Ongoing effects of excessive AI crawlers, and our mitigations, on Pavlovia

Online experiments
,
3

Bots and crawlers have always played a part in the traffic to any website, but over May/Jun 2026 the activity of these bots, at least those irresponsible ones that just constantly poll all website endpoints, has gone through the roof. Rather than thousands of requests per minute, which our servers are designed to handle, we now see a roughly constant barrage of thousands of requests per second. That impacts genuine users and occasionally prevent them from being able to access the site. What is more, those insidious bots are designed to evade standard counter-measures (e.g. rate-limiting).

We have been working on a variety of mitigation approaches to reduce the strain on the servers, and occasionally had to temporarily restrict some edge-case functionality and permissions settings on gitlab.pavlovia.org.

While most functionalities have now been restored, some changes remain in place and may slightly impact how you interact with the platform. We wanted to make you aware of these updates, as below, and we will continue to update this thread as further developments occur.

Changes and current status

  • Access to demo experiment code
  • We temporarily restricted access to the source code of public Pavlovia demo repositories, as these repositories were being heavily targeted by the bots. During this period, the View Code option was unavailable for demo projects. This restriction has now been lifted and access has been restored.
  • Viewing CSV files through GitLab
  • You may currently be unable to view .csv files (for example, individual data files) through View Code → Data → [CSV file]. This access remains restricted. However, all data files can still be accessed either by using PsychoPy (or git) to fetch the files to your local machine or using downloaded using the Download Results button on your Pavlovia project page.
  • Setting projects to public
  • As part of our response, project visibility was temporarily limited to Private and Internal (accessible only to users with a Pavlovia/GitLab account). It is now once again possible to set projects to Public.
  • GitLab Explore and Network pages
  • The GitLab pages /explore and /network (which you were unlikely to use, these are different to the Pavlovia Explore page, for instance) are still currently disabled.
  • You can still browse demos and access experiments directly through Pavlovia.org; only access through these two GitLab pages is currently unavailable.

We apologise for any inconvenience these temporary restrictions may have caused and appreciate your patience while we addressed the issue. We expect any remaining restrictions to have minimal impact on normal workflows, and we will continue to monitor the situation closely.

Thank you for your understanding, we hope everything is smooth from here on!

The Open Science Tools Team