Dealing With Bots: A COAR Resource for Repository Managers

A joint LIBER-COAR webinar

Over the past few years, libraries and their services have been encountering a growing number of bots trying to access their resources. These bots, or crawlers, navigate the internet, gathering data and indexing information for search engines and large language models, as well as for other purposes. While some of these bots are rather innocuous, others are sufficiently aggressive that they cause service disruptions in repositories (and other scholarly communications infrastructures). Machine users have always been a critical constituency for repositories as search engines, aggregators, and other indexing services represent the predominant mechanisms whereby a repository’s resources are discovered. As such, it is very much in the community’s interest to ensure that repositories remain open and accessible to friendly bots and crawlers despite the increasingly aggressive nature of some bots.

In early 2025, the Confederation of Open Access Repositories (COAR) conducted a survey in early 2025 to assess the extent to which repositories were being adversely affected by a sudden and large increase in activity by “bots” – automated Web clients such as crawling or harvesting systems. The ensuing report – The impact of AI bots and crawlers on open repositories: Results of a COAR survey, April 2025 – was published in June 2025. In response to this, COAR convened the Dealing With Bots Task Group to develop advice and supporting information for repository managers to help them to deal with this phenomenon. This resulted in a specific resource  “Dealing with bots” for repository managers.

LIBER has teamed up with COAR to showcase the main takeaways of this resource. In this 60 minute webinar Paul Walk, the technical lead for the Dealing with bots Task Group, will give an overview on the topic and the findings of the group. The 30 minute presentation will be followed by an open Q&A-Session, where all participants can engage.

Session agenda

1. Welcome on behalf of LIBER (Olaf Siegert, ZBW / LIBER Open Access WG) and short introduction on the topic (Kathleen Shearer, COAR)
2. Presentation “Dealing With Bots: A COAR Resource for Repository Managers” (Paul Walk, Antleaf / COAR AI Bots Task Group)
3. Q&A (Olaf Siegert)
4. Wrap-Up and Farewell (Kathleen Shearer / Olaf Siegert)

Speakers

• Paul Walk is a technical consultant with 30 years experience working with information systems and web services primarily in the higher education, research and cultural heritage sectors. Between 2006-2013, he was Deputy Director at UKOLN (University of Bath) and from 2013-2017 Head of Technology Strategy and Planning at EDINA (University of Edinburgh). In 2017 he started his consultancy business, Antleaf, specialising in the development of and support for metadata standards, repository systems and their integration with networked services, and R&D in distributed scholarly communications. Paul has been the lead technical consultant for COAR for over a decade and has been actively involved in the COAR Next Generation Working Group, the COAR Notify Project, and has been the lead of the COAR Dealing with Bots Task Force. 
• Kathleen Shearer is the Executive Director of COAR. COAR (the Confederation of Open Access Repositories) is a global alliance of over 150 members and partners, including libraries, universities, research institutions, and not-for-profits from around the world. COAR brings together individual repositories and repository networks in order to build capacity, align policies and practices, and act as a global voice for the repository community. COAR plays a leading role in defining the standards and practices for open science and is actively involved in helping the global repository community to navigate the rapidly evolving scholarly communications ecosystem. Shearer has been a prominent figure in the open science landscape for close to 20 years. Based in Montreal, Canada, she actively contributes to numerous other organizations including the Research Data Alliance, UNESCO Open Science Working Groups as well as working closely with many regional organizations such as the LIBSENSE Africa (WACREN), SPARC in the US, LIBER in Europe and many others.