The Shadow Traffic Problem: Combating AI-Driven Automated Threats to Websites
Day in and day out, web applications are subject to unwanted automated usage. These events often relate to misuse of inherent valid functionality, rather than the attempted exploitation of unmitigated vulnerabilities. Examples of these events include click fraud, comment spamming, content scraping, password cracking, and many more.
An OWASP project on Automated Threats to Web Applications has produced an ontology providing a common language to facilitate clear communication and help tackle these threats. The project identifies symptoms of these issues and discusses countermeasures against them.
Nowadays, AI is turbocharging the problem into overdrive. Residential proxies, full browser engines, and LLM-driven agents now produce automation that looks human. When combined with emerging Scraping-as-a-Service providers, the cost of doing business on the web goes up significantly.
An early 2026 study of web traffic by AI shows that 51% of web traffic is human and 1% is wanted bots; the rest is unwanted automation. By September, human traffic accounted for less than half of all web interactions. This shift is further highlighted by the arrival of AI agents that are chattier than humans.
This talk will discuss both technical and non-technical countermeasures for combating unwanted automation. We will explore regulatory frameworks and financial incentives to realign human-machine co-existence on the web.
In addition, we will discuss emerging mechanics such as cryptographic agent identity via HTTP message signatures, machine-readable terms and pricing, pay-per-crawl programs, HTTP 402 Payment Required as a settlement primitive, and intermediaries at natural metering points.
About the speaker
Tin Zaw
Read more …