Bot traffic has a major impact on revenue, security, and infrastructure. Some of that impact is beneficial, including crawlers that boost SEO and AI agents that surface your products to new audiences. Some of that impact is malicious, including DDoS attacks and bots that target authentication endpoints. And some is self-inflicted, like misconfigured firewall rules that block legitimate crawlers and quietly cost revenue.
Your security, SEO, ops, and leadership teams all need visibility into bot traffic to understand these impacts, but each team has different questions to answer. The latest updates to Bot Insights include dashboards designed for ops teams, security teams, CISOs, and more, built on a shared data layer that covers your entire web property. And because Bot Insights is a pre-bundled solution, you can get started quickly.
The Right Dashboards for Every Team
Imagine that your business is running a major event such as a real-time media event or sale. Within an hour of launch, page latency doubles, leading to end users abandoning your site. Bot traffic is the most likely suspect. The event is mission-critical, leading to a scramble across teams to understand and mitigate the impact.
- Security engineers need to know whether the traffic is malicious, which endpoints are impacted, and whether there’s enough evidence to escalate.
- Operations engineers need to know whether bots are taxing infrastructure, degrading cache efficiency, and whether to offload traffic.
- Teams managing organic and paid search need to know whether mitigation blocks legitimate crawlers and bots that drive traffic to your site.
- CISOs need to confirm that the right teams are investigating and whether to escalate or intervene.
- CTOs and leadership teams need confidence that the data is reliable and that the right controls are in place.
With Bot Insights, each of these stakeholders has both a unified view and a specialized dashboard designed to answer the questions they need immediate answers to. The dashboards are designed around how each role actually works. Security, marketing, and ops teams start with a high-level signal: is something worth investigating? Then they can drill deeper when investigation is needed. Meanwhile, directors and CISOs can identify who owns the issue and quickly route or escalate as needed while executives have the high-level data they need to inform business decisions.
The following dashboard provides an overview for CTOs and other leaders to quickly understand how bot traffic is impacting your web properties. You get a quick, high-level view of total requests as well as the current status of bot traffic from the perspective of security, governance (including SEO), and ops teams.
This dashboard helps decision makers understand which issues need immediate attention and which are worth keeping an eye on. Here, the edge is critical, warranting follow-up, triage, and possibly escalation with ops teams. Security and governance are both showing warnings, worth a quick check-in during standup, but not in need of immediate attention.
Decision makers can drill down into the operations dashboard or leave that work to operations teams. The next image shows a view designed specifically for operations teams to understand how origin and edge servers are performing.
This dashboard provides answers to crucial questions like:
- What is the P95 and P99 origin latency?
- Which ASNs are having issues based on metrics like average time to first byte and cache miss rate?
Operations teams need these answers to understand how bot traffic is affecting the origin and edge—and also to provide guidance for mitigating negative impacts.
With Bot Insights, each team can answer questions independently or work cross-functionally across dashboards without duplicating work or waiting on another team to interpret the data.
Enhance Bot Insights with an agentic approach: Our publicly available bot-insights skill helps agents investigate automated traffic and produce evidence-backed findings for the teams that need them most: executive strategy and routing, SOC triage and security attribution, crawler governance and AI bot health, cache-busting and origin pressure analysis, control-change reviews, and deterministic entity scorecards.
Better Classification, More Accurate Data
We continue to improve how Bot Insights identifies, classifies, and verifies bot traffic. Every bot now carries a type and an intent, and a new verification layer catches traffic that spoofs known crawler identities.
Bot Insights now recognizes 803 individual bots across 20 categories, up from 28, using the open-source Matomo Device Detector as a foundation for classification. By distinguishing between training scrapers, agentic assistants, and search crawlers, you can better understand whether bot traffic is benefiting your business or leading to negative impacts.
The verification layer cross-references what bots claim to be (based on user agent string) against vendor-published IP ranges and ASN reputation data. A request claiming to be ClaudeBot but originating from a residential ISP is clearly spoofed. Without verification, that traffic appears in marketing dashboards as a legitimate AI crawler and in security dashboards as benign. HUMAN Security research found that spoofed requests account for 5.7% of all traffic claiming to come from AI crawlers.
Get Started With Bot Insights
Bot traffic affects every part of internet-facing businesses, and the teams responsible for managing it need more than a single shared view. Bot Insights delivers specialized dashboards across security, ops, marketing, and leadership, all built on comprehensive, full-fidelity CDN data so that every team shares the same source of truth.
Bot Insights comes pre-bundled with dashboards and transforms, so there’s no complex setup or additional infrastructure required if you’re already using Hydrolix.
- Read the docs to learn more about Bot Insights.
- See how Bot Insights can support your specific use case with a demo.
- Have questions about how Hydrolix can help your business? Contact us to get a quick value assessment.
- Check out our publicly available bot-insights skill, which helps agents investigate your data for automated traffic.
