With automated bot traffic scanner bots accounting for nearly half of web traffic, distinguishing between human and non-human visitors is vital to the security, cost-effectiveness, and ongoing viability of websites, apps, and APIs. While recognizing the signs of good and bad bots is a challenge, there are some key indicators that can help.

Using a combination of automated traffic bot detection techniques, businesses can identify anomalies that deviate from human behavior and identify potential threats. These include: Unusual browser or operating system combinations (e.g., using a Windows machine to access a website that typically supports only Mac or Linux); rapid navigation; inconsistent geographic data; and missing or suspicious referrer headers. Additionally, a number of common tools used by bots to obscure their true identity can be detected: Use of proxies; HTTP requests with non-standard or empty user agent strings; and frequent or excessive use of cURL, a command-line tool commonly used for scripting and automation, that often contains spoofed or dummy parameters.

Advanced Bot Detection Features: How to Identify and Block Malicious Traffic

Traceable’s dashboard provides a centralized view of all automated threat activity to enable better decision-making and continuous refinement of protection policies. This includes a high-level overview of total threat traffic, monitored threat traffic, and blocked threat traffic. It also breaks down traffic into specific threat types and behavioral subcategories, such as media or entertainment search bots, and cURL bots that may be either legitimate or spoofed. The dashboard also displays a time-series of the ratio between these different categories to detect shifts in bot intent or activities.