New research published today by Imperva has found that bad bots made up nearly a quarter of overall website traffic in 2019.
The report was built from data collected from Imperva’s global network and includes hundreds of billions of bad bot requests anonymized over thousands of domains.
Bad bots are responsible for a whole host of problems, including account takeover, price and content scraping, and the creation of spam-spreading accounts on messaging platforms and dating sites.
According to the 2020 edition of Imperva's annual "Bad Bot" report, in 2019, bad bot traffic rose to its highest ever percentage of 24.1 percent of all traffic. Eerily, 37.2% of all traffic on the internet last year wasn't human.
Researchers noted that bad bot sophistication levels remained consistent for the third year running, with 53.6% of malicious bots being moderately sophisticated, 26.3% simple, and 20.1% sophisticated. Sophisticated bots were found to target marketplaces (28.5%) and the real estate industry (24.5%) most of all.
While some bot issues are industry-specific, researchers noted that bad pot problems run across all industries. The top five industries targeted with bad bot traffic are the financial, education, IT & services industries, marketplaces, and government.
To avoid detection, bad bots practice the art of impersonation, often mimicking web browsers.
"Bad bots continue to follow the trends in browser popularity, impersonating the Chrome browser 55.4 percent of the time. The use of data centers reduced again in 2019 with 70 percent of bad bot traffic emanating from them—down from 73.6 percent in 2018," wrote researchers.
The high volume and wide variety of bad bots out there makes defending against this malicious threat tricky.
"Unfortunately, every site is targeted for different reasons, and usually by different methods, so there is no one-size-fits-all bot solution," wrote researchers.
To combat the threat, many companies are deploying geofencing blacklists, blocking traffic from entire countries. Russia tops the list of country-specific block requests at 21.1%, followed closely by China at 19%.
"In some cases, it simply doesn’t make sense that foreign visitors would use a given site, so blocking chunks of foreign IP addresses is good hygiene," wrote researchers.