Tech-Evangelist was a recent victim of a comment spam attack that originated from a wide range of Chinese IP addresses. This is a malicious attack that can temporarily disable web sites and servers. Here is what we saw and what we did to fight off the attacks.
The attacks started to come in waves once or twice per day for about an hour in duration. This was clearly a coordinated attack and not something random. When the attacks hit, our entire server went down due to the severity of the server requests.
At first we thought it was a denial of service attack (DOS), but we also noted a large influx of hundreds of WordPress spam comments every time the attack began. In effect, it was a DOS attack because the intensity of the attack did overload our server, just like a DOS attack. The spam comments led to the identification of the geographic source of the problem and the solution.
Here is what we saw:
As the attacks began, the response time for the web sites slowed down, then WordPress sites stopped responding, and finally the server was so overloaded that the internal nameserver could not respond. Browsers responded with “Server not found” messages.
When the WordPress sites stopped responding, the browser error messages indicated that the problem was a memory allocation error.
Fatal error: Out of memory (allocated 262144) (tried to allocate 245 bytes)
Note that even a small amount of requested memory could not be found due to the server overload. We do allocate memory to each WordPress site, but there just wasn’t any memory available.
We also noticed that Akismet suspended our comment spam plugin API key due to the severity of the attack, so the spam comments flooded in and were held for review. The API key was enabled again a few hours after the attacks were blocked. The spam contained links to sites all over the planet, which indicates that the Chinese network is part of a much larger comment spam network.
WordPress does capture the IP address of the user whenever a comment is submitted, and this can be used to build a list of attacker IPs. We ran the following query to build a list of IPs for the spam that Akismet identified, which were then sorted and identified by country of origin. China dominated the list, followed by a lesser number of spammer IPs in Russia and the Ukraine.
select comment_author_IP from wp_comments where comment_approved = 'spam';
Here is what we did to block the attacks:
- A number of Chinese IP ranges were blocked at the server firewall. We will continue to monitor this and expect the list to grow. This is a large Chinese network, so we expect to find additional IPs that are part of this network.
- Comments were turned off for all older posts. Sorry, but we had to block the comment spam attacks. The comments will probably be turned on again in the future.
If you do not have root access to your server or access to a firewall, any attacker IP ranges that you identify can be blocked via the .htaccess file if the sites are on an Apache web server.
After blocking the Chinese IPs and blocking comments for all older posts the number of spam comments flowing into the site dropped to a trickle.