When hackers from (or at least through) China brought this website crashing to its knees earlier this week, I learned a valuable lesson: Keep up with patches to popular web applications.
For the past few days, I’ve noticed unusual activity on the blog (run with WordPress software), each event progressively more alarming than the previous — my custom theme was switched to the default (dismissed as an errant click on my part); a plugin was disabled (caused me to start paying attention); links to spammers were added to the sidebar (major alarm bells); and then the blog went completely down due to loss of database connectivity (total panic).
I shut everything down. Peeking around the MySQL stats and logs, I noticed that it had been restarted two days before the blog came down completely. In that time, there were 12 million queries, about 56 per second! There’s no way that normal traffic would justify those numbers. No spikes in HTTP requests, no diggs, etc. Appears that I was an unaware victim of a SQL injection exploit that was discovered in v2.1.2, the out-of-date version I was running.
As luck would have it, I had saved a SQL dump just hours before everything went to hëll. Due to the extent of the damage, recovery involved deleting everything (files, databases, MySQL users, etc) and starting with a fresh v2.3.2 install, manually massaging the upgrade process from the older v2.1.2 table structure. It was a nightmare that lasted most of the day, but I recovered everything except the existing category structure which I planned on redoing/replacing anyway.
Follow these steps to keep yourself a bit safer:
- Do not rely on luck like I did. Instead, install the WordPress Database Backup plugin and configure it to email backups daily, weekly, or monthly — whatever is appropriate for you, your blog, and your traffic patterns.
- If you’re mucking around within phpMyAdmin (or something similar), export a full SQL dump while you’re in there anyway, and save it to your local computer. (I frequently do that, much mitigates some of the luck I had.)
- Back up these emails/backups occasionally on CD/DVD/whatever, and sometimes take and keep a copy off-site. (I still need to get better about off-site storage.)
- When WordPress releases a security patch, upgrade as soon as possible. (My new mantra!)
- When WordPress announces a major version, upgrade within two weeks at the latest — not so soon that you are inundated with all sorts of new bugs, but not so late that you remain vulnerable for an extended time. (I still think it’s OK to wait a couple weeks for version 3.01 (for example) to come out after version 3.00 has been announced in order to get major flaws addressed. Especially if it’s a Microsoft product.)
And, who knows, I might regret it someday, but I’m seriously thinking about grounding HTTP requests and blocking IP addresses from China since that’s where 95% of the DOS attacks and comment spam targeting this blog originates. I’m all for free speech and freedom from information repression, but a few bad apples are spoiling the cider.
[This was originally my 500th blog posting before the total overhaul!]
Recent Comments