Sites issues and some hopeful solutions
So you probably noticed the site has been going down a lot lately. The technical reason is it reaches memory and CPU limits. To help with that, I insituted a cacheing feature. What this means is that static pages will be served on heavily trafficked pages. what this means to you is sometimes a successfully posted comment won’t look like it shows up, or page changes won’t seem to show up right away. If you are logged in, this is supposed to be bypassed.
I aslo tweaked some memory limits to see if that helped. It may cause pages to load a wee bit slower.
Some of the reasons for this is we have a load of bloggers here. 
As such, we get hit really hard by spammers. Most of you don’t see it, but the busiest blogs easily get a hundred spam hits a day. Multiply that by the number of prolific and popular bloggers here. In addition, robots were crawling our site roughly every ten minutes. I’ve temporarily blocked them to help weed out the good ones from the bad ones (the bad ones won’t follow the rules and need to be denied a different way). Top that off with feed crawlers. And all that network traffic is before we even have visitors!
In real numbers, we’re getting as much as 1.2 million hits A DAY. That’s 1,200,000 in 24 hours and a heck of a lot of processing.
While I am poking around trying to find causes and solutions, I am taking notes as to what needs to be done on Thanskgiving. I don’t want to go offline for maintenance before then, but if it become unbearable, I will. Just giving what notice I can.
Posted by Andrea on 11-14-2007 at 05:11 pm
Posted in known issues with 3 Comments
Join the conversations