It's search engine bots at the moment. When I logged out this morning, Yahoo had 80 of them on the site. Just a moment ago, there were about 40 from Google, and several from both AskJeeves and Yahoo. It's fluctuating, but about 10x higher than normal. Each of them is like a user who reads an entire page and moves to the next in just a few seconds.
We're getting slammed right now, and it will continue for a couple days, because of some changes made to the site. Essentially, we now offer them a map of the site. A series of files that contain proper links to everything they should be indexing, omitting everything they shouldn't be, all with the proper URLs based on our new structure (term-based, rather than variable based).
So what you have is all the major search engines pounding us, hitting all the URLs (over 1 million of them) in that file, which is refreshed nightly.
I don't expect it to last long, and I don't think it will have too much of an impact on the site. If a given page had a typical load time of .25 seconds for you before, .33-.5 wouldn't shock me now.
1.5s is out of line for any page, but that could have been another issue. There are various crons (automated functions) that run throughout the day. The ones that hit the database hardest (and can cause 'hiccups' in the site lasting between 30 seconds and 3 minutes) are run in the wee hours when we're likely to have less than 10 people on. But there are little ones that run hourly, or every few hours. In those cases, if you just happen to load a page the exact same second as one of them runs (and these things only take about a second to complete their task), then that could cause an extra .5-1s on a page load.
What we have to watch regarding slowdown are trends, rather than spikes. One pageload that takes even like 20 seconds, but then the ones after are .3 seconds, and the ones before were as well -- that could be *anything*. Could be your browser mishandled a java script that didn't load properly, could be your network mishandled a packet, or your local internet with a hiccup, or any of the points between you and our server doing the same, or our connection, or our server running something, etc. Thousands of different possibilities, none directly related to site performance.
But if the average page load time increases over an extended period of time, like it did back before we bought this machine, then we have to start looking at new server-side solutions.
As is, I'll take our mysql stats over to Jelsoft later tonight, and see if it's time to do some settings tweaks to streamline performance.
Do keep posting here and letting me know how the site's performing for you though, particularly in regards to slowdowns -- it's the only way I know when things start to slip.
One last thought. Every now and then, my Firefox, after extended use around BP, just sort of hangs up on a page. Then, if I close Firefox and reopen it, the site flies again. Or if I load IE, I can fly through pages there while Firefox continues to hang up as if the site was stalled out. This is something on my end that I haven't identified yet. It's either a problem extension, or an issue with either Firefox itself, or the version of Sun Java I'm running. If it's happening to me, then it may be happening to others, which is why I mention it. The 'fix' every single time, is to just close and reopen Firefox, and it's all gone. It probably happens to me once every 5-6 hours of use.