Here are the biggest users of this server over the last week:
That's the spider for FAST search, which has been explicitly banned in the robots.txt since it did this a few months ago, and Bloglines, which I signed up with a HubMed RSS feed as a trial and has now grabbed that file over 2000 times this week - that's one (1) subscriber, who has actually looked at the Bloglines page probably twice, at the most.
The search files the FAST spider is requesting are also part of HubMed, and are dynamically built using data from the NCBI's public servers - each request puts strain on their servers as well as mine. That's almost 6000 queries this week for files which *all* spiders are banned from requesting.
Incidentally, the RSS feed that Bloglines is devouring is marked with <sy:updatePeriod>daily</sy:updatePeriod> and <sy:updateFrequency>1</sy:updateFrequency>, which is used to indicate to aggregators that the feed is only updated *once daily*. It's a shame, because I was going to recommend Bloglines to lots of people who wanted to use HubMed feeds but couldn't run an aggregator.