After a few years of deliberating, I finally upgraded to the kX Project drivers (in this case for a Creative Soundblaster Live Platinum). A few installation troubles later (that might have been conflicts with the old drivers, but succeeded in the end with the non-beta version 5.10.0.3534f) and I'm very happy with the results - the ASIO driver is giving 11ms latency in FruityLoops at 44100Hz, which is just about unnoticeable.
Another speed boost recently came from finally dumping Norton Antivirus and switching to the free AVG. Much faster + much less resources used = happy Windows.