Cycling
Mapperz points to TFL's new Google Map of cycle routes in London:
Even better is Camden Cycling Campaign's route map, which has more routes and the option to use either Google or OpenStreetMap.
The TFL Journey Planner's cycle routes are pretty good too, if a bit awkward to read when split over multiple pages.
Walking
Google provides routes for walking between two locations in London, but tends to route along main roads, which is good (safer) if you're a tourist but not so good for regular walkers:
Much better are walkit.com's "air pollution-aware walking routes", which tend to be more direct and avoid main roads:
An "interestingness" threshold, for when you want to explore landmarks and pass through pleasant areas while walking, rather than just getting there, would be good.