Dear Microsoft Word (Office X): How difficult can "I want this picture centred, below this paragraph and above that paragraph, with a caption beneath it" be to understand? You are broken.
Later: It's something to do with the way I'm adding the picture. Insert>Picture From File works fine and gives an inline caption, but Edit>Paste Special>Picture ends up with a floating picture and caption. It seems that copying and pasting an inserted picture is the same as doing Paste Special>Formatted Text (RTF) - if you choose Microsoft Word Document or Picture (the only options available when copying and pasting from another application), everything gets messed up.
More: If you want to understand what I'm dealing with, try reading the answer to this newsgroup query. Note that this was August 2002, and none of the bugs (including "a really embarrassing oversight in the programming") have been fixed.
So the answer seems to be, after pasting in the picture, use Format Picture>Layout>Wrapping Style>Vanilla Ice Inline With Text, then use Keep With Next to make sure the picture doesn't get separated from the caption. Format the Paragraph part of the Caption style so that Keep Lines Together is selected as well.