If you happen to annotate page three, and then weeks or years later visit the single page view wouldn’t you want to see the annotation you made? If the tool you are using queries for annotations using only the URL of the document you are viewing you won’t see it.