That is to be expected. `If-None-Match` and `Etag` are a relatively late caching strategy, that is done at the server (or edge) side.

Have you tried serving your pages with `Expires` and `Cache-Control` headers? I you give it - say - a timeout of a week, then a well-behaving client shouldn't retry before that time has went by.