Yeah, but if it is under, I find, when I pull on the paper, the whole roll comes flying off the holder! Then the holder always ends up lying way back behind the toilet where you can't reach it.
I can remember the newspaper at my grandmothers place, a hole through cut up squares, then tied together with string and hung on a hook