I played 5's exclusively for a good 3 years or so (after starting on 4's obviously), I never had any issues, but going back to 4's after so long was a revelation, they definitly feel like home to me.
I do have a great 5 still, but it is rarely used, usually for dep gigs that I've no idea what key they want songs in, so its positional as opposed to specifially wanting low notes.
I have an Octave pedal that I often just kick in for low notes, more than enough, there aren't too many 5 string players (metal aside) who simply sit down on the B string.
But agreed, its what you feel most comfortable and free on. I understand it when people say "well its good to be challenged, makes you think and play in different ways", but my school of thought is "If you're so at home on an instrument that you forget it's there, you're more likely to concentrate on the music"
Si