I went to five strings 10 years ago after having played 4's for 10 years.
I felt that going to a fiver was a sort of natural "progression", much the same way a racing driver would progress to a higher formula.
While I didn't take long to adjust to 5 strings, I didn't feel my musical ability altered for better or for worse.
Much of the music I play is big band swing, with notation rarely, if ever, going below low E. I could have put in below E notes if I wanted but the music didn't really call for it.
A few months ago I went back to 4 strings with a great P bass, and given the style of music I play, I've found it much more suitable. I found with less strings to concern myself with, I feel more able to 'let go' and my fills/improvs and general playing have improved as a result.
There are doubtless others here that would say the opposite, but this is just my story.
It really is as simple as horses for courses, one man's meat etc.