Probably because of this notion that it is "all in the fingers" that holy grail quest for great tone! Personally, I find it very hard to play bass with a pick, to the degree that if I ever play guitar now I tend to use fingers on that too. I love some pick on bass sounds, particularly the really old school muted flats on something like Labi Siffri's "It Must Be Love" and lots of other 60's pop, and the more gnarly punk players, but never been able to play that way.
I like to think that a lot of what can seem like snobbery is more a bit of a running joke, along with the 5 string debate and good few others.
Pick away - enjoy it!