+1 about EVERYBODY stopping.. could be worth mentioning that at a rehearsal - 'hey chaps, I'm feeling my way into this thing but how about just a skeleton framework for me to work with?' Playing completely solo is mightily exposed!
To my mind, you hit the nail on the head with 'vocabulary'; whether playing a bassline or taking a break you draw on your stock of riffs, lines and phrases. The more you learn, the bigger the store.
Two of the most useful things I ever heard/saw on the subject..
Victor Wooten, in some masterclass video clip I saw on YT then have never been able to find again(!) really stresses this thing where everyone else in the band stops and you're left on your own.. what do you do? Flurry of notes to relieve the pressure of being in total control all the rest of the song.. WRONG! Keep the groove and build on it, otherwise it makes absolutely no sense. A one-note groove-like-a-badger bass break will move people far more than some rhythmically random spray of notes.
David Grier, teaching a guitar picking masterclass.. get so you can think of a melody (indeed, hum a tune) and play it. Sounds daft, but he made his point with Happy Birthday - none of the guitarists in his class could just play it off the top of their heads. 'If you can't just bring up such a well-ingrained tune, how are you expecting to bring up an improvised line that makes any sense.?'
Start with simple lines and let yourself get more complex as you go on. It'll get easier as you do more of it.