Yep, sight reading is constantly reading new stuff. Which, if you are a "trained" classical player, you are doing all the time. Many orchestral concerts are likely to be a 3-hour rehearsal in the afternoon and then a show in the evening, and that is it. It might be repertoire you are familiar with. It might not be.
I do not play multiple instruments. But for me, sight reading is tied into the physical parameters of the instrument I am playing. I started reading on a 'cello. The top string on a 'cello is A. I have not played the 'cello in any meaningful way for over 40 years. When I am under pressure and not thinking, I have been known to read A at the top of the bass clef but play my open G on the Double Bass. But not on bass guitar.
I have also started playing a 6-string bass with a bottom F#. This means that what my hands are used to has changed, and I cannot sight-read reliably on this. I can follow chord charts when I concentrate, and it is getting easier - but I would not take this 6 string to a paying reading gig because I know that the pathways between my brain and my hands are not in tune with each other.
The older I get, the more I realise that none of us perceive music in the same way. I have some students who listen to ......... stuff I would not choose to listen to. But they REALLY relate to the energy and noise coming off what they are hearing, which does nothing for me. I know people who are multi-instrumentalists, and their approach to any instrument is that they pick it up, and it makes sense to them (Keys, Bass, Guitar, Drums, Reeds and Brass), so the reading is an extension of that for them. I am not wired that way, so relating the written to the instrument would be a whole new learning process.