Sounds like you are a proper bassist going above the 12th fret, that’s bandit country for me.
And I think I read you right in that the saddles are high and hence have enough travel to drop the action.
Possibly your shim is an angled shim which has not just raised the height of the neck, but also the heel in proportion to the rest of the neck.
I would take the neck off and see if the shim comes out easy, and you can assess whether it is angled or not. if it does try it without and see what the action goes to, you could even see if you can reverse it so the neck height is still raised, but tapers off.
It May be if it’s an angled shim and a shim is needed it just requires a flat one or less severe angle. You can experiment with playing cards to get what you need and then replace with a proper one.
If the shim is glued in then you can sand down the heel portion to allow it to drop, or build up as above to how you like.
If you want ready made Shim’s StewMac website is great and gives all kinds of angles to use.
Hope it helps and makes sense, others may come with ideas if I am talking bunkum