It all helps. My basses are set up to 2mm at 12th fret or just under. I guess if you use a pick you probably need it a bit higher than that. I'm not great with a pick.
Have you tried lighter gauge strings @Lozz196 ? I'm using Elixir Nickels 45, 65, 80. 100
Dave