Hello mate, frankly, whilst it's not technically correct, I would always adjust the saddles first, simply because a) the action responds immediately (which isn't the case with the neck) and it's therefore easily put back and b) there's less chance of something going catastrophically wrong!
There's also no such thing as the perfect relief/height combination, so whilst it might not work for others, you might find the combination that works best for you with relatively little experimentation. I like dead straight neck and quite high action, other people pick up my basses and say "how do you play it like this....?" Horses and courses
Good luck