I don't think you'll have a problem at all. You did not say what strings you're using; some are more tune-stable than others, particularly Spiros. If you are going to do this regularly I'd recommend laying your bass down on its back, completely dutuning the E and slipping it off the bridge and top nut. Then get a soft pencil, something like a 6B, and coat the nut and bridge slots with graphite. That'll give the string an easy slide when tuning back from D to E and there will be less risk of pulling the bridge out of line.
Incidentally, it's well known that if you completely detune the strings and then bring them back up to pitch they will sound much brighter for about a week. I put a post on TB recently to ask if anyone does this regularly between gigs. The consensus was NO mainly due to the difficulty in keeping some strings, particularly guts, in tune afterwards.