Most pedals, once you plug in a power supply, essentially remove the battery from the circuit. So it matters not one jot whether you leave them in or take them out, if you have a PSU plugged in.
Of course, there is the argument FOR leaving the batteries in, if your PSU goes down mid-gig, all you have to do is unplug the power lead, the battery takes over.
Though saying that, if your PSU goes down mid-gig, there's probably been a power cut, so no one would be able to hear you anyway.