I've just followed your workflow step by step and everything's in time over here. Could be a couple of things - some of the EZ drummer loops can sometimes be a leeeettle bit slow, you could use quantise to bring them in, you might not have 'snap to grid' (command and G) turned on, so your loops aren't starting at exactly the right point, or it could be a RAM issue - as EZ drummer does everything in real time.
This is the first time I've opened Garage Band in a couple of years, so I'm certainly not an expert, but these are the initial things that spring to mind.
There's also a tick box for 'follow tempo and pitch' that automatically takes care of time-stretching loops to the correct tempo, is that selected?
[attachment=106269:Tempo and pitch.png]