Sounds like you already know what's going to happen, you just have to bite the bullet. There are any number of drummers who are 'lovely blokes', but don't confuse this with 'lovely blokes who can play the drums', they are rare and exotic animals. Keep the lovely bloke for social occasions and get the drummer who can play. As said above, if you don't have a good drummer then you don't have a band, it's that simple.
I've wasted lots of time with lovely blokes who can't play, but [i]no more[/i]. If you want to cut it, you can't have any passengers, everyone has to be good. I would also make the point that it's often the drummer who gets the blame for bad timing - and of course a drummer [i]needs[/i] to have good timing - but timing and tempo are the responsibility of the entire band, not just the drummer.