Ultimately I reckon it all comes down to what you want as a musician. If you want to play to full venues full of happy, dancing people (a bit like me, really) then two ways to completely wreck that are (i) play too loud, and (ii) play too fast.
If the band's volume pins the audience to the back wall of the venue, then pretty soon you'll be playing to an empty room, and then you won't be playing anywhere.
If the band's speed means that the audience stand around looking bored and disgusted, then pretty soon ... well, you get the picture.
I can't see either of those issues as being negotiable, and if you're in the minority on these subjects then chances are that you're in the wrong band.
You haven't mentioned the third big issue that causes trouble - repertoire choices (no, that wasn't a failed Tarantino project).