I suppose that depends on how you like the action personally, but I find fretless so lovely to play when the strings are practically touching the fingerboard at the nut and require minimum finger pressure.
Anyway it sounds like the new nut is the problem, have you fitted it 'straight from the packet' or has it been cut to suit, with a angle cut away from the fretboard to help the break? As said a couple of spots of glue would help, first making sure the new nut sits firm, flat, (or contoured to fit), and square into the cutout.