I used to do it with my old Hartke 3500 , but I've since read that it could potentially damage the amp.
The sound only came through one side of the headphones which was annoying, I presume because the speaker output is mono, not stereo.
Rereading your post if you don't actually have amp here yet try looking for one with a dedicated phones output. I think a few of the class Ds have them.
Further edit.
In fact having looked at the ampeg you mentioned it does indeed have a dedicated headphones output. They wouldn't have included it if it was going to damage the amp so you should be golden.