I'm not one for applying rules to music - more from the "if it sounds good and feels good, it IS good" school of thought.
However, my experience has taught me that there are a few little pointers that help make a reggae line really work:
- Simplicity. Reggae is a simple pulsating rhythmic style, and a simple repetitive hook works really well. Root and chord tones for the win.
- Space. As others have mentioned, it works best when you give the other noises space to do their thing. Listen to the groove and ask yourself what are the main defining parts? - maybe a certain hihat shuffle or rim hit, or vocal phrase. Then make sure you accentuate rather than diminish the impact of those parts. Often not playing is an effective way to do this.
- length of notes. This I related to the above. When jamming the groove, thing about the appropriate length of each note. For example, you can play the root on the one, but a short punchy note will have a very different feel to sustaining that note until an immediate cutoff when the guitar skank hits.
Also as per the above, you need to 'get' reggae by immersing yourself in it. Only when you really feel reggae will you get that euphoric "ahhhhhhh" feeling when your groove locks with the band.