I don't think the fret filler is the problem, I've played on a bass that was de-fretted and the had the slots left open, it worked fine.
You have nothing to lose, go nuts with the allen key see what happens,. I can't imagine how the truss rod would get bent, it's more likely to be a problem with the threaded portion. There's only a short thread (just beyond the silver bit) it sound to me that it's stripped or damaged in some way. It is possible to replace a truss rod