Yeah, the same here.
Now, I almost never listen to my own recordings, but when I do, then it's a great kick.
Though I do buy the insecurity argument in itself (as it indeed is a driving force for a lot), in this I don't see it necessarily being a part of the equation.
In my case it sure is not. See, unlike you, I enjoy the sh*t just as much as the good stuff. I know what I thought back then, and why I thought it. I remember the processes that went on in my mind while the recording took place. This makes for fond memories.
(I have a memory dysfunction now, and haven't heard my stuff since getting that dysfunction, so the above describes how it was for me before. Dunno how it would be now.)
best,
bert