Do you feel a certain way about some notes based on the keys with which you're most familiar? For example, due to being most familiar with the "rock keys" of Em, Am etc, I can only think of Bb as a "nasty" note. It wouldn't feel quite right to play a happy song in Bb! C# is a bright, happy note to me, so I struggle with the fact that F (another nasty one!) is its major third. There are many more examples, but I think that's weird enough for now.