Lush Life.
The OP's question is a legitimate one. Given all of the variables, it doesn't matter whether the groove is more important than the notes or the 'feel' etc. One can readily assume that when we say ' the hardest', we are looking for what is the 'hardest' tune in which to get all of these factors to work together to create an identifiable whole. The argument is like saying which building is the tallest and responding by assuming so buildings have taller storeys so will be taller whilst having less floors. It's irrelevant.
For me, the simple truth is that, in strictly musical terms, if it is hard to play, it does not become a standard. In my experience, none of them are hard, They are just different and you either know them or you don't. The observation that they all look pretty easy is entirely accurate because most of them are.
I once worked with a chef who, in his 30s, bought a guitar and an Oasis songbook. After one weekend working with his new toy, he came to me on the Monday morning and said 'Is that it'?