Yeah thats a good point, you ears also hear the room the song is playing in. Headphones might create an illusion for your brain.

But it always amazes me how even when I hear music out of an ancient alarm clock I still have, one tiny speaker, a song can still be enjoyable to listen to. Its all in the mastering I guess