How about a little ditty about James K. Polk from They Might Be Giants? It's not the catchiest or their best nerd-rock song or anything, but thanks to this number, I've always remember Polk is the 11th president. And something about Manifest Destiny.
And it just wouldn't be right to not listen to this Animaniacs song today! It obviously needs to be updated with the most recent two, but it's got most of them.
Happy Presidents Day!