SSH on Windows and the "invalid format" error

So you’re trying to use SSH on Windows, and all it gives you is this crumby old error:

> ssh -T git@github.com
Load key "C:\\Users\\randonomicon/.ssh/id_rsa": invalid format
git@github.com: Permission denied (publickey).

How do you fix it? (I hope to have an answer soon)