SSH on Windows and the "invalid format" error

So you’re trying to use SSH on Windows, and all it gives you is this crumby old error:

> ssh -T [email protected]
Load key "C:\\Users\\randonomicon/.ssh/id_rsa": invalid format
[email protected]: Permission denied (publickey).

How do you fix it? (I hope to have an answer soon, but I ended up switching to Linux for development as the ultimate solution to this problem)