In the beginning, this world was ours. It lay closer to the sun and bore a climate most unsuitable for smaller hairy creatures. Their world was the next one out and quite chilled, requiring a heavy coat of fur and an internal source of heat for survival.
An object came hurtling through our system, striking the fifth planet, shattering it into a field of debris and knocking a large part of the planet out of its original orbit. The sudden reduction of mass in the fifth orbit caused a shift in the gravitational balance, drawing the inner planets out to compensate.
Our cooling planet had doomed us to eventual extinction as the second planet remained too hot for even us. But our world could accommodate the inhabitants of the fourth world. They could adapt to the warmth of it. Their world doomed to become a frozen wasteland.
I cannot explain how we moved them (as I am no expert in such matters), but move them we did. We transported their entire civilization, as if by magic, from one world to another. To them it may as well have been magic as they had no concept of our science and didn’t understand our language.
We stayed on our world for as long as it would have us. As the human civilization grew, they became more violent towards their benefactors. Power-hungry warlords defamed us at every opportunity. We did not demand human sacrifices, but no virgin survived to testify to that fact. Neither rape nor murder was our way. To stay longer, we began to adapt to human form, a painful process that was not always successful.
They hunted us as witches, persecuted us as sorcerers, and drove us to hide or fight for our survival. Eventually, those who had not adapted their form left and those who had stayed died out. So the historical record would state.
Some vestige of what we were had survived.