When you're organizing your continent, the stone where your castle is is not actually the center. If you're organizing your continent using the castle spot as the center, you're going to end up having a displaced continent.