I would of thought a good way would to be add an ocean around the terrain?
Any other suggestions?

Another option is the Total War or Oblivion style, create a skybox with distance mountains or hills and don't let the player get close enough to the edge of the terrain to see the transition clearly. So e.g. if the terrain is 512 x 512, restrict the player to 480 x 480 or w/e looks reasonable. Oblivion did it well. I still prefer water though, but that means your world is an island and that might not be acceptable.

One of the most common ways is to render the terrain using Fog to fade to white or black depending on the skybox and/or time of day.  If you then limit the player's range, they should never be able to see the abrupt edges.

Depending on your terrain generation method, another good option may be to generate different LOD (Level of Detail) models of your terrain and render the lower detail models "in the distance".  This technique, called terrain tiling, also works well when combined with fog.  However, you must take care to add vertices to "stitch" the edges of the lower detail models to match the corresponding high detail edges otherwise holes will appear.  Also be aware the heightfields used to generate the models must blend at the left/right edges and the top/bottom edges to smoothly transition from one terrain tile to another.

The TorqueX engine by GarageGames achieves infinite terrain by tiling a single heightfield and dynamically generating LOD at the triangle level as far as the viewport allows.  It is worth checking out if you need really good looking terrain with full collision support.  The TorqueX terrain can even be flown over at high speed and you'll never see an edge.

My suggestion would be to edit your heightmap slightly, so that around the edge of your map you have mountains or high hills, and then alter the bounds of movement so that you cannot get within x distance of the edge, then add a terrain halo, i.e a ring that acts as a second skybox, that contains an image of mountains off in the distance. that way you will get some paralax effects that add more dimensions to your view. much nicer then having a sudden stop to your terrain, and more realistic then having "fog"