That's strange. ;o I know Photoshop has a wide selection of exotic looking brushes, and if used, your game might just not be able to render it. To fix that you can blur the edges and simply fix up or coloe over what you think would be causing that. Another reason it could be crashing is because there's an alpha channel. You want to remove that in the layers options. Also, save the heightmap as: "(MAPNAME)Terrain.png"
If none of that works, maybe you can try changing the game's rendering system. I do hope that works out for you.