Su, mask issues, nu?
Whenever I've had trouble with terrain masks in the past, it's because the mask has been too blurred, and therefore the game just decides not to render it properly.
So, assuming you're using GIMP or Photoshop, it's a .png image (like your heightmap), and you're sure that you're using only the colors Red, Green, and Blue, then it's probably how you drew your mask that's making it act quirky.
I suggest testing out an entirely different heightmap altogether, at first.
Make a new heightmap, with just a few, simple blobs of red, green, and blue. Don't smudge anything, just make it simple, and test it in Map Maker. If this mask works, then that means you just need to simplify your mask (take away some of the blurs in the lines, mainly).
If you're still unsure as to whether or not your mask is too complex, though, feel free to PM me a screenie of your mask, and I can tell you whether or not it's the image itself, or something within your Map Maker or image editor.