Hmm best I can think of is:
-Did you delete the height map layer if you used layering to create your mask? Check transparency too.
- Did you use red, green, and blue at their proper values when you worked on your RGB masking:
R 225, 0, 0.
G 0, 225, 0.
B 0, 0, 225.
- Did you save the mask as .png?
- Are your map dimensions 513x513
Hope one of these helps