Would you be able to post a screenshot of what it looks like?
I used to make height maps and masks for a little bit, and remember it glowing when the mask itself
was off by a few numbers.
Red: 255, 0, 0
Green: 0, 255, 0
Blue: 0, 0, 255
If one of those numbers are off, it will cause certain areas to glow on the terrain, especially where
the colors meet.