I don't think it will make a difference if you use darker and lighter colours.
If the terrain comes out dark (but it shouldn't) it must be a problem with the files.
If you want a working mask without BRIGHT colours, I'm afraid you might not find a way to do this.
TIP: When you're making the mask, lower the bright colour opacity down. When you finnish, make it the level you want so it doesn't burn your eyes while you're making it.
If you want a realistic look , I would say you use the 'overlap terrain' method.
Steps:
1-Get a shade of red + Green/blue.
2- Do the red places you want on the map.
3- Go OVER them in green/blue on a new layer.
4- Lower the green/blue layer opacity down.
If you want a dark cave terrain, I would use the darkstone (On the mask it would be red at full opacity.) and black. (On the mask it would be green or blue at about 10-20% opacity.)
Done!
Now in the game, load up your terrain mask. Add the terrains, and you should have two of them mixed together.
WARNING: If you mix light terrains together, it will become VERY BRIGHT.
For example; Detail + Snow Tex won't make a pretty one. It will make your eyes EXPLODE.
I hope this helped a little bit with the mask making!