that issue drives me insane as well i stg
try to make the colors line up with each other as much as possible on the UV map. following the horizonal lines makes it easier.
for example
which makes
it's not perfect but not really noticeable unless someone look closely at you character.
in your case the white part would be much higher of course.