Some of overlay files in FHHD aren't compatible with the new version due to changes in the overlay. I haven't noticed any errors in using the vanilla 1.17 overlay files with the rest of FHHD, so you may want to try that for a simple solution.
If you want to change as much to FHHD as possible, I didnt individually check to see what file causes this specific error (I had simply restored everything in feral heart > media > overlay2) but I suspect it may be actions.overlay or some other overlay related to in-game play. I also definitely know that charcreate.overlay can't be replaced when installing FHHD as it causes an error (it can be replaced with the extended items list mod since that was updated though).
Aside from the overlay folder, the only other compatibility issue I can think of off the top of my head with FHHD is that you can't replace cmain or fmain in media > materials, as it causes the tufts to have texture errors (in my case they were pure white).
As for ENBs, I'm a little less knowledgeable in what causes issues so afaik we may just need to scout for and either fix or tolerate visual issues on a case-by-case scenario. As such, you may just need to specify what ENB you want to use and what issue it's causing.
