Problem with NaN's when plugging minimizing vectors (NLopt) back into function

I would recommend looking at the famous “19 dubious ways” article, especially decomposition methods (Section 6), but I did not study your problem in detail.