It may a problem with a specific version of a Python package that breaks… Can you try lowering its version to an older one in CondaPkg.toml?
Just out of curiosity, how do you install the conda packages? I use this command to do it (in a Dockerfile): julia --project -e "using Pkg; Pkg.activate(\".\"); using CondaPkg; CondaPkg.update()"