Thanks for taking a look at this Chris.
I tried doing this but my Julia REPL was crashing in VSCode every time so moved it to a terminal and got a massive stack trace coming from the line that calls adjoint_sensitivities
:
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Core.SSAValue, (38,)}[SSAValue(1), SSAValue(2), SSAValue(3), SSAValue(4), SSAValue(5), SSAValue(6), SSAValue(7), SSAValue(8), SSAValue(9), SSAValue(10), SSAValue(11), SSAValue(12), SSAValue(13), SSAValue(14), SSAValue(15), SSAValue(16), SSAValue(17), SSAValue(18), SSAValue(19), SSAValue(20), SSAValue(21), SSAValue(22), SSAValue(23), SSAValue(18446744073709551615), SSAValue(25), SSAValue(26), SSAValue(27), SSAValue(28), SSAValue(29), SSAValue(30), SSAValue(18446744073709551615), SSAValue(32), SSAValue(33), SSAValue(34), SSAValue(35), SSAValue(36), SSAValue(18446744073709551615), SSAValue(18446744073709551615)], i=(-9223372036854775775,))
ijl_bounds_error_ints at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
getindex at ./array.jl:924 [inlined]
renumber_ssa at ./compiler/ssair/slot2ssa.jl:55
#359 at ./compiler/ssair/slot2ssa.jl:62 [inlined]
ssamap at ./compiler/ssair/ir.jl:503
renumber_ssa! at ./compiler/ssair/slot2ssa.jl:62 [inlined]
renumber_ssa! at ./compiler/ssair/slot2ssa.jl:61 [inlined]
construct_ssa! at ./compiler/ssair/slot2ssa.jl:900
slot2reg at ./compiler/optimize.jl:645 [inlined]
run_passes at ./compiler/optimize.jl:536
optimize at ./compiler/optimize.jl:504 [inlined]
_typeinf at ./compiler/typeinfer.jl:257
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_apply at ./compiler/abstractinterpretation.jl:1357
abstract_call_known at ./compiler/abstractinterpretation.jl:1620
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_apply at ./compiler/abstractinterpretation.jl:1357
abstract_call_known at ./compiler/abstractinterpretation.jl:1620
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_apply at ./compiler/abstractinterpretation.jl:1357
abstract_call_known at ./compiler/abstractinterpretation.jl:1620
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_15369.clone_1 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_type_infer at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_generate_fptr_impl at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-codegen.1.8.dylib (unknown line)
jl_compile_method_internal at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
do_call at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
eval_body at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_interpret_toplevel_thunk at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
top-level scope at REPL[23]:1
jl_toplevel_eval_flex at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_toplevel_eval_in at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
eval at ./boot.jl:368 [inlined]
eval_user_input at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:151
repl_backend_loop at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:247
start_repl_backend at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:232
#run_repl#47 at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:369
run_repl at /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-macmini-x64-5.0/build/default-macmini-x64-5-0/julialang/julia-release-1-dot-8/usr/share/julia/stdlib/v1.8/REPL/src/REPL.jl:355
jfptr_run_repl_63566.clone_1 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#967 at ./client.jl:419
jfptr_YY.967_56999.clone_1 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_f__call_latest at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#invokelatest#2 at ./essentials.jl:729 [inlined]
invokelatest at ./essentials.jl:726 [inlined]
run_main_repl at ./client.jl:404
exec_options at ./client.jl:318
_start at ./client.jl:522
jfptr__start_57423.clone_1 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
true_main at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_repl_entrypoint at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
┌ Warning: Recursive type
│ T = Core.MethodInstance
└ @ Enzyme ~/.julia/packages/GPUCompiler/S3TWf/src/utils.jl:50
┌ Warning: Recursive type
│ T = Core.CodeInstance
└ @ Enzyme ~/.julia/packages/GPUCompiler/S3TWf/src/utils.jl:50
┌ Warning: TypeAnalysisDepthLimit
│ {[]:Pointer, [0]:Pointer, [0,0]:Pointer, [0,0,0]:Pointer, [0,0,0,0]:Pointer, [0,0,0,8]:Pointer, [0,0,0,16]:Pointer, [0,0,0,24]:Integer, [0,0,0,25]:Integer, [0,0,0,26]:Integer, [0,0,0,27]:Integer, [0,0,0,32]:Integer, [0,0,0,33]:Integer, [0,0,0,34]:Integer, [0,0,0,35]:Integer, [0,0,0,36]:Integer, [0,0,0,37]:Integer, [0,0,0,38]:Integer, [0,0,0,39]:Integer, [0,0,0,40]:Integer, [0,0,0,41]:Integer, [0,0,0,42]:Integer, [0,0,0,43]:Integer, [0,0,0,44]:Integer, [0,0,0,45]:Integer, [0,0,0,46]:Integer, [0,0,0,47]:Integer, [0,0,0,56]:Pointer, [0,0,0,56,0]:Integer, [0,0,0,56,1]:Integer,
#
# goes on and on ...
#
Assertion failed: (pp->getNumUses() == 0), function eraseFictiousPHIs, file /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.h, line 1088.
signal (6): Abort trap: 6
in expression starting at REPL[23]:1
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
pthread_kill at /usr/lib/system/libsystem_pthread.dylib (unknown line)
abort at /usr/lib/system/libsystem_c.dylib (unknown line)
__assert_rtn at /usr/lib/system/libsystem_c.dylib (unknown line)
_ZN13GradientUtils17eraseFictiousPHIsEv at /Users/andrearamirez/.julia/artifacts/76377d35d94620b2ca779d981991ee306ffaad8f/lib/libEnzyme-13.dylib (unknown line)
_ZN11EnzymeLogic23CreatePrimalAndGradientEOK15ReverseCacheKeyR12TypeAnalysisPK15AugmentedReturnb at /Users/andrearamirez/.julia/artifacts/76377d35d94620b2ca779d981991ee306ffaad8f/lib/libEnzyme-13.dylib (unknown line)
EnzymeCreatePrimalAndGradient at /Users/andrearamirez/.julia/artifacts/76377d35d94620b2ca779d981991ee306ffaad8f/lib/libEnzyme-13.dylib (unknown line)
EnzymeCreatePrimalAndGradient at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/api.jl:123
enzyme! at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:5176
unknown function (ip: 0x110aacc0d)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#codegen#142 at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6383
codegen##kw at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6052 [inlined]
_thunk at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6870 [inlined]
_thunk at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6864
unknown function (ip: 0x110aa4f29)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
cached_compilation at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6908
unknown function (ip: 0x110687b61)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
#s883#169 at /Users/andrearamirez/.julia/packages/Enzyme/DIkTv/src/compiler.jl:6968 [inlined]
#s883#169 at ./none:0
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
GeneratedFunctionStub at ./boot.jl:582
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_call_staged at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_code_for_staged at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
get_staged at ./compiler/utilities.jl:114
retrieve_code_info at ./compiler/utilities.jl:126 [inlined]
InferenceState at ./compiler/inferencestate.jl:284
typeinf_edge at ./compiler/typeinfer.jl:868
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_edge at ./compiler/typeinfer.jl:877
abstract_call_method at ./compiler/abstractinterpretation.jl:647
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:139
abstract_call_known at ./compiler/abstractinterpretation.jl:1716
abstract_call at ./compiler/abstractinterpretation.jl:1786
abstract_call at ./compiler/abstractinterpretation.jl:1753
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1910
typeinf_local at ./compiler/abstractinterpretation.jl:2386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:2482
_typeinf at ./compiler/typeinfer.jl:230
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_15369.clone_1 at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_type_infer at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
jl_generate_fptr_impl at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-codegen.1.8.dylib (unknown line)
jl_compile_method_internal at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
ijl_apply_generic at /Applications/Julia-1.8.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.8.dylib (unknown line)
Allocations: 370437327 (Pool: 370255013; Big: 182314); GC: 116
[1] 31023 abort /Applications/Julia-1.8.app/Contents/Resources/julia/bin/julia
So it seems that now the error is in adjoint sensitivities. I tried lowering the tolerance but still get that error. I think the choice of the solver algorithm might matter(?).
Any help is much appreciated!!