Zygote limitations: getindex of a DataFrame while training a PINN

Specific Domains Machine Learning
, , ,
1

I want to train a PINN.
I decided, that I want to estimate the physical loss term at static, independent sample times.

For each of the sample times, I have to look up result of the previous sample point, to estimate the delta between both.
I currently solve this, with a look-up in a DataFrame.
The current version of the code is able to run outside the training, but not during the training. I get I error in Zygote, I assume, that the getindex of the DataFrame involves a try-catch bock:

The problematic lines of code are:

	is_previous = static_df.Column1 .== (x.Column1-1)
	xₚᵣₑᵥ = static_df[is_previous, :] |> first

The second lines raises this Error:

Compiling Tuple{Type{Dict}, Dict{Symbol, Int64}}: try/catch is not supported.

Refer to the Zygote documentation for fixes.

https://fluxml.ai/Zygote.jl/latest/limitations

I hope, that someone has an idea

  • to make this look-up without this limitation, or
  • rework the physical loss term, so that this look-up is not needed.

Background:
I currently call the total loss function for reach trained data point cannot get mini batches running).
Therefore I apply a weight, because there are hundred of sample points for the physical loss.

lossₜ(ŷ, y) = (1 - loss_ratio) * mse(ŷ, y)+ loss_ratio * lossₚ()

I compute the physical loss term for every point of the sub sample:

lossₚ() = eachrow(sub_sample_time_df) .|> lossₚ |> sum

	function lossₚ(x)

		Κ = 0.997887
		τ = 1439.8
		θ = -661.1

		ŷ = [x.Q1, x.T1prev] |> NN |> first

		# Column1 contains a zero based index
		is_previous = all_data_points_df.Column1 .== (x.Column1-1)
		xₚᵣₑᵥ = static_df[is_previous, :] |> first
		
		ŷₚᵣₑᵥ = [xₚᵣₑᵥ.Q1, xₚᵣₑᵥ.T1prev] |> NN |> first
		Δyₗ = ŷ - ŷₚᵣₑᵥ
		U = x.Q1
		i₀ = x.duration
		
		yᵣ = (.-ŷ .+ Κ .* U .*(i₀ .- θ)) ./ τ
		
		return sum((Δyₗ .- yᵣ) .^2)
	end

I construct the physical model around the sample point. The function includes a delta term. Do a lookup of the heating and the previous temperature of the previous value, to feed it into a tiny neural network with only 33 parameter.

I perhaps someone with more experience with PINN can share his experience?

2

The first step to try is to convert your dataframe into a normal matrix (or series of vectors/matrices) before passing it to your loss function. Zygote is evidently not able to differentiate through some of the internals of DataFrames.jl, so it needs a friendlier input format to work with.