Integrating symbolic functions?

I’m trying to perform indefinite and/or definite integrals using Symbolics.Integral(), but I can’t seem to find documentation on how to use it (though, most of Symbolics.jl methods appear undocumented). Currently, through trial and error, I have:

import Symbolics
import IntervalSets

@variables x
f = x^2 + x + 1

domain = IntervalSets.ClosedInterval( 0.0, 1.0 )
var_domain = Symbolics.VarDomainPairing( x,  domain )
I = Symbolics.Integral( var_domain )
fI = I(f)
def_integral = fI.val

Which results in:

julia> @variables x
1-element Vector{Num}:
 x

julia> f = x^2 + x + 1
1 + x + x^2

julia> domain = IntervalSets.ClosedInterval( 0.0, 1.0 )
0.0 .. 1.0

julia> var_domain = Symbolics.VarDomainPairing( x,  domain )
Symbolics.VarDomainPairing(x, 0.0 .. 1.0)

julia> I = Symbolics.Integral( var_domain )
(::Integral{Symbolics.VarDomainPairing}) (generic function with 2 methods)

julia> fI = I(f)
Integral(x, 0.0 .. 1.0)(1 + x + x^2)

julia> def_integral = fI.val
Integral(x, 0.0 .. 1.0)(1 + x + x^2)

I’ve also found SymbolicNumericIntegration.jl, though it appears to only support indefinite integration.

import Symbolics
import SymbolicNumericIntegration

@variables x
f = x^2 + x + 1
domain = Symbolics.Num.( [ 0, 1 ] )

fI = SymbolicNumericIntegration.integrate( f, x )[1]
f₀ = Symbolics.substitute( fI, Dict( [ x=>domain[1] ] ) )
f₁ = Symbolics.substitute( fI, Dict( [ x=>domain[2] ] ) )
def_integral = f₁ - f₀

Resulting in:

julia> @variables x
1-element Vector{Num}:
 x

julia> f = x^2 + x + 1
1 + x + x^2

julia> domain = Symbolics.Num.( [ 0, 1 ] )
2-element Vector{Num}:
 0
 1

julia> fI = SymbolicNumericIntegration.integrate( f, x )[1]
x + (1//3)*(x^3) + (1//2)*(x^2)

julia> f₀ = Symbolics.substitute( fI, Dict( [ x=>domain[1] ] ) )
0//1

julia> f₁ = Symbolics.substitute( fI, Dict( [ x=>domain[2] ] ) )
11//6

julia> def_integral = f₁ - f₀
11//6

My primary questions can thus be summarized as:

  1. How do I evaluate the results of Symbolics.Integral()?
  2. Does SymbolicNumericIntegration.jl support definite integration?
    a. Is there a reason SymbolicNumericIntegration doesn’t support definite integrals?
    b. Is it reasonable to create a pull-request to SymbolicNumericIntegration for a definite_integral method using symbolic substitution for evaluating definite integrals?
1 Like

Implement a symbolic integration or convrt it to Sympy until it is implemented.

Just no one has added it yet.

Yes

1 Like

Ok, so I take it then that symbolic integration isn’t implemented? I had assumed it was since .Integral() appeared via Symbolics.<tab><tab> exploring – which is how I do most of my package capabilities exploring.


A related (and perhaps more uplifting) comment, I’m coming back to Julia now to convert some educational material from Matlab and the performance between Julia and Matlab symbolic calculations is ridiculous. Obviously, Matlab supports a broader range of functionality (e.g. multivariate integration), but it appears Julia has a good foundation. Here’s a direct (manual) translation of Matlab code into Julia. Since the ultimate goal is educational, I’m trying to avoid too many performance tricks that might obfuscate the code for non-CS undergrads.

function LagrangeBasis( degree, basis_idx, variate, domain )
	ref_domain = PolynomialReferenceDomain( Lagrange )
	variate = AffineMapping1D( domain, ref_domain, variate )
	node = LinRange( -1.0, 1.0, degree + 1 )
	basis_val = 1.0
	for n = 1 : degree + 1
		if n != basis_idx
			basis_val = basis_val * ( (variate - node[n] ) / ( node[basis_idx] - node[n] ) )
		end
	end
	return basis_val
end

function PolynomialReferenceDomain( basis::Basis )
	if basis == Bernstein
		return [0.0, 1.0]
	elseif basis == Chebyshev
		return [-1.0, 1.0]
	elseif basis == Lagrange
		return [-1.0, 1.0]
	elseif basis == LagrangeCheby
		return [-1.0, 1.0]
	elseif basis == Legendre
		return [-1.0, 1.0]
	elseif basis == Monomial
		return [0.0, 1.0]
	end
end

function AffineMapping1D( domain, target_domain, x )
	x_target = x .- domain[1]
	x_target *= ( ( target_domain[2] - target_domain[1] ) / ( domain[2] - domain[1] ) )
	x_target += target_domain[1]
end

@enum Basis begin
    Bernstein
    Chebyshev
    Lagrange
	LagrangeCheby
	Legendre
	Monomial
end

Benchmarking:

julia> degree = 8
julia> basis_idx = 7
julia> domain = [0, 1]
julia> BenchmarkTools.@btime( LagrangeBasis( $degree, $basis_idx, $x, $domain ) )
  34.200 μs (440 allocations: 21.95 KiB)
91.02222222222221x*(2.0x - 0.25)*(2.0x - 0.5)*(2.0x - 0.75)*(2.0x - 1)*(2.0x - 1.25)*(2.0x - 1.75)*(2.0x - 2.0)

Whereas Matlab’s timeit function on its equivalent code (using Symbolic Toolbox calculations) results in 0.0196 sec execution time – meaning Julia’s Symbolics.jl is roughly 500x-600x faster (in this instance) without the user really worrying about performance.

Yeah I think that’s a good way to put it. Symbolics has a good foundation but needs more features to fill it out. That is happening over time, but integration is one piece that’s notably missing.

Does this also affect MethodOfLines.jl directly which tells it doesn’t support IntegralEquations for discretization? given Symbolics.jl is used for defining

If needed, you can always fall back to SymbolicRegression.jl and learn the symbolic integral with a genetic algorithm, by evaluating the derivatives of candidate expressions and seeing if they match the function. (This is actually an even more powerful strategy than available in Matlab/Mathematica/SymPy!)

Then you can just convert the recovered expression to Symbolics.jl afterwards with node_to_symbolic!

Here’s a demo of integrating the function f(x) = x^2 + 2x + 1 + \sin(x) by numerically evaluating its derivatives inside a genetic algorithm:

Code:

using MLJ  # for fit/predict
using SymbolicRegression  # for SRRegressor
using Zygote  # For `enable_autodiff=true`
using SymbolicUtils

f(x) = x^2 + 2x + 1 + sin(x)

X = reshape(0.0:0.01:10.0, :, 1)
y = f.(X[:, 1])

function derivative_loss(tree, dataset::Dataset{T,L}, options, idx) where {T,L}
    # Column-major:
    X = idx === nothing ? dataset.X : view(dataset.X, :, idx)
    ∂y = idx === nothing ? dataset.y : view(dataset.y, idx)

    ŷ, ∂ŷ, completed = eval_grad_tree_array(tree, X, options; variable=true)

    !completed && return L(Inf)

    mse = sum(i -> (∂ŷ[i] - ∂y[i])^2, eachindex(∂y)) / length(∂y)
    return mse
end


model = SRRegressor(
    binary_operators=[+, -, *, /],
    unary_operators=[sin, cos, exp],
    loss_function=derivative_loss,
    enable_autodiff=true,
    batching=true,
    batch_size=25,
    niterations=100,
)

mach = machine(model, X, y)

fit!(mach)

r = report(mach)
eq = r.equations[r.best_idx]

symbolic_eq = node_to_symbolic(eq, model)
7 Likes

While certainly (very) interesting and (very) cool, it’s a little heavy for the purposes of teaching undergraduates L^2 inner products:

Let’s compute the inner product between x^0 and x^1 on the domain [-1,1]. First we will train a symbolic regression model to integrate the integrand with a genetic algorithm – an NP-hard problem…

Meanwhile the students be like:

Why all this machinery when \int x \ \mathrm{dx} is obviously just \frac{x^2}{2} ?

It’s just much cleaner right now in my Matlab-based materials:

int( x^0 * x^1, [-1, 1] )  % --> 0

Ah, in that case I misread :joy:

No, the ability to represent integrals is separate from the ability to symbolically solve them. Symbolics has the ability to represent integrals right now, which can be used for example to represent integro-differential equations for NeuralPDE and MOL, but it cannot symbolically solve integrals.

You may want to check out SymbolicNumericIntegration.jl which could in theory have a SymbolicRegression.jl backend to how it’s doing the symbolic regression part. It’s a bit smarter than just a direct symbolic regression though, I think you’d enjoy it.

But indeed it’s the kind of algorithm that should exist in a suite of symbolic integration algorithms, and this probably isn’t the the kind of thing you’d tell people to use first. RUBI type methods should come first.

3 Likes

Well, I’m back at it again – needing to integrate Symbolic expressions as part of some educational collateral. I’m wondering what the most robust approach to integrating complicated expressions might be. Consider the following:

import Symbolics
import SymbolicNumericIntegration
using SymbolicNumericIntegration: integrate
import QuadGK

Symbolics.@variables x

m = [ x^n for n in range( 0, 4 ) ]
f = sin( π * x )^2 + cos( x ) - 1

integrand = m * f
julia> display( integrand )
5-element Vector{Symbolics.Num}:
         -1 + cos(x) + sin(πx)^2
     x*(-1 + cos(x) + sin(πx)^2)
 (x^2)*(-1 + cos(x) + sin(πx)^2)
 (x^3)*(-1 + cos(x) + sin(πx)^2)
 (x^4)*(-1 + cos(x) + sin(πx)^2)

Using integrate from SymbolicsNumericIntegration.jl, which now supports definite integration, sometimes (often) returns the wrong answer:

julia> integrate( integrand[end], (x, 0, 1); detailed=false )
-0.06692331486014069

julia> integrate( integrand[end], (x, 0, 1); detailed=false )
-0.06692331486014069

julia> integrate( integrand[end], (x, 0, 1); detailed=false )
-0.009884419990295636

Cranking down the absolute tolerance doesn’t always help:

## abstol = 1e-12
julia> integrate( integrand[end], (x, 0, 1); detailed=false, abstol=1e-12 )
-0.06692331486014069

julia> integrate( integrand[end], (x, 0, 1); detailed=false, abstol=1e-12 )
-0.009884419990287524

julia> integrate( integrand[end], (x, 0, 1); detailed=false, abstol=1e-12 )
-0.009884419990296808

## abstol = 1e-20
julia> integrate( integrand[end], (x, 0, 1); detailed=false, abstol=1e-20 )
[ Info: Integration is partially successful. Pass `detailed = true` to `integrate` for details

This is especially annoying when broadcasting, as (seemingly) randomly some terms don’t integrate and thus return nothing:

var_tuple = fill( ( x, 0, 1 ), ( 5 ) )

julia> integrate.( integrand, var_tuple; detailed=false )
[ Info: Integration is partially successful. Pass `detailed = true` to `integrate` for details
[ Info: Integration is partially successful. Pass `detailed = true` to `integrate` for details
5-element Vector{Union{Nothing, Float64}}:
  0.3414709848078965
  0.13177329067603627
   nothing
   nothing
 -0.06692331486014069

julia> integrate.( integrand, var_tuple; detailed=false )
[ Info: Integration is partially successful. Pass `detailed = true` to `integrate` for details
[ Info: Integration is partially successful. Pass `detailed = true` to `integrate` for details
5-element Vector{Union{Nothing, Float64}}:
  0.3414709848078965
  0.13177329067603627
   nothing
 -0.07826184164390249
   nothing

I’ve generally found trying to tweak kwargs to be an exercise in futility and expression-dependent.


I’ve also tried using QuadGK.jl, resigning myself to using a purely numerical approach, but Symbolics don’t appear compatible with QuadGK?

julia> QuadGK.quadgk( integrand[end], 0, 1 )
ERROR: Sym (x^4)*(-1 + cos(x) + sin(πx)^2) is not callable. Use @syms (x^4)*(-1 + cos(x) + sin(πx)^2)(var1, var2,...) to create it as a callable.

It’s not clear to me how to create a Symbolics callable object in this case (where it’s desired to still be able to do things like transpose and * in other parts of the code):

Symbolics.@syms integrand_5( x )

julia> integrand_5(x) = x^2
ERROR: cannot define function integrand_5; it already has a value
julia> integrand_5
x^2
julia> integrand_5(2)
ERROR: Sym x^2 is not callable. Use @syms x^2(var1, var2,...) to create it as a callable.

Examples of transpose and * not working:

m_1(x) = x^0
m_2(x) = x^1
m_3(x) = x^2
m_4(x) = x^3
m_5(x) = x^4

M = [ m_1, m_2, m_3, m_4, m_5 ]

julia> transpose( M )
1×5 transpose(::Vector{Function}) with eltype Union{}:
Error showing value of type LinearAlgebra.Transpose{Union{}, Vector{Function}}:
ERROR: MethodError: no method matching transpose(::typeof(m_1))

julia> M * f
ERROR: MethodError: no method matching *(::typeof(m_1), ::Symbolics.Num)

julia> reshape( M, 5, 1 ) * reshape( M, 1, 5 )
ERROR: MethodError: no method matching zero(::Type{Function})

Side note #1: the exact integral of integrand[end] can be found in SymPy / Matlab / Mathematica / etc., as:

-\frac{1}{10} + \frac{3}{4\pi^4} - \frac{1}{2\pi^2} + 13\sin(1) - 20\cos(1) \approx -0.009884419990295

Side note #2: the Symbolics.jl documentation claims that it supports “Symbolic Linear Algebra” however this rarely seems to work for me:

julia> var_tuple = fill( ( x, 0, 1 ), ( 5, 5 ) )
julia> A = integrate.( m * transpose( m ), var_tuple; detailed=false )
julia> var_tuple = fill( ( x, 0, 1 ), ( 5 ) )
julia> b = integrate.( m, var_tuple; detailed=false )

julia> A \ b
ERROR: MethodError: no method matching Float64(::Symbolics.Num)

I guess I expected Julia to be a bit more “composable” ala Matlab, isn’t that what multiple dispatch is supposed to provide?

Also, if you use matrix inversion the precision is a bit disappointing:

julia> inv( A ) * b
5-element Vector{Any}:
      1.0000000000005969
 -5.4569682106375694e-12
 -1.8189894035458565e-12
   3.637978807091713e-12
  -3.637978807091713e-12

I thought the benefit of Symbolic.Num was exactness (to the extent possible)? That’s at least a major benefit of other symbolic engines. Consider Matlab:

>> x = sym( "x", "real" )
>> m(x) = [x^0; x^1; x^2; x^3; x^4]
>> A = int( m * transpose( m ), [0, 1] )
>> b = int( m, [0, 1] )
>> sol = inv( A ) * b

sol =
 
1
0
0
0
0

>> isAlways( all( sol == sym( [1; 0; 0; 0; 0] ) ) )

ans =

  logical

   1