# Benchmarking Symbolics.jl against SymEngine.jl

**URL:** https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744
**Category:** Performance
**Tags:** performance
**Created:** [September 11, 2023, 3:42pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744 "2023-09-11T15:42:56Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![certik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/certik/32/2651_2.png) [@certik](https://discourse.julialang.org/u/certik)
#### Post date: [September 11, 2023, 3:42pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/1 "2023-09-11T15:42:56Z")

</div>

I finally found time to benchmark the Symbolics.jl against SymEngine.jl. Here are the results:

> <https://github.com/symengine/symengine/issues/1973>
>
> I want to compare the speed of SymEngine and Symbolics.jl. I don't know if I am …using Symbolics.jl correctly, so I'll document what I do here. I am using Julia 1.8 on Apple M1 Max.
> \`\`\`console
> julia\> using BenchmarkTools, Symbolics
> 
> julia\> vars = @variables a,b,c,d,e,f,g,h,i
> 
> julia\> x = ((a+b+c+1)^20)
> (1 + a + b + c)^20
> 
> julia\> @benchmark y = expand(x)
> BenchmarkTools.Trial: 21 samples with 1 evaluation.
> Range (min … max): 238.961 ms … 244.956 ms ┊ GC (min … max): 2.01% … 3.99%
> Time (median): 240.229 ms ┊ GC (median): 2.24%
> Time (mean ± σ): 241.298 ms ± 2.024 ms ┊ GC (mean ± σ): 2.72% ± 0.84%
> 
> ▃ █▃▃                                                    
> █▁▁▇▁▁▁▁███▁▇▁▁▁▁▁▁▇▁▇▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▇▁▁▁▁▁▇▁▇▁▁▁▁▇▇▁▁▁▁▁▁▇▇ ▁
> 239 ms Histogram: frequency by time 245 ms \<
> 
> Memory estimate: 113.84 MiB, allocs estimate: 447939.
> \`\`\`
> 
> I then compiled SymEngine (default cmake), and applied the following diff:
> \`\`\`diff
> diff --git a/benchmarks/expand1.cpp b/benchmarks/expand1.cpp
> index 4471152e..0c6a897f 100644
> \--- a/benchmarks/expand1.cpp
> +++ b/benchmarks/expand1.cpp
> @@ -30,11 +30,12 @@ int main(int argc, char \*argv\[\])
> RCP\<const Basic\> y = symbol("y");
> RCP\<const Basic\> z = symbol("z");
> RCP\<const Basic\> w = symbol("w");
> \- RCP\<const Basic\> i60 = integer(60);
> + RCP\<const Basic\> i1 = integer(1);
> + RCP\<const Basic\> i60 = integer(20);
>  
> RCP\<const Basic\> e, r;
>  
> \- e = pow(add(add(add(x, y), z), w), i60);
> + e = pow(add(add(add(x, y), z), i1), i60);
>  
> std::cout \<\< "Expanding: " \<\< \*e \<\< std::endl;
>  
> \`\`\`
> And run it:
> \`\`\`console
> $ ./expand1 
> Expanding: (1 + x + y + z)\*\*20
> 3ms
> number of terms: 1770
> \`\`\`
> So I am getting 3ms. I don't know if I am running both benchmarks correctly. It seems SymEngine is 80x faster, which seems that I probably do something wrong.
> 
> I asked the Julia community for help here: https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744.

Am I doing anything wrong with how I run them? So far on my computer Symbolics.jl seems slower. I want to make sure that I am benchmarking it correctly. Thanks for any tips. I tried both Julia 1.8 and 1.9.3. Both global scope and inside a function.

---

<div class="post-metadata">

### Author: ![shashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shashi/32/1824_2.png) [@shashi](https://discourse.julialang.org/u/shashi)
#### Post date: [September 11, 2023, 4:25pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/2 "2023-09-11T16:25:19Z")

</div>

Hi! This is good to know. Is expand the only thing you’re benchmarking?

Most of expand is spent in DynamicPolynomials.jl, there are faster alternatives we could use however. AbstractAlgebra.jl could be faster, but needs more code to convert into. Cc @blegat

---

<div class="post-metadata">

### Author: ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)
#### Post date: [September 11, 2023, 6:15pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/3 "2023-09-11T18:15:45Z")

</div>

Not sure why AbstractAlgebra would be faster. It’s a similar datastructure. Are they using another algorithm?

Maybe using MutableArithmetics to save allocation or switch to TypedPolynomials could help

---

<div class="post-metadata">

### Author: ![shashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shashi/32/1824_2.png) [@shashi](https://discourse.julialang.org/u/shashi)
#### Post date: [September 11, 2023, 7:34pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/4 "2023-09-11T19:34:32Z")

</div>

IIRC AbstractAlgebra uses a representation where there are a fixed number of possible variables, and if I’m right, they map each one to an integer, and represent the monomials as a matrix of exponents.

```julia
julia> using AbstractAlgebra

julia> R, (x, y, z) = polynomial_ring(ZZ, ["x", "y", "z"])
(Multivariate polynomial ring in 3 variables over integers, AbstractAlgebra.Generic.MPoly{BigInt}[x, y, z])

julia> f = x + y + z + 1
x + y + z + 1

julia> p = f^20;

julia> @time q = p*(p+1);
  0.510858 seconds (15.66 M allocations: 294.774 MiB, 28.83% gc time)

# Same thing with DynamicPolynomials:

julia> @time p*(p+1);
 16.334984 seconds (3.14 M allocations: 335.826 MiB, 0.54% gc time)

```

```julia
julia> p.x
1771-element MonomialVector{DynamicPolynomials.Commutative{DynamicPolynomials.CreationOrder}, Graded{LexOrder}}:
 1
 z
 y
 x
 z²
 yz
 y²
 xz
 xy
 x²
 ⋮
 x¹⁷yz²
 x¹⁷y²z
 x¹⁷y³

# vs.

```

```julia
# AbstractAlgebra:

julia> p.exps
3×1771 Matrix{UInt64}:
 0x0000000000000000 … 0x0000000000000000
 0x0000000000000000 0x0000000000000000
 0x0000000000000014 0x0000000000000000

```

---

<div class="post-metadata">

### Author: ![shashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shashi/32/1824_2.png) [@shashi](https://discourse.julialang.org/u/shashi)
#### Post date: [September 11, 2023, 7:42pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/5 "2023-09-11T19:42:00Z")

</div>

> [@blegat](#):
>
> TypedPolynomials

Oh not bad, TypedPolynomials beats AbstractAlgebra:

```julia
f = x + y + z + 1

p = f^20;

@time q = p*(p+1);
  0.279255 seconds (23 allocations: 242.545 MiB)

```

We have to be very careful about compile time in Symbolics though, so it might make sense to allow switching between the two with Typed being the default.

---

<div class="post-metadata">

### Author: ![shashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shashi/32/1824_2.png) [@shashi](https://discourse.julialang.org/u/shashi)
#### Post date: [September 11, 2023, 7:45pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/6 "2023-09-11T19:45:33Z")

</div>

But it’s till way behind SymEngine 😄

---

<div class="post-metadata">

### Author: ![certik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/certik/32/2651_2.png) [@certik](https://discourse.julialang.org/u/certik)
#### Post date: [September 11, 2023, 9:15pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/7 "2023-09-11T21:15:43Z")

</div>

We can do other benchmarks too, we have quite a few here: [https://github.com/symengine/symengine/tree/master/benchmarks](https://github.com/symengine/symengine/tree/master/benchmarks), I just started with the above. If you agree that the speed that I got is accurate, we can move to other benchmarks. I just wasn’t sure if I am doing something wrong, or not.

Here is a simple differentiation benchmark: [Benchmarks against Symbolics.jl · Issue #1973 · symengine/symengine · GitHub](https://github.com/symengine/symengine/issues/1973#issuecomment-1714681036)

---

<div class="post-metadata">

### Author: ![blegat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/blegat/32/217090_2.png) [@blegat](https://discourse.julialang.org/u/blegat)
#### Post date: [September 12, 2023, 8:33pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/8 "2023-09-12T20:33:30Z")

</div>

We could switch to a `Matrix` representation for DynamicPolynomials, it shouldn’t be too hard.  
However, I would use the transpose of what AbstractAlgebra does I think, so that the exponents of the same monomials are contiguous in memory.  
One thing that is would be annoying is that you can’t push a new term at the end anymore since I never found how to append a new column inplace for a Julia `Matrix`. Do you know a way ?

---

<div class="post-metadata">

### Author: ![shashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shashi/32/1824_2.png) [@shashi](https://discourse.julialang.org/u/shashi)
#### Post date: [December 15, 2023, 9:47pm UTC](https://discourse.julialang.org/t/benchmarking-symbolics-jl-against-symengine-jl/103744/9 "2023-12-15T21:47:26Z")

</div>

This is actually very puzzling, I guess SymEngine’s manual memory management would be a big contributor to the performance. But I can’t account for all of the 100x slowdown in Symbolics/DynamicPolynomials/TypedPolynomials. To actually only benchmark term construction rather than expansion of 20th power, I did the following benchmark:

```julia
julia> p(acc, x, y, z) = acc * (x + y + z + 1)
p (generic function with 1 method)

julia> q(x, y, z) = (acc=1; for i=1:20; acc = expand(p(acc, x, y, z)); end; acc)
q (generic function with 1 method)

```

Symbolics (using DynamicPolynomials) is 100x slower, and TypedPolynomials is 150x slower.
