Symbolics.jl bootstrapping a beginner

I’d like to be able to follow along my process of learning geometric algebra from scratch using Symbolics.jl

I like how I’ve seen multiple-dispatch allow symbolic expressions to travel through standard libraries. I am aware of Grassman.jl but it is way too hard for me to bootstrap myself into.

I need to start really low… Please tell me if Symbolics is unsuitable for me.

Here’s something really extra basic I want to do, but I cant find any instruction…
I want to create complex numbers from scratch using Symbolics. I don’t want to use existing packages or built-ins.

How do i create an i such that i squared is -1, and if I multiply two of these guys together it simplifies to -1?

It’s possible that Metatheory.jl will help me out here.
Still no idea how to make a unique entity ‘i’ which has the sqrt(-1) behaviour.
Sure is interesting - I might learn something :shushing_face:

Can some kind person please explain to me why I get no response to this question?

To learn things I need to build systems. In this case I want to build a basic geometric algebra system from the ground up. So I can break things and fix them.

I don’t really learn that well from pushing symbols around on on paper with a pencil.

If need to go somewhere else, please let me know.

Hello @banksiaboy

Sorry you haven’t received a response sooner. Or from somebody more knowledgeable.

For a start, you can overload base functions where applicable, e.g.

import Base: *
struct i
end
*(x::i, y::i) = -1
z = i()
z*z # -1

Also, in the Symbolics GitHub repo it is stated " * SymbolicUtils.jl: This is a rule-rewriting system that is the core of Symbolics.jl. Symbolics.jl builds off of SymbolicUtils.jl to extend it to a whole symbolic algebra system, complete with support for differentation, solving symbolic systems of equations, etc. If you’re looking for the barebones to build a new CAS for specific algebras, SymbolicUtils.jl is that foundation. Otherwise, Symbolics.jl is for you."

So, perhaps SymbolicUtils is a better fit for your purposes?

4 Likes

My guess is: because Symbolics.jl is so new, many questions about it may only be answerable by @shashi and he probably has a lot on his plate.

If you’re looking at Metatheory.jl as well, there’s conversation on Zulip.

Yeah I would do it on SymbolicUtils.jl directly, but you are entering @shashi’s domain there and he’s rarely on Discourse so you should go on the Slack/Zulip for that.

1 Like

@banksiaboy Welcome! So first off, I have to ask, do you actually need this system to be symbolic? E.g. you mention Grassmann.jl but I’ll note that’s not really a symbolic library. Rather it’s a bit of an alternative to using matrices / tensors, and the objects it represents can hold symbolic quantities.

If you do all your geometric algebra manipulations symbolically, it’ll probably be pretty slow and won’t interoperate as well with the rest of the ecosystem, but might have some interesting advantages.

Now, if you do want to go the symbolic route, I’m not sure I have any great recommendations for you. I think SymbolicUtils will eventually be the place to do this, but it currently has some barriers in the way. It technically can support rewrites of non-commutatitve objects, but doing so is actually pretty awkward, and you’d probably not be able to rely on it’s builtin rulesets as a foundation. You’d instead need to reinvent a lot of wheels I suspect.

Metatheory is a very interesting project that will hopefully become something like a backend for SymbolicUtils. I quite like some aspects of it (and get really annoyed by other aspects of it, like some API and interface choices). The most interesting part to me, is that you can write ‘circular’ rulesets without worrying about them causing a never-ending loop like what happens in regular term rewriting systems.

Either way, even just writing for you an example of symbolic complex numbers that ‘work’ correctly is actually a pretty non-trivial task and this is probably why nobody has done it yet.

One thing I should add in addition is that I think one’s best bet to get started with GeometricAlgebra in Julia without too much work is to just use matrices, and interpret them in terms of geometric algebra.

Geometric algebra true believers generally sneer at the use of matrices and claim that using a matrix is inefficient, and sometimes less accurate. If you use StaticArrays.jl, the performance claims don’t really hold up, though the accuracy concerns maybe carry a kernel of legitimacy, I think they’re a little contrived though.

On the upside,you’d benefit from having the full julia linear algebra ecosystem already implemented for you.

Now, you asked for help first doing this in terms of complex numbers first so that you may bootstrap yourself up to the full geometric algebra, but it’s actually easier for me if I show you how to do this with the 2D real geometric algebra. I’ll leave it to you to figure out how to do the 3D version, okay?

Lets start with the usual building blocks. We want an identity element 𝟙, and two linearly independent basis 'vector’s σ1 and σ2 which square to the identity and anticommute. We can write these as

using StaticArrays

𝟙 = SA[1 0
       0 1]

σ1 = SA[0  1
        1  0]

σ2 = SA[1  0
        0 -1]

Here’s your proof that these have the correct behaviour:

julia> σ1^2 == 𝟙
true

julia> σ2^2 == 𝟙
true

julia> σ1 * σ2 == - σ2 * σ1
true

Now, 2x2 matrices span a 4D vector space, but we currently only have 3 basis elements. What’s the third? We construct it with products of the existing elements. For the 2d algebra, there’s only one such independent product: σ1 * σ2, which we’ll call

ℐ = σ1 * σ2

(that’s typed \scrI<TAB>) and this has all the desired properties of complex numbers, e.g.

julia> ℐ^2 == -𝟙
true

julia> exp(π * ℐ) ≈ -𝟙
true

julia> exp(π/2 * ℐ) ≈ ℐ
true

julia> exp(π/4 * ℐ)' * σ1 * exp(π/4 * ℐ) ≈ σ2
true

So you can just use these things for all your algebraic manipulations, but one problem you’ll encounter is that the way these things print isn’t very informative. i.e.

julia> 1𝟙 + 2σ1 + 3σ2 + 4ℐ
2×2 SMatrix{2, 2, Int64, 4} with indices SOneTo(2)×SOneTo(2):
 4  -2
 6  -2

What can we do about this? Well, once we have our basis constructed, it’s actually quite easy to just project any old matrix onto that basis and read off their elements:

function decomp(basis::NamedTuple{basisnames}, M::AbstractMatrix) where {basisnames}
    N = size(M, 1)
    @assert N == size(M, 2)

    map(Tuple(basis)) do xi
        elem = (1//N) * (M ⋅ xi)
    end |> NamedTuple{basisnames}
end 

and print them nicely

GAprint(basis) = M -> GAprint(basis, M)
function GAprint(basis::NamedTuple{basisnames}, M::AbstractMatrix) where {basisnames}
    elems = map(decomp(basis, M) |> Tuple) do elem
        if elem isa Rational && denominator(elem) == 1
            numerator(elem)
        else
            elem
        end
    end
    
    s = join(("$(elems[i]) $(basisnames[i])" for i in eachindex(elems) if elems[i] != 0), " + ")
    if s == ""
        println("𝟘")
    else
        println(s)
    end
end

e.g.,

basis2d = (;𝟙, σ1, σ2, ℐ)
julia> SA[4 -2; 6 -2] |> GAprint(basis2d)
1 𝟙 + 2 σ1 + 3 σ2 + 4 ℐ

julia> exp(π/4 * ℐ)' * σ1 * exp(π/4 * ℐ) |> GAprint(basis2d)
3.3306690738754696e-16 σ1 + 1.0000000000000002 σ2

Is this at all helpful @banksiaboy?

9 Likes

@Mason, @ChrisRackauckas Thanks, and to everyone else also - much appreciated.

I’ve had a long-standing, but unrequited interest in Symbolic programming. Ever since I read an article in the 1970’s where Macsyma was used to destroy a book of standard integrals - 40% wrong from memory.

I have this thing where I want to learn maths by algorithmic construction. I did about 18 months of a Type Theory meetup in Sydney - which was run by good people. I was the last to drop out when we hit Homotopy Type Theory. I didn’t really have the background to do this - but what the heck. The Sydney FP meetup was loaded with people who used Coq, Agda and Isabelle - in the type theory group we tinkered unsuccessfully with Idris :slight_smile: So I can see how hard it might be to do the geometric algebra basis. Complex numbers were a first step for me.

I have just spent some months doing Fastai’s Pytorch course. The 20,000 foot view is good, but I hated Python - architecture by patch.

Started looking at the underpinnings of CNNs got interested in Michael Bronstein’s
Geometric Deep Learning, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Went back over some old work I’d done at a 3D CG Transputer startup on geometry with Homogeneous Coordinates, and then ran into Geometric Algebra Joan Lasenby, David Hestenes - Tutorial on Geometric Calculus.

So from the very basics, and from first principles, in the spirit, and at the level of MIT’s Introduction to Computational Thinking I want to hack my way into combining
Geometric Algebra, Geometric Deep Learning - starting from trying to build simple-enough expressions that describe CNNs in a modern, structured way.

I was hoping that I’d be able to do the basic concepts using the Julia ecosystem. I’m beginning to think that computer algebra, in its current state won’t be much of a help.

Unfortunately Grassman.jl probably does everything I want, but has no visible entry at the beginner’s level. So I’m not up to using it yet. And can’t find a way in.

Ideally Id like to know enough to bang Flux.jl, Grassman.jl, Symbolics.jl, Metatheory.jl together with what I’m seeing in Geometric Deep leaning.

After what people have told me here, I will start prototyping in the Julia type system, and see if I can learn enough to use Grassman.jl

2 Likes

@Mason thanks so much. I’ll start where you suggest. A more practical set of moving parts. I really appreciate the amount of work you put into your replies to me.

3 Likes

Oh yeah - and it could easily be the case that I’m not talented enough to do this, but often my intuition is OK. I’ll see how I go…

Hm, well if you’re passionate about trying to do this in a symbolic system, then maybe give it a try and see if you’re able to fix things or open issues upstream as they appear. Dealing with non-commutative quantities with SymbolicUtils.jl is going to be pretty painful though as it gets complicated. I’ve wanted for a while to contribute some stuff to work well with non-commutative quantities well in SymbolicUtils.jl but just haven’t been able to get around to it.

Regarding Grassmann.jl, it has some great ideas in it and has a large quantity and quality of work put into it, but it also does some dangerous things like abuse the @pure macro and is liable to break in some very subtle and scary ways. It’s developer can rather difficult as well.

1 Like

@banksiaboy Okay, so I’ve drafted a PR for SymbolicUtils.jl that has the necessary infrastructure to deal with non-commutative rules well. https://github.com/JuliaSymbolics/SymbolicUtils.jl/pull/377

This’ll probably take a while to be ready to be merged and might change a lot in the meanwhile, but things are slowly moving :slight_smile:

2 Likes

I had a problem with GAprint(basis::NamedTuple{basisnames}, M::AbstractMatrix) where the dot operator in (M ⋅ xi) is undefined. I fixed that with using LinearAlgebra.

I tried converting your 2d GA example to Pluto - but GAprint output went to the REPL. Which seems to be an existing issue with Pluto. Changed over to a JupyterLab notebook, and all runs and displays fine.

I love the way the matrices have the right behaviour - this is the way to learn algebra!

All set for tinkering :slight_smile:

1 Like

Ah good point. I should have pointed that out.

Oh yeah, I forgot about Pluto’s printing behaviour. Well, one strategy then would be to wrap the matrices in a struct that knows about a basis and can do the printing automatically. I wrote up a quick and dirty package here: GitHub - MasonProtter/GeometricMatrixAlgebras.jl: Geoemtric Algebra using julia matrices

and I wrote some basic stuff in a Pluto notebook here and it seems to work pretty well. I kinda like the idea of using a Pluto notebook as the documentation for a package.

If you’re interested, we could collaborate together on GeomtricMatrixAlgebras.jl. Basically, as you tinker and learn, you can submit documentation with what you found and learned. You will also likely find that the package has certain limitations and you can submit fixes for it. If you don’t know how to do those fixes, I can either do them myself, or guide you on how to do them (when I find the time).

Could be fun, but no pressure if you’re not into it.

3 Likes

Thanks @Mason. Couldn’t be happier - thanks again!
I did a basic check of GeometricMatrixAlgebras.jl, and have created a pull-request which fixes a few typos.

I have a very old project which did wire-frame 3D objects with hidden-surface removal and perspective projection using homogeneous coordinates, which I might try and convert. I think it is still a good demonstration of 3D geometry fundamentals. I’m interested to see the effect of using GA on it. It should let me incrementally build something interesting from a simple base, and exercise GeometricMatrixAlgebras.jl.

I got it working as an under-grad project - which I was allowed to do as an alternative to the vector algebra course. Didn’t have any graphical output devices - so I wrote some software to make it print on the compute-centre’s line-printer. Very entertaining for people wanting their COBOL output.

Crow, Franklin C. Three-Dimensional Computer Graphics Part1 Byte 1981 There never was a part 2 :slight_smile:

Dorst, Leo, Fontijne, Daniel, and Stephen Mann. Geometric Algebra for Computer Science: An Object-Oriented Approach to Geometry. Amsterdam: Elsevier, Morgan Kaufmann, 2010.
– Has an exercise on hidden-surface-removal, which may give me some guidance.

1 Like

I’ll try and modernise the approach using CGPP as a base. Algorithmic conversion to triangle meshes from the model polygons could be interesting.

Hughes, John F. Computer Graphics: Principles and Practice. Third edition. Upper Saddle River, New Jersey: Addison-Wesley, 2014