BoltzmannMachines.jl

Specific Domains Machine Learning
1

Hi everybody,

We just put our package BoltzmannMachines.jl online. It provides a specific type of Deep Learning approach, which isn’t covered by typical Deep Learning packages, but nevertheless rather successful in applications.

The package isn’t registered yet, as it is in its early stages. Nevertheless, it is already rather comprehensive.

We are looking forward to feedback!

Best wishes,

Harald

4 Likes
2

Hi!

That looks quite cool and useful!

Are you aware of https://github.com/dfdx/Boltzmann.jl ? Judging from the readme it looks like there is some overlap, while both provide some things the other one doesn’t. Maybe there is some potential for cooperation.

3

Hi Christof,

Thanks for the suggestion. We initially had a look at the package you mentioned, and it seemed to be focused on restricted Boltzmann machines. We are mainly using the latter only as building blocks for more complex architectures, deep Boltzmann machines, leading us to a kind of lego block design for our restricted Boltzmann parts. This would not have worked as well with the Boltzmann package. Nevertheless, we will look into adding a kind of compatibility layer, also with respect to other deep learning packages.

Best wishes,

Harald

4

Boltmann.jl supports deep belief networks and deep autoencoders, so at first glance adding deep Boltzmann machines seems quite straightforward to me. Do you have examples of what you found hard to implement within that package’s architecture?

5

The two main things were:

  • We wanted to have building blocks that contain more variables than just the weights and the biases
  • We wanted to flexibly paste RBMs (and other components) together. Therefore, we chose to represent deep Boltzmann machines as arrays, where the representation of deep belief networks in Boltzmann.jl have some additional structure (mainly the labels) on top. With the raw array implementation, it is rather easy (also for the user) to add/remove layers by just modifying the array.

Best wishes,

Harald

6

We wanted to have building blocks that contain more variables than just the weights and the biases

As far as I can see from the code, you don’t add any more fields into RBM types themselves, so I assume you have them during training process. In Boltzmann.jl, there’s context (basically a Dict{Any,Any}) that can hold any additional information.

We wanted to flexibly paste RBMs (and other components) together. Therefore, we chose to represent deep Boltzmann machines as arrays, where the representation of deep belief networks in Boltzmann.jl have some additional structure (mainly the labels) on top.

Labels are just names for layers, making them a bit more convenient to work with. Yet, it’s absolutely possible to provide API to generate layer names automatically if they are not needed.

Anyway, I don’t insist on merging packages or something, but if you decide to do it one day, feel free to ping me here or on GitHub.

7

I will look into it. At least it would be convenient if we could provide compatibility to allow for getting a deep belief network for further processing with Boltzmann.jl after starting with BoltzmannMachines.jl, as we don’t offer deep belief networks and probably never will.

1 Like
8

Wherever this compatibility thought leads, I just wanted to say that it is quite nice of you to open-source your implementation of Boltzmann Machines. I am sure people looking for such functionality will appreciate the ability to choose.

Naturally there is a tendency to point out existing work whenever one presents a similar package to encourage cooperation, but at the end of the day people can have quite different visions for the implementation of the same concept. So I hope this feedback/discussion doesn’t feel discouraging to you, because I am sure it is not the intention to discourage you from continuing your work in whatever way seems most sensible to you.

3 Likes
9

Thanks for the feedback. Indeed, we have quite some plans for further development of the package.