Introducing Seep.jl, a tensorflow like library for julia

Specific Domains Machine Learning
1

Introducing Seep.jl, a tensorflow like library for julia, available at https://github.com/mit-ll/Seep.jl.

I wrote this package over the last year to learn about modern neural network architectures. It still immature and there are many areas that need improvement, but I’ve decided to publish it now because it has a foundation and many of the pieces that are needed to train neural network models. We have been able to use it to reproduce recently published results and experiment with adapting techniques to our problems.

Seep builds and evaluates computational flow graphs in julia. A computational flow graph (CFG) is a directed acyclic graph where nodes represent values and edges represent data dependencies. It is similar to the Single Static Assignment form of a program used by compilers, but there is no control flow. All nodes are evaluated each time the graph is executed.

The CFG is first defined as an abstract graph using ANodes. The abstract graph specifies the size and shape of each variable, how they are connected, and how each value will be computed. It does not allocate storage or define the order of operations.

One or more instances of may be constructed from the abstract graph. Each instance is a julia function that evaluates the CFG described by the abstract graph. Storage for the values of each ANode is allocated statically when the instance is constructed. In addition to being callable as a function, the instance provides access to the Arrays on which it operates.

All feedback is welcome. Please let me know if you find it useful or interesting.

Thank you.

4 Likes
2

This looks interesting, but is there a reason to use it over TensorFlow itself?

3

One reason is that it doesn’t depend on any external libraries, a pure Julia approach. This is desirable because it requires less “code footprint” and doesn’t require compiling and installing something like TensorFlow (I still have not been able to compile TensorFlow on my work computer).

1 Like
4

I agree. Isn’t one of the elevator pitches for Julia that it ameliorates the two (or n) language problem?

I’d like to see Seep.jl move to Julia 0.5 or even 0.6 soon.

5

Firewalls are a problem with TensorFlow, but aside from that issue, Julia lends itself well to this problem. I’ve never benchmarked it, but by just outputting julia functions and letting the JIT do the rest, it’s been fast enough for my purposes.

I tried v0.5.0, and made some changes to squelch a few of the deprecation warnings, but found that performance was significantly degraded. I think that I was hitting https://github.com/JuliaLang/julia/pull/18869. I’ll try again when v0.5.1 comes out. It will be interesting to see if I can do anything with the new vectorized function calls.

I haven’t seen anything in 0.6 that looks like it will have a big impact on this, but I haven’t been following very closely. I’m sure there will be something.

6

Why wouldn’t you just download a TensorFlow binary, like the ones that ship with TensorFlow.jl?

7

This is besides the point, but they do not have a binary that works on my system. Nothing wrong if you want to use tensorflow (and I fully support your effort on TensorFlow.jl), this is just a pure Julia framework that tries to take full advantage of the Julia ecosystem.

8

Native Julia libraries, if written correctly, can take advantage of arbitrary Julia types, are easy to extend from within Julia, easier for Julia contributors to get a handle on, easier to install (just Pkg.add, not problems with building/shipping binaries), can have a better Julia-based API, among other reasons. Sure, your TensorFlow.jl wrapper is fine, but native libraries always serve a purpose, even if they aren’t as fast or stable as the dominant interop packages.

6 Likes
9

That does all sound good!

10

Could you please clarify the license of the package? The license file doesn’t actually seem to allow re-use and I doubt that it’s OSI-compatible (Licenses & Standards | Open Source Initiative).

2 Likes
11

Thank you for pointing that out. There was a line missing from the LICENSE file. I’ve added at the top

DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.

That makes it a bit more clear. I think I can apply the MIT license terms, but I need to verify with the contracts office.

12

The license should clearly allow the freedom to run, copy, distribute, study, change and improve the
software, for it to be defined free. OSI has similar requirements for open-source software (it demands freedom to use, modify, and share). I’m not sure your license meets these requirements (it mentions only distribution freedom and the use is regulated by US government).

13

The license is now MIT.

13 Likes
14

worth noting that in the usenix paper describing tensorflow, the authors criticize distbelief for using c++ to implement the layers because

Using a separate, less familiar programming language for implementing layers is a barrier for machine learning researchers.

they are in essence arguing for a pure julia implementation!

2 Likes
15

Bit late on the scene here but this looks interesting. @adambrewster can you comment on how Seep compares to Knet.jl?

Also, is there a status update on supporting newer Julia versions?

16

I’ll have to look at knet, but I won’t be able to work on this for a couple of weeks.

Regarding newer Julia versions, I have it working on 0.6, but the performance ​regression I saw in 0.5 remains. That’s an exciting branch for me because the LLVM code that you might see commented out in this release actually starts to work.

Work remaining to prepare the new version for publication is:

  • Now that 0.6 if feature frozen, I’ll clean up the remaining deprecation warnings.
  • Isolate a minimum non-working test case for the performance problem and report it.
  • Finish and clean up LLVM PTX code generation.
  • See if CUDAnative work can replace my GPU work.
  • Follow up on Defer.jl and the CUDA package resource clean up issues.
1 Like