# Introducing Seep.jl, a tensorflow like library for julia

**URL:** <https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254>\
**Category:** Machine Learning\
**Created:** [January 3, 2017, 1:42am UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254 "2017-01-03T01:42:07Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![adambrewster](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@adambrewster](https://discourse.julialang.org/u/adambrewster)\
**Post date:** [January 3, 2017, 1:42am UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/1 "2017-01-03T01:42:07Z")

</div>

Introducing Seep.jl, a tensorflow like library for julia, available at [https://github.com/mit-ll/Seep.jl](https://github.com/mit-ll/Seep.jl).

I wrote this package over the last year to learn about modern neural network architectures. It still immature and there are many areas that need improvement, but I’ve decided to publish it now because it has a foundation and many of the pieces that are needed to train neural network models. We have been able to use it to reproduce recently published results and experiment with adapting techniques to our problems.

Seep builds and evaluates computational flow graphs in julia. A computational flow graph (CFG) is a directed acyclic graph where nodes represent values and edges represent data dependencies. It is similar to the Single Static Assignment form of a program used by compilers, but there is no control flow. All nodes are evaluated each time the graph is executed.

The CFG is first defined as an abstract graph using ANodes. The abstract graph specifies the size and shape of each variable, how they are connected, and how each value will be computed. It does not allocate storage or define the order of operations.

One or more instances of may be constructed from the abstract graph. Each instance is a julia function that evaluates the CFG described by the abstract graph. Storage for the values of each ANode is allocated statically when the instance is constructed. In addition to being callable as a function, the instance provides access to the Arrays on which it operates.

All feedback is welcome. Please let me know if you find it useful or interesting.

Thank you.

---

<div class="post-metadata">

**Author:** ![malmaud](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/malmaud/32/29_2.png) [@malmaud](https://discourse.julialang.org/u/malmaud)\
**Post date:** [January 4, 2017, 3:54pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/2 "2017-01-04T15:54:46Z")

</div>

This looks interesting, but is there a reason to use it over TensorFlow itself?

---

<div class="post-metadata">

**Author:** ![justin\_goodwin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/justin_goodwin/32/14125_2.png) [@justin\_goodwin](https://discourse.julialang.org/u/justin_goodwin)\
**Post date:** [January 4, 2017, 8:08pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/3 "2017-01-04T20:08:36Z")

</div>

One reason is that it doesn’t depend on any external libraries, a pure Julia approach. This is desirable because it requires less “code footprint” and doesn’t require compiling and installing something like TensorFlow (I still have not been able to compile TensorFlow on my work computer).

---

<div class="post-metadata">

**Author:** ![bpr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bpr/32/2710_2.png) [@bpr](https://discourse.julialang.org/u/bpr)\
**Post date:** [January 4, 2017, 8:59pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/4 "2017-01-04T20:59:43Z")

</div>

> [@justin\_goodwin](#):
>
> One reason is that it doesn’t depend on any external libraries, a pure Julia approach.

I agree. Isn’t one of the elevator pitches for Julia that it ameliorates the two (or n) language problem?

I’d like to see Seep.jl move to Julia 0.5 or even 0.6 soon.

---

<div class="post-metadata">

**Author:** ![adambrewster](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@adambrewster](https://discourse.julialang.org/u/adambrewster)\
**Post date:** [January 4, 2017, 10:15pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/5 "2017-01-04T22:15:06Z")

</div>

Firewalls are a problem with TensorFlow, but aside from that issue, Julia lends itself well to this problem. I’ve never benchmarked it, but by just outputting julia functions and letting the JIT do the rest, it’s been fast enough for my purposes.

I tried v0.5.0, and made some changes to squelch a few of the deprecation warnings, but found that performance was significantly degraded. I think that I was hitting [https://github.com/JuliaLang/julia/pull/18869](https://github.com/JuliaLang/julia/pull/18869). I’ll try again when v0.5.1 comes out. It will be interesting to see if I can do anything with the new vectorized function calls.

I haven’t seen anything in 0.6 that looks like it will have a big impact on this, but I haven’t been following very closely. I’m sure there will be something.

---

<div class="post-metadata">

**Author:** ![malmaud](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/malmaud/32/29_2.png) [@malmaud](https://discourse.julialang.org/u/malmaud)\
**Post date:** [January 5, 2017, 7:21pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/6 "2017-01-05T19:21:57Z")

</div>

Why wouldn’t you just download a TensorFlow binary, like the ones that ship with TensorFlow.jl?

---

<div class="post-metadata">

**Author:** ![justin\_goodwin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/justin_goodwin/32/14125_2.png) [@justin\_goodwin](https://discourse.julialang.org/u/justin_goodwin)\
**Post date:** [January 5, 2017, 9:52pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/7 "2017-01-05T21:52:36Z")

</div>

This is besides the point, but they do not have a binary that works on my system. Nothing wrong if you want to use tensorflow (and I fully support your effort on TensorFlow.jl), this is just a pure Julia framework that tries to take full advantage of the Julia ecosystem.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 5, 2017, 10:27pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/8 "2017-01-05T22:27:18Z")

</div>

> [@malmaud](#):
>
> Why wouldn’t you just download a TensorFlow binary, like the ones that ship with TensorFlow.jl?

Native Julia libraries, if written correctly, can take advantage of arbitrary Julia types, are easy to extend from within Julia, easier for Julia contributors to get a handle on, easier to install (just `Pkg.add`, not problems with building/shipping binaries), can have a better Julia-based API, among other reasons. Sure, your TensorFlow.jl wrapper is fine, but native libraries always serve a purpose, even if they aren’t as fast or stable as the dominant interop packages.

---

<div class="post-metadata">

**Author:** ![malmaud](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/malmaud/32/29_2.png) [@malmaud](https://discourse.julialang.org/u/malmaud)\
**Post date:** [January 6, 2017, 4:50pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/9 "2017-01-06T16:50:14Z")

</div>

That does all sound good!

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [January 7, 2017, 12:40pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/10 "2017-01-07T12:40:40Z")

</div>

Could you please clarify the license of the package? The license file doesn’t actually seem to allow re-use and I doubt that it’s OSI-compatible ([Licenses & Standards | Open Source Initiative](https://opensource.org/licenses)).

---

<div class="post-metadata">

**Author:** ![adambrewster](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@adambrewster](https://discourse.julialang.org/u/adambrewster)\
**Post date:** [January 7, 2017, 4:53pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/11 "2017-01-07T16:53:28Z")

</div>

Thank you for pointing that out. There was a line missing from the LICENSE file. I’ve added at the top

> DISTRIBUTION STATEMENT A. Approved for public release: distribution unlimited.

That makes it a bit more clear. I think I can apply the MIT license terms, but I need to verify with the contracts office.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [January 7, 2017, 5:20pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/12 "2017-01-07T17:20:24Z")

</div>

The license should clearly allow the freedom to run, copy, distribute, study, change and improve the  
software, for it to be defined [free](https://www.gnu.org/philosophy/free-sw.html). OSI has similar requirements for open-source software (it demands freedom to use, modify, and share). I’m not sure your license meets these requirements (it mentions only distribution freedom and the use is regulated by US government).

---

<div class="post-metadata">

**Author:** ![adambrewster](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@adambrewster](https://discourse.julialang.org/u/adambrewster)\
**Post date:** [January 9, 2017, 8:56pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/13 "2017-01-09T20:56:16Z")

</div>

The license is now MIT.

---

<div class="post-metadata">

**Author:** ![bjarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bjarthur/32/9638_2.png) [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Post date:** [March 15, 2017, 3:37pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/14 "2017-03-15T15:37:31Z")

</div>

worth noting that in the usenix paper describing tensorflow, the authors criticize distbelief for using c++ to implement the layers because

> Using a separate, less familiar programming language for implementing layers is a barrier for machine learning researchers.

they are in essence arguing for a pure julia implementation!

---

<div class="post-metadata">

**Author:** ![jobjob](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jobjob/32/160_2.png) [@jobjob](https://discourse.julialang.org/u/jobjob)\
**Post date:** [March 31, 2017, 5:41am UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/15 "2017-03-31T05:41:34Z")

</div>

Bit late on the scene here but this looks interesting. @adambrewster can you comment on how Seep compares to [Knet.jl](https://github.com/denizyuret/Knet.jl)?

Also, is there a status update on supporting newer Julia versions?

---

<div class="post-metadata">

**Author:** ![adambrewster](https://avatars.discourse-cdn.com/v4/letter/a/ee59a6/32.png) [@adambrewster](https://discourse.julialang.org/u/adambrewster)\
**Post date:** [March 31, 2017, 10:46pm UTC](https://discourse.julialang.org/t/introducing-seep-jl-a-tensorflow-like-library-for-julia/1254/16 "2017-03-31T22:46:26Z")

</div>

I’ll have to look at knet, but I won’t be able to work on this for a couple of weeks.

Regarding newer Julia versions, I have it working on 0.6, but the performance ​regression I saw in 0.5 remains. That’s an exciting branch for me because the LLVM code that you might see commented out in this release actually starts to work.

Work remaining to prepare the new version for publication is:

- Now that 0.6 if feature frozen, I’ll clean up the remaining deprecation warnings.
- Isolate a minimum non-working test case for the performance problem and report it.
- Finish and clean up LLVM PTX code generation.
- See if CUDAnative work can replace my GPU work.
- Follow up on Defer.jl and the CUDA package resource clean up issues.