Request: Library examples

Community Meetups
1

Hi,
I’m starting to plan an October meetup in Seattle, and I’m thinking it could be something like some very introductory material, followed by some use cases with libraries. Ideally, I’d like each of the use cases to be one or two slides with these criteria:

  • Relatively short to express
  • Easy to understand for “informed non-specialists”
  • Impressive in conciseness and/or performance for specialists

A few initial ideas for examples:

  • PyCall
  • RCall
  • DifferentialEquations
  • Flux
  • JuMP
  • Queryverse

Is there any kind of repository with something like this? If not, I would love if some library authors could respond with an example that fits these criteria. This could be really useful in promoting the language.

If there’s nothing already available, I’ll take a crack at slides and make any result public.

1 Like
2

Remind me and I’ll try to make a two slide intro. I haven’t been successful at such a succinct introduction yet but maybe with some workshopping it’ll get there

1 Like
3

I’m not a package author, but I use JuMP a lot and recently held a mini-course in Julia & JuMP for PhD students focused on energy systems modeling. As a first introduction to JuMP I used my rewritten version of a multicommodity transport problem from the JuMP Github repo. Most of it could fit on two slides of about 20 lines each if you can accept a smallish font size. Something like this:

Slide 1 (“sets” and parameters). The point is to show that JuMP is just ordinary Julia.

# index sets
ORIG = [:gary, :clev, :pitt]
DEST = [:fra, :det, :lan, :win, :stl, :fre, :laf]
PROD = [:bands, :coils, :plate]

# supply(PROD, ORIG) amounts available at origins
supplytable = [
:_      :gary   :clev   :pitt
:bands  400     700     800
:coils  800     1600    1800
:plate  200     300     300
]
supply = readtable(supplytable)

# demand(PROD, DEST) amounts required at destinations
demandtable = [
:_      :fra :det :lan :win :stl :fre :laf
:bands  300  300  100  75   650  225  250
:coils  500  750  400  250  950  850  500
:plate  100  100   0    50   200  100  250
]
demand = readtable(demandtable)

Slide 2 (the JuMP model):

multi = Model(solver=ClpSolver())   # or CbcSolver() or GLPKSolverLP()

@variables multi begin
    Trans[p in PROD, o in ORIG, d in DEST] >= 0
end

# Changed constraints to inequalities.
@constraints multi begin
    c_supply[p in PROD, o in ORIG],
        sum(Trans[p,o,d] for d in DEST) <= supply[p,o]

    c_demand[p in PROD, d in DEST],
        sum(Trans[p,o,d] for o in ORIG) >= demand[p,d]

    c_total_shipment[o in ORIG, d in DEST],
        sum(Trans[p,o,d] for p in PROD) <= limit[o,d]
end

# For some reason the original file cost maximizes. Changed to cost minimization.
@objective multi Min begin
    sum(cost[p,o,d] * Trans[p,o,d] for p in PROD, o in ORIG, d in DEST)
end

status = solve(multi)

Alternatively, make one slide where you explain what JuMP is and just use my second slide. Delete the comments to make it a bit shorter.

It’s possible to make a complete JuMP demo in just a few lines, but that would just look like a clunkier version of the matrix-based optimization solvers of Matlab. To get a taste of why JuMP is so great for formulating and solving large constrained optimization problems I think this is the smallest useful example size.

Oh, another reason for showing a JuMP slide is that it’s a great demo of how Julia’s macro system makes high level magic possible.

Download the complete executable source code here (note you need Julia 0.6 since JuMP doesn’t have official 1.0 support yet).

EDIT: Sorry, somehow I completely missed that your main criteria were “short to express” and “impressive in conciseness”. My example hardly fits the bill then. Oh well, at least it’s “easy to understand for informed non-specialists”. :slight_smile:

1 Like
4

Thank you @ChrisRackauckas, that would be really great. Not looking for anything to give a deep understanding at this point - ideal reaction is something like “Wow, it can do that? I want to learn more”

@NiclasMattsson this is a really cool example, thank you! Which readtable is this? Seems different than the one in DataFrames.jl

5

GitHub - KristofferC/NearestNeighbors.jl: High performance nearest neighbor data structures and algorithms for Julia. is pretty easy to understand the purpose of and should have good performance.

For example

julia> using NearestNeighbors
[ Info: Precompiling NearestNeighbors [b8a86587-4115-5ab1-83bc-aa920d37bbce]

julia> data = rand(3, 10^6);

julia> kdtree = KDTree(data);

julia> naivetree = BruteTree(data); # does the naive thing

julia> @btime knn(kdtree, $(zeros(3)), 10)  # find 10 closest points to [0,0,0]
  635.988 ns (4 allocations: 368 bytes)
([914205, 493477, 101721, 350699, 751665, 481216, 849755, 603272, 398143, 248106], [0.0272309, 0.0270436, 0.0197171, 0.0179862, 0.0242389, 0.0169534, 0.00533717, 0.0162161, 0.0132264, 0.0101002])

julia> @btime knn(naivetree, $(zeros(3)), 10) # find 10 closest points to [0,0,0]
  4.290 ms (3 allocations: 352 bytes)
([914205, 493477, 751665, 398143, 101721, 350699, 481216, 849755, 248106, 603272], [0.0272309, 0.0270436, 0.0242389, 0.0132264, 0.0197171, 0.0179862, 0.0169534, 0.00533717, 0.0101002, 0.0162161])
1 Like
6

Just a simple function I wrote to read tables with 2D data into a Dict:

function readtable(a)
    colheader = a[1, 2:end]
    rowheader = a[2:end, 1]
    data = a[2:end, 2:end]
    return Dict((rowheader[r], colheader[c]) => data[r,c] for r=1:length(rowheader), c=1:length(colheader))
end

supplytable = [
:_      :gary   :clev   :pitt
:bands  400     700     800
:coils  800     1600    1800
:plate  200     300     300
]

julia> supply = readtable(supplytable)
Dict{Tuple{Symbol,Symbol},Int64} with 9 entries:
  (:coils, :clev) => 1600
  (:coils, :pitt) => 1800
  (:bands, :gary) => 400
  (:plate, :pitt) => 300
  (:plate, :gary) => 200
  (:coils, :gary) => 800
  (:bands, :pitt) => 800
  (:plate, :clev) => 300
  (:bands, :clev) => 700

There’s another version that reads tables with 3D data in the file I linked above.

1 Like
7

Here’s one that might be eye catching. Solving a differential equation simulation and getting error bars in one numerical solution.

http://nbviewer.jupyter.org/format/slides/github/ChrisRackauckas/JuliaAndDiffEq/blob/master/JuliaAndDiffEq.ipynb#/17

The emoji form for defining differential equations always gets people too:

http://nbviewer.jupyter.org/format/slides/github/ChrisRackauckas/JuliaAndDiffEq/blob/master/JuliaAndDiffEq.ipynb#/12

8

This is great, thank you @ChrisRackauckas! I’d still like to learn more about this stuff, especially SDEs and how they connect with Gaussian Processes. (A Wiener process is a GP, but I’m out of my depth quickly after that).

By the way, I really like the idea of DiffEqTutorials.jl. There’s a small problem in the docs getting it going with IJulia - I opened this issue.

9

If you have an inclination towards stochastic processes I have something: