Why do we need the `do` keyword?

Internals & Design
1

According to the documentation, all this keyword does is create a lambda and insert it as the first argument of function call before it. This task could be accomplished with a macro with very similar syntax:

@do
map(coll) x -> x^2

I wanted to know the reason behind integrating this feature into the language as a keyword.

3 Likes
2

I donā€™t know if this is the reason, but I often use it for very complex lambdas. For example, the by() function for dataframes:

by(my_df, :col) do df
    # a whole mess of stuff...
end

Iā€™m not sure how easy your macro would be with a multiline expression, but it also doesnā€™t seem all that much simpler in the example you provided (in that case Iā€™d probably just stick the lambda inside the function call).

2 Likes
3

You can use begin blocks for complex lambdas, theyā€™re apparently just syntactic sugar for anonymous functions. Hereā€™s a macro which does exactly what do keyword does:

macro m_do(func, callable)
    if !isa(func, Expr) || func.head != :call
        error()
    end
    insert!(func.args, 2, callable)
    return func
end

@m_do map([1,2,3]) x -> begin
    println(x)
    return x*x
end

It isnā€™t very complicated because the parser translates the map call and the lambda to two bulk sexprs (Iā€™m not familiar with the source, but it certainly appears so).

4

We donā€™t need it, we want it.

@do f(y) x -> begin
...
end

is longer and less pretty than

f(y) do x
...
end

This is an operation we want to be pretty because it is really commonly used.
It basically takes the place of pythonā€™s with statements and more.
Basically this is a really common and useful pattern.
Some examples of the places I tend to use it:

open("file.txt") do fh
...
end


pmap(1:100) do x
...
end

mapreduce(hcat, 1:100) do x
 # return a vector
end

lock(lk) do
...
end

withenv(API_KEY="123423") do 
...
end

mktempdir() do dirname
...
end

sum(xs) do 
    ...
end

I wouldnā€™t like to make all those uglier.

At the end of the day, why do we do anything?
Since julia is just going to compile to machine code anyway.

Or less reductio ad absurdum: why allow anon functions?
Why not just have people write functions normally?
Or use a macro to create real functions out of lambdas syntax?

Looking at the history do blocks have been in the language since before 0.1 release.
Which is before my time.
The original reason is thus probably buried in a email chain somewhere.

22 Likes
5

So itā€™s an ergonomic feature. I didnā€™t know about these use cases since Iā€™m new to julia. Thanks for clarification.

1 Like
6

So itā€™s an ergonomic feature.

Correct.

I didnā€™t know about these use cases since Iā€™m new to julia. Thanks for clarification.

Youā€™re welcome

7
7 Likes
8

:joy::joy:, best phrase!

9

I never use do blocks because of #15276. Thereā€™s no way to combine the let block trick with do syntax.

10

I donā€™t use do blocks, because I can never quite grok them, or at least only for a few hours at a time. I read the documentation, and then sort of get it, but next time around I have no idea.

I think it might be related to the spelling. ā€œWho is doing what to whom?ā€ I can never read it correctly. Sometimes I try to replace it with Pythonā€™s ā€œwith ā€¦ as ā€¦ā€ and then itā€™s ok, but most of the time that doesnā€™t work. In summary, it might not be the concept thatā€™s so hard, but the word do really throws me off.

Does anyone have a tip for how to make it sound right inside my head?

1 Like
11

I am not sure I understand the requirement, but I would not invest too much in mapping it to a construct/sentence in a natural language, and would recommend trying to understand them as they are.

There is not much to do blocks, just syntactic sugar.

f((x, y) -> body..., args...)

is equivalent to/can be written as

f(args...) do x, y
    body...
end
4 Likes
12

The reason I try to do that is that even when I occasionally figure out what it means, it never sticks, but seems to slip through my mental fingers, as it were. I donā€™t have this problem with other parts of the syntax. The substitution/reshuffling of the sequence of ā€˜thingsā€™, as well as the word do itself trips me up. I canā€™t help reading it as ā€œdo x, yā€ (how can I do ā€˜x, yā€™?)

Anyway, thanks for the explanation. Maybe there is no ā€˜trickā€™.

13

I guess the closest you could get is to read

f(args...) do inner_args...
  ...
end

as

f(args...) does with inner_args...
   these operations
end

which makes sense in many cases:

map(rand(10)) does with x
  x*2
end

Or maybe does to x?

1 Like
14

Thanks. Maybe a sort of visual substitution is the way to go.