How to add (anonymous) functions?

General Usage
1

Hi everybody,

I want to add (reduce?) anonymous functions and I can’t figure out how to do it.

Context: I want to speed up my computation like this:

function g(c, x)
    return hard_computation(c)(x)
end

→ 

function gen_f(c)
    solution = hard_computation(c)
    return x -> solution(x)
end
f = gen_f(5)

Function f is much faster than g because the result of the hard_computation is cached inside f (I assume).

Now my hard_computation returns not a single but a bunch of functions and I’d like to combine them into one, for example, if my hard_computation returns just a cos with frequency and amplitude, I want to turn turn

x -> 1 * cos(1*x)
x -> 2 * cos(2*x)
x -> 3 * cos(3*x)

into

x -> 1 * cos(1*x) + 2 * cos(2*x) + 3 * cos(3*x)

So let’s say I have a Dict or Array which I’m iterating over to get the function parameters:

function g(c, x)
    r = 0.0
    for (a, b) in c
        r += a * cos(b * x)
    end
    return r
end

function gen_f(c)
    f = 0.0
    for (a, b) in c
        f += x -> a * cos(b * x)  # well this won't work, obviously...
    end
    return x -> f(x)
end

Doing it the function g-way is easy, just calculate each result and add them. However, that way I have to perform the “hard computation” every time I run the function, which I would like to avoid.
How do I do this the function f-way?

If anyone has any other comments or tips on this kind of writing programs, I would love to hear them, as this is fairly new to me.

2

Just return solution directly.

There doesn’t seem to be any hard computation involved. Anonymous functions are not magic. If you couldn’t split the function into two parts where one part doesn’t depend on the x while doing some meaningful computation then anonymous function cannot help you.

3

I don’t fully understand what you want to achieve but you could use something like

function gen_f(c)
    functions = [x -> a * cos(b * x) for (a, b) in c]
    x -> sum(x .|> functions)
end
2 Likes
4

If the only goal is to create anonymous functions, then sure. If you want to anything faster or easier to write or easier to understand, then don’t.

5

Thanks for your answer!

Yes, in this case I could return just solution, in my actual use case I have to do some more calculation with solution, that’s why I return an anonymous function here.

Hm, if I understand you correctly, we are currently misunderstanding each other :smiley:
As an example I basically used

hard_computation(a, b) = x -> a * cos(b * x)

which is obviously not really a hard computation at all. But if it actually was a hard computation, this would benefit from being cached like in the very first examples (right?). I’m just not sure on how to do that with more that one return functions.

6

This does exactly what I was hoping for, thank you!

But since my question seemed to raise some questions as well as eyebrows, is this not a reasonable thing to do? Maybe I should have instead written

function g(c, x)
    r = 0.0
    for (a, b) in c
        r += hard_computation(a, b) * cos(x)
    end
    return r
end

If I then do

g.(c, large_array)

the
hard_computation will have to run each iteration (correct?).
If I however do

function gen_f(c)
    functions = [x -> hard_computation(a, b) * cos(x) for (a, b) in c]
    x -> sum(x .|> functions)
end
f = gen_f(c)
f.(large_array)

(assuming this really does what I think it does), this problem is avoided.

Maybe I have been timing things wrongly but at least for me function f performs much better than function g.

7

By not being a hard computation I mean there’s nothing you can precompute here. Creating the anonymous function isn’t hard.

This is why you need to be more specific about the actual structure of your code. If this is actually what you want then yes hard_computation(a, b) does not depend on x so it can be precomputed and a much better way to do it would be.

function gen_f(c)
    c2 = sum(hard_computation(a, b) for (a, b) in c)
    return x->cos(x) * c2
end

As you can see, what you can pull out strongly depend on the structure of the actual computation. i.e.

If you are actually timing the code you write above as is, then I highly doubt that, your ccode doesn’t actually precompute anything.

8

That is, of course assuming the iteration isn’t the most computationally extensive part. It could be for Dict in which case you can just convert it into an array of tuples or pairs first.

In any case, the bottom line is that you need to identify what’s the computation that can be hoisted. After that, it should be very easy to create a datastructure to cache the result. In almost all cases though, creating an array of functions will not help you at all, you’ll be much better off just create an array of the values you want to capture and just do the computation you need in a loop in the anonymous function you return based on those data.

1 Like
9

Thank you for taking the time to guide me through this!

To see if I got your point correctly, I’ll try to make some examples. Let’s say I have the function

function f(x)
    r = 0.0
    for (a, b) in c
        r += hard_computation1(a, b) * cos(hard_computation2(a, b) * x)
    end
    return r
end
f.(big_array)

The hard_computation could be anything time consuming.
Drawing from your suggestions, I would try something like:

function g()
   hc1 = [hard_computation1(a, b) for (a, b) in c]
   hc2 = [hard_computation2(a, b) for (a, b) in c]
   h = function (x)
       r = 0.0
       for i = 1:length(c)
           r += hc1[i] * cos(hc2[i] * x)
       end
       return r
   end
   return h
end
g_ = g()
g_.(big_array)

c must not change often if ever. If it did, I would pass c as an argument to g and create a new function g_ whenever I get a new c.

Would you say this would be the optimal (or at least a good :wink: ) way to do this kind of thing?

10

I think that solution is reasonable, but IMO it’s a bit convoluted (as is perhaps evident by the confusion in this topic). I think something like this would be easier to read and work with:

hard_computation1(a, b) = a + b
hard_computation2(a, b) = a - b

struct PrecalculatedHard
    hc1
    hc2
end

function PrecalculatedHard(c)
    hc1 = [hard_computation1(a, b) for (a, b) in c]
    hc2 = [hard_computation2(a, b) for (a, b) in c]
    PrecalculatedHard(hc1, hc2)
end

function compute(precalculated, x)
    r = 0.0
    for i = 1:length(precalculated.hc1)
        r += precalculated.hc1[i] * cos(precalculated.hc2[i] * x)
    end
    return r
end

precalc = PrecalculatedHard([(3,5), (7,10)])
compute(precalc, 3)
3 Likes
11

I guess you want to explicitly or parametrically type your fields in PrecalculatedHard though.

2 Likes
12

g(c)

Yes.

Anonymous function is always just an automatic way to create a struct that captures the necessary information. Performance wise, they should be the same when both done correctly. I would say once you realize this it’s not easy to get confused anymore so you can just pick whatever syntax you like.

2 Likes
13

Thanks, everybody, for the responses. I guess I have some idea now of how to tackle these kind of problems.

Which leads me, however, to a broader question. During the course of this thread I felt that my general ignorance made it quite difficult to understand what my problem was and to help me out. Could you give any suggestions on how I should go about this situation? For example, if you are now able to reverse engineer my original problem, how would you have phrased it with respect to the limited knowledge of my own problem?

And leading on from this… Coming from a physics rather than a CS background, I guess I still got okay at making programs what I want them to do. However, I have a feeling that my programs often are somewhat inelegant, overly complicated or convoluted. Additionally, I usually I use only a small subset of the programming language I use (Julia, Python, C++) – that’s enough to get me by but makes things rather basic. What would be good to get better at this? Code reviews? Reading through certain packages or base code? Is there a list of recommended practiced and coding ideas for Julia…?

2 Likes
14

I learn a lot from reading code in Base and other packages. If you make pull requests, you essentially get free code review :wink: Start small, with features you would need.

1 Like
15

That’s a great attitude! I think the most important thing to get your question across and get help is to include a minimal working example, including all dependencies so that others can just copy and paste your code into a clean Julia REPL and it should work. See my post above for an example.

Having others review and discuss your code (like in this topic) can be one of the more efficient ways to learn. This assumes however that you’re open for feedback. Many coders are NOT, and can’t stand having others “critique” their code or way of thinking.

Reading base code is awesome, but not always easy to understand. I would start with the manual if you haven’t already.

1 Like
16

To be fair, I have seen quite a few well-intentioned PRs abandonned after multiple rounds of nitpicking (“follow style convention X, even if the rest of the package doesn’t”).

17

The MWE is really quite important. Not only does it significantly ease the work for those who want to help you - - - creating it also helps you understand your own problem better, to the point where you either solve it yourself, or can pin down almost exactly what you don’t understand.

Creating MWEs is good training to become a better programmer, I would say.

3 Likes
18

Thanks for all the suggestions!

When starting this thread I thought a little bit about MWEs. I think when describing something that’s not working or showing buggy behavior it’s absolutely essential. I would love to think I would be better in creating those because it’s rather straight forward: just keep deleting unnecessary lines until the behavior doesn’t show anymore.

But with my slightly more abstract question I found it hard to actually make one. I can make a very simple example to showcase my question, but then the solution I’m looking for might not be the best solution for that particular MWE. As it was rightfully pointed out to me, for my code example the “correct” solution is

function gen_f(c)
   c2 = sum(hard_computation(a, b) for (a, b) in c)
   return x->cos(x) * c2
end

as it is way simpler and faster than anything I was trying to do. However, this won’t work with my actual problem. And unlike the my-stuff-won’t-work-MWEs I couldn’t come up with a way to check whether my MWE was actually any good, besides just posting it and looking for responses.
Maybe I got too entrenched with my own idea on how the solution ought to look like and lost sight of the underlying problem. Definitely an area where I can work on!

I thought the same but so far (I have been reading along for a couple months before I posted this topic) I have come across no requests for code reviews on this page so I am not even sure if that’s a thing here.
Especially since my code is usually just a tool to get a job done, I would hope that I am indeed open for feedback – but then again, I guess that’s what everybody says :smiley: The open and friendly community here definitely helps, though.

Looking into some base code I had pretty much this experience. Most of the stuff depends on some other stuff defined in other files and I didn’t really find a good starting point. I had a similar (albeit less so) experience looking through some of the packages suggested in the “High Quality Packages to learn from” thread. Unless I already know more or less what is going on (which I don’t), I find it pretty hard to grasp what the code is doing without spending ungodly amounts of time brooding over it. I suspect that is mostly due to a lack of practice (= I’ve never done that before).

I read some of the manual and learned quite a lot, but I found it was more of a description of many tools, rather than a explanation on how to use them or when doing so might be a good or a bad idea. Admittedly that isn’t really the function of a manual, still, it’s something that would be immensely helpful to me, I suspect.

19

For MWEs, I would definitely focus on the W more than the M, (even though the M comes before W, anyway). For me, as long as it’s not super long (If you use 5 external packages in 3000 lines of code I’m probably not going to read) it’s usually not a huge problem to post a longer version. Reducing long code is much easier than coming up with non-existing code.