I was just playing around with the best way to compute the mean and standard deviation of a vector of values. Obviously, sum(x), std(x) is the cleanest but it isn’t the fastest, as I assume that approach needs to scan through the vector x twice. We can easily just scan through it once and then calc…

[image] xiaodai: It’s kinda sad that for Julia to hit top performance, we need to use “vectorized” versions of the code. This reminds me of Python/numpy and R. Those Vectorized_ packages are using an entirely different definition of “vectorized” than NumPy and R does. Both packages reference …

Thanks those are probably technical issues but from a users perspective I just care about the precision to an acceptable degree so they are non issues as far as I know. The vectorised point is also correct but I was thinking there might be a ‘mean_std’ or a moments function to compute these by wrap…

[image] xiaodai: Thanks those are probably technical issues but from a users perspective I just care about the precision to an acceptable degree so they are non issues as far as I know. Try your algorithm with x = fill(1/3, 10^5) … the variance should be zero, but your algorithm returns a var…

[image] stevengj: you are using the “naive algorithm” right. didn’t know this. i only did one numerical computing course in first year uni and i recall things like catastrophic cancellation etc etc but don’t recall this issue which i think must’ve been covered. thanks for pointing it out. Bu…

Your fastest version (ignoring numerical instability), i.e. explicit loop, is probably more readable to most programmers than the version using reduce. It’s highly subjective to say that code in a functional style is more “regular” Julia code.

[image] xiaodai: But say I implement the non-naive algorithm, the key points of the posts still stand, no? If your key point is that optimizing code is hard, and extracting maximum performance sometimes requires expertise and lower-level tricks, then I agree. But functional-style programming…

I don’t really understand what your question has to do with functional programming? Functional programming and functional datastructures are the discipline of programming in a world of non-overwriting memory. This is an extremely good fit for certain settings, like e.g. flash memory / SSD (which a…

[image] foobar_lv2: I don’t really understand what your question has to do with functional programming? I think @xiaodai just means programming with higher-order functions like reduce and similar patterns (which are closely associated with functional programming even if they are not precisely…

The pattern for fast numerical and scientific code (large simulations, etc.) is to allocate big fixed-sized arrays to hold everything and then mutate their contents. The functional programming idea is the opposite: pass around immutable data structures. This makes it easier to avoid mistakes when wr…

Can you elaborate on this. I mean how to do large simulations in julia without allocating big fixed-sized arrays?

Functional programming is not capable of achieving absolute top performance

General Usage Performance

Topic		Replies	Views
Compilers, purity, and variance computation Numerics statistics , float , compiler	7	532	September 12, 2024
Reducing allocations to optimize (to match Java speed) Performance	22	1484	December 10, 2018
Optimizing Calculation in Julia compared to C (New to Julia) Performance	25	2719	January 1, 2020
Improving performance elegantly - type stability General Usage type-stability	20	1546	January 13, 2019
In as few lines as possible describe why you love julia New to Julia	113	13170	May 21, 2021

Functional programming is not capable of achieving absolute top performance

Related topics