Question on lazy map function

New to Julia
,
1

Can anyone please point me to overview of using (maybe different kinds) of map/reduce/filter functions in Julia (and it’s standard packages)?

Here is is what I’m missing now: I want any(map(x -> x > 0, big_rand_array)) to lazy evaluate without computing on all elements, yet it appears to allocates big resulting array of map and only then passes it to any.

(this is “first steps” question)
Thank you very much,

1 Like
2

Hi there,

you may be interested in this thread Best Data Manipulation packages 2020-09 [video]

Lazy.jl can give you lazy evaluation: https://github.com/MikeInnes/Lazy.jl

And then there is Transducers.jl and the things that use it, not lazy AFAIK but pretty cool: GitHub - JuliaFolds/Transducers.jl: Efficient transducers for Julia

1 Like
3

One option is to use Iterators (Iteration utilities · The Julia Language)

a = Iterators.filter(x -> x > 0, big_rand_array)
b = Iterators.take(a, 1)
c = length(collect(b)) > 0

Will lazily iterate over your list and return true if any matches are found.

3 Likes
4

You can use a generator, like

any((x > 0 for x in -5:5)) # true
3 Likes
5

Many of these reduction functions can take a function as the first argument: any(>(0), randn(2)) ought to be efficient. There is a lazy map, but here you may as well combine them: any(f, Iterators.map(g, xs)) == any(f∘g, xs).

5 Likes
6

@ericphanson, why not single parentheses?

7

In Julia v1.5, Iterators.map does not appear to be lazy:

julia> Iterators.map(x -> 2x, Iterators.repeated(1, 3))
3-element Array{Int64,1}:
 2
 2
 2
8

Oh I forgot which version, again, sorry, it’s in 1.6 then. It is however identical to writing the generator:

julia> Iterators.map(sqrt, 1:10)
Base.Generator{UnitRange{Int64}, typeof(sqrt)}(sqrt, 1:10)

julia> (sqrt(x) for x in  1:10)
Base.Generator{UnitRange{Int64}, typeof(sqrt)}(sqrt, 1:10)
3 Likes
9

True, forgot you don’t need two sets there

2 Likes
10

Transducers are lazy.

julia> using Transducers

julia> Iterators.repeated(1, 3) |> Map(x -> 2x)
Base.Iterators.Take{Base.Iterators.Repeated{Int64}} |>
    Map(Main.λ❓)

julia> Iterators.repeated(1, 3) |> Map(x -> 2x) |> collect
3-element Vector{Int64}:
 2
 2
 2
3 Likes
11

FYI, there’s Compat.Iterators.map for older Julia.

Somewhat unfortunately, as of lowering: remove incorrect comprehension eta reduction by vtjnash · Pull Request #39139 · JuliaLang/julia · GitHub it’s not the case:

julia> (sqrt(x) for x in 1:10)
Base.Generator{UnitRange{Int64}, var"#1#2"}(var"#1#2"(), 1:10)

(but it’s required for correctness)

1 Like