Is there a generic API (ie a function) for requesting a deferred computation of some value?
Eg if x is a Base.Generator, I want the array, etc.
collect does too much, eg collect(SVector(1, 2, 3)) gives a Vector.
Packages with lazy computation define their own verb, eg LazyArrays.materialize. But there is no generic API I am aware of.
There is no such thing as ādeferred computationā in julia, at the language level.
As such, all possible constructs for deferred computation must use manually boxed objects (e.g. āappliedā from LazyArrays, which returns a box with the function to be applied and the arguments), and must manually unbox them.
Furthermore, the boxed objects are not āspecialā in the language in any way - optimizations aside, you can think of āappliedā as a regular function returning a tuple of itās arguments, except tagged with a special type that, when āmaterializeād just applies the first argument to the rest.
It isnāt really clear what you expected to happen in the collect example, but the most straightforward lazy operations that donāt depend on packages would be using the generator monad,
u=eachline(fn)
v=Base.Generator(u) do line
eachmatch(rā.BOMB.ā,line)
end |> Iterators.flatten
w=(print(m) for m in v)
take(w,3) # materialized up to 3 els.
Note that we didnt specify anywhere number of lines to scan, only how many results we want - ie this is really āon demandā.
Please reread the question because you misunderstood it. I am asking whether there is a package that defines a generic API for this feature (which packages defining lazy computations could define methods for).
I am suspecting the answer is ānoā, but wanted to check before making PRs.
i have read it several times before answering, and once more just now. I still donāt understand what youre looking for in terms of API.
This in particular is unclear. There is a version of collect that takes a type argument as well.
Are you looking for something that would know that your value is already materialized vs held in a package agnostic manner? like resolving promises?
I think I understand somewhat where you are coming from but to me your question seems not well-posed. What is it you want to achieve, why do you want to āmaterializeā lazy objects? I think what is meant by that is quite context-dependent and thus there will not be a general function doing exactly what you want.
To me it seems that might be mainly interested in lazy ālist-likeā objects? Like Vector, Generator,ā¦ Yet your question is about general deferred computation. So if you care about general objects, maybe you can give a couple of examples what you want to happen with e.g.
LinearAlgebra.I or stuff from FillArrays.jlWhat properties do you expect of the output? Maybe fast indexing at arbitrary locations?
There isnāt one generic API for deferring computation either, e.g. generator expressions and LazyArrays donāt work the same way, and neither does the internal Broadcasted. Picking some lazy evaluation types to absorb into a generic materialize API that forwards to their respective materialization implementations could be justifiable, but subjective. For example, are we including view, Iterators.cycle, or other wrappers where all the elementsās values were already evaluated somewhere because we need extra work on indexing?
Sorry, I was a bit unspecific. The indended use is for containers of any kind, which includes anything that supports eg getindex and getproperty.
For example, suppose a calculation takes an array and then uses elements multiple times. Computing them on demand more than once would be wasteful, but collecting just for the sake of fixing values could lead to unnecessary allocation (and type changes) in case the result is already calculated.
Yes, this is a valid point, which is why I wanted to start a discussion. Maybe the use case is so niche that this does not warrant a MaterializeBase.jl package yet 
I ran into the question while refactoring some code that takes an array-like structure and accesses elements in an irregular order and multiple times. After benchmarking some alternatives it appears that even collecting a simple PermutedDimsArray gives me a significant improvement, even with the extra allocation.
I recognize that these things involve a lot of subtle trade-offs (should materialize resolve computations only, or indexing, or leave indexing alone, etc), but I feel that even a very crude API would be a good starting point.
Perhaps you need memoization, which is not the same thing as laziness. There is a package for that if I remember correctly.
EDIT:
Memoization is an alternative, but if you know you will be traversing all indices in the domain it may not be the optimal approach.
FixedSizeArrays.jl has a collect_as function:
So a function taking two arguments, where the first argument is the return type, while the second argument is an arbitrary iterator. The FixedSizeArrays implementation, linked above, only implements collect_as for FixedSizeArrays, however my intention (with @Oscar_Smith giving it a thumbs up) is to make it an interface package:
What do you think of the function interface, the two arguments?
Hereās how collect_as might be used to convert an iterator to Vector{Bool}: collect_as(Vector{Bool}, some_iterator)::Vector{Bool}. This is not implemented, to be clear, although it could forward to collect(Bool, some_iterator) in this specific case.
Thatās a similar idea but I think it is a complement to my use case. I want the calculations and reindexing performed and donāt care about the result type as long as
a[i...] == materialize(a)[i...]
\forall i indices in the domain.
So, you ask a question, i provided an answer which you immediately discarded and ignored.
You specified a bit more what would you want. Now I suggest to go back and reread my answer.
Good luck. I wish i was wrong here.