Sorry @Raf, I can’t seem to find the documentation for this simple question:
Is there a way for Flatten.flatten to return a Vector instead of a Tuple of the parameters?
Right now I simply do:
[flatten(mystruct)...]
Thanks!
2 Likes
I usually use reduce(vcat,A) but never timed it. Let’s do it!
julia> using Base.Iterators
julia> A = map(i->rand(10),1:1000);
julia> @btime reduce(vcat,A);
10.179 μs (2 allocations: 78.20 KiB)
julia> @btime [flatten(A)...];
639.579 μs (30014 allocations: 1019.94 KiB)
3 Likes
That’s cool, but I’m trying to flatten a struct, not a vector of vectors. Actually, a vector of structs…
Flatten isn’t for vectors - its all compile time and we don’t know vector length at compile time.
It flattens arbitrary nested structs and tuples where the fields are known at compile time.
Edit: you are actually timing Base.Iterators.flatten, not Flatten.jl. You would usually find that flattening structs at runtime is best measured in ns. It might even have a similar runtime fro a problem as large as the one you time above if it was using tuples, but the compile time would be ridiculous.
There used to be an argument or flattening to Vector but as I added all the arbitrary type flattening it got confusing having so many types in the method call.
So now you just manually do [flatten(x, Real)...]. That’s all the internal method did anyway and its actually less characters than it was before 
I should probably update the docs a little.
Edit: you might also want to convert the tuple to SVector where performance matters, as it wont allocate.
2 Likes
Thanks!
In my application I want to flatten a known number of structs to an iterable container. Each struct has 4 fields (UInt8), and there are 4 structs, which would result in a tuple with 16 (UInt8) elements. So an SVector would make sense. So something like this (on my phone now, so will check my self later)?
x = [MyStruct() for _ in 1:4]
flatten(x..., SVector{UInt8, 16})
(back on computer) Apparently not… I’m all of a sudden unsure on how to do this with Flatten… I could forgo my custom type and just use a static vector:
using StaticArrays
A = SVector{4, UInt8}
x = (A(rand(UInt8, 4)) for _ in 1:4)
reduce(vcat, x)
How exactly, and would it be any better, do I flatten x with Flatten:
using Flatten
struct B
x1::UInt8
x2::UInt8
x3::UInt8
x4::UInt8
end
x = (B(rand(UInt8, 4)...) for _ in 1:4)
You could look at https://github.com/pao/StrPack.jl. Not sure how well it works since it seems to not have been extensively maintained.
Cant you just do SVector(flatten(x, Real)) ? (or whatever type you want besides Real)
The type in the flatten signature is the types that will be extracted from the struct! So flatten(x, Int) will get all the Int and SVector(flatten(x, Int)) will put all the Int into an SVector.
Flatten args are like: flatten(obj, extracttype::Type, ignoretype::Type), returning a tuple where extract is the type/union to take and ignore is the type/union to ignore. It’s totally arbitrary what actually gets flattened now, but it always produces a tuple. Then you can do whatever you want with the tuple.
Flatten.jl does a lot more than StrPack and is maintained. It’s just very abstract and kinda weird.
1 Like
Sorry, it was unclear to me that Flatten here referred to a package.
No, because x in this example is a vector of structs. But I might be missing something. In this MWE, that won’t work:
using Flatten, StaticArrays
struct B
x1::UInt8
x2::UInt8
x3::UInt8
x4::UInt8
end
x = [B(rand(UInt8, 4)...) for _ in 1:4]
You can’t flatten vectors as the length is unknown to the compiler. Try with a tuple or SVector:
using Flatten
struct B
x1::UInt8
x2::UInt8
x3::UInt8
x4::UInt8
end
julia> x = Tuple(B(rand(UInt8, 4)...) for _ in 1:4)
julia> using StaticArrays
julia> SVector(flatten(x))
16-element SArray{Tuple{16},UInt8,1,16} with indices SOneTo(16):
0x81
0xa4
0x41
0xef
0xf6
0xca
0x9b
0x7d
0xb6
0xe7
0xb4
0xd7
0x69
0x14
0xf7
0x97
1 Like
Also of interest:
julia> @btime SVector(flatten($x))
0.020 ns (0 allocations: 0 bytes)
16-element SArray{Tuple{16},UInt8,1,16} with indices SOneTo(16):
0x81
0xa4
0x41
0xef
0xf6
0xca
0x9b
0x7d
0xb6
0xe7
0xb4
0xd7
0x69
0x14
0xf7
0x97
Thank you so much @Raf, this is great. I was trying to (B(...) for _ in 1:4) before instead of the explicit Tuple(...) version and got stuck on that.
1 Like
It good to know what is confusing about the package, I’ll make it clearer in the docs that length of objects has to be known at compile time, and we are specifically flattening fields, which Array doesn’t have.
2 Likes
I have been using vcat(a, b..., c, d) etc for this (in the vector case). Would have liked to see a proper flatten though, as well as flat_map.
I think it could be done for a struct by using the dynamic lookup functions.
flatten_struct = x -> [getfield(x, n) for n = fieldnames(typeof(x))]
julia> using FlexiMaps
julia> flatten([1:2, 5:10, 1:0])
8-element Vector{Int64}:
<...>
julia> flatmap(i -> 1:i, [1, 3, 2])
6-element Vector{Int64}:
1
1
2
3
1
2
This does a single level of struct flattening. It can be what one needs, but Flatten.jl main usecase is recursive flattening AFAIK.
in this example.
How does flatten() stop unnesting at the rational number level and not unpack the numerators and denominators as well?
I’d be interested in understanding it in principle as well, without the implementation details that I probably wouldn’t be able to understand.
using Flatten
struct NR
q1::Rational{Int64}
q2::Rational{Int64}
end
NSR = Tuple(NR(nn...) for _ in 1:4)
nn=Tuple(NR((Rational(rand(1:10,2)...) for _ in 1:2)...) for _ in 1:4)
julia> flatten(nn)
(5//2, 2//1, 2//1, 7//9, 2//3, 3//5, 9//5, 3//5)
This is just because you are not passing any arguments to flatten, and the default behavior is to flatten Real. Maybe that’s not well enough documented.
If you want the inner values, use flatten(nn, Int64) and it will look inside the Rational numbers for Int64.
1 Like
Okay. Thank you.
I don’t need to unpack the rationali.
I was wondering how the function could be so “smart” that it stops at the right point.
So, conceptually, the recursive function (with default parameters) unpacks everything that isn’t Real?