[ANN] Dictionaries.jl - Improved productivity and performance of dictionaries in Julia

Package Announcements
,
21

I’ve tried to limit the package to stuff that I feel could conceivably put in base someday. I’m sure someone smarter than me will take some of it and make a far better version before that ever happens though.

1 Like
22

Yes I suppose we would statically encode the “insertability” of the indices at the type level, somehow. At the abstract level I’d rather use traits over abstract subtypes of AbstractIndices, though.

I thought you can just go ahead and define custom mutable and immutable indices types for now and then swap its internal based on Base.freeze / Base.melt , once #31630 is merged. (And also maybe add unsafe_freeze API to do “manual borrow checker” to avoid copying until #31630 arrives?) This, of course, relies on that the design of #31630 does not deviate much from the current version. But do you see pitfalls other than that?

Yeah, I’m already prototyping so much that I was afraid of getting too far ahead would start to make things over-ambitious, but yes we can do that. Ideally we’d overload functions that are defined prior to Dictionaries.jl though (really… we’ve needed to expand our traits for arrays and containers in general for a while now, so some shared package where we deal with these “container” ideas may be ideal).

2 Likes
23

Yeah, it would be nice to factor out mutability traits/implementations as a different package, so that Dictionaries.jl does not have to be responsible for all the advanced prototypings. IIUC DataFrames.jl also need something similar for names(df) and df[!, col]. But, previous discussion (and the resulting package ReadOnlyArrays.jl) did not explore mutability of indices/shape, IIRC. ArrayInterface.jl has similar traits (e.g., ismutable) but it focuses on the mutability of values and not indices. Also, it focuses on arrays and not containers in general. Anyway, I guess you are already aware of many of these. I’m just re-emphasising that there are immediate needs for this in Julia ecosystem.

Looking at DataFrames, it looks like it still is not using ReadOnlyArrays. I wonder if there is still a plan to use this. As it needs two kinds of immutabilities (immutable values and immutable indices/shape), it looks like the best place for this to happen.

1 Like