Release announcements for DataFrames.jl

We have already discussed this. The issue is not that simple. In general there are many standard structures and function in Julia Base that potentially can be multi-threaded but are currently single threaded.

Regarding Dict. The issue is not that we would want just ThreadSafeDict as such dictionary would have to use locking for making sure parallel reading and writing is safe. What we need is lock-free parallel writing to a dictionary data structure (i.e. for cases where writing happens in parallel and reading happens in parallel, but we know that writing and reading cannot happen at the same time). Most likely Julia Base devs consider such use case as too specific to include in Julia Base. We will see.

In general we are going to focus on performance of DataFrames.jl after 1.0 so in the worst case we will have our own implementation if it is needed. It will just take time (note how long it took to stabilize DataFrames.jl API enough to warrant its 1.0 release).

10 Likes