Any Julia library for managing data series with automatic transformation

Specific Domains Data
, ,
2

There’s https://github.com/JuliaArrays/MappedArrays.jl. I am not a DataFrames expert, but the following works to take the modulo 2π of any value before storing it:

julia> using MappedArrays, DataFrames

julia> A = mappedarray(identity, x->mod(x, 2π), rand(10))
10-element mappedarray(identity, getfield(Main, Symbol("##13#14"))(), ::Array{Float64,1}) with eltype Float64:
 0.6462246044535898 
 0.3893724745260221 
 0.10819312044797025
 0.7456662437717823 
 0.2259602265381362 
 0.03472737190390074
 0.4670647981623812 
 0.924346515776455  
 0.5413998577384473 
 0.5462417188978359 

julia> df = DataFrame!(Any[A], [:A])
10×1 DataFrame
│ Row │ A         │
│     │ Float64   │
├─────┼───────────┤
│ 1   │ 0.646225  │
│ 2   │ 0.389372  │
│ 3   │ 0.108193  │
│ 4   │ 0.745666  │
│ 5   │ 0.22596   │
│ 6   │ 0.0347274 │
│ 7   │ 0.467065  │
│ 8   │ 0.924347  │
│ 9   │ 0.5414    │
│ 10  │ 0.546242  │

julia> df[5,1] = 10
10

julia> df
10×1 DataFrame
│ Row │ A         │
│     │ Float64   │
├─────┼───────────┤
│ 1   │ 0.646225  │
│ 2   │ 0.389372  │
│ 3   │ 0.108193  │
│ 4   │ 0.745666  │
│ 5   │ 3.71681   │
│ 6   │ 0.0347274 │
│ 7   │ 0.467065  │
│ 8   │ 0.924347  │
│ 9   │ 0.5414    │
│ 10  │ 0.546242  │

julia> mod(10, 2π)
3.7168146928204138

If you’ve been doing this in Matlab, I think you’ll be pleasantly surprised by the performance of MappedArrays:

julia> foo(A) = @inbounds A[2]
foo (generic function with 1 method)

julia> @code_native foo(A)
	.text
; ┌ @ REPL[29]:1 within `foo'
; │┌ @ MappedArrays.jl:161 within `getindex'
; ││┌ @ REPL[29]:1 within `getproperty'
	movq	(%rdi), %rax
; │└└
; │┌ @ array.jl:729 within `getindex'
	movq	(%rax), %rax
	vmovsd	8(%rax), %xmm0          # xmm0 = mem[0],zero
; │└
	retq
	nopl	(%rax)
; └

:smile:

7 Likes