Looking for ways to generalize using IndexedTables to Plot / Graph N-Dimensional data

[Marc.Cox]

Hi All,

I’m looking for ways to generalize using IndexedTables to graph N-Dimensional data,
but I’m stuck on how to collect the “for iter in eachindex(keys(tab_t1.index.columns))”.
In that regard, would you please see MWE pseudocode snippet below and
let me know any suggestions you might have ?

using IndexedTables
using Plots

# 2D = 1Dy Dependent var equation of 1Dx INDependent var
tab_t1 = table( (x=1:5, y=randn(5)); pkey = [:x] )
str_current_equation_approximated = "(x=1:5, y=randn(5))"
println("##500-1 tab_t1.pkey():=",tab_t1.pkey[1])
###500 tab_t1.pkey():=1
# for (k,v) in zip(keys(t.index.columns), t.index.columns)
#             push!(colNames, k)
#         end
println("##500-1 tab_t1.index.columns:=",tab_t1.index.columns)
for iter in eachindex(keys(tab_t1.index.columns)) # COD eachindex(tab_t1.pkey())
   # println(tab_t1[iter])
   global plot1 = Plots.plot(iter, select(tab_t1.y,iter), title = str_current_equation_approximated)	# Mod Template
   global plot2 = Plots.plot(iter, select(tab_t1.y,iter), title = str_current_equation_approximated)	# Mod Template
end
println("##500-1 !! END WORKS !! using IndexedTables - for iter in eachindex(keys(t1.columns)) - println(t1[iter]) !!")
##
##

# 3D = 1Dy Dependent var equation of 2Dxz INDependent vars
tab_t1 = table( (x=1:5, y=randn(5), z=randn(5)); pkey = [:x,:z] )
str_current_equation_approximated = "(x=1:5, y=randn(5), z=randn(5)"
println("##600-1 tab_t1.pkey():=",tab_t1.pkey[1])
###500 tab_t1.pkey():=1
# for (k,v) in zip(keys(t.index.columns), t.index.columns)
#             push!(colNames, k)
#         end
println("##600-1 tab_t1.index.columns:=",tab_t1.index.columns)
for iter in eachindex(keys(tab_t1.index.columns)) # COD eachindex(tab_t1.pkey())
   # println(tab_t1[iter])
   global plot1 = Plots.plot(iter, select(tab_t1.y,iter), title = str_current_equation_approximated)	# Mod Template
   global plot2 = Plots.plot(iter, select(tab_t1.y,iter), title = str_current_equation_approximated)	# Mod Template
end
println("##600-1 !! END WORKS !! using IndexedTables - for iter in eachindex(keys(t1.columns)) - println(t1[iter]) !!")
##
##

Thank You,
Marc Cox

Ps> If you’re interested I’m also happy to share my complete Plotting template
when I’ve completed it - just didn’t want to muddy up the MWE issue right now.

[MA_Laforge]

Apologies if I am not understanding your question correctly. When I read your post, it initially sounded like you have a good solution for manipulating N-Dimensional data using IndextedTables, but only need to figure out how to plot. But the more I read, the more it kind of sounds like you are trying to find a practical way to store/manipulate N-dimensional data from simulation or experiments, that could easily be plotted as well.

MDDatasets.jl (Multi-Dimensional Datasets)

If I am correct, you might want to check out MDDatasets.jl.

[ANN] MDDatasets.jl: Multi-dimensional datasets & continuous x-variable interpolation

The MDDatasets.jl module provides tools to simplify manipulation of multi-dimensional datasets. → GitHub - ma-laforge/MDDatasets.jl: Multi-Dimensional Datasets for Parametric Analysis Insallation: ] add MDDatasets Grouping (x,y) vectors MDDatasets also provides a means to group (x,y) data in a single data structure to simplify calculations (DataF1 structure). For example, we can take the derivative of a dataset (wrt x) very simply using: (Using MDDatasets.jl:) #Assuming f1 has time-doma…

More than plotting

The structures used to store data in MDDatasets basically store the parameters associated with your data. In other words, it stores whatever quantities you wish to vary (in a “hidden” data structure), along with the results of your experiment.

Again, I apologize if I am not quite grasping what you are trying to do. From my understanding, you have to wrap quite a bit of functionality around IndexedTables
in order to make it practical/easy to manipulate N-Dimensional data - which is probably why plotting can seem like a bit of a pain.

But plotting as well

There are a few packages in MDDatasets’s ecosystem that enable plotting of these n-dimensional datasets (I can later elaborate if you wish).

At the moment, these plotting routines simply/automatically overlay 2D slices of the entire dataset - no matter the dimensionality. Note that it would be relatively trivial to generate a “surface” plot of a 3D dataset as well (though overlaying such plots might not be practical for dimensions larger than 3D).

I noticed you mentioned “scatter plots” in “Fast iteration over rows of a DataFrame”. That’s good, because that’s the kind of plotting that is currently supported.

However, you also talk about “graphs” in this thread in which I am currently replying. In this case, it sounds like you wish to also graphs of nodes and edges for n-dimensional data. Sadly, I’m not too sure how this is typically done. I don’t usually deal with these types of graphs, so that might be where I’m getting lost.

[Marc.Cox]

Sorry for my delayed response - **Thank you Very Much ! ** This looks great here >> https://github.com/ma-laforge/MDDatasets.jl

I hope it will work with Base Julia v1.04 or what version would you suggest ?

[MA_Laforge]

Oh good, I wasn’t sure I got what you were trying to do. And yes, at the moment, it should work on all version Julia 1.*. If it doesn’t, let me know.

CData ecosystem

Note that MDDatasets is part of a larger ecosystem which is under the “CData” umbrella (though CData is NOT required):
→ https://github.com/ma-laforge/CData.jl

You can install CData and all its “dependencies”, by adding the library manually, and running the install routine:

] add https://github.com/ma-laforge/CData.jl.git
using CData
CData.install()

Basically, it just installs a bunch of un-registered modules located in my git repo. I would have registered them by now, but I haven’t figured out an effective way to do so.

I really wanted to bundle them all in one or two git repos so that all versions could be more easily kept in sync - and also to avoid having excessive “repo clutter”. There is quite a few of them, even in the early stages:

• Core modules: FileIO2, MDDatasets, NumericIO
• Plotting tools: EasyPlot, EasyData, EasyPlotInspect, EasyPlotGrace, EasyPlotMPL
• Circuit/electrical tools: CircuitAnalysis, SignalProcessing, NetwAnalysis,
  SpiceData, LibPSF, PSFWrite, EDAData

Why not just bundle everything under a single git repo/package?

I would really like to have all the EasyPlots* modules installed as a single monolithic “package”, but the way Julia packages are designed, all dependent modules (InspectDR, HDF5, GracePlot, Plots.jl, PyPlot, …) would automatically get installed.

I don’t like this solution. Most people only intend on using a single plotting package. I am trying hard to find a way to decouple the action of [adding a module and its dependencies to a project] from the action of [adding the .git repo/package housing these modules].

MDDatasets: Examples of what is possible

You might want to take a look at the SignalProcessing.jl module if you really want to see what MDDatasets.jl can do. In particular, you might want to look at the sample plots it can generate:

→ https://github.com/ma-laforge/SignalProcessing.jl
→ https://github.com/ma-laforge/FileRepo/blob/master/SignalProcessing/sampleplots/README.md

Plotting/Saving Multi-dimensional datasets

• EasyData.jl: Save multi-dimensional datasets and entire plots in a single HDF5 file!!!
• EasyPlot.jl: Core library to plot multi-dimensional datasets.
• EasyPlotInspect.jl: Links EasyPlot to the InspectDR.jl backend. I highly suggest using this backend as it receives the most attention.
• EasyPlotPlots.jl: BROKEN!: Links EasyPlot to the Plots.jl backend. If you wish to use Plots.jl as your plotting backend, please let me know. Last I checked, EasyPlotPlots.jl was broken. But if someone wants to use it, I will try to get it up and running again.

[MA_Laforge]

Update: CMDimData.jl

OK, so I decided to re-package all those repositories mentioned above into a single one:
→ GitHub - ma-laforge/CMDimData.jl: Parametric analysis/visualization +continuous-f(x) interpolation
(I’ll make a formal announcement soon)

This one is registered with Julia’s registry, and works better than the CData.jl solution.

It’s basically just a wrapper around MDDatasets.jl that adds plotting and saving functionality for multi-dimensional datasets.

You can check out the sample directory if you wish to see the new plotting API:
→ https://github.com/ma-laforge/CMDimData.jl/tree/master/sample

I suggest looking at:

• demo1_EasyPlotInspect.jl, or
• parametric_sin.jl

There is also some examples of plot construction under plots/.

But if you want more complex examples, you can check out those found in my circuit-related module that builds on CMDimData.jl:
→ GitHub - ma-laforge/CMDimCircuits.jl: Parametric analysis/visualization of model/measurement/simulation results