I want to plot the raw signal data from audio data that is sampled at the standard 44Kbps but there is a challenge if the signal is taken for a very long time; eg. 3 hrs of signal. There is then considerable latency in the presentation of the finalized plot and the time taken to save the figure to disk.
What can be done to reduce the time taken to visualize and save the figure to disk? Are there any ‘tricks’ in GR or Pyplot that can utilize some preset functionality? I was thinking that subsampling the signal would be the only route and visualizing that subsampled signal instead. Are there any recommended avenues for that and for the plotting packages a recommended max number of points for the latency to not grow too large
I have some code I use for plotting audio files. It divides the audio into chunks, computes the minimum and maximum of each chunk, and then plots a filled region between the min and max for each chunk:
using Plots
using DSP
using Statistics
"""
chunkminmax(data, nchunks)
Break `data` into `nchunks` chunks, and return a 3-element `Tuple` where the 1st
item is a `Range` with the starting indices of each chunk, and the second and
third are the lower and upper bounds of each chunk, respectively.
## Example
```julia
sig = randn(1_000_000) .* cos.(linspace(0,2π, 1_000_000))
i, l, u = chunkminmax(sig, 1000)
plot(i, l, fillrange=u)
```
"""
function chunkminmax(data, nchunks)
if length(size(data)) != 1
throw(ArgumentError("`data` must be one-dimensional"))
end
L = length(data)
nchunks < L || return 1:L, data, data
N = cld(L, nchunks)
S = 1:N:L
E = min.(S .+ (N-1), L)
mins = similar(data, length(S))
maxs = similar(data, length(S))
foreach(enumerate(S)) do (i, s)
mins[i], maxs[i] = extrema(data[s:min(s+N-1, L)])
end
S, mins, maxs
end
function sigplot(sig; fs=1, kwargs...)
i,l,u = chunkminmax(sig, 1000)
plot(i/fs, l; fillrange=u, alpha=0.8, legend=false, kwargs...)
end
I find that the inspectdr backend is better suited for this kind of very large signal data, it also has the possibility to zoom and pan along the x axis with good performance, which is often what you want for visualizing signals.
Yes, as @baggepinnen mentionned, InspectDR was basically designed for these kinds of signals.
…But the algorithm that significantly accelerates plotting requires you to plot the audio signal using “lines”. The faster algorithm is automatically disabled when you plot only the symbols because the plot becomes a bit misleading.
As for data:
I am not that familiar with the Kbps notation, but if I am not mistaken, a standard 16-bit 44.1kHz (mono channel) audio signal would take:
So a 3hour long audio signal would take about 1GB. This is well in the range of what InspectDR can handle. If I remember correctly, I would expect plot times for this signal to be about 20s on a typical desktop PC.
Also note that the algorithm downsamples the data to ~1000 points, but this setting can be overwritten with the Plot2D.xres argument. The advantage here is that the plot file will not be excessively large. Please keep the following in mind, though: when the downsampling algorithm is activated, you basically get a plot “image” of the data - so the exported image does not contain the entire dataset.
The good news is that as you zoom in interactively, InspectDR does display the details for that zoom level.
@MA_LaforgeInspectDR is another backend that can be used as pyplot and GR can be used?
I saw on https://libraries.io/github/ma-laforge/InspectDR.jl that Fast, interactive Julia/GTK+ plots (+Smith charts +Gtk widget +Cairo-only images)…? Can I integrate the plots produced with InspectDR in the way that Gtk.jl can incorporate Winston plots with minimal effort? The use of GR produced plots is more of a workaround than expected.
Has it been used for producing spectrograms (heatmaps)?
InspectDR is another backend that can be used as pyplot and GR can be used?
I would guess here that you are asking if InspectDR is a backend to Plots.jl: a package providing a high-level plotting command interface, but leaves the actual rendering of plots to one of many specific “backend” plotting modules.
If that was your question, then YES: InspectDR is a backend to Plots.jl. You just have to select:
using Plots
inspectdr()
(but you must first add the InspectDR package)
NOTE
Please be aware, however, that InspectDR runs significantly faster when used directly (without going through the Plots.jl interface). This will probably be more important to someone plotting large datasets.
Can I integrate the plots produced with InspectDR in the way that Gtk.jl can incorporate Winston plots with minimal effort?
Again, I am not certain if I understand correctly, but InspectDR has a nice GUI that is also built on Gtk.jl. InspectDR.jl is designed in layers so that you could add an InspectDR “widget” in your own Gtk application. The problem is that this feature is not that well documented, so you would probably need my assistance.
Question 3
Has it been used for producing spectrograms (heatmaps)?
No. InspectDR was not designed do produce spectrograms. Other plotting tools did a sufficient job of plotting spectograms for my particular needs - so I did not see a reason to support this feature.
InspectDR was designed as a fast, interactive tool for generating 2D (x vs y) plots. None of the other tools at the time would perform adequately for my requirements.
Also: I do not believe any other Julia plotting backend currently compare to InspectDR in this particular aspect of speed and basic pan/zoom/“add marker” interactivity (though some other backends now have simpler ways to incorporate other types of programmer-provided interactivity).
I have a larger GTK.jl UI, and it has spectrograms etc, can I integrate InspectDR.jl easily into the GTK.jl interface? The examples for the UI I saw, are great, but they do not appear to be as broad as what is possible with GTK.
Function PlotWidget(plot::Plot) creates a new Gtk plot widget from a Plot2D <: Plot object.
I would look at function GtkPlot(mp::Multiplot) to see how one can add a plot widget to a Gtk window.
Note that there is no implementation of a Multiplot widget. There is only a widget for a single plot at a time (PlotWidget). The Multiplot object is always “GUI-fied” into a Gtk.Window object for the time being.
Also note:
base Plot2D <: Plot objects are stored in the Multiplot.subplots list.
PlotWidget objects are synchronized (and constructed) from the sync_subplots(gplot::GtkPlot) function.
Can something like also be applied to spectrograms? Because for a particular chunk of time, you would have to have multiple frequencies shown for a chunk that are not actually present for that chunk’s entirity. Eg for a chunk whose frequencies change from low to high, this interpolation envelope like method would plot high for both frequency bins
import InspectDR
p = InspectDR.Plot2D()
InspectDR.add(p, collect(1:100_000_000), randn(100_000_000))
display(InspectDR.GtkDisplay(), p)
and found that the performance when panning/zooming was very similar between InspectDR standalone and as backend to Plots.
1e6 points is fine, 1e7 points is sluggish, 1e8 points displays but is pretty much not interactive.
@baggepinnen: Yes, sorry. I suppose I should clarify.
What’s mostly faster is time to first plot. This would normally be a problem with small datasets, but when you plot datasets in th GB range, the extra 20sec is not as noticeable.
Then, you get also extra delay for each subsequent plot because there is still overhead in using Plots.jl. I admit, this overhead does not look too bad. It does appear like significantly more memory is used when Plots.jl is active, but that might be due to my poor implementation of the inspectdr() backend wrapper.
But you are absolutely correct: there should be no difference in GUI response times between InspectDR standalone and as a backend to Plots.jl (other than an insignificant difference due to extra memory usage, or something similar).
So I have a small correction:
For large datasets, there should theoretically be no significant difference in using InspectDR in standalone mode, or with Plots.jl.
However, I have seen more crashes due to memory issues when using Plots.jl. Maybe I should try upgrading to the new plots and to Julia 1.2 for further tests when I get a chance.
(I am running on a VM with only 1GB of ram - but it seems like memory issues are happening faster in the newer Julia version than what I remember).
@mantzaris: Can something like also be applied to spectrograms ? […]
Since no one has answered yet:
If I understand correctly, when you talk about spectograms, you mean plots of the frequency spectrum (somewhat like what an FFT might do):
(Just grabbed an image from the web here).
I am a bit confused because initially, you seemed to be saying that your “spectrograms” were heatmaps (hetatmap example below):
Above picture is what Plots.jl calls a heatmap.
But if your concern is having an algorithm that shows both the minimum and maximum peaks of a plot, that IS what InspectDR is doing (as long as you use “F1” acceleration, as it is called). It’s just that the plot values are not binned the same way you would expect if you were writing an algorithm specialized for frequency spectrums.
Alternatives
Note that once came across documentation by one of the spectrum analyzer vendors (I think either Agilent (now Keysight) or Tektronix). If my memory serves me well: this “app note” showed different algorithms that could be used to display spectrums differently (including the trivial algorithm that displayed both the min & max peaks in a bin).
Some of the algorithms described were much better than the trivial one that displays min & max peaks. I really wish I could refer you to this document, but sadly, I just cannot seem to find it anymore. Maybe you will be able to find it if you do a search. I am really bad at choosing the right keywords for Google searches.
Sorry. I just noticed I did not answer this particular issue:
Yes. Despite the fact that InspectDR tends to be fast and “interactive” compared to other backends, eventually, the data to be processed/drawn becomes too large for true interactivity.
In this case, I highly suggest not using the shift+click+mousemove operation to pan the data portion of a plot. It causes too many refreshes for the time required to plot data.
I also don’t suggest using the mouse wheel operation to pan and zoom for similar reasons.
(Maybe I should add a data size threshold to inhibit these operations?!?)
Instead, I suggest using the right mouse button (RMB) box-zoom (+ similar operations), and patiently waiting the few seconds it takes to redraw your plot.
So, eventually, you do lose “true” interactivity, but I still did my best to keep the rendering time as short as I could.
Sorry. I also just realized I never answered your last spectogram question:
Unfortunately, InspectDR does not yet support the heatmaps necessary for this operation. But I have started thinking about it, and it might get added if I have sufficient time.
