Support for MDF format

General Usage
1

It would be great if the MDF format was supported like HDF5 is. Some examples are here for matlab and here (and here) for python. It’s also now supported natively by matlab.

Just to be clear, I’m not referring to this mdf file related to databases.

thx for your time!

1 Like
2

In case you don’t know this, generally speaking your options are (1) to wait (weeks? years?) and hope an energetic volunteer adds this, (2) become that volunteer yourself, or (3) pay someone to do it. For the benefit of whoever decides to create an MDF.jl package, note that FileIO is designed to handle magic byte and extension detection and dispatch.

5 Likes
3

There is a library (https://vector.com/vi_mdf4lib_en.html) that provides a C++ interface.
That might be the easiest way to add MDF support in Julia (with the help of one of the Julia C++ packages)
It was unclear looking at the site whether the library was available for free, or there was a license fee for using it.

Another possibility in the mean-time for you might be to use the Python package from Julia, via PyCall.jl.

1 Like
4

You have to pay to use Vector’s mdf4lib.

On the C++ side there are some alternatives:

1 Like
5

Thx! I think I might be able to manage with one of these libraries and starting here.

6

Thx! I think this can get me going.

1 Like
7

Any progress for this topic?

8

I have not made any progress, but I’m definetly still interested.

9

In the meantime, you can use PyCall to call a Python package like mdfreader.

10

My interest is actually in comparing my Python implementation (GitHub - danielhrisca/asammdf: Fast Python reader and editor for ASAM MDF / MF4 (Measurement Data Format) files) with a julia implementation performance wise.

1 Like
11

This is what worked for me under MS Windows to install “asammdf” inside the miniconda environment of Julia:

ENV["PYTHON"] = raw"C:\bin\WPY64-~1\PYTHON~1.AMD\python.exe"
ENV["PYCALL_JL_RUNTIME"] = raw"C:\bin\WPY64-~1\PYTHON~1.AMD"
Pkg.build("PyCall")
import Pkg; Pkg.add("Conda")
using Conda
Conda.add("asammdf"; channel="conda-forge")

The environment variable ““PYCALL_JL_RUNTIME”” might not be necessary.
And the a 8.3-filename format might not be necessary.

And this my trial to export MF4 → mat,v7.3:

using PyCall
asammdf = pyimport("asammdf")

mdf = asammdf.MDF(raw"C:\data\Rekorder_2020-10-19_16-20-40.MF4")
mdf.export(fmt="mat", filename=raw"C:\data\Rekorder_2020-10-19_16-20-40.mat", single_time_base=true, overwrite=true, format=7.3, oned_as="column")

The analogous python script is as follows:

from asammdf import MDF

mdf = MDF('C:/data/Rekorder_2020-10-19_16-20-40.MF4')
mdf.export(fmt='mat', filename='C:\data\Rekorder_2020-10-19_16-20-40.mat', single_time_base=True, overwrite=True, format=7.3, oned_as='column')

I have the impression that the execution is best, if I run it inside the Julia command window,
with the include command:

include(raw"C:\data\julia\data_im_and_export\export_mf4_to_mat.jl")

The export can take some time (Julia v1.7rc2, Python v3.9.5).
On my machine it is so slow that my first impression was: it is not exporting at all.

12

Using single_time_base=True can be really slow if you have signals with different sampling rates.

It would be faster to first filter only the necessary signals, resample and then export

13

First of all: Thank you for this nice peace of software!!!
The speed is related to the different version states of all the involved packages, even under python it took (depend on the versions installed) quite long to export and under Ubuntu 20.04 I was confronted with a memory buffer overflow).
Now with Julia 1.7.0 I have the impression it works fine. The version that is automatically installed on
my window machine is v5.19.16 (python is v3.9.7 from conda-forge).

14

Daniel told me how to avoid the buffer overflow, which occurs also with the newest
version of asammdf (v7.0.1) under Python. If you also face a buffer overflow issue,
you may add the option: raster=0.1