The right way to tackle this would be to start a BioFormats.jl package; it could be hosted by the JuliaIO organization. You would then only need to “register” (a few lines of code) BioFormats with FileIO; see the FileIO README and its src/registry.jl file.
If you are excited to start working on this, I can create a “blank” BioFormats.jl repository and give you push access. Just let me know.
many thanks Tim,
I’ll try to present status of my attempts to use Julia to the PI’s soon - to get, hopefully, a go-ahead.
or maybe someone else will take it.
most of my time I’m on absolutely different project now.
I’ll let you know in nearest days.
…
I know very little about the particulars here, but I highly doubt that wrapping this library from Julia would take that much time or effort. Most libraries are fairly regular and a little bit of metaprogramming goes a long way. It would probably take one person with the requisite Julia skills and knowledge of the Bioformats library no more than a month to get a solid wrapper. Of course, that depends on how regular and consistent the library API is.
It would seem that writing wrappers for packages in other languages is likely to be a very common julia development activity in the next years; since Julia has so good facilities (in JuliaInterOp) for doing so, and because of the relative youth of the julia package ecosystem. I can think of a few I’d like to wrap myself.
It would be extremely useful if there were a best practices document or a general template for doing these wrappers, which would also have the benefit of allowing some common standards for wrapper libraries. But I haven’t been able to find such. Is there such a document, or e.g. a reference package that would form a good template here? (preferably without requiring one to be very familiar with the code in that underlying library, e.g. I don’t know the underlying imagemagick very well).
to estimate it, again, good example is bfMatalb which can be downloaded from here:
https://downloads.openmicroscopy.org/bio-formats/4.4.9/
sometimes when we had difficulties when saving files with specific metadata, we did it in more manual way, but that was necessary base.
…
I can see only 8 files / functions in bfmatlab.zip, which sounds more like a couple of days to wrap it Julia. If no Matlab-specific libraries are involved of course, but I think it’s not the case.
good to know, but that is rather lower estimate, I afraid for us it would take much longer.
BTW we now have OME.tiff examples of real world data - FLIM, OPT and OPT-FLIM, which is good help.
when we started collaborating with OME/Bioformats team, we had neither, and it was pain to raise it all together, data, I/O and analysis software.
…
MATLAB interop is weakest, mostly because you then have all of the licensing restrictions of MATLAB. I don’t think you can CI test code which uses MATLAB.jl for example. But yes, this would be a quick solution. Though it would be easier for the library to be used if it could try to build off of Python or something like that instead.
I don’t know of “best practices”, but I think it’s generally accepted to say that the functionality when wrapped should probably be given a more Julian API. But that really depends. For example, the Clang wrapper of Sundials.jl leaves the API pretty much intact because then it just refers to the C++ Sundials manual for the lower level parts. However, on top of that it puts a higher level Julian API (a link to DifferentialEquations.jl) so that way it’s easier to use. I don’t know if it would work for all or most wrappers, but this two-tiered approach has worked quite well here.
I think there shouldn’t be anything indispensable from Matlab itself.
yes, but rather, even “essentially Julian”.
for example, some of string outputs from bfMatalb, are “formatted to be used with uigetfile funciton”, which is Matlab function.
basically, the best way to feel how handy it is, from my own experience, is to get bunch of real data and try to “batch-analyse” them (maybe with some minimal GUI if not too complicated), - open, access all needed metadata, process, create output data, then save them - probably in other format; but in our case input was or tiff or OME.tiff and output always OME.tiff.
…
I tried to translate further, digressing from “angles”, as it is simply a specific case of getting numbers from modulo structure.
I likely have some more in Java class path as it passes the first step.
One of crucial components may be “guava.jar”.
however, the error happens just on second line
r = JChannelSeparator((JChannelFiller,), r)
the error is
No constructor for loci.formats.ChannelSeparator with signature (Lloci/formats/ChannelFiller;)V
In the git repo I cited, there is TestData directory where one can find the file named “fluor.OME.tiff”.
It contains angle metadata.
BTW to inspect metadata, the convenient trick is to click on “bioformats_package.jar” - it opens the viewer, then one can load image there, and see what is inside.
I paste below your code a bit changed - with my perceptions of how it should be, likely wrong in many places.
I think even if something like this simple, is made functional, one then can initiate “BioFromats.jl” with it, etc.
using JavaCall
try
# init JVM wtih bioformats_package.jar on classpath
JavaCall.init(["-ea", "-Xmx1024M", "-Djava.class.path=bioformats_package.jar"])
end
# import java classes
const JChannelFiller = @jimport loci.formats.ChannelFiller
const JChannelSeparator = @jimport loci.formats.ChannelSeparator
const JOMEXMLServiceImpl = @jimport loci.formats.services.OMEXMLServiceImpl
const JOMEXMLMetadata = @jimport loci.formats.ome.OMEXMLMetadata
const JOMEXMLService = @jimport loci.formats.services.OMEXMLService
const JMetadataStore = @jimport loci.formats.meta.MetadataStore
const JFormatReader = @jimport loci.formats.FormatReader
###########################################
function getModuloZ(full_filename)
ret = [];
# same as `new ChannelFiller()`
# empty tuple `()` here means that constructor doesn't take any args
r = JChannelFiller(())
# same as new ChannelSeparator(r)
# again, first argument - `(JChannelFiller,)` - a tuple of input types
# the rest - `r` - is a list of actual arguments
r = JChannelSeparator((JChannelFiller,), r)
# same as `new OMEXMLServiceImpl()`
OMEXMLService = JOMEXMLServiceImpl(())
# same as `meta = OMEXMLService.createMEXMLMetadata()`
meta = jcall(OMEXMLService, "createOMEXMLMetadata", JOMEXMLMetadata, ())
# same as `r.setMetadataStore(meta)`
jcall(r, "setMetadataStore", Void, (JMetadataStore,), meta)
# r.setId(full_filename);
jcall(r, "setId", Void, (JMetadataStore,), full_filename)
# modlo = r.getModuloZ();
modlo = jcall(r, "getModuloZ", JFormatReader, ())
if !isempty(modlo)
if !isempty(modlo.labels)
# ret = str2num(modlo.labels)'; %Matlab
# needs an equivalent
end
if !isempty(modlo.start)
if modlo.end > modlo.start
nsteps = round((modlo.end - modlo.start)/modlo.step)
ret = 0:nsteps
ret = ret*modlo.step
ret = ret + modlo.start
end
end
end
return ret
end
Try this for the second line:
r = JChannelSeparator((JIFormatReader,), r)
It turns out actual constructor has a bit different signature, so we need to pass (J)IFormatReader as the type argument.
thanks very much, that allowed to pass another 3 lines,
unfortunately it still stumbles on these lines
(I tried to adjust corresponding class according to Bioformats documentation)
# r.setId(full_filename);
jcall(r, "setId", Void, (JIObject,), full_filename)
# modlo = r.getModuloZ();
modlo = jcall(r, "getModuloZ", JIFormatReader, ())
either of them cause error of the same type
LoadError: Error calling Java: java.lang.NoSuchMethodError: setId
LoadError: Error calling Java: java.lang.NoSuchMethodError: getModuloZ
sorry for bugging you so often, but I’m really intrigued, if it is workable.
…
It seems like you are confused with the arguments of jcall. Here’s what it expects e.g. for a method with 2 arguments:
jcall(<java object>, "<method name>", <type of returned value>, (<type of argument 1>, <type of argument 2>), <argument 1>, <argument 2>)
For example:
"myString".indexOf('A', 5)
can be called as (although I can’t check this code right now):
jcall(someJavaString, "indexOf", Int, (Char, Int), 'a', 5)
According to the signature, setId takes a String, so instead of:
jcall(r, "setId", Void, (JIObject,), full_filename)
you first call should be:
jcall(r, "setId", Void, (JString,), full_filename)
Now try to do the same with getModuloZ and let me know.
got it, - it works except for the last block where one needs to operate with class members
starting from the line
if !isempty(modlo)
… etc.
it throws error
LoadError: MethodError: no method matching start(::JavaCall.JavaObject{Symbol(“loci.formats.Modulo”)})
…
I’m trying to translate further, the “bfopen” which opens reader and then makes calls to “bfGetPlane”.
in “bfopen” I was able to get XYZCT dims of the data, which is encouraging:
sizeX = jcall(r, "getSizeX", jint, ())
sizeY = jcall(r, "getSizeY", jint, ())
sizeZ = jcall(r, "getSizeZ", jint, ())
sizeC = jcall(r, "getSizeC", jint, ())
sizeT = jcall(r, "getSizeT", jint, ())
but now stuck on functions with >2 input parameters in this block from “bfGetPlane” (tried to queue more parameters but that didn’t work):
% convert byte array to MATLAB image
if sgn
# % can get the data directly to a matrix
I = loci.common.DataTools.makeDataArray2D(plane, ...
bpp, fp, little, ip.Results.height);
else
# % get the data as a vector, either because makeDataArray2D
# % is not available, or we need a vector for typecast
I = loci.common.DataTools.makeDataArray(plane, ...
bpp, fp, little);
end
the stuff just before this excerpt looks OK:
pixelType = jcall(r, "getPixelType", jint, ())
bpp = jcall(JFormatTools,"getBytesPerPixel",jint, (jint,), pixelType)
fp = jcall(JFormatTools,"isFloatingPoint",jboolean, (jint,), pixelType)
sgn = jcall(JFormatTools,"isSigned",jboolean, (jint,), pixelType)
little = jcall(r, "isLittleEndian", jboolean, ())
plane = jcall(r, "openBytes", Array{jbyte, 1}, (jint,), iPlane)
hope it is all reconcilable.
…
Note that if you are porting the matlab code directly that appears to be gpl (https://github.com/openmicroscopy/bioformats/blob/v5.3.4/components/formats-gpl/LICENSE.txt), and so your derivative work would be GPL too. The recommended license for julia packages is MIT, though that is not required AFAIK.
thanks,
bfMatlab files have the stamp
% This program is free software: you can redistribute it and/or modify
% it under the terms of the GNU General Public License
what I’m trying to reproduce is the most basic functionality, which is unrelated to Matlab.
as I understand, this code was in its turn, translated to Matlab from some previous medleys of Java excerpts and examples created by Bioformats team.
overall I think bfMatlab is over-engineered for my application in many places, and I will need some subset, but then it may be extended according to Julia logics and needs of other users.
…
From the GitHub page, bioformats library itself is also licensed under GNU GPL, though commercial licenses are also available. If you come up with a library not inferred from bfMatlab, you can probably provide to a user an option to use commercial version of bioformats with your library with whatever license or use both bioformats and your library under GNU GPL (because you cannot link not GPL-licensed software with not GPL-compatible software).
But, honestly, I don’t think you should worry about it right now - getting things working is much more important, and GNU GPL is not as restrictive as it might sound (e.g. it make no difference if you just want to run such software in your lab).
I’ll reply to your other questions a bit later, but for now consider creating a repo on GitHub so we could see the whole code and keep track of a progress.
Just to be sure you know, the OME TIFF format is just a plain TIFF file with some metadata attached. That means you can read it using any of the existing libraries for reading image formats, such as ImageMagick.jl or QuartzImageIO.jl. The image OME metadata can be extracted from the appropriate TIFF tag and parsed with something like LightXML. If you’re using MicroManager, I believe there’s a separate set of JSON metadata that can also be extracted from the file.
This may be easier than firing up a Java library each time you open a file, and probably faster too.
isempty() is a Julia function that is not defined for type JavaCall.JavaObject{Symbol("loci.formats.Modulo")}. Not sure how Matlab handles it, but in Julia you can call e.g. method Modulo.length or create a wrapper function:
function Base.isempty(modlo::JavaCall.JavaObject{Symbol("loci.formats.Modulo")})
len = jcall(modlo, "length", jint, ())
return len == 0
end
I can’t find this class in the repo, where does it come from?
Also, how do you call it from Julia and what is the error?