Bioformats in Julia?

[yalexand]

@rsrock

Just to be sure you know, the OME TIFF format is just a plain TIFF file with some metadata attached. That means you can read it using any of the existing libraries for reading image formats, such as ImageMagick.jl or QuartzImageIO.jl. The image OME metadata can be extracted from the appropriate TIFF tag and parsed with something like LightXML. If you’re using MicroManager, I believe there’s a separate set of JSON metadata that can also be extracted from the file.

This may be easier than firing up a Java library each time you open a file, and probably faster too.

thanks, - yes one can open these files as multi-plane tiffs, but OME convention interprets them as 5-dims and on top we also use “moduo”, at least one real-life case applies both moduloZ (projection angle) and moduloT (FLIM gate), and to unwind all this, plus pixel type conversions, manually, seems more cumbersome than to use Bioformats.
so if JavaCall functionality is enough to handle it, I’d prefer Bioformats…

another thing is that Bioformats are suited for handling many other bioimaging data, not only tiffs.
…

[yalexand]

@dfdx

I can’t find this class in the repo, where does it come from? Also, how do you call it from Julia and what is the error?

for example here -
http://static.javadoc.io/org.openmicroscopy/ome-common/5.3.1/loci/common/DataTools.html

I don’t have this code with me, will reproduce the error tomorrow and let know.

still I don’t get how to access say “start” field in the “Modulo” structure, which is simply a double, where modlo is the corresponding instance:

modlo.start

[dfdx]

@yalexand

still I don’t get how to access say “start” field in the “Modulo” structure, which is simply a double, where modlo is the corresponding instance:

Yep, quite cryptic, but you will get used to this kind or errors very soon. If you check the method list of isempty with:

methods(isempty)

you will see that the only method that conforms to the type of your argument - JavaObject{...} - is the last one:

isempty(itr) in Base at essentials.jl:339

So Julia (wrongly) assumes that if there’s no more specific method for what you have passed to isempty, you should have passed an iterator. In Julia, iterator is any object for which the following methods are defined:

• start - initialize an iterator
• next - get next element
• done - check if there’s no more elements in the iterator

So the very first thing that the generic isempty is trying to do is to initialize an iterator in your JavaObject{...}. But JavaObject{...} isn’t an iterator and doesn’t have start method! And this is exactly what the error message is telling you.

As I’ve already told, the best way to overcome it is to provide a more specific isempty(::JavaObject{:Modulo}).

[yalexand]

I’ll include and check your isempty tomorrow, - but how to access fields in Modulo structure, like “labels”, “step”, etc.
one can’t use jcall because these are not functions.
…

[dfdx]

As far as I know, you can’t access Java object fields via JavaCall.jl. In general, public fields are considered bad practice in Java and instead getters are used (e.g. getLabels(), getStep(), etc.). In your case, the easiest thing is to create and additional class in Java that:

1. Extends Modulo so that it can be used as a drop-in replacement for it.
2. Implements getters and setters (if necessary).

Then you can bundle your own JAR with all classes from bioformats + your extensions.

It may sound hard, but in fact only adds something like 5% to development time.

[yalexand]

@dfdx

It may sound hard, but in fact only adds something like 5% to development time.

OK looks like not the end of the world.
maybe more politically correct way would be to ask Bioformats to implement these getters.
not fastest way, though…
…

[yalexand]

@dfdx

consider creating a repo on GitHub so we could see the whole code and keep track of a progress.

I created the directory “Julia” in the same “dump” project.
hope that it will be an incentive to rescue this stuff to some more decent place(s) when it acquires necessary shape.

Also added there tomography routines designed earlier and improved/corrected by tim.holy and others

Please feel free to request collaborator permissions.

https://github.com/yalexand/Imperial-OMEkit/tree/master/OMEkitFrontEnd/Julia

[yalexand]

@dfdx

Also, how do you call it from Julia and what is the error?

again the same excerpt where functions with one parameter are called without problems via jcall, but at >1 parameters it throws error

pixelType = jcall(r, "getPixelType", jint, ())
bpp = jcall(JFormatTools,"getBytesPerPixel",jint, (jint,), pixelType)
fp = jcall(JFormatTools,"isFloatingPoint",jboolean, (jint,), pixelType)
little = jcall(r, "isLittleEndian", jboolean, ())

iPlane = 180
plane = jcall(r, "openBytes", Array{jbyte, 1}, (jint,), iPlane)

# this passes OK
I = jcall(JDataTools,"makeSigned", Array{jbyte, 1}, (Array{jbyte, 1},), plane)
S = jcall(JDataTools,"bytesToHex", JString, (Array{jbyte, 1},), plane)

# error, despite functions are in the same synopsis
I = jcall(JDataTools,"bytesToShort", Array{jshort, 1}, (Array{jbyte, 1},), (jboolean,), plane, little )

# error
I = jcall(JDataTools,"makeDataArray", Array{jshort, 1},
                     (Array{jbyte, 1},), (jint,), (jboolean,), (jboolean,),
                     plane,bpp,fp,little)

other words, I don’t know how to call functions with >1 parameters.
it would be great to learn.

the error of the type

LoadError: Error calling Java: java.lang.NoSuchMethodError: bytesToShort

[dfdx]

I = jcall(JDataTools,"bytesToShort", Array{jshort, 1}, (Array{jbyte, 1},), (jboolean,), plane, little)

The 4th argument to jcall is a tuple of all input types, so it should look like:

I = jcall(JDataTools,"bytesToShort", Array{jshort, 1}, (Array{jbyte, 1},jboolean,), plane, little)

Accordingly, the last call should be:

I = jcall(JDataTools,"makeDataArray", Array{jshort, 1},
                     (Array{jbyte, 1}, jint, jboolean, jboolean),
                     plane,bpp,fp,little)

Note that this is done to resemble built-in pseudofunction ccall, so you may encounter similar concepts elsewhere in Julia.

[yalexand]

many thanks - I will look at it tomorrow.
for now I implemented a hack there, and “bfopen” is working, at least for the test file I use.
…

[yalexand]

@dfdx

Accordingly, the last call should be:

it throws error of another type.

iPlane = 180
plane = jcall(r, "openBytes", Array{jbyte, 1}, (jint,), iPlane)
#
# ...
#
  object = jcall(JDataTools,"makeDataArray", JavaObject,
                 (Array{jbyte, 1}, jint, jboolean, jboolean),
                 plane,bpp,fp,little)

the error is

LoadError: MethodError: no method matching write(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Void)

also I noticed that the return type of these functions is “Object”, so, even once it is there, the internal array of either type should be somehow extracted.

overall for now I still don’t see how one can implement the “bfGetPlane” properly with these functions but maybe you see some key?

in bfMatlab it is like

I = loci.common.DataTools.makeDataArray(plane, ...
    bpp, fp, little);

switch class(I)
    case 'int8'
        I = typecast(I, 'uint8');
    case 'int16'
        I = typecast(I, 'uint16');
    case 'int32'
        I = typecast(I, 'uint32');
    case 'int64'
        I = typecast(I, 'uint64');
end

[dfdx]

I don’t have access to my laptop right now, so will take a look at the conversion issue later, but this call:

 object = jcall(JDataTools,"makeDataArray", JavaObject,
                 (Array{jbyte, 1}, jint, jboolean, jboolean),
                 plane,bpp,fp,little)

should use JObject instead of JavaObject. Thanks to confusing Java terminology, we have to distinguish between “an object from JVM” which we denote as JavaObject{name-of-java-class} and class java.lang.Object which we denote as JObject. You use JObject as return type to find appropriate Java method to call through JNI, but what pass in and get out instances of JavaObject{...}.

Hope it’s clear now.

[yalexand]

@dfdx

should use JObject instead of JavaObject.

yepp that works, thanks.
still not sure what to do with “object” but error has gone.

on this wrapper issue that you recommended in order to fix Modulo problem - I wrote an extension class that looks like below, can compile and put into jar but still didn’t succeed to let Julia know how to link to this jar - @jimport fails to recognize it.
maybe you have some idea?

import loci.formats.Modulo;

public class ModuloWrapper extends Modulo
{
    public ModuloWrapper(String dimension) 
    {
        super(dimension);
    }
    
    public ModuloWrapper(Modulo modulo)
    {
        super(null);
        parentDimension = modulo.parentDimension;
        start = modulo.start;
        step = modulo.step;
        end = modulo.end;
        parentType = modulo.parentType;
        type = modulo.type;
        typeDescription = modulo.typeDescription;
        unit = modulo.unit;
        labels = modulo.labels.clone(); // not sure if that will work
    }
      
    public String getParentDimension()  {    return parentDimension; }
    public double getStart()            {    return start; }
    public double getStep()             {    return step; }
    public double getEnd()              {    return end; }
    public String getParentType()       {    return parentType; }
    public String getType()             {    return type; }
    public String getTypeDescription()  {    return typeDescription; }
    public String getUnit()             {    return unit; }
    public String[] getLabels()         {    return labels; }

    public void  setParentDimension(String rhs)  {    parentDimension = rhs; }
    public void  setStart(double rhs)            {    start = rhs;  }
    public void  setStep(double rhs)             {    step = rhs; }
    public void  setEnd(double rhs)              {    end = rhs; }
    public void  setParentType(String rhs)       {    parentType = rhs; }
    public void  setType(String rhs)             {    type = rhs; }
    public void  setTypeDescription(String rhs)  {    typeDescription = rhs; }
    public void  setUnit(String rhs)             {    unit = rhs; }
    public void  setLabels(String[] rhs)         {    labels = rhs; }     
}

[dfdx]

public ModuloWrapper(Modulo modulo) {
    super(null);
    parentDimension = modulo.parentDimension;
    start = modulo.start;
    step = modulo.step;
    end = modulo.end;
    parentType = modulo.parentType;
    type = modulo.type;
    typeDescription = modulo.typeDescription;
    unit = modulo.unit;
    labels = modulo.labels.clone(); // not sure if that will work
}

This is, essentially, a copy constructor, but I don’t get why you need it: thanks to inheritance, your ModuloWrapper is already a drop-in replacement for Modulo, just with a few additional methods.

Note, that it’s not essential to have exactly a wrapper, you could have a class, say, JuliaGateway with methods like:

public static double getStart(Modulo modlo) {
    return modlo.start; 
}

and call it instead. You just need to make a method that accesses your objects’ fields and may be called from Julia.

---

but still didn’t succeed to let Julia know how to link to this jar

I guess you haven’t added it to your classpath. Try to update initialization code as follows:

try
    # Windows uses ';' as a delimiter in classpath, *nix systems nornally use ":" instead
    delimiter = @static is_windows() ? ";" : ":"
    JavaCall.init(["-ea", "-Xmx1024M", "-Djava.class.path=bioformats_package.jar" * delimiter * "you_wrapper.jar"])
end

By t he way, I use try-end to reload code in REPL without reinitializing a JVM, but it’s not strictly necessary.

---

overall for now I still don’t see how one can implement the “bfGetPlane” properly with these functions but maybe you see some key?

JavaCall.jl contains routines for Java reflection. In particular, you can find a name of a class of returned object using:

getname(JavaCall.getclass(obj))

JavaCall.jl also provides several functions for conversion between instances of JavaObject{...} and Julia objects (that’s why you can specify String instead of JString or Int instead of JInteger as parameters or return type in jcall). If it’s not enough, you can always convert Java objects to Julia objects manually. Altogether, your code may looks something like this:

c = getname(JavaCall.getclass(obj))
if c == "java.lang.String"
    convert(String, obj)
elseif c == "java.lang.Integer"
    convert(Int, obj)
....
end

There are elegant ways to do it using Julia’s powerful dispatching capabilities, but let’s leave it for another post.

By the way, I’ve cloned your repo, so if you have some code that doesn’t work, I suggest you push it there and give line numbers so I and others could quickly reproduce it.

[yalexand]

@dfdx

I suggest you push it there and give line numbers so I and others could quickly reproduce it.

thanks for suggestions, all very elegant (and if I do something stupid it’s because I am, but it’s OK I got used to it)

there is no code there that isn’t working at all, it’s only that it isn’t properly done.
for now I (sort of, partly) implemented “bfGetModulo” via XML instead Java route you suggested, and “bfGetPlane” only works for 16 bit big-endians, but as our data are from 16 bit cameras and OME.tiffs anyway, maybe even with that I can live.
these are the main 2 drawbacks.
I will comment such places by capital "TODO"s.

next issue for us will be translating the registration procedure (should be lot of fun), then file saving, then adapting it all for running on cluster, writing batch scripts, translating iterative reconstruction, supporting OPT-FLIM data, - and I’m not sure about priorities. further radon/iradon speed-up as well.

so I’ll try to do some more but can’t promise systematic development of Bioformats for Julia.
also, there is other group here, that will inevitably claim (pound of) my time very soon, and then I’ll disappear for long.
…

[dfdx]

No problem. Having a trouble learning new things isn’t stupid, but rather a natural process showing why good community matters

[mkborregaard]

One thing you might consider is writing a blog post detailing your experiences and what you have learned wrapping this library for Julia. You should have plenty of material for that in this thread, and it would be of great benefit for the community (which, as you have seen, is really friendly and supportive).

[yalexand]

@mkborregaard

One thing you might consider is writing a blog post detailing your experiences and what you have learned wrapping this library for Julia.

Certainly, I benefited immensely from this exercise.

Julia team provided guidance and help

• from overall project’s vision viewpoint (suggesting using JavaCall, Java, XML parsing),
• by resolving (really many) low-level issues,
• by teaching
• by making non-trivial technical contributions (transforms speed-up),
• and even on licensing issues

It is well seen from threads I posted (radon/iradon and bioformats).

One striking thing is about that estimate of time needed to code bioformats bindings.
To make initial version took just 4 days, which is closer to the previously discussed optimistic estimate. Albeit still unfinished, for my application I already can use it’s basic functionalities. Without dfdx’s help it would take lot longer and that was reason for my pessimistic estimate. I didn’t know then what the mode of exchange will be.

So, collaborating with Julia team is both advantage and pleasure and I’ll spread the word among colleagues about it!
…

[yalexand]

made an attempt to write image saving function (as OME.tiff), - failed, too many glitches.

the file “bioformats.jl”, places with errors are commented out and marked by “TODO : fix” comments

actually, I don’t need it just now as results can be saved as usual tiffs.
but won’t be enough for OPT-FLIM volumes, which are 3D with T dimension.
…

[yalexand]

Implemented “JuliaGateway” Java class to circumvent incompatibilities.

type conversion not done properly - I don’t know how to make this stuff consistently template based.
For now I can only open it as UInt16 (the type of data in the source file), - data come out negative, - and can save results only as Float64.
But visually reconstruction looks as expected.

BTW by some reason I wasn’t able to apply “imwrite” function, got “not defined” error.
…