Can `load` work faster?

I have to load many files in a loop (to be more specific, I’m working with OBJ files to be used with MeshIO). I guess that including a line like load(filename) in the loop, with filename being a string with the file path, slows down the code because it is not type-stable (a String-type argument is not sufficient to tell load what type of output will be produced).

Is there a type-stable, smarter way to load those files, knowing that I’m always reading the same type of file? (The output of load in my case is always a Mesh{3, Float32, Triangle}.)

I suspect it disk access and memory allocation that is slowing you down. But have you checked @code_warntype to see if type stability is the issue?

You could do this:

for filename in files
    data::Mesh{3, Float32, Triangle} = load(filename)
    # Data operations on type 1

That will allow the compiler to know what data is, and generate the appropriate code in the rest of the loop. If load ever returns something else you will get a exception that the return from load() could not be converted to a Mesh{3,Float32, Triangle}.

I suspect @pixel27 is correct.
How large are the mesh files?
One simple way to see if disk access is the bottleneck would be to time a read, then time a second read. The second should read from memory cache.
And yes - I know this is not really good enough.

A second method might be to create a ramdisk (tmpdisk). Copy the file to that location then time a read.

@code_warntype confirms that it is not type-stable:

julia> @code_warntype load("example.obj")
  #self#::Core.Compiler.Const(FileIO.load, false)

1 ─ %1 = Core.NamedTuple()::Core.Compiler.Const(NamedTuple(), false)
│   %2 = Base.pairs(%1)::Core.Compiler.Const(Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}(), false)
│   %3 = Core.tuple(%2, #self#, s)::Core.Compiler.PartialStruct(Tuple{Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}},typeof(load),String}, Any[Core.Compiler.Const(Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}(), false), Core.Compiler.Const(FileIO.load, false), String])
│   %4 = Core._apply(FileIO.:(var"#load#13"), %3, args)::Any
└──      return %4 

Also tried to declarethe type of the returned variable, as you suggest, although the full parametric type is more convoluted, but I get the exception anyway:

ERROR: MethodError: Cannot `convert` an object of type GeometryBasics.Mesh{3,Float32,GeometryBasics.Ngon{3,Float32,3,GeometryBasics.PointMeta{3,Float32,GeometryBasics.Point{3,Float32},(:uv, :normals),Tuple{GeometryBasics.Vec{2,Float32},GeometryBasics.Vec{3,Float32}}}},FaceView{GeometryBasics.Ngon{3,Float32,3,GeometryBasics.PointMeta{3,Float32,GeometryBasics.Point{3,Float32},(:uv, :normals),Tuple{GeometryBasics.Vec{2,Float32},GeometryBasics.Vec{3,Float32}}}}}} to an object of type GeometryBasics.Mesh{3,Float32,GeometryBasics.Ngon{3,Float32,3,GeometryBasics.PointMeta{3,Float32,GeometryBasics.Point{3,Float32},(:uv, :normals),Tuple{GeometryBasics.Vec{2,Float32},GeometryBasics.Vec{3,Float32}}}},FaceView{GeometryBasics.Ngon{3,Float32,3,GeometryBasics.PointMeta{3,Float32,GeometryBasics.Point{3,Float32},(:uv, :normals),Tuple{GeometryBasics.Vec{2,Float32},GeometryBasics.Vec{3,Float32}}}}}}

Seems that convert is just undefined for such type.

After that, I have also been looking into the specific implementation of load for mesh files in MeshIO, and it seems the type-instability issue is also there:

function loadmesh1(s)
    f = File{DataFormat{:OBJ}}(s)
julia> @code_warntype loadmesh1("example.obj")
  #self#::Core.Compiler.Const(loadmesh1, false)

1 ─ %1 = Core.apply_type(Main.DataFormat, :OBJ)::Core.Compiler.Const(DataFormat{:OBJ}, false)
│   %2 = Core.apply_type(Main.File, %1)::Core.Compiler.Const(File{DataFormat{:OBJ}}, false)
│        (f = (%2)(s))
│   %4 = MeshIO.load::Core.Compiler.Const(MeshIO.load, false)
│   %5 = (%4)(f)::GeometryBasics.Mesh
└──      return %5

(GeometryBasics.Mesh is not Any, but yet an abstract type, labelled in red.)

Anyway, in this particular case you may be right: my OBJ files are ~10MB, not huge, but still big so that probably disk access is the bottleneck.