Method of `open` that applies a function to various files

Package Announcements
1

I have made this very tiny package to help myself in a very common use case, when I want to read data from big files and write results of processing its contents into another, chunk by chunk.

It only contains a new method for Base.open, which essentially mimicks open(f::Function, args...; kwargs...) from base/io.jl, but is specific for the case when the first item of args is a tuple that identifies various files. It opens all the files with the given options by broadcasting, and applys f --which should take as many arguments as files given–, closing the files upon completion.

As far as I know open(::Function, ::Tuple{Vararg}, args...; kwargs...) does not exist in Base, so this should not break anything.

Take it if you think it may be useful for you too. Improvements and extensions are welcome.

2

Since your package does define neither the function nor the argument types, this is type piracy.

https://docs.julialang.org/en/v1/manual/style-guide/#Avoid-type-piracy-1

1 Like
3

Thanks, I have added a note to warn about this in the README file.

What would be the preferred way to avoid type piracy in this case?

  • Move this functionality to a function not from Base -e.g. openmultiple?
  • Make a PR to Base instead of putting this in a separate package?
4

A different name for the function would be the simplest solution IMO.

1 Like
5

Good, I have changed it. Now this is made by a function called openmultiple.
Thanks for the advice!

1 Like
6

A third alternative is to just not overload Base.open and let people use

IOExtension.open(...)
2 Likes