New to Julia (today). Can’t find discussion or tag for #new-platforms, #porting etc hopefully we can save each other some time there…
Rolled my own 1.4.1 on MinGW64 thanks to (team) but I tried to go deeper and get IJulia for my Jupyter install and other things that kept running into dependencies from JuliaBinaryWrappers. That’s where things turn real ugly because those guys are binary-only. Like they can’t even imagine that someone would want to use this on something other than their “listed platforms” it seems
This impression might be wrong and I hope it is. Otherwise… there seems ZERO information for newcomers on porting beyond the base package. Like what exactly does Unable to automatically install ‘Zstd’ from… actually mean in detailed terms esp when I have a fully working / tested zstd lib ready for linking?
Hi and welcome to julia.
The JuliaBinaryWrappers org is an org to keep binary only packages, those packages are generated with GitHub - JuliaPackaging/BinaryBuilder.jl: Binary Dependency Builder for Julia, with their build scripts stored in GitHub - JuliaPackaging/Yggdrasil: Collection of builder repositories for BinaryBuilder.jl. They serve to, if a julia package wants to use an external binary depency i.e FFTW, the user doesn’t have to install it themselves separately. That does mean however that the packages won’t use the libs you already have in your computer, they do that for version reasons, it is possible to use a local version, but most of the time it isn’t worth it.
I do have a question though, why not download from here Download Julia the latest version?
1.4.1 is kind of an old version, do you need this specific version? Why not use e.g. 1.6.2 (the latest stable release?
The whole purpose of those packages is to save the vast majority of users the hassle of having to build dependencies manually. “They” are real people that also frequent this forum.
If you post the error message you’re getting, we may be able to help you with whatever error you’re actually seeing.
Because a package may require a specific version of Zstd and you may not have that installed/other programs on your system don’t work with the version the package needs?
The purpose is to isolate julia dependencies from system dependencies, because crosstalk between different programs requiring different setups/libs/versions is so very often a major point of breakage (e.g. have you tried installing CUDA before?). A lot of packages will give you the option of specifying a system install instead though, which will try to use whatever you have available on your system (e.g. MPI does this).
First, lets be honest the API differences between 1.x.x.x and 1.x.x.2 are usually not very big (project is probably dommed anyway if they are)
MINGW64 uses its own CRT and, when things get complicated, non-sharable C++ manging against MSVC. Also MINGW CRT understands forward-slash and a whole heap of other portable goodies not to mention the butt-load of work done by the MSYS2 team to champion Linux package porting. So yeah there are a ton of reasons to be interested in MSYS2/MINGW64 for windows users that might not be useful for you. So a native solution is my solution. built from scratch with my libs and my flags fair enough?
So in this context I want to use the libs on my system. It might surprise some of you how much work goes into porting those over to MinGW properly. And remember MSVC C++ libs use entirely different symbol mangling than g++ anyway.
Actually I was hoping more for a … duh… go here and follow instructs to re-compile pkg X from source… like in Ruby… or Perl… or Python… or…
It’s more about performance upgrades between versions, I don’t have much experience with windows and julia, https://github.com/JuliaLang/julia/blob/master/doc/build/windows.md this here says that the most recent version of MSYS2 isn’t working correctly, it has instructions for mingw32. To override it follow the instructions here JLL packages · BinaryBuilder.jl. Although your mileage may vary
We’re not surprised here. There has been A LOT of work to get binary builder to build source binaries for Linux, macOS, windows and BSD reliably. You suggesting that this was not a main goal (when portability is a stated goal of the project) is quite disheartening and does not help with the problem you seem to be having, so I ask again:
Please post the error you’re getting so we can help you get it fixed.
Since when does anything work properly? Windows 10 working properly? Debian buster working properly (yeah preach that broken Gnome-Wayland pseudo-gospel).
What is really weird is that nobody has actually answered my question yet.
No need to be rude, you can use local binaries as I said in my previous post, I also said in it that julia isn’t supported on MSYS2 currently, it uses cygwin apparently. To get correct support for the binaries the BinaryBuilder ecosystem will have to able to compile using MSYS2, which to my knowledge it can’t currently. You are asking for support on an unsupported platform. If you want I am sure that the developers would like someone the help add support to it.
I’m 95% certain that this is not the full error message, and I’ve seen my fair share on this forum Please post the full output.
Also, just a note - we’re all volunteers here, spending our free time to help out other julia users. This is not a support forum visited by paid staff (there is no paid support staff). I’d appreciate it if you’d treat us kindly.
gbaraldi is wasting my time. I was never rude to him but he keeps trying to get me to switch platforms (!!!). Bye dude. But I’ll reply to this. Other than the folder locations that you dont need Ill paste a couple of tried so that you truly believe me…
Installed TranscodingStreams ────────── v0.9.6
Cloning [5789e2e9-d7fb-5bc7-8068-2c6fae9b9549] FileIO from https://github.c
m/JuliaIO/FileIO.jl.git
Installed FileIO ────────────────────── v1.11.0
ERROR: Unable to automatically install 'MbedTLS' from '\MSYS2\...\qMb7d\Artifacts.toml'
Stacktrace:
[1] top-level scope at REPL[5]:1
julia> Pkg.add("BinaryBuilder")
Resolving package versions...
ERROR: Unable to automatically install 'MbedTLS' from '\MSYS2\...\qMb7d\Artifacts.toml'
Stacktrace:
[1] top-level scope at REPL[5]:1
julia>
What I am trying to say is, that platform isn’t supported yet. That means it won’t simply work.
The errors you are getting seem to be that some part of the artifact system 8. Artifacts · Pkg.jl isn’t working.
I imagine other parts of Julia would also need to be changed to make it work right. If you want help from the devs to make it work, open an issue on GitHub, support for msys2 is something that Julia is lacking right now. Maybe that sparks the interest in making it work.
… Maybe the part that’s hard to understand here is that I’m not expecting a “supported platform”. What I am expecting is the open-source community spirit that welcomes efforts on new platforms. At this point I can see this isn’t the case here … more like Node.JS where the whole scheme is built to only work with “pre-approved” platforms.
Edit: I’m not looking to be argumentative, but truly, “how to compile a binary extension from source” is not rocket science on most other languages…
I’m going to start looking for a way to delete this thread…
Ok, sorry, I see now that you want to work on a port to MSYS2, I think that the best way to start that would be to either find an open issue in GitHub if it exists, or open one. That way the devs can help you port it.
Hi @scanf, welcome. I’m a moderator here, and this seems to have started off a little rocky. I can assure you folks aren’t trying to waste your time; to the contrary lots of effort has gone into our binary packaging to make it as friendly to newcomers as possible. Folks here are just trying to steer you out of the weeds, but if you’re happy helping carving out new paths then please don’t get scared away!
Rather than the #new-to-julia category (where folks typically deal with basic usage questions), I’d encourage you to get involved — on GitHub (as suggested) or on Slack (where there’s a dedicated binary-builder channel with the core team) or on Zulip (similarly).
The trouble is, currently julia itself is not supported on MSYS2/MinGW64. If you’ve chosen 1.4.1 because that’s the last version MinGW64 building worked, you’re already missing out on a lot of features from 1.5 and 1.6 (large performance improvements across the board etc). Getting support for MinGW64 up and running again would be great though and deeply appreciated, I can assure you of that.
Then on top of that, since julia is not guaranteed to build on MinGW64, neither are the source packages built by BinaryBuilder (since they’re intended to be consumed by julia for the reasons stated above). Using packages that use artifacts on MinGW64 would then mean building the artifacts for MinGW64 (ideally with binary builder, so a recipe for doing so can be contributed to the main repo).
I understand though that you don’t want to/can’t necessarily build packages from source and want to use your existing binaries instead - in which case you can follow the instructions linked to earlier by @gbaraldi on overriding package sources to hard coded paths using Overrides.toml, which exists for the express purpose of using system binaries/libraries instead. The whole system has been designed with system administrators and local constraints in mind.
Thanks I try to keep a minimal profile on GitHub because I always get asked for PRs that I dont have the time for (miss the days of submit a patch).
Look, like I opened with this is my first day of julia and everything was fantastic until I hit your binary wrappers. I just followed the code to https://github.com/JuliaBinaryWrappers
and realized it was all prepackaged which was why julia wouldn’t install it (not like python/ruby/perl/etc that actually grabs source and cranks up gcc). That was a new one for me…
What you’re seeing at JuliaBinaryWrappers is a GitHub organization — and indeed a funny one at that. It’s the output of scripts that are created and maintained at Yggdrasil . We should probably add that bit of context to the org page itself, like it is linked to from all the READMEs therein.
Please post the full output.