Using a C++ library with Julia 1.*.*

There is still other than aesthetic reason to make something privat don’t you think?

I don’t expect you’ll get deprecation warning like this about private mebers, do you?

I was trying to search your private->public construction in cling but was not succesful:

$ ag "#define private" cling/ 
[nothing]
$ ag "#undef private" cling/ 
[nothing]
$ ag -l "private" cling/ | wc -l
74
$ ag -l "#include.*clang.*parser.h" cling/ | wc -l
7
$ ag -l "#include.*clang.*.h" cling/ | wc -l
72

maybe they are successful to use another approach?

Thanks for offer (it would be really nice to cooperate with you) but unfortunately not only Julia core devs have bigger priorities … :frowning:

BTW using C API instead of C++ could not help?

From clang DeveloperPolicy page:

Stability Guarantees: The C API is, in general, a “best effort” for stability. This means that we make every attempt to keep the C API stable, but that stability will be limited by the abstractness of the interface and the stability of the C++ API that it wraps.

I was using aesthetic as opposed to semantics (e.g. some projects have stability guarantees of their public APIs/ABIs but not their private ones). Obviously it encodes the author’s intention about what part of the API you should need if you were writing a C++ compiler. The only problem is that I’m not writing a C++ compiler, so I need to do things that the authors did not anticipate.

I mean, you would if somebody added the deprecation tag to it. However, Clang’s APIs generally don’t go through deprecations cycles. That particular API is a bit of an exception, because it’s one of the most commonly called functions in the code base, so there’s a few thousand internal and external users that needed to get updated for which the warning was helpful.

Cling maintains a fork of clang and IIRC carries a patch to expose the private methods that they need. I didn’t want to do that because maintaining a fork has a significantly higher maintenance burden, and the end effect is the same.

No, the C API is able to be stable, precisely because it is quite limited.

Look nobody is saying here that hacking something together is a great way to get things done, but sometimes it’s the only way to get things done. Clang was not intended to be used by Cxx.jl, so unless and until somebody sits down and goes through the extensive process of changing that, it’s what we’re stuck with.

Lastly, while I have you here, I’d like to turn to a different topic. I would encourage you to reconsider your rhetoric style on this forum. You could have easily made your point by saying something to the extent of “I notice that Cxx.jl uses these anti-patterns - does anybody know why that is and if there’s something we can do to improve that”. Instead you went with:

which reads as “@Keno is an idiot and doesn’t know how to write maintainable code”, which, while that may be perfectly true, is still not a very nice thing to write about somebody. Now, I’ve personally been doing this too long to let that get to me too much, but I’ve noticed your tendency to do this repeatedly on this forum and others have too. I do care about the quality of discourse here and I find that kind of approach harmful. Please consider that point in the future before you post.

23 Likes

There was something upthread about volunteering or funding. Would it make sense, that we ask you to sit down for half an hour and write down in some paragraphs in the README.md of Cxx what the problems are, what code and solutions are needed, and what from your view is stopping progress on Cxx? There are smart people out there and maybe they just need a good starting point.

Would it make sense that we maintain one additional patch to LLVM, besides the ones used in Julia base, to expose the needed interfaces and compile and ship this patched LLVM with BinaryBuilder.jl?
I’m kind of planning to try it, but I don’t know if it makes sense.

Yes. I really could do that.

Thanks! I believed Stefan that I mistakenly wrote “unmaintainable” and I really wrote “hardly sustainable” (which is big difference).

I deeply apologize that my words could be interpreted that I think you are idiot, which I certainly don’t!

But I think you have to agree that code based on unstable API with hack like #define private public really is hardly maintainable.

I support @lobingera’s idea:

Where I am also curious what do you think could be used instead of clang. →

Could you start with some draft ideas ASAP?

Apart from the way you talked about others code, I think the kind of demanding behaviour you’re displaying here and in other discussions rubs many people the wrong way. Keno has a lot going on and he might think there are better uses of his time right now than drafting ideas for you.

This is an open source project. You’re not a paying customer and sometimes if you need something done fast, you have to roll up your sleeves and do it yourself or find a workaround.

Asking someone to do something “ASAP” is usually considered pretty rude, especially if they’ve already made it clear that they have higher priorities.

10 Likes

It starts to look like in this community we have some kind of untouchable themes.

As I remember I was critical just about concrete part of Cxx.jl (which I see (if it works) as state of art of Julia) code and I am still pretty sure that using #define private public is problematic and it has deep consequences.

One of them is that we have Cxx.jl advertised on main Julia page but it doesn’t work several month with actual version.

You could say it is because Keno has no time now. But some others tried to port it to Julia 1.0 (upgrading Julia packages has to be easier than upgrading packages for python3 right?) unsuccesfuly.

I know other possibility for Keno was to write a less attractive package or write it in a much longer time. He choose this way. Now is difficult to find somebody who could help to maintain.

I still don’t say that whole situation is worst possibility! But we are (also) coders here - we could see lessons. If we are open enough to speak about problematic part of code too.

ASAP means as soon as possible. I just show interest and trying to emphasize that just draft some ideas (which Keno surely have already in his mind) is also very good at first! I fully respect Keno’s possibilities.

You are mistaken. In particular, constructive discussions of code are usually very welcome. Usually, these take the form of a pull request (which is implicitly a suggestion for improving parts of the code), or discussions of a PR. Most repositories also welcome questions about existing code (“please explain what this does”), provided the questioner did some research before.

What is less tolerated is trolling, and comments which are purely inflammatory. There is, of course, a spectrum of behavior ranging from these to well-intentioned discussion, but it is usually best to stay away from the kind of tone you are using.

What you are not getting is that all of these contributions are available as free software, so it is pretty pointless to complain about how the author happened to code something, or why he had other priorities than maintaining a particular package for a while. Free software is take it or leave it. You are also encouraged to write your own version if you think you can do better.

Also (and it feels silly to explain this to someone who is presumably an adult): deprecating someone’s work and then two comments later asking that he does something for you “ASAP” is not something that flies well.

7 Likes

This approach is basically what is automated by CxxWrap, and also the reason why CxxWrap requires building a shared library. For users trying to wrap a C++ library, the choice between Cxx.jl and CxxWrap.jl basically boils down to this:

  • I want to write the wrapper in C++ → use CxxWrap.jl
  • I want to write the wrapper in Julia → use Cxx.jl

Other considerations are that with CxxWrap.jl you get a compiled library that loads quiclky, while with Cxx.jl you can do cool things like pass Julia types as template parameters to C++ types on the fly.

There is indeed currently the issue of Cxx.jl not working, but I’m sure that’s temporary. Apart from requiring C++ and Julia knowledge, working on it also requires LLVM knowledge, it seems that at the moment the intersection between those groups is a singleton :wink:

10 Likes

This is a great summary of the different approaches of CxxWrap and Cxx.

It also requires knowing some stuff about how DWARF, ELF, etc. It’s all learnable, but it would take time.

1 Like

Actually Julia community is the most friendly one I have ever experienced. Some forums of new language are filled with hyped fanboys, for example, Rust, Nim, etc. While some forums of old languages are characterized by stone-aged arrogant guys, for example, Fortran, Ada, etc.

Julia community is as friendly as its syntax.

11 Likes

Cxx.jl works with Julia 1.1 to 1.3, i.e. no supported version (I put in the warning in the package docs about that).

I was rereading this thread, feeling a bit responsible, thinking I made a PR to the website, seems I didn’t, I’m recalling to Julia’s docs, actually it was for adding CppWrap.jl:

[Yes, I DID drop “Limited support” wording for C++ in that PR, as it seemed pretty unlimited to me, at the time, not knowing all the (future) issues.]

Anyway should Cxx.jl be dropped there and at the website? Is there I light at the end on the tunnel?

I also think Cxx.jl is state-of-the-art and Keno a genius, and sympathetic to needing hacks (unavoidable it seems), since:

From what I read before, Cxx.jl is a dependency of a specific LLVM, and Julia doesn’t want to stay with old one (understandably). There is seems this will always be a potential problem, even if fixed for say 1.7 (is that likely?), then in some newer, e.g. 1.8. If 1.7 ends up being LTS then at least is gives people some time until it potentially breaks again.

This is why:

but it only seems to shift the problem. Could Cxx depend on Cling rather than LLVM? Would that be even more work to “fix” Cxx.jl? Because I really like that option and would want it back!

There’s another thread on a Python package doing similar (with using Cling), other languages too can by now, and even C++ in Jypyther works interactively. While non-ideal (but often a good hack) we could use that Python package, so this is no deal-breaker, except in some very high-performance situations.

One other intriguing possibility: I read elsewhere that LLVM might be dropped as a dependency. Or optionally. That idea was for PackageCompiler.jl in case you need no code generation at runtime. I’m not sure if when you need LLVM, if it would be done as with libtraompoline (for BLAS), i.e. you could load an appropriate LLVM (an older one that works; even for newer Julia that otherwise would use later LLVM, could maybe two LLVMs potentially work?).

“On 12th of May the initial, minimally functional, clang-repl landed in the LLVM repository.” (Source: Cling Transitions to LLVM’s Clang-Repl - ROOT)

2 Likes