Allowing the object.method(args...) syntax as an alias for method(object, args ...)

[gustaphe]

The secret to a good analogy is to not make obvious stretches.

In what way is the obj.f syntax like a safety feature? How is a team member going to cause catastrophic failure by not understanding the julian paradigm?

The issue here is you have a team who are used to object oriented programming and not enough time to convince them to abandon it: then Julia simply isn’t the right tool for your team. Because it’s not object oriented and it shouldn’t be changed to accommodate object oriented programming.

[pdeffebach]

OP is referring to the fact that there is no way to mark a field as “private” in Julia.

This is true, but there are two solutions

1. Overload getproperty so that a.x only exposes the stuff you want to expose
2. Create a convention of using functions to access internal objects, i.e. a(x). This means you only allow functions you want. It has the added benefit of making documentation easier.

[gianmariomanca]

@pdeffebach yes overloading getproperty seems interesting, but it leaves out methods, or maybe not …

I don’t want to scare anybody with the above example
(see the tab completion in Pluto in the image)

but if you can add self-referencing functions to the instance of a struct, why can’t you have it done automatically at the struct definition level?

[pdeffebach]

That last point seems like a separate issue than getproperty.

As others have said, you are probably overusing OOP design patterns in Julia. Because functions each have their own type, it’s not very useful to put functions inside of structs like that. It will force recompilation of functions for each new IIRFilter encountered.

As others have said, if you really can’t leave OOP, it’s okay not to use Julia. There are other idioms that are used to manage large projects.

Finally, w.r.t. tab completion. Developers are aware that it’s hard to do tab completion on methods. There is an open issue for that on github.

[gianmariomanca]

@pdeffebach

it’s not very useful to put functions inside of structs like that. It will force recompilation of functions for each new IIRFilter encountered.

OK, now we are talking can you expand a bit? or point me to a page to understand this a bit better?
Notice that in this example it will be very rare to instantiate the same object multiple times (maybe for debugging or comparing alternative configurations once in a while).

[Henrique_Becker]

Okay, now your questions makes more sense.

@gianmariomanca

I want a convincing story of why using Julia would be a good idea and have answers for the obvious questions coming from those people.

About the convincing stories, ask for them. I am almost sure there was already a thread about this in the past. I would cite JuMP.jl as a case of success, even if I have some criticism about some of their choices. JuMP got a lot of researchers in my area (Mathematical modeling and optimization) to use it, so they could preprocess the data and build the formulation with high performance and a high-level general purpose language and without hardcoding any specific solver.

For the answers to obvious answers, I recommend both reading the language manual (docs.julialang.org/) and asking the specific questions here. For example, for your current question you got the answer this syntax is not supported nor there are plans for supporting it. I commend your determination to understand the reasons behind it all, but it can be a lot to do this for every question they may have, and sometimes it kinda seems like you want suggest changes to the “problems” you see in the language just to be able to recommend it. This does not seem a good bias to have, if syntax details like object.method are relevant for your team or project (I surely do not understand how if they are highly skilled) then maybe Julia is not the right language and you should re-evaluate your options. If your priority is immediate productivity and it is best achieved by letting a large team to work with the most common paradigm and languages (which they are all accustomed with), then do it.

@gianmariomanca

Within that “context” I was then trying to build a mental model of the current state of the language, community, etc

This is another thing that is hard to rush, but in general: the language is stable now (post version 1.0 there may be improvements but very little breakage if any), the community is active like you can see here in Discourse, the language creators are active, transparent, and check the forums; the packages are somewhat mature but it depends on the specific domain. Like I mentioned before, JuMP.jl+Julia is so good that made researchers jump (no pun intended) from other languages to Julia, but there may be specific domains where Julia does not shine so much (it can call C, Fortran, or even Python in such cases). Obviously, the community and ecosystem are not so big like Java, Python, etc…

@gianmariomanca

Going back to the specific issue, I guess yes, it’s a combination of encapsulation and auto-completion

In Julia, many things are solved by informal contracts. Formal contracts many times go against the dynamic way of the language. For private fields and encapsulation, what the community do in general is to follow this rule of thumb: if it is not documented then it does not exist. Something is a part of the public API only if it is documented (and the documentation does not say otherwise). If fields are to be private, then they should not be documented using using the usual strings above them (which allow Documenter.jl to create HTML pages and querying the documentation with ? in the REPL). Obviously, private fields and methods can yet be documented with comments; or you can use Documenter but add a box/note “Internal use.” in the private methods/fields documentation. People are also trying to start a convention of starting such methods/fields with underscore. I like the idea for private methods, but most structs are composed only of “private” fields (this is, you are only expected to call methods over them) so this seems a little obnoxious/ugly for these private-only structs.

Finally, about guarding rails, I am all about them when they can prevent common mistakes coming from lack of attention or something counterintuitive. However, using an undocumented field of an structure is a conscious decision that a highly-skilled person can easily understand when they should and should not do it. I have already done so for quick fixes, when a package missed some feature, and it was easier to freeze that dependency and break the encapsulation than to try to work around without breaking it. This is the exception that you should be able to do, and avoiding it becomes the rule should be the work of QA and, indirectly, of the HR (by contracting good professionals). Not a problem the language should be solving.

[Tamas_Papp]

@gianmariomanca

want a convincing story of why using Julia would be a good idea

I understand the predicament, but you will have to dive into the language and make up your own mind. Julia is not special in this respect; all modern computer languages with advanced features are like this.

For example, suppose I want to evaluate C++ in a similar manner. I could talk to seasoned C++ programmers, who would tell me what they like and don’t like about the language, but eventually I have to consider the fact that they are using C++, so they must like it enough to live with the things they don’t like about it. For my own purposes, I will have to make my own judgement.

That said, the thought experiment

@gianmariomanca

start a project tomorrow within a large company, with a team of 100 people, all highly skilled but busy and with very different backgrounds/interests

is not something a smart company would ever do. If the project runs for a few years (which is optimistic for software), with 100 people you are close to 1 million person-hours. Multiply it by the hourly wage of those highly skilled people, and reflect on the fact that you want a language chosen by …tomorrow?

Managing 100 participants is a challenge in itself. Julia (the language) is the only project I know of which has this many contributors, but very few of them do it even close to a full-time job. This is uncharted territory at the moment for Julia.

What could make sense for a company considering Julia is to start a pilot project with around 3-10 people interested in the language. In fact, many companies (or groups of academics) are doing or have done something like this. Then see how it goes, and take it from there.

(That said, I would be surprised if the lack or presence of an obj.method syntax had any role in the success of such a project.)

[gianmariomanca]

@Henrique_Becker , @Tamas_Papp thank you for the thoughtful answers.

To give some more context I’m interested in Julia for prototyping algorithms for specialized custom made hardware, typically real-time algorithms. In this context Julia doesn’t solve completely the 2 languages problem because there is no Julia compiler for that hardware, but it can “partially” solve it in the sense that even the prototype pipeline will be subject to extensive testing and the goal is to speed up this testing stage (tons of data, clusters involved, etc…). When everybody is happy then it will be eventually converted to C/C++ or whatever language/compiler the hardware supports.

What happens is that someone has an idea, develops a prototype initially with 3 to 10 people as you said and if the idea is successful the team start expanding, with highly skilled people yes, but skilled in their own field. And they need to be able to be pulled in the project, use the prototype pipeline as reference, and perform all kind of sophisticated test/measurements in the matter of days.
If you are one of the guys that developed the prototype pipeline you will be constantly pulled in to help others understand it, run it, etc etc and it can get easily out of hand.
So nobody will stop you from using Julia if you want to, but then you have to live with your choice.

So I definitely want to take advantage of the language myself and learn it better and I’ll keep asking questions as needed, but in the back of my mind when I ask a question there will also be our “thought experiment”.

@Henrique_Becker

I commend your determination to understand the reasons behind it all

You bet, you don’t go into theoretical physics if you don’t want to get to the bottom of things I’m used to environments where we pound on things until either they turn into a diamond or they turn into dust.

This current thread was one of the possible questions that I though could help me focus the conversation, help me learn and expose some biases, unmotivated assumptions either in myself or in others.

@Tamas_Papp

I would be surprised if the lack or presence of an obj.method syntax had any role in the success of such a project.

Well it is already there apparently if you look at my example Just not very conveniently integrated in the language.
Joking aside, I’m still interested in understanding why it’s an issue to add functions inside structs (the re-compilation issue mentioned by @pdeffebach ) for example, and would it still be an issue if you could specify the type of the arguments while defining the struct (some of them referencing the struct you are defining)? etc etc
I didn’t have the cycles now to open multiple threads now, but maybe little by little.

Same with getproperties and getfields. Not fully clear how you would use it to make some some properties private since there are no internal methods, if you hide some properties overloading getproperties then you need to use getfields in your (external by default) methods to access the “hidden” properties which seems a pain and make everything unreadable (but I might be missing something).

Anyway as we said, one step at a time.

@Tamas_Papp

suppose I want to evaluate C++ in a similar manner

Yep, also C++ gets the same treatment, if people get carried away with template meta-programming and it start to become impossible to control the hardware resources at the % level (within the context of the project/team, I’m not claiming that it’s impossible to do it with C++), there is no problem in enforcing a C-only policy for certain parts of the code or whatever is more appropriate.

Thank you again. I’ll try to probe here and there with focused questions as I manage.

[Henrique_Becker]

@gianmariomanca

To give some more context I’m interested in Julia for prototyping algorithms for specialized custom made hardware, typically real-time algorithms.

Not a context in which Julia really shines, I fear. Julia has a garbage collector. You can disable it and try to work without making any allocations (or at least, very controlled allocations), but without a medium-level understanding of the language you will not understand from where some allocations come (this is common with type-unstable code in Julia).

@gianmariomanca

If you are one of the guys that developed the prototype pipeline you will be constantly pulled in to help others understand it, run it, etc etc and it can get easily out of hand.

This seems a very clear case for a language that is very popular, so either people already know how it works or have loads of material to go through.

[gianmariomanca]

@Henrique_Becker

Not a context in which Julia really shines

I’m a bit puzzled by this comment. I think I can maybe clarify my statement: with the prototype we are testing capabilities at the “mathematical” level. The issue with Python is that sometimes it’s too slow to do, let’s say, some heavy Montecarlo testing throwing at it a variety of simulated noise sources. So we will have to do something more limited and leave some of the heavier testing to when the port to C/C++ is already partially underway, at least for some core parts. Julia could allow a better decoupling of these two stages and allow heavy testing much earlier. This should be the perfect scenario for Julia as far I understand it. Doesn’t really matter if there still be a stage where we will have to go to another language anyway because of the unusual hardware. All the prototyping is done or “standard” machines.
Or are you afraid the cost of getting the full performance out of Julia will be too high in a prototyping phase?

@Henrique_Becker

This seems a very clear case for a language that is very popular

You need to start somewhere

[jzr]

There are various meanings of “real time”. In some settings, “real time” means “must not pause for longer than x microseconds”. Julia’s current garbage collector isn’t good at that. Some garbage-collected systems are better at that, like Go or Erlang, and some systems like Rust don’t have garbage collection. These implementations make them more suitable for such “real time” applications.

Using Julia for those situations is harder because you have to avoid allocation altogether and Julia doesn’t have great tools for diagnosing and eliminating allocation.

[gianmariomanca]

Oh, yes. But that’s an issue only for the final implementation on the specialized hardware (cannot be Julia anyway). The prototype pipeline will have no such constraints. For the prototype the only thing that matters is the total amount of data that it can process in a given time. And of course all the usual qualities that you would expect in that case like easy to use, modify, inspect, inject/extract/parse data etc etc

[Henrique_Becker]

jzr understood correctly what I meant. If you think it is worth using Julia until just before the last version, well, okay. Julia is good for prototyping while keeping high performance, so in this point is a good match.

Rust seems a good idea for a realtime system. However, using two languages that are no so popular may be stretching it. I would consider if Rust is a good prototyping language, if the answer is positive maybe it is a better match for all the way to the final product.

[Tamas_Papp]

@gianmariomanca

why it’s an issue to add functions inside structs (the re-compilation issue mentioned by @pdeffebach) for example, and would it still be an issue if you could specify the type of the arguments while defining the struct (some of them referencing the struct you are defining)? etc etc

I think that if you want to implement a C+±style calling convention this way, you are essentially fighting the language. If you really insist on using Julia this way, a trivial macro transforming

@oop value.f(args...)

to

f(value, args...)

would probably get you there in a much more idiomatic way.

[gianmariomanca]

@Tamas_Papp the calling convention in isolation doesn’t mean much.

It’s just I can’t shake the feeling that what happened is:

• Julia allowed the demonstration in the real world of the power of multiple-dispatch
• people got rightly impressed/excited by it and want to popularize it
• in doing so they somehow identified OOP as the “opposite” missing that while multiple dispatch is superior in certain “computational” scenarios, some aspects of classic OOP offer some practical advantages in very different scenarios and the 2 could probably coexist just fine (see primitive vs complex (almost singleton) objects discussion above)
• they say that freedom + informal contracts is just fine in managing large projects (you can trust the user), but seem afraid that adding the extra option in the language to support internal methods and encapsulation (and potentially better discoverability as a bonus) could somehow lead the users to make bad design choices (don’t trust the users)
• in doing so scare away a large fraction of the potential users, actually reducing the popularization of multiple dispatch in the larger world

Just be clear: I’m not saying that’s what’s really going on. There might be extremely valid reasons why the two approaches cannot coexist in the same language (not sure if in an absolute sense, on at this particular stage of Julia development), I just cannot claim to be fully convinced that’s the case at this stage. My opinion will definitely evolve as I learn more.

[stevengj]

@gianmariomanca

adding the extra option in the language to support internal methods and encapsulation

This has nothing to do with OOP per se. It sounds like you just want a namespace with “private” identifiers (e.g. private fields, private methods). This could certainly be implemented in Julia, and has been discussed multiple times — see this document and links therein.

It could even be accomplished to some degree without any language changes, e.g. as demonstrated in the PrivateModules.jl package or in this discussion.

In practice, it seems like none of these proposals have caught on very much, for reasons that have nothing to do with OOP vs. multiple dispatch. When you want to mark something as private and simply not documenting it seems insufficient, people often begin the identifier with an underscore. The compiler still doesn’t enforce anything, but if someone calls MyModule._some_internal_function(...) they are clearly doing so at their own risk, and this could easily be checked for automatically by linting tools.

[Tamas_Papp]

@gianmariomanca

doing so scare away a large fraction of the potential users

Hypothetical users who are scared away by some feature of Julia come up frequently in these kind of discussions, but since we are unable to verify their existence, they don’t serve any useful function other than rhetorical.

@gianmariomanca

seem afraid that adding the extra option in the language to support internal methods and encapsulation (and potentially better discoverability as a bonus) could somehow lead the users to make bad design choices (don’t trust the users)

I am not sure that “you guys are not adding the features I am asking for because you are chicken” is a meaningful characterization of what is going on.

Adding those things to Julia would be a lot of careful design and implementation work which just no one has done yet. Nothing prevents you (or other interested parties) from working out a detailed proposal and maybe a PR. There are some open issues and previous discussion.

[gianmariomanca]

@stevengj thank you for the pointers.

@Tamas_Papp never wanted to call anybody a “chicken”, but the worry of certain type of syntax being source of confusion came up multiple times in the thread, that’s probably why I used the term “afraid”, didn’t mean to refer to or imply anything else.

@Tamas_Papp

Adding those things to Julia would be a lot of careful design and implementation work which just no one has done yet. Nothing prevents you (or other interested parties) from working out a detailed proposal and maybe a PR. There are some open issues and previous discussion.

This is a totally fair point. Can’t argue with that.

@Tamas_Papp

Hypothetical users who are scared away by some feature of Julia come up frequently in these kind of discussions, but since we are unable to verify their existence, they don’t serve any useful function other than rhetorical.

One was me, and a few colleagues (I know, not really a big sample ) , for a few months at least when I was too busy to dig/learn a little more. It was “dumb” of me, but sometimes we do act based of first impressions for lack of energy or for being overstimulated by other issues.

[gianmariomanca]

@stevengj

see this document and links therein.

Thank you, that’s good summary I would say, including the issue that using overloading getproperties doesn’t really work for that purpose at this point without ruining the syntax inside a module,
that the underscore convention in the structs doesn’t seem widespread, etc

I agree that doing that in modules is not really related to OOP per se. Until you want to be able to instantiate multiple copies, as @Henrique_Becker mentioned, but I agree that for the case we are discussing the ability to instantiate multiple copies is less of a concern at this stage.

PS I still like the “academic” question of the possibility of supporting multiple dispatch and private properties and internal methods in structs at the same time in a more idiomatic way (than my example above), even though it might end up being not practical/advisable in the near future.

[jessymilare]

@gianmariomanca

seem afraid that adding the extra option in the language to support internal methods and encapsulation (and potentially better discoverability as a bonus) could somehow lead the users to make bad design choices (don’t trust the users)

I don’t think the problem is that it would “lead the users to make bad design choices”, for “design choice” is a often personal and relative to the task at hand (e.g. IMO, working with databases makes the object.method syntax an advantage over method(object, ...)). I also haven’t seem anyone claiming such a thing (although I might have missed it - it’s a long thread).

As I see, the biggest problems are:

1. Lack of consistency. The object.method syntax is not consistent with current object.property syntax and introduces ambiguities. If someone writes object.property and property is not a property of object, should the compiler search a method with that name? If so, in which namespace - the module that defined the object or the current module? What if the name is both a property and a method?

2. Lack of clarity. People already using Julia (and users of most programming languages) expect object.thing to mean to take thing which belongs do object in some sense.

Note that Nim allows object.method(...) to be an alias for method(object, ...), which works because that is the only possible meaning of that syntax (e.g. object.field is translated to field(object)).

In Julia, however, introducing that syntax would create two different meanings for the same syntax, so consistency and clarity would be much harder, if not impossible, to achieve.