Allowing the object.method(args...) syntax as an alias for method(object, args ...)

The idea is to support chaining:

rather than

eat(serve(cook(scramble(crack(egg), bowl), :high), :hot))

which is hard to read,

egg.crack().scramble(bowl).cook(:high).serve(:hot).eat()

is easier. The pipeline operator x |> f doesn’t take arguments (at least until the f(_, y) proposal lands) and regardless is substantially longer without an increase in clarity compared to x.f(y) or x⤷f(y) or whatever.

3 Likes

I think that most gains from one-liners are imaginary, so I would go for something like

dish = cook(scramble(crack(egg), bowl), :high)
eat(serve(dish, :hot))

then if necessary I can refactor the last part as

suggested_temperature(_) = :hot
suggested_temperature(::Revenge) = :cold
eat(serve(dish, suggested_temperature(dish)))
6 Likes
serve(dish) = serve(dish, suggested_temperature(dish))
1 Like

@Henrique_Becker , I get your beliefs and preferences, but that doesn’t really move the discussion forward without some concrete examples. What’s the current best approximation already present in the language?

For example, can we use modules as proxy for complex objects that needs to maintain a state from one call to the other? Can I declare a module that represents a block in the pipeline and export only a subset of the variables and functions?

That’s how you can achieve a similar functionality in C using files/static variables. Just put each block in a separate .c file and declare some static variable in that .c file to maintain the “private” state. Would modules allow something similar? Or some other construct?

EDIT:
PS I believe, after looking at the documentation, that modules cannot hide variables/methods, correct me if I’m wrong.
If we put the code of the original question in module called temporalFiltering it would probably alleviate the discoverability issue I’m guessing. But I’m not sure if using modules as objects would be considered good practice.

Best approximation to what? OOP? My point is that one should not come from an OOP way-of-thinking, or an OOP solution, and try to port it to Julia by mapping concepts. Instead, one should understand what is the Julia way-of-thinking and start from the beginning, this is, from the problem, and not from an already existing structure created in another paradigm.

uhhhh, you can do it for singletons, because you can have methods and variables inside a module, but you cannot create multiple instances of a module. What is the problem you are trying to solve here, why do not use struct to create complex objects? It seems very weird to me to try to use modules as objects.

Okay. The idea of using Julia modules is similar to this idea of using .c files with static variables. Here you also cannot instance multiple objects/copies of each “block”/file. However, again, I really need to understand better what is the problem you are trying to solve. The problem is hiding things from the users? (i.e., making variables/fields private?) Julia will not help you in this endeavor. You can work against the language with some success, like redefining the getproperty and getpropertynames of a struct to hide its fields, but getfield will always be able to recover them.

To simplify for the sake of conversation that problem that I’m trying to solve is: if I have to start a project tomorrow within a large company, with a team of 100 people, all highly skilled but busy and with very different backgrounds/interests: quality engineers, electrical engineers, ML researchers, software engineers, some physicist, some mathematicians (some with a background in Python, some Matlab, some C, some C++) I want a convincing story of why using Julia would be a good idea and have answers for the obvious questions coming from those people. Nobody will want to start a project and realize after 6 months that the code is a unmanageable mess, and now we are stuck with it for another 18 months, etc etc. Or that I have to babysit 100 people everyday because the learning curve is too steep, etc, etc.

It’s a very “practical” problem of how a group of people wants to invest time/resources and evaluate risk, etc.

Within that “context” I was then trying to build a mental model of the current state of the language, community, etc.

This process prompted the original question of this thread and then of Regarding `getproperty` and public vs. private APIs. I guess similar in spirit to What don't you like about Julia for "serious work"? .

Going back to the specific issue, I guess yes, it’s a combination of encapsulation and auto-completion, for the case where there is a very specialized set of complex objects/blocks. The question about the modules come from the idea that I could sacrifice the ability of having multiple instantiations, if I can have the other 2 properties right now, while we wait for the language to possibly evolve, etc, etc…

PS In the same spirit I was of course looking at the status of the debugger, VS code integration, Pluto vs Jupyter, plotting, loading dynamic libraries in concrete examples, ability of deploying Julia “jobs” on clusters where Julia is not installed, ability to use GPUs, etc, etc

If you are starting such a project tomorrow, don’t use Julia. If you can’t convince them not to use oop, use oop.

This is like asking for a good way to use the handle of an electric screwdriver as a hammer, because your handymen are more used to hammers and nails than screws.

11 Likes

100 people working on a monolithic application will be a mess in any language. OOP encapsulation is of little help here.

I would first try to divide the application in at least 10 separate packages (or even microservices), each with its own unit tests and specified (public) API.
For example SciML consists of a large number of independent packages designed to work together.

4 Likes

@gustaphe ok, at least your analogy is funny. But some will hear: why do you want to have any safety protocol at all? that’s old fashion and it will slow you down, I’m sure none of the handyman will accidentally drill a few holes in the load bearing beam killing everyone else, no need to hide it and put a locked gate around it.

@lungben yes of course, that is exactly what it would look like. I was just wondering what are the facilities in the language that would facilitate the splitting, accompanied with some “enforcement” tools and some “discoverability” tools (like the tab completion that completes only to states and methods that are part of the public API)

The secret to a good analogy is to not make obvious stretches.

In what way is the obj.f syntax like a safety feature? How is a team member going to cause catastrophic failure by not understanding the julian paradigm?

The issue here is you have a team who are used to object oriented programming and not enough time to convince them to abandon it: then Julia simply isn’t the right tool for your team. Because it’s not object oriented and it shouldn’t be changed to accommodate object oriented programming.

6 Likes

OP is referring to the fact that there is no way to mark a field as “private” in Julia.

This is true, but there are two solutions

  1. Overload getproperty so that a.x only exposes the stuff you want to expose
  2. Create a convention of using functions to access internal objects, i.e. a(x). This means you only allow functions you want. It has the added benefit of making documentation easier.

@pdeffebach yes overloading getproperty seems interesting, but it leaves out methods, or maybe not …

I don’t want to scare anybody with the above example :slight_smile:
(see the tab completion in Pluto in the image)

but if you can add self-referencing functions to the instance of a struct, why can’t you have it done automatically at the struct definition level?

That last point seems like a separate issue than getproperty.

As others have said, you are probably overusing OOP design patterns in Julia. Because functions each have their own type, it’s not very useful to put functions inside of structs like that. It will force recompilation of functions for each new IIRFilter encountered.

As others have said, if you really can’t leave OOP, it’s okay not to use Julia. There are other idioms that are used to manage large projects.

Finally, w.r.t. tab completion. Developers are aware that it’s hard to do tab completion on methods. There is an open issue for that on github.

2 Likes

OK, now we are talking :slight_smile: can you expand a bit? or point me to a page to understand this a bit better?
Notice that in this example it will be very rare to instantiate the same object multiple times (maybe for debugging or comparing alternative configurations once in a while).

Okay, now your questions makes more sense.

About the convincing stories, ask for them. I am almost sure there was already a thread about this in the past. I would cite JuMP.jl as a case of success, even if I have some criticism about some of their choices. JuMP got a lot of researchers in my area (Mathematical modeling and optimization) to use it, so they could preprocess the data and build the formulation with high performance and a high-level general purpose language and without hardcoding any specific solver.

For the answers to obvious answers, I recommend both reading the language manual (docs.julialang.org/) and asking the specific questions here. For example, for your current question you got the answer this syntax is not supported nor there are plans for supporting it. I commend your determination to understand the reasons behind it all, but it can be a lot to do this for every question they may have, and sometimes it kinda seems like you want suggest changes to the “problems” you see in the language just to be able to recommend it. This does not seem a good bias to have, if syntax details like object.method are relevant for your team or project (I surely do not understand how if they are highly skilled) then maybe Julia is not the right language and you should re-evaluate your options. If your priority is immediate productivity and it is best achieved by letting a large team to work with the most common paradigm and languages (which they are all accustomed with), then do it.

This is another thing that is hard to rush, but in general: the language is stable now (post version 1.0 there may be improvements but very little breakage if any), the community is active like you can see here in Discourse, the language creators are active, transparent, and check the forums; the packages are somewhat mature but it depends on the specific domain. Like I mentioned before, JuMP.jl+Julia is so good that made researchers jump (no pun intended) from other languages to Julia, but there may be specific domains where Julia does not shine so much (it can call C, Fortran, or even Python in such cases). Obviously, the community and ecosystem are not so big like Java, Python, etc…

In Julia, many things are solved by informal contracts. Formal contracts many times go against the dynamic way of the language. For private fields and encapsulation, what the community do in general is to follow this rule of thumb: if it is not documented then it does not exist. Something is a part of the public API only if it is documented (and the documentation does not say otherwise). If fields are to be private, then they should not be documented using using the usual strings above them (which allow Documenter.jl to create HTML pages and querying the documentation with ? in the REPL). Obviously, private fields and methods can yet be documented with comments; or you can use Documenter but add a box/note “Internal use.” in the private methods/fields documentation. People are also trying to start a convention of starting such methods/fields with underscore. I like the idea for private methods, but most structs are composed only of “private” fields (this is, you are only expected to call methods over them) so this seems a little obnoxious/ugly for these private-only structs.

Finally, about guarding rails, I am all about them when they can prevent common mistakes coming from lack of attention or something counterintuitive. However, using an undocumented field of an structure is a conscious decision that a highly-skilled person can easily understand when they should and should not do it. I have already done so for quick fixes, when a package missed some feature, and it was easier to freeze that dependency and break the encapsulation than to try to work around without breaking it. This is the exception that you should be able to do, and avoiding it becomes the rule should be the work of QA and, indirectly, of the HR (by contracting good professionals). Not a problem the language should be solving.

2 Likes

I understand the predicament, but you will have to dive into the language and make up your own mind. Julia is not special in this respect; all modern computer languages with advanced features are like this.

For example, suppose I want to evaluate C++ in a similar manner. I could talk to seasoned C++ programmers, who would tell me what they like and don’t like about the language, but eventually I have to consider the fact that they are using C++, so they must like it enough to live with the things they don’t like about it. For my own purposes, I will have to make my own judgement.

That said, the thought experiment

is not something a smart company would ever do. If the project runs for a few years (which is optimistic for software), with 100 people you are close to 1 million person-hours. Multiply it by the hourly wage of those highly skilled people, and reflect on the fact that you want a language chosen by …tomorrow?

Managing 100 participants is a challenge in itself. Julia (the language) is the only project I know of which has this many contributors, but very few of them do it even close to a full-time job. This is uncharted territory at the moment for Julia.

What could make sense for a company considering Julia is to start a pilot project with around 3-10 people interested in the language. In fact, many companies (or groups of academics) are doing or have done something like this. Then see how it goes, and take it from there.

(That said, I would be surprised if the lack or presence of an obj.method syntax had any role in the success of such a project.)

8 Likes

@Henrique_Becker , @Tamas_Papp thank you for the thoughtful answers.

To give some more context I’m interested in Julia for prototyping algorithms for specialized custom made hardware, typically real-time algorithms. In this context Julia doesn’t solve completely the 2 languages problem because there is no Julia compiler for that hardware, but it can “partially” solve it in the sense that even the prototype pipeline will be subject to extensive testing and the goal is to speed up this testing stage (tons of data, clusters involved, etc…). When everybody is happy then it will be eventually converted to C/C++ or whatever language/compiler the hardware supports.

What happens is that someone has an idea, develops a prototype initially with 3 to 10 people as you said and if the idea is successful the team start expanding, with highly skilled people yes, but skilled in their own field. And they need to be able to be pulled in the project, use the prototype pipeline as reference, and perform all kind of sophisticated test/measurements in the matter of days.
If you are one of the guys that developed the prototype pipeline you will be constantly pulled in to help others understand it, run it, etc etc and it can get easily out of hand.
So nobody will stop you from using Julia if you want to, but then you have to live with your choice.

So I definitely want to take advantage of the language myself and learn it better and I’ll keep asking questions as needed, but in the back of my mind when I ask a question there will also be our “thought experiment”.

You bet, you don’t go into theoretical physics if you don’t want to get to the bottom of things :slight_smile: I’m used to environments where we pound on things until either they turn into a diamond or they turn into dust.

This current thread was one of the possible questions that I though could help me focus the conversation, help me learn and expose some biases, unmotivated assumptions either in myself or in others.

Well it is already there apparently if you look at my example :slight_smile: Just not very conveniently integrated in the language.
Joking aside, I’m still interested in understanding why it’s an issue to add functions inside structs (the re-compilation issue mentioned by @pdeffebach ) for example, and would it still be an issue if you could specify the type of the arguments while defining the struct (some of them referencing the struct you are defining)? etc etc
I didn’t have the cycles now to open multiple threads now, but maybe little by little.

Same with getproperties and getfields. Not fully clear how you would use it to make some some properties private since there are no internal methods, if you hide some properties overloading getproperties then you need to use getfields in your (external by default) methods to access the “hidden” properties which seems a pain and make everything unreadable (but I might be missing something).

Anyway as we said, one step at a time.

Yep, also C++ gets the same treatment, if people get carried away with template meta-programming and it start to become impossible to control the hardware resources at the % level (within the context of the project/team, I’m not claiming that it’s impossible to do it with C++), there is no problem in enforcing a C-only policy for certain parts of the code or whatever is more appropriate.

Thank you again. I’ll try to probe here and there with focused questions as I manage.

1 Like

Not a context in which Julia really shines, I fear. Julia has a garbage collector. You can disable it and try to work without making any allocations (or at least, very controlled allocations), but without a medium-level understanding of the language you will not understand from where some allocations come (this is common with type-unstable code in Julia).

This seems a very clear case for a language that is very popular, so either people already know how it works or have loads of material to go through.

1 Like

I’m a bit puzzled by this comment. I think I can maybe clarify my statement: with the prototype we are testing capabilities at the “mathematical” level. The issue with Python is that sometimes it’s too slow to do, let’s say, some heavy Montecarlo testing throwing at it a variety of simulated noise sources. So we will have to do something more limited and leave some of the heavier testing to when the port to C/C++ is already partially underway, at least for some core parts. Julia could allow a better decoupling of these two stages and allow heavy testing much earlier. This should be the perfect scenario for Julia as far I understand it. Doesn’t really matter if there still be a stage where we will have to go to another language anyway because of the unusual hardware. All the prototyping is done or “standard” machines.
Or are you afraid the cost of getting the full performance out of Julia will be too high in a prototyping phase?

You need to start somewhere :slight_smile:

1 Like

There are various meanings of “real time”. In some settings, “real time” means “must not pause for longer than x microseconds”. Julia’s current garbage collector isn’t good at that. Some garbage-collected systems are better at that, like Go or Erlang, and some systems like Rust don’t have garbage collection. These implementations make them more suitable for such “real time” applications.

Using Julia for those situations is harder because you have to avoid allocation altogether and Julia doesn’t have great tools for diagnosing and eliminating allocation.

2 Likes