Understood. But exception handling and fallbacks are still possible, right?
As we are diverging from . already – then it’s not a big leap to go from a..b() to b(a) 
This specific .. syntax won’t work either, because .. is a commonly used operator already, and it parses as:
julia> Meta.@lower 1..2()
:($(Expr(:thunk, CodeInfo(
@ none within `top-level scope`
1 ─ %1 = Core.tuple()
│ %2 = 2 * %1
│ %3 = 1 .. %2
└── return %3
))))
Still, using a.<tab> for function autocomplete is a reasonable idea, you could implement it without any changes to Julia-the-language! Sounds like a nice first step to explore the usefulness of this syntax in practice.
1 Like
I agree, but the aim is to be able to do something in the vein of: obj.meth1(x).meth2(y).meth3(z), which I prefer much more than meth3(meth2(meth1(x),y),z). Any other (non-clashing) operator would suit too.
What’s important in the dot notation is that one has to ensure this won’t clash with other already established practices and notations in Julia. But other symbol would work too (e.g; ::, ->, .., etc.), provided they don’t conflict either with current syntaxes. So no problem to diverge from it.
What makes the object.method notation convenient is to work in conjunction with code completion, for calling a method after a variable allows providing some context to the langage server (or equivalent).
For example, doing object<tab> as you proposed is quite fine, but for just exploration. Once you’ve found the method that suits you (and the variable), you have to go back before the variable to type it. And the more you chain methods, to weirder it gets.
One could assumes doing object<tab>, select a method, type <enter>, then the LS replaces object with method(object [notice the open parens]. But still, very weird, and weirder with method chaining.
So any operator that would allow to do both (exploration and chaining) is much better (in my view) than both separately.
Interesting. Could you explain a use case for this notation? I’m afraid this is the first time I see it.
FWIW, there’s already a way to find possible method matches in the REPL given a set of arguments:
help?> ?(Ref(1)) # [TAB]
ndims(x::Ref) @ Base refpointer.jl:95 fill(v, dims::Union{Integer, AbstractUnitRange}...) @ Base array.jl:582
hcat(X::T...) where T @ Base abstractarray.jl:1615 broadcast(f, x::Number...) @ Base.Broadcast broadcast.jl:844
HTML(content::T) where T @ Base.Docs docs/utils.jl:30 vcat(X::T...) where T @ Base abstractarray.jl:1613
isempty(x::Ref) @ Base refpointer.jl:94 length(x::Ref) @ Base refpointer.jl:93
error(s::Vararg{Any, N}) where N @ Base error.jl:42 iterate(r::Ref) @ Base refpointer.jl:97
widen(x::T) where T @ Base operators.jl:891 bitstring(x::T) where T @ Base intfuncs.jl:918
VecElement(arg::T) where T @ Core boot.jl:408 Some(value::T) where T @ Base some.jl:12
size(x::Ref) @ Base refpointer.jl:91 Tuple(x::Ref) @ Base tuple.jl:388
replace!(A, old_new::Pair...; count) @ Base set.jl:606 isassigned(x::Base.RefValue) @ Base refvalue.jl:36
getindex(b::Base.RefValue) @ Base refvalue.jl:59 replace(A, old_new::Pair...; count) @ Base set.jl:686
only(x::Ref) @ Base.Iterators iterators.jl:1533 Text(content::T) where T @ Base.Docs docs/utils.jl:88
oneunit(x::T) where T @ Base number.jl:371 axes(x::Ref) @ Base refpointer.jl:92
help?> ?(Ref(1), 1) # [TAB]
mapreduce(f, op, A::Union{Base.AbstractBroadcasted, AbstractArray}...; kw...) @ Base reducedim.jl:359
fill(v, dims::Union{Integer, AbstractUnitRange}...) @ Base array.jl:582
broadcast(f, x::Number...) @ Base.Broadcast broadcast.jl:844
rand(X, d::Integer, dims::Integer...) @ Random /mnt/fastdisk/brianc_home/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/Random/src/Random.jl:285
last(itr, n::Integer) @ Base abstractarray.jl:552
edit(f, idx::Integer) @ InteractiveUtils ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/InteractiveUtils/src/editless.jl:271
error(s::Vararg{Any, N}) where N @ Base error.jl:42
iterate(r::Ref, s) @ Base refpointer.jl:98
BigFloat(x, prec::Int64) @ Base deprecated.jl:103
map(f, x::Number, ys::Number...) @ Base number.jl:284
broadcast!(f::Tf, dest, As::Vararg{Any, N}) where {Tf, N} @ Base.Broadcast broadcast.jl:880
ntuple(f::F, n::Integer) where F @ Base ntuple.jl:17
timedwait(testcb, timeout::Real; pollint) @ Base asyncevent.jl:336
setindex!(b::Base.RefValue, x) @ Base refvalue.jl:60
rpad(s, n::Integer) @ Base strings/util.jl:485
first(itr, n::Integer) @ Base abstractarray.jl:502
reduce(op, a::Number) @ Base reduce.jl:492
lpad(s, n::Integer) @ Base strings/util.jl:455
Task(f, reserved_stack::Int64) @ Base task.jl:5
Ref(x::Ref, i::Integer) @ Base refpointer.jl:141
less(file, line::Integer) @ InteractiveUtils ~/.julia/juliaup/julia-1.10.5+0.x64.linux.gnu/share/julia/stdlib/v1.10/InteractiveUtils/src/editless.jl:314
One limitation of this approach is that a lot of Julia functions do not constrain their argument types (see all the args without ::SomeType or that are ::T ... where T), which means the list will have false positives. However, this is a problem independent of syntax (including a hypothetical ..) and more about language semantics.
The other limitation is that the documentation for this feature is buried halfway down The Julia REPL · The Julia Language. I even knew it existed and had to go searching for a couple of minutes to find the official docs! Not sure how to make that info more obvious and accessible, but it would be a good idea.
4 Likes
My two cents are probably already well covered in those Github issues, but here they are anyway.
This isn’t the first time someone wanted the streamed code completion in OOP languages, not even the only time in the last week. But it’s worth looking at what those languages actually do to pull this off, and they don’t all do the same things.
In most OOP languages, object.method(...) calls a member method, which has been thought of as “message” between instances of classes. If you want to discover an object’s possible messages, you only need to search its class. In OOP languages like Java where all methods are encapsulated by classes, tab completion can reach everything, though the leading object is not necessarily the input you’re interested in.
Other OOP languages however have functions outside classes, and typical tab-completion cannot get those from objects at all because of the different syntax. Functions outside a class or the class’s module are very important for extending behavior; while inheritance+overriding and dynamic languages’ monkeypatching can do that within classes, it often traps code in limitations and boilerplate that neither developers nor users want to deal with. Some domains like scientific computing, which Julia started in, use functions much more than member methods, even implementing functions to forward to member methods so users won’t need to occasionally switch to member method syntax.
Like Lisp’s CLOS (also OOP), Julia encapsulates methods in multiply dispatched generic functions instead of types; for brevity, I won’t explain how multiple dispatch was found to mix poorly with class encapsulation in several language’s approaches. So, objects have no member methods to find, only fields and what may be inferred from getproperty. If property syntax were to lower to function calls (a breaking change) to emulate member method syntax, tab-completion might try to find methods that can take our object as a first argument. Seems reasonable right? Let’s check help mode for the single-argument case:
julia> struct X end # totally new type, no methods
help?> ?(X(),)
broadcast(f, x::Number...) @ Base.Broadcast broadcast.jl:844
oneunit(x::T) where T @ Base number.jl:371
Text(content::T) where T @ Base.Docs docs\utils.jl:88
widen(x::T) where T @ Base operators.jl:891
error(s::Vararg{Any, N}) where N @ Base error.jl:42
VecElement(arg::T) where T @ Core boot.jl:408
replace(A, old_new::Pair...; count) @ Base set.jl:686
fill(v, dims::Union{Integer, AbstractUnitRange}...) @ Base array.jl:582
vcat(X::T...) where T @ Base abstractarray.jl:1613
replace!(A, old_new::Pair...; count) @ Base set.jl:606
HTML(content::T) where T @ Base.Docs docs\utils.jl:30
bitstring(x::T) where T @ Base intfuncs.jl:918
Some(value::T) where T @ Base some.jl:12
hcat(X::T...) where T @ Base abstractarray.jl:1615
Wow, we already have some hits in Base even without implementing anything on our own! Let’s try one (pun unintended):
julia> oneunit(X())
ERROR: MethodError: no method matching one(::X)
Closest candidates are:
one(::Type{Union{}}, Any...)
@ Base number.jl:349
one(::Type{Missing})
@ Base missing.jl:104
one(::Missing)
@ Base missing.jl:101
...
Stacktrace:
[1] oneunit(x::X)
@ Base .\number.jl:371
While we would dispatch to the oneunit method, we hit a MethodError on the callee one. In fact most of the printed methods will hit a downstream error, and it’s no surprise because we implemented nothing for X. We don’t actually want to find all the methods that can merely take our object as an argument, we want methods that work on our object; I don’t want to see a method for iterables until after I implement Base.iterate for my type. The reason why this feature isn’t mentioned much is simple; false positives are just not useful.
Barring extreme incompetence, member methods are indeed designed to work on their classes’ instances, so they’re worth discovering. However, discovering other useful functions is still an active area of development; typically we still read library documentation. For Julia in particular, formalizing interfaces could help, especially in the case of iterables.
17 Likes
How would you write dot notation for functions that take the “primary object” as the 2nd argument, as is common in Julia data manipulation?
1 Like
I wonder how useful those suggestions would be if JET filtered out methods that are definitely going to fail.
1 Like
That could be very helpful for reflection, but the downside is the latency of type inference for each method on top of an already very involved search. Searching member methods in an inferred class is a far quicker process that doesn’t check for such errors, fast enough for a real-time feel in code discovery and completion. As I explained, searching for useful functions is not the same thing and is necessarily more complicated, but the differences are relevant for an inspired feature’s feasibility and outcomes. My gut feeling is it would involve restricting searches to specified modules.
When you filter out everything that isn’t an exact match to the type, I think it would work. You would not get any false positives. Yes I know we wouldn’t see all the methods available, but that’s the case already. It just allows me or any other developer to give a nice experience to users if I explicitly call out the type in my methods.
To continue a little of what you said in the other thread, I don’t really see what you mean by
I guess to me there is no option other than documentation currently, so it just gives people the option to do both if they want.
What does make us freer mean exactly. What is being restricted here by allowing users to get a bit more help if they explicitly call out types in methods?
1 Like
That’s the default setting in methodswith, and it’s often almost worthless because of how common dispatching on supertypes are; imagine how much code we have to duplicate if we only had methods with concrete type annotations. methodswith has a setting for supertypes, but it excludes Any to reduce false positives, which is also an issue when types and their behaviors don’t fit into a type hierarchy e.g. iterables and Holy traits. We would often at least get the interface methods, but those are also often buried in other methods. This level of discovery is fundamentally less clean than limiting ourselves to class-encapsulated methods.
In that context, there was a suggestion that we heavily adopt syntax resembling member method chaining, Pipe.jl in particular, to enable tab completion. It’s not an unreasonable idea; if we want code completion to suggest methods for an input object, then the object has to be written first. However, people planning to write function call syntax should also be able to find the same methods as someone pressing TAB after |>, albeit not by completing an expression. If we must refactor our code and write in a style antithetical to the language to use a feature, then it’s not a language feature.
Speaking of pipelines, it superficially resembles member method chaining @pipe obj |> meth1(_,x) |> meth2(_,y) |> meth3(_,z), so people often naively consider it equivalent (I remember one R-to-Python convert saying Python does . where R does |>) and draw too many parallels. But in languages like R that heavily pipe to functions outside of classes, code completion still needs you to type a few characters of the function name to come up with anything useful. Again, class encapsulation matters.
I think people suggesting “obvious” features should consider more often that the developers aren’t ignorant and likely have tried or are working on the same ideas. Spotting nice things and superficial resemblances are nothing compared to actually comparing the languages’ underlying designs and running code to check how well an idea works in practice. Barget’s post is actually a great example of acknowledging ignorance and being open to learning the obstacles to implementing nice features.
2 Likes
Yes. I’ ve used for a year. You can follow Managing Extensions in Visual Studio Code , and disable auto updating of this extension.
Yeah I definitely see how that can be a big pain. Do you think there is an option like:
@lsp ::MyType = [foo, bar, baz]
where this would tell lsp to display those functions in the dropdown? If there are multiple methods and some of them don’t work you can leave that function out or be more specific and write the method/s.
I think the option there is the ?(MyType) and you could decide which option to use for your use case?
I think this really hits the nail the head. Whatever other issues it may have, OOP does a really good job listing the methods you can apply to each object. That’s an issue I have with the approach of just making obj.fun(args…) synonymous with fun(obj, args…) — it doesn’t do anything about namespacing fun, which is one of the main benefits of the OOP syntax. One idea that we kicked around pre-1.0 was to make obj.fun(args…) mean moduleof(obj).fun(obj, args…) or something like that. But this still isn’t great because there can be both functions in thst module that seem like they’d work but don’t as well as there being functions elsewhere that would work that can’t be found in obj’s module.
A slightly different issue is that we haven’t designed argument order to make fun(obj, args…) a particularly useful ordering. The biggest culprits here are classical functional operations like map and filter.
8 Likes
The namespacing is not just about discoverability. It also avoids name collisions. An example I’ve brought up before from quantum computing is something like circ.x(0) in Python. You have methods x, y, z, h, and on and on, and people really like calling them without qualifying the names or worrying about collisions.
If I have another variable x in scope I have to do something like this:
module Amod
struct A
d
end
x(a::A) = a.d
end # module Amod
using .Amod: A
a = A(1)
x = 3
Amod.x(a)
I wrote a package a couple of years ago to try to make this better CBOOCall.jl. (I really want to rename this package to something less terrible, but have not gotten around to it). The above example would be:
module Amod
using CBOOCall: @cbooify
struct A
d
end
x(a::A) = a.d
@cbooify A (x,)
end # module Amod
using .Amod: A
a = A(1)
x = 3
a.x()
It’s not universal. You have to explicitly enroll a function (If I recall, it works on functions, not methods). But in Python, in order to use the dot, the method has to be defined in a class (or registered in a dict). In Rust the function has to be defined in an impl block.
Because the function is enrolled via a macro, you can have syntax to choose the argument order. I did add some features in addition to the one above. But I’m pretty sure I did not allow choosing which is the privileged argument.
4 Likes
I probably won’t ever do that specifically because there would be a distinct area of text I have to keep in sync with my working code, that makes maintenance much harder; you have to do this manually in OOP, but it’s part of implementing the class so there’s no separation.
However you do bring up a very crucial point here; we’re not aiming to list every applicable method, we want tips on using a type or set of types. We don’t expect a comprehensive list, in fact we can’t wait long between keystrokes; taking our time is for reading documentation and source code.
- the one
foo(::MyType, ::AbstractArray)method would suffice, and the dozens of more specific methods would needlessly drown out other functions. I think you were right to prioritize::MyTypebecause it’s more of a guarantee the method was designed for it; supertypes risk false positives. - we usually don’t want to see the internal functions
__foo - we’re not trying to compile hundreds of methods to unlist things that cause a
MethodError,DomainError, etc, in fact we might still want to seeAbstractArraymethods even if our subtype didn’t implement the interface methods yet.
On some level, these take a manual specification of what the important methods are. We do have type annotations, export, docstrings, and the upcoming public, but we’ll probably still need something to deal with interfaces. Even with perfect method display, large packages are an issue because of the sheer number of functions implemented for types; check ?([1]) for >2 pages of methods, many of which are recognizable. The same problem would occur for classes with many member methods.
The documentation of your types (to the extent that they are public and have a non-trivial API) should include a full list of methods anyway. See, for example, the docstring of DocInventories.Inventory. It wouldn’t be terrible to have a solution to make this list of methods “machine readable”, so that LSP/the REPL can access it.
It would certainly be nice of have tooling in my editor and REPL so that I can get a properly filtered (!) list of methodswith for a given type or object. The trick is filtering out the “false positives” of irrelevant methods. I don’t see that there’s any way around manually specifying “relevant” method that should be prioritized.
On the other hand, I often find that when people ask for .<tab> completion of methods, “like in OOP”, they haven’t internalized that Julia (as a non-OOP language) is centered around functions, not objects. And we have .<tab> completion for modules, which is a good organization of functions. If you follow the guidelines that ackages that provide most of their functionality in association with a new type should have pluralized names, so that DataFrames provides the DataFrame type, then a good way to interactively explore the methods of a DataFrame should be to type DataFrames.<tab>. It would be helpful if those tab completions (by default, and going forward) would be filtered to @public symbols. In any case, it’s important to design and document package with this kind of API exploration in mind.
2 Likes
Yes, the “Julia 2.0” thread was not focused. So this was to address this point specifically.
There’s definitely something that I’m missing here. It the property syntax is replaced by a more comprehensive search, with getproperty as first priority, this wouldn’t break breaking, no? The internal would be different, for sure (internals can break). But I don’t see how any existing package that currently uses getproperty expects it to fail. So any packages using getproperty should still work. Something is the vein of (very crudely, with mistakes):
function getcompletion(...)
try
prop = getproperty(...)
catch
funcs = getfuncts(...)
end
...
end
I wouldn’t unfortunately. (see just after)
Indeed, that would be not great. As I see it, introducing a new, non-breaking notation that is not following a pre-existing convention is not exactly an issue. This is not perfect, but trying to satisfy all pre-existing conventions is sometimes impossible, to the point of preventing innovations. So a decision is needed.
Fully agree. Relying solely on the documentation is sometimes (often?) painful, and very dependent of the package’s state.
I don’t exactly understand. That there will be a nudge toward the new notation, sure. But for the rest, nothing will change. So no change, no (new) pain. And those who prefer using |> wouldn’t notice the difference.
Conversely, if, as a result, everybody is now using the new notation, then this would show the usefulness and convenience of the new notation.
Heck, as I see it, this would be more an LSP problem than a language’s. Why not display the same list as after a dot, when pressing <tab>?
Exactly, this worth being stated explicitly. We don’t expect any random function from any module with no type to be listed, otherwise this would be a mess.
Proposition:
- First, typing
object.=> shows list of functions using<:ObjectSuperTypeas first argument BUT only their name (in ordre to remove duplicates) - Then, after selecting the function + parens, typing
<Tab>=> shows sublist of all available methods
Rationale (for tab-completion):
- Clear case 1: If the object has its type|supertype a first argument of the function, should definitely be supported. In all available methods (i.e. imported by either implicitly with
usingor explicitly withimport), from any module. - Clear case 2 (opposite): Given an object, and a function that can accept it. If the object’s type is not in a method’s signature, nor they are in the same module, it’s not possible to know whether the combination works apart from trying and see => no formal link between both => possibility not supported
There’s also a less clearer case, to see whether it’s worth including it’s result in the list?
- Intermediate case 1: if object & function are in the same module, and the function have at least one argument but without a specified type, the function can be listed … but only after clear case 1’s list
1 Like
I like it, but a big reason is that it’s NOT a full list of methods. Obviously it won’t have any methods defined in other packages, but it also doesn’t list all the methods in its own file. It lists the package’s fundamental usage for that type, and that designation should be at the top of any list.
I don’t either, but wider usage-oriented reflection shouldn’t and can’t require explicit lists per type. When I ask “what can I do with this type”, it’s not actually 1 question, I could mean “what does the package’s developer design this type for e.g. Inventory’s docstring” or “what methods in this other package could I try this type in?” I don’t expect dependents to be able or willing to take the maintenance burden of extending Inventory’s list of important methods, and I don’t want to see other package’s methods mixed in any more than people would want functions outside classes to be listed among member methods.
I think I mentioned this before, but there are manual specifications that can make discovery better. methodswith conditionally uses names, though we annoyingly can’t control it directly; the upcoming public keyword could be involved. methodswith ignores Any arguments, but that could be relaxed in a type’s parentmodule by default and involve other user-specified packages.
There’s actually a few different implementations for this described in this thread so far so it’s probably worth writing out examples. First I’ll start with the most stringent, something akin to how Python lowers member method syntax:
obj.meth(args...)
# unconditionally lowers to
meth(obj, args...)
This breaks:
struct ComplexFunctions
real::Function
imag::Function
end
Base.real(c::ComplexFunctions) = c.real
c = ComplexFunctions(abs, -)
c.real(1.2) # v1 abs(1.2), v2 real(c, 1.2)
map(c.real, [1]) # map(abs, [1])
The discrepancy between c.real(...) and c.real gets awful when class encapsulation isn’t around. Sure, c.real could be lowered to real, but remember that modules encapsulate functions. It’s not great to have Base.real and MyPackage.real around and write c.real where the c is actually meaningless.
And that takes us to your last example: we prioritize the type’s properties and fall back to the function call lowering, which makes the MyComplex example behave as it does currently. Now it breaks:
try
c.println("hello") # v2
catch
println("whoops, no println property") # v1
end
The absence of a behavior can be important! Type piracy is considered a terrible practice, and it too is about adding things that weren’t intended to be there.
But honestly this is not nearly as bad as how the property-dependent behavior breaks code within its own major version:
myreal(x) = x.real() # classic duck-typing
myreal(1+2im) # v1 errors, v2 real(1+2im)==1
c2 = ComplexFunctions(zeros, ones)
myreal(c2) # v1 and v2 c2.real()==zeros()==fill(0.0)
#= I forget about this code or someone else wrote this
code in the first place, then I change ComplexFunctions
fields to re and im for brevity =#
myreal(c2) # v1 errors, v2 real(c2)==zeros
While v1’s error indicates exactly why the type’s edit broke the code, v2 would silently switch to another behavior and result, causing opaque problems downstream. This is made even more dangerous in a language where struct fields tend to be internal details that should change without affecting the API. Bad things happen when property syntax is only sometimes used for properties.
This is reasonable. It’s also not what the person in the original thread was suggesting, which was understandably unclear in the string of quotes in this thread.
2 Likes