Except for macros, doesn’t the rest is the same for the documented vs. not documented convention?
No, clearly.
We can’t, but the doc string for all the non-API would be just a warning, while the API one would appear. This is actually way better than the current situation in these cases, where if a package exposes one method and one types ?foo may even have a hard time finding the actual doc string for the exported method.
I’m not sure if a partially @api function is a good idea, seems like an issue when dispatched in a caller method. In a contrived minimal example:
@api f(x) = g(x) + 1
@api g(x::Int) = 2*x
g(x::Float64) = x/2
f(1) # API all the way
f(1.5) # uses an unstable non-API method
f(x::Any) is a single public API method whose behavior is version-unstable depending on the input type. f(x::Any)'s docstring could point out it depends on g and list the “API-stable” input types, or the method could be edited to only accept such types (x::Integer suffices in that example). But in practical examples with more partially @api callees and longer call chains, those are next to impossible. f(::Any) only remains stable if it changes whenever g(::Float64) changes, which is impractical if they are maintained in separate packages. It’s only manageable when a whole function is either all @api or all internal.
The excessive docstrings printout for a function in help mode when you wanted a specific method’s should be solved another way. Argument types or their instances in a function call syntax already can narrow it down to one method, though it’d be nice if nothing is printed for a MethodError. A really neat feature would be to only list methods defined in a specified module, which often won’t be the same as the parentmodule of the whole function.
I think the right level of granularity is symbols, so yes. Having one method public and the other not would be very confusing, and make tooling to check for it much more difficult.
What about some macro @unexported that feels similar to export and add all listed symbols to a variable which all the tools such as Documenter/linters/… could access?
I think that the most painless solution would be just taking the most common approach and somehow producing a list from that, with the option of tweaking that with white- and blacklists.
Specifically, tooling could
exported symbols (exported),docs/src/*.md for Foo.symbol given module Foo (docs),and the API would be
exported ∪ (docs ∩ docstring)
This would require zero or near-zero effort from most package authors, the rest could take care of corner cases with white- and blacklists.
One thing this approach prevents is having more explicit devdocs and docstrings for internal functions at the same time - which may be desirable, like Base has.
I don’t think doing something smart gets us around ultimately wanting to specify explicitly what is considered API of a package (explicitly specifying what is not API is a tougher choice and is in part a reason why I don’t like private in other languages).
Perhaps a smart tool like @Tamas_Papp suggests would assist migration. But I agree this is a case where it would be better to make people be explicit. And since we’re talking I believe about a soft barrier, the cost of not implementing it is low. Give people a way to suppress the warning for dependencies that haven’t been updated and both sides have the ability to deal with the change.
@ToucheSir - thanks for calling out that package, I didn’t know it existed. Looks like exactly what’s needed IMO.
Taka and I do plan on promoting that package further. I’d first like to add PublicAPI support to Documenter, and then we’ll probably start advertising PublicAPI more broadly.
I think it is a nice implementation of three (naturally related) features:
@public)@strict
I think that 1. and 2. are great, especially since packages which just export their API don’t have to do anything whatsoever. This would aid migration and help adoption.
I am not so keen on @strict though. I think that this kind of check belongs in the unit tests or a linter, and should not clutter code.
I’m very happy to see that the discussion has moved toward solutions. I would like to add my 2 cents proposal:
1/ everything is public by default (as now). This way the developers that want to provide unrestricted access can have it, and nothing breaks. Also, all the existing codebase just keeps working.
2/ have a set of private modifiers (maybe private for module level and protected for package level). The compiler would ensure that private/protected members can’t be exported/accessed outside their valid scopes. I expect people will progressively start to rely on these to mark private APIs.
3/ optionally allow for an explicit overrule statement that shows that the consumer of a private API understands and knowingly overwrites the private access modifier. (I would not propose this, but it seems that many users want to still be able to access private fields). The approach has the added benefit that it can help do smart things: if a private function that is accessed via overrule is tagged to be deprecated (to be removed in a minor release), this can result in a breaking build in the consuming application; or tooling can flag overrule statements as warnings.
4/ the idea of access modifiers should be completely uncoupled from documentation - internals need to be documented as well, in order to help with maintenance and foster collaboration.
These options I believe would cover and satisfy all the uses cases.
I’m against this precisely because it requires that a maintainer has to actively think about it. I can see the “Oh shoot, I meant for this to be private - here, have a breaking change where I move it to the internal API” being one of the first things that happens. I thought preventing such trivial mistakes was the whole goal here. As such, adding a private modifier is a change that only attempts to treat a symptom, not the cause.
It should be as easy as possible to both write and use packages, which could be achieved by making it easy for package authors to declare “Hey, this foo is API for your use!” (which should always be a consideration of the author anyway) and not having to think about what is NOT API (which is just waaay too easy to forget during a refactor).
This sounds like quite a lot of Java-esque boilerplate (is anyone here familiar with @override?), which I really wouldn’t want to see in julia. None of the above requires a new keyword; tooling can (and does) warn/error about this already as part of a CI linter pass.
So if anything, I’d say we should go for something like Rust has, where everything is private by default and making things accessible is done explicitly. Having things private by default and moving things to be public is always safe to do, and never a breaking change (which I thought was the entire goal here - preventing breaking changes). Accessing a non-public anyway can (and should) warn, but it should not be prevented entirely.
In practice, this model is already what (at least Base and most packages) use, it’s just not codified/made explicit in code itself. Since that would be a breaking change (everything is accessible right now after all), we can’t do it before 2.0.
Because in the Julia ecosystem we use SemVer, making a method private would require a new major release. Don’t see any issue with this.
By all means, 100% in agreement. I just thought that it would be too much work for package developers to update all their codebases to make things explicitly public. And non-updated packages would suddenly break. I totally dislike the Java way of doing things, so no comment there.
An interesting approach for public/private is in Ruby, which allows declaring a line of code beyond which everything becomes private. This would make it easy to declare large chunks of modules as public or private, by having a line that goes @public and covers all the definitions from that point on, until the end of the module or until the next @private statement (and the reverse for @private).
Regardless of the concrete proposal, I don’t think we should burden/complicate the compiler with features like this.
This is something that a linter should be able to do perfectly well.
Seeing some usage statistics on such a linter would also help us gauge the demand for this feature.
I sincerely do not understand this need to create problems and then solutions to them.
This does not really solve any problems because as public will be the default (\1), then you either restrict yourself to the minority of packages making use of private modifiers (\2) or you may encounter the same problem (“using a non-API function/field without knowing it is not API”).
I also do not see why a complete uncoupling from documentation is desirable. Even if private also needs to be documented (what could be done with comments as I already pointed before) public always need to be documented. So there will always be a coupling.
Yes, this is the way to go. But until 2.0 there is no possibility of it happening. Also, there probably be some macro or way to disable the warning for an annotated call site.