Please explain syntax

General Usage
1

Hi.

I recently started reading through Julia base library code and stumbled on the following syntax (char.jl):

Char
(::Type{T})(x::Number) where {T<:AbstractChar} = T(UInt32(x))
(::Type{AbstractChar})(x::Number) = Char(x)
(::Type{T})(x::AbstractChar) where {T<:Union{Number,AbstractChar}} = T(codepoint(x))
(::Type{T})(x::T) where {T<:AbstractChar} = x

I have read through entire Julia documentation but can’t remember anything like this. Why the ‘Char’ statement in a separate line? What’s with those ‘(::Type{T})’? Have I missed a chapter somewhere?
Thank you!

PS: Its a pity Julia decided not to follow Python design of modules reflected by file & directory structure. It is becoming increasingly hard to read large code bases in Julia without syntactical analysis, even harder than C++. IMHO, Python’s design to reflect program logical structure with file structure is brilliant and allows to quickly grasp code structure and navigate even large code bases without getting lost. The argument for code re-usage with 'include’s can be easily resolved by naming convention. For example *.jl files may reflect modules structure, while *.ji files are includes and ignored by compiler unless included. It is still possible to adopt this system by inverting such naming convention, like *.jl files are includes and *.jm files are modules, so no existing code will be broken.

2

You omitted an important part from the source code. The meaningful code from char.jl is this:

"""
    Char(c::Union{Number,AbstractChar})

`Char` is a 32-bit [`AbstractChar`](@ref) type that is the default representation
of characters in Julia. `Char` is the type used for character literals like `'x'`
and it is also the element type of [`String`](@ref).

In order to losslessly represent arbitrary byte streams stored in a `String`,
a `Char` value may store information that cannot be converted to a Unicode
codepoint — converting such a `Char` to `UInt32` will throw an error.
The [`isvalid(c::Char)`](@ref) function can be used to query whether `c`
represents a valid Unicode character.
"""
Char

(::Type{T})(x::Number) where {T<:AbstractChar} = T(UInt32(x))
(::Type{AbstractChar})(x::Number) = Char(x)
(::Type{T})(x::AbstractChar) where {T<:Union{Number,AbstractChar}} = T(codepoint(x))
(::Type{T})(x::T) where {T<:AbstractChar} = x

As you can see, there is a docstring above Char, which is attached to Char and you can see the docstring in REPL by typing ?Char.

2 Likes
3

In general, using more than one module per package (i.e. using submodules) is an antipattern.
You do not want more namespaces in julia,
you want less namespaces.

Namespaces prevent you doing dispatch.
Dispatch is where the power of the language comes in.

Prefer

foo(::Thing, x)
foo(::Other, x)

over

ThingModule.foo(x)
OtherModule.foo(x)

If one does use submodules,
which sometimes is the right solution,

I prefer to use the convention
Filenames starting with a Capital define modules,
and those starting with lowercase define non-module includes.
If includes exist for those submodules, then they go in a folder with the same name as the module.

5 Likes
4

That is

::Type{T}

is the typof of T,

So for example

julia> foo(::Int) = 1
foo (generic function with 1 method)

julia> foo(::Type{Int}) = 2
foo (generic function with 2 methods)

julia> foo(7)
1

julia> foo(typeof(7))
2

julia> foo(Int)
2

(x::FooType)(args...)

is call overloading,
so it lets you overload what happens if an instance x of FooType is called like a function.

For example

julia> struct PrefixedPrint
       prefix
       end

julia> (pp::PrefixedPrint)(text) = println(pp.prefix, ": ", text)

julia> warnprint = PrefixedPrint("Be Warned")
PrefixedPrint("Be Warned")

julia> warnprint("It happen")
Be Warned: It happen

julia> warnprint("It not stop")
Be Warned: It not stop

So the code combines these two to basically create constructors.

3 Likes
5

Thank you for the explanation!

So Julia uses includes instead and call it a better solution? At least in C, we have a naming convention to tell which is which.

How?

Really, by using a naming convention like in Python one can have both - easy code navigation and includes. And by making this naming convention part of the language it will be possible to eliminate ‘module’ keyword.
Please, don’t get me wrong, I believe Julia has everything to become a mainstream multi-purpose language, and quite soon. I fell in love with it quite quickly. I only hope its creators look beyond scientific computation scope and don’t rush with decisions.

Its just ‘::Type’ construct is never mentioned in the documentation so I was oblivious that ::Type{T} is typeof(T). Thanks for clarification!

6

Really, by using a naming convention like in Python one can have both - easy code navigation and includes. And by making this naming convention part of the language it will be possible to eliminate ‘module’ keyword.

Oh, no. Back in 2010 I’d agree with you, but not anymore. Nowadays Python uses all the tricks to put functions in places other than what package/module path implies. For example, numpy has a function dot() which you normally call as:

import numpy as np
np.dot(...)

So there should be a file somewhere on a system with name “numpy.py” with this function definition, right? Surely, not - numpy is a package, not a module. So there should be numpy/__init__.py with definition of dot(), right? Ok, here it is. Can you see the definition there? Hell, no! Using Google I found 2 definitions of dot() in numpy/ma/core.py (1 and 2), but neither seem to be exported in _all__ section. There’s also numpy.ma.dot() mentioned in docstrings, although I can’t see how it gets to numpy/ma/__init__.py too.

In Julia I would just use methods(dot), @which dot(A, B) or even @edit dot(A, B) if I wanted to update the code.

Also in Python we have only functions while in Julia there are functions and methods (I’m not taking about methods as class attributes in Python, of course, which have nothing to do with multiple dispatch of methods in Julia). Say, we follow your suggestion and prefix all the functions with module name. You see the code:

Databases.connect(...)

Does it mean that function connect() is defined in module Databases? Function - maybe, but actual method can be defined as PostgreSQL.connect(), MySQL.connect(), etc., and all these methods are attached to the function Databases.connect. Placing a module name in front of function call doesn’t help you to figure out where to find the code, method arguments do.

6 Likes
7

Is this a generally held opinion? I am not convinced. Putting everything in the same module increases load time, and could lead to namespace pollution. I think it’s useful to put code that belongs to a package but isn’t part of the core functionality in its own module. A prime example would be sample/test/benchmarking code. Currently, Julia’s solution is to either use separate repositories (which I think is a major anti-pattern for code that belongs together), or to use sub-projects which kind of work but is quite rough around the edges.

3 Likes
8

I think I know the pain. Yes, it sounds horrifying when you know the last line of The Zen of Python is Namespaces are one honking great idea – let’s do more of those! The fact that Julia’s default import syntax using Module is equivalent to infamous start-import from Module import * in Python sounds like the complete opposite of Explicit is better than implicit. But I think there are a few reasons why it turned out to be not a huge issue in Julia. For example, multiple-dispatch reduces the names exposed in a given global scope. It makes name crash less likely. Also, Julia really is a compiled language so that name crash is detected by the compiler in advance and shown as an error (you don’t need an external tool like pyflakes for this). Of course, you can still use submodules in Julia if you need. I don’t think it’s an anti-pattern. For example, Pkg have some submodules. But I don’t think that the file not having its own namespace is a peculiarity of Julia. For example, Go does it too and it is supposed to be a “modern”, “practical” and not science-oriented language.

9

Oh, why people keep implying things I’ve never said? I swear I like Julia! Damn, I love it! Why is that defensive position? I never said anything about prefixing function calls or doubted merits of Julia package manager (which is far superior to most I’ve seen yet). What I was really suggesting is merely a file naming convention to alleviate code structuring. In fact, to implement what I’m asking only two things need to be done:

  1. Add a convention that files with *.jm name extensions represent modules with the same name as the file. ‘module’ keyword in such files can/should be omitted.
  2. Add functionality to existing import and using directives so additionally to existing behavior they will also look in path for a *.jm file with corresponding name/path.

No need to change any existing code, damp existing solutions with includes or do any other sinful things mentioned in above posts.

3 Likes
10

Hm, what I’ve read is that you:

  1. Find it hard to understand where things are defined in Julia.
  2. Believe that Python does it right, binding module path to file system path.

And thus you propose to follow Python way and bind modules names to file names e.g. with extension .jm. Is it what you mean? If so, in my answer I showed by example that:

  1. Modern Python often abuses this pattern using __all__ attribute and all kinds of reexports.
  2. In Julia it won’t help anyway because importing a function from module Foo doesn’t mean that a method you call is defined in that module too.

I also mentioned several tools that you can use to quickly and unambiguously navigate Julia code. So exactly what problem do you want to solve with the proposed *.jm files?

1 Like
11

While Oxinabox has already explained this construct, I’d like to add the documentation entry: Function-like-objects.

12

I am proposing to give developers additional means to organize code with no conflict to the existing system, should they choose to use it. I think it is convenient to know that the ModuleName.jm file represents a module. It is convenient being able to load this module with using ModuleName without specifying its file name or having to previously include its file. No argument this system can be abused, as it happens sometimes in Python. Yet when developers choose not to abuse it, it really helps to organize code. And no need to deprecate existing practice.
And I am just asking to consider this, discuss it maybe. Isn’t this what community forums are for?

13

Thank you for clarification!
The only confusion was because ::Type{...} construct was never mentioned in the documentation so I didn’t understand it was a functor on an abstract type instance.

14

It is in the docs, though I agree it isn’t entirely obvious. See Types · The Julia Language

15

Yes, of course it is fine to make suggestions like this. That said, the benefit is unclear to me; also, when learning a language it may be beneficial to understand existing practice (which takes a lot of time and use) before suggesting changes to something fundamental.

Which is how it works at the moment… so it is unclear what would improve by this.

You may want to look at the documentation for code loading. TL;DR:

  1. using & friends employ a sophisticated, robust and customizable system to actually locate the package,
  2. the standard location is src/PackageName.jl for the main file,
  3. it is customary to just include other files from there.
1 Like
16

Cool. So,

  1. To my understanding packages and modules are generally different things. One package may contain multiple modules and sub-modules. Can it contain sub-packages?
  2. So inside src/PackageName.jl can I also write using AnotherPackage to load src/AnotherPackage.jl?
  3. Having both file name convention and module declaration in the same file looks ambiguous. What if I write module DifferentModuleName inside src/PackageName.jl?
17

A package having a single module (and possibly submodules) is what is currently best supported (other things require hacks and workarounds), so this is what most people use.

I don’t know if “sub-packages” is a meaningful concept at the moment in Julia.

I don’t think so, at least not without adding it to the local project with an explicit path. Are you perhaps looking for submodules?

AFAIK you won’t be able to load it via using etc. So don’t :wink:

18

Do you have a code base, where it’s hard to navigate? I personally stick to what you recommend - most modules I define are defined in a file with the module name!
I do know what you’re talking about, though.
E.g. Core.Compiler isn’t defined in compiler.jl - which made it quite hard to find especially with the bad github search. I had to import Julia Base into Atom, to search for module Compiler :smiley:
But since you propose yet another optional feature to make this easier it doesn’t seem to solve any of this - we’d still be stuck with educating people to only define modules in a file with the same name :wink:

19

Do you now something like this (do it in REPL)?

julia> @edit Char(1)
20

So the sophisticated system you are describing works for packages, not modules. Yet Julia also has modules for some reason, modules have exports, etc., so creators thought about unclogging namespace instead of just putting all the code in one heap. I think it is convenient being able to reflect modules structure with file/directory structure within one package/program. It will help to organise the code and resolve ambiguity with module names and file names. I am not talking about packages here.