Information hiding or modulization in Julia's C/C++ codes

Internals & Design
1

As suggested in https://github.com/JuliaLang/julia/issues/36903, I also believe that Julia’s ABI needs to be sorted out.
In addition to the public/private distinction, IMO, it might be better to promote more modularization, even for private data and definitions.

One of the problems I’m facing is the inclusion of c/cpp files.

$ grep -nrE "#include .+\.cp?p?\"" src | sort
src/builtins.c:401:#include "iddict.c"
src/ccall.cpp:313:#include "abi_llvm.cpp"
src/ccall.cpp:315:#include "abi_arm.cpp"
src/ccall.cpp:316:#include "abi_aarch64.cpp"
src/ccall.cpp:317:#include "abi_ppc64le.cpp"
src/ccall.cpp:318:#include "abi_win32.cpp"
src/ccall.cpp:319:#include "abi_win64.cpp"
src/ccall.cpp:320:#include "abi_x86_64.cpp"
src/ccall.cpp:321:#include "abi_x86.cpp"
src/codegen.cpp:1443:#include "cgutils.cpp"
src/codegen.cpp:2133:#include "intrinsics.cpp"
src/crc32c.c:66:#include "crc32c-tables.c"
src/flisp/flisp.c:411:#include "cvalues.c"
src/flisp/flisp.c:412:#include "types.c"
src/flisp/flisp.c:419:#include "print.c"
src/flisp/flisp.c:811:#include "read.c"
src/flisp/flisp.c:815:#include "equal.c"
src/intrinsics.cpp:7:#include "ccall.cpp"
src/processor.cpp:841:#include "processor_x86.cpp"
src/processor.cpp:845:#include "processor_arm.cpp"
src/processor.cpp:849:#include "processor_fallback.cpp"
src/signal-handling.c:124:#include "signals-win.c"
src/signal-handling.c:126:#include "signals-unix.c"
src/signals-unix.c:229:#include "signals-mach.c"
src/support/hashing.c:51:#include "MurmurHash3.c"

I understand the rationale behind such a technique, but I think the disadvantages are greater. In particular, the chain “codegen.cpp” → “intrinsics.cpp” → “ccall.cpp” seems to inherit too many contexts.

I am unable to understand these codes without the assistance of the IDEs, and I use the VS Code as my IDE, but the IntelliSense and some static code analyzers also work hardly at all. Do you use any useful extensions?

I would like to make some improvements, but I am not sure where to start or what should be kept. I also don’t know whether this issue is shared among the developers.

Do you have any advice for me?

1 Like
2

Hi, I’m not a core developer, but I want to say that I also face problems with IntelliSense on large codebases. I recommend you use ccls ccls - Visual Studio Marketplace / https://github.com/MaskRay/ccls .

ccls is not so easy to install, and may also be slow sometimes like IntelliSense, but at least it usually indexes large codebases correctly.

EDIT: oh, it seems that clangd may also be good (or better), and has automatic installation: https://clangd.llvm.org/ / clangd - Visual Studio Marketplace

1 Like
3

Thanks for introducing the extension.
I am trying the clangd now, and it looks very good in itself! :heart_eyes:

However, it doesn’t seem to automatically resolve even the dependencies of c/cpp files (not header files). The editor pane (minimap) is still bright red. :joy:

Too many errors emitted, stopping now

It is functioning “properly”.

4

Yeah, I’ve just noticed now that Julia uses handwritten Makefiles, so it doesn’t (I think) generates a compile_commands.json file which clangd and cia use to index codebases :frowning:

5

If compilation dependency is a major reason for #include-ing c/cpp files, then I think that is a problem which should be solved by the build system.
I have a lot of :grapes: for CMake, but I think it’s one of the realistic solutions.

6

This might be of interest to you.

https://github.com/rizsotto/Bear

2 Likes
7

IIUC, the most common cause of this problem is file scoping (static).
This is a chicken and egg situation. In order to do a separate compilation, we need to remove statics, and as a prerequisite, we need to prohibit the use of inappropriate global variables. However, we cannot know systematically if the use of global variables is appropriate until we stop #include "*.{c|cpp}".

However, at least for C++, we can use namespaces.