Static Analysis vs. Unit Tests

That sounds like not enough unit tests.

That’s rather beside the point — one of the goals of static analysis in general is to catch errors of this sort without (or in addition to) unit tests.

9 Likes

If you responded to me: my point was that it is so much better to have unit tests, because correctness of the code is not just about typos, but mainly about the internal logic.

I feel like you’re missing the point - typo detection is valuable while writing the very code you don’t/can’t yet test. It might also take quite a long time to run that kind of testsuite, which is very counterproductive when you’re still writing code; you want to iterate fast, and that includes being reasonably sure that your unit tests test functionality and not whether you hit the wrong key somewhere half an hour ago.

In other languages (much more common in software engineering circles than Julia), this kind of early feedback is invaluable, and a large part why the “incumbents” in software are who they are.

16 Likes

Perhaps. Or, you are writing chunks of code which are way too big and should be broken up into testable pieces.

There is no need to blame a user of the language here. No code is small enough to prevent typos from accidentally creeping in - be it capitalization, writing 8 instead of ( (by mistiming pressing SHIFT on my keyboard, for example) or simply hitting the wrong key by slipping to a neighboring one.

Unit tests don’t help here, they are not a solution for this kind of problem. This is something an IDE should be able to report immediately, as is the case in other languages (some of which are much, MUCH more verbose/require more boilerplate than Julia).

14 Likes

No one is blaming anyone here! The point is valid: no IDE will be able to catch logical errors. And unit tests are a necessity. So why not do them right, which will prevent typos popping up much later in production code.

I think the kind of error being discussed here is not the same that is really appropriate to catch in a unit test (or even have a unit test for at all)

exactly, these are not really “logical” errors so much as “obvious” errors. not obvious in the sense that committing one is any reflection on the user’s skill level, but obvious in the sense that the user will likely be able to fix it within seconds of noticing.

nobody in the world is writing a unit test to “make sure I spelled this function correctly.”

2 Likes

That is one of the purposes of UT in Julia: it is a dynamic language. Therefore writing a1 = 1 and a_1 = 1 (the first is missing an underscore) is not an obvious error. It may be another variable being introduced. Logical error? Find it with a UT. No IDE will be able to help you here.

yes, you have identified a type of error that IDEs cannot help with

not sure what relevance that has to the types of errors that IDEs can

5 Likes

I think you’re still missing my point: no ide will be able to catch all errors, therefore UTs are essential, and when done well, they can quickly and efficiently catch all errors.

who said anything about catching all errors?

  • there are errors that IDEs can catch
  • it is useful to the user when said errors are caught

do we disagree on either of these ?

1 Like

Even for an interactive language, catching some “maybe-not-errors” immediately is a useful thing, even if they’re not as solid as unit tests. For example,

julia> pie = 3.14
3.14

julia> circumference(diameter) = diameter*pue
circumference (generic function with 1 method)

would be nice if I was warned that I have yet to define pue in the global scope, even before I throw an actual error in a call. If this was in a source file

pie = 3.14

circumference(diameter) = diameter*pue

then the additional assumption of best practices can highlight pue and say it wasn’t defined in the file yet. I could even have an option to toggle highlighting of global variables if I’m trying to avoid them for performance reasons, even if it’s valid code. Unconditional errors aren’t the only things that should be brought to our attention.

3 Likes

It looks like the VS Code linter already treats references to global variables as missing references:

image

That example is from a vanilla Julia script, i.e. that file is not contained in a module, so the definition of circumference is technically valid. As far as I know, there is no linter option to only highlight missing local references when the method definition is inside a module.

4 Likes

Right, so I actually am not sure what scenario prompted the original complaint about JET, not enough specifics to tell if a linter can handle it. Point was it’s not a bad thing to have several tools to catch different things at different times.

3 Likes

My point is, I’d expect JET.jl to find the obvious mistakes, so that I can save my unit-test-writing time on the actual functionality.

4 Likes

Thanks! This is exactly why I’m avoiding Julia now. Currently, I don’t see the advantage of a dynamic language over one that catches my mistakes.

The language not enforcing declaration of variables just clashes with my basic tendency to make many mistakes when I type code. The set of variables that are in scope being determined at runtime – that’s always been a nightmare characteristic of Julia (and Python) for me.

3 Likes

Local variables are definitely set in stone at lower-time. What exactly is the sort of mistake you’re referring to, it wasn’t made explicit earlier despite people reaching more or less the same guess. Maybe a MWE? It’ll be important to verify whether JET.jl is even the suitable tool to catch such mistakes; JET.jl specifically deals with type inference and method dispatch issues in function calls, not bugs in general.

2 Likes

the original thread was “What don’t you like about Julia for serious work”

I think to get a valuable discussion out of that prompt, it’s important to solicit the opinions of those who in fact do not like Julia for serious work.

Hence, what @DOT is accomplishing is the task set out in the OP: to provide a perspective on why one might not choose Julia

10 Likes