How is the correctness of public open-source packages ensured? If a package contains latent errors that people are unaware of, someone relied on it to conduct scientific research, leading to incorrect conclusions, and published academic papers, who will be responsible for it?

[image] eweiss: A quick look at some of Julia’s statistical modules shows that they have simple test cases that are used to validate the results. This is a benefit. You can see whether or not a particular functionality is tested and how extensively. How well is new functionality in Matlab / S…

Right, if there are concrete places where packages should have more tests, please just open an issue — or better yet, help contribute those tests directly. Contributing tests is the best way to (a) confirm that the functionality you care about works as you want and (b) ensure that it continues to w…

[image] tbeason: How well is new functionality in Matlab / Stata / SAS tested? We don’t know. I will say, however, that Matlab has built up a lot of trust. They have an incredibly-responsive customer service, post known issues publicly, provide regular bug-fixes, etc. There are similar clos…

I myself feel more safe using commercial software like MATLAB than any other open-source software although the latter is indeed more transparent and appealing. Don’t know why.

[image] WuSiren: I myself feel more safe using commercial software like MATLAB […] Don’t know why. To quote Douglas Adams: because it’s reassuringly expensive.

I encountered very bad, undocumented bugs in Matlab/ Simulink… Example: if you try to linearize a Simulink system that has two or more algebraic loops you get completely wrong results without any warning. With ModelingToolkit.jl it just works. I would say your feeling is misleading…

I (like some, unlike others) feel that Julia usage has higher risk of producing incorrect results than I’d like. Given that, I think it’s good to calibrate fear/uncertainty/doubt levels so that (1) new users know they should be extra careful if they can afford that risk and go elsewhere if they can’…

[image] jar1: Julia usage has higher risk of producing incorrect results This reminds me of Pentium FDIV bug - Wikipedia . I don’t think that Julia has a higher risk for incorrect results. Not more than R or python. How is this higher risk argued? At the end it’s native code produced by LLVM…

Laying out concrete issues and appropriately setting expectations is not FUD — it’s the exact opposite. [image] oheil: Can someone enlighten me, where this higher risk comes from? The Julia language is highly permissive. It allows you to combine packages that have never been combined before…

[image] mbauman: something that you just need to be aware of and test/validate and that’s what I strongly argue here for any software you use as a researcher :wink:

Why is it reliable to use open source packages for research?

Offtopic

tbeason May 9, 2024, 4:31pm 19

This is a benefit. You can see whether or not a particular functionality is tested and how extensively. How well is new functionality in Matlab / Stata / SAS tested? We don’t know.

Topic		Replies	Views
Arguing about sloppy science Offtopic	23	1000	May 10, 2024
Discussion on "Why I no longer recommend Julia" by Yuri Vishnevsky Community discussion	292	54380	September 9, 2022
The State of the Julia Ecosystem Community	108	10289	January 31, 2019
Julia stability vs Rust for Scientific Computing Optimization (Mathematical) question	43	4242	August 10, 2026
Why Julia - A Manifesto Community	38	6316	December 7, 2023

Why is it reliable to use open source packages for research?

Related topics