How is the correctness of public open-source packages ensured? If a package contains latent errors that people are unaware of, someone relied on it to conduct scientific research, leading to incorrect conclusions, and published academic papers, who will be responsible for it?

[image] StefanKarpinski: you generally don’t have any legal or financial recourse against the purveyor of commercial software if it gives you a wrong answer. For the curious, this may change in the future, at least in the EU, with the Cyber Resilience Act which aims to solve exactly this liab…

[image] Oscar_Smith: if the tool is broken [such as the open source code], retract the paper that is no longer correct, To get reproducible results you need the same exact versions, for all packages AND their dependencies, i.e. the Manifest.toml file, neither earlier, nor newer or latest vers…

This sounds like Thompson’s trusting trust . It’s trusting all the way down :scream:

In general, in science you are supposed to check and verify every step of your research. If one just uses a piece of software as a black box and proceeds to apply it to an unknown case, there is no guarantee that the result is trustworthy. One should have test cases where there exists an expected co…

All kinds of software and hardware has bugs. For example the x86 instruction for sin returns incorrect results . this has been known for a long time now. Which is why LLVM doesn’t emit a fsin assembly instruction and instead emits call to libm’s sin function.

A quick look at some of Julia’s statistical modules shows that they have simple test cases that are used to validate the results. Since I did not do an extensive survey of what various test directories contain, they will remain unnamed. Will they work properly in edge cases? Who knows.

[image] eweiss: A quick look at some of Julia’s statistical modules shows that they have simple test cases that are used to validate the results. This is a benefit. You can see whether or not a particular functionality is tested and how extensively. How well is new functionality in Matlab / S…

Right, if there are concrete places where packages should have more tests, please just open an issue — or better yet, help contribute those tests directly. Contributing tests is the best way to (a) confirm that the functionality you care about works as you want and (b) ensure that it continues to w…

[image] tbeason: How well is new functionality in Matlab / Stata / SAS tested? We don’t know. I will say, however, that Matlab has built up a lot of trust. They have an incredibly-responsive customer service, post known issues publicly, provide regular bug-fixes, etc. There are similar clos…

Why is it reliable to use open source packages for research?

Offtopic

Topic		Replies	Views
Arguing about sloppy science Offtopic	24	712	May 10, 2024
Discussion on "Why I no longer recommend Julia" by Yuri Vishnevsky Community discussion	298	50044	September 9, 2022
The State of the Julia Ecosystem Community	109	9044	January 31, 2019
Why Julia - A Manifesto Community	38	5769	December 7, 2023
Julia losing popularity among Data Science users (KDnuggets Software Poll) Community	146	20721	June 23, 2018

Why is it reliable to use open source packages for research?

Related topics