# Stdlib tests that fail with v1.3.0 binary

**URL:** <https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159>\
**Category:** New to Julia\
**Tags:** testing\
**Created:** [December 11, 2019, 4:54pm UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159 "2019-12-11T16:54:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![braydenware](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/braydenware/32/2612_2.png) [@braydenware](https://discourse.julialang.org/u/braydenware)\
**Post date:** [December 11, 2019, 4:54pm UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159/1 "2019-12-11T16:54:00Z")

</div>

I was building v1.3.0 from source, and out of habit ran `make testall` to make sure the tests pass with my build before I started using it. I had 52 tests fail from a variety of stdlib packages, so naturally I tried building again without my Make.user tweaks - and got the same 52 failed tests. Then I downloaded the binary distribution, and ran the same tests (located at `share/julia/test/runtests.jl`) - and still, the same 52 tests were failing.

This seems odd, so I took a look at one of the tests that failed, from the LinearAlgebra package, in the file `share/julia/stdlib/v1.3/LinearAlgebra/test/tridiag.jl`.

```julia
@testset "issue #29644" begin
    F = lu(Tridiagonal(sparse(1.0I, 3, 3)))
    @test F.L == Matrix(I, 3, 3)
    @test startswith(sprint(show, MIME("text/plain"), F),
          "LinearAlgebra.LU{Float64,LinearAlgebra.Tridiagonal{Float64,SparseArrays.SparseVector")
end

```

The offending test was the second one, where the result I get for  
`sprint(show, MIME("text/plain"), F)`  
starts with  
` LU{Float64,Tridiagonal{Float64,SparseVector`  
without the `LinearAlgebra.` and `SparseArray.` on the text representation of the types.

Even more strangely, when running the julia binary on just this test file `share/julia/stdlib/v1.3/LinearAlgebra/test/tridiag.jl`, the test _passes_.

I’m not super concerned about the result of this particular test, as the code seems to be working. Just wondering if `make testall` gives a number of test failures for everybody or whether if I’m doing something basic wrong.

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [December 12, 2019, 1:44am UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159/2 "2019-12-12T01:44:19Z")

</div>

If you want to check whether your experience is typical, try opening some of the pull requests to Julia itself and examine the “checks.” In general you’ll see that the large majority of runs pass (there are some platforms that seem flakier than others, though). I’ve not tested 1.3 in a while, but in general `master` passes (otherwise it makes it hard to do development).

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [December 12, 2019, 6:49am UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159/3 "2019-12-12T06:49:53Z")

</div>

This may be a subtle bug in the `testall` target somehow being different from the way that tests are run on CI (which I think has generally been done by directly calling julia from the CI script). You can try comparing the working directories and arguments given to the `choosetests.jl` test runner.

In the past I’ve hit weird edge cases where the test runner can behave differently depending on the number of tests which are invoked. IIRC due to the wrapping of tests within a test module and the detail of how tests are distributed to parallel workers.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [December 12, 2019, 7:00am UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159/4 "2019-12-12T07:00:18Z")

</div>

Julia prints the module prefix or not depending on if the Symbol is available in `Main` (a well intended feature but arguably kinda strange). It likely has something to do with that.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [December 12, 2019, 9:42am UTC](https://discourse.julialang.org/t/stdlib-tests-that-fail-with-v1-3-0-binary/32159/5 "2019-12-12T09:42:11Z")

</div>

I just ran the 1.3.0 tests on Linux and I get failures in Dates/io, of the sort:

```julia
Test Failed at ~/julia/julia-1.3/usr/share/julia/stdlib/v1.3/Dates/test/io.jl:528                                                                              
  Expression: uppercase(t12) == Dates.format(t, "II:MMp") == Dates.format(d, "II:MMp") == Libc.strftime("%I:%M%p", tmstruct)                                             
   Evaluated: "12:00AM" == "12:00AM" == "12:00AM" == "12:00"                        

```

I suspect that that has something to do with the “locale” settings on my machine. Other than that all good, no issues in LinAlg.

(But I did notice that the test suite used at times most of my 16GB of RAM. Seems a bit excessive.)