# Script to find which of your dependencies is not working in julia 0.7
**URL:** https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528
**Category:** Tooling
**Tags:** pkg
**Created:** [June 8, 2018, 10:07am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528 "2018-06-08T10:07:00Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [June 8, 2018, 10:07am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/1 "2018-06-08T10:07:00Z")
So obviously before you can update your package to be compatible with julia 0.7,
you need to update any of your dependencies to julia 0.7
I have put together a script
which recursively runs `Pkg.test` on the specified package and then (if it fails) on its dependencies.
And produces a report.
```julia
using Pkg
const _tested = Dict{String, Bool}()
function passing_all_tests(pkgname)
get!(_tested, pkgname) do
try
Pkg.add(pkgname)
Pkg.test(pkgname)
return true
catch
return false
end
end
end
function get_deps(pkgname)
pkg_dir = Pkg.dir(pkgname)
@show (pkgname, pkg_dir)
reqfile = joinpath(pkg_dir, "REQUIRE")
if !isfile(reqfile)
# anything without a REQUIRE file is not going to be a problem anyway.
#This mostly occurs if you are testing something in stdlib
return String[]
end
deps = String[]
for line in eachline(reqfile)
(strip(line)=="" || line[1]=='#' || line[1]=='@') && continue
#^ Skip comments and things like @osx
pkg_name = split(line)[1]
pkg_name == "julia" && continue
push!(deps, pkg_name)
end
deps
end
function check(pkgname, final_output=stdout, depth=0)
if !passing_all_tests(pkgname)
println(final_output, "\t"^depth * pkgname)
# don't both checking dependencies of things that pass
# Can not use the below with an asyncmap, or it gets weird
for depname in get_deps(pkgname)
check(depname, final_output, depth+1)
end
end
end
function deptree_report(pkgname)
final_output = IOBuffer()
println(final_output)
println(final_output, "-"^70)
println(final_output, "Failing Dependency Tree for " * pkgname)
println(final_output, "-"^70)
check(pkgname, final_output)
println(final_output, "-"^70)
println(String(take!(final_output)))
end
```
Copy and paste it into your repo and call it via: `deptree_report("MyPackage")`.
it will dump out a huge pile of text from all the tests it is calling.
Then at the end it will print out a report.
Any package named in that report is failing its tests on your system.
That might be its own fault, or it might be the fault of its dependencies (esp) if they are listed after. Or both.
(Also failing locally doesn’t nesc mean it is broken on 0.7 for everyone; in the examples below SHA is failing on my computer because of a file-permission error due to where it is installed.)
Also running it for many packages you want to test, one after the other is good,
as it caches test results, so you can avoid rerunning tests.
(Conversely test results won’t updated without a restart/clearing the global `_tested` Dict)
Note this script it will explicitly install all your package’s dependencies, you can probably run it in a throw-away environment if you want.
Also this can take a while, and you might want to turn off depwarn.
## Here are some example outputs:
### MacroTools
Everything is fine.
```julia-auto
julia> deptree_report("MacroTools")
...
----------------------------------------------------------------------
Failing Dependency Tree for MacroTools
----------------------------------------------------------------------
----------------------------------------------------------------------
```
### Flux
```julia-auto
----------------------------------------------------------------------
Failing Dependency Tree for Flux
----------------------------------------------------------------------
Flux
DataFlow
Requires
Adapt
GZip
Colors
ColorTypes
FixedPointNumbers
FixedPointNumbers
ZipFile
AbstractTrees
----------------------------------------------------------------------
```
### TensorFlow
```julia-auto
Failing Dependency Tree for TensorFlow
----------------------------------------------------------------------
TensorFlow
ProtoBuf
PyCall
Conda
BinDeps
URIParser
SHA
TakingBroadcastSeriously
Conda
BinDeps
URIParser
SHA
Distributions
StatsFuns
Rmath
BinDeps
URIParser
SHA
StatsFuns
Rmath
BinDeps
URIParser
SHA
JLD
HDF5
BinDeps
URIParser
SHA
Blosc
BinaryProvider
SHA
CMakeWrapper
BinDeps
URIParser
SHA
FileIO
LegacyStrings
FileIO
MNIST
----------------------------------------------------------------------
```
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [June 8, 2018, 11:49am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/2 "2018-06-08T11:49:48Z")
Very nice! It would be worth integrating something like this into the test functionality of the package manager itself.
---
### Author:  [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)
#### Post date: [June 8, 2018, 1:43pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/3 "2018-06-08T13:43:44Z")
I love this. Would it be possible to compact the output some when a broken dependency has already been displayed? I.e. in the above example, where BinDeps is displayed 5 times, simply having `BinDeps*` after the first would be sufficient, and maybe when there is only one broken dependency, possibly fit it on the same line?)
(and maybe for 2 or 3 broken dependencies that are leaves, or can be displayed with \*), also on same line?
For example:
```julia
TensorFlow
ProtoBuf
PyCall => Conda => BinDeps => (URIParser, SHA)
Conda*
Distributions => StatsFuns => Rmath => BinDeps*`
StatsFuncs*
JLD
HDFS
BinDeps*
Blosc
BinaryProvider => SHA*
CMakeWrapper => BinDeps*
FileIO
LegacyStrings
FileIO
MNIST
```
(I’d be happy to help out trying to make this look nicer, if you’d like, after appropriate bikeshedding over what a good “compact” format should look like.
> [@StefanKarpinski](#):
>
> Very nice! It would be worth integrating something like this into the test functionality of the package manager itself.
Does Pkg have a nice way currently of showing the dependency tree like this (not just the failing ones)?
That would also be useful, not just part of the test functionality.
---
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [June 8, 2018, 2:15pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/4 "2018-06-08T14:15:12Z")
It’s a fairly simple script, should be easy to tweak for output wanted.
I was thinking about only showing things the first time, but that would hide the info that might be needed to determine what to do to fix on of the later ones.
Perhaps the ultimate solution would be to display the DAG, but not sure if there is a good text format for that
---
### Author:  [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)
#### Post date: [June 8, 2018, 3:36pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/5 "2018-06-08T15:36:43Z")
> [@oxinabox](#):
>
> I was thinking about only showing things the first time, but that would hide the info that might be needed to determine what to do to fix on of the later ones.
Well, you would only need to fix `BinDeps` once, to fix all of them that are later marked as “BinDeps\*” for compactness. The output might also need some way of displaying the UUID (or the first part of it), if there are multiple packages with the same name (but different UUIDs), since Pkg3 allows that.
---
### Author:  [@chakravala](https://discourse.julialang.org/u/chakravala)
#### Post date: [June 8, 2018, 6:56pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/6 "2018-06-08T18:56:11Z")
The problem is that the script represents that data as a Tree, while what @ScottPJones wants is a Graph. You could solve his issue by representing the data as a Graph instead of a Tree.
---
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [June 9, 2018, 4:49am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/7 "2018-06-09T04:49:10Z")
> [@chakravala](#):
>
> You could solve his issue by representing the data as a Graph instead of a Tree.
Yes, but displaying a graph in plain-text is hard.
I am fairly sure calculating good node placement for a layered visualization of a DAG is not NP-Hard.
But it is nontrivial.
And getting that to look good (or readable) when displaying with characters is just not going to be easy
It might be a fun project for someone who is interested in that.
---
### Author:  [@bjarthur](https://discourse.julialang.org/u/bjarthur)
#### Post date: [June 30, 2018, 10:42am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/8 "2018-06-30T10:42:30Z")
has recursive dependency testing like this been added to Pkg3 yet? the docs don’t seem to mention it
---
### Author:  [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)
#### Post date: [June 30, 2018, 4:42pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/9 "2018-06-30T16:42:12Z")
> [@oxinabox](#):
>
> deptree\_report
```julia
Warning: Pkg.dir is only kept for legacy CI script reasons
└ @ Pkg.API API.jl:431
ERROR: type Nothing has no field captures
Stacktrace:
[1] getproperty at .\sysimg.jl:18 [inlined]
[2] splitdrive(::String) at .\path.jl:37
[3] joinpath(::String, ::String) at .\path.jl:217
[4] entry_point_and_project_file(::String, ::String) at .\loading.jl:429
[5] implicit_project_deps_get(::String, ::String) at .\loading.jl:583
[6] project_deps_get at .\loading.jl:320 [inlined]
[7] identify_package(::String) at .\loading.jl:253
[8] dir(::String) at .\logging.jl:317
[9] get_deps(::String) at .\REPL[4]:2
[10] check(::String, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Int64) at .\REPL[5]:7
[11] check at .\REPL[5]:2 [inlined]
[12] deptree_report(::String) at .\REPL[6]:9
[13] top-level scope at none:0
```
---
### Author:  [@oxinabox](https://discourse.julialang.org/u/oxinabox)
#### Post date: [July 2, 2018, 5:47am UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/10 "2018-07-02T05:47:22Z")
That to me looks like it might be a bug in `Base`
Base uses regex for some file path operations (like `splitdrive`),
and if nothing else, that is a leaky abstraction right there.
Maybe MWE it and report on JuliaLang/julia github?
---
### Author:  [@Robert\_Honig](https://discourse.julialang.org/u/Robert_Honig)
#### Post date: [August 11, 2018, 3:16pm UTC](https://discourse.julialang.org/t/script-to-find-which-of-your-dependencies-is-not-working-in-julia-0-7/11528/11 "2018-08-11T15:16:05Z")
I think it’s actually an issue with the script:
```julia
Pkg.dir()
```
is deprecated and returns `nothing`, which leads to that confusing error message. Here is a fix for the script for Julia 1.0, using `pathof`:
```julia
using Pkg
const _tested = Dict{String, Bool}()
function passing_all_tests(pkgname)
get!(_tested, pkgname) do
try
Pkg.test(pkgname)
return true
catch
return false
end
end
end
function get_deps(pkgname)
Pkg.add(pkgname)
eval(Expr(:import, Symbol(pkgname)))
pkg_dir = eval(Expr(:call, :pathof, Symbol(pkgname)))
@show (pkgname, pkg_dir)
reqfile = joinpath(pkg_dir, "REQUIRE")
if !isfile(reqfile)
# anything without a REQUIRE file is not going to be a problem anyway.
#This mostly occurs if you are testing something in stdlib
return String[]
end
deps = String[]
for line in eachline(reqfile)
(strip(line)=="" || line[1]=='#' || line[1]=='@') && continue
#^ Skip comments and things like @osx
pkg_name = split(line)[1]
pkg_name == "julia" && continue
push!(deps, pkg_name)
end
deps
end
function check(pkgname, final_output=stdout, depth=0)
if !passing_all_tests(pkgname)
println(final_output, "\t"^depth * pkgname)
# don't both checking dependencies of things that pass
# Can not use the below with an asyncmap, or it gets weird
for depname in get_deps(pkgname)
check(depname, final_output, depth+1)
end
end
end
function deptree_report(pkgname)
final_output = IOBuffer()
println(final_output)
println(final_output, "-"^70)
println(final_output, "Failing Dependency Tree for " * pkgname)
println(final_output, "-"^70)
check(pkgname, final_output)
println(final_output, "-"^70)
println(String(take!(final_output)))
end
```
(The change here is):
```julia
Pkg.add(pkgname)
eval(Expr(:import, Symbol(pkgname)))
pkg_dir = eval(Expr(:call, :pathof, Symbol(pkgname)))
```