How to upgrade from DataFrames 0.10.1 to 0.11.3?

New to Julia
,
1

Is there a path for an upgrade from DataFrames 0.10.1 to 0.11.3 given the dependencies?

This is what I have done so far; using this

find_holdback("DataFrames", 2)
Package StatPlots                                          requires DataFrames
Package RCall                                              requires DataFrames 0.10
Package ODBC                                               requires DataFrames
Package Gadfly                                             requires DataFrames 0.4.2
Package CSV                                                requires DataFrames
Package DataStreams                                        requires DataFrames
    Package ODBC                                               requires DataStreams 0.1.0
    Package CSV                                                requires DataStreams 0.1.0 0.2.0

for pkg in ["StatPlots", "RCall", "ODBC", "Gadfly", "CSV", "DataStreams"]
           Pkg.rm(pkg)
end

INFO: Removing IterableTables v0.6.1
INFO: Removing StatPlots v0.6.0
INFO: Package database updated
INFO: Removing RCall v0.8.1
INFO: Package database updated
INFO: Removing DecFP v0.3.0
INFO: Removing ODBC v0.5.2
INFO: Package database updated
INFO: Removing Gadfly v0.6.4
INFO: Package database updated
INFO: Removing CSV v0.1.5
INFO: Package database updated
INFO: Removing DataStreams v0.1.3
INFO: Package database updated

julia> Pkg.update("DataFrames")
INFO: Updating METADATA...
INFO: Computing changes...
INFO: Package DataFrames was set to version 0.10.1, but a higher version 0.11.3 exists.
      The update is prevented by explicit requirements constraints. Edit your REQUIRE file to change this.
INFO: No packages to install, update or remove
2

Have you tried Pkg.rm for the Dataframes package itself? Then you could try adding it again.

3

Yes, did that too. But It reinstalled 0.10

julia> Pkg.rm("DataFrames")
INFO: Removing DataFrames v0.10.1
INFO: Package database updated

julia> Pkg.add("DataFrames")
INFO: Installing DataFrames v0.10.1
INFO: Building SpecialFunctions
INFO: Package database updated

julia> Pkg.update()
INFO: Updating METADATA...
INFO: Computing changes...
INFO: No packages to install, update or remove
4

i had something similar before. check inside your .julia folder Pkg.dir() look for the REQUIRE file and see if there is a limit on DataFrames inside

5

Note the comment at Comment on DataFrames for DataFrames

6

In this thread, someone explains that there is an outstanding PR for RDatasets that is preventing the upgrade.

7

Hmm… I understand the implication of all these outstanding PRs, but it would be brilliant if the devs could post a beginner friendly upgrade path, if at all possible at this stage.

1 Like
8

The package ecosystem is in a temporary state of flux while developers adapt to changes planned for Julia 1.0 and the new missing type implemented in DataFrames. Things should get better soon. In the meantime, I agree its frustrating. My guess is that there is not a general/simple solution for upgrading DataFrames at the moment.

9

And this PR actually depends on CSV.jl. But hopefully things will start to clear up soon

10

Actually thanks to recent fixes in DataFrames 0.11.4, RDatasets will work again soon with that DataFrames version (albeit with deprecation warnings). See https://github.com/JuliaLang/METADATA.jl/pull/12808.

2 Likes
11

Thanks to everybody who worked towards this solution. Gadfly, Vega, ECharts (my package, currently unregistered) and many others will be able to move foward!

1 Like
12

This fix just got merged into METADATA, which now allows all the packages I care about (i.e. maintain) to move forward with RDatasets/DataFrames 0.11. Thanks everyone!

1 Like
13

Great! No such luck for me though. Will wait for the next Julia Pro and hope that DataFrames 0.11 is included by default.

14

I just spent around 20 minutes figuring this out (again, was somehow silently downgraded when I was not paying attention), removing some packages that depended on Gadly did the trick. Eg do a

grep Gadly ~/.julia/v0.6/**/REQUIRE

or similar to find them.

The fact that a single (albeit important) package can lead to such intricate dependency problems is somewhat disconcerting. I it wasn’t for Pkg3, I would be concerned.

1 Like
15

I think part of the problem was the side-edit to METADATA to cap all packages using DataFrames to 0.10, rather than let people experience broken packages. The effect of this, of course, was to make it really confusing as to why downgrades kept occurring.

16

I disagree. The trend of saying that you support arbitrary future versions of a package is just nonsense. Getting a clear error that a package is not supported is much better than getting run time errors or possibly silently getting wrong results.

Pkg3 will make it much easier to create multiple environments but it cannot help with trying to simultaneously use packages with incompatible dependencies. The only solution to that (ever) will just be to upgrade the packages that are incompatible.

2 Likes
17

You’re reading more into my comment than I was saying. My comment is around the editing of METADATA to cap all of the packages. As a user, I can always go to GitHub and look at what REQUIRE says…except in the case of the DataFrames change.

Any package that depended on DataFrames became out-of-sync in the REQUIRES file. Only if you were saavy enough to read METADATA would you be able to understand why a package was forcing a downgrade (rather than be able to look at the source code)

It’s a decision that was made, and so be it. I’m just saying that it caused me some confusion until I understand how all the packages were capped.

2 Likes
18

Sorry for reading too much. I totally agree that it is (very) confusing when the REQUIRE file in the package and METADATA are out of sync.

In fact, the reason this happens in the first place is that people play loose with their upper bounds, and when a breaking change happens, the upper bounds have to be “backported”. I think in the future, we need to be more careful with this to prevent this situation.

3 Likes
19

What’s the best practice here? Should packages always cap their dependancies at the next (potentially nonexistent) semver that indicates a breaking change? When that next version comes out, the version check can always be relaxed later if everything is ok, correct? Asking for a friend.

20

I’m not sure it’s fair to say that people “play loose” with upper bounds.
It’s simply unclear which upper bounds to put when you release a package. You will either have to revise them up in the future if a new compatible release comes out, or upper-bound them when the new release breaks compatibility. Both of these are unpleasant.

I don’t see an easy solution to this – maybe using the automated JuliaCI testing infrastructure to set such bounds more automatically?

2 Likes