Package size and scope?

Internals & Design
21

I do not understand the problem associated with a 0.9 s loading time that the user do once per session. (?). Am I missing something? I usually do a regular package loading and then work for the whole day on other functions or when finished, my programs can run for days/months and 0.9 s on top of this timescale seems more than reasonable.

2 Likes
22

As packages are organized as github repos, i think a major point to think about is contributors and maintainers:
If packages are too small in terms of functionality, they may not attract a critical mass of contributors and users to write issues or PRs.
This may have a negative impact on the willingness of the maintainer to actively work on the package.

I think it may be a good idea to split up packages if the original maintainer cannot cope with the size of the package and other contributors are willing to maintain their part of the package.

I think we should rather err on the large side of packages in order to guarantee a critical mass of users.

5 Likes
23

Well, organizations also help with this if they are good about building teams to maintain the full set of packages.

24

Yea, all the time. Worst is when I’m on a flight or slow internet. // or in front of an audience/boss

Then I’m like, why did someone include all of StatsBase just so they could pick a random element

also, I got these numbers for running a fresh using:

julia> @time using Pluck
INFO: Recompiling stale cache file /Users/dan/.julia/lib/v0.6/Pluck.ji for module Pluck.
  2.074299 seconds (1.79 M allocations: 90.924 MiB, 2.02% gc time)

julia> @time using StatsBase
INFO: Recompiling stale cache file /Users/dan/.julia/lib/v0.6/StatsBase.ji for module StatsBase.
  5.130169 seconds (160.26 k allocations: 9.352 MiB)
1 Like
25

I think it would be neat if some of these huge packages could be split into sub- packages that could be used by themselves. As you say: bringing in a huge package with multiple package dependencies just so that I can use a single function out of it does not seem an optimal use of anyone’s time.

26

The current package managers is very slow, comparing numbers from it is pointless. As an example, here is a speed comparison between the new one and the old one:

17 Likes