There is a PR that attempts to address this: Generalize `rand` for `OffsetArrays` e.g. `rand(Bool, 0:2)` by goretkin · Pull Request #36558 · JuliaLang/julia · GitHub
3 Likes
OffsetArrays is a nice user-level way to handle that, but it’s not a nice library-level way to handle such an input. For example, in a differential equation solver, optimization library, etc., you just stick to linear algebra, broadcast, iteration with eachindex, start, first, and those primitives that do not assume 1:length(a). If you do this, then your whole library will support OffsetArrays even if your library does not depend on OffsetArrays or make reference to them. In that sense, the mathematical algorithm is abstracted to work with any array type reguardless of their indexing structure, which is different many more general than “any instance of an OffsetArray”.
So then,
If everyone is writing algorithms that, instead of doing if typeof(x) <: OffsetArrays but instead making it naturally offset-aware, the beauty is that if you want different syntax you can just make it. Make a new library OffsetArrays2.jl with an @OA real, dimension(-1:1, 0:4) :: A = 0.0 if you want, the world is your oyster. You’d then be able to use that in any general library. So we can have 10 packages exploring the syntax, some trying to make new better general syntax while others simply making arrays offset for very specific purposes.
9 Likes
Right now - not a whole lot. My current project aside from my job is - finding a job I can sustain at. But, over the past ~2 years I’ve actually tried to work on a lot of this stuff. I’m not out here just throwing wishes around and expecting people to work for my interests… That said, am I a strong contributor? No. I can’t be given my circumstances. I’m not a grad student, I don’t work for a university, and my professional life is on the rocks(for now). For a hobbyist, I think I’m doing okay though…
Some ongoing projects:
- ChemometricsTools.jl - is a user-focussed API. It’s not complete and needs a lot of revisions, but it’s been stable for a long time now and isn’t at 1.0 status. About once every 4 months someone messages me a “thank you” for it. It’s not awesome, I wish I could do it over again right now(but can’t). Basically the goal was to prototype a full-featured DS API which didn’t exist at the time. The mistake I made was I did this while I was still learning the basics of the language. This effort will be mostly washed away due to the sands of politics - but I needed it, and still use it sometimes, so I did it :). Funny side note - one company is using this package but they didn’t even take my resume :P.
- Sherlock.jl - is a high-level package/code introspection tool. Work in progress.
- JuliaTutor.jl - to bring in new people. Kind of like SWIRL. Been stuck on one aspect of it for a while. I should make a thread about it thanks for the reminder.
- ImageChipper.jl & ObjectDetectionStats.jl - to fill in some of the data science gaps, not complete but functional enough for some tasks.
- Nonpublic - physics packages that pre-dates my current employment - to fill in some of the natural science stuff that I have a background in. I have to wait a while to release any of it due to potential(not likely but possibly) conflicts with my noncompete.
-Bringing new people in via social media and other things - I’ve gotten a few people, some were sorta high profile people at FAANG companies(and elsewhere), to at least try it. - GUI stuffs - I have some posts on here about trying to interop Julia to say Java for standing up GUI’s. Other stuff too, but no I haven’t contributed to making GUI’s in Julia.
Stuff I’m not really working on but tried
- Less exclusive culture. I put my real name on VIMKiller, and some other hacky stuff just so people would be comfortable screwing around learning, sharing, and contributing things, and not having to be a hyper professional subject matter expert programming genius’s to use the language to make cool/fun/useful things. I tried kick-starting an organization for “makers” under the same guise but - no real adoption, and I have run out of time for it for now.
- I’ve tried at my current job to get the language adopted. I’ve failed. Politics are too heavy, and yes most people prefer to use Python or even Matlab for what I’d describe as “comfort”.
- I haven’t contributed to package compilation - it’s not my wheelhouse, but it’s been really important for my current job :). I’m a cheerleader for the project.
@Tero_Frondelius - I’m not here to throw stones. It’s never anything major for me. But, there is an attitude some people have some of the time that makes people not want to contribute/be involved. It’s not just me, I have talked to people who are pretty senior in the community about it. It doesn’t rub off on me - I have a Ph.D and know how to ignore it. Some pretty influential package maintainers have confided in me that they were considering or had made plans to leave the community at some point or another for greener pastures. Important to state the factors given were purely social - not technological. Technologically Julia is a no brainer. Joining a small community and contributing, can be pretty political.
18 Likes
I am not sure why you think this is a requirement (of the culture, not the language).
People at various levels of expertise ask for and get help here just fine.
1 Like
I don’t think it’s a requirement. But that, some people, will hold up that expectation. In some cases it can be valid to hold others to that standard. In others, maybe not. The point of that section was to state something more simple “contributing to a language, its culture, or its ecosystem shouldn’t be intimidating. I’ve made efforts to remind people of this, and to make new comers more comfy by just having fun.” A good deal of my posts here have the word “Fun” in it for a reason :).
Like I said, people here are overall great, but I was trying to add to the list of things I’ve done and tried to do for the ecosystem and community to show @lobingera that I wasn’t a complete “noncontributor” or whatever was at stake there…
2 Likes
That’s interesting, I wonder what kind of situations would have people leave the Julia community for social reasons. I have seen heated arguments but mostly felt those were due to individuals having difficulty communicating in a non-escalating way. I have seen plenty of “harsh” feedback to newcomers, but never mean, just very straight to the point, maybe too much so. It’s interesting because there is almost no way for the stewards to control the community’s general behavior, it’s more a side effect of what kind of crowd the language attracts I would say. But of course it would be a pity if that made it less approachable for newcomers. So I value “fun” posts a lot
8 Likes
Something that I didn’t see mentioned that I’m excited about is what Julia’s concurrency/parallelism will evolve into (when it becomes mature and with the structured concurrency properly implemented). There is kind of an empty space for high performance dynamic languages with easy to use lightweight threads. Erlang/Elixir focus on minimal latency makes it not appropriate for many heavy processing tasks, and outside of the dynamic spectrum Go also didn’t get much traction in this area and the most common tool seems to be Scala + Akka (which isn’t a particularly easy language to work with).
Julia’s metaprogramming means that once the low level tools are there, one could create a very concise framework that provides what Akka does for Scala, a way to better scale large projects, as you can structure your system as many simple services/actors that focus on one task, add new layers of safety through let-it-crash strategies and supervision trees, and easy to understand way of parallelizing tasks. Depending on how it evolves, it could be the key to have Julia taking a piece of the data engineer and services pie, since that kind of structure translate well into the development of APIs/web frameworks, workflow management systems and stream processing.
6 Likes
Didn’t realize OffsetArrays committed blatant type-piracy. That seems rather unfortunate in a package that might get pulled in by a user without package authors being aware of it; seems likely to cause extremely difficult to track-down bugs. 
9 Likes
This abstraction clearly has advantages, similar to writing (in mathematical text) a sequence \{x_n\} and a sum \sum_n x_n without specifying its starting index. (In the end, it really doesn’t matter if we sum over n \in \mathbb{N} or n \in \mathbb{Z}_+ or even n \in \{-2,-1\} \cup \mathbb{Z}_+ etc.)
I’m not sure if I would like the added verbosity introduced by keywords like eachindex and start, but I will try it out. Thank you for your more detailed explanation.
Yes, but Fortran-style arbitrary indexing is such a fundamentally nice feature.
So does that bring us back to …?
2 Likes
Writing for j in axes(A,2) is shorter than 1:size(A,2), and more general. Likewise for (i,x) in pairs(V) instead of enumerate(V). Many commands like reshape also accept axes instead of sizes.
8 Likes
There is no need for OffsetArrays to be part of the core language, because packages should just write index-independent code (e.g. use eachindex, axes etc).
6 Likes
Yes, I think that was also the point made by Chris Rackauckas, and while I understand the principle, I think that in certain cases, expressions using indices are simply clearer and closer to mathematical prose. So, I will have to see.
Writing index-independent code is fine.
However, if you want to use non-1-based-indexing, you need an implementation somewhere.
As pointed out, it seems that the external implementation provided by OffsetArrays commits type-piracy.
I guess my point was that OffsetArrays could continue to use simple convenient syntax with type-piracy:
OA = zeros(-1:1, 0:4)
Or OffsetArrays could change implementation to avoid type-piracy with less -convenient syntax:
OA = zeros(OffsetArray, -1:1, 0:4) # or similar
Or OffsetArray implementation could be brought into Base.
Or something else …
1 Like
Sorry why would you call that “selfish” ?
It’s important that things like zeros(eltype(A), axes(A,1), axes(B,2)) work, when B isa OffsetArray, without mentioning the package. That doesn’t in fact require piracy, because these axes are a special type:
julia> B = zeros(0:4, -1:1);
julia> typeof(axes(B,2))
OffsetArrays.IdOffsetRange{Int64,Base.OneTo{Int64}}
julia> axes(axes(B,2))
(-1:1,)
julia> axes(-1:1)
(Base.OneTo(3),)
So I guess it’s sort-of licensed piracy, for convenience of typing zeros(-1:1), and because manipulations easily lose the special type:
julia> typeof(intersect(axes(B,1), axes(B,2)))
UnitRange{Int64}
How much would change if it were made a standard library?
Making it a standard library doesn’t mean it can type pirate freely. It would need to be in Base.
4 Likes
I guess I was thinking about Array (Core), * (Base) and A * B (LinearAlgebra) as an example of standard libraries apparently being granted permission. But perhaps there are plans to put a stop to this.
1 Like
I would say this is something that kind of slipped through the cracks, and no one really knows how to deal with it in a non-breaking way. There is the same with rand() and Random.
4 Likes
I’m hoping we can move away from OffsetArrays to something that supports unique indexing at an axis level (as opposed to specifying this at the level of the array).
1 Like