I’ve lost track of where we are with this. I know @stevengj and @sacha have done a lot of work on this – where do we stand? What’s left to do? What are the blockers? Are we on track to have full dot-syntax fusion by feature freeze at the end of month?
2 Likes
A blocker AFAICS is that we need broadcast methods that handle arbitrary combinations SparseMatrixCSC and scalars. There is already a discussion on how to tackle this, but hasn’t been implemented (more generic broadcast over a single sparse matrix by Sacha0 · Pull Request #19239 · JuliaLang/julia · GitHub).
Is that the only blocker?
Is that the only blocker?
I’d guess some of the @sacha PRs (Pull requests · JuliaLang/julia · GitHub) have something to be sorted out, but I’m not sure what it is in each case. Helping to review those might be necessary.
I don’t think that issue is a blocker for dot operators.
There are a few other cleanups in #17623 that are needed, but I’m hopeful that this will all be ready by the end of the month.
2 Likes
If I recall and understand the plan correctly, #17263 becomes viable when generic broadcast[!] over combinations of sparse matrices and scalars works. With #19518 (generic broadcast[!] over sparse matrices) and the effort @pabloferz mentioned (handling combinations including both sparse matrices and scalars) we should hit that mark.
#19518 should be mergeworthy (with touchups after feature freeze). Re. broadcasting over combinations including scalars, I have work beyond the thread @pabloferz linked. Anticipate more on that front in the next few days.
Additional, related items that should not block #17263 (and completion of which prior to feature freeze may or may not be realistic) but should happen at some point:
The semantics of map[!]/broadcast[!] over sparse vectors should match those for sparse matrices. broadcast[!] over combinations of scalars, sparse vectors, and sparse matrices should also work. map!(f, X) should work for both sparse vectors and matrices.
A vestige (?) from the aggressive zero purging era, several sparse matrix and sparse vector routines automatically trim (resize!) their operands’ storage to the number of entries those operands store (without need). This behavior makes controlling allocation difficult when manipulating sparse matrices, e.g. forcing pessimistic reallocation in broadcast routines. This behavior should probably change.
(I have a day-job deadline at the end of the month as well, but I may be able to complete some of these additional items prior to feature freeze.)
Discussion of generic map[!]/broadcast[!] over structured matrices I leave for another day, but IIRC quasi-consensus was that such operations should behave like those for and yield sparse matrices. That might be achieved (if inefficiently) with little code and effort by promoting structured matrices to sparse matrices in map[!]/broadcast[!] operations.
Best!
4 Likes