# Is there a package for block-sparse matrix processing?
**URL:** https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067
**Category:** General Usage
**Tags:** array, arrays, sparse, matrices, matrix
**Created:** [October 11, 2025, 8:42am UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067 "2025-10-11T08:42:29Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@photor](https://discourse.julialang.org/u/photor)
#### Post date: [October 11, 2025, 8:42am UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/1 "2025-10-11T08:42:29Z")
As in the title, I mean every block is dense, but there are only a small percentage of non-vanishing blocks.
---
### Author:  [@karei](https://discourse.julialang.org/u/karei)
#### Post date: [October 11, 2025, 9:07am UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/2 "2025-10-11T09:07:22Z")
Here you are
> **[JuliaHub](https://juliahub.com/ui/Packages?q=block%2Bmatri)**
(Just search for the keyword “block matri” here)
---
### Author:  [@photor](https://discourse.julialang.org/u/photor)
#### Post date: [October 11, 2025, 2:35pm UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/3 "2025-10-11T14:35:05Z")
Thanks, but this one is actually the opposite of what I want, unfortunately. It can define block matrices, in which each block can be sparse. In my case, each block is dense.
---
### Author:  [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [October 11, 2025, 5:31pm UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/4 "2025-10-11T17:31:14Z")
Is something like this what you want:
[Introduction · BlockSparseMatrices.jl](https://djukic14.github.io/BlockSparseMatrices.jl/stable/) Or CompressedSparseBlocks.jl?
Note searching for “block matrix” gave many results, unlike “block matri” only one at JuliaHub. and seemingly better without a hyphen.
What kind of storage format do you want? BSR seems to be it or BCSR, or even consider Blocked-Ellpack format:
> **[Accelerating Matrix Multiplication with Block Sparse Format and NVIDIA Tensor...](https://developer.nvidia.com/blog/accelerating-matrix-multiplication-with-block-sparse-format-and-nvidia-tensor-cores/)**
>
> Sparse-matrix dense-matrix multiplication (SpMM) is a fundamental linear algebra operation and a building block for more complex algorithms such as finding the solutions of linear systems…
Do you have some kind of structure? E.g. are there as many blocks per row? It seems better for [blocked] Ellpack:
> **[1. Introduction — cuSPARSE 13.0 documentation](https://docs.nvidia.com/cuda/cusparse/#sliced-ellpack-sell)**
>
> The API reference guide for cuSPARSE, the CUDA sparse matrix library.
> The Sliced Ellpack format is standardized and well-known as the state of the art. ..
> [different section below]
> The Blocked Ellpack format is similar to the standard Ellpack, where the column indices represent two-dimensional blocks instead of a single matrix entry.
Do you want to run on a GPU or not? Then consider cuSPARSE, but I also see cuSPARSELt I wasn’t familiar with. You have access to the former, and seemingly also some access to the latter with CUDA.jl:
> **[Is CUSPARSELT supported? · JuliaGPU CUDA.jl · Discussion #1703](https://github.com/JuliaGPU/CUDA.jl/discussions/1703#discussioncomment-10844611)**
>
> When searching around sparse matrix - dense matrix multiplication, I came across this slides and it mentions a new library dedicated for this type of operation. I know CUDA.jl could solve this SmDm...
> **[cuSPARSELt: A High-Performance CUDA Library for Sparse Matrix-Matrix...](https://docs.nvidia.com/cuda/cusparselt/)**
[https://www.osti.gov/servlets/purl/1366653](https://www.osti.gov/servlets/purl/1366653)
> We examine the implementation of block compressed row storage (BCSR) sparse matrix-vector multiplication (SpMV) for sparse matrices with dense block sub-structure, optimized for blocks with sizes from 2x2 to 32x32, on CPU, Intel many-integrated-core, and GPU architectures. .. We give a set of algorithms that performs SpMV up to 4x faster than the NVIDIA cuSPARSE cusparseDbsrmv routine, up to 147x faster than the Intel Math Kernel Library (MKL) mkl dbsrmvroutine (a single-threaded BCSR SpMV kernel), and up to 3x faster than the MKL mkl dcsrmv routine (a multi-threaded CSR SpMV kernel).
> **[Blocking Sparse Matrices to Leverage Dense-Specific Multiplication](https://ieeexplore.ieee.org/document/10027183)**
>
> Research to accelerate matrix multiplication, pushed by the growing computational demands of deep learning, has sprouted many efficient architectural solutions, such as NVIDIA's Tensor Cores. These accelerators are designed to process efficiently a...
What’s your application? ML/LLMs then see also:
> **[Efficient Mixtures of Experts with Block-Sparse Matrices: MegaBlocks](https://mlfrontiers.substack.com/p/efficient-mixtures-of-experts-with)**
>
> A new compute paradigm makes sparse Mixtures of Experts more scalable than ever before
> One of the most challenging research questions around sparse MoE is therefore how to run it efficiently on machines that were never intended to run sparse computations in the first place.
>
> This week, we’ll meet MegaBlocks ([Gale et al 2022](https://arxiv.org/abs/2211.15841)), the perhaps most efficient and scalable sparse MoE implementation in the industry today. The key idea is to re-formulate sparse MoE as a single block-sparse matrix multiplication instead of multiple dense matrix multiplications by leveraging a data format specifically designed to store and manipulate sparse matrices.
> §5.1.3 describes our hybrid _blocked-CSR-COO_ sparse matrix format, which enables efficient matrix products with sparse input and output operands.
Is it matrix-matrix multiply yoy need, and then both same format or one of dense? Or matrix-vector multiply?
> **[Accelerating Block Sparse Matrix Multiplication on IPUs with PopSparse](https://www.graphcore.ai/posts/accelerating-block-sparse-matrix-multiplication-on-ipus-with-popsparse)**
>
> Introducing PopSparse, a library that enables fast sparse operations on the Graphcore IPU.
---
### Author:  [@photor](https://discourse.julialang.org/u/photor)
#### Post date: [October 12, 2025, 2:19pm UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/5 "2025-10-12T14:19:52Z")
Thank you for so much useful infomation. [BlockSparseMatrices.jl](https://djukic14.github.io/BlockSparseMatrices.jl/stable/) seems to do exactly what I want. I need to check.
I need such data structure for domain decomposition in numerical simulation, typically to solve a large scale 2D elliptic equation by decomposing the computational domain into subdomains, in which we use Chebyshev pseudospectra. CPU is enough for the moment.
---
### Author:  [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [October 12, 2025, 7:46pm UTC](https://discourse.julialang.org/t/is-there-a-package-for-block-sparse-matrix-processing/133067/6 "2025-10-12T19:46:11Z")
It has matrix‑vector product so can I assume you need that (and only that?). I didn’t dig into what format is used, but I’m thinking if you want GPUs later, or even just CUDA, they you might want supporting that, or even both. I believe you could even just use Julia’s default sparse format (for correctness), and the only reason for looking for this is that it would be faster. I’m just thinking should some of these packages better serve most users doing sparse? I even think Julia’s sparse capabilities should just go (eventually, to just live in a package), since Julia itself doesn’t need it, nor most users, and those that do even something else. Just let Julia document some options? What do you think?