The future, array API, and so on

This question came up at the SpeechBrain 2023 summit, and I figured it warrants some open discussion and thought. In short: what will it take for librosa to be compatible with the Array API? Motivation here is that doing so would make librosa backend-agnostic and ease interoperability with other packages, platforms (GPU), etc.

The way I see it, the answer is both "a lot" and "not much".

A lot

We make pretty extensive use of several upstream libraries. Specifically:

• Numpy
• Scipy
• numba
• soundfile
• soxr

We also make not-so-extensive use of some other upstream libraries:

• sklearn
• audioread
• matplotlib

I don't think we can plausibly make an attempt at array API until the upstream libraries have implemented it. There could be some exceptions here, eg if the interaction from librosa is one-directional (reading from soundfile, writing to matplotlib for example), we can probably wedge in some appropriate conversions and not have to worry.

However, the core functionality will require at least numpy, scipy, and numba to be on board. Some relevant discussion threads:

• Numpy: https://github.com/numpy/numpy/issues/21135
• Scipy: https://github.com/scipy/scipy/issues/18286
• Numba: https://github.com/numba/numba/issues/9164

So if I understand the state of things (at present) correctly, we're talking about at least numpy 2.0, and probably a ways after that while midstream libraries (scipy, numba) catch up.

Not much

On the other hand, the work we would have to do once our upstream dependencies are in place ought to be rather superficial. In most cases, I expect it to involve swapping object methods (x.sum()) for package functions (np.sum(x)) or similar trivial changes.

There will also have to be some modifications to our input validators and type checkers, obviously.

Some caveats

Things that could cause difficulty include:

• Sparse arrays. For the sake of speed, we have a few spots where we bypass the scipy API and operate directly on the underlying data structures for things like index manipulation (eg util.__shear_sparse). These might cause trouble for us.
• Type hints: could obviously get ugly. Uglier.
• String data: string dtypes are technically out of scope for the array api. We use these extensively in unit conversion and music notation as this was the cleanest way to get dimension- and type-preserving implementations via numpy.vectorize. It's not clear yet (to me) how this will shake out with an abstracted array API backend.

---

There are probably more issues at hand than those noted above, and I'd like to hear from folks if there are major concerns that I'm missing.

From my perspective, I think it makes sense to let this cook a while, and perhaps make it the milestone feature for a 2.0 release of librosa. (Where 1.0 will be the stabilized release of the 0.x series once we wrap up all major outstanding features, which at this point are vanishingly few!)