[ANN] A New Ecosystem for Sampled Signals: SamplesCore, SoundIO, and WavNative

Package Announcements
1

Hi everyone,

I’m excited to announce a new suite of packages designed for sampled signal communications. These tools enable high-performance, low-latency signal processing entirely in native Julia—from hardware I/O to file storage.

The Core Philosophy: Literate & Surgical

My goal is to provide a unified ecosystem that brings the “literate programming” feel of Images.jl to 1D sampled data, while simultaneously providing surgical control over the signal path with extensive customizability and high code reuse.

The ecosystem is built to feel completely integrated yet highly flexible: data loaded via the file parser flows naturally into the hardware transport and can be probed, inspected, or modified at any stage with a literate feel. By leveraging Julia’s type system, the engine annihilates boilerplate by inferring parameters at compile time—enabling self-documenting code that executes with zero-cost efficiency.

The Initial Stack

  • SamplesCore.jl: The foundation. Just as ImageCore.jl treats a pixel as a single unit, this revolves around the Sample—the “Pixel” of the signal world. It provides the abstract traits and literate types (including native Int24 and Q0f23 support) needed to write generic algorithms that treat a single point in time as a first-class citizen.

  • SoundIO.jl: A high-performance transport built on libsoundio. It uses compile-time specialization to generate optimized, branchless hardware callbacks on the fly.

    • Unified Synchronization: The library provides highly customizable, bidirectional mechanisms for both ends of the streaming spectrum:

      • The “Flow” Model (Frozen Audio Buffer): A unique, versatile mechanism that turns standard Julia arrays into managed ring buffers.

      • The “Reactive” Model (Audio Callback Synchronizer): Engineered for the lowest possible latency, notifying Julia tasks for every hardware event.

    • Extensible Infrastructure: Beyond the built-in mechanisms, users can define custom synchronizers and transport layers by inheriting from <:SoundIOSynchronizer, benefiting from the library’s type-specialized pipeline.

    • Interactive Design: Includes a “GUI-like” text interface for REPL-based hardware exploration and Symbol-based error handling.

  • WavNative.jl: A high-performance, pure-Julia .wav parser. It leverages core traits to provide unfiltered, literate input streams. It is designed for speed and raw data integrity, ensuring data is loaded with no modifications unless you explicitly ask for conversion.

Current Status & Roadmap

These libraries are currently in pre-release. I have intentionally “cranked up” performance optimizations, prioritizing raw access and speed.

  • Roadmap: While WavNative currently focuses on a fast-path for mapping disk data to arrays, future updates will include a streaming I/O implementation and allow user-defined functions to process signals while allowing high performance and self-documenting code.

  • Beyond Audio: The architecture is built for any regularly sampled telemetry, from Biomedical signals (ECG/EEG) and Industrial sensors to Low-latency control loops in robotics.

I’d love to hear your thoughts on the architecture and see how these might fit into your DSP and communication workflows!

Acknowledgments

I would like to thank:

  • Andrew Kelley, the developer of libsoundio, for such a fantastic and robust C library.

  • The Images.jl developers for providing the architectural guidance and the concept of literate programming that inspired this ecosystem.

  • The Julia developers for creating such an awesome language that truly allows us to “have our cake and eat it too”—achieving high-level expressiveness without sacrificing low-level performance.

9 Likes
2

Current Work-in-Progress & Opportunities for Contribution

I wanted to share what’s currently on the workbench and where this ecosystem is headed next. If any of these areas interest you, I’d love to collaborate!

:building_construction: In Development: WavPackHybrid.jl

A native-Julia implementation for WavPack Hybrid Lossless files. This codec is particularly interesting because it pairs a space-efficient lossy stream with a correction file to achieve full reconstruction.

  • Signal Processing Utility: The underlying techniques—including error feedback loops, sign-sign LMS adaptation for linear prediction, and adaptive weight factors that update on the fly—provide a robust foundation for high-performance signal processing outside of audio.

  • Modular Logic: Core encoding and decoding functions are exposed as standalone units, allowing for high code reuse.

  • Composable Pipelines: The modular design supports any user-defined workflow—from high-level file operations to direct streaming, where modular components are used to build pure-Julia callbacks that decode and process signals on the fly.

:satellite_antenna: Beyond Audio: Custom Synchronization Demos

I am preparing demo applications for non-audio use cases (IMU, SDR, Medical telemetry) that leverage the Extensible Infrastructure of SoundIO.jl.

  • Bespoke Mechanisms: These demos show how to build custom synchronization for hardware that doesn’t follow a standard audio clock.

  • Real-World Precision: A look at using specialized transport layers to handle high-frequency sensor data with literate, high-performance code.

:handshake: How to Contribute

As these packages are in pre-release, community input is invaluable:

  • Performance & Stability: Help test these optimizations across different OS backends (ALSA, WASAPI, CoreAudio, etc.).

  • Safety & Zero-Cost Abstractions: I am looking to provide a choice between surgical performance and safer implementations (similar to the @inbounds philosophy) without compromising on zero-cost principles.

  • New Codecs: If you’re interested in building high-performance, native-Julia parsers or encoders using the SamplesCore traits, let’s talk!

I’m excited to see how the community uses and extends these tools. Feel free to open an issue or a PR on any of the repos if you want to jump in!

3 Likes
3

Sounds really interesting. I’ll love to test. Where do I find the code and documentation?

1 Like
4

The code is available here:

The documentation is still work in progress, and I’d appreciate any help in making the documentation and api user friendly!

1 Like
5

This is fantastic! A few years ago I found myself wondering how to get started with audio generation in Julia (specifically I wanted to implement a SAME decoder just for kicks) and I didn’t know where to start. This sounds like precisely the place.

1 Like
6

Thanks a lot for sharing @mj2984 .
It’s for sure interesting to have a libsoundio based alternative to PortAudio.jl.

Here’s how libsoundio compares itself to PortAudio: libsoundio-vs-PortAudio.
Can you say a few words on how SoundIO.jl compares to SampledSignals.jl and PortAudio.jl from JuliaAudio?

I wanted to test by just playing a sound like in this example:

using SoundIO

fs = 44100
t = 0:1/fs:1
sine = Int32.(round.(sin.(2π * 440 .* t) .* (2^31 - 1)))
audio = vcat(sine', sine')

SoundIOContext() do ctx
    enumerate_devices!(ctx)
    dev = first(filter(d -> !d.is_input && d.is_raw, ctx.devices))
    play_audio(audio, fs, ctx, dev, :Int32Little)
end

However, I get ERROR: UndefVarError: play_audio not defined in Main, and I can’t find where play_audio() is defined.

7

Sorry I forgot to update the front page. The API changed after I first created the frontpage. I have updated the frontpage and developer guide and committed version 0.1.8. The module no longer exports a play audio function (stays closer to a lower-level library) and users no longer have to create a context (custom contexts are possible but the general implementation is written around ease of use)

You should see the following examples

  1. Loopback test
using SamplesCore, SoundIO
function get_sound_devices()
    enumerate_sound_devices!() # Gets OS permissions and scans available sound devices
    all_devices = list_sound_devices() # Displays available sound devices
    # Getting raw (unprocessed) devices for input and output. Here it connects to the first numbered device it found.
    input_device  = filter(d -> d.is_input && d.is_raw, all_devices)[1]
    output_device = filter(d -> !d.is_input && d.is_raw, all_devices)[1]
    return input_device,output_device
end

function start_loop(input_stream,output_stream)
    input_sync  = input_stream.sync[].stream::FrozenAudioStream # Synchronizer that notifies status at every buffer atoom crossing.
    # For simplicity we only track the input stream status. If required the output stream status can also be tracked but not part of this example.
    println("🎤 Starting Capture...")
    start!(input_stream) # Starts capturing audio and storing into the buffer
    wait(input_sync) # We wait till it sends the first notification (roughly buffer_atom_time + some δ ahead)
    println("🔊 Starting Playback (Real-Time Loopback)...")
    start!(output_stream) # Starts playing back audio
    try
        exchange::FrozenAudioExchange = @atomic input_sync.exchange
        while exchange.status == CallbackJuliaDone # Poll the status and ensure it is stable
            wait(input_sync)
            exchange = @atomic input_sync.exchange
        end
    finally
        close(input_sync)
        destroy_sound_stream_unsafe(input_stream)
        destroy_sound_stream_unsafe(output_stream)
    end
end

sampling_frequency = 48000
buffer_atom_time = 0.5 # Notifications are sent every buffer_atom_time seconds.
total_buffer_atoms = 10 # It goes through 10 such cyles before looping back. (in many cases 2-3 is sufficient)
shared_data = zeros(Sample{2, Int16}, Int(buffer_atom_time * sampling_frequency), total_buffer_atoms) # Pre allocate the array for buffering.

input_device, output_device = get_sound_devices()
# Opening streams. This gets connections to a sound device and ensures the device is active. Opening a stream with this API locks the shared_data array from being garbage collected.
input_stream  = open(input_device,  (shared_data, false), sampling_frequency)
output_stream = open(output_device, (shared_data, false), sampling_frequency)

Threads.@spawn start_loop(input_stream,output_stream)
# At any moment you could peek into shared_data and see the actual data.
  1. Playing Wav files
using SamplesCore, WavNative, SoundIO
function play_audio(audio_data::AbstractArray{T}, sample_rate::Integer, device::SoundIODevice) where {T<:Union{Number,Sample}}
    stream = open(device, (audio_data, false), sample_rate) # The stream captures the audio data from being Garbage collected.
    buffer_stream = stream.sync[].stream::FrozenAudioStream
    start!(stream) #println("🔊 Playback started. Press Ctrl+C to stop.")
    try
        exchange::FrozenAudioExchange = @atomic buffer_stream.exchange
        while exchange.status == CallbackJuliaDone
            wait(buffer_stream)
            exchange = @atomic buffer_stream.exchange
        end
    finally
        close(buffer_stream)
        destroy_sound_stream_unsafe(stream) # Stop stream playback when done or interrupted
    end
end

function play_music(sound_file::String,audio_device::SoundIODevice)
    audio_data,sample_rate = audioread(sound_file,false)
    play_audio(audio_data,Int(sample_rate),audio_device)
end

sound_file = raw"sound_file.wav"
enumerate_sound_devices!()
audio_device = filter(d -> (!d.is_input) & (d.is_raw), list_sound_devices())[1]
println("🎶 Playing: $sound_file")
play_music(sound_file,audio_device)
println("Finished!")

Note: API is still being updated (the context hiding is new implementation, and some of the unsafe operations here will be made safer soon).

At the C level, libsoundio prioritizes lightweight, “raw” access (avoiding automatic format conversions eg: Int24 to Int32), while portaudio is designed to be more convenient but raw access will take more steps (and even then it’s not guaranteed the data is sent without modulation).

The SoundIO.jl package:

  1. Avoids forced format conversions or channel remapping from the libsoundio philosophy but provides zero-cost abstractions for boilerplate—such as auto-inferring formats from data types while still allowing for explicit manual overrides.
  2. Leans toward a functional style but utilizes statefulness where necessary. Certain structs are intentionally immutable with internal ref fields to prevent unintentional modification; these are typically managed by safer helper functions while still providing raw access for developers.
  3. Focuses on zero-boilerplate, maximum speed, and ease of extensibility. It is trivial to implement your own Julia-based Synchronizer or Callback for custom conversions (sample rate, format, or channel broadcasting), keeping the core library transparent and easy to extend.
  4. Is purpose-designed to drive any device that imitates a sound interface, handling your data exactly as it is given (* in raw mode).

SamplesCore.jl maps ImageCore.jl-style features onto sampled signals. It treats a “sample” as the equivalent of a “pixel,” providing methods like channelview and zero(T) for samples. I am just starting to look at SampledSignals.jl and I feel that we can have a merged feature set of both, like Unitful for each axis.

1 Like
8

That sounds like a great idea to demonstrate custom callbacks.

I currently have a developer guide for this (wip) : SoundIO.jl/docs/DeveloperGuide.md at main · mj2984/SoundIO.jl · GitHub

Note: The package is architected to take maximum use of Julia features (for expressive and high performing code), but might be overwhelming (and dense) for users not familiar with the language features. I’ll likely create another document that’s like a “primer for features of Julia for scalable, concise and fast codebase”.

9

Thanks for the update @mj2984.

The first wav file I found apparently does not work. It fails with:

julia> play_music(sound_file,audio_device)
ERROR: Audio data must have at least 2 dimensions: (Channels, Frames, ...)

I guess I need to find one in stereo.

10

You have a few options. If your sound device is 1 channel you can just reshape the array to (1,size(array)). I’ll update wavnative to be consistent with this.

If you have 2 channels in your sound device it can be done by concatenating the same signal have an array of (2,size(array)).

At present there is no check on ensuring device channels match input array channels (will be implemented soon), and it can cause incorrect sound (will feel like num channel times multipled in frequency because each channel receives a fraction of the data but runs at same sample rate in this configuration).

function play_music(sound_file::String,audio_device::SoundIODevice)
    audio_data,sample_rate = audioread(sound_file,false)
    audio_broadcast = repeat(reshape(audio_data, 1, :), inner = (2,1))
    play_audio(audio_broadcast,Int(sample_rate),audio_device)
end

Should do the trick for the mono wav file assuming the sound device is 2 channels. If the audio device is single channel the repeat operation can be ignored.

EDIT::
Updated WavNative.jl to be consistent (and few other tweaks both for user and developers). Now you can use the previous code for mono files as well.

function play_music(sound_file::String,audio_device::SoundIODevice)
    audio_data,sample_rate = audioread(sound_file,false)
    play_audio(audio_data,Int(sample_rate),audio_device)
end

I’ll leave the other code with broadcast as is, in case anyone else has issues with custom arrays that are 1D. The interface expects at least 1-D array of Samples or 2-dimensional array of other types.

1 Like
11

(post deleted by author)