[ANN] PackedStructs.jl

Package Announcements
1

I am happy to announce that we open-sourced PackedStructs.jl.

Shamelessly copying from its README:

PackedStructs.jl provides the @packed macro to annotate structs. These structs will pack together types which do not have a native size, i.e. a power-of-two byte size. This is especially useful when using BitIntegers.jl or EmulatedBitIntegers.jl. Accesses to these packed types take some additional CPU cycles in general. Therefore, speed is effectively traded for space. However, the space savings can lead to a better caching behavior. In some situations, packed structs can be both smaller and faster than regular structs.

Usage

To create a packed struct, do, e.g.:

julia> using EmulatedBitIntegers, PackedStructs
julia> @emulate Int4
julia> @packed struct Foo
           first::Int4
           second::Int4
       end

This struct is smaller than it would be without being @packed.

julia> struct RegularFoo
           first::Int4
           second::Int4
       end
julia> Foo |> sizeof
1
julia> RegularFoo |> sizeof
2

This type can be used like a regular struct, so you can create values by

julia> foo = Foo(2, -1)
Foo(2, -1)

and you can access values by

julia> foo.first
2

julia> foo.second
-1

They have the correct type

julia> foo.first |> typeof
Int4

Internally, the bits of the different fields immediately follow each other

julia> (reinterpret(UInt8, foo), foo.first, foo.second) .|> bitstring
("00101111", "0010", "1111")

Overall, I think the package gets as close to a native implementation as you can get in a package. If you use the internal low-level functions (like getfield instead of the (implicit) getproperty), you can see the wrappers leaking, but otherwise usage should feel totally native (as seen above).

Thanks to the nice people in JuliaData, who kindly allowed me to move and maintain the code there.

This package has some similarities to FieldFlags.jl. However, FieldFlags.jl focuses more on logical bits (fields), whereas PackedStructs.jl with EmulatedBitIntegers.jl focuses more on integers.

2

Hi, nice package and thanks for the mention of FieldFlags.jl! How does your package compare to the @bitfield macro from FieldFlags.jl? The intention of that macro is precisely this kind of packing :slight_smile: There’s an open issue for type annotations on fields, which I haven’t had the time to tackle, but it should be fairly straightforward to implement :thinking:

3

I wondered the same. When we created the package around 2 years ago, we didn’t know about FieldFlags.jl. I only learned later about it and wanted to mention it.

There is definitely the difference in the mental model. A bit field and packed integers are semantically something completely different, although the implementation can be identical and it can be the same just looked from different angles. But how, for example, would you transport the difference between and unsigned and a signed integer in a bitfield?

This might be the reason why I haven’t found the package back then. When I look for packing small integers in structs, “flags” is not really what I would look for.

I wanted to go into some details (bitfields are bound to the struct and can’t (currently) live outside, whereas packed structs are just a collection of types, the amount of work being put into PackedStructs.jl to find an optimum grouping of fields efficiently, …) and let Claude Opus 4.8 do a comparison, too, to not forget important points. However, I think Claude’s answer is so good that I want to share it in the following. That being said, if you see possibilities to cooperate or even merge the packages, I would be interested in this.

PackedStructs.jl vs. FieldFlags.jl (the following is from Claude)

Both are Julia packages for tightly bit-packing data into structs to avoid padding, with getproperty/setproperty! overloads making them feel like ordinary structs. They differ substantially in philosophy and what you put in the fields.

Core abstraction

PackedStructs.jl FieldFlags.jl
Macro(s) @packed @bitflags, @bitfield
Field unit Types (Int4, UInt24, Float32, wrapper/nested structs) Bit counts (a:3), or single bits for flags
Mental model “A normal struct, but fields with non-power-of-two types share storage” “C-style bitfield: declare how many bits each field occupies”
Field type back out The original declared type (e.g. foo.first isa Int4) Bool for width-1, otherwise UInt (custom return types are a planned feature) 
# PackedStructs — types drive the layout
@packed struct Foo
    first::Int4
    second::Int4
end

# FieldFlags — bit widths drive the layout
@bitfield struct Foo
    a:1
    b:2
    _:7      # explicit padding
    c:3
end

Where they overlap

  • Both overload getproperty/setproperty! (true getfield won’t work; introspection sees internal storage slots).
  • Both support mutable struct with in-place field assignment.
  • Both round/collapse layout at macro-expansion time, so type parameters are rejected in both (the bit widths must be known statically to compile away the shifts/masks).
  • Both target negligible overhead via constant-folded bitmask operations.

Key differences

1. Padding & sizing. FieldFlags makes padding a first-class concept (_ and _:n) and always rounds the whole struct up to a multiple of 8 bits (a compiler limitation it documents). PackedStructs derives padding implicitly from the types and packs into Pack<B> groups sized to native widths (8/16/32/64).

2. Field richness. This is the biggest divergence:

  • PackedStructs handles arbitrary isbits types — primitives, BitIntegers/EmulatedBitIntegers, wrapper structs for type safety, and nested immutable structs that compose by logical bits. It’s extensible via PackedStructs.pack (e.g. to pack Float32/Char).
  • FieldFlags stores raw integer bitfields and boolean flags only; custom field types are a documented future plan (#9). Assignments silently truncate leading bits.

3. Ecosystem integration. PackedStructs ships a ConstructionBase extension, so Accessors.@set/@reset (and Setfield, BangBang, StructArrays) work — with a documented load-order requirement. FieldFlags has zero dependencies and emphasizes simplicity, JET debuggability, and subtyping (<: MyAbstract).

4. Dependencies. PackedStructs depends on EmulatedBitIntegers, ExproniconLite, PrecompileTools (+ optional ConstructionBase). FieldFlags is dependency-free.

When to pick which

  • FieldFlags.jl — C-style bitfields/flag registers, hardware/protocol bit layouts, where you think in bit counts, want zero deps and a minimal surface.
  • PackedStructs.jl — you think in types (especially EmulatedBitIntegers/BitIntegers), want fields to round-trip as their declared types, need nested structs or float/char packing, or want Accessors.jl support.

PackedStructs’ own README already nods to this: “FieldFlags.jl uses a similar approach, but focusing more on logical bits (fields) than on integers.” The inverse framing also holds — PackedStructs focuses on typed values, FieldFlags on bit widths.

4

Well.. Claude certainly has a way of interpreting things in such a way that they don’t at all align with my own mental model of the package :sweat_smile: The summary by the LLM seems more like a dissection based on surface level words than what’s already being tracked in the issue tracker :slight_smile: Either way, thanks for the reply!

5

Hmm, Claude’s summary certainly matched my mental model of both packages. However, I never went through the complete code of FieldFlags.jl, so my mental modal seems to be biased, too. How would you phrase it? Claude might not learn from it, but I probably would. :slight_smile: