# \[ANN\] FieldFlags.jl: Bitfield-like structs

**URL:** <https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594>\
**Category:** Package Announcements\
**Tags:** bitfields, bitpacking\
**Created:** [June 20, 2023, 8:45am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594 "2023-06-20T08:45:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 20, 2023, 8:45am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/1 "2023-06-20T08:45:10Z")

</div>

Hi everyone! I’m happy to announce [FieldFlags.jl](https://github.com/Seelengrab/FieldFlags.jl), a small package without dependencies for creating bitfield-like structs, packing integers of various bitsizes into the smallest possible space. The current stable release, after some private experimentation & testing, is v0.3.8.

This package is mostly useful if you want to pack e.g. 2 3-bit numbers into one struct, and only have it take up as much space as necessary instead of (at minimum) two bytes if the two numbers are stored as an `UInt8` each.

Features include:

- Explicit marking of padding bits of various sizes
- Optional mutability
- Subtypability of the created structs
- Good discoverable errors reported by JET.jl
- Dot-access, just like in regular structs
- Efficiency
  - in particular, elimination of error branches and good optimization by the compiler; I can get the compiler to emit the efficient [bextract](https://www.felixcloutier.com/x86/bextr) instruction on my machine.

For more information, check out [the documentation](https://seelengrab.github.io/FieldFlags.jl/stable/) and give the [examples](https://seelengrab.github.io/FieldFlags.jl/stable/examples/) a look! 🙂

---

<div class="post-metadata">

**Author:** ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)\
**Post date:** [June 20, 2023, 10:55am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/2 "2023-06-20T10:55:57Z")

</div>

Very cool package!

I wonder why padding bits are undefined. Guaranteeing that they are zero makes a lot of things easier: Structs can be compared more cheaply (instead of masking before compare, mask when constructing) and bit-for-bit serialized and deserialized, and more predictably cast to another bitstype. I would also suspect (though I am not sure) that it would be very cheap to do so, perhaps even zero-cost.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 20, 2023, 11:10am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/3 "2023-06-20T11:10:15Z")

</div>

It’s a tradeoff. I have to mask incoming data on setting fields either way, because I mustn’t accidentally overwrite other fields! I want to leave padding undefined because I’m not sure about the exact/best semantics for them yet; for julia itself, these are of course not padding bits at all, but rather “well defined” bits that are part of the internal object. This is part of an internal inplementation detail though - the way this currently works doesn’t allow things like SROA (because it’s “one object” from the POV of both Julia and LLVM), leaving them semantically undefined leaves me the option for swapping out the internal implementation later on, should something better come along (perhaps true LLVM level structs?). For the same reason, conversion from a `@bitfield` to some other bitstype is unsupported - that would observe padding; only the conversion from some known integer bitstype like `Bool` or `Int` to a `@bitfield` is supported.

In practice of course, they are going to be zeroed, it’s just not something that I want to expose as guaranteed API, because it locks me into the current design 🙂 There’s also [this related issue about equality and hashing](https://github.com/Seelengrab/FieldFlags.jl/issues/6), which currently does exactly what you describe.

---

<div class="post-metadata">

**Author:** ![rfourquet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rfourquet/32/3610_2.png) [@rfourquet](https://discourse.julialang.org/u/rfourquet)\
**Post date:** [June 20, 2023, 11:35am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/4 "2023-06-20T11:35:34Z")

</div>

Looks super useful!  
A couple of question/remarks:

1. Is there a particular reason for not merging the two main macros, `@bitfield` and `@bitflags` (I could see “clarity” but wondered whether there was something else)
2. I can see the appeal of the `x:3` syntax, but i would personally prefer something like `x::3` and/or `x::UInt32::3` (to also specify the type returned by getproperty), as `::` is already used to specify types (and specifying the bit width can be seen as an extension of that)
3. (more of a feature request) i use more and more `@kwargs`, and it would be lovely if `@bitfield` supported it, and was printed as e.g. `MyBits(a= 0x1, b= 0x2, c= true)` instead of `MyBits(a: 0x1, b: 0x2, c: true)`, such that `show` output can be also used as input.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 20, 2023, 11:46am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/5 "2023-06-20T11:46:06Z")

</div>

> [@rfourquet](#):
>
> Is there a particular reason for not merging the two main macros, `@bitfield` and `@bitflags` (I could see “clarity” but wondered whether there was something else)

They actually use the same code under the hood already; the reason for not merging them further is that (in a potential future of multiple abstract subtyping) we could have the “every field is a single bit” invariant encoded in the type of the object as well. It’s also a syntactic reminder that flags don’t have a width other than the single bit!

> [@rfourquet](#):
>
> I can see the appeal of the `x:3` syntax, but i would personally prefer something like `x::3` and/or `x::UInt32::3` (to also specify the type returned by getproperty), as `::` is already used to specify types (and specifying the bit width can be seen as an extension of that)

[This issue](https://github.com/Seelengrab/FieldFlags.jl/issues/9) will be of interest to you then 🙂 The `a:3` syntax is borrowed from C & C++, which this package is very much inspired by. I personally also prefer not to pun on type assertion/annotation syntax; Core language syntax like that should, in my opinion, be left alone in macros (unless you’re dealing with types).

> [@rfourquet](#):
>
> (more of a feature request) i use more and more `@kwargs`, and it would be lovely if `@bitfield` supported it, and was printed as e.g. `MyBits(a= 0x1, b= 0x2, c= true)` instead of `MyBits(a: 0x1, b: 0x2, c: true)`, such that `show` output can be also used as input.

That’s a nice feature request - can you open an issue so I don’t forget about it? I originally wanted to print it with `:` to remind people that it’s not a regular struct and that putting arguments into the constructor will truncate them if necessary. I can see the appeal for a kwargs constructor though (defaulting to 0 for unspecified fields, presumably).

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 20, 2023, 11:58am UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/6 "2023-06-20T11:58:33Z")

</div>

> [@rfourquet](#):
>
> Is there a particular reason for not merging the two main macros, `@bitfield` and `@bitflags` (I could see “clarity” but wondered whether there was something else)

Actually, that’s also a bug - this currently doesn’t error, although it should:

```julia
julia> @bitflags struct Foo
           a:2
           _
           b
       end

julia> Foo |> methods
# 3 methods for type constructor:
 [1] Foo()
     @ none:0
 [2] Foo(t::Foo_fields)
     @ none:0
 [3] Foo(a::Union{Bool, Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8}, b::Union{Bool, Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8})
     @ none:0

```

Note the last method, taking two arguments `a` and `b`. I’ll add an issue & put a fix up soon-ish.

EDIT: [Issue is here](https://github.com/Seelengrab/FieldFlags.jl/issues/12)

---

<div class="post-metadata">

**Author:** ![mrufsvold](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mrufsvold/32/31600_2.png) [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Post date:** [June 20, 2023, 12:06pm UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/7 "2023-06-20T12:06:22Z")

</div>

Really awesome!

Could you explain for a noob what the purpose/benefit of adding padding to a struct is? Like when would I benefit from putting padding between field A and B?

Would you ever consider allowing `@bitfields` to accept other `FieldFlag` types as fields? It would be nice in Agent-Based Models to store flag attributes of an agent in a nested struct without losing the tight packing.

For example

```julia
@bitflags mutable struct BasicNeeds
       hungry
       tired
       injured
end

@bitfields mutable struct Health
    basic_needs::BasicNeeds
    disease_status:2
ebd

struct Person
    health::Health
end

person.health.basic_needs.hungry = true

```

It would be awesome for `Health` to only take 8 bits, not 16.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 20, 2023, 12:14pm UTC](https://discourse.julialang.org/t/ann-fieldflags-jl-bitfield-like-structs/100594/8 "2023-06-20T12:14:58Z")

</div>

> [@mrufsvold](#):
>
> Could you explain for a noob what the purpose/benefit of adding padding to a struct is? Like when would I benefit from putting padding between field A and B?

My motivation for this package is some (not yet revealed - you’ll have to wait until juliacon 😉 ) microcontroller shenanigans - suffice it to say that being able to place padding bits in exact bit-precise locations is very useful there (and saves a tremendous amount of code when constructing these objects…).

> [@mrufsvold](#):
>
> Would you ever consider allowing `@bitfields` to accept other `FieldFlag` types as fields? It would be nice in Agent-Based Models to store flag attributes of an agent in a nested struct without losing the tight packing.

> [@mrufsvold](#):
>
> It would be awesome for `Health` to only take 8 bits, not 16.

Hm, I know of that problem, yes. The issue is that that would more or less require knowledge that some type annotation already is a `@bitfield` or `@bitflags` type, which I currently don’t track - and getting that information in a macro is difficult, because all I see in there is a `Symbol`. I could just assume that is a type, but that would require `eval` in the macro (or some form of “bitflags/field” registry object in FieldFlags.jl itself, which would need to be guarded against collisions somehow).

It sounds like a good & doable feature, but it would require some engineering & special handling; probably best to do after [Custom field type annotations · Issue #9 · Seelengrab/FieldFlags.jl · GitHub](https://github.com/Seelengrab/FieldFlags.jl/issues/9) already works. Please open an issue with this though, so I don’t forget and you get a notification when it’s implemented!