# RFC: Compressed (base-6) floating point, taking 1/8 the space (new plan 1/64 compression for neural networks)

**URL:** https://discourse.julialang.org/t/rfc-compressed-base-6-floating-point-taking-1-8-the-space-new-plan-1-64-compression-for-neural-networks/87289
**Category:** Numerics
**Tags:** float, neural-network
**Created:** [September 14, 2022, 9:47pm UTC](https://discourse.julialang.org/t/rfc-compressed-base-6-floating-point-taking-1-8-the-space-new-plan-1-64-compression-for-neural-networks/87289 "2022-09-14T21:47:55Z")
**Posts on this page:** 1
**Showing post:** 3

<div class="post-metadata">

### Author: ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)
#### Post date: [September 15, 2022, 11:41am UTC](https://discourse.julialang.org/t/rfc-compressed-base-6-floating-point-taking-1-8-the-space-new-plan-1-64-compression-for-neural-networks/87289/3 "2022-09-15T11:41:32Z")

</div>

This will be a storage format only as described here, but I have other ideas how to work with posits, implicitly, as Float64 until really needing to store, to make faster than Float32.

> [@Posit Standard (2022): Floats/IEEE vs Posits, and ternary math](https://discourse.julialang.org/t/posit-standard-2022-floats-ieee-vs-posits-and-ternary-math/86687/10):
>
> I think we can make posits faster than regular floating-point, in software, on regular CPU, like x86 and ARM (for the vast majority of operations, most that matter?). The trick would be to rely on Julia’s type system to help, by relying on from (code comment) “Posit8, 16 are subsets of Float32” and “Posit32 is a subset of Float64”. Right now: julia\> Posit8(1)+Posit8(1) Posit8(2.0) but if we defer rounding and converting back to Posit8 we can just use Float32 for the calculations: julia\> Po…

I made a new unsigned posit8, because 3 significant _binary_ digits is awfully small, and max 4 fractional bits then at least a bit better.

This is work-in-progress, and as is below dangerous to use (with min or I think also average values stored), as I explain:

Note, unchanged Posit8 has all -16 to +16 integers exact, is are a superset of those numbers, must be where I got 2 significant _digital_ digits from. That’s true, but not for arbitrary XX integer, e.g. 21 or 99.

With that one more bit I suppose I get 0 to 32 exact, then base-30 is more practical, since I want to have 1.0 exact… but it’s only exact up to 0.9, 1.0 and then 1 1/3 (and I thought I would get 1 2/3 exact but seemingly not) but then no more.

Then I can no longer store the average, so I store the min number (likely must store the max number too). I should be able to get some more extra bits for the remaining 6 or 5 numbers. That’s an unfinished idea.

If I store min and max, I can store then in [min, max] or [max, min] order giving me one more bit to work with and use somewhere and can calculate the average from that. I can scale the difference numbers to get some more fractional bits.

As is:

```julia
julia> UPosit8(n) = UInt8((reinterpret(UInt16, Posit16(n)) << 1) >> 8)

julia> UPosit8_to_Float64(n) = Float64(reinterpret(Posit16, UInt16(n) << 7)) # UInt8((reinterpret(UInt16, Posit16(n)) << 1) >> 8)

julia> function encode_8p(tuple; base = 30)
         (a, b, c, d, e, f, g, h) = tuple .* base//2
         m = min(a, b, c, d, e, f, g, h); # mid = (a + b + c + d + e + f + g + h) / 8
         (Posit8(m), UPosit8.((b - m, c - m, d - m, e - m, f - m, g - m, h - m)))
       end

julia> function decode_8p((min, (b, c, d, e, f, g, h)); base=30) # , T=Posit16)
         b, c, d, e, f, g, h = UPosit8_to_Float64.((b, c, d, e, f, g, h))
         min = Float64(min)
         (min, b + c + d + e + f + g + h, b + min, c + min, d + min, e + min, f + min, g + min, h + min) ./ base//2
       end

julia> function decode_8p((min, b, c, d, e, f, g, h); base=30) # , T=Posit16)
         (a, b, c, d, e, f, g, h) = Float64.((b + c + d + e + f + g + h, b + min, c + min, d + min, e + min, f + min, g + min, h + min)) ./ base//2
       end

Alternative, that should be safe:
julia> function encode_8p(tuple; base=30, T=Posit16)
         (a, b, c, d, e, f, g, h) = tuple .* base//2; # mid = (a + b + c + d + e + f + g + h) / 8
         T.((a, b, c, d, e, f, g, h))
       end

```

Note, the first two values are just debug values, I’ve not yet recovered the first value, but it should be possible:

```julia
julia> decode_8p(encode_8p((-0.0, 1.3, 3.0, 4.0, 12.0, 3.0, 5.0, 0.3); base=30); base=30)
(0.0, 27.9, 1.2, 2.933333333333333, 4.0, 11.733333333333333, 2.933333333333333, 4.8, 0.3)

```

As you can see 0.3 exact as promised, and 1 + 1//3 (both unlike in regular floats or posits) but if I change/ lower the min value, then values change:

```julia
julia> decode_8p(encode_8p((-5.0, 1.3, 1 + 1//3, 12.0, 1 + 2//3, 0.9, 5.0, 0.3); base=30); base=30)
(-5.333333333333333, 54.4, 0.5333333333333333, 0.5333333333333333, 10.666666666666666, 1.0666666666666667, 0.5333333333333333, 4.266666666666667, -0.5333333333333333)

```

e.g. 0.3 changes to -0.533 which is kind of bad… 🙂 I think that’s because of the nature of the posit system (and low precision, and catastrophic cancellation), so working with (low-bit) posits has been illuminating, but I think I can work around this and make posits user-friendly.

No, I don’t worry about signed-zero personally, and that’s the least of my worries now. 🙂 I was just informing about posits in general, not just my system based on them, asking if others really want them. Then it would be a deal-breaker and I might as well abandon my investigation.

---

_[View the full topic](https://discourse.julialang.org/t/rfc-compressed-base-6-floating-point-taking-1-8-the-space-new-plan-1-64-compression-for-neural-networks/87289)._