# Calculate precision

**URL:** <https://discourse.julialang.org/t/calculate-precision/7484>\
**Category:** General Usage\
**Created:** [December 3, 2017, 10:57pm UTC](https://discourse.julialang.org/t/calculate-precision/7484 "2017-12-03T22:57:40Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 3, 2017, 10:57pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/1 "2017-12-03T22:57:40Z")

</div>

say I want a number x up to 10^6 decimal precision using BigFloat. What is the value I need to specify in setprecision() ??  
How do I calculate it?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [December 3, 2017, 11:36pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/2 "2017-12-03T23:36:46Z")

</div>

In general, n\_{10} decimal digits correspond to n\_{2} = n\_{10} \log\_{2}(10) binary digits for the mantissa, so in your case you want to specify

```julia
julia> round(Int, 1e6 * log2(10))
3321928

```

Not sure you’ll have some memory issue with such a large precision, though.

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 3, 2017, 11:42pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/3 "2017-12-03T23:42:46Z")

</div>

Well that would mean that all the n2 bits are used to represent the .xxxx… positions but that is not the case.  
Some fraction is reserved for the exponents

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [December 3, 2017, 11:59pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/4 "2017-12-03T23:59:23Z")

</div>

`3321928` is the precision you have to specify with `setprecision`, the size of the mantissa.

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 4, 2017, 12:11am UTC](https://discourse.julialang.org/t/calculate-precision/7484/5 "2017-12-04T00:11:17Z")

</div>

Ah…Maybe I completely missunderstand then…setprecision(n) does specify the precision of the mantissa only to be n bits?  
How big is then my full number in bits (sign + exponents)

When I type:  
eps( big(1.0) )  
I get  
4.6566128731e-10  
which is  
2^-31  
while I specified setprecision(32)  
Where is the 1 bit going?

Isn’t the mantissa only everything on the right of the dot ?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [December 4, 2017, 12:48am UTC](https://discourse.julialang.org/t/calculate-precision/7484/6 "2017-12-04T00:48:09Z")

</div>

> [@Diger](#):
>
> How big is then my full number in bits (sign + exponents)

The default exponent range, which I think is compiled into MPFR and cannot easily be changed in Julia, is so large as to be effectively infinite for most purposes. (I think it is around 62 bits.)

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 4, 2017, 1:03am UTC](https://discourse.julialang.org/t/calculate-precision/7484/7 "2017-12-04T01:03:08Z")

</div>

Ok. So just to clarify. The sign is then incorporated in setprecision(n), meaning that the precision of the mantissa is n-1 because of the 1 bit for the sign?

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 4, 2017, 10:49pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/8 "2017-12-04T22:49:33Z")

</div>

Have you considered trying it?

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 4, 2017, 10:50pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/9 "2017-12-04T22:50:38Z")

</div>

Trying like in Post No5 ?  
Thats why I asked, to maybe get some confirmation? It could be possible that the 1 bit is also used for sth else…you never know.  
Also the exponents are treated separately which I just found out.

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [December 4, 2017, 11:04pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/10 "2017-12-04T23:04:32Z")

</div>

> [@Diger](#):
>
> Ok. So just to clarify. The sign is then incorporated in setprecision(n), meaning that the precision of the mantissa is n-1 because of the 1 bit for the sign?

No, the extra bit is for the leading 1. The precision is defined as the number of bits in the significand. In the case of `big(1.0)`, the first bit will be the initial “1”, then there will be `n-1` trailing zeros.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 4, 2017, 11:13pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/11 "2017-12-04T23:13:11Z")

</div>

You can read about the structure that MPFR uses here:

> **[GNU MPFR 4.1.0](https://mpfr.loria.fr/mpfr-current/mpfr.html)**
>
> How to install and use GNU MPFR, a library for reliable multiple precision
> floating-point arithmetic, version 4.1.0.

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 5, 2017, 5:16pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/12 "2017-12-05T17:16:46Z")

</div>

Hm…When I look at

> **[Single-precision floating-point format](https://en.wikipedia.org/wiki/Single-precision_floating-point_format)**
>
> Single-precision floating-point format (sometimes called FP32 or float32) is a computer number format, usually occupying 32 bits in computer memory; it represents a wide dynamic range of numeric values by using a floating radix point.
> A floating-point variable can represent a wider range of numbers than a fixed-point variable of the same bit width at the cost of precision. A signed 32-bit integer variable has a maximum value of 231 − 1 = 2,147,483,647, whereas an IEEE 754 32-bit base-2 floating-...

then I presume that no bit for the 1 is really needed. It is always there so implicitly assumed and just the exponents and the mantissa are modified? So why do we need a Bit here?

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [December 5, 2017, 5:44pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/13 "2017-12-05T17:44:03Z")

</div>

Because that’s how it is defined: we use a pretty standard definition of precision to be the number of digits in the significand. This is standard throughout languages and numeric literature, and works for binary, decimal, hexadecimal formats, in both normalised and unnormalised forms.

If you are only looking at binary, normalised numbers, then you are correct in that the leading bit will always be 1, in which case there is no need to represent it in the format. However this is simply an implementation detail, and doesn’t change the definition. Note that we are consistent in this, e.g.

```julia
julia> precision(Float32(1))
24

```

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 5, 2017, 5:49pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/14 "2017-12-05T17:49:07Z")

</div>

consistent with the wikipedia article? Well there the 24th bit is the sign…

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [December 5, 2017, 5:51pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/15 "2017-12-05T17:51:16Z")

</div>

> [@Diger](#):
>
> consistent with the wikipedia article? Well there the 24th bit is the sign…

I meant consistent within Julia, but the article is also in agreement in that it states that

> Thus only 23 fraction bits of the significand appear in the memory format, but the total precision is 24 bits

It doesn’t say anything about the 24th bit being the sign.

---

<div class="post-metadata">

**Author:** ![Diger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/diger/32/36786_2.png) [@Diger](https://discourse.julialang.org/u/Diger)\
**Post date:** [December 5, 2017, 6:16pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/16 "2017-12-05T18:16:55Z")

</div>

No it doesn’t say it explicitly, but apart from the exponents it’s the 24th bit (actually in the picture the 31st, but that is just a position)  
In Julia a Float32 has  
23bits for the mantissa  
8 for the exponents, right?  
1 for the 1 (as you said above)  
makes 32. So now we are missing the sign?

Anyway…Maybe I can cope with it if it is just a defintion (the word precision as you said in the number of significands) where however the most significant is basically always 1.

---

<div class="post-metadata">

**Author:** ![simonbyrne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simonbyrne/32/19_2.png) [@simonbyrne](https://discourse.julialang.org/u/simonbyrne)\
**Post date:** [December 5, 2017, 6:30pm UTC](https://discourse.julialang.org/t/calculate-precision/7484/17 "2017-12-05T18:30:03Z")

</div>

No, there is still a bit for the sign. A better way of thinking about it is that the significand is 24 bits, but can be stored using only 23 bits (what the article calls the _fraction_) since the leading bit is always 1.