# \#precision

**URL:** https://discourse.julialang.org/tag/precision/451.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [\[ANN\] MathChecker.jl: Find problems in floating-point calculations](https://discourse.julialang.org/t/ann-mathchecker-jl-find-problems-in-floating-point-calculations/139261)

<div class="topic-metadata">

**Author:** [@moble](https://discourse.julialang.org/u/moble)\
**Replies:** 6\
**Last updated:** [September 10, 2026, 4:01pm UTC](https://discourse.julialang.org/t/ann-mathchecker-jl-find-problems-in-floating-point-calculations/139261 "2026-09-10T16:01:50Z")

</div>

I’m always surprised this isn’t a bigger topic of conversation among Julia users: how to find where problems are coming from in numerics. The gap between our pretty equations and algorithms on one hand, and practical …

---

## [\[ANN\] PrecisionCarriers.jl: Easy Detection of Floating Point Precision Loss](https://discourse.julialang.org/t/ann-precisioncarriers-jl-easy-detection-of-floating-point-precision-loss/130670)

<div class="topic-metadata">

**Author:** [@AntonReinhard](https://discourse.julialang.org/u/AntonReinhard)\
**Replies:** 6\
**Last updated:** [February 6, 2026, 2:52pm UTC](https://discourse.julialang.org/t/ann-precisioncarriers-jl-easy-detection-of-floating-point-precision-loss/130670 "2026-02-06T14:52:08Z")

</div>

I recently registered a new package I’ve been working on: PrecisionCarriers.jl. It’s designed to let you check existing code very quickly and easily for loss of precision in its calculations. I added a macro inspired by…

---

## [Kahan summation in \`sum\`?](https://discourse.julialang.org/t/kahan-summation-in-sum/102723)

<div class="topic-metadata">

**Author:** [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Replies:** 13\
**Last updated:** [December 29, 2024, 1:04am UTC](https://discourse.julialang.org/t/kahan-summation-in-sum/102723 "2024-12-29T01:04:03Z")

</div>

I was under the impression that sum(itr) uses Kahan summation, but after inspecting the Julia repo it appears that it calls mapreduce(identity, add\_sum, itr). And add\_sum is basically just + with some promotions for smal…

---

## [How to do high precision linear algebra with \`exp\` related functions?](https://discourse.julialang.org/t/how-to-do-high-precision-linear-algebra-with-exp-related-functions/122114)

<div class="topic-metadata">

**Author:** [@singularitti](https://discourse.julialang.org/u/singularitti)\
**Replies:** 2\
**Last updated:** [November 1, 2024, 9:21am UTC](https://discourse.julialang.org/t/how-to-do-high-precision-linear-algebra-with-exp-related-functions/122114 "2024-11-01T09:21:18Z")

</div>

Is there a library that supports exp, sin, cos, etc., on matrices with precisions higher than Float64? Here’s my test code: julia\> A = rand(BigFloat, 10, 10); julia\> exp(A) ERROR: MethodError: no method matching exp!(:…

---

## [Handling large values correctly](https://discourse.julialang.org/t/handling-large-values-correctly/117537)

<div class="topic-metadata">

**Author:** [@vh94](https://discourse.julialang.org/u/vh94)\
**Replies:** 4\
**Last updated:** [July 27, 2024, 9:27pm UTC](https://discourse.julialang.org/t/handling-large-values-correctly/117537 "2024-07-27T21:27:35Z")

</div>

I repeatedly run into issues handling large values (values beyond 1e+15), and couldn’t find a clear answer on how to address those. Consider following comparison: (1.0e+16 + 1 + 1 + 1) \> 1.0e+16 will wrongfully return…

---

## [Roundoff error in variance of array of constant value](https://discourse.julialang.org/t/roundoff-error-in-variance-of-array-of-constant-value/116890)

<div class="topic-metadata">

**Author:** [@jgr](https://discourse.julialang.org/u/jgr)\
**Replies:** 7\
**Last updated:** [July 11, 2024, 10:31pm UTC](https://discourse.julialang.org/t/roundoff-error-in-variance-of-array-of-constant-value/116890 "2024-07-11T22:31:28Z")

</div>

In the code below, var(M) is not 0, I guess the reason might be computation error, but I wonder if anyone knows how to deal with this? Thanks in advance. M = fill(5.421241248937501521, (1,10000)) M = Float64.(M) var(M) …

---

## [Avogadro's number as a floating-point value](https://discourse.julialang.org/t/avogadros-number-as-a-floating-point-value/111379)

<div class="topic-metadata">

**Author:** [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Replies:** 18\
**Last updated:** [March 11, 2024, 2:10am UTC](https://discourse.julialang.org/t/avogadros-number-as-a-floating-point-value/111379 "2024-03-11T02:10:14Z")

</div>

Ok, on this topic: What on earth were the SI people thinking when they specced the current value of avogadro’s number? julia\> Int128(6.02214076e23) 602214075999999987023872 I get that they wanted to spec a nice round n…

---

## [I will never understand floating-point arithmetic](https://discourse.julialang.org/t/i-will-never-understand-floating-point-arithmetic/111342)

<div class="topic-metadata">

**Author:** [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Replies:** 17\
**Last updated:** [March 8, 2024, 4:05pm UTC](https://discourse.julialang.org/t/i-will-never-understand-floating-point-arithmetic/111342 "2024-03-08T16:05:15Z")

</div>

I will never understand float point arithmetic… julia\> a = 10000 \* 7 /100 700.0 julia\> b = 10000 \* 0.07 700.0000000000001 julia\> c = 1000 \* 0.69999999999999999 700.0 (but at list I am aware of the danger…)

---

## [And Julia keeps amazing me: high precision computation](https://discourse.julialang.org/t/and-julia-keeps-amazing-me-high-precision-computation/107740)

<div class="topic-metadata">

**Author:** [@Ronis\_BR](https://discourse.julialang.org/u/Ronis_BR)\
**Replies:** 17\
**Last updated:** [December 19, 2023, 1:12am UTC](https://discourse.julialang.org/t/and-julia-keeps-amazing-me-high-precision-computation/107740 "2023-12-19T01:12:06Z")

</div>

Hi! This application is probably very common for some people, but it is the very first time I really needed high precision computation (BigInt and BigFloat). I am implementing an algorithm for designing frozen orbits i…

---

## [Why is there a discrepancy in the value of sqrt(2) from Wolfram Alpha and Julia?](https://discourse.julialang.org/t/why-is-there-a-discrepancy-in-the-value-of-sqrt-2-from-wolfram-alpha-and-julia/107461)

<div class="topic-metadata">

**Author:** [@chandra](https://discourse.julialang.org/u/chandra)\
**Replies:** 5\
**Last updated:** [December 11, 2023, 11:55pm UTC](https://discourse.julialang.org/t/why-is-there-a-discrepancy-in-the-value-of-sqrt-2-from-wolfram-alpha-and-julia/107461 "2023-12-11T23:55:14Z")

</div>

On the Wolfram Alpha website, I get this approximation for sqrt(2) 1.4142135623730950488016887242096980785696718753769480731766797379... With the Julia REPL, I get sqrt(2) 1.4142135623730951 and convert(BigFloat, sq…

---

## [Float64 Lux model (Loss stuck at 1e-8)](https://discourse.julialang.org/t/float64-lux-model-loss-stuck-at-1e-8/103135)

<div class="topic-metadata">

**Author:** [@Baba\_Yara\_Fahiz](https://discourse.julialang.org/u/Baba_Yara_Fahiz)\
**Replies:** 1\
**Last updated:** [August 24, 2023, 7:28pm UTC](https://discourse.julialang.org/t/float64-lux-model-loss-stuck-at-1e-8/103135 "2023-08-24T19:28:05Z")

</div>

It’s very easy to convert a Flux model to Float64 using code like so: model = Chain(Dense(N, 32, tanh), Dense(32, 16, tanh), Dense(16, 8, tanh), Dense(8, 1, s…

---

## [How to increase precision of solution using Roots.jl and ArbNumerics.jl](https://discourse.julialang.org/t/how-to-increase-precision-of-solution-using-roots-jl-and-arbnumerics-jl/102850)

<div class="topic-metadata">

**Author:** [@roi.holtzman](https://discourse.julialang.org/u/roi.holtzman)\
**Replies:** 28\
**Last updated:** [August 18, 2023, 8:59pm UTC](https://discourse.julialang.org/t/how-to-increase-precision-of-solution-using-roots-jl-and-arbnumerics-jl/102850 "2023-08-18T20:59:06Z")

</div>

I am trying to increase precision for a numeric solution using Roots.jl using ArbFloat (of ArbNumerics.jl). But, when I decrease the tolerance, the outcome does not change. I keep comparing it to the numerical solution…

---

## [Arbitrary Precision BesselK (ArbNumerics.jl)](https://discourse.julialang.org/t/arbitrary-precision-besselk-arbnumerics-jl/53343)

<div class="topic-metadata">

**Author:** [@Neutrino155](https://discourse.julialang.org/u/Neutrino155)\
**Replies:** 12\
**Last updated:** [August 16, 2023, 3:58pm UTC](https://discourse.julialang.org/t/arbitrary-precision-besselk-arbnumerics-jl/53343 "2023-08-16T15:58:54Z")

</div>

Hello! I am fairly new to Julia and have been working with ArbNumerics.jl BesselK function as part of evaluating a complicated response function that can be expanded in a series of BesselK functions. The arguments it en…

---

## [How to properly work with changing precision in ArbNumerics](https://discourse.julialang.org/t/how-to-properly-work-with-changing-precision-in-arbnumerics/102775)

<div class="topic-metadata">

**Author:** [@roi.holtzman](https://discourse.julialang.org/u/roi.holtzman)\
**Replies:** 2\
**Last updated:** [August 13, 2023, 9:03pm UTC](https://discourse.julialang.org/t/how-to-properly-work-with-changing-precision-in-arbnumerics/102775 "2023-08-13T21:03:17Z")

</div>

I do not understand how I should properly work with ArbNumerics. I tried setting the number of digits at the beginning of my code, so that later I easily change the number of digits for all of my variables. digits = 20…

---

## [Numerically stable derivative of logerf without catastrophic cancellation](https://discourse.julialang.org/t/numerically-stable-derivative-of-logerf-without-catastrophic-cancellation/92992)

<div class="topic-metadata">

**Author:** [@user22](https://discourse.julialang.org/u/user22)\
**Replies:** 4\
**Last updated:** [January 15, 2023, 4:56pm UTC](https://discourse.julialang.org/t/numerically-stable-derivative-of-logerf-without-catastrophic-cancellation/92992 "2023-01-15T16:56:08Z")

</div>

I need to evaluate the derivative of a logerf function fun with respect to x within the widest possible range of x. At the moment, the derivative is calculated through the function dfun. using LogExpFunctions using Fini…

---

## [Arbitrary Precision Optimization](https://discourse.julialang.org/t/arbitrary-precision-optimization/92803)

<div class="topic-metadata">

**Author:** [@csmarra](https://discourse.julialang.org/u/csmarra)\
**Replies:** 2\
**Last updated:** [January 11, 2023, 7:21pm UTC](https://discourse.julialang.org/t/arbitrary-precision-optimization/92803 "2023-01-11T19:21:19Z")

</div>

Hi, I am trying to perform a function minimization with BigFloat numbers. However, it seems that I cannot get it to work. I am using the BlackBoxOptim.jl package. Is there a way to do that? As an example, I tried to do…

---

## [Inconsistant precision?](https://discourse.julialang.org/t/inconsistant-precision/92674)

<div class="topic-metadata">

**Author:** [@dgagnon](https://discourse.julialang.org/u/dgagnon)\
**Replies:** 13\
**Last updated:** [January 9, 2023, 11:45am UTC](https://discourse.julialang.org/t/inconsistant-precision/92674 "2023-01-09T11:45:16Z")

</div>

Hello I am a bit confuse as to how precision is assigned or indicated in Julia. Consider: julia\> a = Complex(1.0, 0.0) 1.0 + 0.0im julia\> typeof(a) ComplexF64 So by default a Complex number will be built with Float6…

---

## [Is it possible to set a floating point precision while evaluating jldoctest blocks in docstrings?](https://discourse.julialang.org/t/is-it-possible-to-set-a-floating-point-precision-while-evaluating-jldoctest-blocks-in-docstrings/59528)

<div class="topic-metadata">

**Author:** [@jishnub](https://discourse.julialang.org/u/jishnub)\
**Replies:** 5\
**Last updated:** [December 6, 2022, 12:31pm UTC](https://discourse.julialang.org/t/is-it-possible-to-set-a-floating-point-precision-while-evaluating-jldoctest-blocks-in-docstrings/59528 "2022-12-06T12:31:23Z")

</div>

I’m not too familiar with Documenter, so this might have a solution that I’m not aware of. Is it possible to set a filter to compare only the first n digits of floating-point numbers in a jldoctest block? Either of a glo…

---

## [Computation precision with Float](https://discourse.julialang.org/t/computation-precision-with-float/91180)

<div class="topic-metadata">

**Author:** [@dgagnon](https://discourse.julialang.org/u/dgagnon)\
**Replies:** 8\
**Last updated:** [December 3, 2022, 9:39pm UTC](https://discourse.julialang.org/t/computation-precision-with-float/91180 "2022-12-03T21:39:32Z")

</div>

I see strange behavior for the computation of a very simple formula: julia\> rounding(Float64) RoundingMode{:Nearest}() julia\> p=0.8 0.8 julia\> 1-p 0.19999999999999996 I have the same result with Julia 1.8.0 on Windows1…

---

## [Zombies in biological ODE : why is my solver not sticking to zero?](https://discourse.julialang.org/t/zombies-in-biological-ode-why-is-my-solver-not-sticking-to-zero/90409)

<div class="topic-metadata">

**Author:** [@iago-lito](https://discourse.julialang.org/u/iago-lito)\
**Replies:** 25\
**Last updated:** [November 21, 2022, 9:00am UTC](https://discourse.julialang.org/t/zombies-in-biological-ode-why-is-my-solver-not-sticking-to-zero/90409 "2022-11-21T09:00:32Z")

</div>

Hi numerical ODE solvers enthusiasts, we’ve got a problem here that we think necessitates your kind input :slight\_smile: Here are two related questions: The following ODE system represents biological species biomasses B…

---

## [Compare numbers at the stated precision](https://discourse.julialang.org/t/compare-numbers-at-the-stated-precision/86719)

<div class="topic-metadata">

**Author:** [@tp2750](https://discourse.julialang.org/u/tp2750)\
**Replies:** 9\
**Last updated:** [September 5, 2022, 9:29am UTC](https://discourse.julialang.org/t/compare-numbers-at-the-stated-precision/86719 "2022-09-05T09:29:10Z")

</div>

In tests I’ll like to be able to write the following and the test to pass: @test aresame(pi, 3.14) The following works: @test isapprox(pi, 3.14, atol = 0.005) But then I need to adjust the absolute tolerance manual…

---

## [Calculation Pi^Pi^Pi in hexadecimal in High precision](https://discourse.julialang.org/t/calculation-pi-pi-pi-in-hexadecimal-in-high-precision/51785)

<div class="topic-metadata">

**Author:** [@rbdaus](https://discourse.julialang.org/u/rbdaus)\
**Replies:** 61\
**Last updated:** [July 13, 2022, 2:32pm UTC](https://discourse.julialang.org/t/calculation-pi-pi-pi-in-hexadecimal-in-high-precision/51785 "2022-07-13T14:32:51Z")

</div>

Hello! I have a math question. In Wolfram Language I can type this: BaseForm\[N\[Pi^Pi^Pi, 100000\], 16\] and get VERY FAST the result of up to 100,000 hexadecimal digits for the expression “Pi^Pi^Pi” (a few seconds) B…

---

## [Exponentiation and machine precision](https://discourse.julialang.org/t/exponentiation-and-machine-precision/79024)

<div class="topic-metadata">

**Author:** [@amrods](https://discourse.julialang.org/u/amrods)\
**Replies:** 18\
**Last updated:** [June 8, 2022, 9:16am UTC](https://discourse.julialang.org/t/exponentiation-and-machine-precision/79024 "2022-06-08T09:16:10Z")

</div>

In some simulations I am encountering a problem due to machine precision. I have to conduct an exponentiation operation on a number that lies in (0, 1). That number comes from a random number generator, while the exponen…

---

## [Changing precision for a dual number calculation with ForwardDiff](https://discourse.julialang.org/t/changing-precision-for-a-dual-number-calculation-with-forwarddiff/80723)

<div class="topic-metadata">

**Author:** [@andrew\_s](https://discourse.julialang.org/u/andrew_s)\
**Replies:** 3\
**Last updated:** [May 9, 2022, 5:37am UTC](https://discourse.julialang.org/t/changing-precision-for-a-dual-number-calculation-with-forwarddiff/80723 "2022-05-09T05:37:17Z")

</div>

I am relatively new to Julia, but have a lot of experience with modern Fortran. I am minimising an objective function, and I wish to use ForwardDiff.jl to calculate the derivatives. The issue is that there is one calcula…

---

## [Numerical precision](https://discourse.julialang.org/t/numerical-precision/78749)

<div class="topic-metadata">

**Author:** [@F-YF](https://discourse.julialang.org/u/F-YF)\
**Replies:** 15\
**Last updated:** [March 31, 2022, 12:38am UTC](https://discourse.julialang.org/t/numerical-precision/78749 "2022-03-31T00:38:00Z")

</div>

Recently I had a small problem with Julia that confused me. The code is shown below: Julia\> x=2.0e-10 2.0e-10 Julia\> -1.0\*(-0.5\*x^3 - 3.0\*x^2 - 7.5\*x - 7.5)\*exp(-x)/x^6 + 1.0\*(0.5\*x^3 - 3.0\*x^2 + 7.5\*x - 7.5)\*exp(x)/x…

---

## [\[ANN\] ExtremeFloats.jl: Floats with extended range of exponents](https://discourse.julialang.org/t/ann-extremefloats-jl-floats-with-extended-range-of-exponents/78164)

<div class="topic-metadata">

**Author:** [@a5sk6n](https://discourse.julialang.org/u/a5sk6n)\
**Replies:** 1\
**Last updated:** [March 20, 2022, 4:20pm UTC](https://discourse.julialang.org/t/ann-extremefloats-jl-floats-with-extended-range-of-exponents/78164 "2022-03-20T16:20:37Z")

</div>

ExtremeFloats.jl is a new package that allows you to perform floating point operations with a very large range of exponents. Float64 offers 11 bits for the exponent, ExtremeFloat gives you 64 bits. This is especially use…

---

## [Higher precision leads to incorrect results?](https://discourse.julialang.org/t/higher-precision-leads-to-incorrect-results/75248)

<div class="topic-metadata">

**Author:** [@chrisoffner](https://discourse.julialang.org/u/chrisoffner)\
**Replies:** 19\
**Last updated:** [January 28, 2022, 7:32pm UTC](https://discourse.julialang.org/t/higher-precision-leads-to-incorrect-results/75248 "2022-01-28T19:32:55Z")

</div>

I’m working through Arthur & Vassilvitskii’s \\Omega(n^2) lower bound proof for the iterative closest point (ICP) algorithm in \\mathbb{R} (Theorem 3.4 in the paper) and thought it’d be neat to visualize their construction…

---

## [Eigenvalues and extended precision](https://discourse.julialang.org/t/eigenvalues-and-extended-precision/68119)

<div class="topic-metadata">

**Author:** [@zmoitier](https://discourse.julialang.org/u/zmoitier)\
**Replies:** 10\
**Last updated:** [September 14, 2021, 1:31pm UTC](https://discourse.julialang.org/t/eigenvalues-and-extended-precision/68119 "2021-09-14T13:31:15Z")

</div>

I am interested in computing a small proportion (between 3 and 10% of the matrix size) of the eigenvalues of a matrix with extended precision. As a toy example, let’s take the matrix A define by: T = Complex{Float128} N…

---

## [Round Off Error in Iterator Arithmetic](https://discourse.julialang.org/t/round-off-error-in-iterator-arithmetic/63401)

<div class="topic-metadata">

**Author:** [@GpuCoder](https://discourse.julialang.org/u/GpuCoder)\
**Replies:** 1\
**Last updated:** [June 22, 2021, 10:50pm UTC](https://discourse.julialang.org/t/round-off-error-in-iterator-arithmetic/63401 "2021-06-22T22:50:49Z")

</div>

When I first started writing Julia (i.e., recently) I created vectors of intermediate results and operated upon them, that is complicated versions of: temp0 = \[k for k = 1:3\] 3-element Vector{Int64}: 1 2 3 temp1 = m…

---

## [The fastest way to calculate matrix inversion](https://discourse.julialang.org/t/the-fastest-way-to-calculate-matrix-inversion/62892)

<div class="topic-metadata">

**Author:** [@yingqiuz](https://discourse.julialang.org/u/yingqiuz)\
**Replies:** 9\
**Last updated:** [June 15, 2021, 12:14am UTC](https://discourse.julialang.org/t/the-fastest-way-to-calculate-matrix-inversion/62892 "2021-06-15T00:14:32Z")

</div>

Hi all I need to calculate the inverse of a positive definite matrix H of the form H = (X’ \* X + Diagonal(d)). X is a flat matrix, having (dominantly) more columns than rows. d consists of positive, large values. The m…

[Next page](https://discourse.julialang.org/tag/precision/451.md?match_all_tags=true&page=1&tags%5B%5D=precision)