Hello, I just do the Coursera lectures about Deep Leanring by Andrew Ng, and want to implement what I learned in Julia (lecture is in Python), so I can check my understanding. So there is a small neural net, and it takes time, and I change the numerical type from Float64 to Float32 and get like 30%…

Btw. Why is it allocating memory in the first place?

[image] zenon: Why is it allocating memory in the first place Don’t benchmark in the global scope. Put that in a function and run twice.

Float16 is in an odd place right now. For every calculation the Float16 is first converted to a Float32 in which the computation is performed and then converted back to Float16. This is necessary since most hardware has no native support for Float16. The only hardware where Float16 is really relevan…

Check out how Float16 multiplication is defined: julia> x, y = rand(Float16), rand(Float16) (Float16(0.3818), Float16(0.825)) julia> @which x*y *(a::Float16, b::Float16) in Base at float.jl:372 julia> @edit x*y You’ll see it first converts to Float32, performs the multiplication, and then conver…

@ChrisRackauckas Ah, right. Should have known that. > t32(); 0.158636 seconds > t32(); 0.134481 seconds > t16(); 1.785036 seconds > t16(); 1.782366 seconds for function t16() numType = Float16 A = rand(numType, 10000, 10000) B = rand(numType, 10000, 10000) C = Array{n…

Thank you all for the explanations!

Intel added vector instructions to do conversions to/from 16-bit floats many years ago, and in fact, showed that (because of using half the memory, better cache utilization) that using 16-bit could be faster than 32-bit, for larger operations, and not that much slower for smaller vectors. https://s…

[image] vchuravy: For every calculation the Float16 is first converted to a Float32 in which the computation is performed and then converted back to Float16. This is necessary since most hardware has no native support for Float16. The only har Is this still the case now in June 2022 that Floa…

There is not yet widespread support for Float16 in Floating Point hardware. So yes. However, see the next note by @giordano .

The Apple Silicon CPUs such as the M1 have hardware support for float16

Massive performance penalty for Float16 compared to Float32

General Usage Performance

ChrisRackauckas November 3, 2017, 7:23pm 3

Don’t benchmark in the global scope. Put that in a function and run twice.

CSV Reading (rewrite in C?)

Topic		Replies	Views
Apples to apples comparison of A\b with Float64 and Float16 on A64FX Performance question , linearalgebra	12	862	May 2, 2022
Does float16 run natively on a compatible CPU? General Usage	14	728	July 11, 2024
Time it takes to multiply two floats Numerics float	2	453	August 18, 2022
Lux.jl Chain of Dense layers don't see speed up going from 32 bits -> 16 bits Performance machine-learning	5	981	August 7, 2022
Float16 with AMDGPU GPU	10	394	August 30, 2024

Massive performance penalty for Float16 compared to Float32

Related topics