# Faster read only memory

**URL:** <https://discourse.julialang.org/t/faster-read-only-memory/32457>\
**Category:** GPU\
**Tags:** arrayfire, cudanative, cuda, memory, memory-allocation\
**Created:** [December 18, 2019, 10:24pm UTC](https://discourse.julialang.org/t/faster-read-only-memory/32457 "2019-12-18T22:24:07Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [December 19, 2019, 7:15am UTC](https://discourse.julialang.org/t/faster-read-only-memory/32457/2 "2019-12-19T07:15:20Z")

</div>

> [@Michael\_Benton](#):
>
> Which tells me it’s a memory access problem, given that to the compiler, the indices x1 and x2 are random (and there aren’t any ways for me to make them ordered or predictable)

The compiler doesn’t matter here, it’s your code that’s slow 🙂 We don’t do the kind of optimizations you seem to expect (changing or optimizing memory access patterns).

To optimize this, you’ll need to look into the GPU’s architectural details and avoid costly operations. For example, you are hitting global memory here all the time, both loading from and writing to it through `in` and `out`. You could restructure your kernel to buffer this data in local memory. You are also doing so in a random pattern, defeating memory memory coalescing. Again, if you buffer locally and read/write to global memory in consecutive chunks, different memory transactions might be able to get merged. See for example [CuArray is Row Major or Column Major?](https://discourse.julialang.org/t/cuarray-is-row-major-or-column-major/7402)

I suggest you look into optimizing bandwidth-bound CUDA kernels (which yours seem to be) and get familiar with the necessary tools (notably nvprof/nsight-compute).

> [@Michael\_Benton](#):
>
> I’ve tested different methods outside of Julia that can do this much faster

I’m not sure what you mean by this. Does an equivalent CUDA C kernel perform better? That would be the only fair comparison.

And FWIW, please try to post runnable snippets. It’s much harder to help if just brainstorming based on pseudocode.

---

_[View the full topic](https://discourse.julialang.org/t/faster-read-only-memory/32457)._