# Potential Julia 1.3 memory bug. Can someone help me diagnose before I submit an issue?

**URL:** <https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652>\
**Category:** General Usage\
**Created:** [September 11, 2019, 3:13pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652 "2019-09-11T15:13:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 11, 2019, 3:13pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/1 "2019-09-11T15:13:52Z")

</div>

I am trying to define a Vector type where it has a pointer to a compressed vector and a pointer to an uncompressed vector. To save RAM initially only the compressed vector is stored in memory. Upon accessing an element of the vector, the compressed vector gets decompressed and now the uncompressed pointer will point to it. To illustrate

```julia
using Revise
using Blosc
using CmpVectors # defined below

a = rand(Int, 100_000_000)
b = Blosc.compress(a)
c = CmpVector{Int}(b)

c[1]
c[2]

c[1] # invariably crashes 1.3 on Windows 10

```

The above crash Julia 1.3-rc1. I think it might be a bug with Julia but I may not be understanding Julia memory correctly. Is what I am doing OK? Sorry for the long code, but it’s as simple a MWE as I can show.

If I set `a = rand(Int, 3)` then it works and won’t crash. So I think the code is doing thing correctly somewhat.

The full module definition is below

```julia
module CmpVectors

using Blosc

import Base: show, getindex, setindex!, eltype, size

export CmpVector

mutable struct CmpVector{T} <: AbstractVector{T}
    ptr_compressed::Ptr{UInt8}
    ptr_uncompressed::Ptr{T}
    inited::Bool

end

CmpVector{T}(ptr_compress::Ptr{UInt8}) where T= CmpVector{T}(ptr_compress, pointer_from_objref(T[zero(T)]), false)

CmpVector{T}(ptr_compress::Vector{UInt8}) where T= CmpVector{T}(pointer_from_objref(ptr_compress), pointer_from_objref(T[zero(T)]), false)

Base.eltype(cv::CmpVector{T}) where T = eltype(cv.ptr_uncompressed)

Base.size(cv::CmpVector{T}) where T = begin
    decomp(cv)
    size(unsafe_pointer_to_objref(cv.ptr_uncompressed))
end

decomp(cv) = begin
    if !cv.inited
        println("decompressing")
        cv.ptr_uncompressed = Blosc.decompress(
            eltype(cv),
            unsafe_pointer_to_objref(cv.ptr_compressed)
        ) |> pointer_from_objref

        cv.ptr_compressed = pointer_from_objref(UInt8[0])
        cv.inited = true
        println("decompressing: done")
    else
        println("didn't do nothing")
    end
end

Base.getindex(cv::CmpVector{T}, i...) where T = begin
    decomp(cv)
    getindex(unsafe_pointer_to_objref(cv.ptr_uncompressed), i...)
end

Base.setindex!(cv::CmpVector{T}, i...) where T = begin
    decomp(cv)
    setindex!(unsafe_pointer_to_objref(cv.ptr_uncompressed), i...)
end

Base.show(io::IO, A::MIME"text/plain", cv::CmpVector{T}) where T = begin
    if cv.inited
        show(io, A, unsafe_pointer_to_objref(cv.ptr_uncompressed))
    else
        print("data in compressed form; not shown until first used")
    end
end

end # module

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [September 11, 2019, 3:17pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/2 "2019-09-11T15:17:46Z")

</div>

```julia
pointer_from_objref(T[zero(T)])

```

What would keep this array that you created alive (prevent if from being garbage collected)?

Writing code like this is very tricky to get right, why are you using so many pointers and `unsafe` functions?

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 11, 2019, 10:05pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/3 "2019-09-11T22:05:21Z")

</div>

Had a sleep over this. Actually, I don’t think I need the pointers. Thought I was saving some RAM by doing it.

BTW, how do I mark a region of mem and dont let them be garbage collected. I think I just don’t understand how the GC works

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [September 11, 2019, 10:13pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/4 "2019-09-11T22:13:53Z")

</div>

Ram in any GC’d language is use it or lose it. The way you say you care about a piece of memory is by having a struct of some sort that is holding that ram. The reason I (on stackoverflow) and kristov asked “what are you trying to do” rather than giving a more direct answer is because if you want to be able to later fit something into a region of ram, you need to have allocated it, so there’s almost no use case where storing a pointer will save memory, because if that pointer is usable, you must be already using the ram it points to.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 11, 2019, 10:26pm UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/5 "2019-09-11T22:26:28Z")

</div>

I just realised I don’t need pointers because I can just assign a single element array to save RAM and only replace it with large array when I need them

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [September 12, 2019, 12:40am UTC](https://discourse.julialang.org/t/potential-julia-1-3-memory-bug-can-someone-help-me-diagnose-before-i-submit-an-issue/28652/6 "2019-09-12T00:40:48Z")

</div>

> [@xiaodai](#):
>
> BTW, how do I mark a region of mem and dont let them be garbage collected.

FYI checkout [`GC.@preserve`](https://docs.julialang.org/en/latest/base/base/#Base.GC.@preserve) if you need it.