# Creating a new value using \`jl\_new\_struct\_uninit\`

**URL:** <https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825>\
**Category:** Internals & Design\
**Created:** [April 18, 2020, 11:51pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825 "2020-04-18T23:51:09Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 18, 2020, 11:51pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/1 "2020-04-18T23:51:09Z")

</div>

In CxxWrap, C++ objects are wrapped in a struct like this:

```julia
struct CxxPtr{T} <: CxxBaseRef{T}
  cpp_object::Ptr{T}
end

```

Completely new objects are stored in a mutable version of the above struct, with exactly the same layout. On the C++ side, I use `jl_new_struct` to allocate the new struct, with as field argument the boxed pointer to store. However, this requires two allocations, so I tried to work around that using `jl_new_struct_uninit`, as seen here:

[https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/5118cd9b07064b85a943f9835fceacb64ab714ce](https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/5118cd9b07064b85a943f9835fceacb64ab714ce)

Unfortunately, this appears to trigger a segfault. So my question is: what is the correct way to allocate a struct like the above one, using only a single allocation (if possible), for both the mutable and immutable version (immutable is for boxing when needed)?

---

<div class="post-metadata">

**Author:** ![Ronis\_BR](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ronis_br/32/50999_2.png) [@Ronis\_BR](https://discourse.julialang.org/u/Ronis_BR)\
**Post date:** [April 19, 2020, 1:23am UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/2 "2020-04-19T01:23:32Z")

</div>

Do you have a MWE of the segfault? Many times I saw unexplainable segfault was because I was forgetting to protect a variable from the garbage collector.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 19, 2020, 9:55am UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/3 "2020-04-19T09:55:08Z")

</div>

No MWE unfortunately, I can’t even reproduce the problem anywhere except on Travis Linux (Mac is fine).

Checking for a GC problem is indeed always a good idea. Unfortunately running the tests without GC gives exactly the same error.

The log is here, but it doesn’t really say much: [Travis CI - Test and Deploy Your Code with Confidence](https://travis-ci.org/github/JuliaInterop/libcxxwrap-julia/jobs/676817027#L810)

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 19, 2020, 8:41pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/4 "2020-04-19T20:41:21Z")

</div>

Update: it seems that using `memcpy` to the directly dereferenced the pointer to the Julia object fixes it. Not sure what the difference is or if it is correct now, so having someone who is familiar with the Julia C code look at this would reassure me:

[https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/733f403350a7c5ed477905e97328d41bfc4122d5](https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/733f403350a7c5ed477905e97328d41bfc4122d5)

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 30, 2020, 12:14pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/5 "2020-04-30T12:14:54Z")

</div>

Another update: I think this commit makes it more equivalent with the way `jl_new_struct` behaves: [https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/40db46874de18e8f09603d81f8eb18a011051400](https://github.com/JuliaInterop/libcxxwrap-julia/pull/53/commits/40db46874de18e8f09603d81f8eb18a011051400)

However, testing what avoiding the extra allocation actually brings in speed gain is making me question the usefulness of the change: the original version takes 0.0066 s to allocate 100000 C++ objects (2 Julia allocations per C++ object), while the new version takes … 0.0058 s, at indeed only one Julia allocation per C++ object.

@yuyichao Do you have any thoughts on this, please? Is the new version safe, or is tinkering at this low level simply not worth it?

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2020, 4:30pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/6 "2020-04-30T16:30:32Z")

</div>

I can only think of any issue here to be caused by aliasing and if that’s the case using `memcpy` should be enough.

I’m pretty sure travis support remote login that you can use to debug and download the binary from there.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 30, 2020, 6:39pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/7 "2020-04-30T18:39:24Z")

</div>

I think previous crashes were caused by some kind of alignment-related UB, which I don’t understand. In Julia they seem to cast to the integer types because of that, or at least the git comments led me to this:

[https://github.com/JuliaLang/julia/issues/26512](https://github.com/JuliaLang/julia/issues/26512)

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2020, 7:04pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/8 "2020-04-30T19:04:11Z")

</div>

What make you think it is related to alignment and no the julia commit has nothing to do with integer typeps.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 30, 2020, 8:21pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/9 "2020-04-30T20:21:27Z")

</div>

The crash was fixed by using (on 64 bit)

```nohighlight
uint64_t val = *(uint64_t*)(&cpp_ptr);
memcpy(dest, &val, 8);

```

instead of:

```nohighlight
memcpy(dest, &cpp_ptr, sizeof(T*));

```

That change was inspired by the implementation of `jl_store_unaligned_i64`, which is called by `set_nth_field` in `jl_new_struct`, and which were modified in the commit that fixed the issue I cited.

What troubles me is that I don’t understand the difference between the crashing and the working implementation, so I have no idea why this fixes it (or even if it fixes it properly and always).

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2020, 10:28pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/10 "2020-04-30T22:28:56Z")

</div>

Again, the only think I can think of is that you are triggering some UB (or compiler bug) and the best way to debug it would still be having a debugger attached to it one way or another.

And FYI, your “fixed” version made **MORE** assumptions about both alignment and aliasing, which is exactly the opposite of the julia commit you linked.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [April 30, 2020, 11:13pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/11 "2020-04-30T23:13:48Z")

</div>

Possibly, part of the problem is that I don’t understand what those assumptions are and why they are needed, though. But the same casting to `uint` happens in Julia, why is that done?

> <https://github.com/JuliaLang/julia/blob/master/src/datatype.c#L693-L708>

When I had similar crashes locally, valgrind and the debugger didn’t point to anything special, except that accessing the created Julia object results in a crash, and creating it using the higher level `jl_new_struct` fixed it (or at least the crash went away), but introduced an extra allocation for boxing the pointer.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 30, 2020, 11:28pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/12 "2020-04-30T23:28:55Z")

</div>

> [@barche](#):
>
> But the same casting to `uint` happens in Julia, why is that done?

Because it is making the assumption that the alignment and type are correct. Also, there’s no casting to `uint` anywhere, the cast is on the pointer. These are just sligntly more optimized case of the `default` branch. They are there for the sole purpose of giving the compiler some hint to produce better code for the most likely usecase. It will be completely correct and should not crash without any branchs other than the default one.

> [@barche](#):
>
> When I had similar crashes locally, valgrind and the debugger didn’t point to anything special

Well, I’m not sure what do you mean by “debugger didn’t point to anything special”. I assume the debugger will point to where it crashes which will tell you what instruction and point it is and should allow you to inspect the condition it happens, the normal thing you’ll use a debugger to do.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [May 1, 2020, 12:35pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/13 "2020-05-01T12:35:30Z")

</div>

Thanks for the clarification, and indeed after switching to the default branch implementation and also exactly the same implementation that crashed before, it doesn’t crash anymore. Which is actually bad news, because it means there is probably some kind of UB elsewhere.

What I mean with “nothng special” is that when I did manage to reproduce the crash in a debugger, it always segfaulted on this seemingly innocent line, and valgrind didn’t show anything invalid up to that point:

[https://github.com/JuliaInterop/libcxxwrap-julia/blob/fix-double-alloc/include/jlcxx/const\_array.hpp#L94](https://github.com/JuliaInterop/libcxxwrap-julia/blob/fix-double-alloc/include/jlcxx/const_array.hpp#L94)

Here, `ptr` is a Julia object created by the code we discussed.

And while writing this, it crashed again on this change, which should actually be no change: [Comparing 035d575df6b2e30d03d1d8142222aef1e035b39f..a4dce9720f32ba9bb4d06efe0c54deeb65f0a6c2 · JuliaInterop/libcxxwrap-julia · GitHub](https://github.com/JuliaInterop/libcxxwrap-julia/compare/035d575df6b2e30d03d1d8142222aef1e035b39f..a4dce9720f32ba9bb4d06efe0c54deeb65f0a6c2)

(log [here](https://travis-ci.org/github/JuliaInterop/libcxxwrap-julia/jobs/681234993#L799))

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 1, 2020, 1:06pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/14 "2020-05-01T13:06:55Z")

</div>

> [@barche](#):
>
> What I mean with “nothng special” is that when I did manage to reproduce the crash in a debugger, it always segfaulted on this seemingly innocent line, and valgrind didn’t show anything invalid up to that point:

The debugger has way more functions than just line numbers. As I said above, you should look at the instructions and variables.

---

<div class="post-metadata">

**Author:** ![barche](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/barche/32/79_2.png) [@barche](https://discourse.julialang.org/u/barche)\
**Post date:** [May 4, 2020, 1:16pm UTC](https://discourse.julialang.org/t/creating-a-new-value-using-jl-new-struct-uninit/37825/15 "2020-05-04T13:16:26Z")

</div>

OK, I have inspected the value returned by both implementations, and it is always the same, I haven’t been able to reproduce the crash on Travis either anymore. My best guess at this point is that our discussion properly aligned the planets so it works now 😉

I’ve merged the change, since this part of the code was probably never the cause of the crashes in the first place. Thanks for the help!