# Deepcopy dangers and drawbacks for nested data structures

**URL:** <https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998>\
**Category:** General Usage\
**Tags:** deepcopy\
**Created:** [March 22, 2024, 11:56pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998 "2024-03-22T23:56:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 22, 2024, 11:56pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/1 "2024-03-22T23:56:27Z")

</div>

Since reading this issue: [Should `copy` be renamed `shallowcopy`? · Issue #42796 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/42796#issuecomment-951232853)

I have always avoided using `deepcopy`. Now I have a situation in which to define custom copy methods for my types is cumbersome, requires methods for many nested types, and causes a lot of code repetition. `deepcopy`, on the other hand, just works.

Can I fall into any trap by using `deepcopy`, in particular in the context of creating thread-local copies of data structures that must not be shared?

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [March 23, 2024, 12:39am UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/2 "2024-03-23T00:39:44Z")

</div>

Tangential to your question lmiq, but just wanted to throw this in, if someone could explain for me.

I use `deepcopy` too and was not aware of this. Is the only drawback speed or was there more?

I didn’t get the Github Issue completely I think.

Kind regards

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [March 24, 2024, 6:39pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/3 "2024-03-24T18:39:41Z")

</div>

Because `deepcopy` recurses blindly through everything it can find, I think it can’t be trusted to have any specified meaning at all, rather than posing a risk of some particular danger. That’s a somewhat ideological argument but I think it’s right.

For example if `v` is a vector and `x.a.b.c.d.e === v` and `y.a.b.c.d.e === v` then `deepcopy(x), deepcopy(y)` breaks that connection, even if the semantics of `copy(x)` and `copy(y)` would require that the connection be preserved.

Imho implementing `copy` is a good idea. To make it more convenient to define structs, one can use a macro that defines `==`, `hash`, `copy`, etc, specifying custom behavior for particular structs as needed.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [March 24, 2024, 6:48pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/4 "2024-03-24T18:48:45Z")

</div>

It’s hard to say without a MWE, but I think it’s almost always a better idea to use custom constructors to create new objects rather than rely on `deepcopy` (even if writing those constructors is a little tricky).

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [March 24, 2024, 7:11pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/5 "2024-03-24T19:11:06Z")

</div>

> [@jar1](#):
>
> For example if `v` is a vector and `x.a.b.c.d.e === v` and `y.a.b.c.d.e === v` then `deepcopy(x), deepcopy(y)` breaks that connection, even if the semantics of `copy(x)` and `copy(y)` would require that the connection be preserved.

It looks like this is [no longer true](https://github.com/JuliaLang/julia/blob/4a2c59333946c5450c73f4c5ba7d2d28529db8de/base/deepcopy.jl#L14) if it ever was (I don’t know one way or the other).

Edit: I misread what you said, sorry about that. I don’t know how someone could expect anything else other than the result you describe however, copies are copies. It could be avoided with `xcopy, ycopy = deepcopy((x,y))` if the connection needed to be preserved.

Reading the tea leaves, the docstring says “Calling `deepcopy` on an object should generally have the same effect as serializing and then deserializing it.”.

That suggests to me that, unless unusual data structures are involved (and the docstring has some advice there as well), `deepcopy` can be trusted, for some value of “trusted”. An expensive operation sometimes, sure, but that’s the nature of the beast.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 24, 2024, 8:38pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/6 "2024-03-24T20:38:04Z")

</div>

Indeed. The specific problem is that to construct the object I have to read a file that has to be accessed sequentially. Thus, I cannot (could not) construct in parallel.

But, that said, I was already implementing an alternative where I’ll read the data from the file before the threaded loop and create an intermediate structure that will allow me to use constructors in the parallel part. I think in my case that will be easier than implementing all copy methods that would be needed.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [March 24, 2024, 8:40pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/7 "2024-03-24T20:40:56Z")

</div>

> [@jar1](#):
>
> implementing `copy` is a good idea. To make it more convenient to define structs, one can use a macro that defines `==`, `hash`, `copy`, etc,

I can implement a custom copy function that iterates over the fields, but I don’t know if I trust my implementation of something semi-automatic like that more than what I trust `deepcopy`.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [March 24, 2024, 8:58pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/8 "2024-03-24T20:58:38Z")

</div>

Like I said, you don’t have to type out `Base.copy(::T)` for every `T` manually, you can use or create a macro that defines the constructor/==/hash/copy behavior that you want, and just apply it to every struct definition where you want that behavior.

In principle, for a given `T` and `a`, implementing `copy(x::T)` requires specifying whether `x.a === copy(x).a` . It is not true for every type and property. Only the author of `T` can define whether that is true or not, so `copy` must be defined by that person.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [March 24, 2024, 8:59pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/9 "2024-03-24T20:59:13Z")

</div>

`deepcopy` is fundamentally and conceptually unsafe: You can’t copy a datastructure without knowing what it means. Especially for things containing pointers / external references.

To give a Base example:

```julia
julia> r=open("/tmp/foo", "w+")
IOStream(<file /tmp/foo>)

julia> println(r, "foo")

julia> r2=deepcopy(r)
IOStream(<file /tmp/foo>)

julia> close(r)

julia> println(r2, "foo")

julia> GC.gc()
malloc(): unsorted double linked list corrupted

[77602] signal (6.-6): Aborted
in expression starting at REPL[29]:1

```

You can look for other examples by searching github julialang for `finalizer` and cross-referencing with

```julia
julia> methods(Base.deepcopy_internal)
# 12 methods for generic function "deepcopy_internal" from Base:

```

That being said, deepcopy is sometimes useful and pragmatic. Just know that you can only use it on stuff where you know what’s inside.

---

<div class="post-metadata">

**Author:** ![mnemnion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mnemnion/32/206596_2.png) [@mnemnion](https://discourse.julialang.org/u/mnemnion)\
**Post date:** [March 24, 2024, 10:10pm UTC](https://discourse.julialang.org/t/deepcopy-dangers-and-drawbacks-for-nested-data-structures/111998/10 "2024-03-24T22:10:59Z")

</div>

> [@foobar\_lv2](#):
>
> `deepcopy` is fundamentally and conceptually unsafe: You can’t copy a datastructure without knowing what it means. Especially for things containing pointers / external references.

That’s roughly what I was getting at with “for some value of ‘trusted’”, sure. But it isn’t as though the behavior of `deepcopy` is especially difficult to reason about. If someone is using `deepcopy` on an `IOStream`, that probably indicates that they don’t understand what an `IOStream` is, which yeah, fine, handle with care and all.

> [@foobar\_lv2](#):
>
> That being said, deepcopy is sometimes useful and pragmatic. Just know that you can only use it on stuff where you know what’s inside.

Right, it’s neither difficult to understand nor unsafe to apply to most things. Those would be my definitions of “conceptually unsafe” and “fundamentally unsafe”, respectively, but YMMV.

In this case, the topic was started by someone who has learned some ‘lore’ about `deepcopy` which is, perhaps, rooted in overthinking things.

So, this:

> [@lmiq](#):
>
> Can I fall into any trap by using `deepcopy`, in particular in the context of creating thread-local copies of data structures that must not be shared?

Not really, no. You can’t expect good things to happen if you’re copying something which holds state for a stateful process, especially involving syscalls, so y’know. An `IOStream`, don’t copy those. But you know the contents of your types, if they’re plain-old-data, `deepcopy` will work fine. If you’re unsure, you can look at the implementation and find out what will happen for things like `Ref`s. If you’re _really_ unsure, then yeah don’t use it.