# Persistent hash of julia object

**URL:** <https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513>\
**Category:** General Usage\
**Created:** [January 16, 2017, 2:45pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513 "2017-01-16T14:45:28Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [January 16, 2017, 2:45pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/1 "2017-01-16T14:45:29Z")

</div>

I have julia objects and want to hash them such that:

1. If two objects are equal, then their hashes are equal. The other way round is also true with overwhelming probability.
2. The hash should not change across sessions. E.g. hashing an object, saving to jld, rebooting, loading, hashing yields the same answer.
3. Ideally this would work for objects of arbitrary type, but I guess that’s an endless rabbit hole. So I am happy if it works for bitstypes, Strings, Symbols and immutables build up of these.

What is a reasonable way to achieve this? Is there maybe some package? The `hash` function in base seems to be inconsistent across sessions.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 16, 2017, 3:02pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/2 "2017-01-16T15:02:26Z")

</div>

The `hash` function in base frequently creates a hash based on the address of the object, for performance reasons, however, as you have seen, that `hash` value is not useful across sessions or processes.

I don’t believe there currently is any such function. It might best be added to Base, so that it can use the current `hash` methods when possible.  
(I don’t think it would be that difficult, maybe you could submit a PR to do so)

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 16, 2017, 3:10pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/3 "2017-01-16T15:10:27Z")

</div>

> [@ScottPJones](#):
>
> The hash function in base frequently creates a hash based on the address of the object

I was not aware of that. How would that satisfy `==` for hash, if the two objects are otherwise equal but they are at different addresses?

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 16, 2017, 3:24pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/4 "2017-01-16T15:24:07Z")

</div>

This is used for things like symbols, where they are interned, and you cannot have two symbols having the same string contents at different addresses. It avoids having to do an expensive O(N) operation to calculate a hash value.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 16, 2017, 3:29pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/5 "2017-01-16T15:29:01Z")

</div>

Note: this is the cause of fairly large hassles for us, because you can’t precompile `Dicts` reliably, because if the values being hashed use a hash based on the pointer (ObjectId) (which can mean something contained various levels deep in a structure), will not be recreated correctly when the module is reloaded (you have to put the code in ` __init__ ()`).  
Whether that is a known limitation of the implementation, bug, design flaw, or whatever, I’ll leave up to other people!

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [January 16, 2017, 3:30pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/6 "2017-01-16T15:30:55Z")

</div>

> [@Tamas\_Papp](#):
>
> I was not aware of that. How would that satisfy == for hash, if the two objects are otherwise equal but they are at different addresses?

It wouldn’t? [in general, I’m not aware that you’re required to provide hash for your objects to make that happen.]

“implies” below (as always) doesn’t work from right-to-left:

```julia
help?> hash

Compute an integer hash code such that isequal(x,y) implies hash(x)==hash(y). The optional second argument h is a hash code to be mixed with the result.

julia> @edit hash("Palli")

hash(x::Any) = hash(x, zero(UInt))

..

## hashing general objects ##

hash(x::ANY, h::UInt) = 3*object_id(x) - h

[Seems strings are not interned.]

julia> object_id("Palli")
0x2d22622f8fa6f389

julia> object_id("Palli")
0x7d05cdd0cbf4b7f7

[Still this still happens, because of special case hash below]

julia> hash("Palli")
0xd4199fe90d0f820b

julia> hash("Palli")
0xd4199fe90d0f820b

function hash(s::Union{String,SubString{String}}, h::UInt)
    h += memhash_seed
    # note: use pointer(s) here (see #6058).
    ccall(memhash, UInt, (Ptr{UInt8}, Csize_t, UInt32), pointer(s), sizeof(s), h % UInt32) + h
end

```

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 16, 2017, 3:31pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/7 "2017-01-16T15:31:02Z")

</div>

Note: I think this could be solved by having the serialization / deserialization functions not save the hash table, just the pairs, and be smart enough to rehash the Dicts upon deserialization.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 16, 2017, 3:44pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/8 "2017-01-16T15:44:00Z")

</div>

`hash` should satisfy all your requirements.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 16, 2017, 3:45pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/9 "2017-01-16T15:45:21Z")

</div>

With the caveat that we don’t guarantee it to be stable across multiple julia versions and the return type is different on 32 vs 64bits.

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [January 16, 2017, 3:49pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/10 "2017-01-16T15:49:20Z")

</div>

I agree this is easy for stuff build from String, Symbol, bitstype… but the general case is very hard/it is not even clear what the correct behaviour is. For example think of a function depending on global variables.

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [January 16, 2017, 3:50pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/11 "2017-01-16T15:50:56Z")

</div>

Okay the following behaviour is a problem for me:

```julia
   _ _ _(_)_ | A fresh approach to technical computing
  (_) | (_) (_) | Documentation: http://docs.julialang.org
   _ _ _| |_ __ _ | Type "?help" for help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 0.5.1-pre+27 (2016-11-04 08:41 UTC)
 _/ |\ __'_|_|_|\__'_| | Commit 25f7066 (73 days old release-0.5)
|__/ | x86_64-linux-gnu

julia> immutable Foo
       s::String
       end

julia> Foo("a") |> hash
0xe7585c2a6732e304

julia> exit()

```

```julia
               _
   _ _ _(_)_ | A fresh approach to technical computing
  (_) | (_) (_) | Documentation: http://docs.julialang.org
   _ _ _| |_ __ _ | Type "?help" for help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 0.5.1-pre+27 (2016-11-04 08:41 UTC)
 _/ |\ __'_|_|_|\__'_| | Commit 25f7066 (73 days old release-0.5)
|__/ | x86_64-linux-gnu

julia> immutable Foo
       s::String
       end

julia> Foo("a") |> hash
0x56df03a6764b2325

```

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [January 16, 2017, 3:53pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/12 "2017-01-16T15:53:31Z")

</div>

For objects like that, I think it would have to use reflection to build a hash based on all of the values of the fields.  
Of course, you’d still need to have special case code for types that are holding pointers, for example.

---

<div class="post-metadata">

**Author:** ![FANG\_Colin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fang_colin/32/170_2.png) [@FANG\_Colin](https://discourse.julialang.org/u/FANG_Colin)\
**Post date:** [January 20, 2017, 10:55pm UTC](https://discourse.julialang.org/t/persistent-hash-of-julia-object/1513/13 "2017-01-20T22:55:42Z")

</div>

For custom types you have to overload a `hash`

Try `using AutoHashEquals` if you are lazy.