# \[ANN\] EntropyHub.jl

**URL:** <https://discourse.julialang.org/t/ann-entropyhub-jl/64572>\
**Category:** Package Announcements\
**Tags:** package, announcement, time-series, math, nonlinear\
**Created:** [July 13, 2021, 2:09pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572 "2021-07-13T14:09:57Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![MattWillFlood](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattwillflood/32/20722_2.png) [@MattWillFlood](https://discourse.julialang.org/u/MattWillFlood)\
**Post date:** [July 13, 2021, 2:09pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/1 "2021-07-13T14:09:57Z")

</div>

This post is to announce the release of **EntropyHub.jl** - a package for estimating various information-theoretic entropy measures from time series and image data.

EntropyHub.jl (v0.1) features functions for computing:

- standard (base) entropy measures  
e.g. sample entropy, fuzzy entropy, permutation entropy, slope entropy, and much more.
- cross-entropy methods  
e.g. cross-approximate entropy, cross-Kolmogorov entropy, and more.
- multiscale entropy using any of the standard entropies  
e.g. multiscale dispersion entropy, refined multiscale approximate entropy, and much more.
- multiscale cross-entropy using any of the cross-entropies  
e.g. multiscale cross-conditional entropy, composite multiscale cross-distribution entropy, and more.
- bidimensional entropy measure for matrix (image) data  
e.g. bidimensional fuzzy entropy, bidimensional dispersion entropy.

The [EntropyHub project](https://www.EntropyHub.xyz) aims to bring together the many entropy measures in the scientific literature under one complete package that is available in Julia, Python and MatLab  
:juliabouncer: 🐍

 ![Matlab_Logo](https://global.discourse-cdn.com/julialang/original/3X/8/4/84c29f2a95323166d1b3c1c2e4bf603752dea956.png).

EntropyHub is in the Julia Registry and can be installed in the Pkg REPL as follows:

```julia
pkg> add EntropyHub

```

Documentation for EntropyHub.jl, with descriptions of function syntax and examples of use can be found at:  
[MattWillFlood.github.io/EntropyHub.jl/stable](https://mattwillflood.github.io/EntropyHub.jl/stable) and in the _[EntropyHub Guide.pdf](https://github.com/MattWillFlood/EntropyHub/blob/main/EntropyHub%20Guide.pdf)_.

EntropyHub continually seeks to integrate as many established entropy methods into one comprehensive package. We welcome suggestions and support from all in helping to achieve this 🙂

Thanks,  
Matt  
_info@EntropyHub.xyz_

 ![EntropyHub_JuliaLogo](https://global.discourse-cdn.com/julialang/original/3X/1/0/10c2ad2df2e3d47de3b72d728602a4d81b086bc4.png)

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [July 13, 2021, 2:46pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/2 "2021-07-13T14:46:20Z")

</div>

Are you aware of [Entropies.jl](https://juliadynamics.github.io/DynamicalSystems.jl/dev/entropies/api/) ? Have you compared any implementations, especially w.r.t. performance? I’d be interested to see the result.

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [July 13, 2021, 2:53pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/3 "2021-07-13T14:53:05Z")

</div>

I just did a timing, so if you can post your own on permutation entropy I’d appreciate it.

```julia
julia> using DynamicalSystems, BenchmarkTools

julia> m = 3; τ = 1; x = rand(10000);

julia> @btime permentropy(x, m; τ) # base e
  163.200 μs (28 allocations: 470.09 KiB)
1.791651893027263

```

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [July 13, 2021, 3:24pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/4 "2021-07-13T15:24:33Z")

</div>

Are you also aware that the naming convention you use is in opposition with the standard Julia notation, and may bring conflict to potential Julia users?

Julia recommends: [Style Guide · The Julia Language](https://docs.julialang.org/en/v1/manual/style-guide/#Use-naming-conventions-consistent-with-Julia-base/)

But every function I’ve checked uses `CamelCaseNotation` instead of `lower_case_notation`. If the CamelCase is not really necessary due to field-related conventions, I believe you should definitely consider following the lowercase version.

---

<div class="post-metadata">

**Author:** ![MattWillFlood](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattwillflood/32/20722_2.png) [@MattWillFlood](https://discourse.julialang.org/u/MattWillFlood)\
**Post date:** [July 13, 2021, 3:42pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/5 "2021-07-13T15:42:58Z")

</div>

Hi @Datseris 🙂

Yes, we are aware of Entropies.jl and the wonderful functionality it provides.  
While Entropies.jl provides many ways of estimating Shannon, Renyi and similar generalized entropies, EntropyHub.jl focuses on the broad array of entropy measures introduced in the last few decades for various time series applications ([Entropy | Free Full-Text | The Entropy Universe](https://www.mdpi.com/1099-4300/23/2/222)).

Comparing `permentropy` to `PermEn`, indeed the former is a lot faster:

```julia
@btime permentropy(x, m; τ)
  151.700 μs (27 allocations: 470.03 KiB)
1.7916762374702375

@btime PermEn(x, m=3, tau=1, Logx=0)
  7.858 ms (90229 allocations: 7.95 MiB)
([-0.0, 0.6931449751172578, 1.7916762374702377], ...)

```

Actually, it appears that `permentropy` has a bug as it returns the same value regardless of **m** or **tau** …

```julia
julia> permentropy(x, 3; τ = 1)
1.7916762374702375

julia> permentropy(x, 3; τ = 4)
1.7916762374702375

julia> permentropy(x, 4; τ = 4)
1.7916762374702375

julia> permentropy(x, 7; τ = 4)
1.7916762374702375

```

The EntropyHub version of permutation entropy returns all entropy values for embedding dimensions from 1:m, which probably slows it down.

However, one big advantage of the EntropyHub version is that it provides numerous additional arguments so one can estimate the edge [1], modified [2], amplitude-aware [3], weighted [4], uniform-quantization [5] and fine-grained [6] variants of permutation entropy too!

As EntropyHub was originally developed in MatLab and Python, we wanted to keep the syntax consistent across all platforms, that’s why CamelCase was used (otherwise we would have followed the style guide exactly 😉)

Thanks 🙂  
I hope you find the package useful! 👍🏼

_[1] Zhiqiang Huo, et al., Edge Permutation Entropy: An Improved Entropy Measure for Time-Series Analysis,45th Annual Conference of the IEEE Industrial Electronics Soc, (2019), 5998-6003_  
_[2] Chunhua Bian, et al.,Modified permutation-entropy analysis of heartbeat dynamics,_  
_Physical Review E, 85.2 (2012) : 021906_  
_[3] Hamed Azami and Javier Escudero,_  
_Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation,_  
_Computer methods and programs in biomedicine, 128 (2016): 40-51._  
_[4] Bilal Fadlallah, et al.,Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information,Physical Review E, 87.2 (2013): 022911._  
_[5] Zhe Chen, et al.,Improved permutation entropy for measuring complexity of time series under noisy condition, Complexity, 1403829 (2019)_  
_[6] Xiao-Feng Liu, and Wang Yue,_  
_Fine-grained permutation entropy as a measure of natural complexity for time series,_  
_Chinese Physics B, 18.7 (2009): 2690._

---

<div class="post-metadata">

**Author:** ![viraltux](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viraltux/32/15236_2.png) [@viraltux](https://discourse.julialang.org/u/viraltux)\
**Post date:** [July 13, 2021, 4:16pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/6 "2021-07-13T16:16:59Z")

</div>

I have a question on the Licensing, you state:

> ## License and Terms of Use
> 
> EntropyHub is licensed under the Apache License (Version 2.0) and is free to use by all on condition that the following reference be included on any outputs realized using the software:
> 
> ```julia
> Matthew W. Flood and Bernd Grimm, 
> EntropyHub: An Open-Source Toolkit for Entropic Time Series Analysis,
> 2021 www.EntropyHub.xyz
> 
> ```

Does this mean that anybody using your code in their projects must return something like:

```julia
PermEnNewProject(x, m=3, tau=1, Logx=0)

Matthew W. Flood and Bernd Grimm, 
    EntropyHub: An Open-Source Toolkit for Entropic Time Series Analysis,
    2021 www.EntropyHub.xyz

([-0.0, 0.6931449751172578, 1.7916762374702377],...)

```

---

<div class="post-metadata">

**Author:** ![MattWillFlood](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mattwillflood/32/20722_2.png) [@MattWillFlood](https://discourse.julialang.org/u/MattWillFlood)\
**Post date:** [July 13, 2021, 5:02pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/7 "2021-07-13T17:02:59Z")

</div>

Hi @viraltux 🙂

Good question 👍🏼  
That statement just means that if you use EntropyHub in your research, that you include that citation in any scientific outputs (journal/conference papers, presentations, etc.)

Thanks!

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [July 13, 2021, 6:24pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/8 "2021-07-13T18:24:35Z")

</div>

Please provide `underscore_case` versions of functions to reduce unnecessary confusion for Julia users.

---

<div class="post-metadata">

**Author:** ![aramirezreyes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aramirezreyes/32/42573_2.png) [@aramirezreyes](https://discourse.julialang.org/u/aramirezreyes)\
**Post date:** [July 13, 2021, 6:59pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/9 "2021-07-13T18:59:18Z")

</div>

It is always welcome to bring new packages and expand Julia’s application environment!

---

<div class="post-metadata">

**Author:** ![Tero\_Frondelius](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tero_frondelius/32/7629_2.png) [@Tero\_Frondelius](https://discourse.julialang.org/u/Tero_Frondelius)\
**Post date:** [July 13, 2021, 7:55pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/10 "2021-07-13T19:55:36Z")

</div>

> [@jzr](#):
>
> Please provide `underscore_case` versions of functions to reduce unnecessary confusion for Julia users.

Sounds like metaprogramming. Does anyone know how to write such a macro or similar to generate underscore\_case from CamelCase to exported functions?

---

<div class="post-metadata">

**Author:** ![DoktorMike](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/doktormike/32/2736_2.png) [@DoktorMike](https://discourse.julialang.org/u/DoktorMike)\
**Post date:** [July 13, 2021, 8:30pm UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/11 "2021-07-13T20:30:59Z")

</div>

If so that macro should exist for Camel case, No case and Snake case. 😊

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [July 14, 2021, 11:46am UTC](https://discourse.julialang.org/t/ann-entropyhub-jl/64572/12 "2021-07-14T11:46:58Z")

</div>

> [@MattWillFlood](#):
>
> Actually, it appears that `permentropy` has a bug as it returns the same value regardless of **m** or **tau** …

Whooooooooooops sorry about that! Someone was pretty stupid, check out how I defined the “convenience function” `permentropy`:

```julia
function permentropy(x, m = 3; τ = 1, base = Base.MathConstants.e)
    Entropies.genentropy(x, SymbolicPermutation(; τ = 1, m = 3))
end

```

yea…

Fixed now, thanks! Thankfully this has no impact on performance (assuming we measure with order 3). So the Entropies.jl version is a lot faster. We had an original version based on combinatorics that was _really slow_, then @kahaaga re-wrote the algorithm with different approach based on symbolizing. I was kinda hoping you guys would had something even faster so we could steal it from you 😛 But now please do feel free to steal our implementation, as the performance difference will only become bigger with higher `m` and larger timeseries, especially due to the difference in allocations.

> [@MattWillFlood](#):
>
> The EntropyHub version of permutation entropy returns all entropy values for embedding dimensions from 1:m, which probably slows it down.

What is a permutation entropy of order 1? At least according to the original definition of Bandt and Pompe, there is no such thing as permutation entropy of order 1.

We also have amplitude-aware and weighted versions for the entropy but not the rest, good to know!

> [@MattWillFlood](#):
>
> As EntropyHub was originally developed in MatLab and Python, we wanted to keep the syntax consistent across all platforms, that’s why CamelCase was used (otherwise we would have followed the style guide exactly 😉)

I am being annoyingly persistent, but I want to challenge that. Who does this help? A python user that already uses the Python version doesn’t really have a reason to use the Julia version, and so doesn’t really benefit from using the same names. Similarly a Julia user would want to use the Julia version, and once again has no benefit from having the same names.

> [@MattWillFlood](#):
>
> I hope you find the package useful! 👍🏼

Absolutely! I hope you don’t take my remarks the wrong way, I’m playing devil’s advocate here to (hopefully) help you make it even better.