# UTF8 space/difficulty tradeoffs

**URL:** <https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205>\
**Category:** Internals & Design\
**Created:** [May 28, 2018, 6:14am UTC](https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205 "2018-05-28T06:14:03Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 28, 2018, 6:14am UTC](https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205/1 "2018-05-28T06:14:03Z")

</div>

> [@Strftime & strptime bug #27239 is present on all platforms, not just Windows](https://discourse.julialang.org/t/strftime-strptime-bug-27239-is-present-on-all-platforms-not-just-windows/11191/2):
>
> For example, `한` is `0xc7d0` in `EUC-KR` , but `U+d55c` in Unicode and will be encoded as `0xed9f9c` in `UTF-8` .

This is something that has always concerned me about the “UTF-8 is always best” mentality, text from the languages of the vast majority of the world’s population generally takes up 50% to 200% more space encoded with `UTF-8` compared to their older national character sets (compared to `UTF-16`, which is the same size or only 100% larger, not 200%).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 28, 2018, 1:16pm UTC](https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205/2 "2018-05-28T13:16:33Z")

</div>

> [@Strftime & strptime bug #27239 is present on all platforms, not just Windows](https://discourse.julialang.org/t/strftime-strptime-bug-27239-is-present-on-all-platforms-not-just-windows/11191/4):
>
> concerned me about the “UTF-8 is always best” mentality, text from the languages of the vast majority of the world’s population generally takes up 50% to 200% more space encoded with `UTF-8`

This is correct, but another perspective on this issue is that space is usually cheap compared to programmer time and the cost of bugs, so economizing on the latter by choosing a single representation is reasonable in many circumstances.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 28, 2018, 1:18pm UTC](https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205/3 "2018-05-28T13:18:18Z")

</div>

> [@Strftime & strptime bug #27239 is present on all platforms, not just Windows](https://discourse.julialang.org/t/strftime-strptime-bug-27239-is-present-on-all-platforms-not-just-windows/11191/8):
>
> but another perspective on this issue is that space is usually cheap compared to programmer time and the cost of bugs

Actually, that’s one big reason I think that what I’m working on would be much better for the Julia community.  
There’s a constant stream of bugs related to indexing into `UTF-8` code units.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [May 28, 2018, 4:51pm UTC](https://discourse.julialang.org/t/utf8-space-difficulty-tradeoffs/11205/4 "2018-05-28T16:51:16Z")

</div>