# How do you convert a wchar\_t\* returned by a C DLL to a String?

**URL:** <https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535>\
**Category:** New to Julia\
**Tags:** windows, ccall\
**Created:** [March 8, 2017, 7:27am UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535 "2017-03-08T07:27:07Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 8, 2017, 7:27am UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/1 "2017-03-08T07:27:07Z")

</div>

Hi,  
I’m trying to use a C DLL. Most of the functions take int or double or char\* and I think there are examples of passing in all these in the docs.

However, one function I need returns a wchar\_t\* of NULL-terminated UTF16-encoded bytes – it is owned by the DLL and could be up to 2MB. How do I copy this into a String?

Thanks.

---

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 8, 2017, 2:15pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/2 "2017-03-08T14:15:43Z")

</div>

I finally figured out how to do it. I doubt that my solution is the most efficient (which is why I haven’t marked it as solved), but it works for me (using julia 0.6):

```julia
function utf16tostring(p::Ptr{UInt16}; maxlen=100_000)
    text = ""
    i = 1
    while i <= maxlen
        u = unsafe_load(p, i)
        if u == 0; break end
        text *= string(convert(Char, u))
        i += 1
    end
    text
end

```

---

<div class="post-metadata">

**Author:** ![ihnorton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihnorton/32/26_2.png) [@ihnorton](https://discourse.julialang.org/u/ihnorton)\
**Post date:** [March 8, 2017, 3:00pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/3 "2017-03-08T15:00:20Z")

</div>

For 0.6, use [`transcode(UInt8, s::Vector{Cwchar_t})`](http://docs.julialang.org/en/stable/stdlib/strings/?highlight=transcode#Base.transcode) to convert to UTF8. There are also some Windows-only helper functions for `Cwstring` in `base/c.jl`.

(`Cwchar_t == UInt16` on Windows).

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [March 8, 2017, 7:31pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/4 "2017-03-08T19:31:48Z")

</div>

For reference, you really want to avoid calling `*=` in a loop as it’s going to allocate a new string each time. Here `transcode` is the best solution, but in general for this kind of situation you’d better store the characters in an array, fill it with characters, and only then create a string from it. Of course it’s even better if you can directly load the string from memory.

---

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 8, 2017, 8:10pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/5 "2017-03-08T20:10:16Z")

</div>

I looked in base/c.jl and found cwstring() which is great for creating wchar\_t to pass _into_ C functions, so thanks for mentioning that!

But using transcode() doesn’t work because I only have a NULL-terminated Ptr{Cwchar\_t} of unknown length and transcode() won’t accept a pointer. Nor can I see any way to convert a Ptr{Cwchar\_t} to a Vector{Cwchar\_t}. I also tried changing my return type from Ptr{Cwchar\_t} to Cwstring and then doing convert(UInt16, p) where p is a Cwstring but that didn’t work either.

Also, I guessed that \*= is bad in loops (like Python) but I don’t know the length of the string, so I’m stuck.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [March 8, 2017, 9:05pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/6 "2017-03-08T21:05:27Z")

</div>

> [@mark](#):
>
> Nor can I see any way to convert a Ptr{Cwchar\_t} to a Vector{Cwchar\_t}

[http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code.html#Accessing-Data-through-a-Pointer-1](http://docs.julialang.org/en/latest/manual/calling-c-and-fortran-code.html#Accessing-Data-through-a-Pointer-1)

And more specifically [http://docs.julialang.org/en/latest/stdlib/c.html#Base.unsafe\_wrap-Tuple{Union{Type{Array{T,N}%20where%20N](http://docs.julialang.org/en/latest/stdlib/c.html#Base.unsafe_wrap-Tuple%7BUnion%7BType%7BArray%7BT,N%7D%20where%20N)},%20Type{Array{T,N}},%20Type{Array}},Ptr{T},Tuple{Vararg{Int64,N}}}%20where%20N%20where%20T

---

<div class="post-metadata">

**Author:** ![ihnorton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihnorton/32/26_2.png) [@ihnorton](https://discourse.julialang.org/u/ihnorton)\
**Post date:** [March 8, 2017, 9:26pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/7 "2017-03-08T21:26:55Z")

</div>

As far as finding the length, here’s some [helper code](https://github.com/JuliaArchive/LegacyStrings.jl/blob/bd00f162cdef4fbe2a4376de50366db42d58d748/src/utf16.jl#L250) from older Julia versions.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 8, 2017, 11:00pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/8 "2017-03-08T23:00:02Z")

</div>

We could probably stand to make this a little easier to use. The minimalist goal of `transcode` was to allow calling Windows APIs without needing a full-blown UTF-16 string type in Base. PRs and API design ideas welcome.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [March 9, 2017, 3:23am UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/9 "2017-03-09T03:23:42Z")

</div>

This also won’t handle non-BMP characters (i.e. characters that are encoded via two surrogate characters in UTF-16).  
One option might be to use the `UTF16String` type that was moved to the `LegacyStrings` package.  
You can use `String(utf16(p))`, where p is a `Ptr{UInt16}` or `Ptr{Int16}`, to get a `UTF16String` from a pointer to a null terminated UTF-16 string, and then return it as a UTF-8 encoded `String` type.

---

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 10, 2017, 9:25am UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/10 "2017-03-10T09:25:37Z")

</div>

I tried LegacyStrings but it gave lots of warnings so I gave up on it. So I tried a new solution that didn’t keep appending to a string:

```julia
function utf16tostring(p::Ptr{UInt16}; maxlen=100_000)
    chars = zeros(UInt16, maxlen)
    i = 1
    while i <= maxlen
        u = unsafe_load(p, i)
        if u == 0; break end
        chars[i] = u
        i += 1
    end
    transcode(String, chars)[1:i-1]
end

```

It runs about 5% faster on my test data.

I do think that the standard library should provide a Cwstring to String conversion function though.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [March 10, 2017, 12:31pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/11 "2017-03-10T12:31:43Z")

</div>

Unless you know a plausible value for `maxlen`, it’s going to be more efficient to either go over the string to compute its length before allocating the vector, or to start with an empty vector and call `push!` repeatedly. That function will allocate a bigger vector than what is needed, so that this kind of pattern does not trigger a reallocation for each added value.

EDIT: like the code @ihnorton linked to, it would be even better to compute the length of the string, then use `unsafe_wrap(Array, p, len)` and call `transcode` on the result. That way you avoid an intermediate copy.

---

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 10, 2017, 1:46pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/12 "2017-03-10T13:46:05Z")

</div>

Thanks, that’s led to a new (slightly faster) and much shorter version that doesn’t preallocate lots of memory:

```julia
function utf16tostring(p::Ptr{UInt16}; maxlen=100_000)
    len = 0
    while unsafe_load(p, len + 1) != 0 && len < maxlen; len += 1; end
    transcode(String, unsafe_wrap(Array{UInt16,1}, p, len))
end

```

I decided to keep the maxlen limit to be on the safe side.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [March 10, 2017, 10:31pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/13 "2017-03-10T22:31:33Z")

</div>

What sorts of warnings did you see? We use LegacyStrings extensively, and don’t see any warnings (on v0.5).

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 10, 2017, 10:59pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/14 "2017-03-10T22:59:13Z")

</div>

> [@ScottPJones](#):
>
> This also won’t handle non-BMP characters (i.e. characters that are encoded via two surrogate characters in UTF-16).

By “this”, I guess you meant the `utf16tostring` method posted above. The `transcode` function, on the other hand, certainly handles non-BMP characters when translating between UTF-8 and UTF-16 in either direction.

Determining the length by searching for a NUL 16-bit word, then wrapping the `Ptr{UInt16}` in a `Vector{UInt8}`, then calling `transcode`, should be the right thing to do.

I agree that this should be built-into Base, however. In analogy with `unsafe_string`, maybe just an `unsafe_transcode(String, p::Cwstring, length=wcslen(p))` method. LegacyStrings should be updated if needed, but it shouldn’t be needed just to convert data to/from `String`.

---

<div class="post-metadata">

**Author:** ![mark](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mark/32/5216_2.png) [@mark](https://discourse.julialang.org/u/mark)\
**Post date:** [March 11, 2017, 7:24am UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/15 "2017-03-11T07:24:17Z")

</div>

I’m using the 0.6 alpha.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [March 11, 2017, 5:13pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/16 "2017-03-11T17:13:35Z")

</div>

> [@stevengj](#):
>
> By “this”, I guess you meant the utf16tostring method posted above. The transcode function, on the other hand, certainly handles non-BMP characters when translating between UTF-8 and UTF-16 in either direction.

Yes, I meant the original version in this post, which treated each 16-bit codepoint as if it were a character.

The final version with should be better, however, scanning first for a 16-bit 0, wrapping the pointed to words as a `Vector`, and then scanning the string yet again to calculate the length in UTF-8, before finally creating the String version will not be that efficient.

> [@stevengj](#):
>
> I agree that this should be built-into Base, however. In analogy with unsafe\_string, maybe just an unsafe\_transcode(String, p::Cwstring, length=wcslen(p)) method. LegacyStrings should be updated if needed, but it shouldn’t be needed just to convert data to/from String.

Yes, although I’d have separate methods instead of calling wcslen(p), to avoid scanning twice, and Cwstring can be 16-bit or 32-bit characters, depending on platform, so for UTF-16:  
i.e. `unsafe_transcode(String, p::Ptr{UInt16})` and `unsafe_transcode(String, p::Ptr{UInt16}, len::Int)`

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [March 11, 2017, 5:14pm UTC](https://discourse.julialang.org/t/how-do-you-convert-a-wchar-t-returned-by-a-c-dll-to-a-string/2535/17 "2017-03-11T17:14:39Z")

</div>

That explains it - LegacyStrings needs to be updated (or dropped and replaced with something better for v0.6 😉 )