# Impossible Raw String Encoding?

**URL:** <https://discourse.julialang.org/t/impossible-raw-string-encoding/9493>\
**Category:** General Usage\
**Created:** [March 5, 2018, 1:25am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493 "2018-03-05T01:25:05Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Post date:** [March 5, 2018, 1:25am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/1 "2018-03-05T01:25:05Z")

</div>

dear julia experts— is it impossible to encode a quoted quote into a raw string? For example, `("\"abcd\"")` contains two of them:

```julia
println("WANTED: (\"\\\"abcd\\\"\")")
println(raw"NO: (\"\"abcd\\\")")

```

```julia
WANTED: ("\"abcd\"")
NO: (""abcd\\")

```

PS: It’s not important. It’s more a curiosity.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [March 5, 2018, 2:51am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/2 "2018-03-05T02:51:37Z")

</div>

Is this what you want?

```julia
julia> x = IOBuffer(raw"\\\"abcd\\\"")
IOBuffer(data=UInt8[...], readable=true, writable=false, seekable=true, append=false, size=8, maxsize=Inf, ptr=1, mark=-1)

julia> dump(x)
Base.GenericIOBuffer{Array{UInt8,1}}
  data: Array{UInt8}((8,)) UInt8[0x5c, 0x22, 0x61, 0x62, 0x63, 0x64, 0x5c, 0x22]
  readable: Bool true

```

---

<div class="post-metadata">

**Author:** ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Post date:** [March 5, 2018, 6:01am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/3 "2018-03-05T06:01:19Z")

</div>

hi scott—no, I just want to print it. I should have omitted the IOBuffer altogether—it was confusing. will change in questions.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [March 5, 2018, 8:09am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/4 "2018-03-05T08:09:47Z")

</div>

```julia
help?> @raw_str
  @raw_str -> String

  Create a raw string without interpolation and unescaping. The exception is that quotation marks still must be escaped. Backslashes escape both quotation marks and other
  backslashes, but only when a sequence of backslashes precedes a quote character. Thus, 2n backslashes followed by a quote encodes n backslashes and the end of the
  literal while 2n+1 backslashes followed by a quote encodes n backslashes followed by a quote character.

  Examples
  ≡≡≡≡≡≡≡≡≡≡

  julia> println(raw"\ $x")
  \ $x
  
  julia> println(raw"\"")
  "
  
  julia> println(raw"\\\"")
  \"
  
  julia> println(raw"\\x \\\"")
  \\x \"

```

---

<div class="post-metadata">

**Author:** ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Post date:** [March 6, 2018, 1:11am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/5 "2018-03-06T01:11:51Z")

</div>

It’s not what I am getting here:

```julia
julia> println(raw"\\\"")
\\"

```

you seem to be getting `\"`. are you using 0.6.2?

---

<div class="post-metadata">

**Author:** ![greg\_plowman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/greg_plowman/32/8100_2.png) [@greg\_plowman](https://discourse.julialang.org/u/greg_plowman)\
**Post date:** [March 6, 2018, 2:02am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/6 "2018-03-06T02:02:03Z")

</div>

Is this what you want?

```julia
Julia-0.6.2> println("\\\"abcd\\\"")
\"abcd\"

```

---

<div class="post-metadata">

**Author:** ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Post date:** [March 6, 2018, 2:57am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/7 "2018-03-06T02:57:22Z")

</div>

yes, and it is like my original question example, not a raw string but a non-raw ordinary string.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 6, 2018, 3:34am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/8 "2018-03-06T03:34:27Z")

</div>

> [@Liso](#):
>
> 2n+1 backslashes followed by a quote encodes n backslashes

This behavior was introduced in 0.7: [fix #22926, raw string escaping rules that allow writing all strings by JeffBezanson · Pull Request #24621 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/24621)

In Julia 0.6, some strings are impossible to write as `raw"..."` strings; you have to use an ordinary string literal with backslash escapes.

---

<div class="post-metadata">

**Author:** ![iwelch](https://avatars.discourse-cdn.com/v4/letter/i/8c91f0/32.png) [@iwelch](https://discourse.julialang.org/u/iwelch)\
**Post date:** [March 6, 2018, 5:51am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/9 "2018-03-06T05:51:20Z")

</div>

thank you. understood. (looking forward to 1.0!)

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [March 6, 2018, 9:26am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/10 "2018-03-06T09:26:13Z")

</div>

> [@stevengj](#):
>
> In Julia 0.6, some strings are impossible to write as raw"…" strings

How could you write `@raw_str("\t")` without workaround?

BTW. answer to this question imply possibility to write properly something like this:

```julia
replace("abc", r"(b)" => @s_str("\\1\t")) # replace ` group => group * "\t" `

```

Edit:  
Not necessarily imply. Because replace could probably work with next representation too:

```julia
julia> s"\1\t"
s"\\1\\t"

```

But I am not sure if it would be best solution to the problem.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 25, 2018, 1:14pm UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/11 "2018-04-25T13:14:36Z")

</div>

> [@Liso](#):
>
> How could you write `@raw_str("\t")` without workaround?

`raw"\t"` already works. You only run into trouble for backslashes at the _end_ of the string, since it is ambiguous whether they are intended as a backslash or as an escape of the final `"`. In any case, you can always just write an ordinary string literal with backslash escapes: e.g. `raw"\t"` is equivalent to `"\\t"`.

You never _need_ the `raw` string syntax — it is just convenient sugar for strings like `raw"C:\foo\bar\baz"` where you would otherwise need lots of extra backslashes.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [April 26, 2018, 4:56pm UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/12 "2018-04-26T16:56:15Z")

</div>

You wrote: `In Julia 0.6, some strings are impossible to write as raw"…" strings" `.

This is true in Julia 0.7 too…

> [@stevengj](#):
>
> You never need the raw string syntax […]

It could be disputable. Problem described in [SO](https://stackoverflow.com/questions/46410630/how-do-i-use-replace-with-capture-group/46412772#46412772) seems to be still solvable only by using hack.

But you are probably right because it is more problem of SubstitutionString than raw string. Although possibility to escape chars in raw string syntax would help too. (I don’t propose it now)

See:

```julia
# I am trying to create MWE here. 
# We want to add \t to found group in replace 
# So "a\t" has to change to "a\t\t" 
# (or "b\t" to "b\t\t" ) 

julia> grp = r"([a-z]+\t)"

julia> replace("a\t", grp => "\1\t") # this is NOK
"\x01\t"

julia> replace("a\t", grp => "\1\t") # this is NOK
"\\1\t"

# next two examples show where we NEED escape \t 

julia> replace("a\t", grp => s"\1\\t") # this is NOK
"a\t\\t"

julia> replace("a\t", grp => s"\1\t") # this is ERROR

```

So if we want to use “\1” regex syntax we could not use String, we **need** Base.SubstitutionString. But there is problem because we need some kind of mix raw string syntax with string syntax …

This hack works but it looks not like “fresh approach”:

```julia
julia> replace("a\t", grp => @s_str("\\1\t")) # using hack is OK
"a\t\t"

```

We very probably need to fix functions working with SubstitutionString. (replace doesn’t understand ‘\t’ in substitution string)

And for example this is not very nice too:

```julia
julia> s"\1\t"
s"\\1\\t"

```

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [April 26, 2018, 7:53pm UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/13 "2018-04-26T19:53:58Z")

</div>

> [@Liso](#):
>
> You wrote: In Julia 0.6, some strings are impossible to write as raw"…" strings". This is true in Julia 0.7 too…

Not for `raw` strings: [fix #22926, raw string escaping rules that allow writing all strings by JeffBezanson · Pull Request #24621 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/24621)

> It could be disputable. Problem described in SO seems to be still solvable only by using hack.

I don’t see this as a hack. As you point out on SO, you can always do `Base.SubstitutionString("....some literal string....")` with as many backslashes as necessary, or `@s_str("...")` as you note above. For example:

```julia
replace("a\t", r"([a-z]+\t)" => Base.SubstitutionString("\\1\t"))

```

Sometimes you need to call the raw constructor in order to have fine control over what is escaped. This is also important for constructing the string programmatically (not by a literal).

That being said, in cases where it can be done without ambiguity, it would be great to extend the range of escapes recognized by `s"..."` etcetera.

---

<div class="post-metadata">

**Author:** ![Liso](https://avatars.discourse-cdn.com/v4/letter/l/898d66/32.png) [@Liso](https://discourse.julialang.org/u/Liso)\
**Post date:** [May 1, 2018, 6:08am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/14 "2018-05-01T06:08:59Z")

</div>

readme says: “Raw string literal escaping rules have been changed to make it possible to write all strings.”

Maybe it is OK and I just don’t understand that wording. I understand that sentence that this is possible:

```julia
"\t" == raw"<what to write here?>"

```

Which I am afraid is not. But this I believe this could be true:

```julia
repr("<any string>") == raw"<we are able to write this>"

```

So raw strings could be good and we just need to fix substitutional strings.

Or abandon this bitter syntactic sugar? @ScottPJones how do you do replace in your library? Do you use Base.replace?

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 1, 2018, 7:49am UTC](https://discourse.julialang.org/t/impossible-raw-string-encoding/9493/15 "2018-05-01T07:49:41Z")

</div>

Currently, I just use `Base.SubstitutionString`, although that may change (I have had to make my own types to replace `Regex`, `RegexMatch`, and `RegexMatchIterator`, to support string types other than `String`)