### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 13, 2024, 11:55pm UTC](https://discourse.julialang.org/t/named-pipes/115627/1 "2024-06-13T23:55:52Z")
Is there something like “named pipes” in Julia? (I just learned that those things exist).
The goal is to provide a “file” as input to an external program while avoiding explicitly writing the file to disc.
Does that exist?
---
### Author:  [@Yuan-Ru-Lin](https://discourse.julialang.org/u/Yuan-Ru-Lin)
#### Post date: [June 14, 2024, 12:25am UTC](https://discourse.julialang.org/t/named-pipes/115627/2 "2024-06-14T00:25:25Z")
Just use shell mode to create one?
```julia-repl
shell> mkfifo mynamedpipe
julia> write("mynamedpipe", "whateverdatayouwant")
... (blocked here)
```
And then in another terminal, do
```bash
cat mynamedpipe
whateverdatayouwant
```
Back in the REPL, `write` would have returned
```julia-repl
julia> write("mynamedpipe", "whateverdatayouwant") # Executed before
19
```
When writing scripts, you can just do `run(`mkfifo mynamedpipe`)`.
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 12:30am UTC](https://discourse.julialang.org/t/named-pipes/115627/3 "2024-06-14T00:30:51Z")
But then `mkfifo` is something that must be available (I guess probably it isn’t in general - for all platforms). I actually want to do that inside a library.
I think that’s the question: is there a `mkfifo` _in Julia_?
---
### Author:  [@danielmatz](https://discourse.julialang.org/u/danielmatz)
#### Post date: [June 14, 2024, 12:37am UTC](https://discourse.julialang.org/t/named-pipes/115627/4 "2024-06-14T00:37:49Z")
I think you can use the Sockets standard library.
[https://docs.julialang.org/en/v1/stdlib/Sockets/#Sockets.listen-Tuple{AbstractString}](https://docs.julialang.org/en/v1/stdlib/Sockets/#Sockets.listen-Tuple%7BAbstractString%7D)
---
### Author:  [@Dan](https://discourse.julialang.org/u/Dan)
#### Post date: [June 14, 2024, 1:19am UTC](https://discourse.julialang.org/t/named-pipes/115627/5 "2024-06-14T01:19:04Z")
In Julia 1.11 there is `Base.Libc.mkfifo` which wraps the relevant system call.
>
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 1:36am UTC](https://discourse.julialang.org/t/named-pipes/115627/6 "2024-06-14T01:36:28Z")
Any idea if this is/will be in `Compat` or if it is already in some package?
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 6:59am UTC](https://discourse.julialang.org/t/named-pipes/115627/7 "2024-06-14T06:59:58Z")
Here is a small example about named pipes in the docs:
[https://julia-doc.readthedocs.io/en/latest/manual/networking-and-streams/#a-simple-tcp-example](https://julia-doc.readthedocs.io/en/latest/manual/networking-and-streams/#a-simple-tcp-example)
---
### Author:  [@Sukera](https://discourse.julialang.org/u/Sukera)
#### Post date: [June 14, 2024, 7:17am UTC](https://discourse.julialang.org/t/named-pipes/115627/8 "2024-06-14T07:17:10Z")
> [@lmiq](#):
>
> Any idea if this is/will be in `Compat` or if it is already in some package?
This functionality is already provided by the Sockets stdlib. There’s no need to manually wrangle Libc.
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 11:35am UTC](https://discourse.julialang.org/t/named-pipes/115627/9 "2024-06-14T11:35:01Z")
Can someone give me a simple example on how to use `Sockets` for that? From the documentation I cannot figure it out.
For example, let us suppose I want to use `grep` to extract some lines from a string (I’m not _actually_ willing to do that, this is just a simple example to illustrate what I want):
```julia-repl
julia> str = """
abc
def
"""
" abc \n def\n"
julia> open("str.txt", "w") do io
println(io, str)
end
julia> gr = read(pipeline(`grep "abc" str.txt`), String)
" abc \n"
```
Here, I’m writing to disc an intermediate file `str.txt`. How can avoid that using `Sockets`?
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 12:52pm UTC](https://discourse.julialang.org/t/named-pipes/115627/10 "2024-06-14T12:52:12Z")
Here is an stripped down minimal example from the docs. Open up two REPLs, REPL1 is the data receiving part, and REPL2 is the sending part in this example:
REPL1:
```julia
using Sockets
server = listen("\\\\.\\pipe\\testsocket")
while true
sock = accept(server)
while isopen(sock)
println(readline(sock, keep=true))
end
end
```
REPL2:
```julia
using Sockets
clientside = connect("\\\\.\\pipe\\testsocket")
println(clientside,"Hello World from the Echo Server")
println(clientside,"Hello World from the Echo Server")
println(clientside,"Hello World from the Echo Server")
```
You will see the “Hello World” text send through the named pipe and printed on REPL1 three times.
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 1:13pm UTC](https://discourse.julialang.org/t/named-pipes/115627/11 "2024-06-14T13:13:00Z")
I appreciate that, but I do not understand how that relates to the problem of not wanting to generate an intermediate file in the for an external program.
ps: It may be that this is _not_ possible in general, and I having called this thread “Named pipes” just cause confusion.
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 1:33pm UTC](https://discourse.julialang.org/t/named-pipes/115627/12 "2024-06-14T13:33:07Z")
Oh, sorry, missed that point about intermediate file.
On the other hand this is not a regular file despite it looks like because of the path string `"\\\\.\\pipe\\testsocket"`. You can see that as a special file just like the unix files in /proc/…
Let us exactly know what you need, perhaps `named pipes` is really not the right solution. Let’s try to avoid the xy-problem 😉
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 1:45pm UTC](https://discourse.julialang.org/t/named-pipes/115627/13 "2024-06-14T13:45:15Z")
What I need is something like I posted above: [Named pipes - #9 by lmiq](https://discourse.julialang.org/t/named-pipes/115627/9)
Just the program is not `grep`, but another program which is compiled as a binary in Yggdrasil.
The programs reads an input file and returns a result. I currently create this input file explicitly as a temporary file in the hard disc, but I would like to avoid that.
It seems to me (this is new to me) that the `mkfifo` tool is the standard solution for what I want.
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 1:55pm UTC](https://discourse.julialang.org/t/named-pipes/115627/14 "2024-06-14T13:55:41Z")
> [@lmiq](#):
>
> The programs reads an input file and returns a result
Is this a given premise?
If yes, than you have to create a file.
If no, which means you are able to change this binary, than you are probably free to do whatever you want as interprocess communication.
For me it’s still not clear what problem we try to solve. The `grep` example above is special as `grep` accepts a stream from STDIO instead of a existing file. So you have an option without creating a new file. Does your binary accepts STDIO stream as well?
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 2:04pm UTC](https://discourse.julialang.org/t/named-pipes/115627/15 "2024-06-14T14:04:19Z")
> [@oheil](#):
>
> Does your binary accepts STDIO stream as well?
No, it does not.
> [@oheil](#):
>
> If no, which means you are able to change this binary, than you are probably free to do whatever you want as interprocess communication.
Well, yes and no. There is a library which could be used directly without the intermediate file, but would require the implementation of the interface directly to the library in Julia which is some additional work, particularly because everything is written in C++ and I don’t readily understand what’s going on in that interface. And there is the driver (main program) which is easy to use but requires the file as input.
For now I’m just writing the file to disc. I would be happy if I could avoid that, and it seemed to me that `mkfifo` was a mechanism to exactly do that, to create “a file” in the memory that can be used “as if it was a regular disc file” is these cases. But I may have misunderstood that.
(my tries with `mkfifo` in Julia 1.11 failed for now).
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 2:48pm UTC](https://discourse.julialang.org/t/named-pipes/115627/16 "2024-06-14T14:48:29Z")
From [https://www.gnu.org/software/coreutils/manual/html\_node/mkfifo-invocation.html#mkfifo-invocation](https://www.gnu.org/software/coreutils/manual/html_node/mkfifo-invocation.html#mkfifo-invocation)
```julia
A FIFO is a special file type that permits independent
processes to communicate. One process opens the
FIFO file for writing, and another for reading, after which
data can flow as with the usual anonymous pipe in shells or
elsewhere.
```
I am afraid, as far as I understand, that this is not what you can use as an shortcut to avoid creating a real file, but I have never tried. As far as I can see you only have the option to give the “io channel” a name, hence “named pipe”. This is not a file name. The OS is doing the real thing under the hood and gives access using this name. There is probably somewhere in the file system a special file, perhaps in /tmp/… on unix, which is the visible named pipe, but still it’s a special file. If you can open this with your binary just pretending its a normal file is questionable but something you can try.
On which OS are you working? I want to do some experiments this evening with your issue (if my evening expectations will come true).
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [June 14, 2024, 2:51pm UTC](https://discourse.julialang.org/t/named-pipes/115627/17 "2024-06-14T14:51:32Z")
> [@oheil](#):
>
> On which OS are you working?
I’m on Linux (ideally I would like a cross-platform solution, of course).
> [@oheil](#):
>
> (if my evening expectations will come true).
Try having kids 🤣
---
### Author:  [@ericphanson](https://discourse.julialang.org/u/ericphanson)
#### Post date: [June 14, 2024, 5:50pm UTC](https://discourse.julialang.org/t/named-pipes/115627/18 "2024-06-14T17:50:34Z")
here is that example using `mkfifo`, in case it’s helpful:
```julia
julia> str = """
abc
def
"""
" abc \n def\n"
julia> run(`mkfifo my_named_pipe`)
Process(`mkfifo my_named_pipe`, ProcessExited(0))
julia> t1 = Threads.@spawn open("my_named_pipe", "w") do io
println(io, str)
end
Task (runnable) @0x000000032063aef0
julia> t2 = Threads.@spawn read(`grep "abc" my_named_pipe`, String)
Task (runnable) @0x000000032063b080
julia> wait.([t1, t2])
2-element Vector{Nothing}:
nothing
nothing
julia> fetch(t2)
" abc \n"
```
(I know the discussion has moved on to sockets though)
---
### Author:  [@oheil](https://discourse.julialang.org/u/oheil)
#### Post date: [June 14, 2024, 7:06pm UTC](https://discourse.julialang.org/t/named-pipes/115627/19 "2024-06-14T19:06:26Z")
My experiments told me that `mkfifo` indeed works as you are expecting it, but only for linux/unix.
`mkfifo` creates a special file (in linux). This file can just be read just like a regular file (checked with `grep` and `cat`). You can write into the named pipe, e.g.:
```julia
mkfifo /tmp/myNamedPipe
echo "Hello World" > /tmp/myNamedPipe
```
Now you can read from this named pipe with any program which expects opening a file with given name:
```julia
cat /tmp/myNamedPipe
```
(you need a second shell for this simple example, as `echo` waits until all data is read from the pipe with `cat`).
For windows I don’t see an equivalent similar easy pendant. Cross platform is not easy. At least, together with @ericphanson solution, for linux the way to go should be feasible now.