# C program embedding Julia - works in terminal but breaks in pipes

**URL:** <https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973>\
**Category:** General Usage\
**Tags:** interoperability, io, c, piping\
**Created:** [September 9, 2022, 8:53am UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973 "2022-09-09T08:53:29Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [September 9, 2022, 8:53am UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/1 "2022-09-09T08:53:29Z")

</div>

I’d like to write a C program which embeds Julia code, and call this C program from an unrelated program using two-way pipes. While the C program works fine in the terminal, it breaks when called from pipes. Here’s a minimal working example, where the C program initializes the Julia runtime but does not run any actual Julia code. Instead, it runs C code which repeatedly reads integers from stdin and prints them out to stdout:

```
/* test1.c */
#include <julia.h>
#include <stdio.h>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i;
    jl_init();
    
    while (!feof(stdin)) {
        scanf("%d", &i);
        printf("%d\n", i);
    }

    jl_atexit_hook(0);
    return 0;
}

```

I compiled it with

```
gcc -o test1 -fPIC -I$JULIA_DIR/include/julia -L$JULIA_DIR/lib -Wl,-rpath,$JULIA_DIR/lib test1.c -ljulia

```

where `JULIA_DIR` points to my Julia 1.8.0 installation.

The program runs fine in the terminal: whenever I type in any integer, the same integer is printed out, as expected. However, when I try to call it from another program via two-way pipes, it breaks badly. I’d like to interface with a different program, but here I’ll just show the problem by calling the program from a Julia REPL:

```julia
julia> io = open(`./test1`, "r+")
Process(`./test1`, ProcessRunning)

julia> println(io, "10")

julia> readline(io)
"32766"

```

You can see that the output is completely wrong and seems to be an uninitialized integer. I strongly suspects that there is some conflict between Julia’s handling of stdin / stdout and stdio.h in C, because I don’t experience this problem when my C program only performs IO by executing Julia code containing e.g., `print()`. Is it possible that there should be some syncing between the IO buffers in C and in Julia?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [March 27, 2023, 4:58pm UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/2 "2023-03-27T16:58:25Z")

</div>

Running this with

```julia
/* test1.c */
#include <stdio.h>
#include <julia.h>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i = 1337;
    jl_init();

    while (!feof(stdin)) {
        scanf("%d", &i);
        printf("%d\n", i);
        fflush(stdout);
    }

    jl_atexit_hook(0);
    return 0;
}

```

Here’s the julia code in test1.jl

```julia
# test1.jl
io = open(`./test1`, "r+")
println(io, "10")
println(readline(io))
close(io)

```

Then I get the following result:

```julia
$ julia test1.jl
10

```

In the Julia REPL, I get the following result:

```julia
julia> begin
           io = open(`./test1`, "r+")
           println(io, "10")
           println(readline(io))
           close(io)
       end
10

julia> function f(n)
           io = open(`./test1`, "r+")
           println(io, "$n")
           println(readline(io))
           close(io)
       end
f (generic function with 1 method)

julia> f(900)
900

julia> io = open(`./test1`, "r+")
Process(`./test1`, ProcessRunning)

julia> println(io, "100");

julia> readline(io)
"1337"

julia> close(io)

```

If I do not do `jl_init()` then there is not issue:

```julia
/* test2.c */
#include <stdio.h>
#include <julia.h>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i = 1337;
    //jl_init();

    while (!feof(stdin)) {
        scanf("%d", &i);
        printf("%d\n", i);
        fflush(stdout);
    }

    //jl_atexit_hook(0);
    return 0;
}

```

```julia
julia> io = open(`./test2`, "r+")
Process(`./test2`, ProcessRunning)

julia> println(io, "100");

julia> readline(io)
"100"

julia> close(io)

```

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [March 27, 2023, 11:41pm UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/3 "2023-03-27T23:41:58Z")

</div>

> [@mkitti](#):
>
> ```julia
> # test1.jl
> io = open(`./test1`, "r+")
> println(io, "10")
> println(readline(io))
> close(io)
> 
> ```

I can break it with a single extra line, like this:

```julia
# test1_modified.jl
io = open(`./test1`, "r+")
sleep(1)
println(io, "10")
println(readline(io))
close(io)

```

Running `julia test1_modified.jl` now prints 1337. Adding `sleep()` probably makes it similar to the delay of a human user in your REPL example, which also prints out an unexpected result.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [March 29, 2023, 6:57am UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/4 "2023-03-29T06:57:42Z")

</div>

One brief observation is that I’m not sure if `stdin` is still in buffering mode. Effectively, `scanf` may not be blocking.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [March 29, 2023, 7:42am UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/5 "2023-03-29T07:42:07Z")

</div>

# Introduction

The buffering modes on `stdin` and `stdout` are different upon including julia.h. They are no longer in line buffer mode. See [`setvbuf`](https://linux.die.net/man/3/setvbuf).

This means that we need to check the status output of `scanf` before we print anything. It’s possible that `scanf` read nothing and did not update `i`. Thus we need to alter the C code as follows.

## Corrected code

```c
/* test5.c */
#include <stdio.h>
#include <julia.h>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i = 1337;
    int s = 0;
    jl_init();

    while (!feof(stdin)) {
        s = scanf("%d", &i);
        if (s > 0) {
            printf("%d\n", i);
            fflush(stdout);
        }
    }

    jl_atexit_hook(0);
    return 0;
}

```

Above we do the following.

1. Check if `scanf` actually read a value ands stored it in `i`
2. If so, print to stdout and flush stdout.

Reading the output in Julia, we now get the expected result.

```julia
julia> io = open(`./test5`, "r+")
Process(`./test5`, ProcessRunning)

julia> println(io, 10)

julia> println(readline(io))
10

julia> println(io, 30)

julia> println(readline(io))
30

julia> close(io)

```

## Prior code analysis

Previously, scanf was basically reading nothing and not updating `i`. It would print many lines. Consider the following code.

```c
/* test6.c */
#include <stdio.h>
#include <julia.h>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i = 1337;
    jl_init();

    while (!feof(stdin)) {
        i++;
        scanf("%d", &i);
        printf("%d\n", i);
        fflush(stdout);
    }

    jl_atexit_hook(0);
    return 0;
}

```

If we read from this in Julia, we could get the following:

```julia
julia> io = open(`./test6`, "r+")
Process(`./test6`, ProcessRunning)

julia> readline(io)
"1338"

julia> readline(io)
"1339"

julia> readline(io)
"1340"

julia> readline(io)
"1341"

julia> readline(io)
"1342"

julia> readline(io)
"1343"

julia> readline(io)
"1344"

julia> readline(io)
"1345"

julia> begin
           io = open(`./test6`, "r+")
           for line in Iterators.take(eachline(io), 100)
               println(line)
           end
       end
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
...

```

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [March 29, 2023, 11:40am UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/6 "2023-03-29T11:40:21Z")

</div>

Thanks for looking deep into this. That’s very illuminating. Now the slight annoyance is that after running

```julia
julia> io = open(`./test5`, "r+")

```

the process `test5` runs with 100% CPU usage. Presumably `scanf` is running continuously until it finds an integer input.

I guess the workarounds are either doing all IO on the Julia side or avoiding pipes.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [March 29, 2023, 1:13pm UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/7 "2023-03-29T13:13:59Z")

</div>

Just use `setvbuf` to set the buffering to whatever you would like.

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [March 29, 2023, 9:00pm UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/8 "2023-03-29T21:00:40Z")

</div>

I added

```
setvbuf(stdin, NULL, _IOLBF, 16384);

```

right after

```
jl_init();

```

to change `stdin` back to line-buffering mode, but this doesn’t seem to fix the CPU usage when idle. Maybe I got it wrong? I suspect it’s necessary to dig into libuv to fully resolve this, but I know too little about it.

P.S. I can get by without pipes. Still I’m just curious about the problem.

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [October 30, 2025, 2:43pm UTC](https://discourse.julialang.org/t/c-program-embedding-julia-works-in-terminal-but-breaks-in-pipes/86973/9 "2025-10-30T14:43:19Z")

</div>

I revisited this old problem. I’m actually embedding Julia in C++, and I asked GPT-5 how I can safely perform IO with std::iostream. GPT-5 came up with a solution, and I tested it to verify that it worked. Quoting from GPT-5 output:

> Why this happens
> 
> - jl\_init() sets up stdio through libuv and may change FILE\* buffering (setvbuf) and flags (e.g., O\_NONBLOCK). That can break assumptions about line buffering and can cause surprising behavior if you mix C stdio and C++ iostreams on the same descriptors.
> 
> Recommended approach (pure iostreams, decoupled from C stdio)
> 
> - Keep iostreams independent from C stdio and from the stdio FILE\* state that Julia changes.
> 
> - Use your own iostreams bound to fresh file descriptors by opening /dev/stdout and /dev/stderr (POSIX). Those new descriptors are blocking by default and aren’t affected by Julia’s setvbuf or O\_NONBLOCK on fd 1/2.

The corrected version of my original program on Linux, also changed to C++, is as follows:

```cpp
// test2.cpp

#include <julia.h>
#include <fstream>
#include <iostream>
JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[])
{
    int i;
    jl_init();

    // Decouple C++ iostreams from C stdio to avoid interference
    std::ios::sync_with_stdio(false);
    std::cin.tie(nullptr);
    
    // Create dedicated, blocking iostreams to the current stdin/stdout/stderr
    static std::ifstream safe_in("/dev/stdin");
    static std::ofstream safe_out("/dev/stdout");
    static std::ofstream safe_err("/dev/stderr");
    
    while (!safe_in.eof()) {
        safe_in >> i;
        safe_out << i << std::endl;
    }

    jl_atexit_hook(0);
    return 0;
}

```

I compile it with

```shell
g++ -o test2 -fPIC -I$JULIA_DIR/include/julia -L$JULIA_DIR/lib -Wl,-rpath,$JULIA_DIR/lib test2.cpp -ljulia

```

The program works correctly as a process in pipes:

```julia-auto
julia> io = open(`./test2`, "r+")
Process(`./test2`, ProcessRunning)

julia> println(io, "10")

julia> readline(io)
"10"

```

The minor annoyance is that I have to replace e.g. `std::cout` with the custom `safe_out` in the entire C++ program. Also, OS-dependent stuff like `/dev/stdin` would need to be changed on Windows.

P.S. the above program just enables IO on the C++ side. I haven’t tried if I can perform IO in both C++ and Julia.