Help on Live Plotting data which are getting appended to file(.txt)

General Usage
, ,
1

I am very new to Julia, I have a question for the experts, currently, I am appending bunch of data to a text file using c++ , the data is just numbers separated by comma and i just keep apending more numbers in new line of the same text file I was planning to make a 3d plot of these data as they get appended by Julia (basically live ).

the problem starts here that I do not know how to check in Julia that my text file has gotten appended to and I have no clue how to get the last line of the text file after I found out it got updated efficiently.

I googled and for the first portion of my problem found watch_file(path::AbstractString, timeout_s::Real=-1) but I feel this will be slow to plot something live. correct me if I am wrong?
for the second portion, I found that the only way to get the last line is to read the whole thing then I grab the last line and I think that will become very slow as the file starts to become bigger and bigger so is there any other way to do that portion?

in general, do you guys suggest a better way to do this in the first place ??

lastly, to live plot in 3d, do you guys have any suggestions?

2

Here is a julia script that prints numbers slowly:

function myplot()
    θ = 0.0
    for r in 0.0:.01:1.0
        x = r * cos(θ)
        y = r * sin(θ)
        println(x, "  ", y)
        θ += .2
        sleep(0.1)
    end
end

myplot()

You can live plot this with gnu plotutils (this is not gnuplot)

julia ppoints.jl  | graph -x -1 1 -y -1 1 -T X

This uses the X window system. I’m not sure what to do if you have a mac or win32 platform without X. You can also plot into the tektronix emulator that is built into xterm.

This looked pretty slick when RSM showed it to me in 1990 or so. Integrating it with dashboards, notebooks, etc., may be somewhat problematic. Finally, I don’t think you can do 3d plotting this way. At least not directly.

1 Like
3

This would be my attempt, don’t know if it even compiles, it’s off the top of my head and probably can be optimized. And you need to figure out how to identify when the loop is done and exit…maybe if watch_file() times out.

function watch(filename)
    io = open(filename; read=true)
    leftover = Vector{UInt8]()

    while true
        bytes = readavailable(io)
        while length(bytes) == 0
             watch_file(filename, 300)
             bytes = readavailable(io)
        end
        leftover = vcat(leftover, bytes)  
        endofline = -1
        for (i, v) in Iterators.reverse(enumerate(leftover))
            if v == 0xa0
                endofline = i
                break
            end
        end
        if endofline != -1
            newlines = String(leftover[1:endofline])
            leftover = leftover[endofline+1:length(leftover)]
            # Process lines in newlines here and plot
        end
    end
end
1 Like