Save a vector to a txt file asynchronously

New to Julia
1

Everytime I want to save a vector to a txt file the code is hanging…I need a copy of the vector to be passed to function and for it to be saved while the code is running. The saving is only taking a 10th of a second which is not accounting for about a 10x slowdown in julia from java…not sure if this is due to using bigints?

This code was written by chatgpt but the code still hangs…I dont need true concurency just for the compiler to not hang while the file is saving…

using Random, Primes, BenchmarkTools, DelimitedFiles, Profile

function savelr(vec::Vector{Int64}, fn::String)
    save_task = @async begin
		writedlm(fn, vec)
	end
    fetch(save_task)
	println("saved $fn")
end
2

I’m not entirely clear what you’re trying to do here, but spawning one task and then waiting for it is kind of pointless. This function is functionally equivalent to this:

using Random, Primes, BenchmarkTools, DelimitedFiles, Profile

function savelr(vec::Vector{Int64}, fn::String)
    writedlm(fn, vec)
    println("saved $fn")
end
1 Like
3

This code works but I feel it is stalling …so maybe it does need its own thread…I wonder if this is what is causing my code to run so slow…

Also maybe there is a cleaner way to do this…I was watching someone code a quadratic sieve in java and I noticed he would wrap this stuff in a lambda function(I think)

function savelr(vec::Vector{Int64}, fn::String)
  	@async (Asavelr(vec,fn))
end
function Asavelr(vec::Vector{Int64}, fn::String)
  writedlm(fn, vec)
  println("saved $fn")
end

PS and if I just call Asavelr directly I will run into a problem where my vector changes before I can save it. Not sure if this works in Julia or not…if Julia is passing a pointer to my array then I am in problems…hehe

Okay this might actually be working because I tried commenting out the file writing and the code still hangs so its another issue…My question is…is this a correct way of saving files asychronously?

4

You will still encounter this problem with the code as it is. Julia ‘passes by sharing’, so in this case any mutations to your vec array will affect the writing.

If you really need asynchronous writing of the array whilst you go on to do other work, this will work just fine:

@async writedlm(filename, copy(vector))

Or alternatively run that in its own thread:

@Threads.spawn writedlm(filename, copy(vector))

Note that if the contents of the vector was not isbits, you would need to use deepcopy().

Having said all that, unless the writing really is a bottleneck to your code (benchmark!), the simpler, synchronous version will make life easier.

5

Thanks for the tips…I am playing with a quadratic sieve so it’s very computationaly intensive. I really do think it’s necesary to do at least the file saving asychronasly.

The vector is of int64s and in java passing a copy doesnt work You actually need to pass the copy of the vector to a function and then call the thread inside the function.

6

Ok I tried doing the deepcopy and it works…seems to be faster also…The problem in Java was that I was using copy and not deepcopy.

@async writedlm("lr$(deepcopy(happy)).txt",deepcopy(lr))

I dont know if its a problem but when I read from the file it gives me a matrix…which was causing me errors for my save file function(which was waiting for a vector) which is why I am converting to your way.

And my code is running as fast or faster than Java…The file writing is orders of magnatude faster because I was using the processing library to write my code.

1 Like
7

Turns out the deepcopy command is happening asynchronously which means your way of doing it will work most times but the code below is broken using your method:

|||||@async writedlm(gcdcomp.txt,deepcopy(gcdcomp))|
|---|---|---|---|---|
|||||@async writedlm(newprimes.txt,deepcopy(np))|
|||||@async writedlm(mc.txt,deepcopy(mc))|
|||||println(Saving the state of the program.....)|
|||||push!(gcdcomp,rm)|

The push statement executes before the deepcopy which means this is not a secure way to save asychronously…you need to send the array to a function…you say this shouldnt work but it does…maybe the compiler is smart enough to understand what I want to do…not sure…also a self correction java does not have a deepcopy command…I was thinking of clone which only passes a reference to the original array.

8

You’re explicitly asking for those entire expressions to be evaluated asynchronously…

1 Like
9

Exactly!!! everything to the right executes asynchronously and thats the problem with torrance´s solution…However he did make a very helpful point about the arrays needing to be passed as deepcopys…because I wasnt doing that my linear relation number 6 was being stored in lr5.txt for some reason…My new code is as follows…I think this will probably be the end of the discusion:

function savelr(vec, fn)
  	@async writedlm(fn, vec)
end

savelr(deepcopy(lr),"lr$(deepcopy(filename)).txt")

Thanks a lot for the help…