How do computation while transferring data between processes


#1

Hi, there

I do data tranferring via RemoteChannel.
But I found it is impossible that Julia process run sending, receiving and computation at once.
RemoteChannel has its own process and its ID which is determined by an argument when it is created.
When a RemoteChannel is requested by receiving or sending data by a call put! or take!, if the process it belongs to is busy, then data movement call would be blocked to wait for data until the process is available.

There is an example below.
What I want is the second communication does not block, even if it is requested in the middle of other computation.

addprocs(1)
@everywhere BLAS.set_num_threads(1)

@everywhere function do_work(recv_channel, send_channel)
	mat = ones(5000,5000) # for heavy computation
	while true
		data1 = take!(recv_channel)
		println(data1)
		data2 = take!(recv_channel)
		println(data2)
		put!(send_channel, data1)
		put!(send_channel, data2)
		mat *= mat # This line takes long time.
		println("done")
	end	
end

function main()
	mat = ones(5000,5000)
	t = @elapsed mat *= mat;
	println("mat mul takes ", t, " seconds")	

	recv_channel = RemoteChannel(2)
	send_channel = RemoteChannel(2)
	remote_do(do_work, 2, send_channel, recv_channel)
	
	put!(send_channel, "hello")
	put!(send_channel, "world")
	first = @elapsed data = take!(recv_channel) # Because this put! function was waiting in advance, it can receive the data immediately.
	println(data, " takes ", first)
	sleep(t/10)
	second = @elapsed data = take!(recv_channel) # This take! function should be blocked
	println(data, " takes ", second)
	sleep(t/10)
end

main()