I want to calculate human chromosomes’s something, and then using each chromosomes for parallel computing, and finally write it into a csv file, but when I use the Threads.@threads macro, I get an error, and if I don’t add this macro, I can run it directly.
using CSV,DataFrames ...... Threads.@threads for number in 1:chr.ngroups#representing each chromosome chromosome=chr[number] start=chromosome[1,3];final=chromosome[end,3] chromosomename=chromosome[1,1] println('[',Dates.format(now(), "YYYY-m-d HH:MM:SS"),']'," Start calculating ",chromosomename,"'s pdrs") chdf=generatechdf(reader,chromosomename,start,final)#the function was defined by myself chdf=lastchdf(chdf) pdrs=repeat([(NaN,0,0,0)],nrow(chromosome)) pdrspos=calculatepospdrs(chdf,chromosome)#the function was defined by myself pdrsneg=calculatenegpdrs(chdf,chromosome)#the function was defined by myself pdrs[findall(==("C"),chromosome.Column2)]=pdrspos pdrs[findall(==("G"),chromosome.Column2)]=pdrsneg pd=vcat(pd,rename!(DataFrame(pdrs),[:pdr,:discordant,:sum,:allsum]))#this step can intergrate each chromosome println('[',Dates.format(now(), "YYYY-m-d HH:MM:SS"),']'," Finish calculating ",chromosomename,"'s pdrs") end [2022-10-18 14:40:23] Start calculating chr16's pdrs [2022-10-18 14:40:23] Start calculating chr21's pdrs [2022-10-18 14:40:23] Start calculating chr7's pdrs [2022-10-18 14:40:23] Start calculating chr1's pdrs ERROR: TaskFailedException Stacktrace:  wait @ ./task.jl:334 [inlined]  threading_run(func::Function) @ Base.Threads ./threadingconstructs.jl:38  top-level scope @ ./threadingconstructs.jl:97 nested task error: BGZFStreams.BGZFDataError("invalid gzip identifier") Stacktrace:
I still have a problem, I inside the for loop vcat each chromosome calculation results, but I want to let it go output according to the order of chromosome 1, 2, 3 and so on, but it isn’t parallel to calculate according to the order of 1, 2, 3（like 16 ,21 ,7 ,1）and next it will be different.And when they finally vcat two chromosomes is finished at the same time, then this is disordered?