Okay that’s what I thought, it’s just you said
just there, so I wanted the clarification
That’d be cool. I don’t have much of a need for it so I don’t have practice and still suck at concurrency.
That should likely also read @async instead.
One more example of concurrency being one of the two difficult problems in computer science:
- Concurrency
- Cache eviction
- Off-by-one errors
ugh I see now, sorry about that. Yep you’re right, I’ve edited the post.
2 Likes
EDIT: I think this is a bit out of topic, but so near that I am super curious to ask it here
Is there any feature @async gives you due to preventing the migration to another thread?
Both, @async tasks and @spawn tasks can still access all variables, or?
So why isn’t @async made identical to @spawn? Which user code could fail if the task can migrate?
okay
Base.run(`git add -A`)
Base.run(`git commit -m "mymessage")
how to make sure that those are run together and no other task could make another commit message or add other things?
Existing code relying on the fact that @async tasks don’t migrate (which was documented & expected behavior) would become incorrect. They could previously rely on e.g. threadid being stable, and allowing them to move to other threads breaks that.
The reason it’s being discouraged is because that lack of migration effectively also creates a bottleneck in the amount of concurrency you can use; all tasks spawned like that have to share one CPU thread after all, meaning you’re not going to use the full potential of your CPU.
Hide them behind a lock that other tasks have to acquire.
Of course, this is (in general) not possible with code you don’t control, but there’s pretty much nothing you can do about that case. Other non-Julia processes could also invoke git, for example. Locks in your process can’t ever protect you from that (which leads me to think that git also locks something like that on a per-operation basis, but that’s not generally something you’re going to be able to access from Julia).
1 Like
This exactly what I stated two times already as the fallback solution
Ah I see what you mean. I was under the false impression that you were trying to accumulate results from these git operations on multiple tasks without them interferining with eachother. My bad. (In my defence, this is exactly why I asked for a dummy example)
In general, you’re asking here for julia to stop git from comitting a concurrency bug, which just isn’t possible. Probably yeah, the best you can do is lower the likelihood of a concurrency bug in git being triggered from the julia side by using locks.
1 Like
Thank you.
I can assure that no other process is running - only Julia. Hence it is already a huge step forward to get control over Julia’s concurrancy.
Thank you all for this intense and super insightful discussion! I will start to use Threads.@spawn always 
(I never read the @async documentation again after starting to use it).
That sounds like a classical concurrency problem, i.e., accessing a shared resource with transaction semantics for multiple operations. As your resource (git) does not itself ensure atomicity, consistency etc, you will need to make sure that only a single part of your program can interact with the resource (this has nothing to do with where and how this part is executing, e.g., tasks, threads, but just that no other part of the program can concurrently act on the resource). Classical solutions would be locking the critical region or publishing transactions specifying all the commands you want to run on a channel and have a single task executing all commands of a transaction before fetching the next.
3 Likes