@time reporting lock conflicts - what are these?

General Usage
1

When I use @time to time a particular function that reads and write from/to several large files, the output shows (for example):

0.403770 seconds (6.20 k allocations: 233.059 MiB, 8.12% gc time, 4 lock conflicts

I’ve never seen these lock conflicts reported before. Are they new in v1.11? What do they mean?

My code isn’t generating any other errors and, on the face of it, seems to be working. Are these lock conflicts important, can I ignore them, or do I need to eliminate them?

Thanks!

1 Like
2

It seems the line that is giving rise to these lock conflicts says this:

@time arrowTable = Arrow.Table(arrowfile)

It is called five times on 5 different files in one iteration. This generates (for example):

  0.000567 seconds (719 allocations: 34.938 KiB)
  0.000425 seconds (209 allocations: 12.234 KiB, 1 lock conflict)
  0.000492 seconds (218 allocations: 12.781 KiB, 1 lock conflict)
  0.000516 seconds (232 allocations: 13.188 KiB, 1 lock conflict)
  0.000417 seconds (220 allocations: 12.859 KiB, 2 lock conflicts)

So it seems that this is outside my control and probably unimportant (although “lock conflicts” sound like they should be important!)

I found this, which I probably should have found before posting.

2 Likes
3

Lock conflicts are when 2 different locations try to take the same lock at the same time (causing one to have to wait until the other is done). In this case, the lock is likely one in the IO system.

4 Likes
4

The open function takes a lock keyword argument. Setting lock = false will disable locking. If you’re not doing concurrent IO, I suppose you could safely turn off locking.

Also known as lock contention.

1 Like
5

Your code is fine. Like the other metrics reported by @time, lock conflicts can be problematic for performance, but not for correctness. Your code is successfully doing what you asked it to do.

Don’t try to eliminate them unless you know what you’re doing. That could lead to correctness issues.

3 Likes
6

Thank you for these replies.

The conflicts I’m seeing all seem to be arising from Arrow.jl and so I’m not going to try to resolve them. As I say, there is no other sign of trouble in my case.