Measuring network I/O of Master/Worker communication

General Usage
,
1

hello,

Is there any way to measure the network traffic from/to workers? I mean something like Docker NET I/O stats: docker stats | Docker Documentation

We need to measure how much data was exchanged between a worker and the master during our experiments.

Cheers,

2

Not that I am aware off, but that would be a cool addition! Maybe @shashi and @amit.murthy have some additional input.

3

Hello,

Unfortunately the module does not record any statistics w.r.t. bytes sent/recd at a socket connection level. I can think of a couple of indirect ways to get this information though.

  1. If the workers are on a remote host you can look at the system interface stats and compute it. If on the same host you will need to setup virtual interfaces first and use them.

  2. It is possible to the underlying socket fd for each connection to the workers. There may be relevant stats on a per-fd level in /proc - I haven’t looked at this though.

Amit

2 Likes
4

Thanks for your answer, @amit.murthy.

I have finally implemented a simple DockerBackend code which gets container stats, it will be here till I move it to a proper repository:

https://github.com/NaelsonDouglas/DistributedMachineLearningThesis/tree/master/src/docker

Best!