Measuring network I/O of Master/Worker communication

networking
parallel

#1

hello,

Is there any way to measure the network traffic from/to workers? I mean something like Docker NET I/O stats: https://docs.docker.com/engine/reference/commandline/stats/#examples

We need to measure how much data was exchanged between a worker and the master during our experiments.

Cheers,


#2

Not that I am aware off, but that would be a cool addition! Maybe @shashi and @amit.murthy have some additional input.


#3

Hello,

Unfortunately the module does not record any statistics w.r.t. bytes sent/recd at a socket connection level. I can think of a couple of indirect ways to get this information though.

  1. If the workers are on a remote host you can look at the system interface stats and compute it. If on the same host you will need to setup virtual interfaces first and use them.

  2. It is possible to the underlying socket fd for each connection to the workers. There may be relevant stats on a per-fd level in /proc - I haven’t looked at this though.

Amit