Hmm. 26 GBit/sec von 40 naja, besser als 10, aber nur 3,75 sendend auf dem einen Server?
Hat jemand eine Idee? Wieso kommt iperf3 nur auf 3,75 GBit/sec, wenn die .2 an die .1 sendet, jedoch jeweils jenseits 25 GBit/sec, egal ob die .1 an die .2 sendet oder von der .2 empfängt? Also krasse Asymmetrie, je nachdem, wo iperf3 läuft?
root@basiliskn ~ # iperf3 -c 10.40.40.1 -t 60
Connecting to host 10.40.40.1, port 5201
[…]
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 26.2 GBytes 3.75 Gbits/sec 0 sender
[ 5] 0.00-60.00 sec 26.2 GBytes 3.75 Gbits/sec receiver
root@basiliskn ~ # iperf3 -c 10.40.40.1 -t 60 -R
Connecting to host 10.40.40.1, port 5201
Reverse mode, remote host 10.40.40.1 is sending
[…]
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 182 GBytes 26.1 Gbits/sec 474 sender
[ 5] 0.00-60.00 sec 182 GBytes 26.1 Gbits/sec receiver
Hingegen:
root@stanley:~# iperf3 -c 10.40.40.2 -t 60
Connecting to host 10.40.40.2, port 5201
[…]
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 186 GBytes 26.7 Gbits/sec 403 sender
[ 5] 0.00-60.00 sec 186 GBytes 26.7 Gbits/sec receiver
root@stanley:~# iperf3 -c 10.40.40.2 -t 60 -R
Connecting to host 10.40.40.2, port 5201
Reverse mode, remote host 10.40.40.2 is sending
[…]
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-60.00 sec 208 GBytes 29.8 Gbits/sec 10139 sender
[ 5] 0.00-60.00 sec 208 GBytes 29.8 Gbits/sec receiver
Setup:
root@basiliskn ~ # lscpu | egrep '(BIOS Model name|NUMA node)'
BIOS Model name: Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz CPU @ 2.3GHz
NUMA node(s): 2
NUMA node0 CPU(s): 0-15,32-47
NUMA node1 CPU(s): 16-31,48-63
root@basiliskn ~ # lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
root@stanley:~# lscpu | egrep '(BIOS Model name|NUMA node)'
BIOS Model name: Intel(R) Xeon(R) CPU E5-2683 v3 @ 2.00GHz CPU @ 2.0GHz
NUMA node(s): 2
NUMA node0 CPU(s): 0-13,28-41
NUMA node1 CPU(s): 14-27,42-55
root@stanley:~# lspci | grep Mellanox
04:00.0 Ethernet controller: Mellanox Technologies MT27520 Family [ConnectX-3 Pro]
Gretchenfrage: ist das ›nur‹ ein Meßfehler, oder sind derlei schlechte Werte auch zu erwarten, wenn das andere Ende vom DAC in einem Switch statt in einem Server steckt?