Blog/Disable Network Offloading on XenServer and XCP-ng

2020-11-02: How to disable network offloading on XenServer / Xcp-ng

Network cards usually have support to offload some TCP/IP features from the CPU. This leads to lower CPU usage under load.

However NIC offloading can also cause issues in virtualised networks. For examples dropped packets or odd performance metrics.

Here is a quick and dirty script that will turn off various offloading features in your XenServer/XCP-ng pool.

#!/bin/bash

###
# Disable offloading on virtual and physical network interfaces the pool.
#
# References:
# https://xcp-ng.org/docs/guides.html#_3-disable-tx-checksum-offload
# https://support.citrix.com/article/CTX212540
###

echo -e "# Disabling PIF offloading..\n\n"
for uuid in `xe pif-list | awk '/uuid/ {print $5}'| sed '/^$/d'`
do
        echo Disabling offloading on PIF=$uuid
        xe pif-param-set uuid=$uuid other-config:ethtool-gso="off"
        xe pif-param-set uuid=$uuid other-config:ethtool-ufo="off"
        xe pif-param-set uuid=$uuid other-config:ethtool-tso="off"
        xe pif-param-set uuid=$uuid other-config:ethtool-sg="off"
        xe pif-param-set uuid=$uuid other-config:ethtool-tx="off"
        xe pif-param-set uuid=$uuid other-config:ethtool-rx="off"
        echo -e "..Done\n\n"
done

echo -e "# Disabling VIF offloading..\n\n"
for uuid in `xe vif-list | awk '/uuid/ {print $5}'| sed '/^$/d'`
do
        echo Disabling offloading on VIF=$uuid
        xe vif-param-set uuid=$uuid other-config:ethtool-gso="off"
        xe vif-param-set uuid=$uuid other-config:ethtool-ufo="off"
        xe vif-param-set uuid=$uuid other-config:ethtool-tso="off"
        xe vif-param-set uuid=$uuid other-config:ethtool-sg="off"
        xe vif-param-set uuid=$uuid other-config:ethtool-tx="off"
        xe vif-param-set uuid=$uuid other-config:ethtool-rx="off"
        echo -e "..Done\n\n"
done

For a full list of features that can be changed, have a look at the ethtool man page

Use ethtool -k <nic> to list all features supported by the network card:

# ethtool -k eth0
Features for eth0:
rx-checksumming: on
tx-checksumming: on
        tx-checksum-ipv4: off [fixed]
        tx-checksum-ip-generic: on
        tx-checksum-ipv6: off [fixed]
        tx-checksum-fcoe-crc: off [fixed]
        tx-checksum-sctp: off [fixed]
scatter-gather: on
        tx-scatter-gather: on
        tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
        tx-tcp-segmentation: on
        tx-tcp-ecn-segmentation: off [fixed]
        tx-tcp-mangleid-segmentation: off
        tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]

Features labled [fixed] cannot be changed on that NIC.