Limiting the Bandwidth Available to Pods

Using Kubernetes Resource Quotas you can limit number of Kubernetes objects, compute resources (CPU, RAM memory) and Persistent Volumes storage size in the project or multiple projects as per documentation. However with Resource Quotas you can't set ingress and egress traffic bandwidth quotas on a pod level. In this blog post I'll explain how you can do that in OpenShift 3.x or 4.x and Kubernetes. Remember this fuctionality must be supported by your SDN network plugin and is required for CNI networking plugins.

 

You can apply quality-of-service traffic shaping to a pod and effectively limit its available bandwidth. Egress traffic (from the pod) is handled by policing, which simply drops packets in excess of the configured rate. Ingress traffic (to the pod) is handled by shaping queued packets to effectively handle data. The limits you place on a pod do not affect the bandwidth of other pods.

To limit the bandwidth on a pod:

1. Write an object definition and specify the data traffic speed using kubernetes.io/ingress-bandwidth and kubernetes.io/egress-bandwidth annotations. For example, to limit both pod egress and ingress bandwidth to 10M/s in pod definition:

{
    "kind": "Pod",
    "spec": {
        "containers": [
            {
                "image": "openshift/hello-openshift",
                "name": "hello-openshift"
            }
        ]
    },
    "apiVersion": "v1",
    "metadata": {
        "name": "iperf-slow",
        "annotations": {
            "kubernetes.io/ingress-bandwidth": "10M",
            "kubernetes.io/egress-bandwidth": "10M"
        }
    }
}

or in OpenShift deployment config definition in template section:

{
    "apiVersion": "apps.openshift.io/v1",
    "kind": "DeploymentConfig",
    ...
    "spec": {
        ...
        "template": {
            "metadata": {
                "annotations": {
                    "kubernetes.io/egress-bandwidth": "10M",
                    "kubernetes.io/ingress-bandwidth": "10M",
                    "openshift.io/generated-by": "OpenShiftWebConsole"
                },
                ...
             },
             ...
         },
         ...
    },
    ...
}

2. Create the pod using the object definition:

oc create -f <file_or_dir_path>