In one of my previous posts I've described 3 ways to collect metrics stored in OpenShift built-in Prometheus metrics database. Now I'd like to show you how you can limit access to metrics per tenant projects (namespaces).
Built-in Thanos Queries contains dedicated tenancy port which requires namespace parameter to access metrics of objects belonging to the project. If you want to expose this port outside of the cluster you'll need to create custom route (ingress):
kind: Route
apiVersion: route.openshift.io/v1
metadata:
name: multitenant-thanos-querier
namespace: openshift-monitoring
spec:
host: ROUTE_HOSTNAME
path: /api
to:
kind: Service
name: thanos-querier
weight: 100
port:
targetPort: tenancy
tls:
termination: reencrypt
wildcardPolicy: None
Next you can execute follwing commands to query metrics from selected namespace:
PROJECT=sample-app-prod
SA=querier
oc project $PROJECT
oc create sa $SA
TOKEN=$(oc sa get-token $SA)
URL=$(oc get route -n openshift-monitoring | grep multitenant | awk '{print $2}')
You don't need to assign cluster-monitoring-view role to the service account but only view role in the project where you want to query the metrics:
oc adm policy add-cluster-role-to-user view -z $SA
In the thanos querier query you must enter namespace parameter to specify which namespace metrics you want to get:
curl -v -k -H "Authorization: Bearer $TOKEN" "https://$URL/api/v1/query?namespace=$PROJECT&query=kube_pod_status_ready"
In the results you will only see values of metrics which are related to your selected project:
{"status":"success","data":{"resultType":"vector","result":[{"metric":{"__name__":"kube_pod_status_ready","condition":"false","container":"kube-rbac-proxy-main","endpoint":"https-main","job":"kube-state-metrics","namespace":"sample-app-prod","pod":"hello-quarkus-5859859f9f-vbhjh","prometheus":"openshift-monitoring/k8s","service":"kube-state-metrics"},"value":[1651068513.646,"0"]}]}}
That's it. Now you only have access to metrics from the projects where you have the view role.
