Myujth

My real question is, if secrets are mounted as volumes in pods - can they be read if someone gains root access to the host OS.

For example by accessing /var/lib/docker and drilling down to the volume.

If someone has root access to your host with containers, he can do pretty much whatever he wants... Don't forget that pods are just a bunch of containers, which in fact are processes with pids. So for example, if I have a pod called sleeper:

kubectl get pods sleeper-546494588f-tx6pp -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE
sleeper-546494588f-tx6pp 1/1 Running 1 21h 10.200.1.14 k8s-node-2 <none>

running on the node k8s-node-2. With root access to this node, I can check what pid this pod and its containers have (I am using containerd as container engine, but points below are very similar for docker or any other container engine):

[root@k8s-node-2 /]# crictl -r unix:///var/run/containerd/containerd.sock pods -name sleeper-546494588f-tx6pp -q
ec27f502f4edd42b85a93503ea77b6062a3504cbb7ac6d696f44e2849135c24e
[root@k8s-node-2 /]# crictl -r unix:///var/run/containerd/containerd.sock ps -p ec27f502f4edd42b85a93503ea77b6062a3504cbb7ac6d696f44e2849135c24e
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT POD ID
70ca6950de10b 8ac48589692a5 2 hours ago Running sleeper 1 ec27f502f4edd
[root@k8s-node-2 /]# crictl -r unix:///var/run/containerd/containerd.sock# inspect 70ca6950de10b | grep pid | head -n 1
 "pid": 24180,

And then finally with those information (pid number), I can access "/" mountpoint of this process and check its content including secrets:

[root@k8s-node-2 /]# ll /proc/24180/root/var/run/secrets/kubernetes.io/serviceaccount/
total 0
lrwxrwxrwx. 1 root root 13 Nov 14 13:57 ca.crt -> ..data/ca.crt
lrwxrwxrwx. 1 root root 16 Nov 14 13:57 namespace -> ..data/namespace
lrwxrwxrwx. 1 root root 12 Nov 14 13:57 token -> ..data/token
[root@k8s-node-2 serviceaccount]# cat /proc/24180/root/var/run/secrets/kubernetes.io/serviceaccount/namespace ; echo
default
[root@k8s-node-2 serviceaccount]# cat /proc/24180/root/var/run/secrets/kubernetes.io/serviceaccount/token | cut -d'.' -f 1 | base64 -d ;echo 
"alg":"RS256","kid":""
[root@k8s-node-2 serviceaccount]# cat /proc/24180/root/var/run/secrets/kubernetes.io/serviceaccount/token | cut -d'.' -f 2 | base64 -d 2>/dev/null ;echo 
"iss":"kubernetes/serviceaccount","kubernetes.io/serviceaccount/namespace":"default","kubernetes.io/serviceaccount/secret.name":"default-token-6sbz9","kubernetes.io/serviceaccount/service-account.name":"default","kubernetes.io/serviceaccount/service-account.uid":"42e7f596-e74e-11e8-af81-525400e6d25d","sub":"system:serviceaccount:default:default"

It is one of the reasons why it is super important to properly secure access to your kubernetes infrastructure.