3 weeks ago I opened an help request for encrypting at rest here: Where can I find the key_id returned by the EncryptRequest procedure call and the key_id returned by the Status ? · Issue #2 · kubernetes/kms · GitHub but I didn’t receive any answers
Cluster information:
Kubernetes version:
root@k8s-eu-1-master:~# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.2", GitCommit:"fc04e732bb3e7198d2fa44efa5457c7c6f8c0f5b", GitTreeState:"clean", BuildDate:"2023-02-22T13:39:03Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.2", GitCommit:"fc04e732bb3e7198d2fa44efa5457c7c6f8c0f5b", GitTreeState:"clean", BuildDate:"2023-02-22T13:32:22Z", GoVersion:"go1.19.6", Compiler:"gc", Platform:"linux/amd64"}
root@k8s-eu-1-master:~#
Cloud being used: (put bare-metal if not on a public cloud) : bare-metal Contabo Cloud
Installation method: Host OS: Ubuntu 22.04
CNI and version:
root@k8s-eu-1-master:/etc/cni/net.d# ls -lah
total 16K
drwx------ 2 root root 4.0K Mar 13 17:57 .
drwxr-xr-x 3 root root 4.0K Mar 12 20:11 ..
-rw-r--r-- 1 root root 666 Apr 12 08:51 10-calico.conflist
But :
root@k8s-eu-1-master:/etc/cni/net.d# kubectl calico -h
error: unknown command “calico” for “kubectl”
CRI and version:
containerd.service - containerd container runtime
Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-04-12 08:49:12 CEST; 3 weeks 1 day ago
Docs: https://containerd.io
Process: 451 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Main PID: 457 (containerd)
Tasks: 102
Memory: 108.0M
CPU: 9h 22min 6.403s
CGroup: /system.slice/containerd.service
├─457 /usr/bin/containerd
├─603 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id 33e2e09ecaf2d6db85f3904a874aeba88a030a27690032412fc8b22437bc2504 -address /run/containerd/containerd.sock
├─604 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id 841163836577533de7c856a3df2ea3c8846b69dbd2e6f707a19845f47bc9c1c0 -address /run/containerd/containerd.sock
├─610 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id 393aef04fb4d1b189f52c655713fd63348a82b409555bdd4252026b3246f7167 -address /run/containerd/containerd.sock
├─633 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id a750c63ff1ff09d90d40eebbaf6656e553e189e6cd76bc121427fea026ae9354 -address /run/containerd/containerd.sock
├─929 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id e2cd3ab55059ea5eafddab5597ba198efba7186e4a62ff25b0108eeb1a998f44 -address /run/containerd/containerd.sock
└─956 /usr/bin/containerd-shim-runc-v2 -namespace k8s.io -id 73b9de69c53f9799e417e15c08f59e2ae5a1366d98d1f62a60cdabc72e422756 -address /run/containerd/containerd.sock
Notice: journal has been rotated since unit was started, output may be incomplete.
root@k8s-eu-1-master:/etc/cni/net.d#
root@k8s-eu-1-master:~# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/arango-kube-arangodb-operator-844d65cdd5-pl4t2 1/1 Running 11 (7d20h ago) 26d
pod/cluster-agnt-apt6zd1n-75b1c6 1/1 Running 0 26d
pod/cluster-agnt-mwchqsli-75b1c6 1/1 Running 0 26d
pod/cluster-agnt-uhbsq34m-75b1c6 1/1 Running 0 26d
pod/cluster-crdn-2avzgbi1-75b1c6 1/1 Running 0 26d
pod/cluster-crdn-ets3nigq-75b1c6 1/1 Running 0 26d
pod/cluster-crdn-n3r0x9by-75b1c6 1/1 Running 0 26d
pod/cluster-prmr-07wqascu-75b1c6 1/1 Running 0 26d
pod/cluster-prmr-kenmxcns-75b1c6 1/1 Running 0 26d
pod/cluster-prmr-nsgmx9lg-75b1c6 1/1 Running 0 26d
pod/local-storage-4hkdx 1/1 Running 0 26d
pod/local-storage-5xhmh 1/1 Running 0 26d
pod/local-storage-b654d 1/1 Running 0 26d
pod/local-storage-mvndq 1/1 Running 0 26d
pod/local-storage-rk85s 1/1 Running 0 26d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/arango-kube-arangodb-operator ClusterIP 10.99.176.37 <none> 8528/TCP,8628/TCP,8728/TCP 26d
service/cluster ClusterIP 10.100.129.11 <none> 8529/TCP 26d
service/cluster-agent-apt6zd1n ClusterIP 10.100.221.87 <none> 8529/TCP 26d
service/cluster-agent-mwchqsli ClusterIP 10.109.165.148 <none> 8529/TCP 26d
service/cluster-agent-uhbsq34m ClusterIP 10.101.160.58 <none> 8529/TCP 26d
service/cluster-coordinator-2avzgbi1 ClusterIP 10.107.1.167 <none> 8529/TCP 26d
service/cluster-coordinator-ets3nigq ClusterIP 10.107.55.242 <none> 8529/TCP 26d
service/cluster-coordinator-n3r0x9by ClusterIP 10.99.135.125 <none> 8529/TCP 26d
service/cluster-dbserver-07wqascu ClusterIP 10.104.138.185 <none> 8529/TCP 26d
service/cluster-dbserver-kenmxcns ClusterIP 10.96.32.137 <none> 8529/TCP 26d
service/cluster-dbserver-nsgmx9lg ClusterIP 10.101.167.36 <none> 8529/TCP 26d
service/cluster-ea NodePort 10.96.65.142 <none> 8529:32527/TCP 26d
service/cluster-int ClusterIP None <none> 8529/TCP 26d
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 52d
service/local-storage ClusterIP 10.97.243.65 <none> 8929/TCP 26d
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/local-storage 5 5 5 5 5 <none> 26d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/arango-kube-arangodb-operator 1/1 1 1 26d
NAME DESIRED CURRENT READY AGE
replicaset.apps/arango-kube-arangodb-operator-844d65cdd5 1 1 1 26d
root@k8s-eu-1-master:~#
You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.
arangoDeployment.yaml :
apiVersion: "database.arangodb.com/v1"
kind: "ArangoDeployment"
metadata:
name: "cluster"
spec:
mode: Cluster
agents:
volumeClaimTemplate:
spec:
storageClassName: my-local-ssd
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
volumeMode: Filesystem
dbservers:
volumeClaimTemplate:
spec:
storageClassName: my-local-ssd
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
volumeMode: Filesystem
arangoLocalStorage.yaml :
apiVersion: "storage.arangodb.com/v1alpha"
kind: "ArangoLocalStorage"
metadata:
name: "local-storage"
spec:
storageClass:
name: my-local-ssd
isDefault: true
localPath:
- /mnt/big-ssd-disk
encryptionConfiguration.yaml :
# https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/#understanding-the-encryption-at-rest-configuration
# https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/
# https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/#implementing-a-kms-plugin
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
- secrets
- configmaps
- pandas.awesome.bears.example
providers:
- kms:
keys:
apiVersion: v2
name: myKmsPlugin
endpoint: unix:///tmp/socketfile.sock
cachesize: 100
timeout: 3s
In Using a KMS provider for data encryption | Kubernetes :
The API server considers the key_id returned from the Status procedure call to be authoritative. Thus, a change to this value signals to the API server that the remote KEK has changed, and data encrypted with the old KEK should be marked stale when a no-op write is performed (as described below). If an EncryptRequest procedure call returns a key_id that is different from Status, the response is thrown away and the plugin is considered unhealthy. Thus implementations must guarantee that the key_id returned from Status will be the same as the one returned by EncryptRequest. Furthermore, plugins must ensure that the key_id is stable and does not flip-flop between values (i.e. during a remote KEK rotation)
The StatusResponse struct kms/api.pb.go at master · kubernetes/kms · GitHub has a field KeyId , but the EncryptRequest struct has Uid field
Where can I find the key_id returned by the EncryptRequest procedure call, in order to compare this key_id with the one returned by the Status ?
By the way, with :
func (s *Server) getStatus(ctx context.Context, req *kms.StatusRequest) (*kms.StatusResponse) {
resp, _ := s.Status(ctx, req)
return resp
}
func (s *Server) Encrypt(ctx context.Context, req *kms.EncryptRequest) (*kms.EncryptResponse) {
UId := req.Uid
log.Println("UId-EncryptRequest: ", UId)
statusReq := s.StatusRequest{
XXX_NoUnkeyedLiteral: {},
XXX_unrecognized: []byte,
XXX_sizecache: 10,
}
statusKId, _ := s.Status(ctx, statusReq)
if (statusKId.KeyId != UId ) {
log.Printf("Watch out!")
return nil
}
resp := s.Encrypt(ctx, req)
log.Printf("Encrypt-resp: ", resp)
return resp
}
I get this other error s.StatusRequest is not a type :
root@k8s-eu-1-master:~/kms/apis/v2/api# go run ./server/main.go
# command-line-arguments
server/main.go:58:24: s.StatusRequest is not a type
server/main.go:59:39: missing type in composite literal
but here the StatusRequest struct type is defined : kms/api.pb.go at a38ec9832062d14d72f0f4cee8d6904a8be5fbee · kubernetes/kms · GitHub
Where can I find the key_id returned by the EncryptRequest procedure call and the key_id returned by the Status ?
How to correctly setup the *Encryption at Rest* ?
