How to access EFS from kubernetes sa in AWS?

I deployed an EFS in AWS and a test pod on EKS from this document: Amazon EFS CSI driver.

EFS CSI Controller pods in the kube-system

kube-system    efs-csi-controller-5bb76d96d8-b7qhk        3/3     Running   0          26s
kube-system    efs-csi-controller-5bb76d96d8-hcgvc        3/3     Running   0          26s

After deployed a sample application from the doc, when confirm efs-csi-controller sa pod logs, it seems they didn’t work well.

Pod 1

$ kubectl logs efs-csi-controller-5bb76d96d8-b7qhk \
>     -n kube-system \
>     -c csi-provisioner \
>     --tail 10
W1030 08:15:59.073406       1 feature_gate.go:235] Setting GA feature gate Topology=true. It will be removed in a future release.
I1030 08:15:59.073485       1 feature_gate.go:243] feature gates: &{map[Topology:true]}
I1030 08:15:59.073500       1 csi-provisioner.go:132] Version: v2.1.1-0-g353098c90
I1030 08:15:59.073520       1 csi-provisioner.go:155] Building kube configs for running in cluster...
I1030 08:15:59.087072       1 connection.go:153] Connecting to unix:///var/lib/csi/sockets/pluginproxy/csi.sock
I1030 08:15:59.087512       1 common.go:111] Probing CSI driver for readiness
I1030 08:15:59.090672       1 csi-provisioner.go:202] Detected CSI driver efs.csi.aws.com
I1030 08:15:59.091694       1 csi-provisioner.go:244] CSI driver does not support PUBLISH_UNPUBLISH_VOLUME, not watching VolumeAttachments
I1030 08:15:59.091997       1 controller.go:756] Using saving PVs to API server in background
I1030 08:15:59.092834       1 leaderelection.go:243] attempting to acquire leader lease kube-system/efs-csi-aws-com...

Pod 2

$ kubectl logs efs-csi-controller-5bb76d96d8-hcgvc \
>     -n kube-system \
>     -c csi-provisioner \
>     --tail 10
I1030 08:16:32.628759       1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:16:32.628783       1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 5
E1030 08:16:32.628798       1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:16:32.628845       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.628997       1 controller.go:1332] provision "default/efs-claim" class "efs-sc": started
I1030 08:17:04.629193       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "default/efs-claim"
I1030 08:17:04.687957       1 controller.go:1099] Final error received, removing PVC 111111a-d6fb-440a-9bb1-132901jfas from claims in progress
W1030 08:17:04.687977       1 controller.go:958] Retrying syncing claim "111111a-d6fb-440a-9bb1-132901jfas", failure 6
E1030 08:17:04.688001       1 controller.go:981] error syncing claim "111111a-d6fb-440a-9bb1-132901jfas": failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
I1030 08:17:04.688044       1 event.go:282] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"default", Name:"efs-claim", UID:"111111a-d6fb-440a-9bb1-132901jfas", APIVersion:"v1", ResourceVersion:"1724705", FieldPath:""}): type: 'Warning' reason: 'ProvisioningFailed' failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied

From events, I can see

$ kubectl get events
27m         Warning   FailedScheduling         pod/efs-app                                    skip schedule deleting pod: default/efs-app
7m38s       Warning   FailedScheduling         pod/efs-app                                    0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
7m24s       Warning   FailedScheduling         pod/efs-app                                    0/2 nodes are available: 2 persistentvolumeclaim "efs-claim" is being deleted.
7m24s       Warning   FailedScheduling         pod/efs-app                                    skip schedule deleting pod: default/efs-app
17s         Warning   FailedScheduling         pod/efs-app                                    0/2 nodes are available: 2 pod has unbound immediate PersistentVolumeClaims.
27m         Normal    ExternalProvisioning     persistentvolumeclaim/efs-claim                waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
10m         Normal    ExternalProvisioning     persistentvolumeclaim/efs-claim                waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
11m         Normal    Provisioning             persistentvolumeclaim/efs-claim                External provisioner is provisioning volume for claim "default/efs-claim"
11m         Warning   ProvisioningFailed       persistentvolumeclaim/efs-claim                failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
7m47s       Normal    Provisioning             persistentvolumeclaim/efs-claim                External provisioner is provisioning volume for claim "default/efs-claim"
7m47s       Warning   ProvisioningFailed       persistentvolumeclaim/efs-claim                failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied
74s         Normal    ExternalProvisioning     persistentvolumeclaim/efs-claim                waiting for a volume to be created, either by external provisioner "efs.csi.aws.com" or manually created by system administrator
2m56s       Normal    Provisioning             persistentvolumeclaim/efs-claim                External provisioner is provisioning volume for claim "default/efs-claim"
2m56s       Warning   ProvisioningFailed       persistentvolumeclaim/efs-claim                failed to provision volume with StorageClass "efs-sc": rpc error: code = Unauthenticated desc = Access Denied. Please ensure you have the right AWS permissions: Access denied

Service account was created by

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: efs-csi-controller-sa
  namespace: kube-system
  labels:
    app.kubernetes.io/name: aws-efs-csi-driver
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111111111111:role/AmazonEKS_EFS_CSI_Driver_Policy

The AmazonEKS_EFS_CSI_Driver_Policy is the json from https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/v1.3.2/docs/iam-policy-example.json