WaitForFirstConsumer PersistentVolumeClaim waiting for first consumer to be created before binding

The app is waiting for the Pod, while the Pod is waiting for a PersistentVolume by a PersistentVolumeClaim. However, the PersistentVolume should be prepared by the user before using.

My previous YAMLs are lack of a PersistentVolume like this:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: postgres-data
  labels:
    type: local
spec:
  storageClassName: local-storage
  capacity:
    storage: 1Gi
  local:
    path: /data/postgres
  persistentVolumeReclaimPolicy: Retain
  accessModes:
    - ReadWriteOnce
  storageClassName: local-storage
  nodeAffinity:
    required:
      nodeSelectorTerms:
        - matchExpressions:
          - key: app
            operator: In
            values:
              - postgres

The local path /data/postgres should be prepared before using. Kubernetes will not create it automatically.


I just ran into this myself and was completely thrown for a loop until I realized that the StorageClass's VolumeBindingMode was set to WaitForFirstConsumer vice my intended value of Immediate. This value is immutable so you will have to:

  1. Get the storage class yaml:

    kubectl get storageclasses.storage.k8s.io gp2 -o yaml > gp2.yaml
    

    or you can also just copy the example from the docs here (make sure the metadata names match). Here is what I have configured:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: gp2
    provisioner: kubernetes.io/aws-ebs
    parameters:
      type: gp2
    reclaimPolicy: Delete
    allowVolumeExpansion: true
    mountOptions:
      - debug
    volumeBindingMode: Immediate
    
  2. And delete the old StorageClass before recreating it with the new volumeBindingMode set to Immediate.

Note: The EKS clsuter may need perms to create cloud resources like EBS or EFS. Assuming EBS you should be good with arn:aws:iam::aws:policy/AmazonEKSClusterPolicy.

After doing this you should have no problem creating and using dynamically provisioned PVs.