Optional: false
> CALICO\_IPV4POOL\_CIDR: 192.168.0.0/16
> CALICO\_DISABLE\_FILE\_LOGGING: true
> FELIX\_DEFAULTENDPOINTTOHOSTACTION: ACCEPT
> FELIX\_IPV6SUPPORT: false
> FELIX\_LOGSEVERITYSCREEN: info
> FELIX\_HEALTHENABLED: true
> Mounts:
> /lib/modules from lib-modules (ro)
> /run/xtables.lock from xtables-lock (rw)
> /var/lib/calico from var-lib-calico (rw)
> /var/run/calico from var-run-calico (rw)
> /var/run/secrets/kubernetes.io/serviceaccount from calico-node-token-jcc85 (ro)
> Conditions:
> Type Status
> Initialized False
> Ready False
> ContainersReady False
> PodScheduled True
> Volumes:
> lib-modules:
> Type: HostPath (bare host directory volume)
> Path: /lib/modules
> HostPathType:
> var-run-calico:
> Type: HostPath (bare host directory volume)
> Path: /var/run/calico
> HostPathType:
> var-lib-calico:
> Type: HostPath (bare host directory volume)
> Path: /var/lib/calico
> HostPathType:
> xtables-lock:
> Type: HostPath (bare host directory volume)
> Path: /run/xtables.lock
> HostPathType: FileOrCreate
> cni-bin-dir:
> Type: HostPath (bare host directory volume)
> Path: /opt/cni/bin
> HostPathType:
> cni-net-dir:
> Type: HostPath (bare host directory volume)
> Path: /etc/cni/net.d
> HostPathType:
> host-local-net-dir:
> Type: HostPath (bare host directory volume)
> Path: /var/lib/cni/networks
> HostPathType:
> calico-node-token-jcc85:
> Type: Secret (a volume populated by a Secret)
> SecretName: calico-node-token-jcc85
> Optional: false
> QoS Class: Burstable
> Node-Selectors: [beta.kubernetes.io/os=linux](http://beta.kubernetes.io/os=linux)
> Tolerations: :NoSchedule
> :NoExecute
> CriticalAddonsOnly
> disk-pressure:NoSchedule
> memory-pressure:NoSchedule
> network-unavailable:NoSchedule
> not-ready:NoExecute
> unreachable:NoExecute
> unschedulable:NoSchedule
> Events:
> Type Reason Age From Message
>
> * * *
>
> Normal Scheduled 9m11s default-scheduler Successfully assigned kube-system/calico-node-rb8ht to cherokee
> Normal Pulled 9m8s kubelet, cherokee Container image “calico/cni:v3.6.1” already present on machine
> Normal Created 9m7s kubelet, cherokee Created container
> Normal Started 9m6s kubelet, cherokee Started container
I am new to k8s and couldn’t find any useful info from those outputs.
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [April 16, 2019, 1:42am UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/8 "2019-04-16T01:42:18Z")
Ahh it’s a single node cluster. Try removing the master node taint, `kubectl taint nodes --all node-role.kubernetes.io/master-`. That should allow pods to be scheduled on it and bring Calico up.
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [April 16, 2019, 2:36am UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/9 "2019-04-16T02:36:02Z")
Thanks a lot @macintoshprime… that definitely got me one step further, but now I get another strange warning in my mongodb pod:
> Name: mongo-5d89cc6f7f-t7cph
> Namespace: default
> Priority: 0
> PriorityClassName:
> Node:
> Labels: name=mongo
> pod-template-hash=5d89cc6f7f
> Annotations:
> Status: Pending
> IP:
> Controlled By: ReplicaSet/mongo-5d89cc6f7f
> Containers:
> mongo:
> Image: mongo
> Port: 27017/TCP
> Host Port: 0/TCP
> Environment:
> Mounts:
> /data/db from mongo-claim0 (rw)
> /var/run/secrets/kubernetes.io/serviceaccount from default-token-8bngw (ro)
> Conditions:
> Type Status
> PodScheduled False
> Volumes:
> mongo-claim0:
> Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
> ClaimName: mongo-claim0
> ReadOnly: false
> default-token-8bngw:
> Type: Secret (a volume populated by a Secret)
> SecretName: default-token-8bngw
> Optional: false
> QoS Class: BestEffort
> Node-Selectors:
> Tolerations: [node.kubernetes.io/not-ready:NoExecute](http://node.kubernetes.io/not-ready:NoExecute) for 300s
> [node.kubernetes.io/unreachable:NoExecute](http://node.kubernetes.io/unreachable:NoExecute) for 300s
> Events:
> Type Reason Age From Message
>
> * * *
>
> Warning FailedScheduling 3m59s (x2 over 3m59s) default-scheduler 0/1 nodes are available: 1 node(s) had taints that the pod didn’t tolerate.
> Warning FailedScheduling 29s (x7 over 3m38s) default-scheduler **pod has unbound immediate PersistentVolumeClaims**
I will now try to see if I find something wrong in the manifests generated by `kompose`.
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [April 16, 2019, 12:12pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/10 "2019-04-16T12:12:34Z")
So mongo is trying to provision some storage via the PVC and my guess is there is no Storage Class to handle the provisioning. Are you using the Helm Chart or did you build it on your own?
Here are a few of the docs you may find helpful
- [Configure a Pod to Use a PersistentVolume for Storage - Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/)
- [Storage Classes - Kubernetes](https://kubernetes.io/docs/concepts/storage/storage-classes/)
- [Persistent Volumes - Kubernetes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [April 16, 2019, 8:16pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/11 "2019-04-16T20:16:41Z")
Wow! There are a lot of k8s related concepts to digest. I’ve spent more than an hour reading your links to understand a bit.
I did not use Helm, just [`kompose convert`](https://github.com/kubernetes/kompose) in my previus docker-compose.yml:
```
version: "3.3"
services:
mongo:
image: mongo
restart: always
container_name: mongo
volumes:
- /data/mongodb:/data/db
ports:
- "30001:27017"
```
Then it just created a Deployment, Service and a PVC from it. The documentation didn’t tell me a needed to create a PersistentVolume as in [Configure a Pod to Use a PersistentVolume for Storage](https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/). There is also this [Claims As Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims), I am tryng to understand if it is a replacement for a PV, but I think this is what `kompose` did.
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
creationTimestamp: null
labels:
name: mongo-claim0
name: mongo-claim0
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Mi
status: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
name: mongo
name: mongo
spec:
type: NodePort
ports:
- port: 27017
nodePort: 30001
targetPort: 27017
selector:
name: mongo
status:
loadBalancer: {}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
creationTimestamp: null
labels:
name: mongo
name: mongo
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
creationTimestamp: null
labels:
name: mongo
spec:
containers:
- image: mongo
name: mongo
ports:
- containerPort: 27017
resources: {}
volumeMounts:
- mountPath: /data/db
name: mongo-claim0
restartPolicy: Always
volumes:
- name: mongo-claim0
persistentVolumeClaim:
claimName: mongo-claim0
status: {}
```
What is even MORE strange, I that I used `kompose convert` in an almost identical Elasticsearch service and it is `Running` in my single-node without problems.
Trying to figure out the magic in this PVC-PV bindings…
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [April 17, 2019, 12:56am UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/12 "2019-04-17T00:56:43Z")
Ya so looked at what you provided from the docker-compose.yaml and my guess would be that kompose tried to handle the following line by creating a PVC and mounting at `/data/db`
```auto
volumes: - /data/mongodb:/data/db
```
Persistent Volumes can be a little weird at first, think I did that tutorial a few times before I started getting the hang of them. The Docs from GKE have a nice explanation of what they are, [https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes](https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes) .
Saad Ali had a really good talk about Storage which you can find here, [https://www.youtube.com/watch?v=uSxlgK1bCuA](https://www.youtube.com/watch?v=uSxlgK1bCuA)
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [April 23, 2019, 2:18pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/13 "2019-04-23T14:18:05Z")
Very clarifying video and docs @macintoshprime, thanks again. Now I see there is no StorageClass and PersistentVolumes in my single-node, so the PVCs are in Pending state.
I saw hostPath is not recommended if I am going for a cloud provider latter (AWS, Google) because it is not “Workload portability friendly”. Should I use local PersistentVolume or there is an out-of-tree recommended plugin for storage in this case?
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [April 23, 2019, 2:24pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/14 "2019-04-23T14:24:00Z")
Happy to hear!
If you’re already on AWS you can take advantage of their S3 storage, [Storage Classes - Kubernetes](https://kubernetes.io/docs/concepts/storage/storage-classes/#aws-ebs) .
If you’re looking for other solutions here are a few that are pretty awesome.
[https://rook.io/](https://rook.io/)
[https://storageos.com/](https://storageos.com/)
> **[Persistent storage for containers](https://portworx.com/)**
>
> Portworx is persistent storage for containers. Get data services for stateful containers including HA, snapshots, backup & encryption. Try today!
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [May 14, 2019, 8:55am UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/15 "2019-05-14T08:55:39Z")
@macintoshprime there is no way my StorageClasses work on a single-node kubeadm. I tried using AWS-EBS storageclass, but it doesn’t work locally without the actual use of an account. I also tried 3 different approaches for StorageClass:
```auto
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: docker.io/hostpath
reclaimPolicy: Retain
```
After creating the StorageClass I try creating 2 PVCs, but they do not get binded and no PV is created. I also tried the class used in Minikube (which works in my development environment):
```auto
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
namespace: kube-system
name: standard
annotations:
storageclass.kubernetes.io/is-default-class: "true"
labels:
addonmanager.kubernetes.io/mode: EnsureExists
provisioner: k8s.io/minikube-hostpath
```
Same result… and at last I tried a class sugested in Kubernetes Slack:
```auto
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
annotations:
storageclass.beta.kubernetes.io/is-default-class: "true"
labels:
addonmanager.kubernetes.io/mode: EnsureExists
name: standard
provisioner: kubernetes.io/host-path
reclaimPolicy: Delete
volumeBindingMode: Immediate
```
Again, no PVs, no bind. I though it could be some taint related issue, but I’ve already removed all the taint from master using `kubectl taint nodes --all node-role.kubernetes.io/master-`. Any idea?
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [May 14, 2019, 11:52am UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/16 "2019-05-14T11:52:17Z")
If you look at the state of the PVC’s that are not bind what error’s do the give `kubectl describe pvc ` ?
Have you tried either [Portworx](https://portworx.com/) or [StorageOS](https://storageos.com/) both of them have nice Helm installers for a quick install.
---
### Author:  [@mrbobbytables](https://discuss.kubernetes.io/u/mrbobbytables)
#### Post date: [May 14, 2019, 12:23pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/17 "2019-05-14T12:23:09Z")
Just as a general FYI - the storage classes you tried:
- `provisioner: docker.io/hostpath` - Does not exist in vanilla deployments. It is only found in docker managed instances.
- `provisioner: k8s.io/minikube-hostpath` is only available with minikube.
- `kubernetes.io/host-path` - IIRC it has been superceded by [local volumes](https://kubernetes.io/docs/concepts/storage/storage-classes/#local).
If you need something that is just tied to a single host, the local volume route might not be a bad option. It will prevent migration etc, but should be okay for a single node instance.
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [May 15, 2019, 8:20pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/18 "2019-05-15T20:20:12Z")
It seems to be a problem with all these provisioners at @mrbobbytables said. Is this a Kubernetes class I need to deploy? Is there a way to get minikube’s provisioner for example? I also found out there is need to enable the `kubernetes.io/host-path` which is disabled by default. I did not find any announcement of documentation about it being superceded by local volumes…
I got this:
````auto
Name: esdata
Namespace: default
StorageClass: standard
Status: Pending
Volume:
Labels: name=esdata
Annotations: volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/host-path
Finalizers: [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode: Filesystem
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning ProvisioningFailed 36s (x6 over 2m5s) persistentvolume-controller Failed to create provisioner: Provisioning in volume plugin "kubernetes.io/host-path" is disabled
Mounted By: elasticsearch-deployment-56fbc8698-6p4jg```
````
---
### Author:  [@macintoshprime](https://discuss.kubernetes.io/u/macintoshprime)
#### Post date: [May 15, 2019, 8:39pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/19 "2019-05-15T20:39:01Z")
The local volumes option is a good one if your just testing things out but if you need auto-provisioning then using something else would probably be best.
Again storageOS, portworx are good options as they have helm charts you can use to quickly spin things up. OpenEBS just got inducted into the CNCF as an incubating project, it’s worth looking at as well [https://openebs.io/](https://openebs.io/)
---
### Author:  [@static](https://discuss.kubernetes.io/u/static)
#### Post date: [May 15, 2019, 8:55pm UTC](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306/20 "2019-05-15T20:55:11Z")
I will take a look at them too! Thanks again =)
[Next page](https://discuss.kubernetes.io/t/minimal-restricted-kubernetes-cluster-on-ubuntu-in-production/5306.md?page=2)