About a year ago I had just gotten into web development, spinning up a small VPS and hosting a simple LAMP stack for a small website.
Now that I have bigger aspirations and ideas, I wanted to get closer to enterprise techniques. Discovered Docker, loved it, thought it was the most amazing thing since sliced bread. Figured it was everything I needed and began developing my idea. It had several parts, including a MySQL image, a Redis image, an API image, static file server image, and an HAProxy image. Being able to spin the whole thing up with docker-compose and run it/test it locally was awesome.
Then I had the beautiful curse of discovering Kubernetes after only ever hearing offhanded mentions about it. After diving into it headfirst, I was hooked on the idea. Swapped from DO to GCS because of GKE.
Now I’m at the point where I’m realizing I dove into the deep end with way too much way too fast, I have no idea what I’m doing. I can see all the puzzle pieces, but I can’t put it together. I’m not looking for direct code-related help or debugging, I feel like I can break those sorts of problems down perfectly fine on my own. I’m looking for just general overall conceptual ‘help’, I guess is the best way to describe it. I’m overwhelmed, I’m at the point where Googling my problems away isn’t possible because I don’t even know what I’m supposed to be focusing on or doing right now.
I’ll start by describing my situation, what I’d like to be able to do, and methods I’ve found that I think might be related or helpful (but are most likely wrong or incomplete, please don’t get mad if I use a word or definition wrong) I’m going to mentioning a lot of loosely related things, so obviously feel free to answer any one part, any amount of help is appreciated.
When using Docker, I’d usually spin up my images, make changes to my code in VSCode, and visit the related webpages on localhost to see the changes reflected immediately. This is because docker-compose let me attach volumes of code into my images that were immediately “transferred” over to the image when saved in my editor.
As far as I can tell, this is much more complicated with Kubernetes since it works primarily off of pre-built images(?), with the code included in the dockerfile itself. This isn’t a problem for me to do in concept, I know how to put the code in the image itself, build the image, tag it, push it to my repo, and update my Kubernetes deployments to use the new image. This takes a crazy amount of time though, and being a very new developer, I will find myself making dozens of mistakes that require many updates and saves to get working right.
On this topic, so far I’ve heard of a couple methods:
- Push changes to git repo in Google cloud, use Cloud Build to build the image and push it to my image repo, then use some method to detect the image update and automatically deploy it to my GKE cluster(?). Seems slow, but knowing Google who knows. Also doesn’t work for local testing.
- Run a cluster locally, then just manually build each image and run
kubectl applyeach update. This is what I currently do, but it’s long and tedious and I feel like there’s a better way.
- I remember reading something about testing pods(?) locally by proxying(?) them to a cloud cluster, so that they can use resources you have running in the cloud, while you tinker with the pod that’s local. Don’t know if that’s relevant or even meant for what I’m going for.
- I guess I could attach shell into a pod and mess with the code in there, and transfer the code out of the pod when it works? Seems dangerous and dumb because of the ephemeral nature of pods.
- Skaffold: Haven’t tried it? Seems to be the answer I’m looking for. Is it fast enough for my use case of fixing minor bugs from my incompetence?
I’d like to maintain a Production/Next/Test environment in the easiest, most automated and maintainable way possible.
How would I even begin to set this up?
Multiple clusters? A single cluster with multiple namespaces?
Do I need separate docker images?
Separate Skaffold configurations?
Can I have Prod/Next on GCS and ‘test’ be local? Can I tie them to git branches?
Should I have each Docker image in a different git repo or is separate folders fine?
How can I make changes to parts of my code depending on which branch the codeis running in? Say, accessing an external database ‘Project1-next’ from a MySQL server instead of ‘Project1-prod’?
Do I store those db credentials in a configmap, a secret or hardcoded?
Can I access configmap/secret data in my php/python/.net/go/whatever pod?
I think this is the part I’m most overwhelmed with, and my questions can be summed up with ‘what is the methodology of setting up multiple environments in Kubernetes?’.
Needless to say any help is appreciated. And if this isn’t the best way to get help, pointing in the right direction is appreciated. I’ve tried to pay for Kubernetes help on Fiverr twice now with no replies, had to be refunded. I feel confident enough to figure out most issues, what I need most is direction and overarching concepts to work off of.