Communication Between Containers

General Discussions
1

Hi all, I am working on a project using technologies I’ve never worked with before and I was wondering if you guys could help me out.

I am currently working on a project to monitor network traffic using Zeek. The end product is a dashboard which will show alerts based off of Zeek notifications. The requirements state that this needs to be done in Kubernetes, so I will be using Minikube to host the cluster.

Within this cluster, I have a singular pod which should contain multiple containers. Each container should contain one tool. Starting with the first container, we need Zeek which will monitor the network traffic and produce logs. In another container, we will have MariaDB; a database where we will store the log data from Zeek. In another container, we will need to hold the dashboard files (.dart files). For the dashboard, we will be using Flutter.

That should be 3 separate containers within our singular pod. If we wanted to get log data from Zeek to MariaDB and the container for our dashboard, what would be the best way to go about this?

So far, I have searched and found potential solutions for communication between containers…

  • Using localhost:ports
  • Using a shared volume (persistent volume)

Any links/resources I can see for example would be helpful please.

Cluster information:

Kubernetes version:
Host OS: Linux, Ubuntu 22.04

yaml file:

apiVersion: v1
kind: Pod
metadata:
name: zeek-mariadb-pod
spec:
volumes: # Testing possible shared volume option

  • name: shared-data
    persistentVolumeClaim:
    claimName: shared-data-pvc

containers:

  • name: zeek-container
    image: testzeek # Custom zeek image
    imagePullPolicy: IfNotPresent
    volumeMounts:

    • name: shared-data
      mountPath: /zeek-logs

  • name: mariadb-container
    image: mariadb
    env:

    • name: ROOT_PASSWORD
      value: mypassword
    • name: DATABASE
      value: /zeek_logs
      volumeMounts:
    • name: shared-data
      mountPath: /zeek-logs
2

apiVersion: v1
kind: Pod
metadata:
name: zeek-mariadb-pod
spec:
containers:

  • name: zeek-container
    image: testzeek # Custom zeek image
    ports:
    • containerPort:
      env
    • name: MONGO_DB_HOSTNAME
      value: localhost
    • name: MONGO_DB_USERNAME
      value: dbadmin # Give same username as given in db
    • name: MONGO_DB_PASSWORD
      value: admin@123 # Give same password as given in db
  • name: mariadb-container
    image: mongo:4.2.13
    ports:
    • containerPort: 27017
      env:
    • name: MONGO_INITDB_ROOT_USERNAME
      value: dbadmin # Give username of your choice
    • name: MONGO_INITDB_ROOT_PASSWORD
      value: admin@12 # Give password of your choice
      volumeMounts:
    • name: shared-data
      mountPath: /zeek-data
      volumes:
  • name: shared-data
    hostpath:
    path: /zee-logs
    type: DirectoryOrCreate

Though I have not worked much on minikube. You can try with this yaml file. Give correct image and port number for zeek application.
Make sure your container should have root privilege to write to directory /zee-logs created in your machine/laptop.

3

Thank you for this. Using this file, I was able to successfully deploy a new pod with 2 ready containers. I was able to get into the zeek container with
minikube kubectl – exec -it zeek-mariadb-pod -c zeek-container – /bin/bash
and then deployed zeek using
zeekctl deploy

I was a bit confused about where the shared data is actually stored. Since I can’t get into the shared volume the same way I can exec into the containers, so how do I know where to find where the shared volume is mounted to or access the data in the shared volume?

I used minikube ssh -n minikube which brought us into docker@minikube
From there, I was able to find the zeek-logs directory, but it was empty. Is this where the shared volume is? Is this inside or outside of the pod we created?

4

@Jeff1 That’s great.
Ok now there is a directory is created in your Linux machine/laptop in name /zee-logs.
Within this directory there is directory mounted called /zeek-data. The directory /zeek-data as given in above yaml file is basically for your mongodb data base container not for the zeek container.
So if you use the below command
minikube kubectl – exec -it zeek-mariadb-pod -c mariadb-container – /bin/bash
cd /
ls -lsrt
you will find the directory /zee-data.
Let me explain
zee-logs dir created in linux machine is for your minikube
Where as /zee-data dir mounted with /zee-log dir and is created for container mariadb-conatiner not for zeek-container.
So basically whatever data comes to zeek app get stored directly in mariadb of /zee-data.
Hope this will help…

5

Thank you for the explanation.
After using this command; minikube kubectl – exec -it zeek-mariadb-pod -c mariadb-container – /bin/bash and checking for files inside the container, we were able to find /zeek-data, however it was empty.

While looking at the yaml file, we noticed there was no volume mount for the zeek container. We added one to the zeek container since we thought this was preventing it from actually moving data through and reconfigured zeek to save to the /zeek-data instead of the default path using <sed -i ‘s#LogDir = /usr/local/zeek/logs#LogDir = /zeek-data#’ /usr/local/zeek/etc/zeekctl.cfg>

After doing this, we tried going into the mariadb-container and tried to cd into /zeek-data where we were able to find the current directory which should house our log files, however when we try to cd into current, it says “bash: cd: current: No such file or directory.” Do you have any idea why this directory would show up, but we aren’t able to access it? Or how we could access it?

Here is the updated zeek container with our changes.
spec:
containers:
- name: zeek-container
image: testzeek # Custom zeek image
imagePullPolicy: IfNotPresent
ports:
- containerPort: 27017
env:
- name: MONGO_DB_HOSTNAME
value: localhost
- name: MONGO_DB_USERNAME
value: dbadmin # Give same username as given in db
- name: MONGO_DB_PASSWORD
value: admin@123 # Give same password as given in db
volumeMounts:
- name: shared-data
mountPath: /zeek-data

Thank you!

6

On my last line I have mentioned that
Whatever data comes to zeek application get stored directly in mariadb of /zee-data.
So there is no volume mount required for Zeek Application. The Zeek Application data directed to madiadb container volume mount and it stored there.
I think you need to check with application team how they want to store the data.
Also if you want to mount two different volumes to Zeek and mariadb then try to use two different directory, like /zeek-data1 and /zeek-data2 and check.
But in that case whatever data stored in Zeek mount volume /zeek-data1 should be replicated on mariadb mount volume /zeek-data2.