Loading...
Master Docker and Docker Compose for containerized data engineering workflows. This essential guide covers Docker Desktop installation across all platforms, fundamental Docker commands for managing containers and images, and Docker Compose for orchestrating multi-container applications - crucial skills for running Kafka, databases, and other data services.
Docker Desktop is available for Ubuntu and Debian-based distros:
Download Docker Desktop for Linux from the official website.
Install the package using your distro's package manager:
sudo apt-get update
sudo apt-get install ./docker-desktop-<version>-<arch>.deb
Start Docker Desktop from your applications menu.
These commands assume you have a docker-compose.yml file in your current directory. Make sure your file is correctly set up for your Python project before running these commands.
Start your services:
docker-compose up
Start services in detached mode:
docker-compose up -d
Stop services:
docker-compose down
View logs of all services:
docker-compose logs
View logs of a specific service:
docker-compose logs service_name
Rebuild services:
docker-compose build
List running services:
docker-compose ps
Run a command in a service:
docker-compose run service_name command
For example, to run a Python script:
docker-compose run web python script.py
Scale a service:
docker-compose up -d --scale service_name=3
Remember to replace service_name with your actual service name as defined in your docker-compose.yml file.
Make sure to replace my-python-app and my-container with your actual image and container names.
Build a Docker image:
docker build -t my-python-app .
Run a Docker container:
docker run -it --name my-container my-python-app
Stop a running container:
docker stop my-container
Remove a container:
docker rm my-container
List running containers:
docker ps
List all containers (including stopped ones):
docker ps -a
View container logs:
docker logs my-container