Docker Tips and Tricks - NETSEC

Latest

Learning, Sharing, Creating

Cybersecurity Memo

Monday, April 20, 2020

Docker Tips and Tricks

This post summarizes some common docker commands, operations, and tasks.
Some related posts:
Docker logs commands
  • docker logs --since 1h nginx
  • docker logs --tail 50 nginx


2023/12/17 03:15:56 [notice] 24#24: exiting
2023/12/17 03:15:56 [notice] 24#24: exit
2023/12/17 03:15:56 [notice] 1#1: signal 17 (SIGCHLD) received from 26
2023/12/17 03:15:56 [notice] 1#1: worker process 25 exited with code 0
2023/12/17 03:15:56 [notice] 1#1: worker process 26 exited with code 0
2023/12/17 03:15:56 [notice] 1#1: signal 29 (SIGIO) received
2023/12/17 03:15:56 [notice] 1#1: signal 17 (SIGCHLD) received from 24
2023/12/17 03:15:56 [notice] 1#1: worker process 24 exited with code 0
2023/12/17 03:15:56 [notice] 1#1: exit
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: IPv6 listen already enabled
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/12/17 03:15:58 [emerg] 1#1: host not found in upstream "portainer" in /etc/nginx/conf.d/portainer.conf:6
nginx: [emerg] host not found in upstream "portainer" in /etc/nginx/conf.d/portainer.conf:6
Cause: portainer docker was upgraded but there was manually change joined into a Docker network which portainer docker is in. Upgrade portainer docker did not join that network automatically which cause host can not be resolved. 



Docker Failed To Start Troubleshooting


Copy file out from the inside of container:
  • docker cp docker_nginx:/etc/nginx/config.d/portainer.conf .

Copy file back into inside of container:
  • docker cp portainer.conf docker_nginx:/etc/nginx/config.d/portainer.conf .


Are You Inside Docker Environment

While creating a Docker container, a file “.dockerenv” is created at the root of the container. So, searching for this file could help us to know if we are inside the docker container or not.

On the Non-Container machine −

$ ls -la | grep ./dockerenv

This gives no output, hence the file is missing.

Now try on the Container machine −

/ # ls -la | grep .dockerenv

Output

-rwxr-xr-x 1 root root 0 Dec 18 14:00 .dockerenv

Hence this proves we are on the container.

Here is an example from https://ide.goorm.io/ site:
root@goorm:/# ls -la
total 84
drwxr-xr-x  81 root root 4096 Jul 28 14:31 .
drwxr-xr-x  81 root root 4096 Jul 28 14:31 ..
-rwxr-xr-x   1 root root    0 Jul 28 14:23 .dockerenv
drwxr-xr-x   2 root root 4096 Jul 28 14:31 bin
drwxr-xr-x   2 root root 4096 Apr 24  2018 boot
drwxr-xr-x   5 root root  360 Jul 28 14:23 dev
drwxr-xr-x 149 root root 4096 Jul 28 14:31 etc
drwxr-xr-x   5 root root 4096 Feb 24  2020 goorm
drwxr-xr-x   6 root root 4096 Jul 28 14:10 goormService
drwxr-xr-x   2 root root 4096 Apr 24  2018 home
drwxr-xr-x  23 root root 4096 Jul 28 14:30 lib
drwxr-xr-x   2 root root 4096 Jul 18  2019 lib64
drwxr-xr-x   2 root root 4096 Jul 18  2019 media
drwxr-xr-x   2 root root 4096 Jul 18  2019 mnt
drwxr-xr-x   2 root root 4096 Jul 18  2019 opt
dr-xr-xr-x 496 root root    0 Jul 28 14:23 proc
drwx------  11 root root 4096 Jul 28 14:11 root
drwxr-xr-x  12 root root 4096 Jul 28 14:31 run
drwxr-xr-x   2 root root 4096 Jul 28 14:30 sbin
drwxr-xr-x   2 root root 4096 Jul 18  2019 srv
dr-xr-xr-x  13 root root    0 Jul 28 14:32 sys
drwxrwxrwt 833 root root 4096 Jul 28 14:31 tmp
drwxr-xr-x  42 root root 4096 Jul 28 14:30 usr
drwxr-xr-x  33 root root 4096 Jul 28 14:29 var
drwxr-xr-x   3 root root 4096 Jul 28 14:10 workspace
root@goorm:/#



Docker & Docker Compose  Installation

Install Docker
#Ubuntu
One line of command to install docker:
root@Ubuntu18:/# curl -sSL https://get.docker.com/ | sh 
systemctl start docker 
systemctl enable docker
For Ubuntu 20.04, it will be even eaiser to install:

#Ubuntu 20.04
sudo apt install docker.io

#CentOS 6
rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm 
yum update -y 
yum -y install docker-io 
service docker start chkconfig docker on 

#CentOS 7, Debian, Ubuntu 
curl -sSL https://get.docker.com/ | sh 
systemctl start docker 
systemctl enable docker
 
Or , references of my other posts:


Install Docker Compose
curl -L "https://get.daocloud.io/docker/compose/releases/download/1.28.6/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

note: Check the latest version of docker-compose from https://docs.docker.com/compose/release-notes/



#Ubuntu 20.04
sudo apt install docker-compose

Or you can play docker at  Play with Docker online.

Common Docker Commands

  • docker –version
  • docker pull <image name>
  • docker run -it -d <image name>
  • docker ps
  • docker ps -a   //show all the running and exited containers
  • docker exec -it <container name> /bin/bash  
    • or docker exec -it <container name> sh
  • docker stop
  • docker kill    //kills the container by stopping its execution immediately
  • docker commit    //creates a new image of an edited container on the local system
  • docker login     //login to the docker hub repository
  • docker push <username/image name>    //push an image to the docker hub repository
  • docker images    //lists all the locally stored docker images
  • docker rm <container id>
  • docker rmi <image-id>
  • docker build <path to docker file>     //build an image from a specified docker file
  • docker log <container-id>     // show logs about your container, for troubleshooting
[root@shadowdaemon compose]# docker logs --tail 50 --follow --timestamps compose_db_1
2020-03-17T15:24:42.758002000Z chown: changing ownership of ‘/var/lib/postgresql/data’: Permission denied
2020-03-17T15:24:43.410251000Z chown: changing ownership of ‘/var/lib/postgresql/data’: Permission denied

Backup image and load the backup


root@Ubuntu18-Docker:~# docker images
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
wordpress             latest              0d205d4886fe        2 weeks ago         540MB
nginx                 latest              ed21b7a8aee9        2 weeks ago         127MB
mysql                 5.7                 413be204e9c3        2 weeks ago         456MB
mariadb               latest              37f5f0a258bf        4 weeks ago         356MB
portainer/portainer   latest              2869fc110bf7        4 weeks ago         78.6MB
root@Ubuntu18-Docker:~# docker save -o /root/nginx.tar nginx
root@Ubuntu18-Docker:~# ls
nginx.tar  snap
root@Ubuntu18-Docker:~#


You can load this tar file into other machine's image list:
root@controller:~# docker load -i /root/nginx.tar

Export  Container / Import to Image 

docker export / docker import and docker save / docker load serve different purposes.
docker export (and import) are commands to export/import a container's root filesystem; from the command's "help" output;
Export a container's filesystem as a tar archive
When running these commands, you'll only get an archive of the container's filesystem, but not the image's configuration (such as the CMDENTRYPOINT and ENV).
The docker save / docker load commands on the other hand, allow you to save/load an image, including their configuration. From the command description;Using those commands, you can transfer an image between docker hosts (without using a registry), and preserve the layers and image configuration.

root@Ubuntu18-Docker:~# docker container list
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                         NAMES
7f2118c12a0f        nginx:latest                 "nginx -g 'daemon of…"   7 days ago          Up 7 days           0.0.0.0:80->80/tcp, 443/tcp   Nginx1
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days           0.0.0.0:10000->80/tcp         51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days           3306/tcp, 33060/tcp           51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days           0.0.0.0:9000->9000/tcp        portainer
root@Ubuntu18-Docker:~# docker stop Nginx1
Nginx1
root@Ubuntu18-Docker:~# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                      PORTS                    NAMES
7f2118c12a0f        nginx:latest                 "nginx -g 'daemon of…"   7 days ago          Exited (0) 14 seconds ago                            Nginx1
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days                   0.0.0.0:10000->80/tcp    51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days                   3306/tcp, 33060/tcp      51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days                   0.0.0.0:9000->9000/tcp   portainer
root@Ubuntu18-Docker:~# docker export Nginx1 > /root/Container-Nginx1.tar
root@Ubuntu18-Docker:~# docker rm Nginx1
Nginx1
root@Ubuntu18-Docker:~# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                    NAMES
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days           0.0.0.0:10000->80/tcp    51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days           3306/tcp, 33060/tcp      51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days           0.0.0.0:9000->9000/tcp   portainer
root@Ubuntu18-Docker:~# docker import /root/Container-Nginx1.tar Nginx1
invalid reference format: repository name must be lowercase
root@Ubuntu18-Docker:~# docker import /root/Container-Nginx1.tar nginx1
sha256:df44fea67db399580e7cbdd5d09bd882c91bfe96d43c4b0c0f639aa7f74c9e20
root@Ubuntu18-Docker:~# docker ps -a
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                    NAMES
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days           0.0.0.0:10000->80/tcp    51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days           3306/tcp, 33060/tcp      51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days           0.0.0.0:9000->9000/tcp   portainer

root@Ubuntu18-Docker:~# docker images
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
nginx1                latest              df44fea67db3        7 minutes ago       143MB
wordpress             latest              0d205d4886fe        2 weeks ago         540MB
nginx                 latest              ed21b7a8aee9        2 weeks ago         127MB
mysql                 5.7                 413be204e9c3        2 weeks ago         456MB
mariadb               latest              37f5f0a258bf        4 weeks ago         356MB
portainer/portainer   latest              2869fc110bf7        4 weeks ago         78.6MB

root@Ubuntu18-Docker:~# docker run -d --name Nginx1 --restart=always -p 80:80 nginx1
docker: Error response from daemon: No command specified.
See 'docker run --help'.
Docker export command doesnot export all information such history information. It will give you all files and folder structure in the running docker. Use Docker Inspect command to check CMD and entrypoint. Put Entrypoint such as /bin/bash into a Dockerfile

From:nginx1
CMD: /bin/bash

You will be able to bring it up, but the services might still run properly.


Create Own Image Using Your Container


root@Ubuntu18-Docker:~# docker container ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                    NAMES
0457b6159da9        nginx:latest                 "nginx -g 'daemon of…"   16 minutes ago      Up 5 minutes        0.0.0.0:80->80/tcp       nginx1
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days           0.0.0.0:10000->80/tcp    51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days           3306/tcp, 33060/tcp      51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days           0.0.0.0:9000->9000/tcp   portainer
root@Ubuntu18-Docker:~# docker commit nginx1 nginx1netsec
sha256:0cf3a7c347f9bca870bd97b9e40bfc11e959e8220e4529d49e4f452cd5de8e68
root@Ubuntu18-Docker:~# docker image list
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
nginx1netsec          latest              0cf3a7c347f9        6 seconds ago       145MB
nginx                 latest              e791337790a6        3 days ago          127MB
wordpress             latest              0d205d4886fe        2 weeks ago         540MB
nginx                 <none>              ed21b7a8aee9        2 weeks ago         127MB
mysql                 5.7                 413be204e9c3        2 weeks ago         456MB
mariadb               latest              37f5f0a258bf        4 weeks ago         356MB
portainer/portainer   latest              2869fc110bf7        4 weeks ago         78.6MB
root@Ubuntu18-Docker:~# docker stop nginx1
nginx1

root@Ubuntu18-Docker:~# docker run --name nginx2 --restart=always -p 80:80 -d nginx1netsec
5fbe841d1f407db372ef8a69fe5295900b3b5b8eeea7d6d7be45f7eed247a19c
root@Ubuntu18-Docker:~#
root@Ubuntu18-Docker:~# docker container ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS              PORTS                    NAMES
5fbe841d1f40        nginx1netsec                 "nginx -g 'daemon of…"   2 minutes ago       Up 2 minutes        0.0.0.0:80->80/tcp       nginx2
58984786a347        wordpress:latest             "docker-entrypoint.s…"   7 days ago          Up 7 days           0.0.0.0:10000->80/tcp    51sec_wordpress_1
986469bf37d1        mysql:5.7                    "docker-entrypoint.s…"   7 days ago          Up 7 days           3306/tcp, 33060/tcp      51sec_db_1
e1965b3d6e1f        portainer/portainer:latest   "/portainer"             7 days ago          Up 7 days           0.0.0.0:9000->9000/tcp   portainer
root@Ubuntu18-Docker:~#

Notice that certain directories are considered volume directories by docker, meaning that they are container specific and therefore never saved in the image. The \data directory is such an example. When docker commit my_container my_image:my_tag is executed, all of the containers filesystem is saved, except for /data. To work around it, you could do:

mkdir /data0
cp /data/* /data0

Then, outside the container:

docker commit my_container my_image:my_tag

Then you would perhaps want to copy the data on /data0 back to /data, in which case you could make a new image:

On the Dockerfile:

FROM my_image:my_tag
CMD "cp /data0 /data && my_other_CMD"

Notice that trying to copy content to /data in a RUN command will not work, since a new container is created in every layer and, in each of them, the contents of /data are discarded. After the container has been instatiated, you could also do:

docker exec -d my_container /bin/bash -c "cp /data0/* /data"


=========================================================================

Here is an example how to download a image from DockerHub and how to push it to your own repository:
docker pull devisty/xssh:v2
docker image list

Create a new repository xssh from your Dockerhub's johnyan2 account.


docker tag devisty/xssh:v2 johnyan2/xssh:latest
docker push


Publish Your Own Docker Image to Docker Hub



root@Ubuntu18-Docker:~# docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: johnyan2
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

root@Ubuntu18-Docker:~# docker tag nginx1netsec johnyan2/nginx1netsec:latest
root@Ubuntu18-Docker:~# docker push johnyan2/nginx1netsec:latest
The push refers to repository [docker.io/johnyan2/nginx1netsec]
a0ad3b8aa236: Pushed
be91fceb796e: Pushed
919b6770519b: Pushed
b60e5c3bcef2: Pushed
latest: digest: sha256:2ccc1aeb4d69052c9afb6f36a5881bc6b4faf43bc86e33d6922f33382b5bbc28 size: 1160





Pull and run your own Docker:

$ docker pull johnyan2/nginx1netsec
Using default tag: latest
latest: Pulling from johnyan2/nginx1netsec
123275d6e508: Already exists 
6cd6a943ce27: Already exists 
a50b5ac4a7fb: Already exists 
75facb91406e: Pull complete 
Digest: sha256:2ccc1aeb4d69052c9afb6f36a5881bc6b4faf43bc86e33d6922f33382b5bbc28
Status: Downloaded newer image for johnyan2/nginx1netsec:latest
docker.io/johnyan2/nginx1netsec:latest
[node1] (local) [email protected] ~
$ docker image list
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
johnyan2/nginx1netsec   latest              0cf3a7c347f9        11 hours ago        145MB
nginx                   latest              e791337790a6        4 days ago          127MB
[node1] (local) [email protected] ~
$ docker run  -p 80:80 --name Nginx1 -d johnyan2/nginx1netsec
a4e00ef3a26aede705f6519d34baeab2045b31153a4ad2b1a75bb1ec928d27f5
[node1] (local) [email protected] ~
$ netstat -lantp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.11:35667        0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      36/sshd
tcp        0      0 :::22                   :::*                    LISTEN      36/sshd
tcp        0      0 :::2375                 :::*                    LISTEN      20/dockerd
tcp        0      0 :::80                   :::*                    LISTEN      2283/docker-proxy
tcp        0      0 ::ffff:172.18.0.24:2375 ::ffff:172.18.0.1:39999 ESTABLISHED 20/dockerd
tcp        0      0 ::ffff:172.18.0.24:2375 ::ffff:172.18.0.1:39823 ESTABLISHED 20/dockerd
[node1] (local) [email protected] ~


You also can download an image to local then push it to your own repository:


Use Dockerfile to create your own image

Apache with a Dockerfile

FROM php:7.2-apache
COPY src/ /var/www/html/

Where src/ is the directory containing all your PHP code. Then, run the commands to build and run the Docker image:

docker pull php:7.2-apache

docker build -t speedtest .

docker run -dit --name st -p 80:80 speedtest

Run Scheduled Task in Docker

For example , use Certbo command to renew HTTPS/SSL certificate in Docker:
Run it from host( certbotdocker is container 's name):
Put it into crontab -e :

Clean Docker Images and Volumes

1  Delete All Images and Vulumes (even it is running)
Save follow code to docker-cleanup.sh:
#!/bin/bash
docker kill $(docker ps -aq)
docker rm $(docker ps -aq)
docker rmi $(docker images -q)
docker volume rm $(docker volume ls -q)
Run .sh file:
bash docker-cleanup.sh


2  Only Delete non-running volumes and images
#!/bin/bash
docker rm $(docker ps -aq)
docker rmi $(docker images -q)
docker volume rm $(docker volume ls -q)
You will get an error message for those running dockers, but will not be deleted.


Write Keyboard Inputs into a File from Command Line

Using cat command to directly write keyboard inputs into a file. 
root@8a14b324cde5:/etc/nginx/conf.d# cat > portainer.conf <<EOF
> server {
>     listen       80;
>     server_name  awsportainer.51sec.org;
> 
> location / {
>     proxy_pass       http://aws.51sec.org:9000;
>     proxy_redirect             off;
>     proxy_http_version         1.1;
>     proxy_set_header Upgrade   $http_upgrade;
>     proxy_set_header Connection "upgrade";
>     proxy_set_header Host      $host;
>     proxy_set_header X-Real-IP $remote_addr;
>     proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
>     }
> }
> EOF

In this way, you will not need to install nano or vi into your docker. 

Note: $ will be omitted from the file. If you will need to enter $, you might still need install nano or vi. 


Or CTRL+D to end input and save to file. 

  • Make a text file on Linux: $ cat > filename.txt.
  • Add data and press CTRL + D to save the filename.txt when using cat on Linux.

Docker Networking


1)Create a new network

  1.  docker network create <network-name>

2)Connect container into network

  1. docker run --net=<network-name> ...

or join existing docker into network

  1. docker network connect <network-name> <container-name>

 

3)ping docker with its name

  1. docker exec -ti <container-name-A> ping <container-name-B>

Ps:需要注意的是,如果容器没有指定名称(--name),那么就只能用id。
 
64 bytes from c1 (172.18.0.4): icmp_seq=1 ttl=64 time=0.137 ms
64 bytes from c1 (172.18.0.4): icmp_seq=2 ttl=64 time=0.073 ms
64 bytes from c1 (172.18.0.4): icmp_seq=3 ttl=64 time=0.074 ms
64 bytes from c1 (172.18.0.4): icmp_seq=4 ttl=64 time=0.074 ms
请参阅文档的此部分;

此功能当前不支持别名

4)Disconnect from default bridge network

由于容器仍然连接着默认bridge docker0,而现在我们已经不需要它,所以应该将容器与docker0的连接断开,执行以下操作:

  1. # 断开容器与docker0的连接
  2. docker network disconnect bridge name1
  3. docker network disconnect bridge name2

docker network Doc:https://docs.docker.com/network/


[root@centos-nextcloud-aria2 ~]# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
d2daf16b85d2        bridge              bridge              local
ced0ba273013        host                host                local
5bbc16718b13        none                null                local
[root@centos-nextcloud-aria2 ~]# docker network create mynetwork
118b26e6cb77e441be35f823379d8a56b59f28a4d3b5fe680088fae324a10f93
[root@centos-nextcloud-aria2 ~]# docker network connect mynetwork nginx
[root@centos-nextcloud-aria2 ~]# docker network connect mynetwork portainer
[root@centos-nextcloud-aria2 ~]# docker network inspect mynetwork
[
    {
        "Name": "mynetwork",
        "Id": "118b26e6cb77e441be35f823379d8a56b59f28a4d3b5fe680088fae324a10f93",
        "Created": "2021-04-20T16:04:26.972255453Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "2533d99553373027929bdddd7db0de20b8d1bce8fdfc3dca07be28996960b1b8": {
                "Name": "nginx",
                "EndpointID": "13caa8d46093ecd672827998c9a0637f95136b900f846327db50edffb312d20b",
                "MacAddress": "02:42:ac:12:00:02",
                "IPv4Address": "172.18.0.2/16",
                "IPv6Address": ""
            },
            "312eec9453fbf1856dfafe3320fcc7518d15fd2a678e4f60711c1ec2dd9bdb4c": {
                "Name": "portainer",
                "EndpointID": "e115dcf5fc70366ff1e10951fd36879be76979ed78c51f15fb9d25765218eaac",
                "MacAddress": "02:42:ac:12:00:03",
                "IPv4Address": "172.18.0.3/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]


You might need to install ping utility first using comand
  • apt update & apt install iputils-ping

[root@centos-nextcloud-aria2 ~]# docker exec -it nginx ping portainer
PING portainer (172.18.0.3) 56(84) bytes of data.
64 bytes from portainer.mynetwork (172.18.0.3): icmp_seq=1 ttl=64 time=0.082 ms
64 bytes from portainer.mynetwork (172.18.0.3): icmp_seq=2 ttl=64 time=0.083 ms
64 bytes from portainer.mynetwork (172.18.0.3): icmp_seq=3 ttl=64 time=0.050 ms
^C
--- portainer ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 3ms
rtt min/avg/max/mdev = 0.050/0.071/0.083/0.018 ms
[root@centos-nextcloud-aria2 ~]# docker network disconnect bridge nginx
[root@centos-nextcloud-aria2 ~]# docker network disconnect bridge portainer
[root@centos-nextcloud-aria2 ~]#



Troubleshooting


[node1] (local) [email protected] ~
$ ./minkebox.sh 172.18.0.22 /var/data
minke
docker: Error response from daemon: path /var/data is mounted on / but it is not a shared mount.

[node1] (local) [email protected] ~
$ mount --make-shared /
[node1] (local) [email protected] ~
$ ./minkebox.sh 172.18.0.22 /var/data
minke
[node1] (local) [email protected] ~


root@ub20-1-test:~# docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          2         1         834.5MB   15.21MB (1%)
Containers      1         1         1.784MB   0B (0%)
Local Volumes   7         0         112.1kB   112.1kB (100%)
Build Cache     0         0         0B        0B
root@ub20-1-test:~# docker images
REPOSITORY               TAG       IMAGE ID       CREATED        SIZE
alekslitvinenk/openvpn   latest    312262dfa054   13 days ago    15.2MB
qualys/qcs-sensor        latest    12e78646bce8   2 months ago   819MB
root@ub20-1-test:~# 




Docker Data Backup & Restore

Check Docker data size:
root@hpthin:~# docker ps -s
CONTAINER ID   IMAGE                           COMMAND                  CREATED         STATUS                  PORTS                                                                                                                                        NAMES             SIZE
974f72a5b8f4   luodaoyi/kms-server:latest      "/bin/sh -c 'vlmcsdm…"   15 months ago   Up 3 months             0.0.0.0:1688->1688/tcp, :::1688->1688/tcp                                                                                                    kms               0B (virtual 5.74MB)
754f4e9fbbc5   pihole/pihole:latest            "/s6-init"               16 months ago   Up 3 months (healthy)   0.0.0.0:53->53/udp, :::53->53/udp, 0.0.0.0:53->53/tcp, :::53->53/tcp, 0.0.0.0:80->80/tcp, 0.0.0.0:67->67/udp, :::80->80/tcp, :::67->67/udp   pihole            47.2MB (virtual 383MB)
e17a5f818091   openspeedtest/latest:latest     "/docker-entrypoint.…"   16 months ago   Up 3 months             0.0.0.0:3000-3001->3000-3001/tcp, :::3000-3001->3000-3001/tcp, 8080/tcp                                                                      OpenSpeedTest     2B (virtual 55.6MB)
ea61e3109076   portainer/portainer-ce:latest   "/portainer"             17 months ago   Up 3 months             8000/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp                                                                                          hp-tc-portainer   0B (virtual 210MB)
  • Size - The amount of data (on disk) that is used by each container (write)
  • Virtual size -The amount of disk space used for the read-only image of each container.
Note: ncdu program can be used to check folder size in linux. 


Since when a docker created, the required folders in container have been mounted to your local host folders, also the docker image always can be downloaded it again, we only need to backup those mounted local folders.

Usually to make backup process simplified is to create your own specific folder to store your Docker data:
mkdir -p data/docker_data

If you have not done this step before, you can inspect your docker to find out mount settings:
docker inspect <dockername>


root@hpthin:~# docker inspect hp-tc-portainer
[
    {
        "Id": "ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9",
        "Created": "2021-07-04T00:42:29.61723624Z",
        "Path": "/portainer",
        "Args": [],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 1853,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-08-20T12:24:01.245072415Z",
            "FinishedAt": "2022-08-20T12:23:46.134177221Z"
        },
        "Image": "sha256:8bd64518b97697ed2d0d00b5dfd46260f729cdb5ae8120b38e404a05ad08f61b",
        "ResolvConfPath": "/var/lib/docker/containers/ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9/hostname",
        "HostsPath": "/var/lib/docker/containers/ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9/hosts",
        "LogPath": "/var/lib/docker/containers/ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9/ea61e3109076472b9c7922965cdbc1cd2badc462e22922cb9e30bc6fea8935d9-json.log",
        "Name": "/hp-tc-portainer",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/var/run/docker.sock:/var/run/docker.sock",
                "portainer_data:/data"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "9000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "9000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "host",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/167e59329997dad5067d8fd9b9a453d94cb44e22037677fcb5dce21de56a3dee-init/diff:/var/lib/docker/overlay2/91c12c584759da4f38c6a90943da940d5203ea5083a44f1d47d3324964c270c3/diff:/var/lib/docker/overlay2/b58cc223289bfc2f831da6d3cc03a2cb394cc97067b5fae7bac1163fde9a6850/diff:/var/lib/docker/overlay2/36213f2fd54546fef2a7c751f07baba95fe6a76be88152593102f0b8d11be555/diff",
                "MergedDir": "/var/lib/docker/overlay2/167e59329997dad5067d8fd9b9a453d94cb44e22037677fcb5dce21de56a3dee/merged",
                "UpperDir": "/var/lib/docker/overlay2/167e59329997dad5067d8fd9b9a453d94cb44e22037677fcb5dce21de56a3dee/diff",
                "WorkDir": "/var/lib/docker/overlay2/167e59329997dad5067d8fd9b9a453d94cb44e22037677fcb5dce21de56a3dee/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "portainer_data",
                "Source": "/var/lib/docker/volumes/portainer_data/_data",
                "Destination": "/data",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/var/run/docker.sock",
                "Destination": "/var/run/docker.sock",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "ea61e3109076",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8000/tcp": {},
                "9000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": null,
            "Image": "portainer/portainer-ce:latest",
            "Volumes": {
                "/data": {}
            },
            "WorkingDir": "/",
            "Entrypoint": [
                "/portainer"
            ],
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "c7c1f390aeccac751d375d060e93f2b7f154fc5cd5df426df68dc1950c9cdc8c",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8000/tcp": null,
                "9000/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "9000"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "9000"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/c7c1f390aecc",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "4081d3dabcfa98d1d8642bc48a921bd7ec737535ad3391a89991f5bee081f784",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "6c771e8cb72dd4577a7f4f084b0be488bb02b1ac6b361dac88fabc39caa0ea79",
                    "EndpointID": "4081d3dabcfa98d1d8642bc48a921bd7ec737535ad3391a89991f5bee081f784",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]
root@hpthin:~#

Or use following command directly back the folder up if you know the path:

  • docker run --rm --volumes-from CONTAINER -v $(pwd):/backup busybox tar cvfz /backup/backup.tar CONTAINERPATH
Change two parameters : 
  • CONTAINER(docker name)
  • CONTAINERPATH(the path inside of docker)
For example:
  • docker run --rm --volumes-from bitwarden -v $(pwd):/backup busybox tar cvfz /backup/backup.tar /data

Using SCP transfer backed up file to remote server:


Other commands

upzip:
  • tar -zxvf /root/data/docker-data/bitwarden/backup.tar

rename:
  • mv data bw-data   #rename from data to bw-data


Free Docker Playground

1. PWD https://labs.play-with-docker.com/


$ apk add virt-what

$ sudo apk update
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
v3.16.2-203-g16a4499ea3 [https://dl-cdn.alpinelinux.org/alpine/v3.16/main]
v3.16.2-202-ge26245aea1 [https://dl-cdn.alpinelinux.org/alpine/v3.16/community]
OK: 17053 distinct packages available
[node1] (local) [email protected] ~
$ sudo apk add git
OK: 395 MiB in 156 packages
[node1] (local) [email protected] ~
$ git --version
git version 2.36.2
[node1] (local) [email protected] ~
$ 
Note: If you are using PWD (Play with docker), the copy shortcut key is ctrl+insert, paste is ctrl+shift+v or ctrl+insert.

2. https://www.koyeb.com/



Docker Log Issue



[root@centos7-docker-portainer 1f9dad0548c5c1b45b30bf6f61183149d9aff3f575f3ddb4a2dbe9e5b8dee1fc]# ls -l
total 32863468
-rw-r-----. 1 root root 33652035791 Dec 19 15:12 1f9dad0548c5c1b45b30bf6f61183149d9aff3f575f3ddb4a2dbe9e5b8dee1fc-json.log
drwx------. 2 root root           6 Mar 18  2021 checkpoints
-rw-------. 1 root root       10480 Apr 16  2022 config.v2.json
-rw-r--r--. 1 root root        6279 Apr 16  2022 hostconfig.json
-rw-r--r--. 1 root root          13 Apr 16  2022 hostname
-rw-r--r--. 1 root root         174 Apr 16  2022 hosts
drwx------. 3 root root          17 Mar 18  2021 mounts
-rw-r--r--. 1 root root          88 Apr 16  2022 resolv.conf
-rw-r--r--. 1 root root          71 Apr 16  2022 resolv.conf.hash

Note: One Docker log file used 32G disk space

Enable IPv6 and Limit Log File Size (Ubuntu)


Add customized self defined IPv6 address segment to enable container's IPv6 fucntion. And limit log file's size and numbers in case log file to fill all hard drive's space. 

cat > /etc/docker/daemon.json << EOF
{
    "log-driver": "json-file",
    "log-opts": {
        "max-size": "20m",
        "max-file": "3"
    },
    "ipv6": true,
    "fixed-cidr-v6": "fd00:dead:beef:c0::/80",
    "eixperimental":true,
    "ip6tables":true
}
EOF

Restart Docker service:

systemctl restart docker

On CentOS, you might want to remove ipv6 part:


cat > /etc/docker/daemon.json << EOF
{
    "log-driver": "json-file",
    "log-opts": {
        "max-size": "20m",
        "max-file": "3"
    }
}
EOF

If there is any errors or failed restart docker service, you can remove ipv6 part since it might not supported on your VPS.


References






No comments:

Post a Comment