1.Podman容器管理工具基础学习

[TOC]

0x00 基础介绍

Q:什么是Podman? 官网描述: Podman是一个无守护进程的容器引擎,用于在Linux系统上开发、管理和运行OCI容器(开源的容器管理工具)。容器可以作为根运行，也可以以无根模式运行。简单地说:alias docker=podman简单的说它是下一代容器。

官网 : https://podman.io/ Github 项目: https://github.com/containers/podman

前生今世

介绍:Podman 原是 CRI-O 项目的一部分后来被分离成一个单独的项目叫 libpod, Podman 的使用体验和 Docker 类似不同的是 Podman 没有 daemon。

以前使用 Docker CLI 的时候它会通过 gRPC API 去跟 Docker Engine 说「我要启动一个容器」，然后 Docker Engine 才会通过 OCI Container runtime（默认是 runc）来启动一个容器;意味着容器的进程不可能是 Docker CLI 的子进程而是 Docker Engine 的子进程。
Podman 比较简单粗暴它不使用 Daemon，而是直接通过 OCI runtime（默认也是 runc）来启动容器所以容器的进程是 podman 的子进程(相当于是省去了中间商,减少了赚差价 (●ˇ∀ˇ●)),比较像 Linux 的 fork/exec 模型;

fork/exec 模型优势:

1.某个容器进程父进程是谁(即到底是谁启动的一目了然)
2.利用 cgroup 对 podman 限制则利用podman创建的容器都会被限制;
3.可将 podman命令放入systemd单元文件中容器进程可通过podman返回通知(SD_NOTIFY)表明服务已准备好接收任务

Q:Podman有何作用?

描述:podman是作为libpod库的一部分而提供的工具,它可以用来创建和维护容器类似于Docker但又不等同于它;

Podman有何特点:

1.没有 Daemon (守护进程) 直接通过 OCI runtime（默认也是runc）来启动容器，所以容器的进程是 podman 的子进程;
2.采用类似于Linux中"fork/exec模型"相比较于”C/S模型”有一定的优势。
3.能够以非 root 用户的身份去运行容器
4.引入注册表的概念其内部包括docker.io在内的多个容器镜像源,默认的有redhat docker fedora centos quay

Podman VS Docker

(1) 模型对比
- Podman: fork/exec 模型
- Docker: C/S 模型
(2) 启动模式：
- 前者直接OCI containner runtime(runc)进行交互来创建container的
- 后者通过API跟 Docker Engine(引擎)请求才会调用OCI container runtime(runc)来启动一个container
(3) 守护进程
- 前者容器不支持–restart策略但是可以通过编写systemd服务来完成自启动
- 后者因有docker daemon，所以docker启动的容器支持–restart策略
(4) 权限对比
- 前者可以非root用户创建容器
- 后者必须使用root用户来创建容器

Q: Podman 如何使用?

答: Podman 官网的 getting-started 文档永远是你最好的选择;

0x01 安装试用

描述:考虑到本文所写时间可能于读者查看时间有一定的间隔，所此处安装参考官网 installation,首先祭出我们Ubuntu作为一个Linux爱好者必备;

Ubuntu

描述:Kubic项目提供了Ubuntu 18.04、19.04、19.10和20.04的软件包, 在Ubuntu上已经有许多带有libpod前缀的包可用安装环境: Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-46-generic x86_64)

cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"

Ubuntu 下 Podman 安装:

# - Stable Install:
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/Release.key | sudo apt-key add -
sudo apt-get update
sudo apt-get -y upgrade 
sudo apt-get -y install podman

# - Development Install
# Kubic项目为Ubuntu 18.04、19.04、19.10和20.04提供RC/测试包。
. /etc/os-release
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:testing.list
curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/testing/xUbuntu_${VERSION_ID}/Release.key | sudo apt-key add -
sudo apt-get update -qq
sudo apt-get -qq -y install podman

# - Building from scratch
# 构建和运行依赖项运行 make packege-install，它将安装依赖项、构建源代码、为当前平台生成rpm并最终安装它们。
sudo apt-get install 
  btrfs-tools 
  git 
  golang-go 
  go-md2man 
  iptables 
  libassuan-dev 
  libbtrfs-dev 
  libc6-dev 
  libdevmapper-dev 
  libglib2.0-dev 
  libgpgme-dev 
  libgpg-error-dev 
  libprotobuf-dev 
  libprotobuf-c0-dev 
  libseccomp-dev 
  libselinux1-dev 
  libsystemd-dev 
  pkg-config 
  runc 
  uidmap

Podman 命令 - 容器管理工具

描述:了解Podman是如何工作的你可以使用这个帮助以及验证podman安装情况:

$ podman <subcommand> --help
$ man podman-<subcommand>
$ podman --version
podman version 2.0.6

Podman 运行信息:

 podman info
host:
  arch: amd64
  buildahVersion: 1.15.1
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.20, commit: '
  cpus: 2
  distribution:
    distribution: ubuntu
    version: "20.04"
  eventLogger: file
  hostname: ubuntu
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.4.0-46-generic
  linkmode: dynamic
  memFree: 756535296
  memTotal: 4127399936
  ociRuntime:
    name: runc
    package: 'containerd.io: /usr/bin/runc'
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc10
      commit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
      spec: 1.0.1-dev
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 452h 40m 5.75s (Approximately 18.83 days)
registries:
  docker.io:
    Blocked: false
    Insecure: false
    Location: docker.io
    MirrorByDigestOnly: false
    Mirrors:
    - Insecure: true
      Location: xlx9erfu.mirror.aliyuncs.com
    Prefix: docker.io
  search:
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 1
  Built: 0
  BuiltTime: Thu Jan  1 00:00:00 1970
  GitCommit: ""
  GoVersion: go1.14.2
  OsArch: linux/amd64
  Version: 2.0.6

0x02 小试牛刀

1.Podman Hello-world

运行环境说明:

ubuntu 20.04.1 LTS (Focal Fossa)
Linux ubuntu 5.4.0-46-generic 50-Ubuntu SMP Fri Aug 28 15:33:36 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

简单使用： 描述:建议在非根用户运行并在需要根升级的地方使用sudo,下面以Hello-Worlds示例展现podman的使用; 使用示例:

# 1.搜索，拉出和列出图像
$podman search hello-world
INDEX       NAME                                                 DESCRIPTION                                       STARS   OFFICIAL   AUTOMATED
docker.io   docker.io/library/hello-world                        Hello World! (an example of minimal Dockeriz...   1297    [OK]

$podman pull hello-world --log-level debug  # 此处我已经进行加速镜像配置所以是从xlx9erfu.mirror.aliyuncs.com拉取(注意看其拉取过程)
  # DEBU[0000] GET https://xlx9erfu.mirror.aliyuncs.com/v2/
  # DEBU[0000] Ping https://xlx9erfu.mirror.aliyuncs.com/v2/ status 200
  # DEBU[0000] GET https://xlx9erfu.mirror.aliyuncs.com/v2/library/hello-world/manifests/latest

  # --- Manifest List  ---
  # DEBU[0008] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.list.v2+json"
  # DEBU[0008] Using blob info cache at /var/lib/containers/cache/blob-info-cache-v1.boltdb
  # DEBU[0008] Source is a manifest list; copying (only) instance sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042 for current system
  # DEBU[0008] GET https://xlx9erfu.mirror.aliyuncs.com/v2/library/hello-world/manifests/sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042 

  # --- Image Manifest  ---
  # DEBU[0009] Content-Type from manifest GET is "application/vnd.docker.distribution.manifest.v2+json" 
  # DEBU[0009] IsRunningImageAllowed for image docker:docker.io/library/hello-world:latest
  # DEBU[0009]  Using default policy section
  # DEBU[0009]  Requirement 0: allowed
  # DEBU[0009] Overall: allowed
  # DEBU[0009] Downloading /v2/library/hello-world/blobs/sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b
  # DEBU[0009] GET https://xlx9erfu.mirror.aliyuncs.com/v2/library/hello-world/blobs/sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b
  # Getting image source signatures
  # DEBU[0009] Manifest has MIME type application/vnd.docker.distribution.manifest.v2+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.v1+prettyjws, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v1+json]

  # --- Layers  ---
  # DEBU[0009] Downloading /v2/library/hello-world/blobs/sha256:0e03bdcc26d7a9a57ef3b6f1bf1a210cff6239bff7c8cac72435984032851689
  # DEBU[0009] GET https://xlx9erfu.mirror.aliyuncs.com/v2/library/hello-world/blobs/sha256:0e03bdcc26d7a9a57ef3b6f1bf1a210cff6239bff7c8cac72435984032851689
  # DEBU[0010] Detected compression format gzip
  # Copying blob 0e03bdcc26d7 done
  # DEBU[0010] No compression detected

  # --- 镜像ID --- 
  # Copying config bf756fb1ae done
  # Writing manifest to image destination
  # Storing signatures
  # DEBU[0010] Applying tar in /var/lib/containers/storage/overlay/9c27e219663c25e0f28493790cc0b88bc973ba3b1686355f221c38a36978ac63/diff
  # DEBU[0010] setting image creation date to 2020-01-03 01:21:37.263809283 +0000 UTC
  # DEBU[0010] created new image ID "bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b"
  # DEBU[0010] set names of image "bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b" to [docker.io/library/hello-world:latest]
  # DEBU[0010] saved image metadata "{"signatures-sizes":{"sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042":[]}}"
  # DEBU[0010] parsed reference into "[[email protected]/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/hello-world:latest"
  # bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b

$podman images hello-world
REPOSITORY                     TAG     IMAGE ID      CREATED       SIZE
docker.io/library/hello-world  latest  bf756fb1ae65  8 months ago  20 kB

# 2.创建pod(与K8s中含义一致)和查看pod
$ podman pull mirrorgcrio/pause:3.2 && podman tag docker.io/mirrorgcrio/pause:3.2 k8s.gcr.io/pause:3.2 # 创建pod前拉取k8s.gcr.io中pause应用组件(唯一与docker不同的地方)
$ podman pod create --name HelloWorld
  # 73c5a062cb17b5088072ec13c496c101b0b239f9aba1dcad93ba5d746cdfb12d
$ podman pod ls
  # POD ID        NAME        STATUS   CREATED         OF CONTAINERS    INFRA ID(下文ID)
  # 73c5a062cb17  HelloWorld  Created  29 seconds ago  1                15e7d3797552
$ podman container ls -a   # 当前POD运行的容器
  # CONTAINER ID  IMAGE                            COMMAND  CREATED             STATUS   PORTS   NAMES
  # 15e7d3797552  docker.io/mirrorgcrio/pause:3.2           About a minute ago  Created          73c5a062cb17-infra


# 3.在pod中创建并运行容器
$ podman run --pod HelloWorld hello-world  # hello-world镜像无后台程序所以无法后台运行
2fd059b66fb640393394e82404ed895e6a44673e7b1061ade81c1ae2e25e37fb
$ podman logs 2fd059b66  # 将会看见我们最熟悉的Hello-World
  # Hello from Docker!
  # This message shows that your installation appears to be working correctly.
  # ....

# 4.运行在pod中container查看
$ podman ps -ap
  # CONTAINER ID  IMAGE                                 COMMAND  CREATED        STATUS                    PORTS   NAMES               POD ID        PODNAME
  # 15e7d3797552  docker.io/mirrorgcrio/pause:3.2                6 minutes ago  Up 2 minutes ago                  73c5a062cb17-infra  73c5a062cb17  HelloWorld
  # 2fd059b66fb6  docker.io/library/hello-world:latest  /hello   2 minutes ago  Exited (0) 2 minutes ago          friendly_neumann    73c5a062cb17  HelloWorld  # 退出的容器

2.Podman-Hugo-envoy

描述:此处以podman来进行部署Hugo生成的静态页在nginx中运行然后由Envoy进行代理转发实现负载均衡,然后再由前度代理进行内部转发路径的选择; 案例方案:

1.首先会有一个前端代理在某个地方单独运行。前端代理的工作是给其他地方提供一个入口。来自外部的传入连接请求到这里，前端代理将会决定他们在内部的转发路径。
2.其次博客静态页面由 nginx 提供，同时运行一个 “服务 Envoy”，它与 nginx 容器共享 network nemspace（相当于 Kubernetes 的 Sidecar）。
3.所有的 Envoy 形成一个网格，然后在他们之间共享路由信息。

Tips: Envoy 是专为大型现代 SOA（面向服务架构）架构设计的 L7 代理和通信总线,简单的说主要实现高级负载均衡、前端/边缘代理支持等功能;

Step 1.环境准备模拟Nginx网页文件

mkdir -p /opt/HugoBlog/{nginx,envoy}
echo  '<h1>WeiyiGeek Blog With Nginx</h1>' > /opt/HugoBlog/nginx/index.html

Step 2.Envoy-services配置清单声明(重点)

sudo tee /opt/HugoBlog/envoy/service-envoy.yaml <<'EOF'
static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 8080
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.file_access_log
            config:
              path: "/dev/stdout"
          route_config:
            name: local_route
            virtual_hosts:
            - name: service
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.router
            config: {}
  clusters:
  - name: local_service
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    hosts:
    - socket_address:
        address: 127.0.0.1
        port_value: 80
admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8081
EOF

Step 3.利用Podman工具进行创建hugoblog与hugoblog-envoy容器

# 创建一个 hugoblog 容器并指定容器的 IP：
$ podman run -d --name hugoblog 
  --ip=10.88.0.10 
  -v /opt/HugoBlog/nginx:/usr/share/nginx/html 
  -v /etc/localtime:/etc/localtime 
  nginx:alpine
b466e555f6f1d84bc81454503afddf69de4d5812fda2b75b2f17c307f4430768

# 再创建一个 envoy 容器与hugoblog 容器共享 network namespace：
$ podman run -d --name hugoblog-envoy 
  -v /opt/HugoBlog/envoy/service-envoy.yaml:/etc/envoy/envoy.yaml 
  -v /etc/localtime:/etc/localtime 
  --net=container:hugoblog envoyproxy/envoy-alpine:latest

Step 4.查看创建的容器:

podman ps --size --pod # 此处没有直接运行在pod中可以看见PODID与PODNAME为空
  # CONTAINER ID  IMAGE                                     COMMAND               CREATED      STATUS          PORTS   NAMES           POD ID  PODNAME  SIZE
  # b466e555f6f1  docker.io/library/nginx:alpine            nginx -g daemon o...  2 hours ago  Up 2 hours ago          hugoblog                         1.12kB (virtual 22.1MB)
  # c18c914654ac  docker.io/envoyproxy/envoy-alpine:latest  envoy -c /etc/env...  2 hours ago  Up 2 hours ago          hugoblog-envoy                   0B (virtual 52.9MB)

Step 5.访问容器验证envoy工作

# 1.先看Nginx工作是否正常
~$ curl http://10.88.0.10
  # <h1>WeiyiGeek Blog With Nginx</h1>
~$ curl -I http://10.88.0.10   # 返回的头信息
  # HTTP/1.1 200 OK
  # Server: nginx/1.19.2
  # Date: Wed, 23 Sep 2020 04:55:31 GMT
  # Content-Type: text/html
  # Content-Length: 35
  # Last-Modified: Wed, 23 Sep 2020 04:48:24 GMT
  # Connection: keep-alive
  # ETag: "5f6ad398-23"
  # Accept-Ranges: bytes

# 2.再来看envoy是否工作正常
[email protected]:~$ curl http://10.88.0.10:8080
  # <h1>WeiyiGeek Blog With Nginx</h1>
[email protected]:~$ curl -I http://10.88.0.10:8080   # 返回的头信息
  # HTTP/1.1 200 OK
  # server: envoy                                    # 关键点: 可以看见已经成功负载了
  # date: Wed, 23 Sep 2020 06:05:01 GMT
  # content-type: text/html
  # content-length: 35
  # last-modified: Wed, 23 Sep 2020 04:48:24 GMT
  # etag: "5f6ad398-23"
  # accept-ranges: bytes
  # x-envoy-upstream-service-time: 0                # 关键点

Step 6.前面说过podman 创建的容器是 podman 的子进程而实际上 podman 由两部分组成:

(1) podman CLI
(2) container runtime : conmon 是所有容器的父进程并且由 conmon 来负责主要包括监控、日志、TTY 分配以及类似 out-of-memory 情况的杂事,也就是说去做所有 systemd 不做或者不想做的事情并且。即使 CRI-O 不直接使用 systemd 来管理容器它也将容器分配到 sytemd 兼容的 cgroup 中，好处是常规的 systemd 工具比如 systemctl 就可以看见容器资源使用情况了。

# 1.podman 进程查看
pstree -p
# systemd(1)─┬─VGAuthService(733)
#            ├─conmon(824338)─┬─nginx(824363)─┬─nginx(824399)
#            │                │               └─nginx(824400)
#            │                └─{conmon}(824340)
#            ├─conmon(824555)─┬─envoy(824585)─┬─{envoy}(824596)
#            │                │               ├─{envoy}(824597)
#            │                │               ├─{envoy}(824598)
#            │                │               ├─{envoy}(824599)
#            │                │               ├─{envoy}(824600)
#            │                │               ├─{envoy}(824601)
#            │                │               ├─{envoy}(824602)
#            │                │               ├─{envoy}(824603)
#            │                │               ├─{envoy}(824604)
#            │                │               └─{envoy}(826782)
#            │                └─{conmon}(824557)
#            ├─containerd(720296)─┬─{containerd}(720297)
#            │                    ├─{containerd}(720298)
#            │                    ├─{containerd}(720299)
#            │                    ├─{containerd}(720300)
#            │                    ├─{containerd}(720301)
#            │                    ├─{containerd}(720302)
#            │                    ├─{containerd}(720305)
#            │                    ├─{containerd}(720307)
#            │                    ├─{containerd}(720310)
#            │                    ├─{containerd}(734346)
#            │                    └─{containerd}(779760)
#            ├─podman pause(737423)


# 2.通过conmon进程将容器资源使用情况给systemd相关程序所共享比如下面查看cgroup相关信息
systemd-cgtop
# Control Group                                                                                     Tasks   %CPU   Memory  Input/s Output/s
# /                                                                                                   279      -     3.2G        -        -
# init.scope                                                                                            1      -    10.5M        -        -
# machine.slice                                                                                        18      -    17.7M        -        -
# machine.slice/libpod-b466e555f6f1d84bc81454503afddf69de4d5812fda2b75b2f17c307f4430768.scope           3      -     3.1M        -        -  #  hugoblog
# machine.slice/libpod-c18c914654ac4ad1ddcd2f5d7b51917ef6c066fdec353c608e0f1c8d653b253c.scope          11      -    12.7M        -        -  #  hugoblog-envoy  
# machine.slice/libpod-conmon-b46…5f6f1d84bc81454503afddf69de4d5812fda2b75b2f17c307f4430768.scope       2      -   840.0K        -        -
# machine.slice/libpod-conmon-c18…654ac4ad1ddcd2f5d7b51917ef6c066fdec353c608e0f1c8d653b253c.scope       2      -   816.0K        -        -

Step 7.目前为止它还是一个半成品只能在本机访问10.88.0.10该pod地址,外部网络无法直接访问所以为了实现外部网络访问我们网站我需要部署前端代理;

自签证书:(注意对外部访问的生产环境不建议采用自签证书)

mkdir /opt/ssl
/opt/ssl# openssl genrsa -des3 -out server.pass.key 2048
/opt/ssl# openssl rsa -in server.pass.key -out demo.weiyigeek.top.key
/opt/ssl# openssl req -new -key demo.weiyigeek.top.key -out server.csr -subj "/C=CN/ST=Chongqing/L=Chongqing/O=WeiyiGeek/OU=WeiyiGeek/CN=demo.weiyigeek.top"
/opt/ssl# openssl x509 -req -days 3650 -in server.csr -signkey server.key -out demo.weiyigeek.top.crt
/opt/ssl# openssl x509 -in demo.weiyigeek.top.crt -out demo.weiyigeek.top.cer -outform der
/opt/ssl# ls -alh    
# total 28K  # 生成相关证书文件
# -rw-r--r-- 1 root root  895 Sep 23 07:29 demo.weiyigeek.top.cer
# -rw-r--r-- 1 root root 1.3K Sep 23 07:27 demo.weiyigeek.top.crt
# -rw-r--r-- 1 root root 1.1K Sep 23 07:27 server.csr
# -rw------- 1 root root 1.7K Sep 23 07:23 demo.weiyigeek.top.key
# -rw------- 1 root root 1.8K Sep 23 07:23 server.pass.key

envoy 的配置文件中是通过域名来添加 cluster 的 front-envoy.yaml 内容如下,官方参考地址 front-envoy.yaml ：

sudo tee /opt/HugoBlog/envoy/front-envoy.yaml <<'EOF'
static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 80
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.file_access_log
            config:
              path: "/dev/stdout"
          route_config:
            virtual_hosts:
            - name: backend
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/"
                redirect:
                  https_redirect: true
                  response_code: "FOUND"
          http_filters:
          - name: envoy.router
            config: {}
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 443
    filter_chains:
    - filter_chain_match:
        server_names: ["demo.weiyigeek.top"]
      tls_context:
        common_tls_context:
          alpn_protocols: h2
          tls_params:
            tls_maximum_protocol_version: TLSv1_3
          tls_certificates:
            # 证书链此处是自签而不是第三方证书颁发机构
            - certificate_chain:
                filename: "/opt/ssl/demo.weiyigeek.top.crt"  
              private_key:
                filename: "/opt/ssl/demo.weiyigeek.top.key"
      filters:
      - name: envoy.filters.network.http_connection_manager
        config:
          codec_type: auto
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: backend
              domains:
              - "demo.weiyigeek.top"
              routes:
              - match:
                  prefix: "/"
                route:
                  cluster: hugoblog
          http_filters:
          - name: envoy.router
            config: {}
  clusters:
  - name: hugoblog
    connect_timeout: 0.25s
    type: strict_dns
    lb_policy: round_robin
    http2_protocol_options: {}
    load_assignment:
      cluster_name: hugoblog
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: hugoblog
                port_value: 8080
EOF

由于没办法自动服务发现需要通过参数--add-host 手动添加 hosts 到hugoblog容器中(注意映射证书签名)

$ podman run -d --name hugoblog-front-envoy 
--add-host=hugoblog:10.88.0.10 
-v /opt/HugoBlog/envoy/front-envoy.yaml:/etc/envoy/envoy.yaml 
-v /etc/localtime:/etc/localtime 
-v /opt/ssl/:/opt/ssl/ 
--net host envoyproxy/envoy
# 0a3a62265c6d464f86cce41a8cea5f7316d226bbb62be7124dfc44f7aebcd2f5
# [2020-09-23 12:22:50.096][1][info][config] [source/server/configuration_impl.cc:129] loading stats sink configuration
# [2020-09-23 12:22:50.096][1][info][main] [source/server/server.cc:554] starting main dispatch loop
# [2020-09-23 12:22:50.096][1][info][upstream] [source/common/upstream/cluster_manager_impl.cc:171] cm init: all clusters initialized
# [2020-09-23 12:22:50.096][1][info][main] [source/server/server.cc:533] all clusters initialized. initializing init manager
# [2020-09-23 12:22:50.096][1][info][config] [source/server/listener_manager_impl.cc:725] all dependencies initialized. starting workers

Step 8.访问验证envoyproxy前端代理是是否成功

# 1.直接访问非http网站会强制302进行跳转
[email protected]:/mnt/c/Users/WeiyiGeek/Desktop# curl -I demo.weiyigeek.top
HTTP/1.1 302 Found
location: https://demo.weiyigeek.top/
date: Wed, 23 Sep 2020 12:26:17 GMT
server: envoy   # 可以看见server是envoy
transfer-encoding: chunked


# 2.访问https网站结果
[email protected]:/mnt/c/Users/WeiyiGeek/Desktop# curl https://demo.weiyigeek.top --insecure
<h1>WeiyiGeek Blog With Nginx</h1>

[email protected]:/mnt/c/Users/WeiyiGeek/Desktop# curl https://demo.weiyigeek.top --insecure -I
HTTP/2 200
server: envoy
date: Wed, 23 Sep 2020 12:28:37 GMT
content-type: text/html
content-length: 35
last-modified: Wed, 23 Sep 2020 04:48:24 GMT
etag: "5f6ad398-23"
accept-ranges: bytes
x-envoy-upstream-service-time: 0

WeiyiGeek.EnvoyProxy

3.Podman-导出与部署声明式清单

描述:我们将上面的示例整合进指定pod中然后进行导出清单和快捷部署导出的清单;

Step 1.在前面的环境的基础上创建pod以及将创建的容器加入到pod中

# pod 创建并设置其ip地址(非常重要)
$ podman pod create --ip 10.88.0.10 --name Blog

# 创建一个nginx容器并指加入到pod中
$ podman run -d --pod Blog --name hugoblog 
  -v /opt/HugoBlog/nginx:/usr/share/nginx/html 
  -v /etc/localtime:/etc/localtime 
  nginx:alpine

# 创建一个 envoy 容器 与hugoblog 容器共享 network namespace也加入到pod中
$ podman run -d --pod Blog --name hugoblog-envoy 
  -v /opt/HugoBlog/envoy/service-envoy.yaml:/etc/envoy/envoy.yaml 
  -v /etc/localtime:/etc/localtime 
  --net=container:hugoblog envoyproxy/envoy-alpine:latest

# 部署前端代理将pod内部应用进行转发到宿主机上
podman run -d --pod Blog --name hugoblog-front-envoy 
  --add-host=hugoblog:10.88.0.10 
  -v /opt/HugoBlog/envoy/front-envoy.yaml:/etc/envoy/envoy.yaml 
  -v /etc/localtime:/etc/localtime 
  -v /opt/ssl/:/opt/ssl/ 
  --net host envoyproxy/envoy

Step 2.本机访问验证

[email protected]:~# nano /etc/hosts
10.10.107.202 demo.weiyigeek.top

[email protected]:~# curl -I https://demo.weiyigeek.top --insecure
HTTP/2 200
server: envoy
date: Wed, 23 Sep 2020 13:17:57 GMT
content-type: text/html
content-length: 71
last-modified: Wed, 23 Sep 2020 12:35:16 GMT
etag: "5f6b4104-47"
accept-ranges: bytes
x-envoy-upstream-service-time: 0

[email protected]:~# curl https://demo.weiyigeek.top --insecure
<h3 style="color:red">WeiyiGeek Blog , Example Podman pod Create Nginx and Envoy and EnvoyProxy!</h3>

Step 3.外部机器访问

WeiyiGeek.envoyproxy

Step 4.将 pod 导出为声明式部署清单

$ podman generate kube Blog > Blog.yaml 
$ cat Blog.yaml 
# Generation of Kubernetes YAML is still under development!
#
# Save the output of this file and use kubectl create -f to import
# it into Kubernetes.
#
# Created with podman-2.0.6
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2020-09-23T13:50:40Z"
  labels:
    app: Blog
  name: Blog
spec:
  containers:
  - command:
    - envoy
    - -c
    - /etc/envoy/envoy.yaml
    env:
    - name: PATH
      value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    - name: TERM
      value: xterm
    - name: LANG
      value: C.UTF-8
    - name: container
      value: podman
    - name: HOSTNAME
      value: Blog
    image: docker.io/envoyproxy/envoy-alpine:latest
    name: hugoblog-envoy
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
      seLinuxOptions: {}
    volumeMounts:
    - mountPath: /etc/envoy/envoy.yaml
      name: opt-HugoBlog-envoy-service-envoy.yaml
    - mountPath: /etc/localtime
      name: etc-localtime
    workingDir: /
  - command:
    - envoy
    - -c
    - /etc/envoy/envoy.yaml
    env:
    - name: PATH
      value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    - name: TERM
      value: xterm
    - name: container
      value: podman
    - name: HOSTNAME
      value: Blog
    image: docker.io/envoyproxy/envoy:latest
    name: hugoblog-front-envoy
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
      seLinuxOptions: {}
    volumeMounts:
    - mountPath: /etc/localtime
      name: etc-localtime
    - mountPath: /opt/ssl
      name: opt-ssl
    - mountPath: /etc/envoy/envoy.yaml
      name: opt-HugoBlog-envoy-front-envoy.yaml
    workingDir: /
  - command:
    - nginx
    - -g
    - daemon off;
    env:
    - name: PATH
      value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    - name: TERM
      value: xterm
    - name: NGINX_VERSION
      value: 1.19.2
    - name: NJS_VERSION
      value: 0.4.3
    - name: PKG_RELEASE
      value: "1"
    - name: container
      value: podman
    - name: HOSTNAME
      value: Blog
    image: docker.io/library/nginx:alpine
    name: hugoblog
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
      seLinuxOptions: {}
    volumeMounts:
    - mountPath: /usr/share/nginx/html
      name: opt-HugoBlog-nginx
    - mountPath: /etc/localtime
      name: etc-localtime
    workingDir: /
  volumes:
  - hostPath:
      path: /opt/HugoBlog/envoy/service-envoy.yaml
      type: File
    name: opt-HugoBlog-envoy-service-envoy.yaml
  - hostPath:
      path: /etc/localtime
      type: File
    name: etc-localtime
  - hostPath:
      path: /opt/ssl
      type: Directory
    name: opt-ssl
  - hostPath:
      path: /opt/HugoBlog/envoy/front-envoy.yaml
      type: File
    name: opt-HugoBlog-envoy-front-envoy.yaml
  - hostPath:
      path: /opt/HugoBlog/nginx
      type: Directory
    name: opt-HugoBlog-nginx
status: {}
---
metadata:
  creationTimestamp: null
spec: {}
status:
  loadBalancer: {}

怎么样是不是有种熟悉的味道? 答: 它是一个兼容 kubernetes 的 pod 定义，你可以直接通过 kubectl apply -f hugo.yaml 将其部署在 Kubernetes 集群中也可以直接通过 podman 部署 podman play kube hugo.yaml, 回到之前的问题，如果通过声明式定义来创建 pod，还是无法解决服务发现的问题，除非换个支持静态 IP 的 CNI 插件，而支持静态 IP 的这些 CNI 插件又需要 etcd 作为数据库看来还是暂时放弃;

Step 5.pod中的容器systemd管理脚本生成,注意 podman 不再使用 daemon 管理服务所以--restart 参数被废弃了

$ podman generate systemd hugoblog
# container-2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd.service
# autogenerated by Podman 2.0.6
# Wed Sep 23 13:59:13 UTC 2020
[Unit]
Description=Podman container-2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd.service
Documentation=man:podman-generate-systemd(1)
Wants=network.target
After=network-online.target

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
ExecStart=/usr/bin/podman start 2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd
ExecStop=/usr/bin/podman stop -t 10 2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd
ExecStopPost=/usr/bin/podman stop -t 10 2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd
PIDFile=/var/run/containers/storage/overlay-containers/2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd/userdata/conmon.pid
KillMode=none
Type=forking

[Install]
WantedBy=multi-user.target default.target


$ podman generate systemd hugoblog-envoy
# container-23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736.service
# autogenerated by Podman 2.0.6
# Wed Sep 23 13:59:43 UTC 2020
[Unit]
Description=Podman container-23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736.service
Documentation=man:podman-generate-systemd(1)
Wants=network.target
After=network-online.target
After=hugoblog.service

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
ExecStart=/usr/bin/podman start 23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736
ExecStop=/usr/bin/podman stop -t 10 23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736
ExecStopPost=/usr/bin/podman stop -t 10 23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736
PIDFile=/var/run/containers/storage/overlay-containers/23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736/userdata/conmon.pid
KillMode=none
Type=forking

[Install]
WantedBy=multi-user.target default.target


$podman generate systemd hugoblog-front-envoy
# container-25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce.service
# autogenerated by Podman 2.0.6
# Wed Sep 23 14:00:22 UTC 2020

[Unit]
Description=Podman container-25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce.service
Documentation=man:podman-generate-systemd(1)
Wants=network.target
After=network-online.target
After=hugoblog.service hugoblog-envoy.service  # 启动顺序非常重要

[Service]
Environment=PODMAN_SYSTEMD_UNIT=%n
Restart=on-failure
ExecStart=/usr/bin/podman start 25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce
ExecStop=/usr/bin/podman stop -t 10 25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce
ExecStopPost=/usr/bin/podman stop -t 10 25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce
PIDFile=/var/run/containers/storage/overlay-containers/25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce/userdata/conmon.pid
KillMode=none
Type=forking

[Install]
WantedBy=multi-user.target default.target

实际上我们可以直接采用pod为单位进行管理起容器启动和停止：

sudo tee /etc/systemd/system/Blog.service <<'EOF'
[Unit]
Description=WeiyiGeek-Blog-Service
After=network.target
After=network-online.target
After=podman.service
 
[Service]
Type=simple
ExecStart=/usr/bin/podman pod start Blog
ExecStop=/usr/bin/podman pod stop -t 10 Blog
KillMode=none
Type=forking

 
[Install]
WantedBy=multi-user.target default.target
EOF

Step 6.设置开启自启已经systemd守护进程监听

# 1.停止所有的容器
podman stop $(podman ps -aq)

# 2.设置开机自启
systemctl enable Blog.service

# 3.启动pod
[email protected]:~# systemctl start Blog.service
[email protected]:~# systemctl status Blog.service
● Blog.service - WeiyiGeek-Blog-Service
     Loaded: loaded (/etc/systemd/system/Blog.service; disabled; vendor preset: enabled)
     Active: active (running) since Wed 2020-09-23 14:21:28 UTC; 5s ago
    Process: 880757 ExecStart=/usr/bin/podman pod start Blog (code=exited, status=0/SUCCESS)
      Tasks: 8 (limit: 4620)
     Memory: 3.0M
     CGroup: /system.slice/Blog.service
             ├─880804 /usr/libexec/podman/conmon --api-version 1 -c 9a1c6b58305ca3b3b7dc58ed99388ab309384900d54f4d1d71c94c5172c67167 -u 9a1c6b58305ca3b3b7dc58ed993>
             ├─880835 /usr/libexec/podman/conmon --api-version 1 -c 25bcd2f79803a1b7dd653e36a0d0656d6d055df5a29d184e71df728e1c46adce -u 25bcd2f79803a1b7dd653e36a0d>
             ├─880859 /usr/libexec/podman/conmon --api-version 1 -c 2603018909da9c48c7bf4b369a2d540a92573749e120aa0832b19ad908c81cdd -u 2603018909daton9c48c7bf4b369a2>
             └─880906 /usr/libexec/podman/conmon --api-version 1 -c 23444cf45564d9e3a7d0fe96343d8d0e08c83823403bc05cf99e0a0928318736 -u 23444cf45564d9e3a7d0fe96343>

Sep 23 14:21:27 ubuntu systemd[1]: Starting WeiyiGeek-Blog-Service...
Sep 23 14:21:28 ubuntu podman[880757]: 80171e5522608aee384e001ec3267bf091837e0075a64a93c45b1b64381674ac
Sep 23 14:21:28 ubuntu systemd[1]: Started WeiyiGeek-Blog-Service.

# 4.采用systemd停止pod及其容器
[email protected]:~# systemctl stop Blog.service
[email protected]:~# systemctl status Blog.service
● Blog.service - WeiyiGeek-Blog-Service
     Loaded: loaded (/etc/systemd/system/Blog.service; disabled; vendor preset: enabled)
     Active: inactive (dead)

Sep 23 14:23:34 ubuntu systemd[1]: Stopping WeiyiGeek-Blog-Service...
Sep 23 14:23:35 ubuntu podman[880999]: 80171e5522608aee384e001ec3267bf091837e0075a64a93c45b1b64381674ac
Sep 23 14:23:35 ubuntu systemd[1]: Blog.service: Succeeded.
Sep 23 14:23:35 ubuntu systemd[1]: Stopped WeiyiGeek-Blog-Service.

至此每次系统重启后 systemd 都会自动启动这个服务所对应的容器。

0x03 基础配置

1.镜像加速

描述:国内直接用 podman pull 拉取镜像会很慢所以需要配置阿里云容器镜像来加速访问。配置说明: Podman 默认注册表配置文件在/etc/containers/registries.conf,把 location 对应的值修改为你的阿里云容器加速镜像地址然后重新服务即可;

# 参考1
unqualified-search-registries = ["docker.io"]
[[registry]]
prefix = "docker.io"
insecure = true
location = "xlx9erfu.mirror.aliyuncs.com"

# 参考2
unqualified-search-registries = ['docker.io']
[[registry]]
prefix = "docker.io"
insecure = false
location = "docker.io"
[[registry.mirror]]
location = "xlx9erfu.mirror.aliyuncs.com"
insecure = true

主要事项:

版本1中registries.search, registries.insecure, and registries.block 格式已弃用
运行podman info命令可查看设置加速镜像

# 方式1
registries:
  docker.io:
    Blocked: false
    Insecure: false
    Location: docker.io
    MirrorByDigestOnly: false
    Mirrors:
    - Insecure: true
      Location: xlx9erfu.mirror.aliyuncs.com
    Prefix: docker.io
  search:
  - docker.io
# 方式2
podman info -f "{{.Registries}}"
map[docker.io:{docker.io {docker.io false} [{xlx9erfu.mirror.aliyuncs.com true}] false false} search:[docker.io]]r

参考连接: https://github.com/containers/podman/issues/5764#issuecomment-611157552

0x04 入坑出坑

问题.创建pod的提示Error initializing source docker://k8s.gcr.io/pause:3.2:dial tcp 108.177.97.82:443: i/o timeout

问题描述: 首次执行 podman pod create –name HelloWorld 时候会出现以下错误;

ERRO[0060] Error freeing pod lock after failed creation: no such file or directory
Error: error adding Infra Container: unable to pull k8s.gcr.io/pause:3.2: Error initializing source docker://k8s.gcr.io/pause:3.2: error pinging docker registry k8s.gcr.io: Get "https://k8s.gcr.io/v2/": dial tcp 108.177.97.82:443: i/o timeout

解决方式:

$podman pull mirrorgcrio/pause:3.2 --log-level info
$podman images
# REPOSITORY                        TAG     IMAGE ID      CREATED        SIZE
# docker.io/mirrorgcrio/pause       3.2     80d28bedfe5d  7 months ago   686 kB
# docker.io/library/hello-world     latest  bf756fb1ae65  8 months ago   20 kB
$podman tag docker.io/mirrorgcrio/pause:3.2 k8s.gcr.io/pause:3.2
$podman images
REPOSITORY                        TAG     IMAGE ID      CREATED        SIZE
docker.io/mirrorgcrio/pause       3.2     80d28bedfe5d  7 months ago   686 kB
k8s.gcr.io/pause                  3.2     80d28bedfe5d  7 months ago   686 kB