一步步安装部署kubernetes集群（三）3.2

时间:2020-05-09
本文章向大家介绍一步步安装部署kubernetes集群（三）3.2，主要包括一步步安装部署kubernetes集群（三）3.2使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。

3.2 部署kube-apiserver集群

3.2.1 集群规划

主机名 角色 IP地址
pg60-11.k8s.host.com 4层负载均衡 10.20.60.11
pg60-12.k8s.host.com 4层负载均衡 10.20.60.12
pg60-21.k8s.host.com kube-apiserver 10.20.60.21
pg60-22.k8s.host.com kube-apiserver 10.20.60.22

注意：这里 10.20.60.11 和 10.20.60.12 使用nginx做4层负载均衡器，在keepalived上配置vip：10.20.60.10，代理两个 kube-apiserver，实现高可用。

3.2.2 下载并安装kubernetes软件

在 pg60-21.k8s.host.com 虚机实例上操作，另外一台运算节点安装部署方法类似。

shell> wget https://dl.k8s.io/v1.18.0/kubernetes-server-linux-amd64.tar.gz
shell> tar zxf kubernetes-server-linux-amd64.tar.gz -C /opt/
shell> mv /opt/kubernetes /opt/kubernetes-server-linux-amd64
shell> ln -s /opt/kubernetes-server-linux-amd64 /opt/kubernetes
shell> mkdir /opt/kubernetes/server/{ssl,conf,sbin}

3.2.3 签发client证书

在 pg60-199.k8s.host.com 虚机实例上操作

  • 创建生成证书签名请求（csr）的 json 配置文件
shell> cat > client-csr.json << EOF
{
    "CN": "k8s-node",
    "hosts": [
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "91donkey",
            "OU": "ops"
        }
    ]
}
EOF
  • 生成client证书和私钥
shell> cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client-csr.json | cfssl-json -bare client
2020/05/09 12:51:36 [INFO] generate received request
2020/05/09 12:51:36 [INFO] received CSR
2020/05/09 12:51:36 [INFO] generating key: rsa-2048
2020/05/09 12:51:37 [INFO] encoded CSR
2020/05/09 12:51:37 [INFO] signed certificate with serial number 401229144308676946860009337123703016454826971073
2020/05/09 12:51:37 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
  • 检查生成的证书和私钥
shell> ls -l |grep client
-rw-r--r-- 1 root root 1001 May 9 12:51 client.csr
-rw-r--r-- 1 root root 286 May 9 12:50 client-csr.json
-rw------- 1 root root 1675 May 9 12:51 client-key.pem
-rw-r--r-- 1 root root 1371 May 9 12:51 client.pem

3.2.4 签发kube-apiserver证书

在 pg60-199.k8s.host.com 虚机实例上操作

  • 创建生成证书签名请求（csr）的 json 配置文件
shell> cat > apiserver-csr.json  << EOF
{
    "CN": "apiserver",
    "hosts": [
        "127.0.0.1",
        "192.168.0.1",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local",
        "10.20.60.10",
        "10.20.60.21",
        "10.20.60.22"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "91donkey",
            "OU": "ops"
        }
    ]
}
EOF
  • 生成kube-apiserver证书和私钥
shell> cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server apiserver-csr.json | cfssl-json -bare apiserver
2020/05/09 12:57:50 [INFO] generate received request
2020/05/09 12:57:50 [INFO] received CSR
2020/05/09 12:57:50 [INFO] generating key: rsa-2048
2020/05/09 12:57:51 [INFO] encoded CSR
2020/05/09 12:57:51 [INFO] signed certificate with serial number 704123904604325511866815694194275404583444068714
2020/05/09 12:57:51 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
  • 检查生成的证书和私钥
shell> ls -l |grep apiserver
-rw-r--r-- 1 root root 1253 May 9 12:57 apiserver.csr
-rw-r--r-- 1 root root 580 May 9 12:55 apiserver-csr.json
-rw------- 1 root root 1675 May 9 12:57 apiserver-key.pem
-rw-r--r-- 1 root root 1602 May 9 12:57 apiserver.pem

3.2.5 拷贝证书至各运算节点，并创建配置

在 pg60-21.k8s.host.com 虚机实例上操作

  • 拷贝证书、私钥（注意私钥文件属性600）

  • 创建配置

shell> cat /opt/kubernetes/server/conf/audit.yaml 
apiVersion: audit.k8s.io/v1 # This is required.
kind: Policy
# Don't generate audit events for all requests in RequestReceived stage.
omitStages:
  - "RequestReceived"
rules:
  # Log pod changes at RequestResponse level
  - level: RequestResponse
    resources:
    - group: ""
      # Resource "pods" doesn't match requests to any subresource of pods,
      # which is consistent with the RBAC policy.
      resources: ["pods"]
  # Log "pods/log", "pods/status" at Metadata level
  - level: Metadata
    resources:
    - group: ""
      resources: ["pods/log", "pods/status"]

  # Don't log requests to a configmap called "controller-leader"
  - level: None
    resources:
    - group: ""
      resources: ["configmaps"]
      resourceNames: ["controller-leader"]

  # Don't log watch requests by the "system:kube-proxy" on endpoints or services
  - level: None
    users: ["system:kube-proxy"]
    verbs: ["watch"]
    resources:
    - group: "" # core API group
      resources: ["endpoints", "services"]

  # Don't log authenticated requests to certain non-resource URL paths.
  - level: None
    userGroups: ["system:authenticated"]
    nonResourceURLs:
    - "/api*" # Wildcard matching.
    - "/version"

  # Log the request body of configmap changes in kube-system.
  - level: Request
    resources:
    - group: "" # core API group
      resources: ["configmaps"]
    # This rule only applies to resources in the "kube-system" namespace.
    # The empty string "" can be used to select non-namespaced resources.
    namespaces: ["kube-system"]

  # Log configmap and secret changes in all other namespaces at the Metadata level.
  - level: Metadata
    resources:
    - group: "" # core API group
      resources: ["secrets", "configmaps"]

  # Log all other resources in core and extensions at the Request level.
  - level: Request
    resources:
    - group: "" # core API group
    - group: "extensions" # Version of group should NOT be included.

  # A catch-all rule to log all other requests at the Metadata level.
  - level: Metadata
    # Long-running requests like watches that fall under this rule will not
    # generate an audit event in RequestReceived.
    omitStages:
      - "RequestReceived"
  • 创建启动脚本
shell> cat > /opt/kubernetes/server/sbin/kube-apiserver.sh << EOF
#!/bin/bash
/opt/kubernetes/server/bin/kube-apiserver \
  --apiserver-count 2 \
  --audit-log-path /export/kubernetes/logs/kube-apiserver/audit-log \
  --audit-policy-file /opt/kubernetes/server/conf/audit.yaml \
  --authorization-mode RBAC \
  --client-ca-file /opt/kubernetes/server/ssl/ca.pem \
  --requestheader-client-ca-file /opt/kubernetes/server/ssl/ca.pem \
  --enable-admission-plugins NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota \
  --etcd-cafile /opt/kubernetes/server/ssl/ca.pem \
  --etcd-certfile /opt/kubernetes/server/ssl/client.pem \
  --etcd-keyfile /opt/kubernetes/server/ssl/client-key.pem \
  --etcd-servers https://10.20.60.12:2379,https://10.20.60.21:2379,https://10.20.60.22:2379 \
  --service-account-key-file /opt/kubernetes/server/ssl/ca-key.pem \
  --service-cluster-ip-range 192.168.0.0/16 \
  --service-node-port-range 3000-29999 \
  --target-ram-mb=1024 \
  --kubelet-client-certificate /opt/kubernetes/server/ssl/client.pem \
  --kubelet-client-key /opt/kubernetes/server/ssl/client-key.pem \
  --log-dir /export/kubernetes/logs/kube-apiserver \
  --tls-cert-file /opt/kubernetes/server/ssl/apiserver.pem \
  --tls-private-key-file /opt/kubernetes/server/ssl/apiserver-key.pem \
  --v 2
EOF

shell> chmod +x /opt/kubernetes/server/sbin/kube-apiserver.sh
shell> mkdir -p /export/kubernetes/logs/kube-apiserver

3.2.6 在supervisor中创建kube-apiserver.conf的配置

shell> cat > /opt/supervisor/conf.d/kube-apiserver.conf << EOF
[program:kube-apiserver]
command=/opt/kubernetes/server/sbin/kube-apiserver.sh
numprocs=1
directory=/opt/kubernetes/server/bin
autostart=true
autorestart=true
startsecs=22
startretries=3
exitcodes=0,2
stopsignal=QUIT
stopwaitsecs=10
user=root
redirect_stderr=false
stdout_logfile=/export/kubernetes/logs/kube-apiserver/apiserver.stdout.log
stdout_logfile_maxbytes=64MB
stdout_logfile_backups=4
stdout_capture_maxbytes=1MB
stdout_events_enabled=false
stderr_logfile=/export/kubernetes/logs/kube-apiserver/apiserver.stderr.log
stderr_logfile_maxbytes=64MB
stderr_logfile_backups=4
stderr_capture_maxbytes=1MB
stderr_events_enabled=false
EOF

3.2.7 启动kube-apiserver服务并检查

shell> supervisorctl update
shell> supervisorctl status
etcd-server-60-21 RUNNING pid 31686, uptime 0:01:20
kube-apiserver RUNNING pid 31665, uptime 0:01:23

原文地址：https://www.cnblogs.com/91donkey/p/12857568.html

随机文章
本站知识点必读
最近更新