策略引擎>> kubeadm-ha>> 返回
项目作者： cookeem

项目描述 ：
Kubernetes high availiability deploy based on kubeadm, loadbalancer included (English/中文 for v1.15 - v1.20+)
高级语言： Smarty
项目地址: git://github.com/cookeem/kubeadm-ha.git
创建时间： 2017-06-28T06:44:56Z
项目社区：https://github.com/cookeem/kubeadm-ha
开源协议：MIT License
下载


通过kubeadm安装kubernetes高可用集群(支持docker和containerd作为kubernetes的容器运行时)

部署节点信息

hostname ip address comment
k8s-master01 192.168.0.101 kubernetes 控制平面主机 master01
k8s-master02 192.168.0.102 kubernetes 控制平面主机 master02
k8s-master03 192.168.0.103 kubernetes 控制平面主机 master03
k8s-vip 192.168.0.100 kubernetes 浮动IP，通过keepalived创建，如果使用公有云请预先申请该浮动IP 
  1. # 各节点请添加主机名解释
  2. cat << EOF >> /etc/hosts
  3. 192.168.0.100    k8s-vip
  4. 192.168.0.101    k8s-master01
  5. 192.168.0.102    k8s-master02
  6. 192.168.0.103    k8s-master03
  7. EOF

架构说明

  • 演示需要，只部署3个高可用的master节点
  • 使用keepalived和nginx作为高可用的负载均衡器，通过dorycli命令行工具生成负载均衡器的配置，并通过docker-compose部署负载均衡器
  • 容器运行时使用docker，cri-socket使用cri-dockerd连接docker和kubernetes

版本信息

  1. # 操作系统版本: Debian 11
  2. $ lsb_release -a
  3. No LSB modules are available.
  4. Distributor ID:     Debian
  5. Description:        Debian GNU/Linux 11 (bullseye)
  6. Release:            11
  7. Codename:           bullseye
  9. # docker版本: 24.0.5
  10. $ docker version
  11. Client: Docker Engine - Community
  12.  Version:           24.0.5
  13.  API version:       1.43
  14.  Go version:        go1.20.6
  15.  Git commit:        ced0996
  16.  Built:             Fri Jul 21 20:35:45 2023
  17.  OS/Arch:           linux/amd64
  18.  Context:           default
  20. Server: Docker Engine - Community
  21.  Engine:
  22.   Version:          24.0.5
  23.   API version:      1.43 (minimum version 1.12)
  24.   Go version:       go1.20.6
  25.   Git commit:       a61e2b4
  26.   Built:            Fri Jul 21 20:35:45 2023
  27.   OS/Arch:          linux/amd64
  28.   Experimental:     false
  29.  containerd:
  30.   Version:          1.6.22
  31.   GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
  32.  runc:
  33.   Version:          1.1.8
  34.   GitCommit:        v1.1.8-0-g82f18fe
  35.  docker-init:
  36.   Version:          0.19.0
  37.   GitCommit:        de40ad0
  39. # cri-dockerd版本: 0.3.4
  40. $ cri-dockerd --version
  41. cri-dockerd 0.3.4 (e88b1605)
  43. # dorycli版本: v1.7.0
  44. $ dorycli version
  45. dorycli version: v1.7.0
  46. install dory-engine version: v2.7.0
  47. install dory-console version: v2.7.0
  50. # kubeadm版本: v1.28.0
  51. $ kubeadm version
  52. kubeadm version: &version.Info{Major:"1", Minor:"28", GitVersion:"v1.28.0", GitCommit:"855e7c48de7388eb330da0f8d9d2394ee818fb8d", GitTreeState:"clean", BuildDate:"2023-08-15T10:20:15Z", GoVersion:"go1.20.7", Compiler:"gc", Platform:"linux/amd64"}
  54. # kubernetes版本: v1.28.0
  55. $ kubectl get nodes
  56. NAME           STATUS   ROLES           AGE   VERSION
  57. k8s-master01   Ready    control-plane   35m   v1.28.0
  58. k8s-master02   Ready    control-plane   31m   v1.28.0
  59. k8s-master03   Ready    control-plane   30m   v1.28.0

安装docker

  • 在所有节点安装docker服务
  1. # 安装基础软件
  2. apt-get update
  3. apt-get install -y sudo wget ca-certificates curl gnupg htop git jq tree
  5. # 安装docker
  6. install -m 0755 -d /etc/apt/keyrings
  7. curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
  8. chmod a+r /etc/apt/keyrings/docker.gpg
  9. echo   "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \
  10.   "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" |   tee /etc/apt/sources.list.d/docker.list > /dev/null
  11. apt-get update
  12. apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin docker-compose
  14. # 检查docker版本
  15. docker version
  17. # 设置docker参数
  18. cat << EOF > /etc/docker/daemon.json
  19. {
  20.     "exec-opts": ["native.cgroupdriver=systemd"],
  21.     "log-driver": "json-file",
  22.     "log-opts": {
  23.         "max-size": "100m"
  24.     },
  25.     "storage-driver": "overlay2"
  26. }
  27. EOF
  29. # 重启docker服务
  30. systemctl restart docker
  31. systemctl status docker
  33. # 验证docker服务是否正常
  34. docker images
  35. docker pull busybox
  36. docker run --rm busybox uname -m

安装kubernetes

  • 在所有节点安装kubernetes相关软件
  1. # 安装kubernetes相关组件
  2. curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add - 
  3. cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
  4. deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
  5. EOF
  6. apt-get update
  7. apt-get install -y kubelet kubeadm kubectl
  8. kubeadm version
  10. # 获取kubernetes所需要的镜像
  11. kubeadm config images list --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers
  12. export PAUSE_IMAGE=$(kubeadm config images list --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers | grep pause)
  14. # 注意pause镜像用于配置cri-dockerd的启动参数
  15. # 应该是输出 registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.9
  16. echo $PAUSE_IMAGE
  18. # 安装cri-dockerd，用于连接kubernetes和docker
  19. wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.4/cri-dockerd-0.3.4.amd64.tgz
  20. tar zxvf cri-dockerd-0.3.4.amd64.tgz 
  21. cd cri-dockerd/
  22. mkdir -p /usr/local/bin
  23. install -o root -g root -m 0755 cri-dockerd /usr/local/bin/cri-dockerd
  25. # 创建cri-docker.socket启动文件
  26. cat << EOF > /etc/systemd/system/cri-docker.socket
  27. [Unit]
  28. Description=CRI Docker Socket for the API
  29. PartOf=cri-docker.service
  31. [Socket]
  32. ListenStream=%t/cri-dockerd.sock
  33. SocketMode=0660
  34. SocketUser=root
  35. SocketGroup=docker
  37. [Install]
  38. WantedBy=sockets.target
  39. EOF
  41. # 创建cri-docker.service启动文件
  42. # 注意设置pause容器镜像信息 --pod-infra-container-image=$PAUSE_IMAGE
  43. cat << EOF > /etc/systemd/system/cri-docker.service
  44. [Unit]
  45. Description=CRI Interface for Docker Application Container Engine
  46. Documentation=https://docs.mirantis.com
  47. After=network-online.target firewalld.service docker.service
  48. Wants=network-online.target
  49. Requires=cri-docker.socket
  51. [Service]
  52. Type=notify
  53. ExecStart=/usr/local/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-image=$PAUSE_IMAGE
  54. ExecReload=/bin/kill -s HUP \$MAINPID
  55. TimeoutSec=0
  56. RestartSec=2
  57. Restart=always
  59. # Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
  60. # Both the old, and new location are accepted by systemd 229 and up, so using the old location
  61. # to make them work for either version of systemd.
  62. StartLimitBurst=3
  64. # Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
  65. # Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
  66. # this option work for either version of systemd.
  67. StartLimitInterval=60s
  69. # Having non-zero Limit*s causes performance problems due to accounting overhead
  70. # in the kernel. We recommend using cgroups to do container-local accounting.
  71. LimitNOFILE=infinity
  72. LimitNPROC=infinity
  73. LimitCORE=infinity
  75. # Comment TasksMax if your systemd version does not support it.
  76. # Only systemd 226 and above support this option.
  77. TasksMax=infinity
  78. Delegate=yes
  79. KillMode=process
  81. [Install]
  82. WantedBy=multi-user.target
  83. EOF
  85. # 启动cri-dockerd
  86. systemctl daemon-reload
  87. systemctl enable --now cri-docker.socket
  88. systemctl restart cri-docker
  89. systemctl status cri-docker
  91. # 通过kubeadm预先拉取所需的容器镜像
  92. kubeadm config images pull --image-repository registry.cn-hangzhou.aliyuncs.com/google_containers --cri-socket unix:///var/run/cri-dockerd.sock
  93. docker images
  1. # 安装dorycli
  2. cd /root
  3. wget https://github.com/dory-engine/dorycli/releases/download/v1.7.0/dorycli-v1.7.0-linux-amd64.tgz
  4. tar zxvf dorycli-v1.7.0-linux-amd64.tgz
  5. chmod a+x dorycli
  6. mv dorycli /usr/bin/
  8. # 设置dorycli的自动完成，可以通过键盘TAB键自动补全子命令和参数
  9. dorycli completion bash -h
  10. source <(dorycli completion bash)
  11. dorycli completion bash > /etc/bash_completion.d/dorycli
  13. # 使用dorycli打印高可用负载均衡器配置信息，并保存到kubeadm-ha.yaml
  14. dorycli install ha print --language zh > kubeadm-ha.yaml
  16. # 根据实际情况修改kubeadm-ha.yaml的配置信息
  17. # 可以通过以下命令获取各个主机的网卡名字
  18. ip address
  20. # 本例子的配置如下，请根据实际情况修改配置
  21. cat kubeadm-ha.yaml
  22. # 需要安装的kubernetes的版本
  23. version: "v1.28.0"
  24. # kubernetes的镜像仓库设置，如果不设置，那么使用官方的默认镜像仓库
  25. imageRepository: "registry.cn-hangzhou.aliyuncs.com/google_containers"
  26. # keepalived镜像
  27. keepalivedImage: "osixia/keepalived:release-2.1.5-dev"
  28. # nginx-lb镜像
  29. nginxlbImage: "nginx:1.27.0-alpine"
  30. # 使用keepalived创建的高可用kubernetes集群的浮动ip地址
  31. virtualIp: 192.168.0.100
  32. # 使用nginx映射的高可用kubernetes集群的apiserver映射端口
  33. virtualPort: 16443
  34. # 浮动ip地址映射的主机名，请在/etc/hosts配置文件中进行主机名映射设置
  35. virtualHostname: k8s-vip
  36. # kubernetes的容器运行时socket
  37. # docker情况下: unix:///var/run/cri-dockerd.sock
  38. # containerd情况下: unix:///var/run/containerd/containerd.sock
  39. # cri-o情况下: unix:///var/run/crio/crio.sock
  40. criSocket: unix:///var/run/cri-dockerd.sock
  41. # kubernetes集群的pod子网地址，如果不设置，使用默认的pod子网地址
  42. podSubnet: "10.244.0.0/24"
  43. # kubernetes集群的service子网地址，如果不设置，使用默认的service子网地址
  44. serviceSubnet: "10.96.0.0/16"
  45. # keepalived的鉴权密码，如果不设置那么使用随机生成的密码
  46. keepAlivedAuthPass: "input_your_password"
  47. # keepalived的virtual_router_id设置
  48. keepAlivedVirtualRouterId: 101
  49. # kubernetes的controlplane控制平面的主机配置，高可用master节点数量必须为单数并且至少3台
  50. masterHosts:
  51.     # master节点的主机名，请在/etc/hosts配置文件中进行主机名映射设置
  52.   - hostname: k8s-master01
  53.     # master节点的IP地址
  54.     ipAddress: 192.168.0.101
  55.     # master节点互访使用的网卡名字，用于keepalived网卡绑定
  56.     networkInterface: eth0
  57.     # keepalived选举优先级，数值越大优先级越高，各个master节点的优先级不能一样
  58.     keepalivedPriority: 120
  59.     # master节点的主机名，请在/etc/hosts配置文件中进行主机名映射设置
  60.   - hostname: k8s-master02
  61.     # master节点的IP地址
  62.     ipAddress: 192.168.0.102
  63.     # master节点互访使用的网卡名字，用于keepalived网卡绑定
  64.     networkInterface: eth0
  65.     # keepalived选举优先级，数值越大优先级越高，各个master节点的优先级不能一样
  66.     keepalivedPriority: 110
  67.     # master节点的主机名，请在/etc/hosts配置文件中进行主机名映射设置
  68.   - hostname: k8s-master03
  69.     # master节点的IP地址
  70.     ipAddress: 192.168.0.103
  71.     # master节点互访使用的网卡名字，用于keepalived网卡绑定
  72.     networkInterface: eth0
  73.     # keepalived选举优先级，数值越大优先级越高，各个master节点的优先级不能一样
  74.     keepalivedPriority: 100
  76. # 通过dorycli创建可用负载均衡器配置信息，并且把生成的配置输出到当前目录
  77. # 执行命名后，会输出生成的文件说明，以及启动配置文件说明
  78. dorycli install ha script -o . -f kubeadm-ha.yaml --language zh
  80. # 查看dorycli生成的kubeadm-config.yaml配置文件，该配置文件用于kubeadm init初始化kubernetes集群用途
  81. # 本例子生成的配置如下:
  82. cat kubeadm-config.yaml
  83. ---
  84. apiVersion: kubeadm.k8s.io/v1beta3
  85. kind: ClusterConfiguration
  86. kubernetesVersion: v1.28.0
  87. imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
  88. apiServer:
  89.   certSANs:
  90.     - "k8s-vip"
  91.     - "192.168.0.100"
  92.     - "k8s-master01"
  93.     - "192.168.0.101"
  94.     - "k8s-master02"
  95.     - "192.168.0.102"
  96.     - "k8s-master03"
  97.     - "192.168.0.103"
  98. controlPlaneEndpoint: "192.168.0.100:16443"
  99. networking:
  100.   podSubnet: "10.244.0.0/24"
  101.   serviceSubnet: "10.96.0.0/16"
  102. ---
  103. apiVersion: kubeadm.k8s.io/v1beta3
  104. kind: InitConfiguration
  105. nodeRegistration:
  106.   criSocket: unix:///var/run/cri-dockerd.sock
  108. # 设置master节点的kubernetes高可用负载均衡器的文件路径
  109. export LB_DIR=/data/k8s-lb
  111. # 把高可用负载均衡器的文件复制到k8s-master01
  112. ssh k8s-master01 mkdir -p ${LB_DIR}
  113. scp -r k8s-master01/nginx-lb k8s-master01/keepalived root@k8s-master01:${LB_DIR}
  115. # 在 k8s-master01 节点上启动高可用负载均衡器
  116. ssh k8s-master01 "cd ${LB_DIR}/keepalived/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  117. ssh k8s-master01 "cd ${LB_DIR}/nginx-lb/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  119. # 把高可用负载均衡器的文件复制到k8s-master02
  120. ssh k8s-master02 mkdir -p ${LB_DIR}
  121. scp -r k8s-master02/nginx-lb k8s-master02/keepalived root@k8s-master02:${LB_DIR}
  123. # 在 k8s-master02 节点上启动高可用负载均衡器
  124. ssh k8s-master02 "cd ${LB_DIR}/keepalived/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  125. ssh k8s-master02 "cd ${LB_DIR}/nginx-lb/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  127. # 把高可用负载均衡器的文件复制到k8s-master03
  128. ssh k8s-master03 mkdir -p ${LB_DIR}
  129. scp -r k8s-master03/nginx-lb k8s-master03/keepalived root@k8s-master03:${LB_DIR}
  131. # 在 k8s-master03 节点上启动高可用负载均衡器
  132. ssh k8s-master03 "cd ${LB_DIR}/keepalived/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  133. ssh k8s-master03 "cd ${LB_DIR}/nginx-lb/ && docker-compose stop && docker-compose rm -f && docker-compose up -d"
  135. # 在各个master节点上检验浮动IP是否已经创建，正常情况下浮动IP绑定在 k8s-master01 上
  136. ip address
  • 初始化高可用kubernetes集群
  1. # 在k8s-master01上使用kubeadm-config.yaml配置文件初始化高可用集群
  2. kubeadm init --config=kubeadm-config.yaml --upload-certs
  3. # kubeadm init命令将会输出以下提示，使用该提示在其他master节点执行join操作
  4. You can now join any number of the control-plane node running the following command on each as root:
  6.   kubeadm join 192.168.0.100:16443 --token tgszyf.c9dicrflqy85juaf \
  7.     --discovery-token-ca-cert-hash sha256:xxx \
  8.     --control-plane --certificate-key xxx
  10. Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
  11. As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
  12. "kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
  14. Then you can join any number of worker nodes by running the following on each as root:
  16. kubeadm join 192.168.0.100:16443 --token tgszyf.c9dicrflqy85juaf \
  17.     --discovery-token-ca-cert-hash sha256:xxx 
  20.   kubeadm join 192.168.0.100:16443 --token tgszyf.c9dicrflqy85juaf \
  21.     --discovery-token-ca-cert-hash sha256:xxx \
  22.     --control-plane --certificate-key xxx
  24. # 在k8s-master02 和 k8s-master03节点上执行以下命令，把k8s-master02 和 k8s-master03加入到高可用kubernetes集群
  25. # 记住kubeadm join命令需要设置--cri-socket unix:///var/run/cri-dockerd.sock
  26. kubeadm join 192.168.0.100:16443 --token tgszyf.c9dicrflqy85juaf \
  27.         --discovery-token-ca-cert-hash sha256:xxx \
  28.         --control-plane --certificate-key xxx --cri-socket unix:///var/run/cri-dockerd.sock
  30. # 在所有master节点上设置kubectl访问kubernetes集群
  31. mkdir -p $HOME/.kube
  32. cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  33. chown $(id -u):$(id -g) $HOME/.kube/config
  35. # 在所有master节点上设置kubectl的自动完成，可以通过键盘TAB键自动补全子命令和参数
  36. kubectl completion -h
  37. kubectl completion bash > ~/.kube/completion.bash.inc
  38. printf "
  39. # Kubectl shell completion
  40. source '$HOME/.kube/completion.bash.inc'
  41. " >> $HOME/.bash_profile
  42. source $HOME/.bash_profile
  44. # 在k8s-master01节点上安装cilium网络组件
  45. wget https://github.com/cilium/cilium-cli/releases/download/v0.15.6/cilium-linux-amd64.tar.gz
  46. tar zxvf cilium-linux-amd64.tar.gz 
  47. mv cilium /usr/local/bin/
  48. cilium install --version 1.14.0 --set cni.chainingMode=portmap
  50. # 设置所有master允许调度pod
  51. kubectl taint nodes --all node-role.kubernetes.io/control-plane-
  53. # 检查所有pod状态是否正常
  54. kubectl get pods -A -o wide
  55. NAMESPACE              NAME                                         READY   STATUS    RESTARTS      AGE     IP              NODE           NOMINATED NODE   READINESS GATES
  56. kube-system            cilium-mwvsr                                 1/1     Running   0             21m     192.168.0.102   k8s-master02   <none>           <none>
  57. kube-system            cilium-operator-b4dfbf784-zgr7v              1/1     Running   0             21m     192.168.0.102   k8s-master02   <none>           <none>
  58. kube-system            cilium-v27l2                                 1/1     Running   0             21m     192.168.0.103   k8s-master03   <none>           <none>
  59. kube-system            cilium-zbcdj                                 1/1     Running   0             21m     192.168.0.101   k8s-master01   <none>           <none>
  60. kube-system            coredns-6554b8b87f-kp7tn                     1/1     Running   0             30m     10.0.2.231      k8s-master03   <none>           <none>
  61. kube-system            coredns-6554b8b87f-zlhgx                     1/1     Running   0             30m     10.0.2.197      k8s-master03   <none>           <none>
  62. kube-system            etcd-k8s-master01                            1/1     Running   0             30m     192.168.0.101   k8s-master01   <none>           <none>
  63. kube-system            etcd-k8s-master02                            1/1     Running   0             26m     192.168.0.102   k8s-master02   <none>           <none>
  64. kube-system            etcd-k8s-master03                            1/1     Running   0             25m     192.168.0.103   k8s-master03   <none>           <none>
  65. kube-system            kube-apiserver-k8s-master01                  1/1     Running   0             30m     192.168.0.101   k8s-master01   <none>           <none>
  66. kube-system            kube-apiserver-k8s-master02                  1/1     Running   0             26m     192.168.0.102   k8s-master02   <none>           <none>
  67. kube-system            kube-apiserver-k8s-master03                  1/1     Running   1 (25m ago)   25m     192.168.0.103   k8s-master03   <none>           <none>
  68. kube-system            kube-controller-manager-k8s-master01         1/1     Running   1 (26m ago)   30m     192.168.0.101   k8s-master01   <none>           <none>
  69. kube-system            kube-controller-manager-k8s-master02         1/1     Running   0             26m     192.168.0.102   k8s-master02   <none>           <none>
  70. kube-system            kube-controller-manager-k8s-master03         1/1     Running   0             24m     192.168.0.103   k8s-master03   <none>           <none>
  71. kube-system            kube-proxy-gr2pt                             1/1     Running   0             26m     192.168.0.102   k8s-master02   <none>           <none>
  72. kube-system            kube-proxy-rkb9b                             1/1     Running   0             30m     192.168.0.101   k8s-master01   <none>           <none>
  73. kube-system            kube-proxy-rvmv4                             1/1     Running   0             25m     192.168.0.103   k8s-master03   <none>           <none>
  74. kube-system            kube-scheduler-k8s-master01                  1/1     Running   1 (26m ago)   30m     192.168.0.101   k8s-master01   <none>           <none>
  75. kube-system            kube-scheduler-k8s-master02                  1/1     Running   0             26m     192.168.0.102   k8s-master02   <none>           <none>
  76. kube-system            kube-scheduler-k8s-master03                  1/1     Running   0             23m     192.168.0.103   k8s-master03   <none>           <none>
  78. # 检查所有节点状态是否正常
  79. kubectl get nodes
  80. NAME           STATUS   ROLES           AGE   VERSION
  81. k8s-master01   Ready    control-plane   31m   v1.28.0
  82. k8s-master02   Ready    control-plane   27m   v1.28.0
  83. k8s-master03   Ready    control-plane   26m   v1.28.0
  85. # 测试部署应用到kubernetes集群
  86. # 部署一个nginx应用，并暴露到nodePort31000
  87. kubectl run nginx --image=nginx:1.23.1-alpine --image-pull-policy=IfNotPresent --port=80 -l=app=nginx
  88. kubectl create service nodeport nginx --tcp=80:80 --node-port=31000
  89. curl k8s-vip:31000

[可选] 安装管理界面 kubernetes-dashboard

调整kubernetes-dashboard服务使用nodePort暴露端口

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:

  • port: 443
    protocol: TCP
    targetPort: 8443
    nodePort: 30000
    selector:
    k8s-app: kubernetes-dashboard
    type: NodePort
    EOF

创建管理员serviceaccount

kubectl create serviceaccount -n kube-system admin-user —dry-run=client -o yaml | kubectl apply -f -

创建管理员clusterrolebinding

kubectl create clusterrolebinding admin-user —clusterrole=cluster-admin —serviceaccount=kube-system:admin-user —dry-run=client -o yaml | kubectl apply -f -

手动创建serviceaccount的secret

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
name: admin-user-secret
namespace: kube-system
annotations:
kubernetes.io/service-account.name: admin-user
type: kubernetes.io/service-account-token
EOF

获取kubernetes管理token

kubectl -n kube-system get secret admin-user-secret -o jsonpath=’{ .data.token }’ | base64 -d

使用浏览器访问kubernetes-dashboard: https://k8s-vip:30000

使用kubernetes管理token登录kubernetes-dashboard

  2. ## [可选] 安装ingress控制器 traefik
  4. - 要使用kubernetes的[ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/)功能，必须安装ingress controller，推荐使用`traefik`
  5. - 要了解更多，请阅读官方网站文档: [traefik](https://doc.traefik.io/traefik/)
  7. - 在kubernetes所有master节点部署traefik: 
  8. ```shell script
  9. # 拉取 traefik helm repo
  10. helm repo add traefik https://traefik.github.io/charts
  11. helm fetch traefik/traefik --untar
  13. # 以daemonset方式部署traefik
  14. cat << EOF > traefik.yaml
  15. deployment:
  16.   kind: DaemonSet
  17. image:
  18.   name: traefik
  19.   tag: v2.10.5
  20. ports:
  21.   web:
  22.     hostPort: 80
  23.   websecure:
  24.     hostPort: 443
  25. service:
  26.   type: ClusterIP
  27. EOF
  29. # 安装traefik
  30. kubectl create namespace traefik --dry-run=client -o yaml | kubectl apply -f -
  31. helm install -n traefik traefik traefik/ -f traefik.yaml
  33. # 检查安装情况
  34. helm -n traefik list
  35. kubectl -n traefik get pods -o wide
  36. kubectl -n traefik get services -o wide
  38. # 检验traefik安装是否成功，如果输出 404 page not found 表示成功
  39. curl k8s-vip
  40. curl -k https://k8s-vip

[可选] 安装性能数据采集工具 metrics-server

```shell script

拉取镜像

docker pull registry.aliyuncs.com/google_containers/metrics-server:v0.6.1
docker tag registry.aliyuncs.com/google_containers/metrics-server:v0.6.1 k8s.gcr.io/metrics-server/metrics-server:v0.6.1

获取metrics-server安装yaml

curl -O -L https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.6.1/components.yaml

添加—kubelet-insecure-tls参数

sed -i ‘s/- args:/- args:\n - —kubelet-insecure-tls/g’ components.yaml

安装metrics-server

kubectl apply -f components.yaml

等待metrics-server正常

kubectl -n kube-system get pods -l=k8s-app=metrics-server

查看节点的metrics

kubectl top nodes
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
k8s-master01 146m 7% 2284Mi 59%
k8s-master02 123m 6% 2283Mi 59%
k8s-master03 114m 5% 2180Mi 57% 

  2. - 安装metrics-server后kubernetes-dashboard也可以显示性能数据
  4. ![](images/kubernetes-dashboard.png)
  6. ## [可选] 安装服务网格 istio
  8. - 要使用服务网格的混合灰度发布能力，需要部署istio服务网格
  9. - 要了解更多，请阅读istio官网文档: [istio.io](https://istio.io/latest/docs/)
  11. ```shell script
  12. # 安装istioctl，客户端下载地址 https://github.com/istio/istio/releases/tag/1.18.2
  14. # 下载并安装istioctl
  15. wget https://github.com/istio/istio/releases/download/1.18.2/istioctl-1.18.2-linux-amd64.tar.gz
  16. tar zxvf istioctl-1.18.2-linux-amd64.tar.gz
  17. mv istioctl /usr/bin/
  19. # 确认istioctl版本
  20. istioctl version
  22. # 使用istioctl部署istio到kubernetes
  23. istioctl install --set profile=demo \
  24. --set values.gateways.istio-ingressgateway.type=ClusterIP \
  25. --set values.global.imagePullPolicy=IfNotPresent \
  26. --set values.global.proxy_init.resources.limits.cpu=100m \
  27. --set values.global.proxy_init.resources.limits.memory=100Mi \
  28. --set values.global.proxy.resources.limits.cpu=100m \
  29. --set values.global.proxy.resources.limits.memory=100Mi
  31. # 检查istio部署情况
  32. kubectl -n istio-system get pods,svc

[可选] 非常简单的开源k8s远程开发环境 Dory-Engine

🚀🚀🚀 使用k8s快速搭建远程开发环境 (https://www.bilibili.com/video/BV1Zw4m1r7aw/)

  • Dory-Engine 非常简单的开源k8s远程开发环境，开发人员不用学、不用写、不用配就可以自行把自己编写的程序从源代码，编译、打包、部署到各类k8s环境中。
  1. 不用学: 不用学习复杂的k8s技术原理，5分钟即可快速上手部署应用
  2. 不用配: 不需要配置任何代码仓库、镜像仓库和k8s连接参数
  3. 不用写: 不需要编写任何k8s部署清单和流水线脚本