k8s 集群环境搭建

发布时间：2023-06-13 13:49:09

本教程通过虚拟机进行环境搭建虚拟机联网配置虚拟机设置静态IP 设置网卡连接方式准备两台虚拟机机器 IPhostname192.168.243.1

本教程通过虚拟机进行环境搭建

虚拟机联网配置

虚拟机设置静态IP

设置网卡连接方式
准备两台虚拟机

机器 IP	hostname
192.168.243.134	k8s-master
192.168.243.136	k8s-node1

配置静态IP (master 和 node 都要配置)

vi /etc/sysconfig/network-scripts/ifcfg-ens33nnTYPE="Ethernet"nPROXY_METHOD="none"nBROWSER_ONLY="no"nBOOTPROTO="static"nDEFROUTE="yes"nIPV4_FAILURE_FATAL="no"nIPV6INIT="yes"nIPV6_AUTOCONF="yes"nIPV6_DEFROUTE="yes"nIPV6_FAILURE_FATAL="no"nIPV6_ADDR_GEN_MODE="stable-privacy"nNAME="ens33"nUUID="d00801e4-2486-4c94-9402-018fdb60fc77"nDEVICE="ens33"nONBOOT="yes"nnn#### 以下是静态ip配置时新增内容nIPADDR="192.168.243.134" # 此处给自己设置一个静态ip, master: 192.168.243.134 node1: 192.168.243.136. 根据自己的实际ip进行填写nNETMASK="255.255.255.0" nGATEWAY="192.168.243.1" # 网关配置.没有什么特殊需求,前三位和 IPADDR 前三位保持一致. 最后一位使用 1. 如: 192.168.243.1nDNS1="223.5.5.5" # 固定不变nn####### 使用命令重启网络nservice network restart

关闭防火墙 (master 和 node 都要执行)

systemctl stop firewalldnsystemctl disable firewalld

关闭selinux (master 和 node 都要执行)

setenforce 0 # 临时关闭nsed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config # 永久关闭

关闭swap(master/node)

swapoff -a # 临时关闭；关闭swap主要是为了性能考虑nfree # 可以通过这个命令查看swap是否关闭了nsed -ri 's/.*swap.*/#&/' /etc/fstab # 永久关闭

修改host文件(master 和 node 都要执行)

vi /etc/hostsnn192.168.243.134 master.com master # 主机 hostn192.168.243.136 node1.com node1 # node hostn199.232.28.133 raw.githubusercontent.com # 后面的步骤会在这个网站进行文件下载,如果无法下载文件,可以在 host 文件中添加这个地址

修改主机名(master 和 node 都要执行)

主机名配置

master 调整

hostnamectl set-hostname master ##重启后永久生效

node1 调整

hostnamectl set-hostname node1 ##重启后永久生效

桥接设置(master 和 node 都要执行)

cat > /etc/sysctl.d/k8s.conf << EOFnnet.bridge.bridge-nf-call-ip6tables = 1nnet.bridge.bridge-nf-call-iptables = 1nEOFnnnsysctl --system

添加阿里云源 (master 和 node 都要执行)

rm -rf /etc/yum.repos.d/*ncurl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

安装常用包 (master 和 node 都要执行)

yum install vim bash-completion net-tools gcc -y

安装 docker (master 和 node 都要执行)

yum install -y yum-utils device-mapper-persistent-data lvm2nnyum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.reponnyum -y install docker-ce

添加aliyundocker仓库加速器(master 和 node 都要执行)

mkdir -p /etc/dockernntee /etc/docker/daemon.json <<-'EOF'n{n "registry-mirrors": ["https://fl791z1h.mirror.aliyuncs.com"]n}nEOFnnsystemctl daemon-reloadnnsystemctl restart docker

安装kubectl、kubelet、kubeadm(master 和 node 都要执行)

添加阿里kubernetes源(master 和 node 都要执行)

cat <<EOF > /etc/yum.repos.d/kubernetes.repon[kubernetes]nname=Kubernetesnbaseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/nenabled=1ngpgcheck=1nrepo_gpgcheck=1ngpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpgnEOF

安装 kubectl、kubelet、kubeadm (master 和 node 都要执行)

yum install kubectl kubelet kubeadmnn#### 此时，还不能启动kubelet，因为此时配置还不能.nsystemctl enable kubelet

初始化k8s集群(仅 master 需要安装)

查看 kubeadm 版本

[root@localhost ~]# kubeadm versionnkubeadm version: &version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:25:59Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}nn## 当前版本: GitVersion:"v1.20.2"

初始化集群

kubeadm init --kubernetes-version=1.20.2 n--apiserver-advertise-address=192.168.243.134 n--image-repository registry.aliyuncs.com/google_containers n--service-cidr=10.10.0.0/16 --pod-network-cidr=10.122.0.0/16

这里注意有两个参数是需要调整的 --kubernetes-version: 引用 kubeadm 的版本号 --apiserver-advertise-address: 需要替换为 master ip 地址

创建kubectl(仅 master 需要安装)

mkdir -p $HOME/.kubennsudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/confignnsudo chown $(id -u):$(id -g) $HOME/.kube/config

查看节点 pod

[root@localhost ~]# kubectl get nodenNAME STATUS ROLES AGE VERSIONnlocalhost.localdomain NotReady control-plane,master 139m v1.20.2nnn[root@localhost ~]# kubectl get pod --all-namespacesnNAMESPACE NAME READY STATUS RESTARTS AGEnkube-system coredns-7f89b7bc75-4cvgf 0/1 Pending 0 2mnkube-system coredns-7f89b7bc75-nfdvg 0/1 Pending 0 2mnkube-system etcd-master 1/1 Running 0 2m10snkube-system kube-apiserver-master 1/1 Running 0 2m10snkube-system kube-controller-manager-master 1/1 Running 0 2m10snkube-system kube-proxy-hk47n 1/1 Running 0 2mnkube-system kube-scheduler-master 1/1 Running 0 2m10s

node节点为NotReady，因为corednspod没有启动，缺少网络pod

安装calico网络(仅 master 需要安装)

[root@localhost ~]# kubectl apply -f https://docs.projectcalico.org/manifests/calico.yamlnn#### 执行结果nconfigmap/calico-config createdncustomresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org createdncustomresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org creatednclusterrole.rbac.authorization.k8s.io/calico-kube-controllers creatednclusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers creatednclusterrole.rbac.authorization.k8s.io/calico-node creatednclusterrolebinding.rbac.authorization.k8s.io/calico-node createdndaemonset.apps/calico-node creatednserviceaccount/calico-node createdndeployment.apps/calico-kube-controllers creatednserviceaccount/calico-kube-controllers creatednpoddisruptionbudget.policy/calico-kube-controllers created

查看pod和node

命令可以等一会执行,因为部分服务在启动,状态不会立马变更为 Running 状态

[root@localhost ~]# kubectl get pod --all-namespacesnn#### 执行结果nNAMESPACE NAME READY STATUS RESTARTS AGEnkube-system calico-kube-controllers-744cfdf676-djfcb 1/1 Running 0 135mnkube-system calico-node-r8g7m 1/1 Running 0 135mnkube-system coredns-7f89b7bc75-2c8c4 1/1 Running 0 142mnkube-system coredns-7f89b7bc75-zl49d 1/1 Running 0 142mnkube-system etcd-localhost.localdomain 1/1 Running 0 142mnkube-system kube-apiserver-localhost.localdomain 1/1 Running 0 142mnkube-system kube-controller-manager-localhost.localdomain 1/1 Running 0 142mnkube-system kube-proxy-lvwhk 1/1 Running 0 142mnkube-system kube-scheduler-localhost.localdomain 1/1 Running 0 142mnkubernetes-dashboard dashboard-metrics-scraper-79c5968bdc-hdzlm 1/1 Running 0 100mnkubernetes-dashboard kubernetes-dashboard-7448ffc97b-d2q5v 1/1 Running 0 100m

安装kubernetes-dashboard(仅 master 需要安装)

官方部署dashboard的服务没使用nodeport，将yaml文件下载到本地，在service里添加nodeport

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc7/aio/deploy/recommended.yaml

如: dashboard 界面报错

namespaces is forbidden: User "system:serviceaccount:kubernetes-dashboard:kubernetes-dashboard" cannot list resource "namespaces" in API group "" at the cluster scope

原因: 发现是dashboard的版本和kubernetes的版本不一致

解决方案: 从 https://github.com/kubernetes/dashboard/releases 找到对应版本的 dashboard 的 yaml 重新部署, 即可解决

如果访问失败: 在 hosts 文件中添加 199.232.28.133 raw.githubusercontent.com

备用下载地址

编辑 recommended.yaml 文件

vim recommended.yamlnnkind: ServicenapiVersion: v1nmetadata:n labels:n k8s-app: kubernetes-dashboardn name: kubernetes-dashboardn namespace: kubernetes-dashboardnspec:n type: NodePortn ports:n - port: 443n targetPort: 8443n nodePort: 30000n selector:n k8s-app: kubernetes-dashboard

找到 kind: service 的配置

- 创建 dashboard

kubectl create -f recommended.yaml

通过 token 方式登录 k8s dashboard

获取 token

### 创建 service accountnkubectl create sa dashboard-admin -n kube-systemnn### 创建角色绑定关系nkubectl create clusterrolebinding dashboard-admin --clusterrole=cluster-admin --serviceaccount=kube-system:dashboard-adminnn### 查看 dashboard-admin 的 secret 名字nADMIN_SECRET=$(kubectl get secrets -n kube-system | grep dashboard-admin | awk '{print $1}')nn### 打印 secret 的tokennkubectl describe secret -n kube-system ${ADMIN_SECRET} | grep -E '^token' | awk '{print $2}'

进入 dashboard 页面

浏览器输入master主机地址 + 30000,使用https协议如: https://192.168.243.134:30000

dashboard 登录页

node 加入到集群

在 master 主机上生成 token

默认token的有效期为24小时，当过期之后，该token就不可用了，在master节点上执行 kubeadm token create

创建token (仅 master 需要执行)

这个 token 可以自动设定,需要按照规则生成token

## token 规则 A([a-z0-9]{6}).([a-z0-9]{16})znkubeadm token create token1.tokentokentoken1

查看 token(仅 master 需要执行)

kubeadm token listnn### 执行结果nTOKEN TTL EXPIRES USAGES DESCRIPTION ntoken1.tokentokentoken1 23h 2021-01-30T17:33:23+08:00 authentication,signing <none> system:bootstrappers:kubeadm:default-node-token

获取ca证书sha256编码hash值(仅 master 需要执行)

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'nn### 执行结果n0654fa65a6a2b7fe09cb605f24809e7fe61cdc910d7b2b74165c6c8843c197c7

节点加入集群(仅 node 需要执行)

### 清理环境nkubeadm reset nn### 链接集群nkubeadm join 192.168.243.134:6443 --token token1.tokentokentoken1 n--discovery-token-ca-cert-hash sha256:0654fa65a6a2b7fe09cb605f24809e7fe61cdc910d7b2b74165c6c8843c197c7