努力挣扎的生活
yangfk
2022-12-20

kubeadm证书替换

//

kubeadm创建的集群内部证书默认过期时间大多为一年，为避免后期维护更换证书，在集群创建好之后，先把证书重新更新更长过期时间

kubeadm_1.jpg

certificates官方说明 (opens new window)

官方文档PKI证书 (opens new window)

# 替换主控服务的certs

手动执行替换命令：kubeadm certs renew all，证书过期时间为1年，默认开启自动轮换机制，

警告： kubeadm 不能管理由外部 CA 签名的证书

# 查看kubelet证书

cd /etc/kubernetes/pki
openssl x509 -in ./ca.crt -noout -text |grep Not
1
2

建议在node节点kubelet开启csr，将允许启动引导 kubelet 的服务证书 ：vi /var/lib/kubelet/config.yaml

添加行：serverTLSBootstrap: true

重启kubelet：systemctl restart kubelet.service

# 报错Error from server: error dialing backend: remote error: tls: internal error

kubeadm_1.jpg

kubectl exec -it nginx-dp-69df87f76d-6vrzm bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Error from server: error dialing backend: remote error: tls: internal error
  • 修复命令，原因未知
for i in `kubectl get csr|awk '{print $1}'`;do kubectl certificate approve $i;done
1

# kubeadm自签证书-1.26

参考前面的部署方式

手动签发的证书存放目录 /opt/self-signed/certs

cd /opt
wget http://download.yfklife.cn/blog/cloud/k8s/certs/cfssl -O ./cfssl
wget http://download.yfklife.cn/blog/cloud/k8s/certs/cfssl-json -O ./cfssl-json

chmod +x ./cfssl*
mv ./cfssl* /usr/local/bin/

mkdir -p /opt/self-signed/certs /opt/self-signed/pki/etcd
1
2
3
4
5
6
7
8
9

# ca 签发承载式证书

  • ca-csr
cd /opt/self-signed/certs

cat >ca-csr.json<<EOF
{
    "CN": "kubernetes",
    "hosts": [	
    ],
    "key": {			
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "kubernetes",
            "OU": "kubernetes"
        }
    ],
    "ca": {
        "expiry": "175200h"
    }
}
EOF
cfssl gencert -initca ca-csr.json | cfssl-json -bare ca
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
  • ca-config.json

证书的过期时间20年

cat >ca-config.json<<EOF
{
    "signing": {
        "default": {
            "expiry": "175200h"
        },
        "profiles": {
            "server": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth"
                ]
            },
            "client": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "peer": {
                "expiry": "175200h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}

EOF
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

# 签发etcd 相关证书

使用kubeadm创建一个高可用etcd集群-官方文档 (opens new window)

多预留个IP

cat >etcd-peer-csr.json<<EOF
{
    "CN": "k8s-etcd",
    "hosts": [
        "192.168.255.20",
        "192.168.255.30",
        "192.168.255.31",
        "192.168.255.32",
        "192.168.255.33"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "system:masters",
            "OU": "kubernetes"
        }
    ]
}
EOF

#生成证书对

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare peer
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare server
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare healthcheck-client
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=peer etcd-peer-csr.json |cfssl-json -bare apiserver-etcd-client
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# 签发kube-api-server相关证书

  • apiserver-client
cat >apiserver-client-csr.json<<EOF
{
    "CN": "kube-apiserver-kubelet-client",
    "hosts": [
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "system:masters",
            "OU": "kubernetes"
        }
    ]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client apiserver-client-csr.json |cfssl-json -bare client
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
  • api-server

注：control-plane-endpoint.yfklife.cn 是controlPlaneEndpoint地址，指向虚拟HA-VIP，也可以是部署的主控节点

cat >apiserver-csr.json<<EOF
{
    "CN": "kube-apiserver",
    "hosts": [
        "control-plane-endpoint.yfklife.cn",
        "127.0.0.1",
        "10.96.0.1",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local",
        "192.168.255.20",
        "192.168.255.30",
        "192.168.255.31",
        "192.168.255.32"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "beijing",
            "L": "beijing",
            "O": "kubernetes",
            "OU": "kubernetes"
        }
    ]
}
EOF
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server apiserver-csr.json |cfssl-json -bare apiserver
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# 签发front-proxy相关证书

  • front-proxy-ca
cat >front-proxy-ca-csr.json<<EOF
{
    "CN": "front-proxy-ca",
    "key": {			
        "algo": "rsa",
        "size": 2048
    },
    "ca": {
        "expiry": "175200h"
    }
}
EOF
cfssl gencert -initca front-proxy-ca-csr.json | cfssl-json -bare front-proxy-ca
1
2
3
4
5
6
7
8
9
10
11
12
13
  • front-proxy-ca-client
cat >front-proxy-client-csr.json<<EOF
{
  "CN": "front-proxy-client",
  "key": {
     "algo": "rsa",
     "size": 2048
  }
}
EOF
cfssl gencert   -ca=front-proxy-ca.pem   -ca-key=front-proxy-ca-key.pem   -config=ca-config.json   -profile=client   front-proxy-client-csr.json | cfssl-json -bare front-proxy-client
1
2
3
4
5
6
7
8
9
10

# 签发ServiceAccount Key

cd /opt/self-signed
openssl genrsa -out pki/sa.key 2048
openssl rsa -in pki/sa.key -pubout -out pki/sa.pub
1
2
3

# 拷贝证书

与kubeadm证书目录对应文件名

cd /opt/self-signed

#etcd 证书
cp certs/ca.pem pki/etcd/ca.crt
cp certs/ca-key.pem pki/etcd/ca.key
cp certs/peer.pem pki/etcd/peer.crt
cp certs/peer-key.pem pki/etcd/peer.key
cp certs/healthcheck-client.pem pki/etcd/healthcheck-client.crt
cp certs/healthcheck-client-key.pem pki/etcd/healthcheck-client.key
cp certs/server.pem pki/etcd/server.crt
cp certs/server-key.pem pki/etcd/server.key


#kube-system 证书
cp ./certs/ca.pem pki/ca.crt
cp ./certs/ca-key.pem pki/ca.key
cp ./certs/client.pem pki/apiserver-etcd-client.crt
cp ./certs/client-key.pem pki/apiserver-etcd-client.key
cp ./certs/apiserver.pem pki/apiserver.crt
cp ./certs/apiserver-key.pem pki/apiserver.key
#cp ./certs/ca.pem pki/front-proxy-ca.crt
cp ./certs/client.pem pki/apiserver-kubelet-client.crt
cp ./certs/client-key.pem pki/apiserver-kubelet-client.key
cp ./certs/front-proxy-ca.pem  pki/front-proxy-ca.crt
cp ./certs/front-proxy-ca-key.pem  pki/front-proxy-ca.key
cp ./certs/front-proxy-client.pem  pki/front-proxy-client.crt
cp ./certs/front-proxy-client-key.pem  pki/front-proxy-client.key


- 修改证书权限
chmod 600 pki/*.key pki/*.pub pki/etcd/*.key 


#拷贝证书,提示：需要提前安装好kubeadm,kubelet
cp -a /opt/self-signed/pki/* /etc/kubernetes/pki/
cp -a /opt/self-signed/pki/etcd /etc/kubernetes/etcd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# kubeadm更新kubernetes根证书(ca,etcd-ca,front-proxy-ca)

kubeadm_1.jpg

证书过期时间检查：kubeadm certs check-expiration

# 备份删除根证书

  • 备份配置文件和证书
mkdir -p /opt/backup-kubernetes
cp -a /etc/kubernetes/pki /opt/backup-kubernetes/
cp -a /var/lib/kubelet/pki /opt/backup-kubernetes/kubelet-pki
cp /etc/kubernetes/kubelet.conf /opt/backup-kubernetes/
cp /etc/kubernetes/admin.conf  /opt/backup-kubernetes/
1
2
3
4
5
  • 删除需要替换的根证书
rm -rf /etc/kubernetes/pki/ca.*
rm -rf /etc/kubernetes/pki/etcd/ca.*
rm -rf /etc/kubernetes/pki/front-proxy-ca.*
1
2
3

# 更新ca根证书

  • 生成新的 CA 证书,生成目录：/etc/kubernetes/pki

kubeadm init phase certs ca

  • 分发 CA 证书到其他节点：
scp /etc/kubernetes/pki/ca.* <node-ip>:/etc/kubernetes/pki/
1
2

# 更新etcd-ca根证书

  • 生成新的 etcd-ca 证书

kubeadm init phase certs etcd-ca

  • 分发 CA 证书到其他节点：
scp /etc/kubernetes/pki/etcd/ca.* <node-ip>:/etc/kubernetes/pki/etcd/
1
2

# 更新front-proxy-ca根证书

  • 生成新的 front-proxy-ca 证书：

kubeadm init phase certs front-proxy-ca

  • 分发 CA 证书到其他节点：
scp /etc/kubernetes/pki/front-proxy-ca.* <node-ip>:/etc/kubernetes/pki/
1
2

# kubelet证书替换

cp -a /etc/kubernetes/kubelet.conf{,_bak}
rm -rf /etc/kubernetes/kubelet.conf
rm -rf /var/lib/kubelet/pki/kubelet*
1
2
3

$NodeName替换为你当前节点名

kubeadm kubeconfig user --org system:nodes --client-name system:node:$NodeName > /etc/kubernetes/kubelet.conf
1

# 下发证书

kubeadm certs renew all

  • 重启kubelet

systemctl restart kubelet

重新生成 kubectl 访问集群权限：

ln -sf /etc/kubernetes/admin.conf ~/.kube/config

  • 重启k8s组件
crictl ps |grep -E 'kube-apiserver|kube-controller-manager|kube-scheduler|etcd' | awk -F ' ' '{print $1}' |xargs crictl stop
1

# 检查集群状态

kubeadm certs check-expiration
kubectl get nodes
kubectl get pods -A
1
2
3
4

借鉴博客 (opens new window)

官方文档参考 (opens new window)

//
如果此文章对您有帮助，点击 -->> 请博主喝咖啡 (opens new window)
上次更新: 2025/06/30, 17:43:00
kubeadm安装k8s-自签证书
Pod探针

← Pod探针→

最近更新
01
postgreSQL通过keepalived配置主从自动切换
 06-30
02
ubuntu2204编译安装php7.4.33
 06-30
03
nginx之resolver解析
 06-19
更多文章>
Theme by Vdoing | Copyright © 2019-2025 yangfk | 湘ICP备2021014415号-1
×
//