一, 单机
说明: 执行 etcd 备份数据的恢复的机器必须和原先 etcd 所在机器一致
1, 单机备份
- ETCDCTL_API=3 etcdctl --endpoints="https://10.25.72.62:2379" \
- --cert=/etc/etcd/ssl/etcd.pem \
- --key=/etc/etcd/ssl/etcd-key.pem \
- --cacert=/etc/kubernetes/ssl/ca.pem \
- snapshot save snapshot.db
2, 单机数据恢复
- [root@SZD-L0105331 ~]# systemctl stop etcd # 停止 etcd 服务
- [root@SZD-L0105331 ~]# export ETCDCTL_API=3; # 使用 ETCDCTL API 3
- [root@SZD-L0105331 ~]# etcdctl snapshot restore snapshot.db \
- --name=SZD-L0105331 \
- --endpoints=https://10.25.72.62:2379 \
- --cacert=/etc/kubernetes/ssl/ca.pem \
- --cert=/etc/etcd/ssl/etcd.pem \
- --key=/etc/etcd/ssl/etcd-key.pem \
- --initial-cluster=SZD-L0105331=https://10.25.72.62:2380 \
- --initial-advertise-peer-urls=https://10.25.72.62:2380 \
- --initial-cluster-token=etcd-cluster-0 \
- --data-dir=/var/lib/etcd4 # 注意 --data-dir 参数, 以下步骤需要使用
- [root@SZD-L0105331 ~]# # 修改 etcd 启动参数 --data-dir 指向上一步的数据回复目录, 一般在 / etc/etcd/etcd 文件中
- [root@SZD-L0105331 ~]# cat /etc/etcd/etcd # 完整参数文件如下
- ETCD_OPTIONS="--name=SZD-L0105331 \
- --client-cert-auth=true \
- --cert-file=/etc/etcd/ssl/etcd.pem \
- --key-file=/etc/etcd/ssl/etcd-key.pem \
- --peer-cert-file=/etc/etcd/ssl/etcd.pem \
- --peer-key-file=/etc/etcd/ssl/etcd-key.pem \
- --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
- --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
- --initial-advertise-peer-urls=https://10.25.72.62:2380 \
- --listen-peer-urls=https://10.25.72.62:2380 \
- --listen-client-urls=https://10.25.72.62:2379,https://127.0.0.1:2379 \
- --advertise-client-urls=https://10.25.72.62:2379 \
- --initial-cluster-token=etcd-cluster-0 \
- --initial-cluster=SZD-L0105331=https://10.25.72.62:2380 \
- --initial-cluster-state=new \
- --data-dir=/var/lib/etcd"
- [root@SZD-L0105331 ~]# systemctl start etcd # 启动 etcd 服务
二, 集群
1, 模拟写入数据到 Etcd 集群
- # 使用 API 3 写入数据库
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem put /name/1 zxg
- # 读取数据
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem get /name/1
- 2018-08-16 20:47:13.017586 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- /name/1
- zxg
- # 使用 API 2 写入数据库
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=2 etcdctl --cert-file /root/etcd-cluster/etcd.pem --ca-file /root/etcd-cluster/ca.pem --key-file /root/etcd-cluster/etcd-key.pem --endpoints="https://10.25.73.25:2379,https://10.25.73.150:2379,https://10.25.84.251:2379" set /name1 zxg1
- 2018-08-16 18:27:35.060559 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- zxg1
- # 读取数据
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=2 etcdctl --cert-file /root/etcd-cluster/etcd.pem --ca-file /root/etcd-cluster/ca.pem --key-file /root/etcd-cluster/etcd-key.pem --endpoints="https://10.25.73.25:2379,https://10.25.73.150:2379,https://10.25.84.251:2379" get /name1
- 2018-08-16 18:27:47.884908 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- zxg1
2, 备份 etcd 数据
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem snapshot save mysnapshot.db
- 2018-08-16 20:50:47.206334 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- Snapshot saved at mysnapshot.db
- [root@SZD-L0097856 etcd-cluster]#
3, 停止 etcd 集群
停止方法: 分别在 3 台 etcd 的宿主机上执行以下命令停止 etcd 服务
systemctl stop etcd
停掉 Leader 10.25.73.25, 查看集群状况, 重新选举出了 leader, 集群可正常使用
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem endpoint status --write-out=table
- 2018-08-16 18:31:50.686352 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- Failed to get the status of endpoint https://10.25.73.25:2379 (context deadline exceeded)
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
- | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
- | https://10.25.84.251:2379 | 2a53eb67dfa37cc0 | 3.1.10 | 25 kB | false | 14 | 19 |
- | https://10.25.73.150:2379 | c267a0ca02c6bff7 | 3.1.10 | 25 kB | true | 14 | 19 |
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
- [root@SZD-L0097856 etcd-cluster]#
停掉 10.25.73.150, 查看集群状况, 集群已经无法正常使用, 说明 3 节点的 Etcd 容错为 1
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem endpoint status --write-out=table
- 2018-08-16 18:33:44.526382 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- Failed to get the status of endpoint https://10.25.73.25:2379 (context deadline exceeded)
- Failed to get the status of endpoint https://10.25.73.150:2379 (context deadline exceeded)
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
- | ENDPOINT | ID | VERSION | DB SIZE | IS LEADER | RAFT TERM | RAFT INDEX |
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
- | https://10.25.84.251:2379 | 2a53eb67dfa37cc0 | 3.1.10 | 25 kB | false | 14 | 19 |
- +---------------------------+------------------+---------+---------+-----------+-----------+------------+
删除 etcd 数据, 方法: 登录 etcd 所在主机执行:(注意: 危险操作, 请谨慎操作, 确保在有数据备份并且确定 Etcd 集群无法正常工作后操作)
rm -rf /var/lib/etcd
4, 使用备份数据进行恢复
恢复 10.25.84.251 节点数据到 / var/lib/etcd
- [root@SZD-L0097856 etcd-cluster]# ETCDCTL_API=3 etcdctl --name=SZD-L0097856 --endpoints="https://10.25.84.251:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem --initial-cluster-token=etcd-cluster-0 --initial-advertise-peer-urls=https://10.25.84.251:2380 --initial-cluster=SZD-L0101798=https://10.25.73.25:2380,SZD-L0103739=https://10.25.73.150:2380,SZD-L0097856=https://10.25.84.251:2380 --data-dir=/varlib/etcd snapshot restore mysnapshot.db
- 2018-08-16 19:52:33.409406 I | etcdserver/membership: added member 2a53eb67dfa37cc0 [https://10.25.84.251:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:52:33.409481 I | etcdserver/membership: added member 304bcbfa92e84c75 [https://10.25.73.25:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:52:33.409499 I | etcdserver/membership: added member c267a0ca02c6bff7 [https://10.25.73.150:2380] to cluster 4741a9e2cf17e1fa
- [root@SZD-L0097856 etcd-cluster]#
恢复 10.25.73.25 节点数据到 / var/lib/etcd
- [root@SZD-L0101798 etcd-cluster]# ETCDCTL_API=3 etcdctl --name=SZD-L0101798 --endpoints="https://10.25.73.25:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem --initial-cluster-token=etcd-cluster-0 --initial-advertise-peer-urls=https://10.25.73.25:2380 --initial-cluster=SZD-L0101798=https://10.25.73.25:2380,SZD-L0103739=https://10.25.73.150:2380,SZD-L0097856=https://10.25.84.251:2380 --data-dir=/varlib/etcd snapshot restore mysnapshot.db
- 2018-08-16 19:55:30.735518 I | etcdserver/membership: added member 2a53eb67dfa37cc0 [https://10.25.84.251:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:55:30.735601 I | etcdserver/membership: added member 304bcbfa92e84c75 [https://10.25.73.25:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:55:30.735617 I | etcdserver/membership: added member c267a0ca02c6bff7 [https://10.25.73.150:2380] to cluster 4741a9e2cf17e1fa
- [root@SZD-L0101798 etcd-cluster]#
恢复 10.25.73.150 节点数据到 / var/lib/etcd
- [root@SZD-L0103739 etcd-cluster]# ETCDCTL_API=3 etcdctl --name=SZD-L0103739 --endpoints="https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem --initial-cluster-token=etcd-cluster-0 --initial-advertise-peer-urls=https://10.25.73.150:2380 --initial-cluster=SZD-L0101798=https://10.25.73.25:2380,SZD-L0103739=https://10.25.73.150:2380,SZD-L0097856=https://10.25.84.251:2380 --data-dir=/varlib/etcd snapshot restore mysnapshot.db
- 2018-08-16 19:58:21.892719 I | etcdserver/membership: added member 2a53eb67dfa37cc0 [https://10.25.84.251:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:58:21.892796 I | etcdserver/membership: added member 304bcbfa92e84c75 [https://10.25.73.25:2380] to cluster 4741a9e2cf17e1fa
- 2018-08-16 19:58:21.892812 I | etcdserver/membership: added member c267a0ca02c6bff7 [https://10.25.73.150:2380] to cluster 4741a9e2cf17e1fa
- [root@SZD-L0103739 etcd-cluster]#
5, 启动 Etcd 服务
分别在 etcd 所在主机执行如下命令:
systemctl start etcd
6, 验证数据完整性
经过验证, 使用 ETCDCTL_API=2 存放的数据会丢失, 使用 ETCDCTL_API=3 存放的数据能正常恢复
附录:
查询 etcd API3 的键
- [root@SZD-L0101798 ~]# ETCDCTL_API=3 etcdctl --endpoints="https://10.25.84.251:2379,https://10.25.73.25:2379,https://10.25.73.150:2379" --cert=/root/etcd-cluster/etcd.pem --key=/root/etcd-cluster/etcd-key.pem --cacert=/root/etcd-cluster/ca.pem get / --prefix --keys-only
- 2018-08-16 20:38:06.954368 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
- /name/1
- /name/2
- /name/3
- /name/4
来源: https://yq.aliyun.com/articles/680554