前情提要
以下所有操作均在单 master 群集已完成部署的情况下进行.
所有服务器均保证防火墙常闭, 核心功能 selinux 关闭.
服务器角色分配
角色 | 地址 | 安装组件 |
---|---|---|
master | 192.168.142.220 | kube-apiserver kube-controller-manager kube-scheduler etcd |
master02 | 192.168.142.120 | kube-apiserver kube-controller-manager kube-scheduler |
node1 | 192.168.142.136 | kubelet kube-proxy docker flannel etcd |
node2 | 192.168.142.132 | kubelet kube-proxy docker flannel etcd |
nginx1 | 192.168.142.130 | nginx keepalived |
nginx2 | 192.168.142.140 | nginx keepalived |
VIP | 192.168.142.20 | 虚拟地址 |
一, nginx 端部署
建立 nginx 的 YUM 源
- [[email protected] ~]# cat> /etc/yum.repos.d/nginx.repo <<EOF
- [nginx]
- name=nginx-repo
- baseurl=http://nginx.org/packages/centos/7/\$basearch/
- gpgcheck=0
- EOF
安装 nginx 并进行配置
- [[email protected] ~]# yum install nginx -y
- # 添加 stream 模块实现四层转发
- [[email protected] ~]# VIM /etc/nginx/nginx.conf
- ### 添加
- stream {
- log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
- access_log /var/log/nginx/k8s-access.log main;
- upstream k8s-apiserver {
- server 192.168.142.220:6443;
- server 192.168.142.120:6443;
- }
- server {
- listen 6443;
- proxy_pass k8s-apiserver;
- }
- }
- # 开启服务
- [[email protected] ~]# systemctl start nginx
- [[email protected] ~]# systemctl enable nginx
安装 Keppalived 服务
- [[email protected] ~]# yum -y install keepalived
- ## 修改 keepalived 配置文件
- [[email protected] ~]# VIM /etc/keepalived/keepalived.conf
- ### 原本的全部删除, 按下面新建
- ! Configuration File for keepalived
- global_defs {
- # 接收邮件地址
- notification_email {
- [email protected]
- [email protected]
- [email protected]
- }
- # 邮件发送地址
- notification_email_from [email protected]
- smtp_server 127.0.0.1
- smtp_connect_timeout 30
- router_id NGINX_MASTER
- }
- vrrp_script check_nginx {
- script "/usr/local/nginx/sbin/check_nginx.sh"
- }
- vrrp_instance VI_1 {
- state MASTER #备服务器改为 BACKUP
- interface ens33 #监控 ens33 网卡
- virtual_router_id 51 # VRRP 路由 ID 实例, 每个实例是唯一的
- priority 100 # 优先级, 备服务器设置 90
- advert_int 1 # 指定 VRRP 心跳包通告间隔时间, 默认 1 秒
- authentication {
- auth_type PASS
- auth_pass 1111
- }
- virtual_ipaddress {
- 192.168.142.20/24 #VIP 地址
- }
- track_script {
- check_nginx
- }
- }
建立监控脚本
一旦 nginx 处于 down 状态, 将会自动关闭 keeplived
- [[email protected] ~]# mkdir -p /usr/local/nginx/sbin/
- [[email protected] ~]# VIM /usr/local/nginx/sbin/check_nginx.sh
- ## 手动进行编写
- count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
- if [ "$count" -eq 0 ];then
- systemctl stop keepalived
- fi
- [[email protected] ~]# chmod +x /usr/local/nginx/sbin/check_nginx.sh
- # 开启 keeplived
- [[email protected] ~]# systemctl start keepalived
- [[email protected] ~]# systemctl enable keepalived
此时, 前面的负载均衡已经配置完毕, 但是并不能起到实际的作用, 原因就是后方 node 节点中负责进行身份识别的 kubeconfig 文件中的地址没有改变, 无法识别.
二, node 端修改
更改 kubeconfig 中的地址
- [[email protected] ~]# VIM /opt/kubernetes/cfg/Bootstrap.kubeconfig
- [[email protected] ~]# VIM /opt/kubernetes/cfg/kube-proxy.kubeconfig
- [[email protected] ~]# VIM /opt/kubernetes/cfg/kubelet.kubeconfig
- ## 三个文件全部改为
- server: https://192.168.142.20:6443 #指向 VIP 地址
重启 kubelet & kube-proxy 服务
- [[email protected] ~]# systemctl restart kubelet
- [[email protected] ~]# systemctl restart kube-proxy
以上, 就是 nginx 做负载均衡, keppalived 做双机热备的全部部署过程
DEMO: 建立 POD 进行检测
master 端建立测试 pod
- [[email protected] ~]# kubectl run nginx --image=nginx
- ## 建立 pod
- kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
- deployment.apps/nginx created
- # 查看建立打的 pod
- [[email protected] ~]# kubectl get pods
- NAME READY STATUS RESTARTS AGE
- nginx-dbddb74b8-7tdvp 1/1 Running 0 21s
此时, pod 只能进行简单的查看, 一旦查看日志会报错. 为了解决这个问题, 可采用下面的办法解决.
- # 注意日志问题
- [[email protected] ~]# kubectl logs nginx-dbddb74b8-7tdvp
- Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-7tdvp)
- ### 解决办法:
- [[email protected] ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
- clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
来源: http://www.bubuko.com/infodetail-3415572.html