这是 kubernetes 二进制部署的第三篇
如果没有看过前面第一篇的朋友可以看看下面的:
Kubernetes 二进制部署 (一) 单节点部署 https://blog.51cto.com/14449541/2470009
kubernetes 二进制部署 (二) 多节点部署 https://blog.51cto.com/14449541/2470012
实验环境:
负载均衡
- Nginx1:192.168.35.104/24
- Nginx2:192.168.35.105/24
Master 节点
- master1:192.168.35.100/24
- master2:192.168.35.103/24
Node 节点
- node1:192.168.35.101/24
- node2:192.168.35.102/24
nginx 安装部署
lb01 和 lb02 操作
1, 关闭防火墙
- [[email protected] ~]# systemctl stop firewalld.service?
- [[email protected] ~]# setenforce 0
2, 安装 nginx 服务, 把 nginx.sh 和 keepalived.conf 脚本拷贝到家目录
[[email protected] ~]# ls
anaconda-ks.cfg ? ? ? keepalived.conf ? 公共 ? 视频 ? 文档 ? 音乐
initial-setup-ks.cfg ?nginx.sh ? ? ? ? 模板 ? 图片 ? 下载 ? 桌面
3, 构建 nginx 的 yum 源环境
(1)复制配置文件
- [[email protected] ~]# VIM nginx.sh
- cat> /etc/yum.repos.d/nginx.repo <<EOF
- [nginx]
- name=nginx repo
- baseurl=http://nginx.org/packages/centos/7/$basearch/
- gpgcheck=0? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? #复制四行内容
- EOF
- stream {
- ? ?log_format ?main ?'$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
- ? ? access_log ?/var/log/nginx/k8s-access.log ?main;
- ? ? upstream k8s-apiserver {
- ? ? ? ? server 10.0.0.3:6443;
- ? ? ? ? server 10.0.0.8:6443;
- ? ?
- }
- ? ? server {
- ? ? ? ? ? ? ? ? listen 6443;
- ? ? ? ? ? ? ? ? proxy_pass k8s-apiserver;
- ? ?
- }
- ? ?
- }
(2)创建 yum 源, 重新加载 yum 仓库, 并安装 nginx
- [[email protected] ~]# VIM /etc/yum.repos.d/nginx.repo
- [nginx]
- name=nginx repo
- baseurl=http://nginx.org/packages/centos/7/$basearch/
- gpgcheck=0
- [[email protected] ~]# yum list
- [[email protected] ~]#?yum install nginx -y
4, 添加四层转发
(1)复制配置文件
- [[email protected] ~]# VIM nginx.sh
- cat> /etc/yum.repos.d/nginx.repo <<EOF
- [nginx]
- name=nginx repo
- baseurl=http://nginx.org/packages/centos/7/$basearch/
- gpgcheck=0? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
- EOF
- stream {
- ? ?log_format ?main ?'$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
- ? ? access_log ?/var/log/nginx/k8s-access.log ?main;
- ? ? upstream k8s-apiserver {
- ? ? ? ? server 10.0.0.3:6443;
- ? ? ? ? server 10.0.0.8:6443;
- ? ?
- }
- ? ? server {
- ? ? ? ? ? ? ? ? listen 6443;
- ? ? ? ? ? ? ? ? proxy_pass k8s-apiserver;
- ? ?
- }
- ? ?
- }? ? ? ? ? ? ? ? ? ? ? ? ? #复制此段内容
(2)添加配置文件并修改
- [[email protected] ~]# VIM /etc/nginx/nginx.conf
- user ?nginx;
- worker_processes ?1;
- error_log ?/var/log/nginx/error.log warn;
- pid ? ? ? ?/var/run/nginx.pid;
- events {
- ? ? worker_connections ?1024;
- }
- stream {
- ? ?log_format ?main ?'$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';
- ? ? access_log ?/var/log/nginx/k8s-access.log ?main;
- ? ? upstream k8s-apiserver {
- ? ? ? ? server 192.168.35.100:6443;
- ? ? ? ? server 192.168.35.103:6443;? ? ? ? ? ? ? ? ? #只需修改 IP 为 master1 和 master2 的地址? ? ??
- ? ?
- }
- ? ? server {
- ? ? ? ? ? ? ? ? listen 6443;
- ? ? ? ? ? ? ? ? proxy_pass k8s-apiserver;
- ? ?
- }
- ? ?
- }
- http {
- ? ? include ? ? ? /etc/nginx/mime.types;
- ? ? default_type ?application/octet-stream;
- ? ? log_format ?main ?'$remote_addr - $remote_user [$time_local]"$request" '
- ? ? ? ? ? ? ? ? ? ? ? '$status $body_bytes_sent"$http_referer" '
- ? ? ? ? ? ? ? ? ? ? ? '"$http_user_agent" "$http_x_forwarded_for"';
- ? ? access_log ?/var/log/nginx/access.log ?main;
- ? ? sendfile ? ? ? ?on;
- ? ? #tcp_nopush ? ? on;
- ? ? keepalive_timeout ?65;
- ? ? #gzip ?on;
- ? ? include /etc/nginx/conf.d/*.conf;
- }
(3)验证语法是否错误
- [[email protected] ~]# nginx -t
- nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
- nginx: configuration file /etc/nginx/nginx.conf test is successful
(4)修改首页, 进行区分, 一个 master, 一个 backup.
lb01 操作:
- [[email protected] ~]# cd /usr/share/nginx/HTML/
- [[email protected] HTML]# ls
50x.HTML ?index.HTML
- [[email protected] HTML]# VIM index.HTML?
- /14 <h1>Welcome to master ginx!</h1>
lb02 操作:
- [[email protected] ~]# cd /usr/share/nginx/HTML/
- [[email protected] HTML]# ls
50x.HTML ?index.HTML
- [[email protected] HTML]# VIM index.HTML?
- /14 <h1>Welcome to backup?ginx!</h1>
(5)开启服务, 并访问首页
- lb01:
- [[email protected] ~]# systemctl start nginx
用浏览器访问 http://192.168.35.104/
- lb02:
- [[email protected] ~]# systemctl start nginx
用浏览器访问 http://192.168.35.105/
keepalived 安装部署
lb01 和 lb02 操作:
1, 安装 keepalived
[[email protected] ~]# yum install keepalived -y
2, 修改配置文件
[[email protected] ~]# ls
anaconda-ks.cfg ? ? ? keepalived.conf ? 公共 ? 视频 ? 文档 ? 音乐
initial-setup-ks.cfg ?nginx.sh ? ? ? ? 模板 ? 图片 ? 下载 ? 桌面
[[email protected] ~]# cp keepalived.conf /etc/keepalived/keepalived.conf
cp: 是否覆盖 "/etc/keepalived/keepalived.conf"? yes
[[email protected] ~]# VIM /etc/keepalived/keepalived.conf?
lb01 是 Master 配置如下:
- ! Configuration File for keepalived
- global_defs {
- ? ?# 接收邮件地址
- ? ?notification_email {
- ? ? [email protected]
- ? ? [email protected]
- ? ? [email protected]
- ? ?
- }
- ? ?# 邮件发送地址
- ? ?notification_email_from [email protected]
- ? ?smtp_server 127.0.0.1
- ? ?smtp_connect_timeout 30
- ? ?router_id NGINX_MASTER
- }
- vrrp_script check_nginx {
- ? ? script "/etc/nginx/check_nginx.sh"
- }
- vrrp_instance VI_1 {
- ? ? state MASTER?
- ? ? interface ens33
- ? ? virtual_router_id 51 # VRRP 路由 ID 实例, 每个实例是唯一的
- ? ? priority 100 ? ?# 优先级, 备服务器设置 90 ?
- ? ? advert_int 1 ? ?# 指定 VRRP 心跳包通告间隔时间, 默认 1 秒
- ? ? authentication {
- ?
- ? ? ? ? auth_type PASS
- ? ? ? ? auth_pass 1111
- ? ?
- } ??
- ? ? virtual_ipaddress {
- ? ? ? ? 192.168.35.200/24
- ? ?
- }
- ? ? track_script {
- ? ? ? ? check_nginx
- ? ?
- }
- }
lb02 是 Backup 配置如下:
- ! Configuration File for keepalived
- global_defs {
- ? ?# 接收邮件地址
- ? ?notification_email {
- ? ? [email protected]
- ? ? [email protected]
- ? ? [email protected]
- ? ?
- }
- ? ?# 邮件发送地址
- ? ?notification_email_from [email protected]
- ? ?smtp_server 127.0.0.1
- ? ?smtp_connect_timeout 30
- ? ?router_id NGINX_MASTER
- }
- vrrp_script check_nginx {
- ? ? script "/etc/nginx/check_nginx.sh"
- }
- vrrp_instance VI_1 {
- ? ? state?BACKUP?
- ? ? interface ens33
- ? ? virtual_router_id 51 # VRRP 路由 ID 实例, 每个实例是唯一的
- ? ? priority 90 ? ?# 优先级, 备服务器设置 90 ?
- ? ? advert_int 1 ? ?# 指定 VRRP 心跳包通告间隔时间, 默认 1 秒
- ? ? authentication {
- ?
- ? ? ? ? auth_type PASS
- ? ? ? ? auth_pass 1111
- ? ?
- } ??
- ? ? virtual_ipaddress {
- ? ? ? ? 192.168.35.200/24
- ? ?
- }
- ? ? track_script {
- ? ? ? ? check_nginx
- ? ?
- }
- }
3, 制作管理脚本
- [[email protected] ~]# VIM /etc/nginx/check_nginx.sh
- count=$(ps -ef |grep nginx |egrep -cv "grep|$$")
- if [ "$count" -eq 0 ];then
- ? ? ? ? systemctl stop keepalived
- fi
4, 赋予执行权限并开启服务
- [[email protected] ~]# chmod +x /etc/nginx/check_nginx.sh
- [[email protected] ~]# systemctl start keepalived
5, 查看地址信息
(1)查看 lb01 地址信息
- [[email protected] ~]# ip a
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
- ? ? link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- ? ? .NET 127.0.0.1/8 scope host lo
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 ::1/128 scope host?
- ? ? ? ?valid_lft forever preferred_lft forever
- 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- ? ? link/ether 00:0c:29:ba:e6:18 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.35.104/24 brd 192.168.35.255 scope global ens33
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? .NET 192.168.35.200/24 scope global secondary ens33? ? ? ? ? ? ?# 漂移地址在 lb01 中?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::6ec5:6d7:1b18:466e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::2a3:b621:ca01:463e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::d4e2:ef9e:6820:145a/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
- ? ? ? ?valid_lft forever preferred_lft forever
- 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
(2)查看 lb02 地址信息
- [[email protected] ~]# ip a
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
- ? ? link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- ? ? .NET 127.0.0.1/8 scope host lo
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 ::1/128 scope host?
- ? ? ? ?valid_lft forever preferred_lft forever
- 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- ? ? link/ether 00:0c:29:1d:ec:b0 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.35.105/24 brd 192.168.35.255 scope global ens33
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::6ec5:6d7:1b18:466e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::2a3:b621:ca01:463e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::d4e2:ef9e:6820:145a/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
- ? ? ? ?valid_lft forever preferred_lft forever
- 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
6, 测试故障转移切换
(1)验证地址漂移(lb01 中使用 pkill nginx, 再在 lb02 中使用 ip a 查看)
- [[email protected] ~]# pkill nginx
- [[email protected] ~]# systemctl status nginx
● nginx.service - nginx - high performance Web server
? ?Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor preset: disabled)
? ?Active: failed (Result: exit-code) since 六 2020-02-08 16:54:45 CST; 11s ago
- ? ? ?Docs: http://nginx.org/en/docs/
- ? Process: 13156 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=1/FAILURE)
- ?Main PID: 6930 (code=exited, status=0/SUCCESS)
2 月 08 16:54:45 localhost.localdomain kill[13156]: -q, --queue <信号> ? ...2)
2 月 08 16:54:45 localhost.localdomain kill[13156]: -p, --pid ? ? ? ? ? ?... 号
2 月 08 16:54:45 localhost.localdomain kill[13156]: -l, --list [=<信号>] ... 称
2 月 08 16:54:45 localhost.localdomain kill[13156]: -L, --table ? ? ? ? ?... 值
2 月 08 16:54:45 localhost.localdomain kill[13156]: -h, --help ? ? 显示此... 出
2 月 08 16:54:45 localhost.localdomain kill[13156]: -V, --version ? 输出版... 出
2 月 08 16:54:45 localhost.localdomain kill[13156]: 更多信息请参阅 kill(1).
2 月 08 16:54:45 localhost.localdomain systemd[1]: nginx.service: control...
2 月 08 16:54:45 localhost.localdomain systemd[1]: Unit nginx.service ent...
2 月 08 16:54:45 localhost.localdomain systemd[1]: nginx.service failed.
- Hint: Some lines were ellipsized, use -l to show in full.
- [[email protected] ~]# systemctl status keepalived.service? ? ? ? ? ? ?#keepalived 服务也随之关闭, 说明 nginx 中的 check_nginx.sh 生效
● keepalived.service - LVS and VRRP High Availability Monitor
- ? ?Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
- ? ?Active: inactive (dead)
2 月 08 16:39:26 localhost.localdomain Keepalived_vrrp[7319]: VRRP_Instan...
2 月 08 16:39:26 localhost.localdomain Keepalived_vrrp[7319]: Sending gra...
2 月 08 16:39:26 localhost.localdomain Keepalived_vrrp[7319]: Sending gra...
2 月 08 16:39:26 localhost.localdomain Keepalived_vrrp[7319]: Sending gra...
2 月 08 16:39:26 localhost.localdomain Keepalived_vrrp[7319]: Sending gra...
2 月 08 16:54:46 localhost.localdomain Keepalived[7317]: Stopping
2 月 08 16:54:46 localhost.localdomain systemd[1]: Stopping LVS and VRRP ...
2 月 08 16:54:46 localhost.localdomain Keepalived_vrrp[7319]: VRRP_Instan...
2 月 08 16:54:46 localhost.localdomain Keepalived_vrrp[7319]: VRRP_Instan...
2 月 08 16:54:47 localhost.localdomain systemd[1]: Stopped LVS and VRRP H...
Hint: Some lines were ellipsized, use -l to show in full.
在 lb01 查看地址:
- [[email protected] ~]# ip a
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
- ? ? link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- ? ? .NET 127.0.0.1/8 scope host lo
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 ::1/128 scope host?
- ? ? ? ?valid_lft forever preferred_lft forever
- 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- ? ? link/ether 00:0c:29:ba:e6:18 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.35.104/24 brd 192.168.35.255 scope global ens33
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::6ec5:6d7:1b18:466e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::2a3:b621:ca01:463e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::d4e2:ef9e:6820:145a/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
- ? ? ? ?valid_lft forever preferred_lft forever
- 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
在 lb02 查看地址:
- [[email protected] ~]# ip a
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
- ? ? link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- ? ? .NET 127.0.0.1/8 scope host lo
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 ::1/128 scope host?
- ? ? ? ?valid_lft forever preferred_lft forever
- 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- ? ? link/ether 00:0c:29:1d:ec:b0 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.35.105/24 brd 192.168.35.255 scope global ens33
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? .NET 192.168.35.200/24 scope global secondary ens33? ? ? ? ? ? ? ? #漂移地址转移到 lb02 中
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::6ec5:6d7:1b18:466e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::2a3:b621:ca01:463e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::d4e2:ef9e:6820:145a/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
- ? ? ? ?valid_lft forever preferred_lft forever
- 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
(2)恢复操作(在 lb01 中先启动 nginx 服务, 再启动 keepalived 服务)
- [[email protected] ~]# systemctl start nginx
- [[email protected] ~]# systemctl start keepalived.service?
- [[email protected] ~]# ip a
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
- ? ? link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- ? ? .NET 127.0.0.1/8 scope host lo
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 ::1/128 scope host?
- ? ? ? ?valid_lft forever preferred_lft forever
- 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- ? ? link/ether 00:0c:29:ba:e6:18 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.35.104/24 brd 192.168.35.255 scope global ens33
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? .NET 192.168.35.200/24 scope global secondary ens33? ? ? ? ? ? ?? #漂移地址又转移回 lb01 中
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::6ec5:6d7:1b18:466e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::2a3:b621:ca01:463e/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- ? ? inet6 fe80::d4e2:ef9e:6820:145a/64 scope link tentative dadfailed?
- ? ? ? ?valid_lft forever preferred_lft forever
- 3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
- ? ? .NET 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
- ? ? ? ?valid_lft forever preferred_lft forever
- 4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN qlen 1000
- ? ? link/ether 52:54:00:14:39:99 brd ff:ff:ff:ff:ff:ff
(3)nginx 站点 / usr/share/nginx/HTML
在宿主机上 ping 虚拟 IP, 保证可以 ping 通, 说明可以去进行访问
在 lb01 查看 index.HTML
[[email protected] ~]# cat /usr/share/nginx/HTML/index.HTML?
- <!DOCTYPE HTML>
- <HTML>
- <head>
- <title>
- Welcome to nginx!
- </title>
- <style>
- ? ? body { ? ? ? ? width: 35em; ? ? ? ? margin: 0 auto; ? ? ? ? font-family:
- Tahoma, Verdana, Arial, sans-serif; ? ? }
- </style>
- </head>
- <body>
- <h1>
- Welcome to master ginx!
- </h1>
- <p>
- If you see this page, the nginx Web server is successfully installed and
- working. Further configuration is required.
- </p>
- <p>
- For online documentation and support please refer to
- <a href="http://nginx.org/">
- nginx.org
- </a>
- .
- <br/>
- Commercial support is available at
- <a href="http://nginx.com/">
- nginx.com
- </a>
- .
- </p>
- <p>
- <em>
- Thank you for using nginx.
- </em>
- </p>
- </body>
- </HTML>
在 lb01 查看 index.HTML
[[email protected] ~]# cat /usr/share/nginx/HTML/index.HTML?
- <!DOCTYPE HTML>
- <HTML>
- <head>
- <title>
- Welcome to nginx!
- </title>
- <style>
- ? ? body { ? ? ? ? width: 35em; ? ? ? ? margin: 0 auto; ? ? ? ? font-family:
- Tahoma, Verdana, Arial, sans-serif; ? ? }
- </style>
- </head>
- <body>
- <h1>
- Welcome to backup nginx!
- </h1>
- <p>
- If you see this page, the nginx Web server is successfully installed and
- working. Further configuration is required.
- </p>
- <p>
- For online documentation and support please refer to
- <a href="http://nginx.org/">
- nginx.org
- </a>
- .
- <br/>
- Commercial support is available at
- <a href="http://nginx.com/">
- nginx.com
- </a>
- .
- </p>
- <p>
- <em>
- Thank you for using nginx.
- </em>
- </p>
- </body>
- </HTML>
因为漂移地址是在 lb01 上, 所以访问漂移地址时现实的 nginx 首页应该是包含 master 的
node 节点绑定 VIP 地址
1, 修改 node 节点配置文件统一 VIP(Bootstrap.kubeconfig,kubelet.kubeconfig)
- [[email protected] ~]# VIM /opt/kubernetes/cfg/Bootstrap.kubeconfig
- [[email protected] ~]# VIM /opt/kubernetes/cfg/kubelet.kubeconfig
- [[email protected] ~]# VIM /opt/kubernetes/cfg/kube-proxy.kubeconfig
- # 全部都改为 VIP 地址
- server: https://192.168.35.200:6443
2, 替换完成直接自检
- [[email protected] ~]# cd /opt/kubernetes/cfg/
- [[email protected] cfg]# grep 200 *
- Bootstrap.kubeconfig: ? ?server: https://192.168.35.200:6443
- kubelet.kubeconfig: ? ?server: https://192.168.35.200:6443
- kube-proxy.kubeconfig: ? ?server: https://192.168.35.200:6443
3, 重启服务
- [[email protected] cfg]# systemctl restart kubelet.service?
- [[email protected] cfg]# systemctl restart kube-proxy.service?
4, 在 lb01 上查看 nginx 的 k8s 日志
- [[email protected] ~]# tail /var/log/nginx/k8s-access.log
- 192.168.35.102 192.168.35.100:6443 - [08/Feb/2020:17:42:00 +0800] 200 1119
- 192.168.35.102 192.168.35.103:6443 - [08/Feb/2020:17:42:00 +0800] 200 1121
- 192.168.35.101 192.168.35.100:6443 - [08/Feb/2020:17:42:14 +0800] 200 1121
- 192.168.35.101 192.168.35.100:6443 - [08/Feb/2020:17:42:14 +0800] 200 1121
在 master01 上操作
1, 测试创建 pod
- [[email protected] k8s]# kubectl run nginx --image=nginx
- kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
- deployment.apps/nginx created
2, 查看状态
- [[email protected] k8s]# kubectl get pods
- NAME ? ? ? ? ? ? ? ? ? ?READY ? STATUS ? ?RESTARTS ? AGE
- nginx-dbddb74b8-cfggf ? 1/1? ? ? ? 0 ? ? ContainerCreating? ? ?64s? ? ? ? ? ? ? ? #正在创建中
- [[email protected] k8s]# kubectl get pods
- NAME ? ? ? ? ? ? ? ? ? ?READY ? STATUS ? ?RESTARTS ? AGE
- nginx-dbddb74b8-cfggf ? 1/1 ? ? Running ? 0 ? ? ? ? ?74s? ? ? ? ? ? ? ? ? ?# 创建完成, 运行中
3, 注意日志问题(无法查看, 会报错)
- [[email protected] k8s]# kubectl logs nginx-dbddb74b8-cfggf
- Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-cfggf)
解决办法:
(1)权限提升
- [[email protected] k8s]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
- clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
(2)在去进行日志查看
- [[email protected] k8s]# kubectl logs nginx-dbddb74b8-cfggf
- // 并不会出现报错, 只是因为没有去访问, 所以没有产生日志
4, 查看 pod 网络
- [[email protected] k8s]# kubectl get pods -o wide
- NAME ? ? ? ? ? ? ? ? ? ?READY ? STATUS ? ?RESTARTS ? AGE ? IP ? ? ? ? ? ?NODE ? ? ? ? ? ? NOMINATED NODE
- nginx-dbddb74b8-cfggf ? 1/1 ? ? Running ? 0 ? ? ? ? ?14m ? 172.17.45.2 ? 192.168.35.101 ? <none>
5, 在对应网段的 node 节点上操作可以直接访问
[[email protected] cfg]# curl 172.17.45.2
- <!DOCTYPE HTML>
- <HTML>
- <head>
- <title>
- Welcome to nginx!
- </title>
- <style>
- ? ? body { ? ? ? ? width: 35em; ? ? ? ? margin: 0 auto; ? ? ? ? font-family:
- Tahoma, Verdana, Arial, sans-serif; ? ? }
- </style>
- </head>
- <body>
- <h1>
- Welcome to nginx!
- </h1>
- <p>
- If you see this page, the nginx Web server is successfully installed and
- working. Further configuration is required.
- </p>
- <p>
- For online documentation and support please refer to
- <a href="http://nginx.org/">
- nginx.org
- </a>
- .
- <br/>
- Commercial support is available at
- <a href="http://nginx.com/">
- nginx.com
- </a>
- .
- </p>
- <p>
- <em>
- Thank you for using nginx.
- </em>
- </p>
- </body>
- </HTML>
6, 访问就会产生日志, 回到 master01 操作
- [[email protected] k8s]# kubectl logs nginx-dbddb74b8-cfggf
- 172.17.45.1 - - [08/Feb/2020:10:10:29 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
来源: http://www.bubuko.com/infodetail-3414959.html