高可用实现方法汇总

作者: 李佶澳   转载请保留:原文地址   发布时间:2015/07/02 11:04:18

摘要

高可用有两种场景: “单活”与”多活”。分析实际的场景,考虑是用”主备”,还是用”负载均衡”。

Keepalived

Keepalived

Keepalived是用于实施两台或者两组LVS服务器之间的热备, 当一组LVS服务器失活的时候,另一组LVS服务器自动接管工作。

keepalived is a userspace daemon for LVS cluster nodes healthchecks and LVS directors failover.

Keepalived的目的只是用于LVS,可以通过巧妙的配置,拓展Keepalived的应用范围。

Keepalive配置文件的语法: man keepalived.conf。 文献5中介绍了Keepalived使用和配置。

Keepalived注意事项

初次使用Keepalived的时候很容易遇到一个让人哭笑不得的大坑,所以非常有必要把这个单列为一章。

Keepalive不会对配置文件的格式进行检查,而且配置文件中的”{“与前面的字符之间必须至少有一个空格分隔。

vrrp_sync_group VG_REDIS {  !注意VG_REDIS后面必须有个空格,不然会被当作"VG_REDIS{"
	group {               
		VG_REDIS                    
	}                   
}

至少在1.2.7的版本中还是这样的。

使用参数”-d”(在/etc/sysconfig/keepalived中添加)时,keepalived会把配置文件的解析结果记录到日志文件(/var/log/messages)中,可以通过查看解析后的结果,来判断配置文件是否被正确解读。例如文献6中:

Sep 12 14:13:11 example-01 Keepalived: ------< Global definitions >------
Sep 12 14:13:11 example-01 Keepalived:  LVS ID = LVS_EXAMPLE_01
Sep 12 14:13:11 example-01 Keepalived:  Smtp server = 127.0.0.1
Sep 12 14:13:11 example-01 Keepalived:  Smtp server connection timeout = 100
Sep 12 14:13:11 example-01 Keepalived:  Email notification from = admins@example.com, fakepager@example.com
Sep 12 14:13:11 example-01 Keepalived:  Email notification = admins@example.com
Sep 12 14:13:11 example-01 Keepalived: ------< SSL definitions >------
Sep 12 14:13:11 example-01 Keepalived:  Using autogen SSL context
Sep 12 14:13:11 example-01 Keepalived: ------< VRRP Topology >------
Sep 12 14:13:11 example-01 Keepalived:  VRRP Instance = VI_1
Sep 12 14:13:11 example-01 Keepalived:    Want State = MASTER
Sep 12 14:13:11 example-01 Keepalived:    Runing on device = eth0
Sep 12 14:13:11 example-01 Keepalived:    Virtual Router ID = 51
Sep 12 14:13:11 example-01 Keepalived:    Priority = 150
Sep 12 14:13:11 example-01 Keepalived:    Advert interval = 1sec
Sep 12 14:13:11 example-01 Keepalived:    Preempt Active
Sep 12 14:13:11 example-01 Keepalived:    Authentication type = SIMPLE_PASSWORD
Sep 12 14:13:11 example-01 Keepalived:    Password = example
Sep 12 14:13:11 example-01 Keepalived:    VIP count = 1
Sep 12 14:13:11 example-01 Keepalived:      VIP1 = 192.168.1.11/32
Sep 12 14:13:11 example-01 Keepalived:  VRRP Instance = VI_GATEWAY
Sep 12 14:13:11 example-01 Keepalived:    Want State = MASTER
Sep 12 14:13:11 example-01 Keepalived:    Runing on device = eth1
Sep 12 14:13:11 example-01 Keepalived:    Virtual Router ID = 52
Sep 12 14:13:11 example-01 Keepalived:    Priority = 150
Sep 12 14:13:11 example-01 Keepalived:    Advert interval = 1sec
Sep 12 14:13:11 example-01 Keepalived:    Preempt Active
Sep 12 14:13:11 example-01 Keepalived:    Authentication type = SIMPLE_PASSWORD
Sep 12 14:13:11 example-01 Keepalived:    Password = example
Sep 12 14:13:11 example-01 Keepalived:    VIP count = 1
Sep 12 14:13:11 example-01 Keepalived:      VIP1 = 10.20.40.1/32
Sep 12 14:13:11 example-01 Keepalived: ------< VRRP Sync groups >------
Sep 12 14:13:11 example-01 Keepalived:  VRRP Sync Group = VG1, MASTER
Sep 12 14:13:11 example-01 Keepalived:    monitor = VI_1
Sep 12 14:13:11 example-01 Keepalived:    monitor = VI_GATEWAY
Sep 12 14:13:11 example-01 Keepalived: ------< LVS Topology >------
Sep 12 14:13:11 example-01 Keepalived:  System is compiled with LVS v1.0.4
Sep 12 14:13:11 example-01 Keepalived:  VIP = 192.168.1.11, VPORT = 22
Sep 12 14:13:11 example-01 Keepalived:    delay_loop = 10, lb_algo = rr
Sep 12 14:13:11 example-01 Keepalived:    protocol = TCP
Sep 12 14:13:11 example-01 Keepalived:    lb_kind = NAT
Sep 12 14:13:11 example-01 Keepalived:    RIP = 10.20.40.11, RPORT = 22, WEIGHT = 1
Sep 12 14:13:11 example-01 Keepalived: ------< Health checkers >------
Sep 12 14:13:11 example-01 Keepalived:  10.20.40.11:22
Sep 12 14:13:11 example-01 Keepalived:    Keepalive method = TCP_CHECK
Sep 12 14:13:11 example-01 Keepalived:    Connection timeout = 10
Sep 12 14:13:11 example-01 Keepalived:    Connection port = 22 

最简应用-可抢占

Simplest-preempt

最简应用-不可抢占

Simplest-nopreempt

最简应用-VRRP-script

可以通过vrrp-script自行定义存活的条件。

Simplest-VRRP-script

拓展场景-高可用的Redis

Keepalived的配置文件(man keepalived.conf)中,可以设置Master和Backup在进行角色转变时执行的脚本。如下:

notify_master /path/to_master.sh    
notify_backup /path/to_backup.sh   
notify_fault "/path/fault.sh VG_1"

我们可以丰富脚本中的内容,拓展Keepalived的应用范围。

我们需要找到一种方法使keepalived可以检测的是Redis进程的存活,而不是机器的存活,这可以利用上一节的中的vrrp-script。

最终效果:

两台物理机器为主备,主为master,备为slave,主死掉后,备成为master,主恢复后成为备。主备切换只会有瞬间的服务不可用,且不丢失数据。

              _|_
              \ /
         +-----'-----+            +-----------+  
         |   VIP     | keepalived |   VIP     |
         |   LVS1    +------------+   LVS2    |
         |  Master   |            |  Backup   |
         +-----------+            +-----------+   
        192.168.192.37            192.168.192.38

具体实现: Keepalived - Redis Master Slaver

常规用法-Keepalvied + LVS

2015-07-09 16:43:28 @ 暂时不需要这种场景,以后需要的时候,再补充。

如下图所示:

1. 两台LVS服务器,一主一备
2. 三台Machine拥有同样的VIP,且不会对VIP的Arp报文进行回应(这很重要!)
3. 同一时刻, LVS1和LVS2之间只有“Master”才会被设置VIP,“Master”会回应VIP的Arp报文(所以发送VIP的报文会被“Master”接收)

                    _|_
                    \ /
               +-----'-----+            +-----------+             VIP:192.168.1.10
               |   VIP     | keepalived |   VIP     |
               |   LVS1    +------------+   LVS2    |
               |  Master   |            |   Backup   |
               +-----------+            +-----------+   
                     |
          +----------+----------+---------------------+
         _|_                   _|_                   _|_
         \ /                   \ /                   \ /
   +------'-----+        +------'-----+        +------'-----+ 
   |    VIP     |        |    VIP     |        |    VIP     | 
   |  Machine1  |        |  Machine2  |        |  Machine3  | 
   +------------+        +------------+        +------------+ 
    192.168.1.11         192.168.1.12          192.168.1.13

LVS1与LVS2会互相检测对方的存在,根据配置文件中的内容(默认角色、优先级),决定谁作为Master。当Backup发现Master失活,就会自动转变成Master,并设置VIP,开始接管工作。

如果在配置文件中,设置了Master是”可以抢占的”,当拥有高优先级的Master复活后,将会抢回Master的身份。

问题: 如果Master和Backup之间的网络断开,是不是两个都会成为Master,导致VIP冲突?因该是会的。所以keepalived的应用前提: Master与Backup之间的网络要是可靠的。

HeartBeat & Pacemaker

2015-07-09 16:43:28 @ 暂时不需要这种场景,以后需要的时候,再补充,

HeartBeat解决结点的存活监测的问题。

Pacemaker

参考

  1. http://www.ibm.com/developerworks/cn/cloud/library/cl-cloudhighavailability/ “ 实现基于云的高可用性的实用方法”
  2. http://www.ibm.com/developerworks/cn/linux/l-haccmdb/ “实现复合应用程序的高可用性”
  3. http://www.ibm.com/developerworks/cn/linux/l-drbd/ “Distributed Replicated Block Device的高可用性”
  4. http://www.ibm.com/developerworks/cn/opensource/os-cn-RabbitMQ/index.html “高可用RabbitMQ集群自动化部署解决方案”
  5. http://keepalived.org/pdf/sery-lvs-cluster.pdf “负载均衡及服务器集群(lvs)”
  6. http://keepalived.org/LVS-NAT-Keepalived-HOWTO.html LVS-NAT-Keepalived-HOWTO”
  7. http://abcve.com/redis-keepalived/ “ Redis主从配置及使用Keepalived实现高可用”
  8. https://tobrunet.ch/2013/07/keepalived-check-and-notify-scripts/ “Keepalived Check and Notify Scripts”

欢迎加微信,最好备注姓名和方向

QQ交流群

区块链实践互助QQ群:576555864

Kubernetes实践互助QQ群:947371129

Prometheus实践互助QQ群:952461804

API网关Kong实践互助QQ群:952503851

Ansible实践互助QQ群:955105412

Copyright @2011-2018 All rights reserved. 转载请添加原文连接,合作请加微信lijiaocn或者发送邮件: lijiaocn@foxmail.com,备注网站合作 友情链接: lijiaocn github.com