搭建LVS-DR负载均衡集群、Keepalived-LVS高可用负载均衡集群

[root@director ~]# vim /usr/local/sbin/lvs_dr.sh
#! /bin/bash
echo 1 > /proc/sys/net/ipv4/ip_forward
#开启端口转发
ipv=/usr/sbin/ipvsadm
vip=192.168.8.100
rs1=192.168.8.134
rs2=192.168.8.135
#注意这里的网卡名字
ifdown ens33
ifup ens33
#在此重启网卡的目的是避免重复设置命令行提供的IP
ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 up
#绑定VIP到dir的虚拟网卡ens33:2
route add -host $vip dev ens33:2
#添加网关
$ipv -C
$ipv -A -t $vip:80 -s wrr
$ipv -a -t $vip:80 -r $rs1:80 -g -w 1
$ipv -a -t $vip:80 -r $rs2:80 -g -w 1
#设置ipvsadm规则，-g=gateway：使用默认网关（DR模式）

[root@director ~]# sh /usr/local/sbin/lvs_dr.sh
成功断开设备 'ens33'。
成功激活的连接（D-Bus 激活路径：/org/freedesktop/NetworkManager/ActiveConnection/2）

[root@director ~]# ip add
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:be:0e:17 brd ff:ff:ff:ff:ff:ff
    inet 192.168.8.133/24 brd 192.168.8.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.8.100/32 brd 192.168.8.100 scope global ens33:2
       valid_lft forever preferred_lft forever
    inet6 fe80::592f:39cc:1b50:1d07/64 scope link 
       valid_lft forever preferred_lft forever

[root@real1 ~]# vim /usr/local/sbin/lvs_rs.sh
#/bin/bash
vip=192.168.8.100
#把vip绑定在lo上，是为了实现rs直接把结果返回给客户端
ifdown lo
ifup lo
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
#以下操作为更改arp内核参数，目的是为了让rs顺利发送mac地址给客户端
#参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

[root@real1 ~]# sh /usr/local/sbin/lvs_rs.sh

[root@real1 ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.8.2     0.0.0.0         UG    100    0        0 ens33
192.168.8.0     0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.8.100   0.0.0.0         255.255.255.255 UH    0      0        0 lo

[root@real1 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.8.100/32 brd 192.168.8.100 scope global lo:0
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

arp_ignore - INTEGER
	Define different modes for sending replies in response to
	received ARP requests that resolve local target IP addresses:
	0 - (default): reply for any local target IP address, configured
	on any interface
	1 - reply only if the target IP address is local address
	configured on the incoming interface
	2 - reply only if the target IP address is local address
	configured on the incoming interface and both with the
	sender's IP address are part from same subnet on this interface
	3 - do not reply for local addresses configured with scope host,
	only resolutions for global and link addresses are replied
	4-7 - reserved
	8 - do not reply for all local addresses

	The max value from conf/{all,interface}/arp_ignore is used
	when ARP request is received on the {interface}

arp_announce - INTEGER
	Define different restriction levels for announcing the local
	source IP address from IP packets in ARP requests sent on
	interface:
	0 - (default) Use any local address, configured on any interface
	1 - Try to avoid local addresses that are not in the target's
	subnet for this interface. This mode is useful when target
	hosts reachable via this interface require the source IP
	address in ARP requests to be part of their logical network
	configured on the receiving interface. When we generate the
	request we will check all our subnets that include the
	target IP and will preserve the source address if it is from
	such subnet. If there is no such subnet we select source
	address according to the rules for level 2.
	2 - Always use the best local address for this target.
	In this mode we ignore the source address in the IP packet
	and try to select local address that we prefer for talks with
	the target host. Such local address is selected by looking
	for primary IP addresses on all our subnets on the outgoing
	interface that include the target IP address. If no suitable
	local address is found we select the first local address
	we have on the outgoing interface or on all other interfaces,
	with the hope we will receive reply for our request and
	even sometimes no matter the source IP address we announce.

	The max value from conf/{all,interface}/arp_announce is used.

	Increasing the restriction level gives more chance for
	receiving answer from the resolved target while decreasing
	the level announces more valid sender's information.

Assume that a linux box X has three interfaces - eth0, eth1 and eth2. Each interface has an IP address IP0, 

IP1 and IP2. When a local application tries to send an IP packet with IP0 through the eth2.  Unfortunately, 

the target node’s mac address is not resolved. Thelinux box X will send the ARP request to know 

the mac address of the target(or the gateway). In this case what is the IP source address of the 

“ARP request message”? The IP0- the IP source address of the transmitting IP or IP2 - the outgoing

 interface?  Until now(actually just 3 hours before) ARP request uses the IP address assigned to 

the outgoing interface(IP2 in the above example) However the linux’s behavior is a little bit 

different. Actually the selection of source address in ARP request is totally configurable 

bythe proc variable “arp_announce”  

If we want to use the IP2 not the IP0 in the ARP request, we should change the value to 1 or 2. 

The default value is 0 - allow IP0 is used for ARP request.  

[root@director sbin]# yum install -y keepalived

自定义Keepalived配置文件：
[root@director ~]# vim /etc/keepalived/keepalived.conf
vrrp_instance VI_1 {
    #备用服务器上为 BACKUP
    state MASTER
    #绑定vip的网卡为ens33，你的网卡和阿铭的可能不一样，这里需要你改一下
    interface ens33
    virtual_router_id 51
    #备用服务器上为90
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.8.100
    }
}
virtual_server 192.168.8.100 80 {
    #(每隔10秒查询realserver状态)
    delay_loop 10
    #(lvs 算法) 
    lb_algo wlc 
    #算法(DR模式)
    lb_kind DR
    #(同一IP的连接60秒内被分配到同一台realserver)
    persistence_timeout 0 
    #(用TCP协议检查realserver状态)
    protocol TCP 
    real_server 192.168.8.134 80 {
        #(权重) 
        weight 100
        TCP_CHECK {
        #(10秒无响应超时)
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }   
    real_server 192.168.8.135 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }  
}    

启动Keepalived服务：
[root@director ~]# systemctl start keepalived

查看网卡信息：
[root@director ~]# ip add
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:be:0e:17 brd ff:ff:ff:ff:ff:ff
    inet 192.168.8.133/24 brd 192.168.8.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.8.100/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::592f:39cc:1b50:1d07/64 scope link 
       valid_lft forever preferred_lft forever
#虚拟IP（VIP）在ens33网卡上

查看ipvsadm规则：
[root@director ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.8.100:80 wlc
  -> 192.168.8.134:80             Route   100    0          0         
  -> 192.168.8.135:80             Route   100    0          0         

配置路由转发脚本：
[root@real2 ~]# vim /usr/local/sbin/lvs_rs.sh
#/bin/bash
vip=192.168.8.100
#把vip绑定在lo上，是为了实现rs直接把结果返回给客户端
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
#以下操作为更改arp内核参数，目的是为了让rs顺利发送mac地址给客户端
#参考文档www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce


[root@real2 ~]# sh /usr/local/sbin/lvs_rs.sh