LVS-Tun is an LVS original. It is based on LVS-DR and has the same high scalability/throughput of LVS-DR.
LVS-Tun can be used with realservers that can tunnel (==IPIP encapsulation). The director encapsulates the request packet inside an IPIP packet before sending it to the realserver. The realserver must be able to decapsulate the IPIP packet (currently linux only). (With LVS-DR, the realservers can have almost any OS.)
Unlike LVS-DR, with LVS-Tun the realservers can be on a network remote from the director, and can each be on separate networks. Thus the realservers could be in different countries (eg a set of ftp mirror sites for a project). If this is the case the realservers will be generating reply packets with VIP->CIP. Not being on the VIP network, the routers for the realservers will have to be programmed to accept outgoing packets with source=VIP. Routers normally drop these packets as an anti-spoofing measure.
If the realservers are on the same network as the director, then LVS-DR and LVS-Tun are equivalent in terms of performance and ease of setup (see the performance page)
Here's an example set of IPs for a LVS-Tun setup. For (my) convenience the servers are on the same network as the client. The only restrictions for LVS-Tun with remote hosts are that the client must be able to route to the director and that the realservers must be able to route to the client (the return packets to the client come directly from the realservers and do not go back through the director).
Normally for LVS-Tun,the client is on a different network to the director/server(s), and each server has its own route to the outside world. In the simple test case below where all machines are on the 192.168.1.0 network there would be no default route for the servers, and routing for packets from the servers to the client would use the device on the 192.168.1.0 network (presumably eth0). In reallife, the realservers would have their own router/connection to the internet and packets returning to the client would go through this router. In any case reply packets do not go back through the director.
Machine IP client CIP=192.168.1.254 director DIP=192.168.1.1 VIP=192.168.1.110 (arps, IP clients connect to) realserver-1 RIP1=192.168.1.2, VIP (tunl0, non-arping, 192.168.1.110) realserver-2 RIP2=192.168.1.3, VIP (tunl0, non-arping, 192.168.1.110) realserver-3 RIP3=192.168.1.4, VIP (tunl0, non-arping, 192.168.1.110) . . realserver-n RIPn=192.168.1.n+1, VIP (tunl0, non-arping, 192.168.1.110)
#lvs_tun.conf LVS_TYPE=VS_TUN INITIAL_STATE=on VIP=eth0:110 192.168.1.110 255.255.255.255 192.168.1.110 DIP=eth0 192.168.1.9 192.168.1.0 255.255.255.0 192.168.1.255 DIRECTOR_DEFAULT_GW=client SERVICE=t telnet rr realserver1 realserver2 SERVER_VIP_DEVICE=tunl0 SERVER_NET_DEVICE=eth0 SERVER_DEFAULT_GW=client #----------end lvs_tun.conf------------------------------------
________ | | | client | |________| CIP=192.168.1.254 | | VIP=192.168.1.110 (eth0:1, arps) __________ | | | director | |__________| DIP=192.168.1.1 (eth0) | | ------------------------------------- | | | | | | RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=192.168.1.4 (eth0) _____________ _____________ _____________ | | | | | | | realserver | | realserver | | realserver | |_____________| |_____________| |_____________| VIP=192.168.1.110 VIP=192.168.1.110 VIP=192.168.1.110 (all tunl0,non-arping) | | | (router) (router) (router) | | | ----------------------------------------------> to client
Here's part of the rc.lvs_dr script which configures the realserver with RIP=192.168.1.8
#setup servers for telnet /sbin/ipvsadm -A -t 192.168.1.110:23 -s rr /sbin/ipvsadm -a -t 192.168.1.110:23 -R 192.168.1.1 -i -w 1
With LVS-Tun, the target port numbers of incoming packets cannot be remapped. A request to port 23 on the VIP will be forwarded to port 23 on a realserver, thus no port number is used for setting up the IP of the realserver.
Here's the packet headers as the request is processed by LVS-Tun.
packet source dest data 1. request from client CIP:3456 VIP:23 - 2. ipvsadm table: director chooses server=RIP1, encapsulates into IPIP packet DIP RIP1 IP datagram source=CIP:3456, dest=VIP:23, data= - 3. realserver recovers IP datagram CIP:3456 VIP:23 - 4. realserver looks up routing table, finds VIP is local, processes request locally, generates reply VIP:23 CIP:3456 "login: " 5. packet leaves realserver via default gw, not via DIP.
For the verbally oriented...
A packet arrives for the VIP. The director looks up its tables and decides to send the connection to realserver_1. The director encapsulates the request packet in an IPIP datagram with DIP->RIP. The packet arrives at realserver1, the realserver recovers the original IP datagram, looks up its routing table, finds that the VIP (on an otherwise unused, non-arping and nonfunctional device) is local and processes the packet locally. A reply packet is generated with VIP:23->CIP:3456. The realserver looks up its routing table and finds that a packet to CIP goes out its default gw (not to the DIP).
The tunl0 device does not arp with 2.0.36 kernels, but does with 2.2.x kernels. Go look up the section on the arp problem to see if you need to patch the kernel on the realserver.
Edit the template lvs_tun.conf and run configure.
$ ./configure_lvs.pl lvs_nat.conf
Load the the parameters into the director and then the realservers with the command
$ . ./etc/rc.d/rc.lvs_tun
(the script knows whether it is running on a realserver or the director).
(later put in /etc/rc.d or /etc/init.d and put mon_xxx.cf in /etc/mon)
check the output from ipvsadm, ifconfig -a and netstat -rn, to see that the services/IP's are correct. If not re-edit and re-run the script(s).
If the realserver is on a different network to the director, then it will be replying with packets having a source address of the VIP. In this case the router will have to be programmed to pass packets in the outward direction with the source address of the VIP. This may be a problem with some firewalls as they will be forwarding packets with a source address not in their network.
JoeHow does a packet get to a tunl device from a remote machine without a MAC address?
Julian
tunl, lo and dummy are used just to configure the VIP. We don't send any packets through these devices. The requests are delivered to the real servers using their RIP. The director asks only about their RIP from ipvsadm. Only their gateway asks about VIP but only the director must reply. When the packet is received in the real server it is delivered locally (not forwarded or dropped) due to configured VIP. This is the only role of these "dummy" interfaces: the kernel to treat the received packet as it is destined to our host (the real server). Nothing more. No IPIP encapsulations (for tunl), no MAC address definitions, nothing more. When we answer the request we use eth0. The tunl/lo/dummy is not selected as device for the outgoing packets. We have routes for eth0 (default gateway) which we use for the outgoing traffic. This is for DROUTE and TUNNEL mode.
If two linux boxes (not in an LVS) are joined by an IPIP tunnel and there is no MAC address associated with the tunl0 devices at each end of the link, then how do the packets get from one machine to the other?
Julian
The packets are encapsulated via IPIP and sent to the tunnel ends real IP where they are decapsulated again and appear on the tunl interface. You don't need a MAC address for point-to-point links, or logical interfaces like tunnels.