Next
Previous Contents
LVS-HOWTO
Joseph Mack (C) 1999-2001, released under GPL
jmack@wm7d.net
v1.12, Dec 2001
Install, testing and running of a Linux Virtual Server with 2.2.x and 2.4.x kernels
1.
Introduction
1.1 ChangeLog
1.2 Purpose of this HOWTO
1.3 Nomenclature/Abbreviations
1.4 What is an LVS?
1.5 LVS Failure
1.6 Thanks
1.7 HOWTO source is sgml
1.8 Mailing list, subscribing, unsubscribing and searchable archives
1.9 Minimal knowledge required, getting technical help
1.10 Posting problems/questions to the mailing list
1.11 ToDo List
1.12 Other load balancing solutions
1.13 Software/Information useful/related to LVS
2.
Getting Files
2.1 Director Code
2.2 RealServer Code
2.3 Configure Script(s)
2.4 Priority Routing and ifconfig
2.5 Go setup an LVS
3.
The arp Problem
3.1 The problem
3.2 The cure(s)
3.3 The ARP problem, the first inklings
3.4 A posting to the mailinglist by Peter Kese explaining the "arp problem"
3.5 random mailings on the arp problem
3.6 Is the arp behaviour of 2.2.x kernel a bug?
3.7 How to tell if an interface is replying to arp requests
3.8 Arp caching defeats Heartbeat switchover
3.9 The device doesn't reply to arp requests, the kernel does.
3.10 Properties of devices for the VIP
3.11 Topologies for LVS-DR and LVS-Tun LVS's
3.12 A discussion about the arp problem
3.13 ATM/ethernet and router problems
4.
Collect Hardware
4.1 minimum setup
4.2 Gotchas
4.3 Test with telnet (or netcat)
5.
Choose LVS Forwarding Type
5.1 Comparison of LVS-NAT, LVS-DR and LVS-Tun
5.2 Expected LVS performance
5.3 Initial setup steps
6.
Install - General
6.1 Director
6.2 Realservers
7.
Ipvsadm
7.1 Using ipvsadm
7.2 Compile a version of ipvsadm that matches your ipvs
7.3 schedulers
7.4 does rr and lc weighting equally distribute the load?
7.5 Changing weights with ipvsadm
7.6 experimental scheduling code
8.
Persistent connection
8.1 netscape/database/tcpip persistence
8.2 LVS persistence
8.3 persistent client connection, pcc (for kernel =<2.2.10)
8.4 persistent port connection (ppc) (for kernel >= 2.2.12)
8.5 Problems: Removing persistant connections after a realserver crash
8.6 Persistent and regular services are possible on the same realserver.
8.7 Examples of persistence
8.8 AOL and proxies
8.9 PPC (persistent port connection) (kernels >2.2.12)
8.10 Related to PPC - Sticky connections
9.
Fwmarks
9.1 Introduction
9.2 single port service: telnet with fwmarks
9.3 Grouping services: single group, active ftp(20,21)
9.4 Grouping services: two groups, active ftp(20,21) and e-commerce(80,443)
9.5 passive ftp
9.6 fwmark with LVS-NAT
9.7 Collisions between fwmark and VIP rules
9.8 persistence granularity with fwmark
9.9 fwmark allows LVS-DR director to be default gw for realservers
9.10 Routing to director and realservers in an LVS setup with fwmark
9.11 fwmark simplifies configuration for large numbers of addresses
9.12 Example: firewall farm
9.13 Example: LVS'ing a CIDR block
9.14 Example: forwarding based on client source IP
9.15 Example: load balancing multiple class C networks
9.16 Example: proxy server
9.17 Example: transparent web cache
9.18 Example: Multiply-connected router
9.19 Example: Dynamically generated images in webpages
9.20 Example: Balancing many IPs/services as one block
9.21 Appendix 1: Specificiations for grouping of services with fwmarks
9.22 Appendix 2: Demonstration of grouping services with fwmarks
9.23 Appendix 3: Announcement of grouping services with fwmarks
10.
Configure tools
10.1 Configure
11.
Services
11.1 setting up a new service
11.2 services must be setup for forwarding type
11.3 ftp general
11.4 ftp (active) - the classic command line ftp
11.5 ftp (passive)
11.6 ftp is difficult to secure
11.7 evaluation of SuSE ftp proxy
11.8 sshd
11.9 telnet
11.10 dns
11.11 sendmail/smtp/pop3/qmail
11.12 Mail farms
11.13 authd/identd (port 113) and tcpwrappers (tcpd)
11.14 http name and IP-based (with LVS-DR or LVS-Tun)
11.15 http with LVS-NAT
11.16 httpd normally closes connections
11.17 Persistence with http; browser opens many connections to httpd
11.18 Dynamically generated images on web pages
11.19 http: logs, shutting down, cookies, url_parsing, squids, mod_proxy, indexing programs, htpasswd
11.20 HTTP 1.0 and 1.1 requests
11.21 https
11.22 Named Based Virtual Hosts for https
11.23 Databases
11.24 Cookies
11.25 r commands; rsh, rcp, and their ssh replacements
11.26 NFS
11.27 RealNetworks streaming protocols
11.28 Synchronising content and backing up realservers.
11.29 Timeouts for TCP/UDP connections
12.
LVS-NAT
12.1 Introduction
12.2 Example Two Network LVS-NAT (VIP and RIPs on different network)
12.3 All packets from the realserver to the outside world must go through the director
12.4 Run configure
12.5 Setting up demasquerading on the director; 2.4.x and 2.2.x
12.6 masquerading clients on realservers
12.7 re-mapping ports, rewriting in slow
12.8 masquerade timeouts
12.9 Julian's step-by-step check of a L4 LVS-NAT setup
12.10 How LVS-NAT works
12.11 In LVS-NAT, how do packets get back to the client, or how does the
12.12 One network LVS-NAT
12.13 Performance of LVS-NAT, 2.0 and 2.2 kernels
12.14 Performance of LVS-NAT, 2.4 kernels
12.15 Various debugging techniques for routes
12.16 Postings from the mailing list
13.
LVS-DR
13.1 How LVS-DR works
13.2 Handling the arp problem for LVS-DR
13.3 LVS-DR scales well
13.4 LVS-DR director is default gw for realservers
13.5 Director has 1 NIC, accepts packets via transparent proxy.
13.6 Julian's martian modification
13.7 Accepting packets on LVS-DR director by fwmarks
13.8 security concerns: default gw(s) and routing with LVS-DR/LVS-Tun
13.9 routing to realserver from director
13.10 Setting up NAT clients on LVS-DR realservers
14.
LVS-Tun
14.1 How LVS-Tun works
14.2 Configure LVS-Tun
14.3 Realservers on different network(s) to director
14.4 LVS-Tun Questions
15.
localnode
15.1 You can't rewrite ports with localnode
15.2 Testing LocalNode
16.
Transparent proxy (TP or Horms' method)
16.1 General
16.2 How you use TP
16.3 The original 2.2 TP setup method
16.4 Transparent proxy for 2.4.x
16.5 Transparent proxy Q&A
16.6 Experiments showing that 2.4TP is different to 2.2TP
16.7 What IP TP packets arriving on?
16.8 Take home lesson for setting up TP on realservers
16.9 Handling identd requests from 2.4.x LVS-DR realservers using TP
16.10 Performance of Transparent Proxy
17.
Authd/Identd
17.1 What is authd/identd?
17.2 symptoms of the ident problem
17.3 comp.os.linux.security FAQ on identd
17.4 Why identd is a problem for LVS
17.5 tcpdumps of connections delayed by identd
17.6 There are solutions to identd problem in some cases
17.7 Turn off tcpwrappers
17.8 Identd and smtp/pop/qmail
18.
Squid Real-Servers (poor man's L7 switch)
18.1 Terminology
18.2 Preview
18.3 Let's start assembling
18.4 One squid
18.5 Another squid
18.6 Combining pieces with LVS
18.7 Problems
19.
Details of LVS operation (including Security, DoS)
19.1 Director Connection Hash Table
19.2 Port range: limitations, expanding port range on directors
19.3 DoS
19.4 DoS, from the mailing list
19.5 Writing Filter Rules
19.6 Active/Inactive connnection
19.7 Creating large numbers of InActConn: testlvs; testing DoS strategies
19.8 Debugging LVS
19.9 Security Issues
19.10 MTU discovery
19.11 Filesystems for realserver content: the many reader, single writer problem
19.12 netfilter hooks
20.
Failover protection
20.1 Director failure
20.2 Saving connection state on failover: Director demon for server state synchronisation
20.3 Realserver failure
20.4 ethernet NIC failure
20.5 Service/realserver failout
20.6 Mon for server/service failout
20.7 BIG CAVEAT
20.8 About Mon
20.9 Mon Install
20.10 Mon Configure
20.11 Testing mon without LVS
20.12 Can virtualserver.alert send commands to LVS?
20.13 Running mon with LVS
20.14 Why is the LVS monitored for failures/load by an external agent rather than by the kernel?
21.
Setting up Linux-HA for directors using rpms
21.1 linux-ha howto
21.2 Stop ldirectord from starting, ensure heartbeat starts on reboot
21.3 starting heartbeat and verifying functionality
21.4 Test your fail-over features, understand HA.
21.5 Configuration of mon - recommended
22.
Monitoring director throughput
22.1 ipvsadmm
22.2 /proc system (originally /proc/net/ip_vs_stats)
22.3 MRTG and LVSGSP
22.4 MIB
23.
Misc/FAQ/Wisdom from the mailing list
23.1 Multiple VIPs
23.2 Who is connecting to my LVS?
23.3 Limiting number of clients connecting to LVS
23.4 Setting up an LVS with inetd
23.5 How to bring down a realserver for maintenance (eg swap disks)
23.6 Howto turn your single node ftp/http server into an LVS without taking it off-line
23.7 shutdown of LVS
23.8 Other projects like LVS - Beowulf
23.9 Projects like LVS - Eddie
23.10 Any recommendations for a NIC?
23.11 NIC problems - eepro100
23.12 NIC problems - tulip
23.13 Thundering herd problem, when down machine(s) come on line
23.14 on the need for extended testing
23.15 loopback on Solaris
23.16 Having one director handling multiple LVS sites
23.17 Running multiple directors (each with their own IP)
23.18 Running clients (eg telnet) on realservers
23.19 ICMP
23.20 tcpdump
23.21 Bringing down aliased devices
23.22 Malicious attacks (SYN floods)
23.23 Does SMP help on the director?
23.24 Multiple IPs on the Director
23.25 Performance Hints from the Squid people
23.26 Testimonials
23.27 Transport Layer Security(TLS)
23.28 rcp and friends on LVS (better to use ssh)
23.29 Forwarding an httpd request based on file name not load (mod_proxy)
23.30 URL parsing
23.31 can I run my ipchains firewall and LVS on the same box?
23.32 Setting up a hot spare server
23.33 An LVS of LVSs
23.34 Connecting through multiple parallel links to the clients
24.
L7 Switching
24.1 from the mailing list about L7 switching
25.
Geographically distributed load balancing
26.
Useful things that have other place (yet)
26.1 Files which are kernel version dependant eg System.map and ipvsadm
26.2 Ramdisk
26.3 cscope
27.
Patches
27.1 machine readable error codes from ipvsadm
27.2 machine compatible ipsvadm entries
27.3 Threshhold patch
27.4 Martian modification patchs
27.5 fwmark name-number translation table
28.
FAQ
28.1 Help! My LVS doesn't work
28.2 My LVS doesn't work: ipvsadm shows entries in InActConn, but none in ActiveConn
28.3 My LVS still doesn't work: what do I do now?
28.4 initial connection is delayed, but once connected everything is fine
28.5 How fast/big should my director be?
28.6 What is the minimum hardware requirements for a director
28.7 Does the director handle ICMP?
28.8 I get "connection refused" from the client
28.9 Does SMP help?
28.10 When will LVS be ported to Solaris, xxxBSD...?
28.11 Is there a HOWTO in Japanese, French, Italian, Chinese...?
Next
Previous Contents