Being a hero

How to configure caching-nameserver (named) in cascading mode

2009-07-01T01:10:00.000-07:00

Mission

To configure a caching nameserver on a local machine which will cascade to another previously configured and functional nameserver (may or may not be caching. It'll generally be your ISP nameserver or the one provided by your organization).

Advantage

Reduces the delay in domain name resolution drastically as the requests for frequently accessed websites are served from cache.

Working

named gets a request for domain resolution.
It checks whether the request can be satisfied from cache. If the answer is in cache and not stale, the request is satisfied from cache itself saving a lot of time :)
If request can't be satisfied from cache, named queries the first parent. If it replies with the answer, then named will cache the response and subsequent requests for the same domain name will be satisfied from the cache.
In case first parent fails to reply, named will query the second parent and so on.

(The working is my understanding of caching-nameserver using wireshark as traffic analysis tool and caching-nameserver may not behave exactly as explained above.)

How to install

named is by default on most of the systems by the package name 'caching-nameserver'. If its not present on your system, install using

[root@localhost ~]# yum install caching-nameserver [ENTER]

How to configure

The main configuration file for named resides in /var/named/chroot/etc/named.caching-nameserver.conf which is also soft linked from /etc/named.caching-nameserver.conf . named configuration file supports C/C++ style comments.

For a caching nameserver which will cascade to another nameserver, there is nothing much to be configured. You need to configure "options" block. Below is a configuration file for a machine with IP address 172.17.8.64 cascading to two nameserver 192.168.36.204 and 192.168.36.210. The comments inline explain what each option does.

options {
 // Set the port to 53 which is standard port for DNS.
 // Add the IP address on which named will listen separated by semi-colons.
 // It'll be your own IP address.
 listen-on port 53 {127.0.0.1; 172.17.8.64;};
 // These are default. Leave them as it is.
 directory   "/var/named";
 dump-file   "/var/named/data/cache_dump.db";
 statistics-file "/var/named/data/named_stats.txt";
 memstatistics-file "/var/named/data/named_mem_stats.txt";
 // The machines which are allowed to query this nameserver.
 // Normally you'll allow only your machine. But you can allow other machines also.
 // The address should be separated by semi-colons. To allow a network 172.16.31.0/24,
 // the line would be
 // allow-query {localhost; 172.16.31.0/24; };
 // Don't forget the semi-colons.
 allow-query     { localhost; 172.17.8.64; };
 recursion yes;
 // The parent nameservers. List all the nameserver which you can query.
 forwarders { 192.168.36.204; 192.168.36.210; };
 forward first;
};
logging {
       channel default_debug {
               file "data/named.run";
               severity dynamic;
       };
};
zone "." IN {
 type hint;
 file "named.ca";
};
include "/etc/named.rfc1912.zones";

Start caching-nameserver

Now start the caching-nameserver using the following command

[root@localhost ~]# server named start [ENTER]

[root@localhost ~]# /etc/init.d/named start [ENTER]

To make named start every time your reboot your machine use following command

[root@localhost ~]# chkconfig named on [ENTER]

Using caching-nameserver

To use your caching-nameserver, open /etc/resolv.conf file and add the following line

nameserver 127.0.0.1

Comment all other lines in the file, so that finally the file looks like

; generated by /sbin/dhclient-script
#search wlan.iiit.ac.in
#nameserver 192.168.36.204
#nameserver 192.168.36.210
nameserver 127.0.0.1

Now your system will use your own nameserver (in caching mode) for resolving all domain names. To test if your nameserver use the following command

[root@localhost ~]# dig fedora.co.in [ENTER]

Now if you use that command for the second time, the resolution time will be around 2-3 milli seconds while first time it would be around 400-700 milli seconds.

Example

Below is two subsequent runs of dig for fedora.co.in . Notice the Query time.

[root@bordeaux SPECS]# dig fedora.co.in
; <<>> DiG 9.4.2rc1 <<>> fedora.co.in
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7839
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; QUESTION SECTION:
;fedora.co.in.                  IN      A
;; ANSWER SECTION:
fedora.co.in.           83629   IN      A       72.249.126.241
;; AUTHORITY SECTION:
fedora.co.in.           79709   IN      NS      ns.fedora.co.in.
;; ADDITIONAL SECTION:
ns.fedora.co.in.        79709   IN      A       72.249.126.241
;; Query time: 531 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 19 18:04:47 2008
;; MSG SIZE  rcvd: 79
[root@bordeaux SPECS]# dig fedora.co.in
; <<>> DiG 9.4.2rc1 <<>> fedora.co.in
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64233
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1
;; QUESTION SECTION:
;fedora.co.in.                  IN      A
;; ANSWER SECTION:
fedora.co.in.           83625   IN      A       72.249.126.241
;; AUTHORITY SECTION:
fedora.co.in.           79705   IN      NS      ns.fedora.co.in.
;; ADDITIONAL SECTION:
ns.fedora.co.in.        79705   IN      A       72.249.126.241
;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Nov 19 18:04:51 2008
;; MSG SIZE  rcvd: 79
[root@bordeaux SPECS]#

How to install and configure HAProxy as an HTTP load balancer

2009-06-30T21:13:00.000-07:00

HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for web sites crawling under very high loads while needing persistence or Layer7 processing. Supporting tens of thousands of connections is clearly realistic with todays hardware. Its mode of operation makes its integration into existing architectures very easy and riskless, while still offering the possibility not to expose fragile web servers to the Net.

This post will explain how to install HAProxy on FreeBSD and how to configure it as a simple HTTP load balancer.

1. Getting HAproxy

The first step is to download HAProxy. You can find the latest version at this address:

http://haproxy.1wt.eu/#down

As of writing this post, the latest version is 1.3.16, which you can find here:

http://haproxy.1wt.eu/download/1.3/s...-1.3.16.tar.gz

2. Prepare for installation

Log on your FreeBSD machine and change to the temporary directory of your choice. You can now issue the easy following commands:

$ wget http://haproxy.1wt.eu/download/1.3/s...-1.3.16.tar.gz
$ tar xvfz haproxy-1.3.16.tar.gz
$ cd haproxy-1.3.16

HAProxy is now downloaded, unpacked and ready to be compiled.
3. Compile HAProxy

If you are running FreeBSD, you can simply issue the following command:

$ make -f Makefile.bsd REGEX=pcre DEBUG= COPTS.generic="-Os -fomit-frame-pointer"

If you are running another version of Linux, please refer to the README file (included with HAProxy) to find the good way to compile HAProxy on your system.

4. Install HAProxy

Once HAProxy is compiled, you will find a light-weight, standalone "haproxy" executable.

You can copy it where you want, it's your choice! Something like this can be a good choice:

$ cp haproxy /usr/local/sbin

Why? Simply because /usr/local/sbin is usually in $PATH, so haproxy will then be easy to run simply by typing "haproxy".

5. Configuring HAProxy

The configuration of HAProxy, like its installation, is really easy. In fact, it can be very easy, or very complex. HAProxy is VERY flexible and it has literally thousands of parameters you can tweak. You can find the whole documentation on the HAProxy official Web site there:

http://haproxy.1wt.eu/#docs

As of writing this post, the latest official documentation is there:

http://haproxy.1wt.eu/download/1.3/d...figuration.txt

For this post, we'll keep it simple and configure a simple load balancer.

You can put the HAProxy configuration file where you wish, and name it as you wish! This post will be using this path/filename:

/etc/haproxy.conf

And here's our simple configuration file:

global
maxconn 4096
pidfile /var/run/haproxy.pid
daemon

defaults
mode http
retries 3
option redispatch
maxconn 2000
contimeout 5000
clitimeout 50000
srvtimeout 50000

listen GALAXY aaa.bbb.ccc.ddd:80
mode http
cookie GALAXY insert
balance roundrobin
option httpclose
option forwardfor
stats enable
stats auth myuser:mypass
server EARTH 192.168.0.2:8080 cookie GALAXY_SERVER_01 check
server MOON 192.168.0.3:8080 cookie GALAXY_SERVER_02 check

> aaa.bbb.ccc.ddd should be the public IP of your server.
> 192.168.0.2 should be the LAN IP to your server 1.
> 192.168.0.3 should be the LAN IP to your server 2.

Everything that is in CAPS is customizable:

* GALAXY
* EARTH
* MOON
* GALAXY_SERVER_01
* GALAXY_SERVER_02

Basically, this configuration file is defining a group ("listen" block) called GALAXY, which contains 2 servers: EARTH and MOON. The "roundrobin" balance option is telling to HAProxy to alternate between the 2 servers all the time. For the other options in the global, defaults and listen blocks, they're pretty standard. If you need to tweak those or to add options to suit your needs, I'm sure HAProxy can do it! Simply refer to the whole documentation.

Cookie

When the user will reach the GALAXY group (using http://aaa.bbb.ccc.ddd), the cookie GALAXY will be created and the server ID specified for "cookie" in the servers definitions will be stored in it (GALAXY_SERVER_01 or GALAXY_SERVER_02).

Then, for the whole session, HAProxy will read the cookie and force the use of the server stored in it.

This behavior is controlled by the "cookie GALAXY insert" line and the "cookie GALAXY_SERVER_XX" parts in the GALAXY group block. If you don't want this feature, simply remove or comment (using a "#") these parts.

Stats

There's a pretty complete load balancer stats page built-in HAProxy. You can reach it there:

http://aaa.bbb.ccc.ddd/haproxy?stats

With the configuration above, the username will be "myuser" and the password "mypass". If you don't want the stats to be enabled, you can simply remove or comment these lines:

stats enable
stats auth myuser:mypass

Nginx

In the configuration above, we used the port 8080 for the local/private Web servers. If these Web servers are running Nginx, you will need to include the following configuration in your http, server or location block (nginx.conf):

port_in_redirect off;

This is needed because if you try to reach this:

http://aaa.bbb.ccc.ddd/test (NO ending slash)

Then Nginx will try to redirect you there:

http://aaa.bbb.ccc.ddd:8080/test/

Which will obviously not work. So if you're running Nginx and that you decided to use a port different of 80, you will need this setting.

6. Starting HAProxy

To start HAProxy, simply issue the following command:

haproxy -f /etc/haproxy.conf

For the initial testing, you might want to disable the cookie in the configuration file and test your Web page to actually see that you're switching from a server to another.

Conclusion

That's it! You've got a high-performance, high-scalability and highly-tweakable load balancer configured in a couple of minutes!

Publibed by Michel Nadeau, 03-26-2009