Let us first install Ubuntu Server, and then, from another system, connect using the PuTTY SSH client to install FTP support and phpMyAdmin, and get a phpBB website up and running.

Setting up Ubuntu Server

Download the ISO from the Ubuntu website. Burn a CD, or prepare a USB boot disk, as per your preference. Boot the server machine from it. The boot menu is shown in Figure 1. Select Install to the Hard Disk. The installer will boot and ask you to select your language. Select English and continue.

Figure 1: Ubuntu Server boot menu

I will fly through the next few screens, since they’re all what you usually see during any standard OS installation. For the keyboard layout, the default US layout works for us. The installer will then load components and configure hardware. It will also try to configure the network interface using DHCP. If that fails, you will be asked to manually enter network information like Host IP, Gateway, Subnet Mask, etc. It will also prompt you to enter the hostname; use a name you would like the server to be known by on the network. For this article, I have used ‘mylamp’ with IP address 210.22.23.24.

Next comes partitioning. I used the Erase Entire Disk option; if you have any specific partitioning requirements, feel free to use Manual partitioning. After this stage, under Clock Configuration, you will be asked about your system time setting. The new user (non-root) creation screen is next; enter your full name, user name (I used ‘lampuser’) and password.

The installer will now show you a software selection screen; select LAMP Server and OpenSSH Server.
Wait as the progress bars fill up — first for the base system, then for the package installation, and then for the GRUB bootloader. Once all these are done, it will tell you that it is time to reboot and take your new server for a ride!

Figure 2: Reboot prompt

Server installation is now complete. Remove the disc and return to your desktop. Let us now remotely configure the server for a LAMP website.

Preparing for deployment

Whether your desktop runs Windows or Linux, you can use the PuTTY SSH client. Download it, fire it up, and enter the connection details for your server (Figure 3).

Figure 3: PuTTY connection screen

Click Open to connect to the server. This will bring up a terminal prompt that asks you to enter the username. It may throw up an error similar to the one in Figure 4. Just click on Yes. Once the password is accepted, you will be greeted with a shell prompt. The first thing to do is set up a FTP server so we can upload files to the server. Next, let us install phpMyAdmin to manipulate MySQL databases.

Figure 4: Server host key alert

FTP service

FTP support is by the vsftp daemon; installation is straightforward: sudo apt-get install vsftpd. After installation, tweak the configuration file to point the default path to /var/www (the Apache ‘Webroot’ for hosted content/sites). Edit /etc/vsftpd.conf as the root user, with a text editor and Make sure the settings match those shown below:

write_enable=YES
local_enable=YES
local_root=/var/www
file_open_mode=0777

Restart the FTP service to bring the configuration changes into effect: sudo restart vsftpd.

Installing phpMyAdmin

phpMyAdmin is a great tool for administering MySQL databases. Install it using the following command: sudo apt-get install phpmyadmin.
We can now exit the PuTTY session, using the exit command.

Deploying a LAMP website

For the purpose of this article, let us deploy a standard version of phpBB — the most famous AMP-powered bulletin board system and forum content management system (CMS). I will not go into the installation details, since the aim here is to learn how to generally deploy a LAMP website remotely — however, you can refer to this article for installation instructions. Download the archive from the phpBB home page and extract its contents.

Now, using an FTP client, connect to the server with the username and the password used in the PuTTY session. Create a folder called phpBB on the server, and copy the extracted files and folders to the root of that directory. Open a browser and navigate to http://210.22.23.24/phpmyadmin (replace the IP address with yours) to bring up the phpMyAdmin page (Figure 5).

Figure 5: phpMyAdmin status page

Create a new database named phpbb — go to the Database tab and use the Create New Database section at the bottom. You can now proceed to a site-specific installation by navigating to http://210.22.23.24/phpbb. Follow the instructions on the page.

We have now successfully set up our server, and learnt how to remotely control it! I hope you had fun learning, during this process. Until next time, cheers!

WordPress runs on something knows as an AMP (Apache, MySQL, PHP) stack. Apache is your web server software. MySQL is your database engine. And PHP is the language WordPress is written in. Out of these three, the web server is the only component that you can replace with something else. Most people use Apache just because it’s very simple to set up — and that WordPress supports it out of the box. But what if you could tweak around and install another web server that is immensely faster than Apache?

Enter Nginx. Pronounced Engine X (Engine 10), this web server software was created by a Russian developer to solve something known as the C10K problem. Most traditional Web servers (Apache included) uses threads to serve each request. I know this is kind of technical, but you’ll see — creating new threads is a very expensive operation for operating systems, and there’s a limit to the number of threads that can can be spawned. As such, the theoretical limit for Apache is 10,000 concurrent connections.

Nginx is a breed of new Web servers that use an event-driven technique to Web serving. Jobs are performed asynchronously (which basically means that tasks are executed in parallel, and that one task doesn’t have to complete for the next to commence) — which makes it immensely fast. Every new connection uses a small, and more importantly predictable amount of memory. The end result? A fast Web server which uses very less memory, and can serve more than 10,000 connections concurrently.

‘Nuff said.

Will WordPress Like Nginx?

Good question. One of WordPress’s quirks is that it uses files named .htaccess (which is an Apache-specific feature) which are used to control redirections if you use pretty permalinks. If you don’t use pretty permalinks (e.g., your blog has links like www.myblog.com/index.php?p=100&d=200) then you’re pretty much set. If you do use pretty permalinks, (like www.myblog.com/2011/01/01/happy-new-year.html) then you’ll need some additional configuration (which I’m going to show you here).

Now there’s another problem. WordPress is very extensible, which means it supports plugins. WordPress itself doesn’t rely much on .htaccess files other than to handle redirections, but plugins may use it to handle access-control and permissions. Now the list of such plugins is infinitesimally small, and any such plugin which uses .htaccess files is badly designed anyway, since plugins shouldn’t make assumptions about the Web server its running under, but if you must use those plugins, then you’ll either have to do without the access-control (you can chmod the files appropriately), or you’ll have to do without Nginx.

Us? We still aren’t big enough to warrant moving to VPS, so we use whatever server out hosting provider provides. But here’s some trivia for you — WordPress.com itself runs on Nginx.

Getting Started

I’m assuming you’ve got a VPS with Apache, MySQL and PHP already hosting your blog. What we’ll do is install Nginx and the PHP connector, and switch the site over from Apache to Nginx in a single step that will typically involve less than a second of downtime.

Most servers run either Debian/Ubuntu Server or RHEL/CentOS. If you’re one of the lucky ones to afford SUSE Linux Enterprise, you’ll have to figure out how to install Nginx yourself.

Nginx development takes place very fast. For this reason, your distribution binaries are generally going to be way out of date. But there’s a way out — Nginx provides official repositories (yes, complete repos, not just binaries) for many operating systems.

If you run Debian Squeeze, add these lines to your /etc/apt/sources.list:

deb http://nginx.org/packages/debian/ squeeze nginx
deb-src http://nginx.org/packages/debian/ squeeze nginx

If you run Ubuntu Server, the lines for /etc/apt/sources.list are:

deb http://nginx.org/packages/ubuntu/ lucid nginx
deb-src http://nginx.org/packages/ubuntu/ lucid nginx

Needless to say, you should be running the LTS version of Ubuntu Server. Nginx also has official versions for RHEL and CentOS. To add their Yum repo, create a new repo file at /etc/yum.repos.d/nginx.repo with the contents (for CentOS):

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

And for RHEL:

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/$releasever/$basearch/
gpgcheck=0
enabled=1

Now you can install Nginx using your package manager. For Debian/Ubuntu, run:

sudo apt-get update && sudo apt-get install nginx

You’ll get Certificate errors for Nginx because apt does not automatically add certificates for repositories. The certificate links are in the Appendix section at the end of this article. Anyway, for RHEL/CentOS, run:

sudo yum install nginx

You now have Nginx. You’ll have to install the PHP FastCGI connector now.

You’ll need to add a repository if you’re on Debian Squeeze. Add these lines to the /etc/sources.list file:

deb http://packages.dotdeb.org stable all
deb-src http://packages.dotdeb.org stable all

You’ll also need to add a repository if you’re on RHEL/CentOS 5. Run the following commands:

sudo rpm -Uvh http://repo.webtatic.com/yum/centos/5/latest.rpm
sudo yum --enablerepo=webtatic update

Now, to install, on Debian/Ubuntu type:

sudo apt-get update && sudo apt-get dist-upgrade
sudo apt-get install php-fpm

And on CentOS/RHEL, type:

sudo yum --enablerepo=webtatic install php-fpm

You’re officially on the road. All you need to do now is configure Nginx, start php-fpm, and then in one step stop Apache and start Nginx.

Configuring Nginx

I’m assuming your Webroot is at /var/www. If not, adjust the following file appropriately. Open up the file /etc/nginx/sites-available/default, and replace everything in there with the following:

server {
    listen       192.168.1.1:80;                # Your server's public IP address
    server_name  example.com;                   # Your domain name
    root         /var/www/;                     # Absolute path to your WordPress installation

    try_files $uri $uri/ /index.php;            # Handle permalinks
    if (!-e $request_filename) {
          rewrite ^.*$ /index.php last;         # More code to handle permalinks
    }                                           # This is what the HTACCESS is basically for

    location ~ \.php$ {
        include        fastcgi_params;
        fastcgi_pass   localhost:9000;
        fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
    }
}

That’s it. Now, we’ve got to start php-fpm, by doing:

sudo /etc/init.d/php-fpm start

And then in one fell swoop, switch the Web server:

sudo /etc/init.d/httpd stop && sudo /etc/init.d/nginx restart

Don’t worry about the restart thing in the last command — some ditributions think its really cool to start nginx on port 8000 as soon as is it’s installed.

Now access your site. Happy?

You can now go ahead and remove all Apache packages from your system. That should be it. Oh, you should go ahead and install this plugin.

What If It Doesn’t Work?

Leave Comment below, and we’ll help you out.

Appendix: Certificates For Debian/Ubuntu

To add the DotDeb certificate, do:

wget http://www.dotdeb.org/dotdeb.gpg
cat dotdeb.gpg | sudo apt-key add -

And to add the Nginx certificate, do:

gpg --recv-key 7BD9BF62
gpg --armor --export 7BD9BF62 | sudo apt-key add -

Before installing iSCSI (Internet Small Computer Systems Interface) on Openfiler, let’s cover some of its basics.

The SCSI standard is used to connect high-speed peripherals (especially hard disks) to computers. It can connect up to seven devices to a single controller. Over a period of time, the requirements have changed to connect various devices on a network rather than on an individual computer. This required the SCSI protocol to be mapped to network transport protocols.

iSCSI enables SCSI packets to be transferred over TCP/IP (Ethernet). Just think about it — iSCSI enables access to disks across the network — not only to the local network, but even across the Internet. iSCSI requires an initiator (the client system, in this case, a desktop PC) and a target (the storage device — here, Openfiler).

SCSI data is transferred via an iSCSI session in five layers. The bottom three layers, Transport (TCP), Internet (IP) and Network Interface (Link) belong to the TCP/IP stack. iSCSI sits between the SCSI and transport layer, encapsulating and de-encapsulating packets.

Consider a case of a write cycle — the iSCSI initiator will send SCSI packets through the SCSI layer down to the Link layer. After data transfer to the target, it will go up from the Link layer to the SCSI layer to be written on the disk. The many important benefits of iSCSI are:

• It uses standard Ethernet devices and works on TCP/IP
• It eliminates distance limitations; remote replication over the Internet is possible
• It is documented in RFC3720 by the IETF (Internet Engineering Task Force)
• It is supported by Intel, Cisco, various Linux flavours and other OSs

Creating an iSCSI target

Let’s now proceed to the installation of an iSCSI target in Openfiler. Open the Openfiler Web interface; proceed to Services –> iSCSI Target Server and enable the service. To create an iSCSI volume: Volume –> Add Volume –> Volume Name iSCSI1 –> 100000 MB –> Volume type –> iSCSI. Add a new iSCSI target from Volumes –> Target. (See Figure 1; I used the default target name, you can edit and change it, if required.)

Figure 1: Add new iSCSI target

The target now needs to be mapped to a LUN (Logical Unit Number) used to identify a SCSI disk, from a stack of a maximum of seven disks. Click the MAP button to complete this step (Figure 2).

Figure 2: LUN Mapping

Under Network ACL, allow access to the required network subnet. Complete the setup by assigning the CHAP Authentication username and password (Figure 3).

Figure 3: CHAP Authentication

The Openfiler iSCSI set-up is now complete.

The client setup

Download and install iSCSI Initiator on your client box. Go to the Discovery tab and add a target portal (the IP address and default port 3260). See Figure 4.

Figure 4: Add target portal

Proceed by clicking on Targets –> Logon –> Advanced –> CHAP logon information –> User name and Password as defined earlier in iSCSI target setup in Openfiler (see Figure 5).

Figure 5: CHAP log-on information

You should see the new device detected. Format the device, make partitions, format them, and start using your Openfiler iSCSI drive as if it were a local machine drive.

Figure 6: Dynamic disk discovered

Figure 7: iSCSI status

You can check open sessions in Openfiler by selecting Status –> iSCSI targets (Figure 7).

In a nutshell, Openfiler provides an excellent iSCSI target, usable by various client operating systems as local storage. The various deployment scenarios detailed on the Openfiler website are:

To sum up, Openfiler is a great file server, and supports various configurations including:

• Various access methods such as FTP, NFS, CIFS, HTTP/DAV, rsync, etc.
• Network access controls.
• Group- and user-based quota allocation.
• Various RAID levels to improve data reliability.
• Bonded interfaces to increase network throughput/Ethernet redundancy.
• Intuitive Web interface for configuration.
• Uses the full potential of 64-bit CPUs for excellent performance (the latest release is only for 64-bit CPUs).
• Scalability to a capacity of more than 60 TB.
• Last, but not the least, it is open source, with great community support.

The latest stable release of Openfiler is 2.99; one of the most important additions is GUI-based configuration for High-Availability (HA) clusters. Keep a watch on this column for more articles on various Linux distros!

References

• Openfiler website
• Openfiler community forums
• Linux iSCSI initiator downloads

The Openfiler RAID 1 setup that we discussed last month uses mirroring for data volumes, and a single additional OS hard disk. Here, to recover from a crashed OS disk, one must replace the disk, reinstall the OS and restore backups. The RAID data volumes remaining intact, the server is back in action after that. Disadvantages here are the need for an additional OS disk (and its power consumption) and having to reinstall the OS to recover. It is also possible to have both OS partitions and RAID 1 volumes on just two disks, in two steps:

1. Create required OS partitions, configure them in RAID 1, modify GRUB to include the second hard disk as a bootable disk.
2. Create RAID 1 volumes, and assign user rights from the Web-based Openfiler GUI.

Hardware configuration used was an Intel Dual-Core CPU-based PC with 2 GB RAM, 2×500 GB SATA hard disks, and a DVD drive for installation. Openfiler version 2.3 was used.

Start by booting from the Openfiler CD, and continue with GUI-based installation. The most important step required for RAID 1 installation is creating the RAID partitions manually using Disk Druid. Check the following table for the brief layout.

Figure 1: Creating RAID partitions

Figure 2: Verifying RAID 1 partitions

The remaining installation options are straightforward — Ethernet interface parameters (don’t forget to set the interface to activate at boot); setting timezone and assigning a root password. Once done, open the Web GUI from any computer (https://ipaddress:446). Verify the RAID configuration –> navigate to Volumes –> Software RAID and verify all the three RAID volumes created earlier. If any change is required, now is the time to fix the partitions.

Next, log in as root and update Openfiler with the following commands:

conary updateall
conary update conary

Confirm that all updates have been applied, by running the same command till you receive the message: “no new troves were found”.

One important step remains. The Openfiler setup has only the GRUB register hda as a boot drive; hdb should also be made bootable. To do this, use the following commands (apply GRUB commands corresponding to the detected hard disks; here, hda was detected as sda and hdb as sdb):

root@localhost:~# grub
grub> device (hd0) /dev/sda
grub> root (hd0,0)
grub> setup (hd0)
grub> device (hd1) /dev/sdb
grub> root (hd1,0)
grub> setup (hd1)
grub> quit
root@localhost:~#

Figure 3 lists the terminal output for the above commands.

Figure 3: Grub commands

Verifying the installation

You can remove one hard disk at a time, and check whether the system boots properly. Once up and running using a single disk, check under Volumes –> Software RAID; you will see the “clean” and “degraded” volumes (which remain unsynchronised), as shown in Figure 4.

Figure 4: sda removed from RAID1 Array

After re-installing the removed disk, proceed to add the member to the RAID 1 array, for synchronisation to start. Then select Add Member from the ADD column; carefully check the partition sizes, and add the required RAID 1 member to the array. You can see the synchronisation process status (Figure 5) immediately after this. If a new hard disk is added, you will require to create the partitions and then add them to the RAID array.

Figure 5: RAID synchronization

The remaining setup requires you to configure network access via System –> Network Setup; create the Network Access Configuration to allow access to the full subnet.

Important tips

1. Use only manual partitioning. Auto partitioning will use the full disk for the OS, leaving no space for data storage.
2. Make all software RAID partitions primary partitions — check the “Force primary” checkbox.
3. By default, logs are stored on the root (/) partition. If it runs out of space, the system stops functioning. To protect against this, a separate /var/log partition is recommended.
4. Do not configure swap partitions as either RAID type, or primary partitions.
5. You must configure GRUB to allow booting from both disks — else, if hda fails, the system will not boot — and OS redundancy by RAID 1 will be useless.
6. Further configuration steps, with screenshots, are given in the August and September articles. Refer to those for more information.

Enabling FTP

In the Web GUI, create a user group: Accounts –> Administration –> Group Administration; create ftpgroup, overriding GID to 1001. Then create the ftpuser1 user in Accounts –-> Administration –> User Administration, overriding UID to 501. Go to Services –> FTP Server and click Enable.

After you add volumes to your volume group and create an ext3 partition (of 150 GB, for instance), use Make share as FTP and configure ftpgroup as the primary group (PG); allocate read-write (RW) access to ftpgroup. Under the host access configuration, select RW access for FTP to the Internal PCs group. Now, you can access the FTP share with standard FTP clients from your internal network.

Accessing the FTP server from the Internet

To access this server from the Internet, you require to set up a small network with a firewall capable of port forwarding. IPCop is an excellent GNU firewall distribution for this purpose. Internal PCs/servers access the Internet via the firewall, which passes requests from internal PCs, to the requested Internet address, and passes replies back to the originating internal PCs. To allow external computers to access internal servers, we use a firewall feature called port forwarding. For more information, see Wikipedia on port forwarding.

Basically, connections to the firewall at certain port numbers used for different services can be passed on to servers behind the firewall, and the servers’ responses relayed back to the computer on the Internet that requested the service. Some well-known ports are TCP 20, 21 for FTP data and FTP; TCP and UDP 53 for DNS; TCP 80 and 443 for HTTP/HTTPS; and UDP 161 for SNMP.

Prudent firewall installations first block all incoming Internet traffic to the internal network, and vice versa. Then, depending on the requirements, access is allowed for the requested services.

Configuring IPCop for port forwarding

Internet traffic to TCP ports 20 and 21 on the public Internet interface must be forwarded to the internal FTP (Openfiler) server. Port forwarding configuration instructions will differ depending on firewall type. Figure 6 shows IPCop’s port forwarding screen. Access can be restricted to a particular source IP address/range of IP addresses by entering the corresponding IP or network address in the Source IP field, as you can see in Figure 6. The ports are forwarded to the internal FTP server only if the originating IP address is the configured source IP.

Figure 6: IPCop port forward

That is all, folks! Now, the internal Openfiler FTP server can be accessed from preconfigured public IP addresses with any standard FTP client software such as Filezilla or CoreFTP.

Once installation is complete, don’t forget to back up your system and LDAP configuration via System–Backup/Restore –> Download and Services –> LDAP setup –> Backup LDAP.

For several important installation details, please refer to the August and September articles.

PHP installation is similar to Apache’s installation process, with GNU autoconf involved (./configure script). We’ve already discussed setting up the optimisation flags for Apache (CFLAGS, CXXFLAGS) and hence I won’t describe the process here — please read the earlier article in this series from June 2011 for that. To install PHP 5.3.6 (the latest available, as of this writing), you need (these requirements have been taken from the INSTALL file in the tarball):

• ANSI C Compiler (GCC on Linux)
• Flex 2.5.4
• Bison 1.28, 1.35 or 1.75
• Web server
• Module specific components (gd, pdf libs, etc)

The PHP source configuration is a bit tricky; extensions enabled by default (or by specifying an option) are compiled statically. You need to take special care about this. Statically compiling extensions means the extension will be embedded into the PHP binary itself; you cannot disable it at will. If you can’t disable the extension, it causes memory wastage if you aren’t using the extension in any of your applications on the server.

Telling the PHP build system to build all extensions takes effort. You have to state “shared” in the extension enabler switch: --with-EXT=shared or --enable-EXT=shared. There are two switches, --enable-shared and --disable-static that can be passed to ./configure, but they never worked for me — the build system always compiled extensions statically.

Get the PHP source tarball from its official website, and also note the MD5 sum shown on the download page, for file verification (similar to what we did when installing MySQL, in July). Do not skip file verification; often, a proper tarball wasn’t downloaded and my installation had problems like memory leaks, etc.

Extract the archive to get the php-5.3.6 subdirectory. Before we begin configuring the source, if you want to install some PECL extensions directly with PHP (if you choose not to install PEAR/PECL with PHP), then you must place them in the ext directory, and subsequently run the autoconf tool present in the source code folder itself, with ./autoconf).

Configuring and building PHP

So here’s the table of options that the PHP ./configure accepts, and a short description of what they do. This list isn’t exhaustive; run ./configure --help | less to get a full list. Please note that you have to set the hardware-dependent CFLAGS, CXXFLAGS for optimisation.

Another thing to remember is that if an option’s disable version is shown in the exhaustive list, then its enable counterpart also exists and can be used. For example, --disable-libxml is listed, but not --enable-libxml — yet it can be used.

After you have completed configuration with ./configure and the options, run make and make install to install PHP, keeping your fingers crossed that it builds and installs successfully.

PHP configuration

Two configuration file candidates for php.ini are provided in the root of the source tarball: php.ini-production and php.ini-development. Unless you will be testing and developing on this system, choose the production version.

Copy php.ini-production to /usr/local/etc/php.ini or your /php.ini and start modifying it with your favourite text editor. Here is a table of options that can be specified in php.ini, along with a description of each:

In addition to this configuration, to get maximum performance, you should install an opcode cache like APC, XCache, eAccelerator, etc. Remember, PHP is an interpreted language; source is compiled every time the script is run. Caching compiled code saves CPU cycles.

You can also use Facebook’s Hiphop to convert PHP code to C++, which will help you boost performance immensely — but that is beyond the scope of this article.

Configuring PHP with Apache

There are three methods to configure PHP on Apache: CGI (the worst option), mod_php (better), and mod_fastcgi (the best). I’ll discuss only mod_php and mod_fastcgi methods.

The mod_php method

This is the most commonly used method to configure PHP with Apache. It is applicable only if you built the Apache SAPI for PHP (--with-apxs2). Add these lines to httpd.conf to enable PHP support with mod_php:

LoadModule php_module modules/libphp5.so
AddHandler php5-script .php
AddType text/html .php

If you have read the PHP documentation for installation, then you might know that PHP recommends the use of application/x-httpd-php for PHP scripts — but that never worked properly for me; hence, I won’t advise it.

The mod_fastcgi method

With PHP-FPM, FastCGI usage has been increasing ever since, because of its advantages over CGI and mod_php. FastCGI’s advantage over others is that the PHP processing stack is separated from the server — there are some processes running separately, independent of the Web server on the machine where PHP is being used itself, or some remote destination. Because of this, opcode caches are able to share data across multiple processes, and their data is not destroyed when you change the Web server configuration and restart (or reload) it. Also, you can have dedicated PHP processing machines on the network to enable load sharing — very useful if you have a heavily trafficked site.

Again, there are two methods to use PHP with mod_fastcgi; the older uses spawn-fcgi or something similar, which sets up a PHP interpreter stack on a TCP port or a UNIX socket. The newer method uses PHP-FPM. mod_fastcgi is not provided with the default Apache installation — you have to download and install as per the INSTALL file in the tarball. Sometimes, the module is not automatically installed to the Apache modules directory; you need to copy it there from /.libs/. After you have installed the module, to enable it in Apache, add (or uncomment) this directive:

LoadModule fastcgi_module modules/mod_fastcgi.so

The older method using spawn-fcgi

Run the following spawn-fcgi command (spawn-fcgi is a part of the lighttpd project, but is available as a separate package in many distributions):

spawn-fcgi -f /usr/local/bin/php-cgi -s /tmp/php.sock -u apache -g apache -C 10

This will launch the PHP interpreter stack (consisting of 10 processes and one manager process) that will listen for requests at /tmp/php.sock. You can also make the stack listen on a TCP port using -p (port) and -a (address) option instead of -s. These are mutually exclusive.

Add these lines to the Apache configuration to enable PHP:

AddHandler php-fcgi .php
FastCgiExternalServer /var/www/cgi-bin/php.external -socket /tmp/php.sock -pass-header Authorisation
Action php-cgi /cgi-bin/php.external

The above lines are the same if you use FPM — it is just that you may have to change the socket path if you specify a different path in FPM configuration.

You need to configure FastCgiExternalServer‘s path into a cgi-bin directory, or you may have to set the ExecCGI option for PHP scripts if php.external is not in a cgi-bin. There are various ways to set up php-fastcgi on Apache, so use the one that works for you. Use Google to search for more information.

FPM configuration

A sample FPM configuration may be placed in /usr/local/etc or <PREFIX>/etc. The default location seems to be /etc/php-fpm.conf (as per my installation, on Gentoo), but that may differ across distributions. Look for the sample configuration in /etc and /usr/local/etc or <PREFIX>/etc; copy it to php-fpm.conf in the same directory, and start modifying it. The configuration file is well documented, so I’ll describe options specific to the PHP pool required to get PHP running, not the others.

A pool configuration section starts with the pool name in square brackets:

[www] ; Pool name = www

To make a pool listen on a socket or TCP port, you need to use the listen option:

listen = <path-to-socket|address:port>

Also note the options listen.owner, listen.group and listen.mode. If you make a mistake while configuring these, you may have a setup that doesn’t work. These should be configured so that the user and group Apache is running under should be able to read and write to the socket. The options user and group in the FPM configuration specify the user and group PHP runs as. This means that the user and group specified there should have read-write permissions to the directories/files they may be processing.

Other options you need to configure are pm, pm.max_children, pm.start_servers, pm.min_spare_servers and pm.max_spare_servers. The pm=dynamic setting is best — it will launch PHP processes when required (of course, min_spare_servers number of processes will always be running).

Other options depend on your server’s capacity. Read the documentation provided in the file itself to configure them.

The earlier article used a standard PC based on an Intel dual-core 2.8 GHz CPU, 1 GB RAM, a single 100 MBps Ethernet card, and a SATA 80 GB hard disk. To create bonded interface and software RAID configurations, one Fast Ethernet card and two 500 GB hard disks were added to this original hardware.

Before starting the actual set-up, let us understand the basics of bonded interfaces and various RAID levels.

Bonded interface: A standard SATA II hard disk gives a write speed of 6 Gigabits per second. Fast Ethernet transfers 100 Megabits per second and Gigabit Ethernet 1 Gigabit per second. Thus, the network speed is six to sixty times lower than hard disk write speeds. Bonding two interfaces into a single one bridges this speed difference. Here, typically, two independent Ethernet interfaces with their own IP addresses are bonded to be accessed as a single interface by an additional third IP address, thus creating twice the bandwidth. Bonding is also possible for more than two interfaces. Bonded interface also provide redundancy if one of the Ethernet cards interface fail.

RAID: Openfiler supports RAID levels 0, 1, 5, 6 and 10, which are explained in brief for ready reference. For more information, please see the Wikipedia article on RAID.

1. RAID 0 (block-level striping without parity): Given two hard disks (the minimum), a block of data to be written is split into two equal parts and written on the two disks. This almost doubles write performance. There is no redundancy; if one disk fails, all data is lost.
2. RAID 1 (mirroring without parity or striping): This requires a minimum of two hard disks. Whatever is written on Disk A is duplicated on Disk B, providing 1-to-1 redundancy. This level can sustain failure of one disk without losing data. When a faulty disk is replaced, the RAID controller automatically synchronises the data.
3. RAID 5 (block-level striping with distributed parity): This level requires a minimum of three hard disks and tolerates failure of one disk without loss of data. Given three disks, A, B and C, and three data blocks to be written on them, Table 1 shows how the blocks are stored, after splitting each into two equal parts.
   

   Table 1: RAID 5 storage structure
   Block	Data	Parity
   First block	Disk A and B	Disk C
   Second block	Disk B and C	Disk A
   Third block	Disk C and A	Disk B

   The parity information is calculated using the logical Exclusive-OR (XOR) function. Table 2 helps understand the function, and how it is used to reconstruct data to be written on a replacement hard disk.

   Table 2: XOR truth table and disks where data and parity are written
   Data A	Data B	Parity
   0	0	0
   0	1	1
   1	0	1
   1	1	0
   Disk A	Disk B	Disk C

   With XOR, when data A and data B are equal, the parity is 0; when not equal, parity is 1. Now let’s assume Disk A fails. At this point, we have correct values for data B and parity, stored on disks B and C. After the faulty disk is replaced, the RAID controller recalculates and writes the correct data to it, using the following rules:

   • If Data B and Parity both are 0, Data A is 0.
   • If Data B is 1 and Parity is 0, Data A is 1.
   • If Data B is 0 and Parity is 1, Data A is 1.
   • If Data B and Parity both are 1, Data A is 0.

   There are two disadvantages of using this RAID level:

   • During synchronisation (rebuilding the RAID array), the array’s read-write performance is greatly reduced due to calculation overhead.
   • If any other disk fails during synchronisation, the whole array is destroyed.
4. RAID 6 (block-level striping with double distributed parity): This level uses two parity disks instead of one as in the case of RAID 5; thus, it can tolerate failure of two hard disks in an array, at a time.
5. RAID 10 (stripe of mirrors): Combines RAID 0 and 1. Two RAID 1 arrays are striped using RAID 0, thus providing speed as well as redundancy.

Now, after the basics, let’s proceed with Openfiler configuration of bonded interfaces and RAID levels.

Bonded interface configuration

First, the prerequisites: a minimum of two Ethernet cards (eth0 and eth1) and two additional IP addresses in the same range as the Openfiler installation (the third IP is for the bonded interface bond0).

Now that we have the bare minimum, open the Openfiler Web interface (https://ipaddress:446), log in and go to System –> Network Interface Configuration. Edit the eth1 configuration, and enter the IP address and subnet mask for this interface (see Figure 1).

Figure 1: Network interface configuration

Proceed with bonding the two interfaces once the second interface is configured (Figures 2 and 3).

Figure 2: Bonding two interfaces

Figure 3: Network bonding configuration1

Continue to give the bonded IP address and subnet mask, leaving all other parameters at their default values (Figure 4).

Figure 4: Network bonding configuration2

Select the bonding type as balance-alb (adaptive load balancing). Various other modes available provide combinations of load balancing methods and failover. Verify the new network configuration (Figure 5).

Figure 5: Configured bonded interfaces

Now, you won’t be able to access the Web GUI via the eth0 address. Instead, connect to https://bond0address:446.

RAID configuration

Now, let us proceed to add RAID volumes to the box. We shall create RAID 1 100 GB volumes on the two new 500 GB hard disks.

Proceed to Volume –> Block devices. This (Figure 6) reflects all hard disks. Openfiler detects SATA hard disks as SCSI, but check the description — it is still ATA with exact model numbers. You can easily figure out from the description that the sda and sdc hard disks are Seagate, and sdb is Hitachi.

Figure 6: Block device management

Define 100 GB RAID array partitions on /dev/sda and /dev/sdc (Figure 7).

Figure 7: Creating RAID partition

Select Volume -–&ft; Software RAID and create a new RAID array /dev/md0 (Figure 8).

Figure 8: Creating a new RAID array

Create a new volume group (let’s name it “raid1group” in /dev/md0 (Figures 9 and 10).

Figure 9: Creating new volume group in RAID array

Figure 10: raid1group array successfully created

The first article explained addition of volumes for firstvolume. Exactly the same steps are required to be followed to create volumes on raid1group. (For installation purposes, you can treat a RAID volume as any standard volume created in Openfiler.) So, proceed with: Add Volume –> Raid1Group. Give the volume name as “Database”, and description as “database storage”. Allot the full 100 GB space to the volume, and select ext3 partition, to finally create the volume. The time required for creating the volume will depend on machine speed.

Proceed to define groups, users, and assign quotas to groups and users. Please refer to the previous article for detailed steps.

The benefits of using a separate disk for OS, and two additional disks for RAID 1 volumes, are:

1. The operating system resides on a separate hard disk, and doesn’t occupy RAID space.
2. Important data is mirrored in a RAID 1 volume.
3. If the OS disk fails, you can install a fresh OS and restore the backup of the configuration, to restore the Openfiler configuration.
4. If one of the 500 GB hard disks fails, you can replace that hard disk and reconstruct the RAID 1 array.

Important troubleshooting tips for bonded interface

If you misconfigure or cancel the bonded interface setup, the GUI access to Openfiler stops working. Here is what you should do in such a case. Connect a keyboard and monitor to the Openfiler system, and log in as root. Run the following commands:

cd /etc/sysconfig/network-scripts ## Change directory to network
rm ifcfg-bond0 ## Remove bond0 interface

Ensure that ifcfg-eth0 file has the following lines (adjusted for your configuration) or add them:

DEVICE=eth0
BOOTPROT=static
BROADCAST=192.168.51.255
IPADDR=192.168.51.200
NETMASK=255.255.255.0
NETWORK=192.168.51.1
ONBOOT=yes
TYPE=Ethernet
GATEWAY=192.168.51.1

Restart the network service (/etc/init.d/network restart or service network restart). Access the Web GUI at the configured eth0 IP address. Redo the bonding configuration properly.

That’s all for this time, folks!

Amanda is probably one of the best open source network backup solutions available in the market today. The Advanced Maryland Automatic Network Disk Archiver (Amanda), as the name suggests, was developed at the University of Maryland. It allows the administrator to set up a single master backup server to backup multiple hosts over the network, to tape drives, to disks or to optical media, and even the cloud (with the help of Amazon S3 Web services). It provides an array of options for the type of media that can be backed up to, and a multitude of client environments.

Amanda is basically like an enterprise solution that does require a bit of configuration initially, but after that, it’s a breeze. But don’t let this put you off, because along with a tool like Amanda comes great flexibility and reliability, which allows it to work efficiently in the most complex of architectural environments. A variety of client operating systems are now supported, ranging from UNIX to all flavours of Linux, Solaris, Mac OSX and Windows. Overall, it’s a great utility, which you will fall in love with once you get used to it.

Why Amanda?

The most basic questions are:

• Why should you spend so much time struggling with Amanda, when you can use simple command-line automation scripts for your Linux servers?
• Even if you do decide to use a backup utility, why should you spend so much time and resources to set up software like Amanda?

The answer lies in the following three factors: scalability, flexibility and architectural complexity, which decide whether you need Amanda or not. Basically, Amanda is suited for those who have a multitude of systems with varying server environments, and it is really expensive to have a separate backup facility for each of these.

Amanda provides you with a dedicated backup server for the whole kit, making the backup process reliable, well-planned and efficient. Also, it allows for really good scalability and freedom when it comes to expanding your network. After adding new clients to the network, all you need to do is make a few changes in the configuration and you’re done; you can let Amanda handle the rest.

This essentially makes it quite future-proof. Otherwise, if your network is not that complex, I would frankly recommend a simpler solution for you, because Amanda might prove to be a bit of overkill in such scenarios.

One big advantage that Amanda has is that the data can easily be obtained with native tools, even if the Amanda server is rendered completely unusable (that, of course, does not mean that the data is not safe; it does have really good security features).

Features

• Client-server architecture: Obviously, this is one of the most important features. A unique feature is that it is the server that schedules and decides which client is to be backed up, and requests clients for data, instead of clients requesting the server. Also, all configuration related to the backup plan is done on the server. Amanda also has a really good media interface, without having any device driver-based dependencies.
• Automatic backup level selection: Amanda uses the concept of a “backup level” to distinguish different kinds of backups. Each backup type has a level number assigned to it; for example, a full backup is Level 0. Backing up the system at any level means saving all the files that have changed since the last backup at the previous level. Thus, a Level 1 backup saves all the files that have changed since the last full (Level 0) backup; a Level 2 backup saves all the files that have been changed since the last Level 1 backup, and so on. It determines the backup level automatically, rather than making the administrator determine this ahead of time.
• A consistent backup window and resource utilisation: Amanda provides a consistent plan for all your backups, such that there are no spikes in media and server resource utilisation — and you can set a specific time-frame within which backups are to happen.
• An intelligent backup scheduler: It determines the amount of data changed for a client, and schedules accordingly. The administrator only specifies a few parameters according to which backups are to occur. It basically distributes full backups with incremental ones over the backup cycle, to balance the amount of data that is backed up at a time. The scheduler skips any clients that could not be backed up, or were not available at the instant, and reschedules when they are available again.
• Data encryption and compression: Data compression options are plentiful. Compression can happen on the client or server, depending on the configuration specification. Encryption with OpenSSH and Kerberos is available, making it secure enough.
• Reporting and verification: The amreport tool provides reports on each backup run, along with detailed statistics. It also sends overnight email notifications to the administrators. The amverify tool checks the Amanda format on a drive, and whether it can be restored to a healthy state.

Installation and configuration

As far as installation is concerned, it turns out to be pretty easy; it’s the setting up and initial configuration that takes time. For most Linux systems, Amanda is available in repositories as the packages amanda-common and amandaserver, which, as the names suggest, need to be installed at their respective places. Some of the main dependency requirements for Amanda, which must be installed on the system before trying to install Amanda, are:

• GNU Tar 1.15 or later
• Samba for communication with Windows clients
• Perl 5.6 or later
• Glib version 2.2 or later
• Awk and Gnuplot for the amplot utility

The installation creates a new user, ‘amanda’ (or something similar), to run the Amanda backup and other tools. Configuration files are created in /etc/amanda, including an example configuration to play around with; this can serve as an overview, but is not of much practical use. To create a new configuration, we need to create a folder under /etc/amanda, the name of which will represent a particular configuration for Amanda. In this directory will be the amanda.conf file, which contains the following pieces of major information:

• org: The email subject, to differentiate between various backups.
• mailto: Administrator email address(es) to which to send reports (multiple addresses to be separated with spaces).
• tapecycle: The number of tapes that are available, and to be circulated.
• dumpcycle: The number of days in the total dump cycle.
• runspercycle: The number of daily full backups to be taken.
• tapedev /dev/null: This should be changed to tapedev /dev/nst0, which is the non-rewinding device for Linux.
• tapetype: The configuration of the tape drive that Amanda will be using. There is a utility called amtapetype, which performs writes to the tape to determine the capacity and speed.

The values for tapecyle, dumpcycle and runspercycle depend on the backup plan you choose, and the strategy you plan to undertake. There are a couple of other parameters in the file, which should be self-explanatory.

Now, we need to label the tapes. This is very important, because tapes are rejected if they are found to be improperly labelled during a backup run. This should be done with the amlabel utility, as the ‘amanda’ user, and to create labels that match the regular expression specified in the particular amanda.conf file. We’ll now want to add entries to the disklist file in the configuration directory, i.e., tell Amanda which directories on the client need to be backed up. Finally, the amcheck utility is used to check the validity of the configuration; if it reports no errors, you are good to go!

Go on and run amdump to take a trial run. The command won’t print anything, but a report of the backup can be generated with amreport (it will also probably send an email to report everything when it is done). Once we have everything set up well, it’s time for cron to take over and automate the process every night.

Recovering data from Amanda is pretty easy, though this time, the client needs to be set up to pull data from the server. This can be done in the amanda-client.conf file of the configuration directory, after which the amrecover utility handles everything.

In conclusion, this article tries to present to the user what it takes to set up and get an Amanda backup running. Though it is recommended to go through the official documentation while actually setting up Amanda, this involves taking care of a lot of the nitty-gritties of the configuration. Though the set-up process might look easy after reading this article, it is actually a bit harder, and might require more than one attempt. But once everything falls into place, it becomes pretty easy to run such a powerful tool on your server.

Further reading

1. Amanda Wiki
2. Using Amanda to Backup your Linux Server

We will try to appraise this interesting and feature-rich Linux distro in a series of articles. This first part covers basic Openfiler installation and configuration, and its various storage options for a 64-bit standard installation. Openfiler can convert any computer system complying with the minimum requirements to a storage server. The specifications, from the Openfiler website, are in the following table.

The website further mentions that Openfiler is compatible with 32-bit and 64-bit industry standard server hardware too. It can also be installed in a virtual-machine environment such as VMWare or XenServer, as a guest OS. Future releases of Openfiler may only support 64-bit processors, so they are highly recommended for new Openfiler installations. Some important advantages of using Openfiler are:

• No licensing cost
• Multiple NIC bonding
• Support for iSCSI targets
• Software RAID support
• Remote replication for Disaster Recovery
• High Availability (HA) cluster fail-over capability

Openfiler also has various other features available on standard Network Attached Storage (NAS) boxes, including access via FTP, Web server and SMB/CIFS services; authentication against a local (or remote) LDAP server, or integration with a Windows domain controller; user/group-wise access control and quota management; configuration via an easy Web-based GUI; free updates and patches. Help is available via excellent community support.

Getting started

To start exploring this wonderful distro, download your desired image from the Openfiler website. Various images are available: 32/64 bit ISOs, and various appliances for VMWare, VMWare ESX and Citrix XenServer. Our installation is based on the Openfiler 2.3 Installation ISO image (for x86/64). The hardware used here is a simple PC with an Intel 2.8 GHz dual-core CPU, with 1 GB of RAM and an 80 GB SATA hard disk.

Start the installation by booting from a CD prepared from the downloaded ISO image. Openfiler supports a text- as well as GUI-based installation; continue with the latter. There are practically no surprises during this installation process. The most important steps during installation are hard-disk partitioning and networking, which are explained here.

Disk partitioning

If you have two hard disks installed in the computer, select the first hard disk for installation. Partition the hard disk manually, if you have a single hard disk.

Create three partitions, as detailed in the table below. Tick the “Force to be a primary partition” check-box while creating all the three partitions.

Network configuration

Ensure you enter correct values for hostname, default gateway and DNS servers. These settings are important for patching Openfiler. Though Openfiler can get an IP address from a DHCP server, use the Edit tab at the top right corner to specify a static IP address and subnet mask. Don’t forget to tick the “Activate on boot” check-box.

Continue through the rest of the installation, and reboot to complete the first phase of installation. The Openfiler box can now be accessed in these ways:

1. Use the username root and the password configured during the installation process for access at the Openfiler PC console, or via SSH from another system.
2. Access the Web administration interface by navigating to https://ipaddress:446 (the IP address of the Openfiler PC) from any node. Here, the default login name is openfiler, and the password is password.

Updates

Immediately after the first reboot, patch the new installation. Log in at the console, and run the command conary updateall. The time required for updates depends on your Internet connection speed. Repeat the same command till you get a “Nothing to be updated” message. Now the installation is at the stage where you can create volumes, shares, groups and users, and start using the Openfiler box.

Log in to the Web administration interface. The first screen you see is status (Figure 1), where hardware and important system information is displayed.

Figure 1: Status screen

Initial set-up

Proceed to the System tab (Figure 2), which has the options to (re-)configure Openfiler system parameters like network configuration; clock setup (define a Network Time Protocol (NTP) server address and time zone); UPS setup (Openfiler, if connected to an intelligent UPS, can be configured to shutdown when the UPS signals low battery state); backup/restore configuration; and secure console (SSH access).

Figure 2: System screen

Complete the “Network Access Configuration”. You can allows access to a single host, or the full subnet. Under our test setup, we allowed access to the 192.168.51.0/255.255.255.0subnet (Figure 3).

Figure 3: Network access configuration

The next step is to create a volume group on the desired hard disk, using the Volumestab (Figure 4).

Figure 4: Volume group management

Under the test setup, the group name was firstvolume. Continue by selecting firstvolume from the Add Volumes tab. The new volume should be defined along with disk-space allocation and volume type (ext3/XFS or iSCSI). Here, we defined Songsas the volume name (Figure 5), allotting 37000 MB of disk space. We preferred ext3, from the data-recovery perspective. The time required to complete this step will vary depending on CPU and hard disk speed.

Figure 5: New volume creation

Groups and users

Next, we create groups and users. The LDAP service must be active to do this. From the Accounts tab, select the Authentication menu. Fill in the LDAP settings (Figure 6), and tick the “Use Local LDAP Server” check-box. Submit the configuration information, wait for about a minute, and under the Servicestab, verify that the LDAP service is enabled.

Figure 6: Fill in the LDAP server settings

Instead of LDAP, you can configure authentication with a Windows domain controller from this page. Administrator credentials are required to enable access control based on preconfigured groups and usernames. LDAP service configuration may give a few surprises, which will result in failure to create groups and users. Try the following if you are facing problems at this juncture:

1. Clear and rebuild the LDAP server from the Services –> Setup menu.
2. Make sure that you have updated Openfiler fully using conary updateall as discussed earlier. If not, do so now. (Community forums report that there was a bug in the original distro, which was fixed by updates. While preparing screenshots for this article, I had taken a shortcut of not updating the distro. Ultimately, LDAP failed, and I had to complete updates before proceeding (Figure 7).

Figure 7: Console output after a successful 'conary updateall' run

Create a new group, overriding the GID. Here, a GID range starting from 501 is used for the SongLoversgroup. Additional groups can be created as and when required, for granular access control. Go on creating user accounts with their passwords, and assign them their primary group (Figure 8).

Figure 8: Add new user

Quota allocation and shares

After creating groups and users, the next task is to allocate quotas. Here, for the SongLovers group, we assign a 20 GB quota. User-wise quota definition is also possible.

We also created the shares: Firstvolume –> songs –> marathi and Firstvolume –> songs –> hindi.

A new share is accessible only after granting specific access. Click the share name and configure the three required parameters — listed below:

• Share access control mode: Public guest access or Controlled access
• Group access configuration: Primary group and type of access — No access/Read only (RO)/Read-Write (RW)
• Host access configuration: Type of access to be granted (from 5 possible services: SMB/CISF, NFS, HTTPS, FTP and RSYNC). Select all which you want, and set the share’s type of access (No/RO or RW).

Here, we defined the “marathi” share with Controlled access, giving the primary group SongLovers RW access. Further, we enabled RW access via SMB/CISF and FTP protocols (Figure 9). (Remember to use the Servicesmenu to enable the FTP service if using FTP shares!)

Figure 9: Share access control

That is all, folks!! Start browsing the shared resources by the IP address. You should see your shares—in our case, firstvolume.songs.marathi—as a shared folder. Enter your user credentials, and start using your NAS!

In future articles, we will continue to discuss various interesting features of Openfiler.

References

• Installation screenshots are available here
• Openfiler forum
• Openfiler download page

Special thanks to my friend Achyut Ghare, BE, CISA for introducing me to, this great distro!

Before I jump into the topic, let me explain what OpenOTP is, and why we may need it. Password-based authentication has been around for a long time — perhaps even during the days of the Colosseum! — but is a debatable method, because our secrets are not as secret as we would like them to be. Social engineering, shoulder-surfing (when you log in), eavesdropping, etc., can capture your secret/password. Besides, some people use weak passwords based on easily-found personal data, like ex-girlfriends’ or spouse’s names, dates of birth, etc. This is where two-factor authentication comes in, helping to keep your network secure.

Two-factor authentication is any mechanism where more than one thing is required to authenticate a user. The two components of two-factor authentication are:

• Something you know (e.g., password/PIN, etc)
• Something you have (a token, cell phone, etc)

The “two-factor” or “strong authentication” mechanism is based on one-time passwords (OTP). In this mechanism, instead of authenticating with a simple password, every user will use an OTP, generated from a device (hardware token), or by an application, perhaps on their cellphone. This is, no doubt, more secure than a static password, because the attacker needs to know both the PIN, and possess the token/device you are using. If the attackers try eavesdropping on your login session too, they won’t succeed, since a one-time password is valid for only one session. Once an OTP is generated, it cannot be reused.

One of the biggest issues with two-factor authentication is expense, because special-purpose hardware tokens have a cost — and also, the authentication server has license costs. Besides, the algorithm used to produce the codes is only known to the company — i.e., it is proprietary. The solution I’m proposing is open source, and the hardware token will be replaced by your cellphone.

In my current assignment, I had to come up with a solution that would support OTP for applications like RADIUS, SSH, Apache, and some other Web services as well.

Out of personal interest, I had in the past, used open source tools like OPIE (One-time Passwords In Everything), Mobile-OTP, Google Authenticator, etc., but the problem with these is that the secret and passphrase are stored in plain text files on the desktop filesystem. There are no schema available to integrate these applications with OpenLDAP or any directory service.

I also had to invest a lot of time and effort on scalability, service uptime, and plugins for different applications. I was looking for something that could store everything centralised in a directory server like OpenLDAP, and also was very scalable and highly available. After a bit of searching, I found a couple of open source enterprise product candidates. After some testing, I approved of OpenOTP.

An advantage of OpenOTP is that we can use cellphones as our token device, which reduces the solution cost, since we don’t need to buy hardware tokens (though this product supports those as well). Also, it’s certified on the Open Authentication Framework (OATH). OpenOTP supports OATH HOTP (event-based), TOTP (time-based) and OCRA (challenge-based) standards for both software and hardware tokens. Many OATH-compliant software tokens are available for Android, iPhone, J2ME cellphones, Windows Mobile, Palm, BlackBerry, etc.

Overview

OpenOTP is an enterprise-grade two-factor authentication solution based on open standards and technologies. It provides multiple authentication methods for LDAP users, which rely on OTP, including Mobile-OTP software tokens, SMS OTP, Secure Mail OTP and YubiKey. It is completely free for up to 25 users. If you need more users, then you have to buy a licence. To learn more, please visit the official website.

The OpenOTP solution is composed of the WebADM server application, the OpenOTP SOAP service, the optional RADIUS Bridge, User Self-service Desk, and the end-user Web application. The main components in the solution are:

• WebADM, Web services (e.g., OpenOTP), Web apps (e.g., SelfService)
• The LDAP server
• RADIUS Bridge
• PAM Bridge

The WebADM server contains:

• The SOAP APIs for the installed Web services
• The Web apps
• The admin portal
• The PKI server
• The session server

This article will not cover all these components; for a better idea of each component, please have a look at the product website. We will cover installation and configuration of OpenOTP, and how to configure some applications like SSH and RADIUS Bridge. Figure 1 shows the interaction of the components.

Figure 1: The suite components

The current version of WebADM supports any 32-bit or 64-bit Linux with GLIBC >=2.5 and installed 32-bit binaries. The installation package contains the required dependencies, so you don’t need to install anything extra. However, WebADM needs an LDAP server like OpenLDAP, and for logging, an SQL database (MySQL or PostgreSQL). I am using OpenLDAP and MySQL. This product can run both on a physical machine, or on a virtual machine. For the hardware requirements, please check the product website.

Figure 2 shows (at a high level) an overview of how OpenOTP might be integrated into your infrastructure.

Figure 2: Integrating OpenOTP in your environment

Installing the product

Download the tarball from the product website, uncompress it, and run it, as follows:

gunzip webadm-all-in-one-1.1.x.sh.gz
sh webadm-all-in-one-1.1.x.sh

For a basic installation, just keep hitting Enter till the installation is over. My installation directory is /opt/webadm. This script creates the WebADM system user, sets filesystem permissions, creates directories for configuration files, logs, etc.

Configuring OpenLDAP

The steps below are if you are setting up a new LDAP server. If you already have an existing OpenLDAP installation, then edit the configuration files /opt/webadm/conf/webadm.conf and /opt/webadm/conf/servers.xml to make the necessary changes. These files are self-explanatory and well-documented, but you can scan the procedure below as well.

Make sure that OpenLDAP related packages are installed, using your distribution’s package manager. Copy the WebADM schema to the existing schema directory, and add this schema to the /etc/openldap/slapd.conf file:

cp /opt/webadm/doc/OpenLDAP.schema /etc/openldap/schema/webadm.schema

include /etc/openldap/schema/webadm.schema

Now we have to modify our suffix and rootdn, and also add ACLs for the webadm user. You could instead use the default LDAP user for webadm too, but it’s not recommended, for security reasons. Here, I’m using example.com as my DC:

Suffix 	"dc=example,dc=com"
Rootdn	"cn=Manager,dc=example,dc=com"

Add admin ACLs for the WebADM user (read and write permission to the entire LDAP tree) in slapd.conf:

Access to *
by dn="cn=admin, dc=example, dc=com" write
by self write

Now, we add the domain to the OpenLDAP tree. Create an LDIF file, with your favourite editor, and save it as example.ldif, with these contents (modify them to suit your domain):

dn : dc=example,dc=com
dc: example
ou: rootObject
objectClass: top
objectClass: dcObject
objectClass: organizationalUnit
dn: cn=admin,dc=example,dc=com
cn: admin
sn: admin
objectClass: person
objectClass: inetOrgPerson

Initialise the OpenLDAP directory by adding the just-created LDIF:

# slapadd –v –l example.ldif

Finally, restart OpenLDAP with /etc/init.d/ldap restart.

Configuring MySQL

We must create a webadm database to store audit logs and localised messages, and a webadm user with a password of our choice, with full permissions on that database. My example uses the username webadm and the password password. Make sure the MySQL service is running; log in with the client, as root; create the webadm database, and assign permissions, as follows:

mysql –u root –p
mysql> create database webadm;
GRANT ALL ON *.* TO webadm@”localhost” identified by ‘password’;

Configuring WebADM

Let’s then configure WebADM to use our OpenLDAP and MySQL servers. All the configuration files are in /opt/webadm/conf. Edit server.xml and change the following, customised according to your setup:

<LdapServer name="LDAP Server"
host="localhost"
port="389"
encryption="NONE"
cert_file=""
key_file=""
/>
<SqlServer name="SQL Server"
type="MySQL"
host="localhost"
port="3306"
user="webadm"
password="password"
database="webadm"
/>

Now we come to WebADM’s main configuration file, webadm.conf. Most of the settings work out-of-the-box with any supported LDAP backend, and with very minor changes. The file itself is very well documented so please read the comments before making changes. We need to change the LDAP tree base (suffix) in the following settings (adapting the domain name to your domain):

auth_mode DN
list_domain yes
proxy_user	       "cn=webadm,dc=example,dc=com"
proxy_password         "testing123"
super_admins           "cn=admin,dc=example,dc=com”
optionsets_container   "dc=OptionSets,dc=example,dc=com"
webapps_container      "dc= webApps,dc=example,dc=com"
websrvs_container      "dc= WebSrvs,dc=example,dc=com"
mountpoints_container  "dc=MountPoints, dc=example,dc=com"
domains_container      "dc=Domains, dc=example,dc=com"
clients_container      "dc=Clients, dc=example,dc=com"

Now we are ready to start the webadm service. If you want to change the port it listens on, you can modify /opt/webadm/bin/webadm. By default, the Web server listens on port 443 (SSL), and the SOAP server on ports 8080 and 8443 (SSL). The settings are HTTP_PORT, SOAP_PORT_SSL, etc. To start the service, run:

#/opt/webadm/bin/webadm start

Navigate to WebADM (https://<your-server-address>). Only the user(s) given as super_admins in webadm.conf can run the graphical setup. So, log in with your admin account and run the graphical setup via the Setup button that appears on the home page.

WebADM requires a DN-based login until the setup is completed. Then it will use the login mode as configured in webadm.conf. The graphical setup process will:

• Create the required database tables (as specified in conf/database.xml).
• Register the required LDAP schema objectclasses and attributes.
• Create the proxy user (if it does not already exist).
• Create the WebADM LDAP containers (as defined in conf/webadm.conf).

This article will not describe in detail how to use the WebADM GUI — for that, take a look at the WebADM manual. This GUI is the centralised point for LDAP and OpenOTP administration activities like creation, deletion, modification, etc.

Once you log in to the console, you can see that some domains have already been created, like dc=clients,dc=MountPoints,dc=ObjectSets,dc=WebApps,dc=WebSrvs, etc. These are created by the setup script for use by WebADM.

Test user

Now we have to create a user for our test setup. For this, let’s create a different OU called People. Once the OU is created, you can see it in the left pane. To create the WebADM user, follow the steps shown below:

1. Click Create, and you will see the screen shown in Figure 3.
   

   Figure 3: WebADM Account

2. Select WebADM Account and click Proceed. Provide all necessary information, like CN, Last Name, Login Name, password, etc. Then click Proceed and click Create the object.
3. After creating testuser, Figure 4 is the screen you will see.
   

   Figure 4: OpenOTP setting

4. Now we change OpenOTP settings like Token, PIN, etc. Click the Application menu, select OTP Authentication Server and click Configure. You will see the screen shown in Figure 5.
   

   Figure 5: Configuration

5. Now configure the OpenOTP application as per your needs, like the default OpenOTP authentication mode, domain name, host-name of your server, etc. The basic setting is Login Mode (authentication mode). I used LDAPOTP, to require both LDAP and OTP passwords, which is the default. If you enable Password Swapping, it will accept either LDAP or OTP.
6. Now click Proceed, then click testuser and click OTP  Server. You will see the screen shown in Figure 6.
   

   Figure 6: OTP Authentication server

7. Select Register/Unregister Token; you will see the screen shown in Figure 7.
   

   Figure 7: Token registration

8. Select Mobile OTP (Time-Based). If you have an existing key, you can put it in the Token Key field, or you can let the server generate a key.
9. Then you enter your Token PIN, click Register, and you are done.

So now our test user is ready to use Mobile-OTP. You can download the relevant application from your Apps store. (I’m using “Droid OTP” for my Android device.) Launch it, and enter your secret PIN to generate the passwords you can use to log in to different applications. Next, let us look into how to configure applications like SSH and RADIUS.

Configuring SSHD for OTP passwords

Download libopenotp-1.0.4-1.tgz and pam_openotp-1.0.4.tar.gz from the downloads section on RCDevs website and decompress them. For 32-bit architectures, the default modules are present in the build_linux32 subdirectory of the extracted libopenotp tar, but for 64-bit, you have to compile it. For my 64-bit setup, I compiled and installed libopenotp with make && make install. I then moved /usr/local/lib/libopenotp* to /usr/lib64.

After getting libopenotp into the appropriate directory, run an ldconfig.

Next, compile pam_otp with make. Copy pam_openotp.so to the architecture-specific security directory: for 64-bit, /lib64/security/ and for 32-bit, /lib/security. On 64-bit, remove the pam module from /lib/security/.

Edit /etc/pam.d/sshd and replace the line “auth include system-auth” with the following, replacing myserver with the WebADM server’s IP address:

auth required pam_openotp.so server_url="http://myserver:8080/openotp/" client_id="SSH"

My file now looks like this:

auth    required 	pam_env.so
auth   required	pam_openotp.so server_url=http://192.168.0.1:8888/openotp/ client_id="SSH" password_mode=2, default_domain="example"

Next, let’s configure the OpenSSH service. Edit /etc/ssh/sshd_config. Enable OpenSSH challenge response mode with ChallengeResponseAuthentication yes. Enable OpenSSH password authentication with PasswordAuthentication yes. Enable OpenSSH with PAM with UsePAM yes. Save and exit. Restart the SSH service.

Now if you try to connect as the test user (for example, ssh testuser@192.168.0.1) from any box, it will prompt you for an OTP password, and not the normal password.

RADIUS Bridge (RB)

OpenOTP RADIUS Bridge provides the RADIUS API for OpenOTP. It is an optional server component to be deployed on your OpenOTP installation and is implemented over the open source FreeRADIUS software.

Installation and configuration

Download RADIUS Bridge from the downloads section on RCDevs website and install it using the self-installer script (for example, sh radiusd.1.0.6-1.sh). The installation creates the RADIUS Bridge system user and sets filesystem permissions. Edit /opt/radiusd/conf/openotp.conf and change the following parameters, adapting them to your environment:

Server_url = http://192.168.0.1:8080/openotp/
Password_mode = 2
Default_domain = "example"
Data_is_vps = yes

For more information regarding the settings, please look at the README file.

Start the radius service with /opt/radiusd/bin/radius restart. If required, add your RADIUS network access clients (for example, your VPN server IP address) in /opt/radiusd/conf/clients.conf. Each client must be configured with an IP address and a RADIUS secret. By default, RADIUS Bridge accepts requests from any client using the RADIUS secret “testing123″. If you want the RADIUS server to reply with a value-attribute pair, make sure all the attribute and value pairs exist in the /opt/radiusd/lib/dictionaries/dictionary.XXX file, and specify the same in the Reply-Data section of the user (in the WebADM GUI console) as follows:

192.168.0.1:Vendor-XXX-Profile-ID=2000,192.168.0.2:Vendor-XXX-Profile-ID=3000

This means that if testuser logs in from 192.68.0.1, the network access client will get a value of 2000, while if the person logs in from 192.168.0.2. it will get a value of 3000.

We are done!

OpenOTP is pretty vast, and everything can’t be covered in a single article. We covered the basic setup and configuration. For other services, please check the documentation available at the website. This product is scalable and you can also configure the services in HA mode.

If you want to go ahead with two-factor authentication, and don’t want to invest, then you can also check OPIE, Google Authenticator, etc.

Though MySQL provides binaries for various Linux distributions and even FreeBSD, we’re not interested; we need performance, and so we’ll be compiling our own. As in the previous article, I do not claim that optimisations will definitely increase performance; sometimes, they can cause degradation as well, because it depends more on hardware, CPU cache and DRAM.

MySQL, unlike the traditional ./configure build methods, uses CMake, an advanced and easy-to-use build system with a nice UI called ccmake (command-line) or cmake-gui (the GUI, if you run X). However, we’ll specify options at the command line itself. Along with CMake, you’ll also need a C++ compiler installed. This usually comes with GCC in most distributions, but sometimes is in a separate package (e.g., Fedora has gcc and gcc-c++).

To download the MySQL source tarball, navigate to the downloads page on the MySQL website, and in the platform drop-down list, choose “Source Code”. When the download option is displayed, make a note of the MD5 sum in a text file. After the download is over, to ensure you have the original file and not a tampered version with security holes, verify the file checksum as follows.

Copy the saved MD5 sum to the clipboard, and paste it at the shell prompt, to assign to a variable, as shown; then run the following commands:

$ ORG_SUM=<Paste the MD5 sum here>
$ FLE_SUM=$(md5sum mysql-5.5.14.tar.gz | awk -F ' +' '{ print $1 }')
$ [ "$ORG_SUM" == "$FLE_SUM" ] && echo verification success || echo verification failed

If your download was not okay, redownload it, else read on.

Source code configuration

CMake accepts options on the command line with the -D switch; for example, cmake -D<OPTION NAME>=<OPTION VALUE>. Significant options while configuring the code-base are:

• CMAKE_INSTALL_PREFIX: The location where the MySQL server will be installed.
• COMMUNITY_BUILD: Enables/disables the community features in MySQL.
• ENABLED_PROFILING: Enables/disables query profiling code.
• MYSQL_DATADIR: This is the default directory where data will be stored. It can be set in the configuration file as well, after installation.
• WITH_INNOBASE_STORAGE_ENGINE: Enables/disables InnoDB, a transactional storage engine.
• WITH_DEBUG: Enables/disables debugging capability; this increases the binary size, but can be very useful while getting support from forums, etc, in case your installation fails for some reason. A related option is ENABLE_DEBUG_SYNC.
• WITH_PARTITION_STORAGE_ENGINE: Enables/disables the partition storage engine.
• WITH_SSL: Enables/disables SSL support in the server and client. Enable this if you see any need for replication in the future; unencrypted replication is a risk.
• CMAKE_CXX_FLAGS: Sets the C++ compiler flags -O3 -march=native -mtune=native -msse -msse2 -mmmx (provided you have a CPU with a decent cache size — else use -Os instead of -O3).
• CMAKE_C_FLAGS: Sets the C compiler flags. MySQL has no C code, so this is not required. You might want to set it to the same value as CXXFLAGS, for sanity.
• DISABLE_SHARED: Builds MySQL statically, with all plugins compiled in. This is not preferred; it will unnecessarily consume memory for unwanted features/plugins.

I have described the most important options, all of which are enabled by default. If you want a look at many more, use ccmake or cmake-gui, or refer to the MySQL documentation here.

Server configuration

MySQL’s single configuration file, my.cnf, is located in CMAKE_INSTALL_PREFIX/etc or MYSQL_DATADIR (deprecated). You can also pass the location of the configuration file to mysqld while starting it. Some of the configuration options in my.cnf in the [mysqld] section are:

• bind-address: The address on which the server will listen for connections. If you’re only going to connect from localhost, don’t use this; instead, use the skip-networking option, as a security measure.
• user: The account as which mysqld will run; again, a security measure. This user must have R/W access to MYSQL_DATADIR.
• datadir: By default, the compile-time path in MYSQL_DATADIR. Change it if you want to relocate the data directory.
• character-set-server: Keep as UTF8 (Unicode), unless you have special reason to change it.
• character-set-client: Same as the previous option, but this will tell the client to use the specified character set.
• default-character-set: The default set to use while creating tables.
• default-storage-engine: This default (MyISAM) will be used if an engine is not specified in the CREATE statement.
• skip-innodb: Disable the InnoDB storage engine. If you statically compiled MySQL, you need to use ignore-builtin-innodb.
• key_buffer: The MyISAM key buffer; should be kept as large as possible (of course, sparing DRAM for other applications and services). The larger it is, the better the performance. On a dedicated MySQL server, it is usually at least a quarter of the total memory (and not more than half). It should be enough to hold all the MyISAM indexes (.MYI files) in memory. This can be checked via variables, which can be obtained by the SHOW VARIABLES command at the MySQL command prompt, or using tools like phpMyAdmin. Obtain these variable values, and divide key_read by key_read_requests; the value should be < 0.01. And if you divide key_write by key_write_requests, the value should be < 1.
• table-cache: Every time MySQL opens a table, it saves it in cache, to speed up access. The variable opened_tables in server status will tell you what you need to set this to; some use cases have seen values as high as 20,000. Keep opened_tables as low as possible.
• read_buffer_size: The default is 128K, and usually this is set to 1M or 2M. The performance depends on CPU cache, disk speed and other factors; you should run a test in your environment, as described here.
• query_cache_type: The value 0 means the query cache is disabled; 1 implies that the query cache is enabled for all queries that don’t have SQL_NO_CACHE specified in the query; and 2 means that no query is cached unless SQL_CACHE is specified in the query.
• query_cache_size: The larger the better, but also check the available DRAM.
• query_cache_limit: The maximum size of the result set that will be stored in the query cache. This should be large if you have queries producing large result sets and have a lot of DRAM to spare. Watch the variables %Qcache% in the server status to see how well the query cache is being used.
• tmp_table_size: MySQL sometimes needs to create temporary tables automatically, depending on the query. When the result set grows larger than the size specified here, the temporary table is converted to a disk-based temporary table, which results in a loss in performance. Keep this size as large as possible, considering the available DRAM and other applications running on the site. In the server status, observe created_tmp_disk_tables.
• max-connections: The maximum number of simultaneous connections. This is limited by CPU power and the available DRAM.
• sort_buffer: This should be set to a high value if your queries involve a lot of sorting. This seems to have some negative effect on performance as well, so do a few test cases before and after changing the value.
• thread_cache: If you have a lot of very-short-duration connections, increase this value till the hreads_created value stops increasing in the server status, to reduce CPU usage.
• long_query_time: This is the time a query will be allowed to run before it is considered slow and logged in slowlog. Keep this value small — say 5s; the default value is 10s.
• slow_query_log_file: This is the path to the slow query log file. MySQL 5.5 has a new option, log-queries-not-using-indexes. If specified, it will send only slow queries that are not using indexes to the slow query log.

Tips for designing efficient tables and queries

Although I have talked a lot about options and optimisations, the actual optimisation of MySQL is done by properly designing tables and the database. Also, it depends on how efficient your queries are. Generally, one query can be executed in multiple ways, and each way has its own advantages and disadvantages. We need to choose a query that suits our environment and isn’t costly on the performance side. Here are some points:

• Keep the design simple; split your columns across tables. Columns you don’t access often should be put in a different table, to avoid memory wastage when the table is cached. You can link data across tables using foreign keys, which are supported only in InnoDB, and not in MyISAM.
• Use MyISAM if you are going to read a lot and write less — and InnoDB otherwise. There are some absolutely astounding results in this aspect, and you should switch engines after proper testing.
• The columns frequently used in WHERE statements and in filtering results should be indexed. Indexes point to the data location on the disk. If indexes are not used, MySQL has to scan the whole table to find the required row, which is very costly in terms of performance.
• Your tables should conform to normal form rules, have normalised data, and use efficient column types.
• Keep queries simple and short. Longer queries take more time to execute. Use application-level caching.
• Turn on slow query logging in my.cnf by setting long_query_time and slow_query_log_file.
• Use EXPLAIN and ANALYZE to see what a slow query does, and how long it takes. This can give you a lot of hints for optimising it.
• Run OPTMIZE TABLE frequently. But be aware: this locks the table, so run it only when MySQL traffic is low. This basically rearranges disk files, reclaiming disk space consumed by deleted rows.

Since MySQL uses a lot of disk, actual performance is pretty much limited by disk performance, rotation speed, etc. Obviously, you need a high-speed disk. Another optimisation tip would be to use a filesystem that supports compression on the MySQL data partition, to reduce disk seeks; this is currently supported in ZFS (FreeBSD) and Btrfs. The latter is under development as of now, and not recommended for production use; try it only after proper backups.

Using RAID striping with a stripe size of 1 MB or so can help a lot, since the data is split across two disks, and reading from two disks is obviously faster than reading from one disk sequentially.

There’s another possibility for speeding up disk performance, which is not actually tested: using SquashFS and AUFS. I have never tried it for a MySQL database directory but have used it on my system, and it does speed up things at the cost of CPU usage. What you basically do is compress /usr into a squashfs file — say, /squashed/usr.sqfs. Then, you mount usr.sqfs at /squashed/usr/ro (loop, readonly).

Using AUFS, you can mount /usr and specify read (/squashed/usr/ro) and write (/squashed/usr/rw) locations as its branches. The only problem with this method is that you have to re-squash whenever large changes are made, in order to ensure performance. This is probably not feasible with MySQL in cases where the database will have a lot of writes.

More information on this squashing method can be obtained from this Gentoo Linux forum thread. Please note that I’m not responsible for any loss. Backup before trying any tricks. :-)