Creating EBS based Amazon EC2 instances

So I had to create an Amazon EC2 instance quickly and ideally it had to be as plain a CentOS instance as possible. Now the first thought that I had was to find a way of creating one from scratch somehow, i.e. use the EC2 tools to bootstrap an S3-based AMI image but whenever I booted this it kept giving me kernel initrd errors and didn’t work.

In the end I just gave up on bootstrapping my own instance from scratch and simply used one of the pre-existing Rightscale CentOS 5.4 images as a starting point. I started it up, removed almost everything from it, including the rightscale rpm. Then I snapshotted it, and converted the snapshot to an AMI using ec2-register.

Now I finally had somewhere to start from. I started up my new instance, and thanks to the instructions on this webpage I proceeded to create a blank EBS volume. I created a filesystem on it and mounted it under /mnt/ebs. Now I could rsync the running instance across. I created the /dev files, unmounted and shut down.

I now had an EBS volume from which I could create a snapshot. I then registered this snapshot again and I finally ended up with an appropriate AMI to use. Job done. Thanks once again to the webkist blog for the detailed instructions.

If I do figure out how to properly bootstrap my own EBS-based AMI image I’ll be sure to post the details here.

Turn off the linux console display

I needed a way to turn off the display on an old netbook that I am using as a low-powered server. I installed CentOS 5.5 on the netbook and set the ‘inittab’ to boot into console mode by default however I did not know how to turn the display off completely when in console mode. I wanted to do this for various reasons; saving power and reducing the heat generated were my primary concerns.

I was thinking of using ‘setterm’ for this, but I decided to google this first and found the perfect solution in the Arch forums. Just in case that link stops working I’m going to include the script below, hopefully it will be helpful to other people.


# Check if X is running or not, turn off monitor, #
# wait for a key press and turn it on again.      #


# Check if X is running.
ps -e | grep -e "\bX\b" > $grep_result_file
ps -e | grep -e "\bxorg\b" >> $grep_result_file
ps -e | grep -e "\bxserver\b" >> $grep_result_file

## If you want to check result file, uncomment following lines.
#echo "===== $grep_result_file - begin ====="
#cat $grep_result_file
#echo "===== $grep_result_file -  end  ====="

if [ ! -s $grep_result_file ] || [[ $(tty) =~ tty ]] || [[ $(tty) =~ vc ]]; then
    echo 'Detected X not runnig or you are at console...'
    if [ $UID -ne 0 ]; then
        echo 'You need super user privileges to run this script at console.'
        echo 'Rerun as super user or start X and run from a terminal.'
        exit 0
    turn_off='vbetool dpms off'
    turn_on='vbetool dpms on'
    echo 'Detected X running...'
    turn_off='xset dpms force off'

echo 'Turning off monitor...'

echo 'Waiting for a key press...'
read -n1 -s

echo 'Turning on monitor...'

rm $grep_result_file

echo 'Finished: monitor_off'
kill -9 $PPID

Creating a bootable USB stick with Centos 5.4

OK, other people have also written about this, but the information seems to be cluttered all over the place in different forums or the CentOS wiki.

The most straightforward way I found to do this is as follows. Credit for this method goes to Neil Aggarwal, who posted it on the centos mailing list.

In case that link stops working someday, it is as simple as:

1. Using fdisk, I created two partitions on the drive:
/dev/sdc1 = 15 MB
/dev/sdc2 = The rest
I marked the first one bootable

2. I used dd to copy the diskboot.img to /dev/sdc1

3. Formatted /dev/sdc2 as ext3 and copied the
iso images there

4. When I boot from the usb drive, I selected hard
drive install and pointed the installer to

It worked perfectly!

Thanks, Neil!

OpenBSD and PF

On the first day of the UKUUG Spring Conference I attended the OpenBSD and PF tutorial. I was asked to write a short review of that tutorial for the UKUUG newsletter. I’m posting what I wrote below:

The superiority of OpenBSD when it comes to security is legendary. The OpenBSD community continuously do security audits of their codebase and their website proudly boasts of having only two remote holes in the default install, in more than 10 years!” PF is the default packet filter used in OpenBSD from version 3.0 onwards.

Having setup OpenBSD firewalls using PF in the past, I was interested in expanding my knowledge and this tutorial provided the perfect opportunity for me to do so. The tutorial was given by Peter M. Hansteen, who is a consultant, writer and sysadmin based in Bergen, Norway, and also the author of the excellent “The Book of PF“, published by No Starch Press.

Peter started off by answering some common questions that people might have about PF such as: Can I run it on Linux (Answer: No, but some are trying). He recommended not trusting any GUI tools, and simply using a text editor to edit pf.conf as that is simpler and faster. In addition, some tools claim to automatically convert “other” firewall rules to PF, but he recommended implementing a fresh PF config yourself.

The first firewall I ever tried to configure for my home network used iptables; when I had to implement one in PF, I found it to be a breath of fresh air. I have always found the concept of “chains” that iptables uses confusing. PF doesn’t have any concept of a chain; you simply start your rules by first “blocking everything”, then enabling the things you need, one line after the other. Although in principle this sounds exactly like what you are supposed to do with iptables, in practice the rules you generate are much simpler, and easier to understand.

Leading on from this, he showed examples of how PF should be set up in an environment where you need a “gateway” between 2 networks, and how to deal with problems faced by people who try to use FTP from behind a NAT firewall (ftp-proxy). Tables and filtering by services (http, ftp, etc) were introduced next. A table in PF is basically a list of IP addresses; listing them in a table makes it easier to apply a single rule to a collection of hosts.

Peter then moved on to the subject of dealing with the huge volume of spam that besieges us all. The two main concepts that he focused this section of the tutorial on were “tarpitting” and “greylisting“. In tarpitting, when a blacklisted host connects to you, you send replies to them very very slowly, let’s say around 1 byte at a time. When doing greylisting, you lie to unknown connecting clients using SMTP 45n errors (temporary local error). This usually thwarts spammers, who simply want to quickly connect, deliver their payload and leave. Many spammers don’t attempt to reconnect after seeing this error, while legitimate clients will automatically retry after a short period of time. These legitimate hosts are then added to a whitelist, which means that the next time they try to connect, they will no longer be given a 45n temporary error, their mail will be accepted immediately.

We then got a look at how to thwart SSH bruteforce attacks by using rate-limiting, and a short introduction to wireless networking in OpenBSD. The next thing that Peter talked about, authpf, was quite interesting. Basically users need to authenticate to authpf first; once authenticated, only then is traffic generated by these users allowed to pass through the firewall. Special rules can be setup specifically for authpf users.

The next topic was load balancing and Peter showed how to configure a “web server pool” using PF. Requests to this pool were alternated using a form of round-robin. To solve a common round-robin problem where machines in the pool go down, you can use “hoststated“, which monitors the state (up/down) of the certain specified hosts and compensates accordingly. hoststated has been renamed to relayd in OpenBSD 4.3.

You can tag incoming packets, so you can quickly pass/block packets marked with a certain tag. Setting up a OpenBSD bridge was discussed next. A bridge in this context simply refers to a transparent firewall that sits between 2 or more networks and filters packets at the link level.

Using ALTQ, you can do bandwidth allocation and traffic shaping. You can used class based queues (percent, kilo, or mega bytes), priority based or hierarchical queues. In a class based queue, you can say for example that FTP is only allowed 20% of your bandwidth.

The last major aspect of PF that Peter discussed was CARP (Common Address Redundancy Protocol) and pfsync. Put simply, CARP and pfsync allow you to setup 2 redundant firewalls instead of 1, and in case one firewall fails, everything switches over to the other firewall automatically. pfsync is used to keep the rules between the 2 firewalls in sync.

Overall, I am very pleased that I attended this tutorial. Obviously I was familiar with some of the concepts, but things like authpf, hoststated and CARP were completely new to me. I will definitely use the things I learned here when considering any OpenBSD based firewalling solutions in the future.

UKUUG Spring Conference

I’m a member of the UKUUG, and I just came back from their Spring Conference, which was held in Birmingham this year. It was a great conference! I met lots of other Unix/Linux people and learned a lot simply by chatting with people in the corridors! I will try my best to make it to the next one, which I think will be held sometime later this year.

In any case, I was asked by Alain (UKUUG Chairman) to write a few words about one of the tutorials that I attended on the first day of the conference. It will be published in the next UKUUG newsletter. I will make sure I post a copy of what I write here.