Atlantic Linux Blog

The top 10 supercomputers in the world run Linux

Pingdom (a Swedish uptime monitoring company) took the results of the latest Top500.org list (Top500 is a site that publishes a list, usually twice a year, of the top-performing supercomputers in the world where the performance is ranked against the LINPACK benchmark) and published a study of what operating systems the top 20 supercomputers in world are running. It turns out that 19 of the top 20 are running some form of Linux (with the top 10 all running Linux).

While most of you reading this probably don’t plan on purchasing a supercomputer on the scale of Roadrunner or Jaguar in the near future (if you do, we’d love to assist you in the process) our own experience with deploying smaller HPC clusters for organisations is that Linux is equally suitable for smaller clusters for a variety of reasons including the cost of deployment, the flexibility of Linux as a server OS, the wide range of HPC software that is developed on or targetted at the Linux operating system and the excellent support provided by the Linux community for new hardware (see the recent announcement of Linux being the first operating to support USB 3.0 devices as a good example).

Tags: clusters, hpc, linux, supercomputer

Marine exploration using Linux

It is always interesting to see the different environments and ways in which Linux is used in the real world. I was recently onsite with the Irish Marine Institute to install a new Linux server for them. The server configuration itself is pretty standard – using the latest stable CentOS 4.x on a new HP ProLiant server.

What is interesting about this particular piece of work is the equipment that the Linux server is being used to control. The Marine Institute is Ireland’s national agency for marine research, technology and innovation. As part of their work they are involved in a climate change program which requires them to measure data including the temperature, salinity, chlorophyll and oxygen levels of the ocean. Using this data they can model the behaviour of the ocean and understand its role in climate change in Ireland.

As well as using traditional methods for recording this data such as sensors on buoys and ship surveys – the Marine Institute is deploying an unmanned remote controlled vehicle which includes a range of sensors. The remote controlled vehicle or glider can be programmed to follow a specific course, taking measurements at various depths as it travels and sending them back to base periodically. Obviously, this gives far more flexibility than data from a static buoy and costs far less to run than a research vessel.

Which brings us back to the Linux install. The Linux server will operate as the controller for this remote glider – a Slocum electric glider developed by Webb Research. Webb provide software to talk to the glider any time it is on the surface using an Iridium satellite modem. The software allows data to be retrieved from the glider and new routes to be sent to it.

The initial installation went well – after a few teething problems. The glider, which is currently out of the water (having successfully completed initial trials), was powered on and we immediately starting seeing communications between the server and the glider. Initial tests look good and we hope to hear of the Marine Institute successfully deploying this technology in the near future. When in operation, the glider can travel distances of over 1000km and to depths of 1000m.

Tags: linux, marine research, oceanography

Setting UUIDs on new partitions

If you take a look at /etc/fstab on a recent Linux installation, you may notice it’s using lines like

UUID=663f1349-3d37-4633-af59-849eda89bae4 / ext3 defaults 0  1

instead of the more traditional

/dev/sda1       /               ext3    defaults 0  1

These universally unique identifiers (UUIDs) can be generated by the uuidgen command and can reasonably be considered unique among all UUIDs created on the local system, and among UUIDs created on other systems in the past and in the future (from the uuidgen(1) man page).

Why would you want to use a UUID rather than the far more readable /dev/sda1 (well, readable if you’re somewhat comfortable with Linux anyway)? The problem with traditional device names (like /dev/sda1) is that they are assigned by the Linux kernel at boot-time and depend on the order in which devices are found. So /dev/sda is the name the Linux kernel assigns to the first SCSI or SATA hard drive it finds, /dev/sdb is the name used for the second SCSI or SATA hard drive and so on.

This works very well until the Linux kernel detects your devices in a different order to the usual. In practice, this is a relatively rare event but it is certainly possible when moving between kernel versions and if you make changes to your system hardware (such as adding in a second SATA controller). If your device order does change, all of a sudden, your root device which used to be known as /dev/sda1 becomes /dev/sdb1 or similar. Normally this will cause a kernel panic at boot time as your Linux system attempts to mount a filesystem which doesn’t exist or which doesn’t contain the expected filesystem.

One solution to this is the use of UUIDs. When you create a new filesystem, a UUID is generated and written to the filesystem’s superblock. The /etc/fstab then refers to this UUID which will remain constant regardless of what kernel you are running or how the drive is connected to your system (indeed, if you move the drive to an entirely different system, the UUID will still be valid).

If you only create partitions during the initial installation of your OS – you won’t have to deal with creating UUIDs for your partitions, the installer should take care of it automatically.

If you create any new partitions though, you will need to assign UUIDs to those partitions manually if you wish to refer to those partitions in your /etc/fstab by UUID (you can continue to use a mix of /dev/sda1 type entries and UUID based entries in /etc/fstab if you wish). To assign new UUIDs you need to

1. Generate new UUIDs with uuidgen

   uuidgen
   f15f8aed-0073-4d2f-abec-aa5da4f72e8c

2. Write this uuid to your new partition (WARNING: Do not run these commands against an existing partition),
   for ext2, ext3 or ext4:

   sudo tune2fs -U f15f8aed-0073-4d2f-abec-aa5da4f72e8c /dev/sdc5

   for xfs:

   sudo xfs_admin -U f15f8aed-0073-4d2f-abec-aa5da4f72e8c /dev/sdc5

   for reiserfs:

   reiserfstune -u f15f8aed-0073-4d2f-abec-aa5da4f72e8c /dev/sdc5

Update: When you  create a new filesystem on a device, the command seems to allocate a uuid at that stage. To view existing uuids (in order to use them in your fstab and so on, run the blkid command).

Tags: filesystem, fstab, linux, partition, uuid