OpenMosix- Linux Supercomputer
Most of us dream of using a Linux Supercomputer, something with so much raw processing power and memory that operations get completed in nanoseconds rather than minutes. With hardware becoming cheaper, most of us are accustomed to working on machines with 4-8 GB of RAM, and occasionally even using dual or quad core processors.
However, what if we told you that there's a simple way to build your own supercomputer. That too, using nothing more than GNU/Linux and any old hardware you happen to have lying around. The basic idea is to cluster multiple systems together, and use their combined CPU power and combined RAM as if it is one system.
This concept of multiple physical machines contributing their processing power and behaving like a single system is known as 'Single System Image' clustering. In other words, the cluster behaves like a normal single system to the end-user.
The key to doing this is to use a system known as 'openMosix' in conjuction with Linux. OpenMosix is an extension to the Linux kernel that allows for seamless clustering and load balancing of processing power over systems on a network. This means that you can have say 5 low-end machines with 256 MB RAM, install an openMosix enhanced Linux kernel on them, and effectively have a system that has 5 CPUs and 1,280 MB RAM! This idea scales very nicely, imagine a setup with 10 systems, each with 512 MB RAM... you can cluster them, and get an extremely powerful 10 CPU, 5 GB RAM monster to play with!
Anyway, now that we've got you drooling, we'll show you the simplest way to set up your own GNU/Linux cluster, explain the technology behind it, show you how to optimize it, and finally give you a couple of interesting ideas on what to do with your behemoth cluster.
Before we start, let's take a quick look at what we've got covered in the following pages for you:
- Section 1: Understanding OpenMosix.
- Section 2: Building An OpenMosix Cluster.
- 2.1: Getting & Installing OpenMosix.
- 2.2: Installing from Source.
- 2.3: Installing from RPM.
- 2.4: Installing in Debian.
- Section 3: Using Cluster Knoppix.
- Section 4: Starting Up Your Cluster.
- Section 5: Testing Your Cluster.
- Section 6: Controlling Your Cluster.
- Section 7: Openmosix File System.
- Section 8: Using SSH Keys Instead of Passwords.
- Section 9: Interesting Ideas.
- 9.1: Distributed Password Cracking
- 9.2: Clustered Audio Encoding
This is a great project to take up as there are lots of practical uses for clusters, especially in scenarios where you suddenly require a large amount of processing power (sudden mail server load? ;) ).
Having a basic knowledge of Linux will make things easier for you to understand, since we'll be patching and compiling the kernel, but we've written this tutorial so that it will be accessible to newbies as well.
So, without any more delay, lets start going through this awesome tutorial!
Most of us dream of using a Linux Supercomputer, something with so much raw processing power and memory that operations get completed in nanoseconds rather than minutes. With hardware becoming cheaper, most of us are accustomed to working on machines with 4-8 GB of RAM, and occasionally even using dual or quad core processors.
However, what if we told you that there's a simple way to build your own supercomputer. That too, using nothing more than GNU/Linux and any old hardware you happen to have lying around. The basic idea is to cluster multiple systems together, and use their combined CPU power and combined RAM as if it is one system.
This concept of multiple physical machines contributing their processing power and behaving like a single system is known as 'Single System Image' clustering. In other words, the cluster behaves like a normal single system to the end-user.
The key to doing this is to use a system known as 'openMosix' in conjuction with Linux. OpenMosix is an extension to the Linux kernel that allows for seamless clustering and load balancing of processing power over systems on a network. This means that you can have say 5 low-end machines with 256 MB RAM, install an openMosix enhanced Linux kernel on them, and effectively have a system that has 5 CPUs and 1,280 MB RAM! This idea scales very nicely, imagine a setup with 10 systems, each with 512 MB RAM... you can cluster them, and get an extremely powerful 10 CPU, 5 GB RAM monster to play with!
Anyway, now that we've got you drooling, we'll show you the simplest way to set up your own GNU/Linux cluster, explain the technology behind it, show you how to optimize it, and finally give you a couple of interesting ideas on what to do with your behemoth cluster.
Before we start, let's take a quick look at what we've got covered in the following pages for you:
- Section 1: Understanding OpenMosix.
- Section 2: Building An OpenMosix Cluster.
- 2.1: Getting & Installing OpenMosix.
- 2.2: Installing from Source.
- 2.3: Installing from RPM.
- 2.4: Installing in Debian.
- Section 3: Using Cluster Knoppix.
- Section 4: Starting Up Your Cluster.
- Section 5: Testing Your Cluster.
- Section 6: Controlling Your Cluster.
- Section 7: Openmosix File System.
- Section 8: Using SSH Keys Instead of Passwords.
- Section 9: Interesting Ideas.
- 9.1: Distributed Password Cracking
- 9.2: Clustered Audio Encoding
This is a great project to take up as there are lots of practical uses for clusters, especially in scenarios where you suddenly require a large amount of processing power (sudden mail server load? ;) ).
Having a basic knowledge of Linux will make things easier for you to understand, since we'll be patching and compiling the kernel, but we've written this tutorial so that it will be accessible to newbies as well.
So, without any more delay, lets start going through this awesome tutorial!
Tags: Linux OpenMosix File System installation Supercomputer clustering Knoppix Cluster ssh multi-processing
