HOWTO: Multi-boot Solaris, RHEL, SLES and Nevada

Once upon a time I let it be known that I’d done a fair bit of “playing” with Linux and as a result I attended a RedHat training course. As a result of this, I’ve been given the task of being one of the primary x86/x64 engineers in the UK solution centre. With this responsibility came a lovely new Ultra 20 to keep my feet warm and the requirement that I need to be able to offer assistance in supporting all the *NIX varieties that Sun supports on it’s hardware.

Well, never put off my a new opportunity and a chance to play with learn something new, I gratefully accepted the Ultra 20 and set myself the task of installing RedHat Enterprise Linux, SuSE Linux Enterprise Server, Solaris 10 and Nevada onto the same disk on the same machine.

I can happily report, I’ve done this now and thought I’d share my method.

The Partition Table

First, the partition/slice table. As we’re installing several operating systems on the one disk, we need to do a bit of planning. What’s more, the current Solaris and OpenSolaris installer will only allow one Solaris partition (primary or extended) per disk. You can actually cheat and work around this using GRUB to change the partition types on the fly, but I haven’t gone this route.

This is what I’ve gone for:

Part #	Size	Type	Mount Point	Notes
1	20G	EXT3	/ (RHEL) /mnt/redhat (SLES)	This will be the root partition for RHEL
2	50G	UFS	s0 - Sol 10 (10G) s1 - Swap (2G) s3 - Nevada (10G) s4 - ABE (10G) s5 - ZFS (4G) s6 - ZFS (4G) s7 - /export (10G)	s1 will be used by Linux for SWAP too [#] s3/4 are interchangeable and will be used for LiveUpgrading Nevada and Solaris 10 s5/6 are so I can play with ZFS without affecting my OS or data s7 will be for Solaris specific apps needed on both Solaris and Nevada
3	20G	ReiserFS	/ (SLES)	I’ve stuck with the default filesystem for SLES
4	60G	Extended Partition
5	10G	FAT32	/mnt/data	This gives me a partition I can mount on ALL of the operating systems on this machine. I use it to store all host config info to start off with.
6	50G	TBA	TBA	This is a scratch partition that can be used by any OS for any reason. It can be formatted with any filesystem as and when it’s needed. I only envision this being used for things like testing other operating systems and other OS functionality.

The Installation Order

Whilst the installation isn’t too important, I found the following to be the easiest method and removed the need to reload boot blocks or re-install GRUB. Ultimately, as Nevada will be my primary OS, I’ll use Nevada’s GRUB.

1. Install RHEL 4u4 and create the basic partition table (as above). It doesn’t matter what filesystems you use for partitions 2 and 6 as we can reformat these later. Select EXT3 for partition 3 - this isn’t too important though as the SLES installer will reformat it later.

   Once the OS has been installed, there is one thing you need to do - change your /etc/fstab so partitions are mounted by device name and number and not label. ie replace LABEL=/ with the correct /dev/sd?# device. This is needed as only RedHat seems to use the labels, and if they are overwritten later (by SLES or Solaris), then RedHat will still boot correctly.

2. Install SLES 10 onto partition 3 - reformatted to ReiserFS - as this is the desired default for SLES 10.

   Once the OS is installed and booted, I did the following to ease the next few steps:

   • changed the partition type for partition 2 to “be” using fdisk (# fdisk /dev/sda > t > 2 > be > w). This is needed so the Solaris 10 installer can find the Solaris partition.
   • copied /boot/grub/menu.lst to /mnt/data - I need this for Nevada as it won’t pick up the Linux partitions automatically like Linux does.
3. Install Solaris 10 11/06 (update 3) onto partition 2. As part of the installation, slice up the partition as specified above. To get the installer to install into partition 2, I needed to use one of the “text” options as the graphical installer wouldn’t play ball nicely. The text options also give you a lot more freedom with your settings.
4. Install Nevada onto partition 2, slice 3. Be sure to preserve all other slices.

   Once the OS was installed I added the RHEL and SLES grub entries into Nevada’s /boot/grub/menu.lst from the file I copied earlier to /mnt/data.

Additional Notes

Linux Using the Solaris SWAP slice:

Now you may be curious to know how Linux is going to be able to use the Solaris swap slice for it’s own swap partition.

Well, when Linux creates the devices for each partition, it will detect the presence of the Solaris fdisk partition table on partition 2, and will process this too and create the appropriate devices. The pseudo-partition numbers start immediately after the last physical partition. This can be seen in /var/log/messages or the dmesg output with lines that look similar to this (your output may differ):

sda: sda1 sda2 sda3 < sda5 sda6 sda7 >
sda2: <solaris: [s0] sda8 [s1] sda9 [s2] sda10 [s3] sda11 [s4] sda12 [s5] sda13 [s6] sda14 [s7] sda15 >

You can now add the appropriate entry to your /etc/fstab file.

One more hurdle though - to use it as Linux swap, it must be properly formatted. Run a mkswap from the Linux side to format the swap partition. Fortunately Solaris skips over the header so it won’t get mangled.

Easy SSH:

As you’re multibooting, each OS will have a different SSH host key. This plays havoc with your known_hosts file, so to ease ssh connections, copy the /etc/ssh/ssh_host_*_key* from one of the operating systems to /etc/ssh/ on each of the other operating systems installed.

Problematic NFSv4:

As part of my configuration, I export a filesystem for access from remote systems. Unfortunately, the NFSv4 implementation on Linux is just plain wrong and causes havoc with the Solaris automounter. Accordingly, I had to disable NFSv4 on RHEL and SLES.

Screenshots

Of course we need some screenshots too. I personally think the SLES desktop is the cleanest and most pleasing on the eye with Nevada coming in a close 2nd. Solaris 10 is horrible.