BareMetal restoration of Linux (RedHat or CentOS) by ReaR.



Lets talk about two scenarios here.
1.       Consider that you have a physical system which has an OS crash recently. If the system is so critical, you need to restore it as soon as possible to control the business impact. How do you do that?
What we usually do is that, we will install a new OS, make necessary configuration in the system and application side and restore the data. This will take hell lot of time and our business cannot wait till that much time. What is an alternative option to it to reduce the downtime?
2.       You have a physical system and you need to migrate the OS to another system or create an exact replica of that system. How do you do that?
Same solution as what we discussed above? Is it going to take too much time to get it done?
What I suggest is, ReaR (Relax and Recover) . You can restore the exact configuration (replica) of a system from its previous backup.  

Let’s see how it is going to work here.
In this demonstration, We are going to use two systems with different configuration in terms of hard disks capacity and number of network interfaces. I am going to take a back of one system with 10GB root disk and restore it to an 8GB root disk. Since this is a demo, I used two VMs instead of physical systems. I have experience in restoring and migrating physical systems with Rear, so I am pretty sure that the demo in VMs would work exactly like how it works in physical systems.

Our Source system is LABSVR1 with below configuration.
Hard Disk/s : 1 with 20 GB in size used for RootVG.
                         1 with 5GB in size used for dataVG.
Network Card: 1 network card.




Our Destination system is LABSRV2 with below configuration.
Hard Disk/s:  2 disks with 10GB size (No data in it)
                        1 disk with 5GB size (No data in it and planned to install OS in it.)

Now I am going to install the necessary packages in the source system or the system which to be backed up. The packages are
1.       rear – Relax and Recover package, the base utility package for this solution.
2.       syslinux – To create bootable media for recovery.
3.       genisoimage -  as name indicated, to create ISO images.

These packages are available in AppStream  and BaseOS  repositories of CentOS/RHEL 8 systems. For other version of OS, you may need to download them from the external repositories.

# yum install rear syslinux genisoimage -y






The packages are installed now. We need to configure the backup now. The main configuration file for Rear is “/etc/rear/local.conf”. You can refer “/usr/share/rear/conf/default.conf” to get the default configuration of ReaR.

Before we start configuring the ReaR, let me tell you that, I am going to take the backup in to an NFS share. I have an NFS server in my environment which is called as “nfsserver” with a shared directory “/data/nfs_share”. The backup will be taken to that share in this demo.

Edit the file /etc/rear/local.conf and add the below details in it.

OUTPUT=ISO                                                                                   # ISO image as the bootable disk
BACKUP=NETFS                                                                                    # Take backup into a mounted directory.
BACKUP_PROG=tar                                                                    # Tar backup to be taken
BACKUP_TYPE=full                                                                 # To take the full backup
BACKUP_PROG_COMPRESS_OPTIONS="--gzip"                 # gzip method
BACKUP_PROG_COMPRESS_SUFFIX=".gz"                               # .gx suffix for backup file
BACKUP_PROG_EXCLUDE=( '/tmp/*' '/dev/shm/*' '/data/*' )               # directories to be excluded
BACKUP_OPTIONS="nfsvers=3,nolock"                                     # NFS mount options
BACKUP_URL="nfs://nfsserver/data/nfs_share"                 # Destination of backup
AUTOEXCLUDE_MULTIPATH=n                                                    # include multipath disks
AUTORESIZE_PARTITIONS=true                                                 # if source and dest  disks sizes are different
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE=55           # % of shrink.


In case of Physical disks where you use UEFI partitions, you may need to add below entries as well in the configuration.

###option added for UEFI###
USING_UEFI_BOOTLOADER=1
UEFI_BOOTLOADER=( '/boot/efi/grub.efi' 'grub*.efi' '*.efi' )
ISO_MKISOFS_BIN="$( type -p xorrisofs || type -p mkisofs || type -p genisoimage )"
###



  
Now let’s take the backup of the system. Issue the below command in the system to take the backup.

# rear mkbackup -v




The backup has been completed now. The WARNING messages can be ignored as it is related to some open sockets which is expected as per this configuration. If we go to the NFS share, then we can see the backup files taken now.




There are two main files containing the backup and bootable media. The file “backup.tar.gz” contains the actual data backup and the file “rear-labsrv1.iso” is the bootable media used for the restoration. When we restore the system, we start booting it from the ISO image. The content of image will locate the backup and start restoring the system from there.

To schedule the regular backup, put the commands in a script, make it executable and schedule it in crontab like below.

[root@labsrv1 ~]# cat /usr/local/bin/make-rear-backup.sh
#!/bin/sh
set -x
/usr/sbin/rear mkbackup -v  > /tmp/make-rear-backup.out 2>&1
#
[root@labsrv1 ~]#

[root@labsrv1 ~]# chmod 700 /usr/local/bin/make-rear-backup.sh
[root@labsrv1 ~]#

[root@labsrv1 ~]# crontab -l
00 02 15 * * /usr/local/bin/make-rear-backup.sh
[root@labsrv1 ~]#

Now we are going to test the recovery of this system from this backup. We are going to restore it to an entirely different system with less capacity of root disk to see if the features AUTORESIZE_PARTITIONS &
AUTOSHRINK_DISK_SIZE_LIMIT_PERCENTAGE are working fine as expected.

We will boot the system from ISO image which we created by our backup. In my system I selected the bootable CD ROM as iso image which we have created by backup. I have copied the ISO image from the NFS to my /tmp directory of local system and pointed my CD drive to it.




While booting the system, you will get the below options. Select the first one and press Enter key to start recovering the system.



The system started booting used the below kernel now.




As we see here, the system will show up the interfaces available in it and we need to select one of it to configure the IP address of it. Per our configuration, the system will get assigned the source IP address, this may lead to a network glitch with the source server if we plan to close the system. If the source system is already down, then no need to worry on it.

I selected interface 2 and pressed the Enter key as shown below.



Now the system is configured with the IP address same as in the source system and prompted for login. Give “root” as the user and press Enter Key. This will not prompt for root password and will lead directly to the shell.



Limited commands will work here in this recovery shell. To see the IP configuration, use the “ifconfig” command.




Since my source system is still up and running, I made the interface down on the recovery system to control the IP conflict issues.

   
I am going to assign a new IP to this system before we start the recovery.



Since our backup is taken to NFS, the system will look at the backup file (xxx.tar.gz) in the same location. So, the network configuration to be setup before start recovering the system.

To start recovering the system, issue the below command:

# rear recover



In my destination server, I am trying to restore the OS from a 20GB root disk to 10GB one. In the source system /dev/vda is the root disk and the recovery is asking to select the destination disk. We are going to select 3 (/dev/sdc – 6GB) and press Enter key.

In the source system we have another disk /dev/vdb for /data mount point. The recovery is asking to select the destination for it. Selected option 2 which is /dev/sdb.



Now it is showing the mapping which we selected. Left side one indicates the source and the right one for destination. Like wise there are two disks we have selected for the restoration. Selecting the default option to confirm it.


Prompted for the reconfirmation for disk layout. If needed, we can edit it by selecting option 2 (will lead to a file edited by vi command.). In this case we are continuing with the default option.


Partition size has been reduced as shown here since the destination disk is with lesser capacity. Proceed with the selection of default option to start the recovery.




Disk layout has been created now, still it is editable. Check it and proceed with default option if everything looks good.




When we confirmed, the system started restoring the data from the backup file.



Recovery has been completed now. If you need to make any permanent changes to the system, we can edit the files and do it before reboot. The root file system is mounted under /mnt/local now.




As we found above, this system is having the source IP address by default. We need to change it by editing the file “/mnt/local/etc/sysconfig/network-scripts/ifcfg-en9

Changed the IP address from 192.168.1.71 to 192.168.1.72 as shown below. If you want to change the root password, you can change it here too.



Now reboot the system and check all the configurations after booting. Root password will be the one same as its source system.

System has been rebooted now with below details.

1.       System has been restored with IP 192.168.1.72

[root@labsrv1 ~]# ifconfig |grep 192.168.1.7
        inet 192.168.1.72  netmask 255.255.255.0  broadcast 192.168.1.255
[root@labsrv1 ~]#

2.       All the mount points have been created.

[root@labsrv1 ~]# df -Phx tmpfs
Filesystem                 Size  Used Avail Use% Mounted on
devtmpfs                   901M     0  901M   0% /dev
/dev/mapper/rootvg-rootlv  6.5G  1.6G  5.0G  24% /
/dev/mapper/datavg-datalv  2.0G   47M  2.0G   3% /data
/dev/sdc1                  488M  123M  335M  27% /boot
[root@labsrv1 ~]#

3.       No data is available in /data directory since we excluded it from backup [BACKUP_PROG_EXCLUDE=( '/tmp/*' '/dev/shm/*' '/data/*' )]

[root@labsrv1 ~]# ls -l /data |wc -l
1
[root@labsrv1 ~]#

With this we conclude that the restoration is successful.




Comments

Popular posts from this blog

AIX Firefox Installation

AIX Commands

How to increase swap size on CentOS 8 / RHEL 8

How to configure NTP in RHEL8 or CentOS 8 by chronyd.

System Administration

How to manage Linux systems through web browser (cockpit for Linux): CentOS 7, CentOS 8, RHEL 7, RHEL 8, Fedora, Ubuntu and Debian systems.

How to find big files in Linux, Unix, AIX

Remote X11 forwarding for AIX in command line.

How to enable VNC server (get GUI remotely) in CentOS 7, RHEL 7, CentOS 8 and RHEL 8 servers.

How to change Time Zone in CentOS 8, RHEL 8 and Ubuntu systems