Sooner or later, your operations are going to need to scale out to bigger and better servers. You’ll have to migrate your application to new hardware. Here’s how to do it. This is more of a checklist than a full guide, as a full guide would be far too specific to your requirements. We’ll be assuming a typical LAMP stack, along with email hosting, as this is a very common combination among small web startups.
First and foremost, take inventory of what needs to be moved, and make a full backup of absolutely everything. Due to hardware differences, and hosting company requirements, it probably won’t be possible to simply restore your backup on the new server.
Go in to your DNS provider’s settings and change your site’s TTL time to the minimum of one hour. This means it will take at most one hour for users to see the new DNS entries when you make the migration. Once you’re done moving, be sure to change it back.
We will be doing this move in two stages, the first stage moves the services and configurations and allows for testing; the second stage shuts down the previous server for good, and migrates all of the application data to its new host.
For the purposes of this tutorial, we will assume a Debian-based system, and that you’ll be working as root the entire time. These commands are mostly distribution-agnostic, however some file locations might be Debian specific.
To make a full backup, first stop all of the services that write to the database.
service apache2 stop service postfix stop service dovecot stop service mysql stop
Then do the backup. We’re not going to compress it at this time, as compressing takes a very long time, and during this time your services are offline. We’ll also take this time to make proper SQL backups. Make sure you have enough room in the root partition to hold all of this, or put them in a place where there is enough room (such as under /var or /home).
mysqldump -u root -p reallyimportantdatabase > /reallyimportantdatabase.sql mysqldump -u root -p someotherdatabase > /someotherdatabase.sql tar cvpf /backup.tar --exclude=/proc --exclude=/dev --exclude=/lost+found --exclude=/backup.tar --exclude=/mnt --exclude=/sys
Once it is done, start your services back up again.
service apache2 start service postfix start service dovecot start service mysql start
Check the size of the backup. If it is less than, say, 10GB, don’t bother compressing it, as you’ll spend more time compressing and decompressing than you will transferring it. If it is significantly over 10GB, I suggest using pbzip2 to compress it, which is a backwards compatible bzip2 algorithm that does the compression in parallel. This means its speed increases linearly with how many cores you have available. If a bzip2 operation should take ten minutes, and you have two cores, it will take about five minutes with pbzip2. You can get it at http://compression.ca/pbzip2/, which also contains usage examples. It is contained within many distribution’s repositories, so try those first.
pbzip2 /backup.tar scp /backup.tar.bz2 firstname.lastname@example.org:/home/admin
We’re now done with the old server for the rest of Stage One. All of the following instructions will take place on the new server, until we’re ready for Stage Two. Make sure you take the time to install your preferred management utilities such as GNU Screen and vim before proceeding. Also be sure to set up networking properly.
Let’s unpack the backup that we transferred over.
cd /home/admin mkdir oldmachine cd oldmachine mv ../backup.tar.bz2 . pbzip2 -d backup.tar.bz2 tar xvf backup.tar
While that's going, now's a good time to install the services and support utilities that were on the old server. This list will vary wildly depending on your needs, but a minimal command for Debian might look a bit like this:
apt-get install fail2ban postfix postfix-mysql dovecot-imapd dovecot-pop3d dovecot-mysql apache2 libapache2-mod-php5 php5-apc php5-gd php5-mcrypt php5-mysql mysql-server
Once all of that is done, we’ll move the stock configurations out of the way in preparation for restoring the old configurations.
cd /etc mv dovecot dovecot.dist mv postfix postfix.dist mv php5 php5.dist mv mysql mysql.dist mv apache2 apache2.dist
Now copy all of the old configurations and support files from the backup to their new home. This list depends on your exact setup, but this should give you a general idea of what to look for. Note that we are not copying the SQL database files during this, they will be restored in their proper manner in just a bit. Don’t forget to copy over any SSL keys you may have!
cd /home/admin/oldmachine cp -a var/www/* /var/www/ cd etc cp -a dovecot /etc cp -a postfix /etc cp -a php5 /etc cp -a mysql /etc cp -a apache2 /etc cd ../var cp -a mail /var
Now we hope for the best and restart our services. Everything should go smooth, but, you never know. If your distribution changed, there will likely be problems with minor things like the location of pid files. These things should be very easy to fix, just be sure to take a look at your logs.
service apache2 restart service postfix restart service dovecot restart service mysql restart
Now it’s time to import your databases the proper way.
mysql -u root -p reallyimportantdatabase < /home/admin/oldmachine/reallyimportantdatabase.sql mysql -u root -p someotherdatabase < /home/admin/oldmachine/someotherdatabase.sql
Stage One is now complete. Test absolutely everything, then test it some more. Once you’re done testing, test it all again. This is your only chance to get things right without incurring undue downtime on your users, so make use of it. The exact method of testing depends on your application, but it will at the least involve some fake entries in to your local machine’s hosts file.
Now that you are satisfied that your new server is working properly, head over to your old machine and put your application in some sort of maintenance mode. The idea is to present the users with a page saying “we’re moving servers, thank you for your patience”, and for the application to not write to the database at all.
Now we back up and move the application’s database one final time.
mysqldump -u root -p reallyimportantdatabase > /reallyimportantdatabase.sql scp /reallyimportantdatabase.sql email@example.com:/home/admin
And then we import it on the new machine.
mysql -u root -p reallyimportantdatabase < /home/admin/reallyimportantdatabase.sql
The last step is to move your DNS to point at the new servers. Don't forget to change your SPF records to reflect the new email server's IP address. That's it! You're done! To make sure it worked, keep a close eye on your Apache access logs.
tail -f /var/log/apache2/access.log