Best Practices for Identifying Data for Online Backup
Data can be categorized into three tiers: critical (integral to ongoing operations), important (valuable but not mission critical) and archival (required but not typically retrieved).
The Online Backup Guide for Service Providers is a complete 196-page guide on starting and operating an Online Backup Service – the latest revision of Rob Cosgrove’s industry defining RBS Book originally published in 1987. The entire book is being published here, chapter by chapter.
Usually, the customer just wants everything backed up. Of course this isn’t possible at a reasonable price with today’s technology. Most hard drives contain many gigabytes of data, most of which is non-critical and can be recovered from original distribution CDs and other sources. Computers are not yet fast enough to prepare that much data to transmit over the Internet in a reasonable amount of time, and upload bandwidth is inadequate.
Here’s an example. The computer I’m using right now contains 43.3 gigabytes of data. My Internet connection is fairly fast – 12 Mbps downstream and 3 Mbps upstream. So, assuming 100% efficiency, I can theoretically send all this data up to the Internet in 1 day, 8 hours, 4 minutes, and 26.67 seconds.
That doesn’t sound too bad. But, no Online Backup Service Provider is going to take my connection at 3 Mbps, and I’ll never achieve 100% efficiency. The best I can hope for is an overall bit rate of (maybe) 900 Kbps, which is going to take 4 days, 10 hours, and 54 minutes. That’s too long.
Most businesses have an upstream bandwidth less than 900 Kbps. 512 Kbps is more common. That would take twice as much time.
In addition to the time it takes to do a backup, we must consider the time it takes to restore, called the Recovery Time Objective (RTO). This is a more important number because it directly impacts a company’s down time in the event of a disaster.
What is the acceptable amount of time between the loss of data and its recovery? If there is too much data (because of bandwidth) to download in an acceptable period of time, the Service Provider may have to deliver the restored data on a USB drive. This is where physical proximity to your clients is important. You could overnight a drive if your customer’s RTO is twelve hours, and depending on what time of day the data loss occurred.
You can test your upstream and downstream bandwidth at the following URL:
http://www.dslreports.com/tools
Then you can use one of my handy dandy online calculators to figure out how long it will take to do a backup:
http://www.remote-backup.com/calculators
Clearly you have to be selective about which files you back up.
Include & Exclude
Online Backup software contains rules that govern which files are backed up and which files are not. In RBackup and Mercury files and folders can be included and excluded from backups many ways. I won’t go into their specifics, but I will explain the basic concepts.
Backup Sets
Because different groups of files have different tiers (critical, important and archival) which coexist on the same hard drive, Online Backup software needs a way to assign include/exclude parameters and schedules by file groups called Backup Sets.
Some Backup Sets contain critical data that needs to be backed up every day and kept for twelve months, with a Recovery Time Objective of three hours, and the ability to do a point-in-time restore.
Some Backup Sets contain less important data that only needs to be backed up every two days, only kept for ninety days, and doesn’t need point-in-time.
Part of your job as a consultant is to help your customers design a backup strategy that backs up the correct files often enough to meet the company’s RTO and RPO, while archiving data for the required amount of time, and providing the ability to roll back through multiple versions by date or version number.
Synthetic Full Backups
In the old days when people backed up to tape drives a certain strategy had to be used. The most popular was Grandfather/Father/Son tape rotation. This method required 19 to 23 tapes and ensured that there were always backups of the seven most current days, the four most current weeks, the eleven most current months, and the most current year.
This method required keeping close tabs on which tape was which, and rotated them so most of the tapes received even wear from the tape machine, so they could be replaced in groups.
Restoring from tapes required finding the tape containing the backup that had the version of the file you want, and restoring it from tape. To do a complete disaster recovery required restoring first the most recent full backup, then all incremental backups since then. It was a long and arduous task. Tapes would break, or would become unusable because of heat, humidity, wear, and magnetic distortion.
When I designed the first Online Backup software, I predicted a problem with that kind of strategy, and saw an opportunity to employ a powerful function that tape didn’t have – Indexing and database-driven restore.
This gave rise to what was later called Synthetic Full Backup – full backups that are built out of pieces of previous incremental and full backups already on the Server without the need to re-send data to the Server. It saves a lot of time on restore and saves space on the Server.
In addition, it allows for creating full restores to any point in time, and the on-the-fly custom design of virtually any type of restoration of any subset or groups of subsets of full backups by date, file size, file type, backup set, drive location, customer-defined “tags,” file name or folder name, or any other file attribute. Restores became very granular, and very fast, and can be done all in a single pass without the need to restore from multiple backup sets.
As far as I know, my RBackup and Mercury are the only two Online Backup packages currently available that can process Synthetic Backups.
Rob Cosgrove, CEO Remote Backup Systems
Rob Cosgrove is the President of Remote Backup Systems, founder of the Online Backup Industry, and a vocal advocate for maintaining the highest standards in Online Backup software. His latest book, the Online Backup Guide for Service Providers: How to Start and Operate an Online Backup Service, is available online now, on Amazon.com, and at bookstores.
Remote Backup Systems provides brandable, scalable software and solutions to MSPs and VARs enabling them to offer Online Backup Services.
