Why Are Server Backups So Painful?

{excerpt}The reasons why backups have such a dramatic impact on server performance. {excerpt}

h3. Data Sets Are Growing Rapidly

!disk-capacity-by-year-graph.png|width=754,height=313!

Data sets are growing rapidly and storage is always the most precious commodity a server has. Here is a graph of hard disk capacity available as it has grown over the years. These numbers are compiled using data available at [http://www.mattscomputertrends.com/harddiskdata.html]. If you haven't ever seen [Matt's Computer Trends|http://www.mattscomputertrends.com/], then you are missing out. According to his site, Matt spends his spare time scrubbing old computer magazines to find historical specs on computer components, and the data he has compiled over the last few years is awesome.

h3. Disk Performance Improvements are Not Keeping Up with Capacity Growth

!relative-disk-capacity-performance.png|width=748,height=298!

Here is a graph of relative capacity of hard disks vs. performance. Comparing performance of hard disks is not completely straightforward, so here hard disk transfer rate measured in KB/s is used. The performance measurements are hard to come by. The transfer rates used here are from an excellent article at [Tom's Hardware|http://www.tomshardware.com] that covers [15 years of hard disk performance|http://www.tomshardware.com/reviews/15-years-of-hard-drive-history,1368-7.html] and actually benchmarks the disks to get real performance numbers. While the article is dated from the fall of 2006 and the fastest drive it covers is a 10,000 RPM Western Digital SATA drive the numbers don't matter much when put on a scale compared to disk capacity. Even if we tossed in a 15,000 RPM SAS drive manufactured in 2008, it would not even be noticeable compared to the growth in capacity and [the article on Tom's|http://www.tomshardware.com/reviews/15-years-of-hard-drive-history,1368-7.html] does a good job of driving this point home.

!performance_graph.png!

Here is a graph from the UCLA computer science department used in [UCLA's Computer Science and Engineering|http://cs.ucla.edu/] (CSE 111) [Introduction to Operating Systems lectures|http://read.cs.ucla.edu/111/2006fall/notes/lec15#incommensurate-scaling]. You can see the growth of CPU performance vs. the hard disk. Clearly disk I/O is the most precious resource a computer has.

h3. Backup Technology vs. Hard Disk Capacity

|| *Date* || *Capacity* || *Cost* || *Typical File Counts on Servers* \\ ||
| *1986 GNU Tar was Born* \\ | *70 MB* \\ | *$3,592* | *Hundreds or Thousands* |
| *1990 Legato Networker 1.0* \\ | *270 MB* \\ | *$3,500* | *Thousands* |
| *1999 rsync was born* \\ | *32,000 MB* \\ | *$299* | *100,000 - 500,000* \\ |
| *2005 CDP for Linux Invented by R1Soft* \\ | *300,000 MB* \\ | *$300* | *1,000,000\+* \\ |