Backup

FOSS Project Spotlight: Bareos, a Cross-Network, Open-Source Backup Solution

Heike Jurzik and Maik Aussendorf — Fri, 17 May 2019 12:00:00 +0000

Bareos (Backup Archiving Recovery Open Sourced) is a cross-network, open-source backup solution that preserves, archives and recovers data from all major operating systems. The Bareos project started 2010 as a Bacula fork and is now being developed under the AGPLv3 license.

The client/server-based backup solution is actually a set of computer programs (Figure 1) that communicate over the network: the Bareos Director (BD), one or more Storage Dæmons (SD) and the File Dæmons (FD). Due to this modular design, Bareos is scalable—from single computer systems (where all components run on one machine) to large infrastructures with hundreds of computers (even in different geographies).

Figure 1. A Typical Bareos Setup: Director (with Database), File Dæmon(s), Storage Dæmon(s) and Backup Media

The director is the central control unit for all other dæmons. It manages the database (catalog), the connected clients, the file sets (they define which data Bareos should back up), the configuration of optional plugins, before and after jobs (programs to be executed before or after a backup job), the storage and media pool, schedules and the backup jobs. Bareos Director runs as a dæmon.

The catalog maintains a record of all backup jobs, saved files and volumes used. Current Bareos versions support PostgreSQL, MySQL and SQLite, with PostgreSQL being the preferred database back end.

The File Dæmon (FD) must be installed on every client machine. It is responsible for the backup as well as the restore process. The FD receives the director's instructions, executes them and transmits the data to the Bareos Storage Dæmon. Bareos offers pre-packed file dæmons for many popular operating systems, such as Linux, FreeBSD, AIX, HP-UX, Solaris, Windows and macOS. Like the director, the FD runs as a dæmon in the background.

The Storage Dæmon (SD) receives data from one or more File Dæmons (at the director's request). It stores the data (together with the file attributes) on the configured backup medium. Bareos supports various types of backup media, as shown in Figure 1, including disks, tape drives and even cloud storage solutions. During the restore process, the SD is responsible for sending the correct data back to the FD(s). The Storage Dæmon runs as a dæmon on the machine handling the backup device(s).

Go to Full Article

Weekend Reading: Backups

Carlie Fairchild — Sat, 19 May 2018 12:48:00 +0000

by Carlie Fairchild

Public Service Announcement: please do a backup if you haven't in awhile. This weekend we feature articles varying from scary backup stories to how-to safeguard your data with encrypted backup solutions.

Scary Backup Stories

by Paul Barry

Backups. We all know the importance of making a backup of our most important systems. Unfortunately, some of us also know that realizing the importance of performing backups often is a lesson learned the hard way. Everyone has their scary backup stories. Here are mine.

Reliable, Inexpensive RAID Backup

by Brian C. Lane

As a topic, backups is one of those subject likely to elicit as many answers as people you ask about it. It is as personal a choice as your desktop configuration or your operating system. So in this article I am not even going to attempt to cover all the options. Instead I describe the methods I use for building a reliable, useful backup system. This solution is not the right answer for everyone, but it works well for my situation.

LVM and Removable IDE Drives Backup System

by Mike Fogarty

When the company I work for, a civil engineering and surveying firm, decided to move all its AutoCad drawings onto a central fileserver, we were presented with a backup situation orders of magnitude larger than anything we had confronted before. We had at that time (now considerably larger) about 120,000 files, totaling 200GB, that were in active change and needed to be backed up at least daily.

My first thoughts were of some sort of tape backup system, but as I began to research them, I was shocked at the prices I encountered. A tape autoloader large enough to contain our filesystem ran about $12,000 and a 40Gig tape was $89. When I first convinced my boss to let me run Linux on our servers, cheap was a big selling point. So, what are the alternatives?

Backup and Update

by Shawn Powers

In this video, editor Shawn Powers shows us how to do a basic backup in Linux. Or as he puts it, a public service announcement to please do a backup if you haven't in awhile!

Go to Full Article

Ideal Backups with zbackup

David Barton — Mon, 19 Jan 2015 19:28:30 +0000

by David Barton

Data is growing both in volume and importance. As time goes on, the amount of data that we need to store is growing, and the data itself is becoming more and more critical for organizations. It is becoming increasingly important to be able to back up and restore this information quickly and reliably. Using cloud-based systems spreads out the data over many servers and locations.

Where I work, data has grown from less than 1GB on a single server to more than 500GB spread out on more than 30 servers in multiple data centers. Catastrophes like the events at Distribute IT and Code Spaces demonstrate that ineffective backup practices can destroy a thriving business. Enterprise-level backup solutions typically cost a prohibitive amount, but the tools we need to create a backup solution exist within the Open Source community.

zbackup to the Rescue

After switching between many different backup strategies, I have found what is close to an ideal backup solution for our particular use case. That involves regularly backing up many machines with huge numbers of files as well as very large files and being able to restore any backup previously made.

The solution combines zbackup, rsync and LVM snapshots. zbackup works by deduplicating a stream—for example, a tar or database backup—and storing the blocks into a storage pool. If the same block ever is encountered again, the previous one is reused.

Combining these three elements gives us a solution that provides:

Multiple versions: we can store complete snapshots of our system every hour, and deduplication means the incremental storage cost for each new backup is negligible.
Storing very large files: database backups can be very large but differ in small ways that are not block-aligned (imagine inserting one byte at the beginning of a file). Byte-level deduplication means we store only the changes between the versions, similar to doing a diff.
Storing many small files: backing up millions of files gives a much smaller number of deduplicated blocks that can be managed more easily.
Easily replicating between disks and over a WAN: the files in the storage pool are immutable; new blocks are stored as new files. This makes rsyncing them to other drives or machines very fast and efficient. It also means we can synchronize them to virtually any kind of machine or file storage.
Compression: compressing files gives significant size reductions, but using it often stops rsync or deduplication from working. zbackup compresses the blocks after deduplication, so rsyncing is still efficient. As mentioned previously, only new blocks need to be rsynced.

Go to Full Article

Crashplan, the Only Reason I Install Java

Shawn Powers — Tue, 05 Mar 2013 15:41:29 +0000

by Shawn Powers

I'm the sort of person who doesn't like to install Java. I actually don't like to install Flash either, but it's still tough to survive browsing the Internet without Flash installed. There is one program that makes me break my own rules, however, and that's Crashplan.

For years, I've been singing the praises of BackupPC, and for servers, I still think it's the best thing going. The problem with BackupPC, however, is in order for it to work reliably, your workstations need to be on all the time. This is especially difficult with laptops.

Crashplan is an incredibly powerful backup utility that allows local or offsite backup, and the company offers cloud-based storage for reasonable rates. Normally, I wouldn't be so excited by a paid service and a non-open-source software package, even if it does offer a Linux-native client. The folks at Code 42, however, have given away the ability to swap storage with friends as an alternative to their paid-cloud-based service. If you have a computer at work, and a computer at home, you can back them up over the Internet to each other completely free!

As I already mentioned, I normally don't like Java-based programs like Crashplan, but its functionality is so great, I don't mind breaking my own rules. More than that, I give a lot of credit to Code 42 for not only making a native Linux client, but also for giving away incredible functionality for free. If you're not backing up your data, be sure to consider Crashplan at http://www.crashplan.com. Its price, feature set and generous non-paid features make it this month's Editors' Choice!

Go to Full Article

BackupPC

Shawn Powers — Thu, 17 Mar 2011 13:00:00 +0000

by Shawn Powers

Some tools are so amazing, but unfortunately, if no one ever talks about them, many folks never hear of them. One of those programs is BackupPC. You may have heard Kyle Rankin and myself talk about BackupPC on the Linux Journal Insider podcast, or perhaps you've seen us write about it here in Linux Journal before. If you haven't checked it out, you owe it to yourself to do so. BackupPC has some great features:

Hard drive-based backups, no tape swapping.
Support for NFS, SSH, SMB and rsync.
Hard linking to save valuable disk space.
Individual files can be restored in place in real time.
Powerful and simple Web interface.
E-mail notification on errors.
Free!

BackupPC is one of those projects that doesn't get updated terribly often. It doesn't have flashy graphics. It doesn't require constant maintenance. It just works, and it works well. Check it out: backuppc.sourceforge.net.

Go to Full Article

Clonezilla Live

Michael Reed — Fri, 03 Sep 2010 15:44:17 +0000

by Michael Reed

Clonezilla is a bootable CDROM designed for partition backup and restoration. Unlike SystemRescueCD, Clonezilla Live doesn't contain an array of utilities, rather, it is a single, focused tool. However, if you're interested in simply backing up or restoring whole partitions to or from files, or copying one partition onto another, Clonezilla might be just what you're looking for.

There are two primary uses for a tool such as this one: backup and subsequent restoration in the event of a mishap or creating a clone of an existing system. So, you could install Linux on one machine, backup the entire disk to a file and then copy the setup to other machines. On the other hand, the partition imaging allows you to do a system backup that can restore a complete system, unlike a traditional backup utility that can only restore your files.

When imaging to a file, the resulting file should be smaller than the entire size of the partition because Clonezilla doesn't back up the free space. It has support for most of the file systems that you are likely to encounter and it can backup those that it doesn't recognise, although this results in larger files. When restoring a partition, the hard disk drive must be the same size or larger than the source hard drive, but you can copy a smaller hard drive onto a larger one.

Note that another version of Clonezilla, Clonezilla SE (Server Edition) is designed for restoring partitions to multiple machines via a network for mass cloning. Clonezilla Live, the version that we are discussing here, can restore or backup a single partition over a network or a removable storage device such as a USB stick, or even another local hard drive. A partition image file can't reside upon a partition that is going to be operated upon.

Now that we've determined what Clonezilla is for, how easy is it to use? The answer is that the procedure is very simple. The start up menu is, as you might expect, mainly orientated towards starting the partition copying utility, although it does feature options for network booting, starting FreeDOS or running Memtest. This means that, if armed with only a Clonezilla Live disc, you might find yourself stuck if you needed to edit some files or even edit the partition table of a disk.

Go to Full Article

Stupid afio Tricks

LJ Staff — Fri, 23 Jul 2010 13:00:00 +0000

by LJ Staff

I've already covered tar and all the wonderful ways to use it, but it's not the only tool at your disposal. Another popular backup tool is afio. Depending on your distribution, it already may be installed. On Ubuntu, for example, run the following to install it:

sudo apt-get install afio

Now you have a fairly powerful tool at your disposal for archiving files and making backups.

By default, afio reads and writes the files being archived on standard input and standard output. This means you can create a list of files to archive with another program, like find, and pipe it to afio to do the actual archive. Once you have your list of files, you can apply five basic commands to those files:

-o — create an archive.
-i — install (or unpack) an archive.
-t — test (or list) the files stored in an archive.
-r — verify the files stored in an archive against the filesystem.
-p — copy the files to a given directory location.

If you want to create a simple archive of all of your C source code files, execute:

find . -name *.c -print | afio -o -Z source_code

When you want to extract these files again, execute:

afio -i -Z source_code

When you run afio as a regular user, all file paths are stored with no leading /. This means when you unpack an archive, it unpacks in the current directory. The idea is to avoid overwriting system files accidentally. To keep the leading /, use the command-line option -x. Running afio as the superuser reverses this behavior—any leading / is maintained, and you need to use the command-line option -X to get the usual behavior of stripping the leading /.

If space is at a premium, afio also can compress your archive, just like tar, with the -Z command-line option. There is one very big difference, however. When you compress a tar archive, the entire archive file is compressed. This means if one part of the file is corrupted, you potentially could lose all the files in the archive. When you compress an afio archive, the archived files actually are compressed individually. So if one file becomes corrupted, by whatever means, you won't lose any of the other files in the archive.

When you compress an archive, afio uses gzip by default. You can tell gzip what compression factor to use with the -G num command-line option, where num is the amount of compression gzip will apply to the archived files. This is a number between 0 (for no compression) and 9 (for maximum compression), with a default of 6.

Go to Full Article