Tag Archives: Amazon S3

How-To: Automated incremental daily backups to Amazon S3 using Duplicity

This guide shows how to use Amazon S3 with duplicity to make secure GPG encrypted automated daily incremental backups (snapshots) of a Linux server or desktop. I have been using this method on various servers for several months and it has proved to be a reliable, secure, cheap, and robust method to create automated backups.

I have used this method on Fedora, YDL, and CentOS but the instructions should equally apply to other Linux distributions including Debian and Ubuntu. It will even work on OS X using the MacPorts version of duplicity.

Aims of this guide

This guide explains how to create a simple wrapper script for duplicity that allows you to automatically create GPG encrypted incremental backups that are saved to an Amazon S3 bucket. The script is designed to be executed as a daily cron job so that incremental snapshot backups are created each day. The script creates a full backup set on the 1st day of each month (or when an appropriate full backup cannot be found) and then creates incremental backups on subsequent days.

This guide provides a walk-through of how to create the GPG encryption key, and provides full scripts and example usage for both backup and restore. You could easily adapt the backup script so that it makes full backups each week, or otherwise adjust it to suit your individual needs.

This guide is written with the general Linux user in mind: you do need some understanding of basic linux concepts such as cron, permissions, and directory structures.

Continue reading

Simple Linux to Amazon S3 backup script

There are many tools available to help backup Linux systems to Amazon S3 but finding the right one to use can be difficult. Jeremy Zawodny made a good list of various S3 backup tools which is very helpful, if a little outdated. I experimented with a few tools, including some of the standard scripts published by Amazon but I found each had their own shortcomings.

One tool that is very simple to use is s3cmd which is a linux command line tool to upload, retrieve, and manage data in Amazon S3. The tool is written in python so should install and run on pretty much any modern linux distro and I have found it works very nicely and seems to be an ideal tool to use if you want to write a basic backup script.

Continue reading