cenolan.com » Linux

Simple time lapse video in Linux

Chris — Sun, 10 May 2009 16:24:37 +0000

Time lapse (or stop-motion) video is really cool, and this post explains how to make a simple time lapse style video in Linux using entirely free open source software.

There is more than one way to skin a cat and when I was searching for how to do this I came across many different methods and suggestions but not really anything that suited what I wanted. Andrew Wells suggests making a movie and then processing it with ffmpeg to only store 1 in every n frames. That seems a neat solution but I wanted to take a series of still shots and string those together into a movie. Tim Nugent published a teaser of some nice looking time lapse software he wrote but as yet there is no published source or binary. There were various other suggestions dotted around the web but each one I tried had some problem or other. So here's how I did it.

First of all, you need a working webcam. Webcam support has really improved recently in Linux and I found the built-in iSight on my MacBook and my cheapo Logitech USB webcam work just fine without any intervention from me.

Next we need some way to save an image from the webcam every x seconds. I found that camstream does the trick just fine despite it's sucky 1996-looking website and lack of a release since 2006. Under Fedora 10 to install camstream just use yum or your favourite GUI package manager:

yum install camstream

Camstream is pretty easy to use. Really, a child could do it. Just use the "File" menu to open up your webcam device. Then click the little configuration icon (it looks like a spanner) to adjust the file settings you'd like to use for capture. I found that JPG images work much better than PNG.

As the "Basename" enter the full path of the filename you'd like to save your images as, for example if you want to save your images as "/home/yourname/Pictures/Webcam/MyTimeLapse/image001.jpg" (and so on) then set the basename as "/home/yourname/Pictures/Webcam/MyTimeLapse/image" then choose "Number sequence" and set the maximum sequence number to something pretty big (like 100000). Close the settings. Click the icon to "show last snapshot".

Now click "take snapshot at regular intervals" and set your interval. I chose 3 seconds. Choose whatever you think is appropriate. Now you're all set. Camstream will take images at the interval chosen and save them into the directory you configured. Sweeeeet.

Ok, so now we have a directory full of images that looks something like this:

$ ls
image000.jpg  image003.jpg  image006.jpg  image009.jpg  image012.jpg  image015.jpg  image018.jpg
image001.jpg  image004.jpg  image007.jpg  image010.jpg  image013.jpg  image016.jpg
image002.jpg  image005.jpg  image008.jpg  image011.jpg  image014.jpg  image017.jpg

We're gonna feed these files to mencoder for it to pull together into a movie. To do this we need a text file containing a list of the files in the order that we want them. You could write this file yourself (boring) but we can create it using something like this:

ls -1tr > files.txt

This provides a listing (ls) of the current working directory outputting 1 file per line (1) sorted by modification time (t) in reverse order (r) and sends the output (>) to a file named files.txt. In the example above my files.txt looks like this:

image000.jpg
image001.jpg
image002.jpg
image003.jpg
image004.jpg
image005.jpg
image006.jpg
image007.jpg
image008.jpg
image009.jpg
image010.jpg
image011.jpg
image012.jpg
image013.jpg
image014.jpg
image015.jpg
image016.jpg
image017.jpg
image018.jpg

Now download and install mencoder. Again it's available on nearly all modern distros. For Fedora 10 just do "yum install mencoder". Now all we need do to make our move is issue a command like this:

mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4 -o test.avi -mf type=jpeg:fps=20 mf://@files.txt

This will encode all the files listed in files.txt into a movie called test.avi using the mpeg4 encoder with no sound and a framerate of 20 frames per second. Messing with the fps gives very different results. See "Encoding from multiple input image files" in the mencoder documentation for more info.

Here's an example video made using this method:

Installing Fedora 10 on a MacBook

Chris — Sat, 29 Nov 2008 16:55:45 +0000

This guide will help you through all the steps necessary for installing Fedora 10 on a MacBook Santa Rosa. This guide is currently aimed at Fedora 10 x86_64 but will also work on i386 version (adjust as necessary). Most of the steps equally apply to pre-Santa Rosa models too.

These instructions will work with the MacBook 3,1 (Late 2007) and newer but will NOT work with the new aluminium MacBooks since they have different hardware (especially the graphics card). These instructions are not suitable for the MacBook Pro either!!

This guide is based largely around my previous guide for Fedora 8, and again much credit goes to the people who created the original Ubuntu wiki guide.

What works and what doesn't?

There are a few things that need fixing (covered in detail below) but the following all work "out of the box" with Fedora 10: video/graphics, compiz effects, firewire, CD/DVD reading and writing, function keys (brightness, volume etc), sound, and ethernet. Even the new "plymouth" graphical boot screen works fine once activated.

I have yet to try the infrared or connecting an external monitor. Everything else works with the tweaks described below, except for suspend and hibernate which seem generally pretty broken in F10 (as it was in F9 too). Update 7 November 2008: Suspend and hibernate are fixed with 2.6.27.7-134 kernel and newer. See bug report.

Basic installation instructions

If you're new to installing Linux then these steps should help you. If you're an old hand then skip below to view the post-installation steps necessary to fix the things that don't "just work".

Update the MacBook firmware to the latest version. If you prefer, you may use the standard "Software Update" in OS X.
Get your Fedora 10 installation DVD ready. Note: you should use the x86_64 version of Fedora 10 and that I have only tested this using the Install Media, not the Live DVD.
Install rEFIt on OS X if you want a boot menu to select between OS X/Fedora to appear on every boot. This is highly recommended but optional - this guide assumes you have installed rEFIt. You can make Fedora boot by default and change the menu timeout by editing the "refit.conf" file (described on the rEFIt website).
Use the OS X partition manager in Disk Utility to resize your OS X partition and make space for Fedora. On my 260GB hard drive, I shrank the OS X (hfs+) partition to 120GB to leave about 130GB for Fedora. Insert your Fedora 10 installation DVD in the drive and reboot.
If you have rEFIt installed, then on boot you will be given the option to boot from the DVD, choose it. If you don't have rEFIt installed, then hold down "C" to boot from the DVD.
Install Fedora as usual, except:

In the partitioner, select to manually edit partition table
Create a 200MB boot partition with mountpoint '/boot' (this will be /dev/sda3)
Create a new ext3 partition with mountpoint '/' - this will be your main Fedora partition on /dev/sda4. Make sure to leave 2GB free space after this partition for your swap partition
With the remaining 2GB, create a linux swap partition - this will be /dev/sda5

Finish the installation and reboot.
If you installed rEFIt, you should have a choice between booting OS X and Linux. Use the arrow keys and Enter to select Linux.
Log in to Fedora
Follow the steps in the sections below to work around things that still do not work out-of-the-box in Fedora 10.

Post installation

Note: The wireless on the MacBook doesn't work out of the box in Fedora, so you're going to need an ethernet internet connection available until we get round to fixing the wireless later in this guide.

It is very important now to update your Fedora installation to the latest packages using yum. To do so:

su -
yum update

Once yum has finished updating, reboot, and then do a yum update again. Repeat this process until yum finds no more packages to update.

Fixing the wireless

The MacBook comes with a Broadcom 4328 wireless chipset. This is not supported by default in Fedora 10 and there are two options to make it work: using ndiswrapper and Windows drivers or using the new official Broadcom STA wireless driver. In this guide we will use the official Broadcom driver because it is easier and more reliable.

1) Enable the rpmfusion non-free repository.

The broadcom-wl and wl-kmod RPMs that we need are in the rpmfusion.org non-free repository which also requires the rpmfusion.org free repository. To enable these repos in Fedora simply do this:

su -c 'rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm'

2) Update and install the driver package

Now that you have the appropriate repository enabled, to install the driver package we first ensure that we have the latest updates and then install the "broadcom-wl" package which will bring with it the required "kmod-wl" package:

su -
yum update
yum install broadcom-wl

That's all there is to it. At this stage you may need to reboot in order to enable the new driver and any new kernel that was installed during the update. For more details or if you have any difficulties see this post.

When future kernels are released a simple "yum update" command should install the new kernel and also pull in the updated Broadcom driver for that new kernel.

Fixing the touchpad

(Thanks to bodom_lx for this tip)

The touchpad works out of the box but there is no way to right or middle click. Traditionally this was fixed by adding the appropriate options to /etc/X11/xorg.conf but the new xorg-server in F10 doesn't use the traditional xorg.conf file. Instead it is configured through HAL fdi configuration files.

To create the appropriate file for configuring the touchpad:

su -
gedit /etc/hal/fdi/policy/appletouch.fdi

Copy and paste the following into the empty file:

Save the file and reboot. You should now be able to right-click by using a two finger tap on the touchpad and middle-click by using a three finger tap. You can also right-click by holding two fingers on the touchpad and clicking the button, and middle-click by holding three fingers and clicking the button.

If you want to adjust any of the options (or add new ones) see "man synaptics" for an explanation of what they do.

Disable touchpad while typing

The placement and sensitivity of the MacBook touchpad means that it is prone to moving your cursor while you're typing which is very annoying. We can fix this:

To disable the touchpad while typing use this command:

syndaemon -i 1 -d

This disables the touchpad for 1 second after you type something. This works well but if you go back and forth a lot it can be a hassle to wait that one second. NOTE you can use ".5" for half a seconf instead and it works much better.

Alternatively,

syndaemon -t -d

This disables tapping motions but not movement while typing.

Both work fine, and the one that works best for you will likely be determined by the type of computing you do. Try them both and decide for yourself.

Once you've found the one you want to use, add it as an application to start in your gnome session.

iSight camera

First of all, you will need a copy of the firmware from Apple. You can either download this here, or copy it from a Mac OS X partition. While the latter is usually preferred I can't guarantee that the firmware distributed with your MacBook will be supported, so sometimes its easier just to download it.

To acquire the firmware, save the file AppleUSBVideoSupport to your home directory. Next, we need to convert the AppleUSBVideoSupport to a useable firmware file. The isight-firmware-tools will do this for us so we need to install that package:

su -c "yum install isight-firmware-tools"

Now, finally, we can convert the AppleUSBVideoSupport to a useable firmware file. From your home directory where you downloaded the AppleUSBVideoSupport file:

su -c ift-extract --apple-driver AppleUSBVideoSupport

Reboot and your webcam should be fully functionally. Test it with Cheese, Ekiga, or Skype2.

Bluetooth

Bluetooth works out of the box but due to some kernel bug it needs to be enabled properly first.

Edit /etc/sysconfig/bluetooth and make sure it says 'HID2HCI_ENABLE=true'.

That'll fix it on subsequent reboots. For now, just run '/sbin/hid2hci' as root.

Apple SMC (Optional)

This is an optional but recommended step. The Apple SMC (System Management Controller) provides an interface to devices such as fans, temperature sensors, motion sensors, leds and controls these.

The applesmc kernel module is not loaded by default - I guess this is a bug. The correct way to load a module on boot in Fedora 10 is to create a .modules file in /etc/sysconfig/modules/ - here is how to do it for applesmc:

su -
gedit /etc/sysconfig/modules/applesmc.modules

Paste the following into the file:

#!/bin/sh

if [ ! -d /sys/devices/platform/applesmc.768 ] ; then
        exec /sbin/modprobe applesmc >/dev/null 2>&1
fi

Save and exit, then change the file permissions as follows:

chmod 755 /etc/sysconfig/modules/applesmc.modules

On next boot the applesmc module will load. Meanwhile, just do "modprobe applesmc" as root.

Plymouth graphical boot (Optional)

To activate the plymouth graphical boot you must edit /boot/grub/grub.conf and add your framebuffer resolution at the end of the kernel line. For the MacBook with Intel graphics the resolution that works is "0x318". So, become root and edit the kernel line to look something like this:

title Fedora (2.6.27.5-117.fc10.x86_64)
	root (hd0,2)
	kernel /vmlinuz-2.6.27.5-117.fc10.x86_64 ro root=UUID=afb18b52-bfdf-49aa-95f0-234d3013f87d rhgb quiet vga=0x318
	initrd /initrd-2.6.27.5-117.fc10.x86_64.img

By default only the solar and text themes are installed. To add additional plymouth themes type:

su -
yum install plymouth-plugin-{fade-in,pulser,spinfinity}

Now you can try any theme you want like shown below, just replace pluginname with one of the following: solar, fade-in, pulser, spinfinity, text:

su -
plymouth-set-default-plugin pluginname
/usr/libexec/plymouth/plymouth-update-initrd

After that just reboot your computer to enjoy a lovely graphical boot.

What next?

Now your MacBook hardware should be fully functional so it is time to get busy adding the software you want and tweaking the system to your needs.

This guide by Mauriat Miranda is a great place to look for details of how to install common media players, codecs and fonts.

For the native 64bit flash plugin see this post.

Have fun!

Some useful linux commands

Chris — Mon, 30 Jun 2008 12:46:32 +0000

A few useful linux commands and their explanations.

Recursively find in files...

find . -name \*.php -exec grep -inH discount {} \;

This finds all files ending with .php (-name \*.php) starting in the current path (.) and recurse through all subdirectories, then search (grep) case-insensitive (i) within those files ({}) for the word "discount" and outputs the filename (H), line number (n), and the line for any matches.

Find all files older than 14 days...

find /home/*/Maildir/.Junk/cur -type f -mtime +14 -exec ls {} \;
find /home/*/Maildir/.Trash/cur -type f -mtime +14 -exec ls {} \;

These two commands find files within system users' Maildir .Junk and .Trash folders that are older than 14 days. You could change the "-exec ls" to "-exec rm -rf" to remove the files.

Find top 10 directories using disk space...

du -x --block-size=1024K | sort -nr | head -10

This finds and lists the top 10 directories using disk space in the current path. Useful if you're trying to track down what is filling up your disk.

Create and extracting a tar.gz archive of a directory...

tar -czvf (output filename) (directory to archive)

e.g. tar -czvf myhome.tar.gz /home/myhome/

To extract this archive while preserving permissions, use:

tar --preserve -zxvf (filename)

Change permissions of directories and files...

find . -type d -exec chmod 755 {} \;

This finds all directories (indicated by the -type d) in the current path (.) and then chmods them to 755. If you wanted to recursively chmod files, you could issue this:

find . -type f -exec chmod 644 {} \;

This is really for my own reference, but you may find it useful.

Mounting an FTP filesystem under Yellow Dog Linux 6

Chris — Tue, 13 May 2008 15:18:25 +0000

This guide shows how to use FUSE and CurlFtpFs to mount an FTP filesystem on Yellow Dog Linux 6. This guide presumes that you have already built and installed your own FUSE capable kernel for YDL 6.

Unless otherwise specified, all commands are run as a normal user, NOT root!

1) Install the FUSE userspace tools

wget http://downloads.sourceforge.net/fuse/fuse-2.7.3.tar.gz?modtime=1203456610&big_mirror=0
tar xvzf fuse-2.7.3.tar.gz
cd fuse-2.7.3
./configure
make
su -c "make install"

This should all go smoothly and you should now have the FUSE tools such as "fusermount" installed.

2) Install CurlFtpFs

wget http://downloads.sourceforge.net/curlftpfs/curlftpfs-0.9.1.tar.gz?modtime=1175788159&big_mirror=0
tar zxvf curlftpfs-0.9.1.tar.gz
cd curlftpfs-0.9.1
jm_cv_func_working_malloc=yes
export jm_cv_func_working_malloc
ac_cv_func_malloc_0_nonnull=yes
export ac_cv_func_malloc_0_nonnull
jm_cv_func_working_realloc=yes
export jm_cv_func_working_realloc
ac_cv_func_realloc_0_nonnull=yes
export ac_cv_func_realloc_0_nonnull
./configure
make
su -c "make install"

Again, this should all go smoothly and you should now have the "curlftpfs" program installed.

3) Mounting the FTP server

Mounting procedure is as follows:

mkdir MountPoint (eg: mkdir /backup)
curlftpfs -o user=USERNAME:PASSWORD ftp://FTPHost/ MountPoint/
eg: curlftpfs -o user=testusr:mypass ftp://127.0.0.1/ /backup/

Building a custom kernel RPM with FUSE support on YDL 6

Chris — Tue, 13 May 2008 13:39:29 +0000

This guide explains how to build and install a custom kernel RPM with FUSE support on Yellow Dog Linux 6. The guide is written based on my experience with YDL on an Apple XServe G4 using kernel-2.6.23-9 on ppc architecture. If your system is different then adjust the guide as required.

Follow all the commands as a normal user, NOT root!

1) Create an rpm build environment

cd ~
mkdir ~/rpmbuild
mkdir ~/rpmbuild/BUILD
mkdir ~/rpmbuild/RPMS
mkdir ~/rpmbuild/SOURCES
mkdir ~/rpmbuild/SPECS
mkdir ~/rpmbuild/SRPMS

echo "%_topdir /home/`whoami`/rpmbuild" > ~/.rpmmacros

2) Download and install the source rpm of the current kernel.

Note: this is not the kernel-source rpm!

wget http://www.mirrorservice.org/sites/ftp.yellowdoglinux.com/pub/yellowdog/releases/yellowdog-6.0/SRPMS/kernel-2.6.23-9.ydl6.1.src.rpm
rpm -ivh kernel-2.6.23-9.ydl6.1.src.rpm

3) Prepare the kernel source tree

cd ~/rpmbuild/SPECS
rpmbuild -bp --target=`uname -m` kernel.spec

4) Configure the kernel

cd ~/rpmbuild/BUILD/kernel-2.6.23/linux-2.6.23.9/
cp configs/kernel-2.6.23-ppc.config .config
make oldconfig
make menuconfig

Using the down arrow key navigate to "File systems" then press Enter and then scroll down to "Filesystem in Userspace support" and press "M" to select it as a module. Press Tab to select Exit in the bottom bar, then do it again, then press Y when prompted to save the configuration.

Copy the config file to ~/rpmbuild/SOURCES/:

cp .config ~/rpmbuild/SOURCES/kernel-2.6.23-ppc.config

5) Prepare the build files

cd ~/rpmbuild/SPECS

Open up the kernel.spec file for editing and ensure the kernel has a unique name by changing the following line (line 32)

%define release %{subrev}.ydl6.%{rpmrev}

%define release %{subrev}.ydl6fuse.%{rpmrev}

6) Build the new kernel

rpmbuild -bb --with baseonly --without debuginfo --target=`uname -m` kernel.spec

The build process takes a long time to complete. A lot of messages will be printed to the screen. These messages can be ignored, unless the build ends with an error. If the build completes successfully, the new kernel packages in the ~/rpmbuild/RPMS directory.

7) Install the new kernel

cd ~/rpmbuild/RPMS/ppc
su -c "rpm -ivh kernel-2.6.23-9.ydl6fuse.1.ppc.rpm kernel-headers-2.6.23-9.ydl6fuse.1.ppc.rpm --oldpackage"

8) Make the new kernel boot by default

As root, open up /boot/etc/yaboot.conf for editing. You should now see two kernel images listed as boot options. Change the label of the ydl6fuse image to something friendly like "linuxfuse". Now change the "default" line to the same.

For example:

default=linuxfuse
image=/vmlinux-2.6.23-9.ydl6fuse.1
label=linuxfuse
read-only
initrd=/initrd-2.6.23-9.ydl6fuse.1.img
append="root=LABEL=/"

image=/vmlinux-2.6.23-9.ydl6.1
label=linux
read-only
initrd=/initrd-2.6.23-9.ydl6.1.img
append="root=LABEL=/"

Important: Ensure you also have a line that says "delay=5" (or a greater number) so if your new kernel won't boot you can still access the yaboot menu to boot your old kernel!

Save and exit the file.

Now run "ybin" (again, as root) to commit the changes. Reboot!

9) Load the fuse module

su -
modprobe fuse

To make the fuse module load on boot. Create the file /etc/sysconfig/modules/my.modules and give it the following contents:

#!/bin/sh
modprobe fuse >/dev/null 2>&1

Save and exit. Now make it executable:

chmod +x /etc/sysconfig/modules/my.modules

10) Finished!

Your kernel now supports FUSE. Now you can use any of the various userspace tools to mount the various filesystems supported by FUSE.

Simple Linux to Amazon S3 backup script

Chris — Mon, 05 May 2008 13:05:53 +0000

There are many tools available to help backup Linux systems to Amazon S3 but finding the right one to use can be difficult. Jeremy Zawodny made a good list of various S3 backup tools which is very helpful, if a little outdated. I experimented with a few tools, including some of the standard scripts published by Amazon but I found each had their own shortcomings.

One tool that is very simple to use is s3cmd which is a linux command line tool to upload, retrieve, and manage data in Amazon S3. The tool is written in python so should install and run on pretty much any modern linux distro and I have found it works very nicely and seems to be an ideal tool to use if you want to write a basic backup script.

Once installed you just run "s3cmd --configure" to configure the script with your Amazon S3 access keys. You can then use s3cmd from the command line to manage your S3 buckets and their contents. Full details of the various options can be obtained through running "s3cmd --help".

I have used s3cmd to create automated daily rotated backups of a single directory (e.g. /home) using a simple bash script which I adapted from the automysqlbackup script. The script I have used is shown below, and provides rotated backups of the previous 7 days, each day individually archived in a single tar.gz file. This is a fairly limited solution as it only provides a backup of a single directory, does not do incremental backups, and it will break if your tar.gz file exceeds the S3 individual file storage limit of ~6GB. However, if you just need a simple solution to maintain daily backups of a single folder then its a nice tidy option. It will also email you the results, and can optionally include jobs to run pre- and post- backup.

#!/bin/bash
#
# Backup Script
#
#=====================================================================
#=====================================================================
# Set the following variables to your system needs
#=====================================================================

# Directory to save daily tar.gz backup files to e.g /backups
BACKUPDIR="/backups"

# Directory to backup
BACKUPSRCDIR="/home"

# Maximum size of backup files in MB (larger files will be split into separate archives)
# Note: not implemented yet
MAXTARSIZE="1000"

# S3 Settings
# The name of the S3 bucket to upload to e.g. "my_s3_bucket"
S3BUCKET="my_s3_bucket"

# Mail setup
# What would you like to be mailed to you?
# - log   : send only log file
# - files : send log file and tar.gz files as attachments
# - stdout : will simply output the log to the screen if run manually.
# - quiet : Only send logs if an error occurs to the MAILADDR.
MAILCONTENT="log"

# Set the maximum allowed email size in k. (4000 = approx 5MB email [see docs])
MAXATTSIZE="4000"

# Email Address to send mail to? (user@domain.com)
MAILADDR="user@domain.com"

# Command to run before backups (uncomment to use)
#PREBACKUP="/etc/backup-pre"

# Command run after backups (uncomment to use)
#POSTBACKUP="/etc/backup-post"

#=====================================================================
#=====================================================================
#=====================================================================
#
# Should not need to be modified from here down!!
#
#=====================================================================
#=====================================================================
#=====================================================================
PATH=/usr/local/bin:/usr/bin:/bin:
DATE=`date +%Y-%m-%d_%Hh%Mm`                            # Datestamp e.g 2002-09-21
DOW=`date +%A`                                          # Day of the week e.g. Monday
DNOW=`date +%u`                                         # Day number of the week 1 to 7 where 1 represents Monday
DOM=`date +%d`                                          # Date of the Month e.g. 27
M=`date +%B`                                            # Month e.g January
W=`date +%V`                                            # Week Number e.g 37
VER=0.1                                                 # Version Number
HOST=`hostname`                                         # Hostname for LOG information
LOGFILE=$BACKUPDIR/$HOST-`date +%N`.log                 # Logfile Name
LOGERR=$BACKUPDIR/ERRORS_$HOST-`date +%N`.log           # Error log Name
BACKUPFILES=""

# Create required directories
if [ ! -e "$BACKUPDIR" ]                # Check Backup Directory exists.
then
mkdir -p "$BACKUPDIR"
fi

# IO redirection for logging.
touch $LOGFILE
exec 6>&1           # Link file descriptor #6 with stdout.
# Saves stdout.
exec > $LOGFILE     # stdout replaced with file $LOGFILE.
touch $LOGERR
exec 7>&2           # Link file descriptor #7 with stderr.
# Saves stderr.
exec 2> $LOGERR     # stderr replaced with file $LOGERR.

# Functions

# Backup function: removes last weeks archive from S3, creates new tar.gz and sends to S3
SUFFIX=""
dobackup () {
s3cmd ls s3://"$S3BUCKET" | grep s3 | sed "s/.*s3:\/\/$S3BUCKET\//s3:\/\/$S3BUCKET\//" | grep "$DOW" | xargs s3cmd del
tar cfz "$1" "$2"
echo
echo Backup Information for "$1"
gzip -l "$1"
echo
s3cmd put "$1" s3://"$S3BUCKET"
return 0
}

# Run command before we begin
if [ "$PREBACKUP" ]
then
echo ======================================================================
echo "Prebackup command output."
echo
eval $PREBACKUP
echo
echo ======================================================================
echo
fi

echo ======================================================================
echo BackupScript VER $VER
echo http://www.cenolan.com/
echo
echo Backup of Server - $HOST
echo ======================================================================

echo Backup Start Time: `date`
echo ======================================================================
# Daily Backup
echo Daily Backup of Directory \( $BACKUPSRCDIR \)
echo
echo Rotating last weeks Backup...
eval rm -fv "$BACKUPDIR/*.$DOW.tar.gz"
echo
dobackup "$BACKUPDIR/$DATE.$DOW.tar.gz" "$BACKUPSRCDIR"
BACKUPFILES="$BACKUPFILES $BACKUPDIR/$DATE.$DOW.tar.gz"
echo
echo ----------------------------------------------------------------------
echo Backup End Time: `date`
echo ======================================================================
echo Total disk space used for backup storage..
echo Size - Location
echo `du -hs "$BACKUPDIR"`
echo
echo ======================================================================
echo ======================================================================

# Run command when we're done
if [ "$POSTBACKUP" ]
then
echo ======================================================================
echo "Postbackup command output."
echo
eval $POSTBACKUP
echo
echo ======================================================================
fi

#Clean up IO redirection
exec 1>&6 6>&-      # Restore stdout and close file descriptor #6.
exec 1>&7 7>&-      # Restore stdout and close file descriptor #7.

if [ "$MAILCONTENT" = "files" ]
then
if [ -s "$LOGERR" ]
then
# Include error log if is larger than zero.
BACKUPFILES="$BACKUPFILES $LOGERR"
ERRORNOTE="WARNING: Error Reported - "
fi
#Get backup size
ATTSIZE=`du -c $BACKUPFILES | grep "[[:digit:][:space:]]total$" |sed s/\s*total//`
if [ $MAXATTSIZE -ge $ATTSIZE ]
then
BACKUPFILES=`echo "$BACKUPFILES" | sed -e "s# # -a #g"` #enable multiple attachments
mutt -s "$ERRORNOTE MySQL Backup Log and SQL Files for $HOST - $DATE" $BACKUPFILES $MAILADDR < $LOGFILE       #send via mutt
else
cat "$LOGFILE" | mail -s "WARNING! - Backup exceeds set maximum attachment size on $HOST - $DATE" $MAILADDR
fi
elif [ "$MAILCONTENT" = "log" ]
then
cat "$LOGFILE" | mail -s "Backup Log for $HOST - $DATE" $MAILADDR
if [ -s "$LOGERR" ]
then
cat "$LOGERR" | mail -s "ERRORS REPORTED: MySQL Backup error Log for $HOST - $DATE" $MAILADDR
fi
elif [ "$MAILCONTENT" = "quiet" ]
then
if [ -s "$LOGERR" ]
then
cat "$LOGERR" | mail -s "ERRORS REPORTED: Backup error Log for $HOST - $DATE" $MAILADDR
cat "$LOGFILE" | mail -s "Backup Log for $HOST - $DATE" $MAILADDR
fi
else
if [ -s "$LOGERR" ]
then
cat "$LOGFILE"
echo
echo "###### WARNING ######"
echo "Errors reported during Backup execution.. Backup failed"
echo "Error log below.."
cat "$LOGERR"
else
cat "$LOGFILE"
fi
fi

if [ -s "$LOGERR" ]
then
STATUS=1
else
STATUS=0
fi

# Clean up Logfile
eval rm -f "$LOGFILE"
eval rm -f "$LOGERR"

exit $STATUS

I have posted this largely for my own information but maybe somebody else will find this useful.

Converting a MySQL database to UTF-8

Chris — Wed, 16 Apr 2008 12:57:44 +0000

I occasionally need to convert a mysql database to UTF-8. This can be a painful process and if it goes wrong can result in a nightmare of character-set collisions. Some digging around on the lazyweb led me to a nice solution I found over at oscarm.org that shows you how to quickly convert the actual data, but I wanted to convert the database too.

Here's what I did:

1) Dump the database into a sql file using mysqldump:

mysqldump --opt --quote-names -u username -p password my_database > my_database.sql

2) Convert the dumped sql data from the current character set to utf-8, for example:

iconv -f ISO_8859-1 -t UTF-8 my_database.sql > my_database-utf-8.sql

3) Change the CHARSET definitions in the sql file to utf8. You could do this by manually editing the file, but this little perl snippet will do it for you (here we're going from latin1 to utf8 - change to fit your needs):

perl -pi -w -e 's/CHARSET=latin1/CHARSET=utf8/g;' my_database-utf-8.sql

4) Import the database back into mysql:

mysql -u username -p password my_database < my_database-utf-8.sql

Voila. Your database and data is now converted to utf-8.

Fedora kernel fixes and files for MacBook 3,1 (Santa Rosa)

Chris — Tue, 08 Apr 2008 18:03:00 +0000

Following on from my previous post, I have been making some RPM packages available for Fedora 8 and the MacBook 3,1 Santa Rosa.

The kernel packages are not needed now since Fedora 8 kernel 2.6.24.3-50 (and newer) already contains the MacBook specific fixes. However, at the time of writing the gstreamer packages are still required if you want to use gstreamer based applications with the MacBook iSight camera.

Before downloading, please take care of my bandwidth. If you don't need the package, please don't download it.

You can download the packages here.

There is also this thread at fedoraforum.org which may be helpful.

If you find any problems or have any suggestions please let me know.

Installing Fedora 8 on a MacBook

Chris — Tue, 08 Apr 2008 16:16:18 +0000

The MacBook is great, but OS X is not really my cup of tea and I choose not to use Windows. Luckily Fedora 8 works like a charm on the MacBook and with a bit of configuring you can get all the hardware working properly.

I couldn't find any information on the lazyweb about installing Fedora 8 on a MacBook so I recorded what I did and made a detailed how-to which is posted in the wiki over at mactel-linux.org. The guide shows you step-by-step how to install and configure Fedora 8 x86_64 on the MacBook and works with both MacBook version 3,1 (from late 2007) or version 4,1 (from early 2008).

Big up to the guys at Fedora who gave in to my relentless nagging and integrated some of the mactel-linux patches into the latest Fedora kernels. Without them I'd still be spending my weekends rolling kernels!

If you find the guide useful or have any comments or suggestions then let me know.