Matt/ links
Atom Add a new post titled:
Dan Simmons

Visiting some friends, they recommended reading Dan Simmons. I'll check him out.

Posted Thu 28 Feb 2008 08:46:48 PM PST
Apple TV Hacking

I've been looking for a Media Center Thing for a long time and haven't found one. Friends are looking at hacking Apple TVs for that purpose, but we'll see. Anyway, sites to watch are :

Posted Mon 18 Feb 2008 09:42:42 PM PST
Backups for the rest of us

'Time Machine' RSync Backups.

Copying the page contents here for a backup.

Time Machine for every Unix out there

Using rsync to mimic the behavior of Apple's Time Machine feature

rsync is one of the tools that have gradually infiltrated my day to day tool-box (aside Vim and Zsh).

Using rsync it’s very easy to mimic Mac OS X new feature called Time Machine. In this article I’ll show how to do it, but there is still a nice GUI missing – for those who like it shiny.

What Time Machine does

Time Machine makes a snapshot of your files every hour. The files are usually stored on a external hard drive connected to your Mac via USB or Firewire. Earlier Leopard versions (ADC preview versions) had the ability to make the backups to a remote drive (I’ve heard).

So if you lose a file, or did a devastating change to one of your files, simply go back in time until you find your file or a version that’s not corrupted.

Incrementally backing up all files every hour so that you can access them in reversed chronological order isn’t that hard with standard Unix utilities like rsync. The only missing thing is a nice GUI for which Apple is known to be quite good at.

Making full backups in no time every hour

You can use this method to make a backup every hour or every ten minutes if you like. There are many many features you can tune or configure to your own taste – excluding files that are larger than 1GB for example.

So, here the command to make the backup:

rsync -aP --link-dest=PATHTO/$PREVIOUSBACKUP $SOURCE $CURRENTBACKUP

Lets go through the parameters step by step.

-a means Archive and includes a bunch of parameters to recurse directories, copy symlinks as symlinks, preserve permissions, preserve modification times, preserve group, preserve owner, and preserve device files. You usually want that option for all your backups.
-P allows rsync to continue interrupted transfers and show a progress status for each file. This isn’t really necessary but I like it.
--link-dest this is a neat way to make full backups of your computers without losing much space. rsync links unchanged files to the previous backup (using hard-links, see below if you don’t know hard-links) and only claims space for changed files. This only works if you have a backup at hand, otherwise you have to make at least one backup beforehand.

PATHTO/$PREVIOUSBACKUP is the path to the previous backup for linking. Note: if you delete this directory, no other backup is harmed because rsync uses hard-links and the operating system (or filesystem) takes care of releasing space if no link points to that region anymore.
$SOURCE is the directory you’d like to backup.
$CURRENTBACKUP is the directory to which you’d like to make the backup. This should be a non-existing directory.

As said earlier, rsync has many many features. To exclude files over a certain size for example, use the option --max-size (unfortunately this is not available on the rsync version shipped with Mac OS X Leopard). The man page or the documentation can give you plenty of ideas in this direction.

So much for the theory of the most important command for our purpose. Here a simple script that makes an incremental backup every time you call it:

#!/bin/sh

date=`date "+%Y-%m-%dT%H:%M:%S"`
rsync -aP --link-dest=$HOME/Backups/current /path/to/important_files $HOME/Backups/back-$date
rm $HOME/Backups/current
ln -s back-$date $HOME/Backups/current

The script creates a file called “back” appended by the current date and time, for example back-2007-11-13T22:03:32 which contains the full backup. Then there is a symbolic link called “current” which points to the most recent directory. This directory-link is used for the --link-dest parameter.

You should look at the --exclude parameter (or better, --exclude-from= parameter) and learn how to exclude certain files or directories from the backup (you shouldn’t backup your backup for example).

The script above only works on the local machine because making links on a remote machine needs some extra work. But not much:

#!/bin/sh

date=`date "+%Y-%m-%dT%H:%M:%S"`
rsync -azP --link-dest=PATHTOBACKUP/current $SOURCE $HOST:PATHTOBACKUP/back-$date
ssh $HOST "rm PATHTOBACKUP/current && ln -s back-$date PATHTOBACKUP/current"

To get that working you either use a public/private key authentication scheme or something else to avoid typing in your password. Another possibility is, of course, to mount the remote file-system on the local computer using the above script.

On my setup the script takes about 6 seconds to synchronize 46968 files and 29GB – this takes 20MB for the file structure (with no actual files to transfer of course). But afterwards, I have a complete backup of my system in a new directory.

On a much bigger setup (1.2 million files and 50GB of data) the backup takes about 30 minutes and takes about 3GB of space (just for links!), so it isn’t exactly free, but very convenient.

The space needed for the backup is determined by the shape of your directory structure. On the larger setup I have lots of Maildirs and a very deep directory structure so it takes much more space than my home-directory backup above. 3GB is quite a lot, but 20MB doesn’t hurt.

Advanced rsync parameters

Additional to the parameters described above, I usually employ a combination of these parameters in my backup:

--delete and --delete-excluded this tells rsync to remove files from my backups either if they are gone on my local machine, or if I decided to exclude them from my backup.
--exclude-from=FILE the file specified here is a simple list of directories of files (one per line) which should not be backed up. My Trash folder oder some .cache folders are candidates for this file.
-P is used to give more information on how far the backup is, and how many files are to be backed up. Additional it could resume an interrupted transfer (which doesn’t apply here because we create a blank backup each time we call the script).
-x this one is important because it prohibits rsync to go beyond the local filesystem. For example if you backup you Linux-root partition, you should not include the /proc directory because rsync will get stuck in it. -x excludes all mounted filesystems from the backup which is probably what you want in most cases.

Posted Mon 18 Feb 2008 09:37:31 PM PST
Speachy Stuff

How to Speak. Supposed to be legendary.

Posted Mon 18 Feb 2008 09:11:23 PM PST
Java Links Posted Tue 05 Feb 2008 08:29:30 PM PST
20080129 Weird links Posted Tue 29 Jan 2008 07:34:17 PM PST
Good site about BASH

Bash cures cancer is a bit of hyperbole, but has some good advice when scripting with BASH.

Posted Mon 28 Jan 2008 10:22:10 AM PST
A better JNI

JNA looks like a better JNI.

Posted Mon 28 Jan 2008 10:14:41 AM PST
BSG Propaganda

If you're a BSG fan like I am, you'll probably be interested in these posters. Quite cool.

Posted Mon 28 Jan 2008 10:10:16 AM PST
JavaSpaces and Jini, or Distributed Computing

I've been skeptical of JavaSpaces and Jini due to past experiences with some of it's inherent assumptions and, because of those assumptions, limitations.

I want to learn about it again and figure I'll start with these links :

Offtopic

Posted Sun 27 Jan 2008 09:08:17 PM PST
Javassist

Javassist appears to be an "easier to use" ASM/BCEL.

Posted Sun 27 Jan 2008 12:44:31 AM PST
Voices that Matter Summary

Voices that Matter Summary, specifically focused on GWT at this one.

Associated Video

Posted Sun 27 Jan 2008 12:16:17 AM PST
Java Collection, Apache Collection, now Google Collections

Since Apache isn't updating commons collections for Java 5, Google has taken on the task:

Posted Sat 26 Jan 2008 11:21:40 PM PST
Resume Links Roundup

Links, primarily about how NOT to write a resume.

Posted Sat 26 Jan 2008 11:19:45 PM PST
5 Whys

5 Whys is described as a methodology to help find the root cause of a problem. Neat!

Posted Sat 26 Jan 2008 11:07:05 PM PST