If you cannot see the embedded spot, click on this link.

The spot runs for three days, showing Ubuntu to a remarkable amount of passengers.

Please note: The background image is from Marvin Kubiak, you can find it among other interesting Jaunty background images on:

https://wiki.ubuntu.com/Artwork/Incoming/Jaunty/AlphaBackgrounds

Happy release!

So there are methods like stupid .zip encryption or a bunch of packages in the repositories like “bcrypt” that provide you with their implementations. But there is a tool, you already know and maybe even use, but don’t think of when considering symmetric encryption: “gpg“. Actually gpg heavily relies on symmetric algorithms as you might know. The public/private key encryption is a combination of asymmetric and symmetric encryption as the latter is quite more cpu efficient. In our case, gpg will use the strong cast5 cipher by default.

Encrypting

So as gpg already knows about a bunch of symmetric encryption algorithms, why not use them? Let’s just see an example. You have a file named “secretfile1.txt” and want to encrypt it:

$ gpg --symmetric secretfile1.txt

You will be prompted for a password. Afterwards you’ll have a file named secretfile1.txt.gpg. You are already done! Please note: The file size of the encrypted file might have decreased as gpg also compresses during encryption and outputs a binary. In my test case the file size went down from 700k to 100k. Nice.

Armoring

In case you need to have an easy portable file that is even ready to be copy-pasted, you can ask gpg to create an ascii armor container:

$ gpg --armor --symmetric secretfile1.txt

The output file will be called “secretfile1.txt.asc”. Don’t bother to open it in a text editor of your choice. The beginning will look similar to this:

$ head secretfile1.txt.asc
-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.6 (GNU/Linux)

jA0EAwMChpQrAA/o8IFgye1j3ErZPvXumcnIwbzSvENDD/fYlWMRiY/qqvn949kV
+mo/v+nQi7OFrrA45scQPuPbj8I1T+2f7XAT4ouW2kuHIJ/2zkyrxBMvO04fDH82
273NwUrXd/s+JJXe+wJz149K324rE7+FIHvfImiez8lRs5qyRI/drp/wFK8ZHRvF
gzhDGaTe8Dgj1YqHgWAY4eAjrXhYLI1imbIYrV1OVPia6Roif37FV7C1AT/i/2HX
2ytI2mBhQLdqkSVeqXZ74lgZhsitnOeqZH65IuTLi77PUcroFOuefw6+4qSpMIuM
8dyi4jCqQ1jCR7PRorpGvm3RtXhlkB689vrknKmOa5uztTj3MGrPOgC6jegBpu/L
3419sAxRtw8bj2lP76B+XXPZ2Tuzpg01hC/BWlifSexy+juYXv7iuF5BuQ1z7nTi

(In this case I used ‘head” for displaying the first ten lines. Head is similar to tail, which you might already know.) Though the ascii file is larger than the binary .gpg file it is still much smaller than the original text file (about 200k in the above case). When tampering with binary files like already compressed tarballs the file size of the encrypted file might slightly increase. In my test, the size grew from a 478kB file to a 479kB file when using binary mode. In ascii armor mode the size nearly hit the 650kb mark, which is still pretty acceptable.

Decrypting

Decrypting is as easy. Just call “gpg –decrypt”, for instance:

$ gpg --decrypt secretfile1.txt.gpg
# or
$ gpg --decrypt secretfile1.txt.asc

gpg knows by itself, if it is given an ascii armored or binary file. But nevertheless the output will be written to standard output, so the following line might be much more helpful for you:

$ gpg --output secretfile1.txt --decrypt secretfile1.txt.gpg

Please note, that you need to stick to the order first –output, then –decrypt. Of course you can also use a redirector (“>”).

Piping

So, for the sake of it: The real interesting thing is that you can use gpg symmetric encryption in a chain of programs, controlled by pipes. This enables you to encrypt/decrypt on the fly with shell scripts helping you to write strong backup scripts. Gpg already detects, if your are using it in a pipe. So let’s try it out:

$ tar c directory | gpg --symmetric --passphrase secretmantra \
| ssh hostname "cat > backup.tar.gpg"

We just made a tarbal, encrypted it and sent it over ssh without creating temporary files. Nice, isn’t it? To be honest, piping over ssh is not a big deal anymore. But piping to ftp? Check this:

$  tar c directory | gpg --symmetric --passphrase SECRETMANTRA \
| curl --netrc-optional --silent --show-error --upload-file - \
--ftp-create-dirs ftp://USER:PASSWORD@SECRETHOST/SECRETFILE.tar.gpg

With the mighty curl we just piped from tar over gpg directly to a file on a ftp server without any temporary files. You might get a clue of the power of this chain. Now you can start using a dumb ftp server as encrypted backup device now completely transparently.

That’s all for now. If you like encryption, you should also check symmetric encryption and the possibilities of enhancing daily business scripts security by adding some strong crypto to it. Of course you can complain about the security of the password, the possible visibility via “ps aux”, but you should be able to reduce risks by putting some brain in it. In the meantime check “bashup“, the bash backup script, which uses methods described here to provide you with a powerful and scriptable backup library written in Bash with minimum depencies. And yes, gpg will be added soon.

Bashup is written in heavy Bash syntax and has few dependencies to external programs. You should image it as a scripting library for backups as it allows to call different backup methods and is more a framework than a fully integrated solution. The power of this is the ability to be free in the creation of a backup process while still using easy methods.

The following is a fairly easy setup of bashup for backing up some directories over ssh. You see that we only source the bashup library here, setup some variables and call the bashup method then which executes the backup.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#!/bin/bash
# source bashup lob
. bashup_lib.sh
 
# set setup source - we want to backup a tree
SOURCE=tree
# set source dirs
TREE_DIRECTORIES="$HOME/PDF $HOME/ogg"
# setup filter for compression
FILTER=bzip2
 
# set destination - we want to ssh
DESTINATION=ssh_file
# set ssh host, you could be clever
# and setup access in .ssh/config
SSH_HOST=host.name.tld
# set remote file name
FILE=file.name.bz2
 
# we want two types of logging
USE_REPORTERS="console log"
 
# set the log file names
COMBINED_LOG=combined.log
ERROR_LOG=error.log
 
# call bashup
bashup

I admit this example won’t win a Noble award but if you are busy with setting up backups on very different servers you might like the idea of a scripting environment which only needs bash. Imagine your power when digging deeper into the bashup lib and calling the special methods directly while piping output and more. This is possible of course.

In the next weeks I’ll show you how to use backup rotation (yes, also over ssh, ftp and other methods), mysql/postresql/oracle/subversion backup, nagios feedback integration and more.

If you like to test and tell me what you think or even want to provide patches, feel free to checkout bashup:

svn co https://bashup.svn.sourceforge.net/svnroot/bashup bashup