1) Ubuntu Community booth

The “ubuntu Deutschland e.V.”, Ubuntu and Kubuntu community presented their work at a community hosted booth. A bunch of Ubuntu Berlin members supported the booth, answered hundreds of questions and helped with the proceedings.

2) Talks

Saturday Featured a lot of Ubuntu focussed talks. For the first time “Ubuntu Berlin” and its members hosted a remarkable amount of them, e.g.:

• Anselm Helbig – Ubuntu Berlin Lightning Talks (tmux)
• Benjamin Drung – Ubuntu in 50 minutes, Packaging for Debian and Ubuntu
• Caspar Clemens Mierau – Ubuntu Berlin Lightning Talks (Vimperator), Gnome-Do, Ubuntu in 50 minutes
• Daniel Holbach – Fixing Bugs in Ubuntu (see slides), Ubuntu in 50 minutes, Ubuntu Berlin Lightning Talks (liblaunchpad)
• Marcel Eichner – Ubuntu Berlin Lightning Talks (Franklin)
• Torsten Franz – Supporting Ubuntu

minutes before the “Ubuntu in 50 minutes” talk

3) Interviews for Radio Tux

Several members of Ubuntu Berlin gave interviews to the well known German Podcast “Radio Tux“. Daniel’s talk on bugjamming has been released already. Be sure to check the archive for other releases within the next days.

4) LinuxTag BBQ

Last but not least: The  ”End of LinuxTag BBQ” at c-base, sponsored by Canonical, has been a great success again. Three barbacues and about ten “Grillmeister” (you cannot translate this) provided more than hundred visitors with tasty food. Members of various types of open source communities had interesting chats while relaxing at the banks of the Spree.

As Ubuntu Berlin’s support for the LinuxTag continuously increased within the last years, I am sure next year will even better. We’ll see.

At Ubucon 2008 in Göttingen, a German Ubuntu User Conference, I noticed that a lot people hanging around discussed how nicely their parents and especially moms are using Ubuntu. No hazzle, no further explanations needed – it just worked. And often they don’t even know or notice that they are using Ubuntu, as they just standard software like Firefox and Thunderbird. As this user group doesn’t belong to the tweeting, facebooking, social web society, I decided to found a launchpad group named “My Mum Runs Ubuntu“, that has no further meaning than joining it means your mother runs Ubuntu – a simple way of giving this “silent” user group at least a number and a marker on a map.

I was surprised how fast the member list grew and happy to see that it’s members came and come from all over the world as you can see below:

"My Mom Runs Ubuntu" global map (map by Google)

So this is not about a single heroine in technology – it is about a general movement:  I am convinced, especially Ubuntu with it’s focus on an intuitive interface seems to keep the entry level very low and therefore attracts user groups that might be a suprprise for a lot of people. I know dozens of techie people stating that free operating systems are way too complicated to use for them. When telling about “My Mom Runs Ubuntu” they run out of reasons. At least there is nothing more convincing on using free software than people that are just using it on a daily basis without the need of telling everybody as they just take it as normal. I am sure, this user group continues to grow and am

So, if you already have a Launchpad account and your mum runs Ubuntu, too, give her a voice by just joining the group.

And if you think, this post misses real techie heroines, check the “Ubuntu Women” project, featuring some of the most active members of the Ubuntu community.

Ada Lovelace Day Logo

While caring for a remarkable amount of MySQL server instances, troubleshooting becomes a common task. It might of interest for you which

Recovering a crashed MySQL server

After a server crash (meaning the system itself or just the MySQL daemon) corrupted table files are quite common. You’ll see this when checking the /var/log/syslog, as the MySQL daemon checks tables during its startup.

Apr 17 13:54:44 live1 mysqld[2613]: 090417 13:54:44 [ERROR]
  /usr/sbin/mysqld: Table './database1/table1' is marked as
  crashed and should be repaired

The MySQL daemon just told you that it found a broken MyISAM table. Now it’s up to you fixing it. You might already know, that there is the “REPAIR” statement. So a lot of people enter their PhpMyAdmin afterwards, select database and table(s) and run the REPAIR statements. The problem with this is that in most cases your system is already in production – for instance is up again and the MySQL server already serves a bunch of requests. Therefore a REPAIR request gets slowed down dramatically. Consider taking your website down for the REPAIR – it will be faster and it’s definitely smarter not to deliver web pages based on corrupted tables.

The other disadvantage of the above method is, that you probably just shut down your web server and your PhpMyAdmin is down either or you have dozens of databases and tables and therefore it’s just a hard task to cycle through them. The better choice is the command line in this case.

If you only have a small number of corrupted tables, you can use the “mysql” client utility doing something like:

$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 10
Server version: 5.0.75-0ubuntu10 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> REPAIR TABLE database1.table1;
+--------------------+--------+----------+----------+
| Table              | Op     | Msg_type | Msg_text |
+--------------------+--------+----------+----------+
| database1.table1   | repair | status   | OK       |
+--------------------+--------+----------+----------+
1 row in set (2.10 sec)

This works, but there is a better way: First, using OPTIMIZE in combination with REPAIR is suggested and there is a command line tool only for REPAIR jobs. Consider this call:

$ mysqlcheck -u root -p --auto-repair --check --optimize database1
Enter password:
database1.table1      OK
database1.table2      Table is already up to date

As you see, MySQL just checked the whole database and tried to repair and optimize it.

The great deal about using “mysqlcheck” is, that it can also be run against all databases in one run without the need of getting a list of them in advance:

$ mysqlcheck -u root -p --auto-repair --check --optimize \
  --all-databases

Of course you need to consider if an optimize of all your databases and tables might just take too long if you have huge tables. On the other hand a complete run prevents of thinking about a probably missed table.

[update]

nobse pointed out in the comments, that it’s worth having a look at the automatic MyIsam repair options in MySQL. So have a look at them if you want to automate recovery:

option_mysqld_myisam-recover

Recovering a broken replication

MySQL replication is an easy method of load balancing database queries to multiple servers or just continuously backing up data. Though it is not hard to setup, troubleshooting it might be a hard task. A common reason for a broken replication is a server crash – the replication partner notices that there are broken queries – or even worse: the MySQL slave just guesses there is an error though there is none. I just ran into the latter one as a developer executed a “DROP VIEW” on a non-existing VIEW on the master. The master justs returns an error and ignores. But as this query got replicated to the MySQL SLAVE, the slave thinks it cannot enroll a query and immediately stopped replication. This is just an example of a possible error (and a hint on using “IF EXISTS” as often as possible).

Actually all you want to do now, is telling the slave to ignore just one query. All you need to do for this is stopping the slave, telling it to skip one query and starting the slave again:

$ mysql -u root -p
mysql> STOP SLAVE;
mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
mysql> START SLAVE;

That’s all about this.

Recreating databases and tables the right way

In the next topic you’ll recreate databases. A common mistake when dropping and recreating tables and databases is forgetting about all the settings it had – especially charsets which can run you into trouble later on (“Why do all these umlauts show up scrambled?”). The best way of recreating tables and databases or creating them on other systems therefore is using the “SHOW CREATE” statement. You can use “SHOW CREATE DATABASE database1″ or “SHOW CREATE TABLE database1.table1″ providing you with a CREATE statement with all current settings applied.

mysql> show create database database1;
+-----------+--------------------------------------------------------------------+
| Database  | Create Database                                                    |
+-----------+--------------------------------------------------------------------+
| database1 | CREATE DATABASE `database1` /*!40100 DEFAULT CHARACTER SET utf8 */ |
+-----------+--------------------------------------------------------------------+
1 row in set (0.00 sec)

The important part in this case is the “comment” after the actual create statement. It is executed only on compatible MySQL server versions and makes sure, your are running utf8 on the database.

Keep this in mind and it might save you a lot of trouble.

Fixing replication when master binlog is broken

When your MySQL master crashes there is a slight chance that your master binlog gets corrupted. This means that the slaves won’t receive updates anymore stating:

[ERROR] Slave: Could not parse relay log event entry. The possible reasons are: the master’s binary log is corrupted (you can check this by running ‘mysqlbinlog’ on the binary log), the slave’s relay log is corrupted (you can check this by running ‘mysqlbinlog’ on the relay log), a network problem, or a bug in the master’s or slave’s MySQL code. If you want to check the master’s binary log or slave’s relay log, you will be able to know their names by issuing ‘SHOW SLAVE STATUS’ on this slave. Error_code: 0

You might have luck when only the slave’s relay log is corrupted as you can fix this with the steps mentioned above. But a corrupted binlog on the master might not be fixable though the databases itself can be fixed. Depending on your time you try to use the “SQL_SLAVE_SKIP_COUNTER” from above but actually the only way is to setup

Setting up replication from scratch

There are circumstances forcing you to start replication from scratch. For instance you have a server going live for the first time and actually all those test imports don’t need to be replicated to the slave anymore as this might last hours. My quick note for this (consider backing up your master database before!)

slave: STOP SLAVE;
slave: RESET SLAVE;
slave: SHOW CREATE DATABASE datenbank;
slave: DROP DATABASE datenbank;
slave: CREATE DATABASE datenbank;

master: DROP DATABASE datenbak;
master: SHOW CREATE DATABASE datenbank;
master: CREATE DATABASE datenbank;
master: RESET MASTER

slave: CHANGE MASTER TO MASTER_USER="slave-user", \
MASTER_PASSWORD="slave-password", MASTER_HOST="master.host";
slave: START SLAVE

You just started replication from scratch, check “SHOW SLAVE STATUS” on the slave and “SHOW MASTER STATUS” on the master.

Deleting unneeded binlog files

Replication needs binlog files – a mysql file format for storing database changes in a binary format. Sometimes it is hard to decide how many of the binlog files you want to keep on the server possibly getting you into disk space trouble. Therefore deleting binlog files that have already been transferred to the client might be a smart idea when running low on space.

First you need to know which binlog files the slave already fetched. You can do this by having a look on “SHOW SLAVE STATUS;” on the slave. Now log into the MySQL master and run something like:

mysql> PURGE BINARY LOGS TO 'mysql-bin.010';

You can even do this on a date format level:

mysql> PURGE BINARY LOGS BEFORE '2008-04-02 22:46:26';

Conclusion

The above hints might save you same time when recovering or troubleshooting a MySQL server. Please note, that these are hints and you have – at any time – make sure, that your data has an up to date backup. Nothing will help you more.

By chance the Ubuntu Developer Sprint happened to be in Berlin for the next Jaunty release this week. If I got it right, the Canonical Ubuntu developers meet five days around two weeks before a release feature freeze and work in groups and issues that need to be decided/designed or just fixed immediatly. The incredible Daniel Holbach had the idea of inviting the bunch of developers right into the c-base after their work. So he did and we scheduled it for an evening when the Ubuntu Berlin crew also meets at c-base for their monthly jour fix.

We did not announce this meeting externally as we tried to make the whole evening as comfortable as possible for everyone. And we did, I think. Right in time at 19:30 this Wednesday evening about thirty Ubuntu developers entered the c-base. Just among them Mark who seemed to like the whole c-base hackerspace, Ubuntu Berlin, community, space and future thing a lot. The Canonical crowd got several guided tours through the whole base by __t, while housetier provided the others with “German beer” and Club Mate. We had a lot of chats in smaller groups, things, you always wanted to ask the developer of your choice and just relaxed smalltalks about space, canooeing the c-base project “OpenMoon” (trying to send a rocket to the moon), and more. Mark seemed to be amazed about asking people why they joined to Ubuntu Berlin team, what they were currently doing and so on – so, what the community thing is about right here and right now.

We had no schedule for the evening. Therefore we spent about two and a half hour at c-base without any official part and a cosy diner in smaller groups afterwards. We only asked some people for an interview for the c-base statement studio channel where already people like Nokias Peter Schneider and Mozillas CEO John Lilly showed up. Mark took the time for an interview by jocognito. The results of this short talks are already online on Youtube:

Interview #1: (when the inbound video doesn’t show up, click here)

Interview #2: (when the inbound video doesn’t show up, click here)

Again, a couple of lectures will be held – ranging from new features in Intrepid (that’s my part, I’ll give the lecture at the same day also at the BLIT), over Gnome eyecandy stuff to presentations of Freifunk project and the DeepaMehta semantic desktop (don’t miss!) directly from it’s lead developer. So either if you are new to Ubuntu and want to make first steps and contact other Ubuntu users or you are a Linux guru – you’ll find somebody to have a chat with, I promise. There’ll even be a “tux tinker corner” where you or your girl friend have the possibility to try out some Tux Origami.

The event’ll mainly be in German, but a lot of people are speaking English, so don’t hesitate asking for help/translations.

Entrance is free, there is a free wifi, so feel free to bring your notebook in. You can also “buy” a freshly burned Ubuntu/Kubuntu/Xubuntu cd for a service charge of 1 Euro or check the Ubuntu merchandising table.

See you there?

Links:
Official Party Announcement Page
how to get to c-base?
Freifunk
DeepaMehta
Ubuntu Berlin

If you want to give this a try, call the ssh client this way:

$ ssh -o VisualHostKey=yes your.host.name
Host key fingerprint is ff:aa:a8:dc:0b:5e:e3:9f:96:f1:75:d4:24
+--[ RSA 1024]----+
|            +o   |
|             o. .|
|            E  + |
|       .   . .. .|
|      . S   ..   |
|   . o o..  . .  |
|    + + .+.. .   |
|   . + ooo.      |
|    . ooo        |
+-----------------+

Nice, isn’t it? Now try your different ssh hosts and compare the figures. Hope you don’t start generating ssh host keys for getting a special figure, do you? Actually I don’t know if I’ll really remember figures of dozens of machines, but hey: it’s just additional fun.

In case you want to make this behavior default, add “VisualHostKey yes” to your “~/.ssh/config”. In case you don’t have this file, make a new one with the following content (and find out that this file makes ssh really poweful in combination with command line completion, but that is another topic):

Host *
	VisualHostKey		yes

Please note: This might break applications that rely on the ssh console client as they don’t expect graphical art popping up. So if some other clients don’t work anymore, play around with aliases or your ~/ssh/config file.

Thank you, OpenSSH guys, I really appreciate your work.

There are, of course, PDF frameworks available. Let’s take PHP as an example: If you want to create a PDF from a php script, you can choose between FPDF, Dompdf, the sophisticated Zend Framework and more (and commercial solutions). But to be honest, they are all either complicated (as you often have to use a specific syntax) to use or just quite limited in their possibilities to create a pdf file (as you can only use few design features). As I needed a simple solution for creating a 50+ pages pdf file with a huge table on the fly I tested most frameworks and failed with most of them (often just as I did not have enough time to write dozens of line of code).

So I hoped to find a solution that allowed me just to convert a simple HTML file to a PDF file on the fly providing better compatibility than Dompdf for instance. The solution was … uncommon. It was no PHP class but a neat command line tool called “htmldoc” available as a package. If you want to give it a try just install it by calling “aptitude install htmldoc”.

You can test htmldoc by saving some html files to disk and call “htmldoc –webpage filename.html”. There a lot of interesting features like setting font size, font type, the footer, color and greyscale mode and so on. But let’s use htmldoc from PHP right away. The following very simple script uses the PHP output buffer for minimizing the need for a write to disk to one file only (if somebody knows a way of using this without any temporary files from a script, let me know):

// start output buffer for pdf capture

ob_start();
?>
your normal html output will be places here either by
dumping html directly or by using normal php code

As you can see, this is neither rocket science nor magic. Just a wrapper for htmldoc enabling you to forget about the pdf when writing the actual content of the html file. You'll have to check how htmldoc handles your html code. You should make it as simple as possible, forget about advanced css or nested tables. But it's actually enough for a really neat pdf file and it's fast: The creating of 50+ page pdf files is fast enough in my case to make the on demand access of htmldoc feel like static file usage.
Please note: Calling external programs and command line tools from a web script is always a security issue and you should carefully check input and updates for the program you are using. The code provided should be easily ported to another web language/framework like Perl and Rails.
]]>
			http://www.screenage.de/blog/2008/07/07/my-package-of-the-day-htmldoc-for-converting-html-to-pdf-on-the-fly/feed/
		11
		
		
		
		http://www.screenage.de/blog/2008/06/23/my-package-of-the-day-htop-as-an-alternative-top/
		http://www.screenage.de/blog/2008/06/23/my-package-of-the-day-htop-as-an-alternative-top/#comments
		Mon, 23 Jun 2008 07:23:47 +0000
		ccm
				
		
		
		
		
		
		
		

		http://www.screenage.de/blog/2008/06/23/my-package-of-the-day-htop-as-an-alternative-top/
		Continue reading →]]>
			“top” is one of those programs, that are used quite often but actually nobody talks about. It just does its job: showing statistics about memory, cache and cpu consumption, listing processes and so on. Actually top provides you some more features like batch mode and the ability to kill processes, but it’s all quite low level – e.g. you have to type the process id (pid) of process you want to kill.

So, though an applications like top makes sense on the console, a more sophisticated one would be great, extending the basic top functionality with enhancements to it’s usage. This tool already exists: It’s the ncurses based “htop” and we’ll have a closer look at it now.
For the beginning: Install “htop” by running “aptitude install htop”, Synaptic or the package manager of your choice. As you can see, htop is quite colorful, which is, of course, a matter of taste. In my opinion, colors make sense, when the they mean something or provide better readability. So let’s check the output in brief:

At the upper left corner you see statistics about the usage of cpu cores (in my case there are two of them, marked “1″ and “2″), memory and swap statistics, while on the right side, you have the common uptime/load stats. The interesting part is the usage of colors in cpu/ram/swap bars. If you are new to htop you have to look the colors up at least once. Therefore just stroke “h” (“F1″ should work, too, but Gnome might get in your way) and you’ll see a nice explanation in the help:

Quite interesting is the distribution between green and red in the cpu stats, as a high kernel load often means something goes wrong (with the hardware i/o for instance). In the memory bar the real used ram is marked green – blue and orance actually could be cleared by the kernel if necessary. (People are often confused that their ram seems to be full, when calling a tool like/htop though they are not running that many programs. It’s important to understand, that the memory is also used for buffering/caching and that this memory can often be used by “real” data later on).
So what’s the next htop feature? Use your mouse, if you like! You can test it by clicking on “Help” on the menu bar at the bottom. Maybe while clicking around a bit you already noticed that you can also click on processes and mark them. What for? Well, htop enables you to kill processes quit easy, as you don’t have to type a process id, write a pattern or something, you just can mark them with a mouse or cursor and either click on “Kill” in the menu or stroke the “F9″ or “k” key. “htop” will let you choose from a list of signals afterwards:

Of course you cannot kill processes that belong to your user when htop does not run as root (i.e. with “sudo”). “htop” marks processes that belong to user it is run by with a brighter process id:

Sadfully this also means, that running htop as root/sudo, marks processes that belong to non-root with a darker grey. But hey, that’s a nice missing feature for patch, isn’t it?
If you like to become an advanced htop user, you can check the “Setup” menu (click it or press the “F2″ or “S” key). You will see a menu for configuring the output of htop, enabling you to switch off and on the display of certain information:

Of course you can also sort the process list (click “Sort” or press “F6″) which give you a list of possible sort parameters:

In spite of this, you can switch to a process tree display and sort it by pressing one of the keys showed below:

So let me give you a last nice gimmick and then end for today: You can try to attach “strace” to a running process by marking the process typing “s”. If you don’t know, what strace is, don’t bother, if you do, you will probably like this feature pretty much.
I hope you got the clue about using htop, which is a really neat, full featured console top replacement that is even worth to be used when running X as it supports mouse usage and brings everything you need while still having a small footprint. If you have alternatives, you like mention, feel free to drop them as a comment.
]]>
			http://www.screenage.de/blog/2008/06/23/my-package-of-the-day-htop-as-an-alternative-top/feed/
		7
		
		
		
		http://www.screenage.de/blog/2008/06/20/my-package-of-the-day-mtr-as-a-powerful-and-default-alternative-to-traceroute/
		http://www.screenage.de/blog/2008/06/20/my-package-of-the-day-mtr-as-a-powerful-and-default-alternative-to-traceroute/#comments
		Fri, 20 Jun 2008 15:03:27 +0000
		ccm
				
		
		
		
		
		
		
		
		
		

		http://www.screenage.de/blog/2008/06/20/my-package-of-the-day-mtr-as-a-powerful-and-default-alternative-to-traceroute/
		Continue reading →]]>
			Know the situation? Something is wrong with the network or you are just curious and want to run a “traceroute”. At least under most Debian based systems your first session will probably look like this:

$ traceroute www.ubuntu.com
command not found: traceroute
Maybe on Ubuntu you will at least be hinted to install “traceroute” or “traceroute-nanog”… To be honest, I really hate this lack of a basic tool and cannot even remember how often I typed “aptitude install traceroute” afterwards (and press thumbs your network is up and running).
But sometimes you just need to dig a bit deeper and this time the surprise was really big as the incredible Mnemonikk told me about an alternative that is installed by default in Ubuntu and nearly no one knows about it: “mtr“, which is an abbreviation for “my traceroute”.
Let’s just check it by calling “mtr www.ubuntu.com” (i slightly changed the output for security reasons):
                 My traceroute  [v0.72]
ccm        (0.0.0.0)          Wed Jun 20 6:51:20 2008
Keys:  Help   Display mode   Restart statistics   Order
of fields      Packets               Pings
 Host        Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 1.2.3.4   0.0%   331    0.3   0.3   0.3   0.5   0.0
 2. 2.3.4.5   0.0%   331   15.6  16.3  14.9  42.6   2.6
 3. 3.4.5.6   0.0%   330   15.0  15.5  14.4  58.5   2.7
 4. 4.5.6.7   0.0%   330   17.5  17.3  15.4  60.5   5.3
 5. 5.6.7.8   0.0%   330   15.7  24.3  15.6 212.3  30.2
 6. ae-32-52 58.8%   330   20.6  22.1  15.9  42.5   4.7
 7. ae-2.ebr 54.1%   330   20.6  25.0  19.0  45.4   4.7
 8. ae-1-100  0.0%   330   21.5  25.4  19.2  41.1   5.1
 9. ae-2.ebr  0.0%   330   27.5  34.0  26.7  73.5   5.2
10. ae-1-100  0.3%   330   28.8  33.6  26.7  72.5   6.0
11. ae-2.ebr  0.0%   330   30.8  32.9  26.7  48.5   5.0
12. ae-26-52  0.0%   330   27.6  34.8  26.9 226.8  26.8
13. 195.50.1  0.3%   330   27.7  28.4  27.2  42.5   1.7
14. gw0-0-gr  0.0%   330   27.9  28.1  27.0  40.5   1.4
15. avocado.  0.0%   330   27.8  28.0  27.2  36.2   1.0
You might notice, that the output is quite well formed (“mtr” uses curses for this). The interesting point is: Instead of running once, mtr continuously updates the output and statistics, providing you with a neat network overview. So you can use it as an enhanced ping showing all steps between you and the target.
For the sake of it: The package installed by default in Ubuntu is actually called “mtr-tiny” as it lacks a graphical user interface. If you prefer a gui you can replace the package with “mtr” by running “aptitude install mtr”. When running “mtr” from the console afterwards you will be prompted with a gtk interface. In case you still want text mode, just append “–curses” as a parameter.
Yes, that was a quick package, but if you keep it in mind, you will save time, you normalle spend for installing “traceroute” and you’ll definitely have better results for network diagnose. Happy mtr’ing!
[update]
sherman noted, that the reason for traceroute not being installed is, that it’s just deprecated and “tracepath” should be used instead. Thank you for the hint, though I’d prefer “mtr” as it’s much more reliable and verbose.
]]>
			http://www.screenage.de/blog/2008/06/20/my-package-of-the-day-mtr-as-a-powerful-and-default-alternative-to-traceroute/feed/
		4
		
		
		
		http://www.screenage.de/blog/2008/06/19/my-package-of-the-day-gpg-for-symmetric-encryption/
		http://www.screenage.de/blog/2008/06/19/my-package-of-the-day-gpg-for-symmetric-encryption/#comments
		Thu, 19 Jun 2008 16:00:42 +0000
		ccm
				
		
		
		
		
		
		
		
		
		

		http://www.screenage.de/blog/2008/06/19/my-package-of-the-day-gpg-for-symmetric-encryption/
		Continue reading →]]>
			Though asymmetric encryption is state of the art today, there are still cases when you probably are in need of a simple symmetric encryption. In my case, I need an easy scriptable interface for encrypting files for backup as transparent as possible. While you can, of course, use asymmetric encryption for this, symmetric methods can save you a lot of time while still being secure enough.

So there are methods like stupid .zip encryption or a bunch of packages in the repositories like “bcrypt” that provide you with their implementations. But there is a tool, you already know and maybe even use, but don’t think of when considering symmetric encryption: “gpg“. Actually gpg heavily relies on symmetric algorithms as you might know. The public/private key encryption is a combination of asymmetric and symmetric encryption as the latter is quite more cpu efficient. In our case, gpg will use the strong cast5 cipher by default.
Encrypting
So as gpg already knows about a bunch of symmetric encryption algorithms, why not use  them? Let’s just see an example. You have a file named “secretfile1.txt” and want to encrypt it:
$ gpg --symmetric secretfile1.txt
You will be prompted for a password. Afterwards you’ll have a file named secretfile1.txt.gpg. You are already done!  Please note: The file size of the encrypted file might have decreased as gpg also compresses during encryption and outputs a binary. In my test case the file size went down from 700k to 100k. Nice.
Armoring
In case you need to have an easy portable file that is even ready to be copy-pasted, you can ask gpg to create an ascii armor container:
$ gpg --armor --symmetric secretfile1.txt
The output file will be called “secretfile1.txt.asc”. Don’t bother to open it in a text editor of your choice. The beginning will look similar to this:
$ head secretfile1.txt.asc
-----BEGIN PGP MESSAGE-----
Version: GnuPG v1.4.6 (GNU/Linux)

jA0EAwMChpQrAA/o8IFgye1j3ErZPvXumcnIwbzSvENDD/fYlWMRiY/qqvn949kV
+mo/v+nQi7OFrrA45scQPuPbj8I1T+2f7XAT4ouW2kuHIJ/2zkyrxBMvO04fDH82
273NwUrXd/s+JJXe+wJz149K324rE7+FIHvfImiez8lRs5qyRI/drp/wFK8ZHRvF
gzhDGaTe8Dgj1YqHgWAY4eAjrXhYLI1imbIYrV1OVPia6Roif37FV7C1AT/i/2HX
2ytI2mBhQLdqkSVeqXZ74lgZhsitnOeqZH65IuTLi77PUcroFOuefw6+4qSpMIuM
8dyi4jCqQ1jCR7PRorpGvm3RtXhlkB689vrknKmOa5uztTj3MGrPOgC6jegBpu/L
3419sAxRtw8bj2lP76B+XXPZ2Tuzpg01hC/BWlifSexy+juYXv7iuF5BuQ1z7nTi
(In this case I used ‘head” for displaying the first ten lines. Head is similar to tail, which you might already know.) Though the ascii file is larger than the binary .gpg file it is still much smaller than the original text file (about 200k in the above case). When tampering with binary files like already compressed tarballs the file size of the encrypted file might slightly increase. In my test, the size grew from a 478kB file to a 479kB file when using binary mode. In ascii armor mode the size nearly hit the 650kb mark, which is still pretty acceptable.
Decrypting
Decrypting is as easy. Just call “gpg –decrypt”, for instance:
$ gpg --decrypt secretfile1.txt.gpg
# or
$ gpg --decrypt secretfile1.txt.asc
gpg knows by itself, if it is given an ascii armored or binary file. But nevertheless the output will be written to standard output, so the following line might be much more helpful for you:
$ gpg --output secretfile1.txt --decrypt secretfile1.txt.gpg
Please note, that you need to stick to the order first –output, then –decrypt. Of course you can also use a redirector (“>”).
Piping
So, for the sake of it: The real interesting thing is that you can use gpg symmetric encryption in a chain of programs, controlled by pipes. This enables you to encrypt/decrypt on the fly with shell scripts helping you to write strong backup scripts. Gpg already detects, if your are using it in a pipe. So let’s try it out:
$ tar c directory | gpg --symmetric --passphrase secretmantra \
| ssh hostname "cat > backup.tar.gpg"
We just made a tarbal, encrypted it and sent it over ssh without creating temporary files. Nice, isn’t it? To be honest, piping over ssh is not a big deal anymore. But piping to ftp? Check this:
$  tar c directory | gpg --symmetric --passphrase SECRETMANTRA \
| curl --netrc-optional --silent --show-error --upload-file - \
--ftp-create-dirs ftp://USER:PASSWORD@SECRETHOST/SECRETFILE.tar.gpg
With the mighty curl we just piped from tar over gpg directly to a file on a ftp server without any temporary files. You might get a clue of the power of this chain. Now you can start using a dumb ftp server as encrypted backup device now completely transparently.
That’s all for now. If you like encryption, you should also check symmetric encryption and the possibilities of enhancing daily business scripts security by adding some strong crypto to it. Of course you can complain about the security of the password, the possible visibility via “ps aux”, but you should be able to reduce risks by putting some brain in it. In the meantime check “bashup“, the bash backup script, which uses methods described here to provide you with a powerful and scriptable backup library written in Bash with minimum depencies. And yes, gpg will be added soon.
]]>
			http://www.screenage.de/blog/2008/06/19/my-package-of-the-day-gpg-for-symmetric-encryption/feed/
		11