Script to copy my home directory

Posted by Patrice Neff Tue, 20 Jan 2009

Since a few months I keep configuration parts of my home directory managed with git. This allows me to easily check out my home directory on my servers and have access to the same environment.

Today I wrote a script to make that slightly easier. I wrote the following script which is stored as ~/bin/init-host.

 #!/bin/bash
 #
 # Copies my git repository to the given host.
 cd ~/
 echo “You’ll be asked for your password on each host.”

for host in $*; do tar -czf – .git | ssh $host “git —version && \ tar -xvzf – —exclude '._*’ && \ git reset —hard HEAD” done

It logs into all hosts given as command line parameters. On each it deploys the home directory using the ~/.git repository. A few specialties:

  • tar on OS X creates ugly resource fork files (._original_filename). Those are not copied over.
  • The file is only extracted if git is installed on the target server. That’s the purpose of the git —version line.

The motivation for this is that unlike subversion git does not allow me to check out a repository into an existing directory. At least not with git clone – lower-level commands may be able to do this. So I always ended up doing a git clone into a temporary directory, followed by moving around some files to get the setup working.

This script now allows me to get around this problem.

Skolelinux 2

Posted by Patrice Neff Tue, 14 Mar 2006

Well, look at what we've got here. Because of this this subversion checkin I saw that Skolelinux 2 is now live. And indeed, on the English download page you are now presented with the option of downloading Skolelinux 2.0r0. Great!

This version brings a ton of improvements. First and foremost, this is based on the Sarge release of Debian and therefore includes a lot more recent software and an improved installer. Second it includes very good Spanish translation. And it generally just works very nicely.

This new release is a major step and am I very glad for the Skolelinux/Debian-Edu project that they (or rather we, as I also helped my little bit) got it out.

Another Skolelinux lesson

Posted by Patrice Neff Tue, 28 Feb 2006

I taught another lesson on Saturday. This time to five teachers who are not from the . I showed them how to install the standalone client including how to partition the disk. They were very content and one of the participants commented that it's easier to install than Windows. And that's exactly my opinion as well. The only thing that makes a Linux installation more difficult than Windows nowadays is the fact, that it doesn't come pre-installed. Only very few people actually install Windows themselves, while most people just use the pre-installed OEM versions.

I'll have a lesson with the same people again at one of their colleges to install Skolelinux clients in their network.

Skolelinux 2.0 is getting ready

Posted by Patrice Neff Tue, 21 Feb 2006

The next release will soon be released. This will finally be based on Sarge. For me one of the major advancements is the language support. The installation and most of the system can be used in Spanish. This was not the case with Skolelinux 1.

In the past weeks we have translated the default Apache homepage (for the server installation) and the LDAP user admin in Webmin. Also parts of the installation suddenly were in English again, but with help of the mailing list we fixed that as well.

So it seems to me that we have a nice Spanish Skolelinux system. A few parts are not translated yet, such as the Netgroup administration. I hope to address that in future Skolelinux upgrades.

And here at the we already use the Sarge-based system. We installed it from the release candidates and it works just fine.

iTALC

Posted by Patrice Neff Tue, 14 Feb 2006

At our network here at the I installed yesterday. This is a software for teaching environments which allows a teacher computer to control the student computers. He can block the screen, watch what's happening on the screens, show his screen on the student's computers, start programs, send messages, etc. And all of this functions come with a nice () user interface - though of course it also works in (where we use it).

In order for the demo modus to work, I had to patch the application slightly. Technically the whole thing is based on VNC and SSH - proven protocols. And I distributed the software installation, configuration and teacher public keys to the clients with cfengine. Nice.

I will post some screenshots later when I'll use that in my first Python class.

Spanish Skolelinux

Posted by Patrice Neff Tue, 07 Feb 2006

I'm working on translating some more components of to Spanish. The default Web site of a server installation is done and I have also translated the LDAP user administration (a Webmin plugin). Though I currently don't know how to include the translation into the official distribution.

And it seems that in the following days a release candidate for Skolelinux 2 will come out. That version will finally be based on Debian Sarge and will include very good Spanish support. Version 1 of Skolelinux was not very good in that aspect, yet.

Version control for /etc

Posted by Patrice Neff Sat, 28 Jan 2006

Last year I tried using RCS to do a version control of a server's /etc direcotry. That didn't work out too well, because the extra copies of every archive turned out to cause errors for some applications. Also it was very cumbersome to use, because you have to manually add every archive and use 'co' every time before editing.

So I'd much rather use something like or darcs. Especially darcs because it doesn't create a separate control directory for every directory unter version control (CVS creates a directory called "CVS" everywhere, Subversion one called ".svn" and darcs just creates "_darcs" in the top level directory.

Today for the new server at the Diego Thomson I did some experiments with Subversion. This went from "cool" to "oh now" to "cool" to "oh now" now. But let me extend on this story. First I thought, well this would be very cool. Also because most applications will ignore the hidden folder (".svn") while having problems with visible folders ("CVS") this should work. And: subversion stores symlinks without problems. Then I realized that it doesn't store file permissions and was demoralized by this. A Yahoo! search then turned up the Subversion FAQ, specifically the entry How can I do an in-place 'import'.

# svn mkdir file:///root/svn-repository/etc -m "Make a directory in the repository to correspond to /etc"
# cd /etc
# svn checkout file:///root/svn-repository/etc .
# svn add apache samba alsa X11
# svn commit -m "Initial version of my config files"

I modified this commands, executed them and really: my file permissions were preserved. The I tested some changes at the sudoers file. Added a line but realized I didn't like it. So an svn revert later my sudo calls suddenly reported "sudo: /etc/sudoers is mode 0644, should be 0440" and didn't work. Ouch. So when doing revert the file permissions will not be preserved after all.

Has anyone a solution or does know a system which will preserve my permissions (darcs won't either and doesn't even know about symlinks, yet). I prefer systems that do not clobber my file tree and only create one single directory.

Update: Later I tried out Bazaar as mentioned here by Maxi. Bazaar-NG doesn't know anything about file permissions either. I tried it first and my discovery is also confirmed in this mailinglist thread. (Though it does mention that GNU Arch knows about permissions, so I'm off to testing that now.)

Details for cfengine and Debian-Edu patch

Posted by Patrice Neff Mon, 19 Dec 2005

Philipp asked for details about my cfengine patch to Debian-Edu. It's not as breath-taking as he seems to think. What it does is adding a new cfservd default configuration for Debian-Edu workstations. This way all newly installed workstations can then be configured using cfrun from the server. There two things which are really cool about this. First you can install new workstations and don't have do to any manual configuration. Second there is a working cfservd configuration on that new workstation which you can use to push your central configuration from the server. The workstations also automatically pull changes from the server on reboot. That's there for workstations not online during the last execution of cfrun.

Some more details are included on my mailing list post on the Debian-Edu mailing list.

cfengine for Debian-Edu

Posted by Patrice Neff Sat, 17 Dec 2005

During my work at the Diego Thomson we're going to improve the network a bit. One of the things I want to incorporate is a network-wide configuration using . Because I want to make (re-)installation of single clients easy without using any fancy distribution tools, I want to incorporate some cfengine configuration into the (better known as ) installer.

We had some discussion about this on the mailing list and people generally seemed to like my idea. So I went to work writing a patch. It was quite some work. Especially building the ISO image took a while. And then testing it. And reinstalling the test system, because I had forgot a colon somewhere. And so on.

And this is my patch proposal, if anyone is interested: cfengine.patch. Or you may just want to follow the discussion on the Debian-Edu mailing list if you're interested in that kind of stuff. ;-)

Using Squid as an APT cache

Posted by Patrice Neff Fri, 18 Nov 2005

I'm currently re-installing Linux a lot of times, because I'm testing Skolelinux. Also building a custom CD (not successfully yet) required downloading a lot of packages. So instead of wating an hour or so every time I have now changed the configuration to allow caching of APT packages.

The relevant changes in the configuration file:
# Keep deb packages in cache for 30 days
refresh_pattern deb$ 43200 100% 43200
refresh_pattern Packages.gz$ 1440 100% 1440

maximum_object_size 102400 KB

cache_dir ufs /var/spool/squid 1000 16 256

The two refresh_pattern lines allow caching deb packages for 30 days and Packages.gz for one day. The maximum_object_size directive is necessary because by default objects larger than 4 megabytes are not cached. And finally I changed the cache_dir line to have a gigabyte spool instead of the 100 megabytes of before (Debian default configuration if I remember correctly).

I could also have used , but that was not readily available on the Linex version we currently use on that server.

Update November 21: Seems to have been a very good idea: Fetched 255MB in 36s (6989kB/s)

Custom configuration of Skolelinux clients

Posted by Patrice Neff Thu, 17 Nov 2005

I'm currently investigating how (or Debian-Edu which is the same) could provide means for custom network configurations. What I mean by this is, that the Skolelinux installer could query the network server for a list of packages or some configuration files to install on every newly installed client in the network. This would greatly simplify customizing the network for us at the Diego Thomson.

A few examples of the customizations I'm thinking of:
  • GNOME instead of KDE
  • Different sources.list, so the APT packages would automatically be fetched from our file server (which would run apt-proxy for example)
  • Putting a few important applications on the Desktop by default (though this would probably rather be done on the server using /etc/skel)

An inquiry on the debian-edu mailinglist seems to indicate, that such a solution is not yet integrated. So I'm now looking at solutions. I discovered Cfengine yesterday. It's already installed by default on Skolelinux clients, and is already used during setup. So I guess it would be possible to pull some Cfengine files from the server and execute them during installing.

I'm currently in the process of building a custom ISO image. Then I can test some installer modifications to find a sensible solutions.

Cfengine

Posted by Patrice Neff Wed, 16 Nov 2005

I'm currently investigating how to use Cfengine here at the Diego Thomson. It's a tool to configure all hosts in a network centrally. Had I only known that tool before, when I was responsible for Linux servers. Well, from know on I'll know it.

There is a Tutorial and a Reference in case you're interested.

Automated network re-configuration

Posted by Patrice Neff Wed, 16 Nov 2005

Sometimes the network has to be reconfigured on the fly. Two examples when this might happen are a laptop on the move (plugging into a different network) or a local computer which had a bad network connection on booting up. Of course with , reconfiguring the network is as easy as:
# ifdown eth0
# ifup eth0

What's easier than that? Automated configuration. As a mac user I'm spoiled, as OS X always reconfigures the network automatically when I plug in my cable. That's what I wanted for Linux as well. So if there is already an existing solution, please tell me. But if there isn't, I have one for you now.

First install the bash script network-status.sh somewhere on your computer (in /etc/network for example). Then execute the following command on boot:
mii-tool --w eth0 | while read line; do /etc/network/network-status.sh $line; done

How does this work? Well, mii-tool using the -w flag prints a line every time the link status changes. This line is then parsed by network-status.sh to run ifup or ifdown automatically.

This was hacked together in the last few minutes, so expect bugs and reports them to me. Also, if there is already a better solution for that out there, please tell me. I'm sure that other people already had the same idea before me.

Versionskontroller für Konfigurationsdateien

Posted by Patrice Neff Thu, 31 Mar 2005

Die letzten beide Tage habe ich für einen Kunden zwei neue Server aufgesetzt. Dabei habe ich zum ersten mal richtig mit RCS für Konfigurationsdateien experimentiert. Ich wollte das eigentlich schon lange, habe mich aber irgendwie immer wieder gescheut. Nun kann ich aber sagen: es lohnt sich. Der Aufwand ist relativ klein, der Nutzen wird aber mit der Zeit sicherlich immens. Zum Beispiel:
Hm, letzte Woche ging doch dieses beim Apache noch. Wieso jetzt nicht? Naja, mal schauen was da in der Konfiguration geändert wurde.

Wenn eine Datei bearbeitet werden soll, die bereits unter Versionskontroller steht, sieht das so aus:
co -l httpd.conf
vi httpd.conf
ci -u httpd.conf

co (checkout) entfernt den Schreibschutz von der Datei und lockt die Datei für den aktuellen Benutzer. ci (checkin) fragt nach einer Beschreibung, schreibt die Änderungen in das Log und setzt einen Schreibschutz auf die Datei. Ach ja, das -u Flag bewirkt, dass die Datei am alten Ort bleibt. Ohne dieses Flag würde die Datei selber gelöscht (könnte aber natürlich mit co wieder ausgecheckt werden).

Ist die Datei noch nicht unter Versionskontrolle, muss vor den oben angegeben Schritten erst folgendes gemacht werden.
mkdir RCS
ci -u httpd.conf

Erst wird das RCS Verzeichnis angelegt. In dieses Verzeichnis werden die Revisionsinformationen gelegt (jeweils mit dateiname,v also z.B. httpd.conf,v). Existiert das Verzeichnis RCS nicht, werden die Revisionsinformationen in das aktuelle Verzeichnis gespeichert. Mit persönlich ist es da lieber in einem Unterverzeichnis.

Der drauffolgende ci-Befehl bewirkt nun, dass die Datei in die Revisionskontrolle kommt.

Postfix + MySQL + Kommandos ausführen

Posted by Patrice Neff Tue, 19 Oct 2004

Nach langen Jahren klassischer qmail Installationen habe ich meinen Server nun auf Postfix mit MySQL umgestellt. So kann ich zum Beispiel mit dem PHP Tool Postfix Admin meine Mail-Benutzer verwalten.

Jedoch hat mir da zuerst ein sehr wichtiges Feature gefehlt. Ich habe einige Adressen, welche direkt Kommandos ausführen. Zum Beispiel für das Moblog oder auch für ein Ticketing-System, das ich für einen Kunden im Einsatz habe. Das wieder hinzukriegen hat ein wenig gedauert, klappt nun aber wunderbar. Wie das geht beschreibe ich hier.

Erstens muss die Alias-Datei (/etc/aliases bei mir) bearbeitet werden. Links vom Doppelpunkt ein Name, welche für den Empfang von Mails verwendet wird. Hier wird aber nur ein interner Name verwendet. Ich schreibe hier kunde-applikation, damit das übersichtlich bleibt. Danach ein Doppelpunkt, eine Pipe und dann das Kommando. Also z.B. wie folgt:

patrice-test: |/var/virtual/bin/patrice-test.sh

Jetzt muss die Alias-Datenbank noch in das binäre Format umgewandelt werden. Das geht schnell mit dem folgenden Kommando:

postalias /etc/aliases

Danach wird in der MySQL Datenbank ein Alias für die wirkliche Adresse angelegt. Dieser muss auf den bereits angelegten Alias zeigen. Also zum Beispiel test-AT-patrice.ch zeigt auf patrice-test-AT-zion.patrice.ch, wobei zion.patrice.ch der Hostname meines Servers ist (-AT- = @ in den Beispielen).

Linux kostet hunderte von Millionen

Posted by Patrice Neff Wed, 13 Oct 2004

In der Linux Newsgruppe hat ein Amerikaner angeboten, eine Linux Version für 50'000 Dollar zu kaufen um diese dann unter die BSD Lizenz zu stellen. Neben all den politischen Fragen kam dann auch die Frage auf, wie viel Linux wert ist. Mit dem kostenlosen Tool SLOCCount hat Ingo Molnar berechnet, dass die Neu-Entwicklung von Linux etwa 176 Millionen US-Dollar kosten würde. David Wheeler hat das dann noch detaillierter geprüft und kam auf ca. 612 Millionen US-Dollar.

Sehr interessant. Vor allem, wenn man bedenkt, dass es dabei "nur" um den Kernel handelt. Noch keine zusätzlichen Programme sind dabei berücksichtigt worden. (Via Golem.de)