I needed to install the Citrix ICA client on CentOS 5.2 (RHEL 5.2), but it has very strange dependencies — it complains about a version of libXaw which is present, demands an older version of libXm, and requires manual installation of openmotif 2.2.

The trick (thanks, FriedChips!) was yum --nogpgcheck localinstall ICAClient-10.6-1.i386.rpm, rather than rpm -Uvh yum ICAClient-10.6-1.i386.rpm. This way yum chased the dependencies for me, and didn’t refuse to install the unsigned Citrix package.

Next I associated launch.jsp with /usr/lib/ICAClient/wfica.sh — Citrix should have used .ica instead, because .jsp is used for other things. IIRC, EMC NetWorker used .jsp to launch their graphical console.

Unfortunately the ICA client insists on being wider than the physical display, but I can work around that. I wonder if it’s because I simultaneously connected to the same XP system via RDP from both Linux and a Mac with different resolutions.

Update: Citrix is fixed on the size of my MBP’s 1440900 main display, which means it doesn’t fit properly on the MBP’s external 12801024 (or landscape 10241280) or my Linux box’s 12801024.

Annoyingly, Citrix assigns the Mac’s Command key to Alt on the Windows host. This doesn’t work well, because although they avoid most Command key combinations in the ICA Client, Command-Tab switches Mac apps rather than Windows windows. Guys, just use the Option key! It even says alt on it, and nobody needs that key for Mac specific functions! Today’s happy discovery: Command-Option-Tab switches Windows apps.

Next I have to figure out how to de-assign Alt-Tab from switching virtual workspaces in KDE. Copy & Paste don’t work consistently when connected from KDE either, presumably because some events are being interecpted locally and others are being passed through. I won’t need to use KDE as a Citrix terminal for much longer, though.

Crud. After all that, the Citrix ICA client doesn’t display most text, making it useless. I can get some things to display by selecting them, but many things (including dialog boxes) are un-selectable. Junk!

Ouch! At 10:31pm last night, I started patching both Linux servers running reppep and associated domains, prompted by Rich’s BIND alert. At 12:33am, www.reppep.com finished installing approximately 255 CentOS patches (including BIND), and I rebooted. Everything looked fine, and I went to bed. This morning, I thought it a bit odd that I didn’t have any new email, but not that unusual.

Melissa left me a message that mail wasn’t flowing, but I couldn’t fix it at work. Tonight I discovered that amavisd-new, which handles filtering for reppep email, was unable to start. Strangely, it was complaining about the Compress::Zlib perl module, which was actually installed (version 2.008, via the perl-Compress-Zlib-1.42-1.fc6 RPM). Some more digging indicated Scalar-List-Utils-1.19 needed to be reinstalled, which enabled amavisd-new to start (it checks for Compress::Zlib and refuses to start if it finds something wrong, which was apparently triggered by the Scalar-List-Utils issue).

mailq showed me postfix was now getting errors from amavisd-new about MIME::Parser and File::Temp. CPAN reinstalled MIME::Parser and said File::Temp was already current.

I bounced amavisd-new again, and tried postfix flush. Over the past 15 minutes, postfix has delivered the ~~650 outstanding messages, and all seems well.

Separately, Alex noticed our blogs were inaccessible, but bouncing BIND tonight cleared that — odd, as I checked http://www.bertpepper.com/ and got valid DNS resolution from both nameservers immediately after patching, but obviously something I didn’t notice was still scrambled.

Anyway, at 8:45pm, all seems present and correct.

Sorry for the disruption!

A while ago I posted a script for driving HandBrakeCLI. But it was lame that I had two not-quite-identical versions of the script — one for iPhone output and another for Apple TV. At a guess, Brian Beardmore only needed one type. Now that we watch movies on the Apple TV and I watch on the iPhone, it was silly to have two different scripts. So I added simple argument processing.

hb.sh v1.0.3

If there first argument is iphone, then hb.sh optimizes for iPhone. If the argument is appletv or there are no arguments, hb.sh optimizes for Apple TV. It’s very easy to tweak or add your own types — just look for myArgs in the script and add or adjust as desired. I run this script on my Linux server, which has lots of disk space and is generally idle.

On my MacBook Pro, I have a couple aliases to facilitate things. I copy DVD folders to ~/tivo/tivo-inspector/input and run one of these. When done, the script opens up ~/tivo/tivo-inspector/. I move the DVD folders out of input and the processed .m4v video out of output; then I drop the .m4v files onto iTunes’ LIBRARY area (so it doesn’t stop whatever it’s currently playing) and check iPhone videos to sync to the iPhone (the Apple TV has plenty of space, so everything syncs to it). Note that these lines may be too wide to display properly in WordPress — just Copy and Paste, and you’ll get the full text.

alias hbatv="ssh -t inspectore time screen bin/hb.sh appletv; open ~/tivo/tivo-inspector"
alias hbip="ssh  -t inspectore time screen bin/hb.sh iphone;  open ~/tivo/tivo-inspector"

Note that inspectore is the name of my Linux server. This would work just as well with HandBrakeCLI on a Mac “server” — or even Windows, if you set it up to accept remote commands (CygWin, anyone?).

Reminder: You must adjust the inputSearchDir and outputDir paths for the running HandBrakeCLI.

pepper@inspector:~$ egrep tivo bin/hb.sh
inputSearchDir="$HOME/tivo-inspector/input"
outputDir="$HOME/tivo-inspector/output"

In the future version I’d like to support for arbitrary HandBrakeCLI arguments on the hb.sh command line, but I first have to see if HandBrakeCLI can handle gracefully conflicting arguments from built-ins and the command line.

I use Handbrake to rip DVDs for my iPhone and Apple TV (and previously for my Treo 650, at much lower quality). It’s excellent, but takes a long time, and slows down the whole system while compressing. My PowerBook G4 wasn’t powerful enough to do H.264 compression in reasonable time, so I got into the habit of ripping the DVDs on the Mac with Mac The Ripper, copying them to the server, converting on the server, and copying back down to the iPhone through iTunes on the laptop. Transferring a full DVD via gigabit Ethernet takes under 10 minutes, which is why we have 3 1/2 gigabit Ethernet switches (8-port, a pair of 5-ports, and the 3 ports built into the Time Capsule) in a 3 1/2 person (2 grown-ups, a 5-year-old, and the cat) home.

Since then I’ve upgraded the laptop and replaced the server, so they’re now both 2.4GHz Core Duos, but I still prefer to avoid the load on my personal machine — especially now that we have an Apple TV, which likes large high-quality video files. The MacBook Pro runs HandBrake.app (Mac GUI), while the server runs HandBrakeCLI (Linux).

HandBrakeCLI is designed to convert individual files and is awkward for converting whole DVDs. Fortunately I found MediaForkCLI-batch.sh, a wrapper which is good at converting all the real titles from one or more DVDs; it automatically ignores short titles such as previews, and titles that have already been transcoded. Unfortunately MediaForkCLI-batch.sh is languishing, and hasn’t been updated for HandBrake v0.92, but it was straightforward to update it to the new syntax.

In the process I made some changes and added an alternate version with different settings optimized for our Apple TV. On the Linux box (named inspector), the scripts are ~/bin/appletv.sh & ~/bin/iphone.sh. I put newly ripped files into ~/tivo-inspector/input/ on inspector, and these scripts rip whatever they find there, generating files in ~/tivo-inspector/output/.

On the MacBook Pro, I keep video files in ~/tivo, and ~/tivo/tivo-inspector is a symbolic link to tivo-inspector on inspector, the server — mounted via AFP. This makes it easy to move files back and forth.

For convenience, I have a couple aliases on the MacBook Pro which log into the Linux machine, start a screen session, and run the appropriate script. This means that if I disconnect the laptop, I can later ssh back into the server and use screen -DR to reconnect to my running conversion session. When the conversion is done, the aliases open the remote folder in the Finder. I move the files back up out of input & output, drop them onto iTunes, and then either Sync the Apple TV (which gets copies of my whole iTunes Library), or sync them to the iPhone.

The scripts are appletv.sh & iphone.sh.

If you use these scripts, you should carefully review the encoding settings. In particular, I encode Apple TV audio to AAC stereo, because our Apple TV is connected to a pair of stereo speakers with a subwoofer; they cannot handle Dolby Pro Logic, which is better for most TVs.

You will also have to update paths in the scripts — to the input and output directories, and to HandBrakeCLI itself — to make them work on your own system.

Here are the aliases in sh/bash format; change the equal signs to spaces for csh/tcsh:

alias appletv="ssh -t inspectore time screen bin/appletv.sh; open ~/tivo/tivo-inspector"
alias  iphone="ssh -t inspectore time screen bin/iphone.sh;  open ~/tivo/tivo-inspector"

Half because WordPress really needs to stay upgraded, and half in hopes of fixing the Admin-SSL bug which was blocking posting, I upgraded to WordPress 2.5, a compatible beta of Admin-SSL (now under new management), and a few other plug-ins.

Not knowing how well the upgrade would go, I did the safe thing — I installed WP 2.5 separately from the live Extra Pepperoni site, installed and configured all the plugins I use (with my personal patches), created a new MySQL database, and configured everything, including a couple test comments (not as myself). After I got it working, I brought down the old site, moved the new one in place, reconnected it to the old MySQL DB (with all posts and comments), clicked the button to upgrade, and we’re up.

Unfortunately, there’s still a problem with comments. When I log into a new account to comment, I get a link to https://secure.reppep.com/wp-admin/profile.php, which is bogus; it needs to be https://secure.reppep.com/ep/wp-admin/profile.php. If you have an existing account (Tony), you might be able to login through https://secure.reppep.com/ep/wp-admin/ and comment, but it seems that viewing an actual post (which must be non-SSL) still loses its association with the login session, so you can visit the HTTP site as an anonymous user, or use the HTTPS site as your registered user, but the plaintext side has no access to comment, and the encrypted side doesn’t show the posts you would want to comment on. Hopefully BCG will be able to fix the problem in Admin-SSL. He’s already fixed the Preview function.

Also freaky: When I log into EP as a brand-new user (to comment), I get the Dashboard, telling me I (the brand-new user) have 184 posts. I didn’t think Subscriber users saw the Dashboard, but the post count is definitely bogus.

I did the initial installation as a Subversion checkout, which is very cool. Now, though, I have to create my own private WP hacks repos (easy), and figure out how to set up externals to pick up my additions.

A tip: Don’t try to check out the WordPress source over AFP; the permissions weren’t right, and the checkout couldn’t complete; when I did it locally on the Linux server, there was no problem. I hadn’t even noticed I was running “svn co” on the Mac instead of the server, but it was easy to fix once I noticed the cause.

I got Exceed, and after reassigning my left modifiers to X instead of Windows, kate is quite reasonable. It uses kompare for graphical diff, and comes with some CVS plugins. I prefer BBEdit’s diff display (although BBEdit’s diff has been broken for years). I’m not sure how I managed to view a couple windows in xemacs from kompare, but I can probably avoid that in the future…

kate is clunkier and less featureful, and not as configurable as I expect — the commands I want to assign to the toolbar, for instance, are not available in that context. On the other hand, BBEdit doesn’t use a toolbar at all, and rearranging menus is only supported in limited ways, so I’m not convinced kate is inferior here — it may just feel like that to me as a BBEdit user. Hopefully Subversion support is available for kate, but that doesn’t actually matter to me right now.

I need to get Copy & Paste working between Windows (including PuTTY) and Exceed; hopefully this will be straighforward, but it doesn’t just work.

Per IDM, UltraEdit cannot be installed without admin rights, which I do not expect to get, so that’s out — at least until they offer an alternate installer.

kate icons are a bit fuzzy, but they fit the Linux aesthetic, and the fonts are very nice.

pctony (congratulations on your Apache httpd PMC membership, Tony!) just informed me that comments here are broken. I knew Preview was broken, and am guessing that it’s a problem with my configuration of Admin-SSL, but hadn’t known it affected anyone other than myself. Admin-SSL in this configuration creates a disruption between the public (reading) side and the SSL-encrypted authenticated side, and preview & user logins for commenting both appear to be falling into that crack.

If I can’t get Admin-SSL working this way, I’ll come up with something else, although at this point I’m hoping Haris can tell me how to sort myself out.

In the meantime, I’m sorry for the inconvenience (especially Tony’s).

His two suggestions were to quote the path in the UltraEdit installer, or to use “dir /x” in CMD.COM to find the DOS-style 8.3 pathname of the destination folder. Unfortunately, I seem to have been wrong about the cause for their installer’s terribly vague “1925″ error message, as I tried another viable path (not containing spaces) today, and UE failed to install there too. Perhaps it’s a registry access issue — I sent email to IDM Software, and hope they have a more useful suggestion than “become an administrator”.

After DreamHost’s breach 8 months ago, I was aggravated at their poor handling of the situation, but willing to give them the benefit of the doubt, and still happy with their low prices and flexible services.

With the new bad news and worse confirmation (still with poor incident handling), though, it’s time to get out of dodge.

I have moved Extra Pepperoni back onto my own hardware. I started blogging on Apple’s Blojsom install, but gave up on Tiger Server for Blojsom (and Mailman) because the services kept silently shutting down, leaving me to notice they were disabled days or weeks later (no fault of Blojsom or Mailman — Apple didn’t do a good job porting SpamAssassin either). Bringing up a WordPress blog and mailing lists at DreamHost was easy and cheap, but that’s no good if they are unsafe.

I’ll look at moving a couple very light-duty Mailman lists off DH next, but the lists are so lightly used I’m not too concerned. There just isn’t any confidential information on the mailing lists, aside from their tiny subscriber lists.

Ah, well. I now know much more about WordPress and MySQL than I cared too, but the setup wasn’t too bad. I hadn’t realized how many customizations and tweaks I made to WordPress until it came time to recreate them on my own system:

1. Almost Spring theme (included by DreamHost); with minor hack
2. PHP Markdown Extra; with minor hack
3. MySQL admin UI
4. WP-DB-Backup (DH included one, which I’m no longer using)
5. mod_rewrite for permalinks
6. Admin-SSL, with “Shared SSL” tweak, integrated into my existing SSL site (meaning EP is available through two different “sites”, and I have to keep the Apache configurations reconciled)
7. Twitter
8. WP-Cache (DH standard)
9. Akismet anti-spam registration
10. Technorati pinger (came over automatically with the DB).
11. Fix for widget.php to use legal JavaScript tag.

It feels very very strange to be unemployed — it’s been 7 years since the last time, and I was too freaked out at Shooting Gallery laying me off to feel this way. Now that I’m a grown-up (having kid(s) means you’re responsible, even when you’re irresponsible!) it’s a good thing that we’re covered by RU insurance past the start date for GS insurance, but the whole experience is still very odd. I wiped the third computer today at 5:30pm, and am copying data off computer #4 (old reppep.com) right now in preparation for retiring it (it’s falling apart, apparently — optical drive died an hour ago).

Now I just need Apple to update the MBP15s, so I can replace this PowerBook. It’s doing better than I thought, though — doesn’t seem any doubt that it will serve until the next update.

RU IT did right by me today — a grand spread, consisting of John’s pizza, baby back ribs, beef ribs (they looked like something from The Flintstones), and chicken wings. A nice (short) speech by Armand, and well wishes all around. Elaine hung a bunch of colorful signs, which delighted Julia.

I closed out my helpdesk tickets, turned in my keys (forgot to turn in my ID/swipe card, though), and updated the documentation on our load balancers again, as well as re-re-recapping for my co-workers. I had to say “Look, when you feel like you’re an idiot, don’t worry — I felt like that repeatedly for years while working with these. The Big-IPs are absurdly complicated. Two kernels, a super ’switch card’ that’s doing all kinds of crazy (non-switch) stuff, over 20 IP addresses, 8 networks, plenty of bugs, and delays in getting technical support. It’s not you!”

Maybe I’ll have some time to investigate Linux & Windows text editors.

On Feb 19, 2008, I shut down the old reppep.com server, which ran Mac OS X 10.4 “Tiger” Server, and replaced it with a new (cheaper and faster) PC running Linux. Unfortunately, the password formats are incompatible, so I apologize to app reppep users for the disruption.

Please call me if you have an account on reppep.com and haven’t received your password already, or find anything not working right.

I switched from Apple’s jabberd to Openfire, which doesn’t use the UNIX system accounts, so let me know if you want a chat account (compatible with iChat & GTalk).

[Done] I forgot SquirrelMail address books — should be able to bring those over too.

• Firewall problem fixed. SMTP MX issue fixed.
• Virus filtering problem fixed.
• Webmail certificate fixed.
• Quota problem fixed.
• Virtual domains for email fixed.

As of 5pm, I don’t know anything that doesn’t work (aside from SquirrelMail address books) [fixed Thursday].

Thanks for your patience!

As of 10:30 on the 20th, things seem to be working. Something’s screwy with amavisd-new’s quarantine, but mail is going through. I reinstalled Openfire, and chat seems okay under the correct hostname/certificate name now (will try signing it as ca.reppep.com later).

Good timing — the optical drive on the old server died tonight.

I have distributed all the new temporary passwords, so any users having trouble logging in should let me know.

Markdown.cgi is still broken, but I’m the only person who uses it here, so I’ll get to it.

On Thursday the 21st, I found a problem with amavisd-new — it had quarantined 32,000 messages in a single directory, and was stuck (apparently ext3 doesn’t support more than 32,000 files in a directory). I cleared it out and finally managed to disable quarantine, which wasn’t as easy as it should have been, and the backlog of messages have been delivered as of 9:15pm.

At 11pm, I fixed an issue preventing SMTP AUTH from working properly, which was interfering with sending email to non-reppep addresses.

I was quite impressed by Joel’s description of the hiring process, and we’ve been doing a lot of interviewing for System Admins lately. I put together a list of standard questions to ask during interviews, which has been quite helpful in judging a) how much technical knowledge people have, and b) (just as important) how good a match they are for the skills void we were trying to fill at the time. Here they are, for the next person who needs to perform a similar exercise.

1. How many systems does your team manage (Linux, Solaris, Windows, etc.)?
2. How large is your team?
3. Which OS are you most comfortable/familiar with?
4. Which Linux flavors are you most comfortable/familiar with?
5. Which Red Hat versions are you familiar with?
6. Are you familiar with kernel programming or configuration?
7. Have you done any custom packaging or kickstarting?
8. Have you used or managed Sun JumpStart?
9. How much experience do you have with Sendmail?
10. … NetWorker? Version? Managing backups, or just configuring clients?
11. … LDAP? Brand & version? LDIF or just querying?
12. … firewalls (iptables, ipf, etc.)?
13. … network administration (Cisco, sniffing, etc.)?
14. … Apache httpd?
15. … Tomcat & Java?
16. … EMC (Clariion, PowerPath)?
17. … shell scripting, and with which shells?
18. … perl scripting?
19. … Veritas VM/FS? Versions?
20. … Veritas Cluster, or other HA? Versions?
21. … snapshots? In which products?
22. … load balancing
23. … Oracle (as SA, not DBA)?
24. … HPC?
25. Please briefly explain the difference between RAID 1 and 5. What are layered RAID levels, and when are they appropriate?
26. What sizable projects have you done recently?
27. Why are you leaving your current employer / did you leave your last employer?
28. Please give specific examples of some routine tasks you’ve performed recently.
29. Have you done systems specification and design (servers, multi-server configurations)?
30. Have you worked with customers directly, or primarily with/for other IT personnel?

It didn’t make sense to publish a list of questions when I was involved in the interviewing process, but now that I’m leaving Rockefeller and no longer interviewing UNIX Admins for them, I can post my sample questions.

For several years, I’ve been saying Apple made a bad choice when they picked Cyrus IMAPd as the POP/IMAP server for Mac OS X Server. It’s a huge and complicated system, encompassing IMAP, POP, SSL, Sieve filtering, LMTP delivery, USENET news, clustering/proxy (Murder), pluggable authentication (SASL), etc. I cannot think of a single company outside Cupertino where it would make sense to run an enterprise mail system on Mac OS X Server, but Apple continues to add these inexplicable high-end features to its mail server, most recently XSan-based email clustering in Leopard Server.

The statement that convinced me (shortly after I had migrated to Cyrus IMAPd on Mac OS X Server 10.4 “Tiger”) that I would never choose to run Cyrus for my personal use, was the following — which I came across again today:

Installation Overview

This system should be expected to have the same order-of-magnitude installation complexity as a netnews system. Maintenance should have similar complexity, except administrators will have to deal with creation and deletion of users and will have the option of managing quotas and access control lists.

USENET news is infamously demanding and bandwidth intensive. It would be wonderful if Apple had taken Cyrus IMAPd, repackaged it (without too many changes!), and put a powerful and simple interface on top. The did this quite successfully with Apache httpd (although Server Admin breaks down on complicated configurations and has obscure bugs). Lots of people use Mac OS X Server to run websites and think it’s easy & simple. Considering the typical reactions of those same people to the httpd .conf files “under the hood”, this is a noteworthy triumph. Similarly, Time Machine provides a reasonable approximation of scheduled snapshots on a high-end NAS for do-it-yourself file recovery, with a simple interface that insulates users from the nitty-gritty of copy-on-write and hard links.

Cyrus did not get as much attention, though. Basically, Apple makes it pretty easy to create email accounts, provides a Repair button for the overall Cyrus database, and provides a Reconstruct button for individual accounts. That’s about it. Unfortunately, Apple doesn’t really document maintenance beyond “press the button and it will fix your problem”. I’ve had several serious database problems which Apple’s Repair button did not help with. Those were bad times.

Similarly, I have had problems where users could not log in, but Workgroup Manager claimed their accounts were usable. I eventually discovered that resetting passwords with passwd works sometimes, and re-setting passwords in Workgroup Manager works consistently, but when I asked Apple about it, the eventual response was basically, “Yes, that’s bad; you should restore your accounts from your recent Open Directory export.” Not a good answer.

It doesn’t help that Apple’s SpamAssassin and ClamAV installations are broken, as these result in more spam and slower deliveries.

So why am I planning to migrate to Cyrus IMAPd on CentOS 5.1? Well, I’d really like to just copy my 5gb mail directory to the new system and have my clients not notice the difference. Eudora doesn’t handle (IMAP) change well — renaming a single IMAP directory can force it to download all messages again, and various other things can cause Eudora to lose date stamps on sent mail, or message state information (when it gets disassociated from the actual message on the IMAP server). If I can make Cyrus work, I’ll be very happy, and if I can’t I’ll try Dovecot (Red Hat’s default) or Courier (which I hear is also good).

Also, I know it can work, and I have a rough model to work from on my Tiger Server, but if I wasn’t using Cyrus already I would stay away from it, as I wish Apple had done.

I’m digging deeper into CentOS (basically a free version of Red Hat Enterprise Linux) v5.1, and for me that entails dozens of runs through the installer, testing out kickstart configuration variations.

This led me to wonder if it is faster to install via NFS or HTTP. I couldn’t find a useful answer online, so I ran a couple simple tests. My client is a 2.4GHz Dell PowerEdge 600SC using SATA disks on a Promise TX4. My server is a dual 1.25GHz Power Mac G4, running Mac OS X Server 10.4.11. They’re connected via private network, using a NetGear gigabit Ethernet switch.

For NFS installations, anaconda takes a directory containing a DVD ISO (or set of CD ISOs), and automatically loopback mounts them as part of the installation process. This is very handy with the CD ISOs, as it doesn’t require much configuration on the server — just an NFS export.

In contrast, HTTP installation doesn’t work against ISOs — the web server must serve up the individual files, whether from a loopback mount on the server, or a directory where the files have been extracted. With the 6 CD ISO files, this is quite a nuisance; with the DVD ISO, it’s not so bad.

My fairly complete kickstart configuration installs 2,103mb of packages. Installation times were quite similar, but a bit faster for HTTP, at 18:28 for package installation and 24:37 total. Via NFS packages took 19:57; total was 27:38.

For reference, I used the following partitioning configuration, which factors into total installation time:

part /boot --onpart=sda1 --fstype=ext2
part /     --onpart=sda2 --fstype=ext3
part swap  --onpart=sda3 --size=2048
part /var5  --onpart=sda5 --fstype=ext3
part /home6 --onpart=sda6 --noformat
part /sdb1  --onpart=sdb1 --noformat

The partitions already existed:

[root@pe ~]# df -hl|grep -v tmp
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             7.7G  2.8G  4.5G  39% /
/dev/sda6             664G  198M  629G   1% /home6
/dev/sda5             3.9G   73M  3.7G   2% /var5
/dev/sdb1             688G  198M  653G   1% /sdb1
/dev/sda1             251M  7.4M  231M   4% /boot

I used the following set of packages for testing:

%packages
@core
@base
@editors
@system-tools
@text-internet
@dns-server
@base-x
@network-server
@games
@web-server
@mail-server
@server-cfg
@admin-tools
@development-tools
@graphical-internet
@office
@development-libs
@x-software-development
@gnome-desktop
@graphics
@kde-desktop
@sound-and-video
kdegraphics
kdemultimedia
-bluez-gnome
-bluez-hcidump
-bluez-utils
-evolution
-evolution-connector
-evolution-webcal
-ibmasm
-krb5-auth-dialog
-mod_perl
-mod_python
-numactl
-openldap-clients
-openoffice.org-calc
-openoffice.org-draw
-openoffice.org-graphicfilter
-openoffice.org-impress
-openoffice.org-math
-openoffice.org-writer
-openoffice.org-xsltfilter
-pam_krb5
-pcmciautils
-php-ldap
-planner
-rp-pppoe
-sabayon
-sendmail
-sendmail
-sendmail-cf
-squid
-stunnel
-sysreport
-system-config-samba
-tux
-vconfig
-ypbind

In Leopard, Tab completion in bash doesn’t immediately append trailing slashes to symbolic links that point to directories. When I complained about this change to Apple, I was told it was user configurable, and I should just configure the old behavior. It took me a while to actually find the solution — partially because it isn’t within bash itself, and partially because it was quite a nuisance but not a serious problem.

For example, ~/www is a symlink to /Volumes/www, and I cd to directories below it quite frequently. I’m in the habit of typing “cd w[Tab]/pu[Tab]“, which should expand to “cd www/public_html“. This broke in Leopard — I needed an extra Tab to get the / — otherwise I’d end up with the ugly and non-functional “cd wwwpu“. Anyway, the fix is:

echo "set mark-symlinked-directories on" >> ~/.inputrc

Now Tab completion works the way I want it to again. Thanks, stylishpants!

Red Hat has, with good consideration and foresight, been pushing people to use logical volume management for a while. It’s not completely integrated into the RHEL5 installer, but they’re pushing hard to make it ubiquitous, and telling people this is the right way to do things. Unfortunately, the syntax for specifying logical volumes within DOS-style partitions is still a bit obscure, and the manual page examples don’t show the LV syntax; this is fixable, but will take time.

I used software RAID and LVM on my new installation, but it doesn’t boot — I’ve found several articles on making GRUB work with software RAID, so I believe I’ll be able to get it working. The docs say I should be able to just use “lvm” (which is present) to get an lvm shell, but neither lvm nor lvm.static does anything — they just dump me back in bash.

Fortunately, “linux rescue” finds my partitions (this time), but not being able to even list out physical volumes is worrisome.

I want mirrored /boot, but it’s RHEL’s mirrored /boot capabilities are pretty limited:

If you are making a RAID partition of /boot/, you must choose RAID level 1, and it must use one of the first two drives (IDE first, SCSI second). If you are not creating a seperate RAID partition of /boot/, and you are making a RAID partition for the root file system (/), it must be RAID level 1 and must use one of the first two drives (IDE first, SCSI second).

Speaking of LVM being immature, check out What is the process to fsck lvm volumes? in the Red Hat Knowledgebase:

First, boot into rescue mode by using the correct media. This is very important: When prompted to mounted the drives, do not. Using fsck on a mounted filesystem will destroy all the data on that file system. This is unrecoverable. The data will be gone forever–save for very expensive hardware-level data recovery.

For over a year now, I’ve been following the development of Mac OS X Server 10.5 Leopard and testing betas, and anticipating upgrading reppep.com from Tiger Server on a dual 1.25GHz Power Mac G4 to Leopard Server on a dual 2GHz Power Mac G5. Over the weekend I had a change of plans, though.

Although I support Mac OS X Server at Rockefeller, I don’t recommend it for most requirements, as Linux compares favorably for transparency (some of the MOSXS internals are unique and poorly documented), server software compatibility (although Macs are quite good here too), and price/features at the low end. A Core Duo Mac mini has plenty of juice to saturate our 768kbps/3mbps DSL circuit, but adding a couple drives more than doubles its price, and Apple’s software RAID is quite broken; Linux software RAID is apparently quite good; I might eventually switch to hardware RAID. An Xserve is a great piece of hardware, but it’s a bit exotic and I can get a fast generic PC cheaper; I don’t want all the high-end features for a box that sits in our apartment.

Additionally, I’ve read perhaps 600 pages of docs on Leopard Server, and had at another 400-1500 yet to go. This is an investment I was finding hard to justify. The migration process is quite complicated, and Apple doesn’t support migrating accounts from a Tiger system to a Leopard system — I don’t want to do an upgrade. I could clone the G4 to the G5 and upgrade it there, but I prefer to handle upgrades as scratch installations with manual migration of applications, so I know exactly what’s been done. A lot of this is masked by upgrade procedures.

As part of this, I’ve decided to invest a bit more time in learning RHEL5 — we have a couple systems at Rockefeller, but not much in production yet, and now seems like a good time to dig in some more.

Fortunately, all the services I’ve been using on reppep.com are available on Linux (and FreeBSD), so aside from another incredibly inconvenient password change cycle (for which it is arguably time anyway), the switch should be largely transparent to reppep.com users, although I still have plenty of research to do.

A brief timeline of reppep.com

1. 1999: I left the National Audubon Society, and bought the Power Mac 7300 with accelerator card I’d been using there. I set it up with LinuxPPC and Apache, and started offering free web hosting to friends & family. LinuxPPC was eventually discontinued.
2. I upgraded from LinuxPPC to Yellow Dog Linux, which was better than LinuxPPC, but had serious flaws.
3. 2001: I was working on a couple remote FreeBSD machines (as admin of the Info-Mac server, and a user on the Apache Software Foundation userhost), and decided to learn more; I bought a cheap Celeron PC and installed FreeBSD 4.3 (IIRC); I upgraded through about v5.1 and a Pentium 4 (giving the Celeron box to the Info-Mac Archive, where it became the Info-Mac server for a while). I learned a lot about FreeBSD and UNIX in general, but eventually realized I was investing more time learning FreeBSD than I could justify. The best thing about FreeBSD is not a technical feature, but rather that the user community is so rich with knowledge. Reading the FreeBSD-STABLE list was amazing, as there was so much depth, freely shared with the community. While running on FreeBSD, I added mail services to the web services I had been offering. Note: Disruptions to personal email service are much worse than problems with personal web service.
4. 2005: It became clear that I needed anti-spam, so I began researching SpamAssassin. While I was figuring out how to build the SMTP sandwich, with a public untrusted Postfix listener on port 25 & 587, and a filter, and then a listener on a high port like 10025 to accept and deliver mail to actual users, I installed a beta of Mac OS X Server 10.4 “Tiger”, which had the whole thing implemented, plus ClamAV as a bonus. I started testing heavily before the release, and switched to MOSXS 10.4 shortly after it was finalized. It’s been very good, but as time has passed, I’ve had more and more problems. In particular, Apple chose to use Cyrus as an IMAP/POP server, and Cyrus is complicated, but Apple ignores the complexity; this can make troubleshooting impossible. The SpamAssassin installation is slightly broken; it’s a bit too old to offer the newer SpamAssassin self-upgrade mechanism. Server Admin is great, but has a bunch of bugs around SSL certificates, some of which destroy the certificates. Blojsom was nice, but Apple’s installation was very unstable; I eventually moved my blog to WordPress hosted externally.
5. 2008: I intend to switch to CentOS 5.1, which is basically a (legal) no-charge clone of Red Hat Enterprise Linux 5.1. This should make future upgrades a bit more straightforward, as I won’t have to deal with Apple’s Open Directory (OpenLDAP); it will also give me a bit more experience with RHEL5, which is a better investment for my time than Leopard Server.

With thanks to Daring Fireball’s JotW.

I like The Register because they cover the stuff I’m interested in, and their leanings correlate reasonably well with my own. But they don’t edit their stuff, and have no shame about being wrong or just lost in left field. Today’s example:

Android: developer dream or Google cash machine?

By Phil Manchester

Published Friday 16th November 2007 18:49 GMT

However, it will take more than a $10 million “incentive” (http://www.sci-tech-today.com/story.xhtml?story_id=030002Y7BBKU) to truly galvanize people and generate a powerful and self-sustaining grassroots developer movement and ISV community. Some of the open source technologies changing today’s market, after all, built up critical mass because they were good, useful or employed a community friendly license - not because early developers got huge cash dongles.

Um, no. People write free and cheap Palm, Symbian, Windows Mobile, Google Maps, and (now) iPhone apps all the time. You don’t need to pay them $10,000,000 to do so.

Google’s Android agenda is far from clear, but it seems money is a driving factor, rather than a genuine desire to liberate developers and phone users from the nasty old telcos with an open platform. After all, Android’s backers include some of those very carriers that liked to lock you in (http://www.openhandsetalliance.com/oha_members.html) and have proved nothing more than an anchor on software and service innovation, but who just happen to be lagging the US market leaders.

See, the logical fallacy here is more subtle, but still big enough to throw a phone through. Google has never claimed that money wasn’t a driving factor. There are lots of people who are interested in Android for primarily non-commercial reasons. Nobody who’s awake ever thought Google (or the other Open Handset Alliance members) were among them. It’s “Don’t be evil.”, not “Liberate developers.” Whether or not you think Google is evil (I think they’re scary and cool, but not evil), they never pretended Android wasn’t supposed to make much money. Is there anyone, aside from Phil Manchester, who didn’t know that Google likes to make money and is quite good at it?

Enlightened capitalism maybe - but capitalism just the same.

Duh.

Update: It’s sillier and sadder than I thought. See below.

Today (2007/11/12), Oracle announced Oracle VM, their free competitor to VMware and (Citrix) Xen. A few months ago, Oracle announced “Unbreakable Linux”, which is their re-branding of Red Hat Enterprise Linux. There are already many free Red Hat flavors, including CentOS, but not too many companies have built business models on attempts to take Red Hat support business away from Red Hat.

Oracle has. They made many loud claims of being cheaper and better than RHEL, while claiming this wasn’t an attack on Red Hat. Red Hat was pretty quiet about Oracle Linux, but did point out that Oracle’s claims to be actively fixing bugs in RHEL (supposedly faster than Red Hat does) without forking RHEL were impossible — as soon as there’s a fix which isn’t available from Red Hat, that’s a fork.

There’s been a lot of ill feeling both ways over this, but of course neither company is willing to publicly and unambiguously badmouth the other.

Today we see another step in Oracle’s (Linux) plan: Oracle VM is free, but Oracle offers paid support. The best part is this, though:

What is the difference between Oracle VM and the virtualization that comes bundled with Oracle Enterprise Linux?

As part of the Unbreakable Linux Support program, Oracle supports virtualization that is included with Oracle Enterprise Linux 5. Please note that Oracle products are not supported to run in that environment. Any customer who wants to deploy Oracle products in a virtual environment should use Oracle VM, and subscribe to Oracle VM support. Oracle customers should refer to MetaLink note 466538.1

Translation: We sell RHEL5 (which includes Xen as part of the base price) but we don’t like it, because we want you to pay more for Oracle VM instead. We cannot realistically either break or drop support for Xen, even though we’d really like to, but we do get to chose what “platforms” we support Oracle on, so we’ll support Xen, and Oracle on Linux, but not Oracle on Xen. Please don’t think too hard about that one. It makes our heads hurt!

Update 2007/11/13: I missed the fact that Oracle VM is based on Xen. This means Oracle wants to sell you “Unbreakable Linux”, but wants to charge an extra $500 to virtualize its own software on “Oracle’s” Linux platform. I thought they were claiming Oracle VM was better than RHEL’s VM, but that can’t stand even cursory scrutiny, given that they’re basically the same code. Additionally, their

• Three times greater efficiency than current x86 based server virtualization products;

has to be in relation to VMware which is not paravirtualized, but there is no way Oracle’s brand-new Xen build is significantly faster than Red Hat’s Xen kernel, running on Red Hat’s Linux distribution.

Given that Oracle now recommends RHEL + Xen (from Oracle) as a platform for running Oracle Database & Applications products, Oracle’s lack of support for running on RHEL + Xen (when purchased from Red Hat) looks — I was going to say even more absurd, but this can’t be an oversight, so it’s just transparent corporate greed.

Update 2008/2/3: Adam, thanks for the suggestion — I’d forgotten about the vnc:// scheme. But who’s Geoff?? I prefer aliases to functions because they’re simpler, and like to leave an ssh shell open, both for my own use and as a reminder to close the tunnel when done. Here’s a simpler alias — note that you must still supply the hostname on the command line after the alias, e.g., “stss salt“.

alias stss="(sleep 4; open vnc://127.0.0.1:5901) & \
ssh -C -4 -L 5901:127.0.0.1:5900"

Update 2007/12/14: I added a pbcopy command to put ‘127.0.0.1:5901′ on the Clipboard (pasteboard), so now I can just Paste and then delete (pbcopy appends an undesired Return to the Clipboard), which makes the whole thing easier. New alias (note that this is really properly one line, but it doesn’t wrap properly without help):

alias stss='echo 127.0.0.1:5901 | pbcopy; open \
/System/Library/CoreServices/Screen\ Sharing.app; \
ssh -C -4 -L 5901:127.0.0.1:5900'

I have a couple licenses for Apple Remote Desktop at work, for managing our 8+1 Mac cluster (”the orchard”) and for managing other Mac servers on campus. I find ARD very useful because although Remote Desktop uses VNC as the underlying protocol, Apple’s compatibility has been poor, so I had lots of trouble connecting from Chicken of the VNC and other clients. While I like ARD (particularly the automatic ssh tunneling in v3), I only use the remote control feature, never its other management capabilities.

With Mac OS X 10.5 Leopard, Apple has bundled /System/Library/CoreServices/Screen Sharing.app, which provides the VNC capabilities I use from ARD and skips the other features I don’t care about. It’s my favorite Leopard feature, accessible from the Finder Sidebar, iChat, Server Admin, and through Back to My Mac (which seems to have some problems with security).

The only thing I don’t like about Screen Sharing is that Apple apparently built encryption into the VNC protocol. Apple’s encryption is of course incompatible with all the other clients & servers, since it’s Apple proprietary (just like their proprietary compression encodings). It’s confusing because the Preferences options look identical to the ones in ARD3, which actually uses an ssh tunnel to provide encryption. It’s a firewall problem because there are lots of places we a) allow ssh, b) block unencrypted VNC, and c) would allow encrypted VNC. ARD3’s ssh tunneling is usable here but Screen Sharing’s port 5900 connection is blocked. Fortunately the workaround is simple — build the ssh tunnel manually, as is normal for non-ARD3 VNC users. I have this alias:

alias stss='open /System/Library/CoreServices/Screen\ Sharing.app/; ssh -C -4 -L 5901:127.0.0.1:5900'

I use it with a hostname, as in: stss www

That makes an ssh connection to the specified host (www in this case), sets up a tunnel from 5901 on my admin workstation to 5900 on the server (since the admin workstation is likely to be running the Remote Management/Screen Sharing agent on 5900 already), and gives me a shell on www. As a convenience, it also launches Screen Sharing for me. In the Screen Sharing Connect window, I type 127.0.0.1:5901, and connect to the local end of the tunnel on port 5901; it goes through ssh and I get secure remote control via the ssh port (so it works across any firewalls that allow ssh). It’s actually doubly encrypted if I’m going across the Internet, since I always leave Screen Sharing’s encryption on too — if I forget to start the tunnel or connect to a machine that’s not firewalled on port 5900, I want to be sure I’m not transmitting passwords in plaintext.

I needed a place to keep openssl commands for reference. See http://www.reppep.com/~pepper/writing/tidbits/ssl-article/ for much more depth.

Read a cert (I use this to build all my .crt files, so I can easily tell what I’m working with later):

openssl x509 -text -fingerprint -sha1 -in certificate.crt

Read a CSR (most fields should match the account with your CA, or your private CA cert):

openssl req -text -in request.csr

The classic, for testing availability of an SSL server, is:

openssl s_client -connect server:port — e.g., openssl s_client -connect www:443

For web sites, I generally use a browser to review the certificate, but for other protocols openssl is invaluable. Apple’s /System/Library/CoreServices/Certificate\ Assistant.app/ (available from Keychain Access’ Keychain menu) is also good for verifying SSL status of arbitrary SSL servers.

For traffic analysis, ssldump can (with the server’s private key) decrypt tcpdump captures or live traffic.

From a Windows admin, requesting a cert for IIS (I have not tested):

I need for you to combine the crt with the key to make a pfx file.

openssl pkcs12 -export -out canonicalName.pfx -inkey canonicalName.key -in canonicalName.crt