A while ago I posted a script for driving HandBrakeCLI. But it was lame that I had two not-quite-identical versions of the script — one for iPhone output and another for Apple TV. At a guess, Brian Beardmore only needed one type. Now that we watch movies on the Apple TV and I watch on the iPhone, it was silly to have two different scripts. So I added simple argument processing.

hb.sh v1.0.3

If there first argument is iphone, then hb.sh optimizes for iPhone. If the argument is appletv or there are no arguments, hb.sh optimizes for Apple TV. It’s very easy to tweak or add your own types — just look for myArgs in the script and add or adjust as desired. I run this script on my Linux server, which has lots of disk space and is generally idle.

On my MacBook Pro, I have a couple aliases to facilitate things. I copy DVD folders to ~/tivo/tivo-inspector/input and run one of these. When done, the script opens up ~/tivo/tivo-inspector/. I move the DVD folders out of input and the processed .m4v video out of output; then I drop the .m4v files onto iTunes’ LIBRARY area (so it doesn’t stop whatever it’s currently playing) and check iPhone videos to sync to the iPhone (the Apple TV has plenty of space, so everything syncs to it). Note that these lines may be too wide to display properly in WordPress — just Copy and Paste, and you’ll get the full text.

alias hbatv="ssh -t inspectore time screen bin/hb.sh appletv; open ~/tivo/tivo-inspector"
alias hbip="ssh  -t inspectore time screen bin/hb.sh iphone;  open ~/tivo/tivo-inspector"

Note that inspectore is the name of my Linux server. This would work just as well with HandBrakeCLI on a Mac “server” — or even Windows, if you set it up to accept remote commands (CygWin, anyone?).

Reminder: You must adjust the inputSearchDir and outputDir paths for the running HandBrakeCLI.

pepper@inspector:~$ egrep tivo bin/hb.sh
inputSearchDir="$HOME/tivo-inspector/input"
outputDir="$HOME/tivo-inspector/output"

In the future version I’d like to support for arbitrary HandBrakeCLI arguments on the hb.sh command line, but I first have to see if HandBrakeCLI can handle gracefully conflicting arguments from built-ins and the command line.

We got an Apple TV this week, and it’s excellent, although I tripped over some serious network problems (more Mac problems than Apple TV problems, actually).

Compared to our TiVo (upstairs) or our Time Warner Scientific Atlanta HD DVR, the Apple TV is surpringly advanced. The SA box keeps losing signal (probably TWC’s wiring at fault, but they keep not fixing it), and is much larger (and noisier) than the Apple TV; basically it’s a piece of junk, but it’s substantially cheaper than another TiVo. We’ll probably get rid of this DVR and our downstairs cable connection in favor of the Apple TV very soon.

Comparing the Apple TV to the TiVo is more interesting, not least because people have been comparing the two companies for years, and keep demanding that Apple build a TiVo killer (both before and after the Apple TV release). Given how badly cable companies stink, it’s hard to believe Apple should embroil themselves in this mess, but they seem to be doing okay with the iPhone, and phone companies aren’t much better than cable companies. People also want Apple TVs to play DVDs, which is an obvious feature, but would be less profitable for Apple than iTunes Store rentals and purchases. But back to the comparisons.

The SA DVR has exactly one advantage over the TiVo (aside from price): its “Ouija board” — when you need to “type” with a very limited keyboard, the TiVo makes it possible but not easy. The SA box improves the experience dimming (and skipping over) invalid letters (which would spell words that don’t match the list of available shows). The Apple TV, interestingly, has an unimpressive on-screen keyboard and a very limited remote (it’s the same one Macs ship with, meaning 6 buttons: 4 directions, play/pause, and menu/back). But it’s easier to use, because the Apple TV doesn’t lag behind user input as much (it doesn’t have to match input against all possible titles, remember), and tactile response is very good; I only made one typo when entering usernames of several friends, and it was easy to correct, even though Delete is an onscreen selection (no Clear key, as on the TiVo remote).

This brings us to another interesting comparison: the SA box has Internet connectivity (I think it’s channel 996 that shows the current IP), but doesn’t use it for anything except the electronic program guide and purchasing pay-per-view (which we don’t do). The TiVo adds TiVo-to-TiVo transfers of shows (we only have one, so haven’t tried it), scheduling via http://www.tivo.com/tco/, an unsupported web server which allows downloading encrypted/watermarked television shows, and the ability to run applications from a server (either at home or across the Internet). Applications allow you to play music or slide shows from a Mac or Windows PC, or slide shows (from your Picasa or Yahoo Pictures account — but not Flickr, even though Yahoo owns Flickr!). Unfortunately, you cannot combine these applications, so it’s impossible to listen to music while watching a slide show on the TiVo. TiVo has apparently dropped support for third-party development.

The Apple TV, on the other hand, does this all much better. Out of the box, it comes with a set of high-quality flower photos, which run as a slide show when idle. Music can be a) played from the Apple TV’s hard drive, b) streamed from iTunes on a Mac or PC (controlled from the Apple TV), or c) streamed from within iTunes in AirTunes+ mode — iTunes sends audio and ID3-style metadata including cover art over the network to the Apple TV. In any of these modes, track information is displayed onscreen, and if the Apple TV is left idle, the it starts showing a slide show (ours is photos of Julia, of course); this doesn’t interfere with music playback at all.

Compared to TiVo’s lousy support for Yahoo Photos (!?!) and Picasa (they want you to create your own account and log into it before downloading any photos), the Apple TV supports Flickr and .Mac photos, as well as the owner’s own via iTunes, of course. There is a clear hierarchy of user experience here: no support on SA/TWC; poor slide shows or mediocre media streaming on TiVo; high-quality music and photos on the Apple TV, pre-loaded with nice photos for a superior out-of-box experience.

One of the few things I regret about the Apple TV is that I bought it from Apple; I didn’t get an educational or corporate discount, so I could have gotten it faster for $15 less from Amazon (via Prime), but when I tried to cancel the order at store.apple.com it had already gone through (less than 5 minutes after pressing Submit). This should be the worst problem I have with the new gadget!

Unfortunately, it wasn’t. The Apple TV would not synchronize content from iTunes; I was able to play music through it (AirTunes), but it mostly refused to show up in iTunes’ DEVICES list. I got a warning about port 3689 possibly being blocked by a firewall, which I initially ignored, knowing I had specifically allowed iTunes to connect through Leopard’s “socket firewall”.

pepper@prowler:~$ grep 3689 /etc/services 
daap            3689/udp    # Digital Audio Access Protocol
daap            3689/tcp    # Digital Audio Access Protocol

The second time I got this message, with iTunes’ Preferences claiming the Apple TV was synching even while it wasn’t fully accessible, I did some searching, and found out that indeed several people needed to open up the socket firewall before Apple TV synching would work. I did this, and lo and behold, our Apple TV now has the proper 12gb of video, 51gb of audio, and 3gb of photos it should. It’s bad that iTunes wasn’t properly whitelisted in the firewall, but it’s much worse that people need to turn off a security feature to make the Apple TV work. Fortunately, after I switched the firewall back to “Set access for specific services and applications” (where it should be), the Apple TV continued to appear and synch properly; bug filed with Apple.

That brings up another bug: we have a Gigabit Ethernet network (3.5 switches — 8-port, a couple 5-port, and the 3-port built into our Time Capsule) and an 802.11n network, but unfortunately the wireless doesn’t work right. At 5GHz, I keep losing my connection; at 2.4GHz it stays up everywhere except the guest room (which has no Ethernet), but speeds throughout the apartment are poor and connectivity is less reliable than our 802.11g Airport Extreme network. Since I haven’t fixed this yet, I much prefer to do large transfers over the wired network.

The Apple TV connects to a running copy of iTunes to download content; in my case, most of the connections (once I got past the firewall issue) were to the AirPort IP address, which prevented them from making progress on the 65gb transfer. I had to disable AirPort to force the Apple TV over to the Ethernet connection, which was much faster; after it was done I re-enabled AirPort, but that’s another bug (also reported, and yes, I do have System Preferences set to prefer Ethernet to AirPort).

I found a couple pages of Outlook keyboard shortcuts (the online help lists shortcuts too):

• http://www.hongkiat.com/blog/80-keyboard-shortcuts-for-microsoft-outlook/
• http://www.rnib.org.uk/xpedio/groups/public/documents/PublicWebsite/public_rnib003594.hcsp

Many of these are standard Windows shortcuts, but a few are useful and news to me.

Eudora stopped working on my home MBP recently, so I’m back to Thunderbird, and it strikes me how similar Thunderbird is to Outlook, even extending to some misfeatures (design flaws, not quite bugs), such as over-using the Esc key. Compared to Eudora, pine, and Apple Mail, Thunderbird is clearly much closer to Outlook. A few things are notable improvements, such as being able to mark messages Read and Unread with the M key, instead of Ctl-Q/Ctl-U, or S to flag messages (stored as an IMAP tag; this shows up in Eudora as Label 15). And with a mailbox selected, Ctl-Q doesn’t mark all its unread messages read, which it should.

In both Outlook and Thunderbird, Esc closes message windows; this is inconsistent with all other full windows, which are closed with Command-W, and makes messages feel particularly ephemeral. In Outlook, when I open a received message and hit the Space bar to scroll to the next page (which works in every other email client and browser I know), it instead inserts spaces at the beginning of the received message, which if course is not what I want.

I cannot find a good way to sort threads by date; I’d like every thread (perhaps every thread with new messages) grouped together, with the messages in each thread sorted internally by date, and the threads sorted by date (typically of the first message). In Outlook I can group “Conversations” by Subject: or group by From: line, but new messages keep showing up at the top of the mailbox, instead of the bottom (where they should sort, by date).

I have figured out more what’s wrong with Refresh. First, I have to hit F5 repeatedly to make Outlook clear more and more read messages from unread-only views; second, collapsed conversations are not cleared; I have to expand them out and then hit F5 again. This is particularly annoying because Outlook has such a strong tendency to always keep one message selected and thus read (although it’s not marked read, so I cannot simply mark it unread; I have to mark it read, then mark it unread, and then make sure Outlook doesn’t preview it again), so it’s quite difficult to reorganize a mailbox and get to a “clean” view (only new messages/threads) without losing some messages which Outlook insisted on selecting/previewing/marking read while rearranging.

And a little attention (not “love”) for IE: I still hit Ctl-L to select the URL for copying, and IE7 still fails to do it, bringing up a blank URL entry dialog, instead of selecting the URL in the current window as Safari & Firefox do. I shouldn’t need the mouse to copy the current URL.

This is odd. I have to use View Options to see full headers (in the Message Options window), but while that’s open, the main Outlook program is visible but completely unresponsive. Message Options is apparently a super-modal dialog, which blocks “other applications”, and the Alt-Tab task switcher doesn’t even show the main Outlook icon. I thought Outlook had crashed, until I realized it was accessible again after I dismissed Message Options.

In fairness to Apple, I’ve seen cases in Leopard where Apple’s Command-Tab task switcher only shows some of the currently running applications (it sorts itself out fairly quickly), but Windows’ super-modal behavior is fscked up, and designed into the application (or the OS!).

After DreamHost’s breach 8 months ago, I was aggravated at their poor handling of the situation, but willing to give them the benefit of the doubt, and still happy with their low prices and flexible services.

With the new bad news and worse confirmation (still with poor incident handling), though, it’s time to get out of dodge.

I have moved Extra Pepperoni back onto my own hardware. I started blogging on Apple’s Blojsom install, but gave up on Tiger Server for Blojsom (and Mailman) because the services kept silently shutting down, leaving me to notice they were disabled days or weeks later (no fault of Blojsom or Mailman — Apple didn’t do a good job porting SpamAssassin either). Bringing up a WordPress blog and mailing lists at DreamHost was easy and cheap, but that’s no good if they are unsafe.

I’ll look at moving a couple very light-duty Mailman lists off DH next, but the lists are so lightly used I’m not too concerned. There just isn’t any confidential information on the mailing lists, aside from their tiny subscriber lists.

Ah, well. I now know much more about WordPress and MySQL than I cared too, but the setup wasn’t too bad. I hadn’t realized how many customizations and tweaks I made to WordPress until it came time to recreate them on my own system:

1. Almost Spring theme (included by DreamHost); with minor hack
2. PHP Markdown Extra; with minor hack
3. MySQL admin UI
4. WP-DB-Backup (DH included one, which I’m no longer using)
5. mod_rewrite for permalinks
6. Admin-SSL, with “Shared SSL” tweak, integrated into my existing SSL site (meaning EP is available through two different “sites”, and I have to keep the Apache configurations reconciled)
7. Twitter
8. WP-Cache (DH standard)
9. Akismet anti-spam registration
10. Technorati pinger (came over automatically with the DB).
11. Fix for widget.php to use legal JavaScript tag.

I’ve been using Windows on a daily basis for 9 days now. At Rockefeller, I kept it on a VM (earlier, on a physical PC) which I could easily wipe and reinstall. I kept the few installers I need on a Mac so I could easily reinstall and be back in business. Now I have to do much more in XP/Outlook, and I have many gripes.

Things I Miss

These are mostly lacks in Windows, although not entirely.

• In the Alt-Tab task switcher, I cannot Hide (Command-H), Quit (Command-Q), or click an application’s icon to switch directly to it. This is aggravated by the fact that icons in the switcher often correspond to windows rather than on the Mac, where they correspond to applications (each with one or more windows), so there are many more icons to Tab through, and often several indistinguishable windows (4 Firefox windows generate 4 identical unlabeled icons; so do 4 open messages in Outlook). Considering Windows has had this feature for longer than Apple, it’s shockingly underpowered.
• I cannot hide the current application (window) from the keyboard (Command-H), or Hide Others (Command-Option-H).
• I miss BBEdit — working with notepad and vi for now; UltraEdit’s installer (recommended on TidBITS-Talk) doesn’t work in my environment. This will get worse as I start writing and editing more (code).
• BBEdit (particularly side-by-side diff and interactive reconciliation, which I should be able to find an alternative for, but sdiff isn’t it).
• bbedit (I miss opening files from the shell, including via sftp and from for loops).
• ssh keys for authentication.
• Seeing my personal email throughout the day — the financial/SEC/Sarbanes-Oxley environment requires a lot more separation between personal and work activities.
• Options (program-wide preferences) is not available from message windows — only from the mailbox viewer window. This is true in Thunderbird/Mac too; presumably copied from the MS model.
• Good filtering: Outlook’s filtering is very much wizard-driven, but not very flexible (no booleans & very limited criteria available); two filters that try to file the same message will put 2 “copies” of the message in different mailboxes; some options are handled on the server, while others are only performed on the client.
• BlackBerry filtering: The BlackBerry Curve shows everything in one mailbox; I’d like to see things grouped as I do on Outlook.

Things that bug me

• I can make Outlook sort a mailbox with the newest messages at the bottom (the default is newest-at-top), but then when I click the mailbox, Outlook selects the bottom (newest) message in a mailbox, whereas I want to read oldest first.
• I can make Outlook allow commas as address delimiters (which is what they are in the actual mail messages), but then I cannot type names, because we have autocomplete disabled and Outlook doesn’t recognize a correct “Last, First” recipient when it’s set to allow commas as delimiters (even though Outlook uses them once I click Check Names).
• Keyboard nickname completion (Command-L in Eudora); I can’t find a way to assign a keyboard shortcut to Check Names.
• I try to keep my mailbox “caught up” or “clean” (all messages read). Outlook doesn’t mark a message read until I deselect it. This means that when I’m done, to have it stop showing that last message as unread, I have to select something else.
• When I’m reading a bunch of new messages, and Outlook selects one I don’t want to read (see above about selecting the bottom/newest message), I have to click another message to get the Mark Unread contextual menu command; then I scroll up to the top and click on the first message.
• AutoCorrect absolutely would not let me type “SAs” (System Administrators) until I killed it.
• MS Office Communicator flashes in the Start bar and the Alt-Tab switcher, but it flashes the main window’s icon, instead of the one for the conversation with new activity. That’s just dumb.

Things I like

• Outlook can show me mailing list (group) membership. It’s called “Outlook Properties” in the menu, despite being maintained on the Exchange side, but after I got over thinking that couldn’t be the right place, this is quite handy.
• Outlook checks group memberships automatically when filtering; this cannot be turned off, so I cannot filter messages sent to a list separately from messages sent to a member of the list.

DreamHost wrote back, and the news isn’t good. Someone sent them a list which is apparently circulating, of username/password pairs for “FTP” accounts; one was mine. I had hoped that if a password leaked it was my old password, which I replaced back in June (on my birthday) when DreamHost told me they got hacked. No joy, though — the password they received was active on Extra Pepperoni (and chrispepper.com) until they sent me mail yesterday; I don’t use it elsewhere and changed it last night, but that means someone had access to EP very recently. It looks like nobody ever used the account, but methinks it’s time to install MySQL and WordPress on www.reppep.com, and probably Mailman too.

Crud on a cracker!

http://www.finjan.com/Pressrelease.aspx?id=1868&PressLan=1819&lan=3

And I still have no idea how they got me.

I got a message from DreamHost tonight which both confused and disturbed me.

Telling me there’s evidence that I have been intruded upon is scary — but what was the evidence?? Without more information, this is upsetting but not helpful.

I only access this account from fully patched Macs under my direct control. None of them were running Windows spyware, and I know there hasn’t been a hardware keylogger in operation on my equipment recently (I don’t believe every, but I’ve been doing lots of work on my equipment lately, so I know not recently). It’s certainly possible I got hacked by some brand-new Mac OS X exploit, but (especially given my understanding of DreamHost’s security model, which entails emailing plaintext passwords at the drop of a hat) I consider it considerably more likely this is a false alarm or miscommunication.

Especially given that, despite “we have reset your password”, the affected account’s password was NOT changed. I logged in normally and changed it myself. This makes me very glad that I created a brand-new password only for DreamHost last time they got hacked. On the other hand, I could have been sniffed logging in over the Internet (most of their access is unprotected); I only set up SSL for administration of Extra Pepperoni a month ago…

We’ll see how they respond to my request for clarification.

In the meantime, I am worried and aggravated.

It’s also somewhat suspicious that the timezone is UTC, considering that DreamHost is in Los Angeles. If it wasn’t the right panel.dreamhost.com hostname, I’d think this was an attempt to get me to submit my DH account information to a spammer, but that information isn’t worth much.

To: “Chris Pepper” <—-> From: DreamHost Support <support@—-> Subject: [reppep ----] Account Concerns… Date: Fri, 7 Mar 2008 02:20:34 +0000 (UTC)

Dear DreamHost customer,

We have found evidence indicating that your ‘reppep’ web server account may have been subject to intrusion by a malicious 3rd party. As a precautionary measure, we have reset your password and ask that you change it, here:

https://panel.dreamhost.com/index.cgi?tab=users&subtab=users& current_step=Index&next_step=Edit&usid=1532237

At this time we have found no evidence to suggest that there has been a breach of our internal security. We believe that the passwords in question were likely obtained through the use of spyware/keyloggers/malware, possibly installed on your personal computer.

In order to secure your account, we ask that you immediately follow the recommendations provided in the DreamHost AbuseCenter - particularly those involving the removal of malware. You may visit the AbuseCenter, here:

http://abuse.dreamhost.com/cracking/#exploits

If you have any questions or concerns, please let us know.

• DreamHost Abuse/Security Team

Apple put a lot of effort into making network sharing (Mac and Windows networking using the AFP & SMB/CIFS protocols) easier in Leopard. One of the things they did was introduce credential caching at the system level, so once you mount another Mac via AppleShare (for instance), you could then connect to it with Screen Sharing too, without authenticating. This is neat, but a bit problematic. I have had cases where:

1. I had to kill NetAuthAgent (the background process that appears to hold username/password pairs on your behalf) to make mounting work
2. I had to rearrange windows around onscreen, because a (stalled) progress window was hiding a username/password window, and never going to get anywhere without some help; other times I have dismissed the progress dialog without realizing it was waiting for a concealed window.
3. I have had to Force Quit and relaunch the Finder before it could (re-)mount some or all network volumes.
4. I have had to reboot the Leopard server before I could (re-)mount its volumes.
5. I have had Leopard systems fail to share out volumes, and had to re-share them manually. Part of this appears to be a different issue, where Leopard systems don’t even mount additional drives until a user logs in (obviously unmounted volumes cannot be mounted over the network). That’s not right!

Tonight’s problem was a bit different — I was connecting to a Windows server running Samba, and not getting the right permissions. When I looked in the server’s /var/log/samba/smbd.log (because I cannot find any way to see the account used for a network mount in in the Finder), I discovered that the share was mounted as the wrong user. I had never gotten the username/password dialog for this mount, as I had (the wrong) user credentials cached in NetAuthAgent.

The Tiger behavior is to default to the client username (the account mounting the share from the server). Leopard instead uses whichever user it has a cached credential for. I have now changed my scripts to always specify the username when mounting shares, e.g., open smb://pepper@inspectore/inspector.

It feels very very strange to be unemployed — it’s been 7 years since the last time, and I was too freaked out at Shooting Gallery laying me off to feel this way. Now that I’m a grown-up (having kid(s) means you’re responsible, even when you’re irresponsible!) it’s a good thing that we’re covered by RU insurance past the start date for GS insurance, but the whole experience is still very odd. I wiped the third computer today at 5:30pm, and am copying data off computer #4 (old reppep.com) right now in preparation for retiring it (it’s falling apart, apparently — optical drive died an hour ago).

Now I just need Apple to update the MBP15s, so I can replace this PowerBook. It’s doing better than I thought, though — doesn’t seem any doubt that it will serve until the next update.

RU IT did right by me today — a grand spread, consisting of John’s pizza, baby back ribs, beef ribs (they looked like something from The Flintstones), and chicken wings. A nice (short) speech by Armand, and well wishes all around. Elaine hung a bunch of colorful signs, which delighted Julia.

I closed out my helpdesk tickets, turned in my keys (forgot to turn in my ID/swipe card, though), and updated the documentation on our load balancers again, as well as re-re-recapping for my co-workers. I had to say “Look, when you feel like you’re an idiot, don’t worry — I felt like that repeatedly for years while working with these. The Big-IPs are absurdly complicated. Two kernels, a super ’switch card’ that’s doing all kinds of crazy (non-switch) stuff, over 20 IP addresses, 8 networks, plenty of bugs, and delays in getting technical support. It’s not you!”

Maybe I’ll have some time to investigate Linux & Windows text editors.

On Feb 19, 2008, I shut down the old reppep.com server, which ran Mac OS X 10.4 “Tiger” Server, and replaced it with a new (cheaper and faster) PC running Linux. Unfortunately, the password formats are incompatible, so I apologize to app reppep users for the disruption.

Please call me if you have an account on reppep.com and haven’t received your password already, or find anything not working right.

I switched from Apple’s jabberd to Openfire, which doesn’t use the UNIX system accounts, so let me know if you want a chat account (compatible with iChat & GTalk).

[Done] I forgot SquirrelMail address books — should be able to bring those over too.

• Firewall problem fixed. SMTP MX issue fixed.
• Virus filtering problem fixed.
• Webmail certificate fixed.
• Quota problem fixed.
• Virtual domains for email fixed.

As of 5pm, I don’t know anything that doesn’t work (aside from SquirrelMail address books) [fixed Thursday].

Thanks for your patience!

As of 10:30 on the 20th, things seem to be working. Something’s screwy with amavisd-new’s quarantine, but mail is going through. I reinstalled Openfire, and chat seems okay under the correct hostname/certificate name now (will try signing it as ca.reppep.com later).

Good timing — the optical drive on the old server died tonight.

I have distributed all the new temporary passwords, so any users having trouble logging in should let me know.

Markdown.cgi is still broken, but I’m the only person who uses it here, so I’ll get to it.

On Thursday the 21st, I found a problem with amavisd-new — it had quarantined 32,000 messages in a single directory, and was stuck (apparently ext3 doesn’t support more than 32,000 files in a directory). I cleared it out and finally managed to disable quarantine, which wasn’t as easy as it should have been, and the backlog of messages have been delivered as of 9:15pm.

At 11pm, I fixed an issue preventing SMTP AUTH from working properly, which was interfering with sending email to non-reppep addresses.

After getting burned too many times, I dropped my .Mac subscription. I never trusted my Apple keychains to iDisk anyway, but this means I have different subsets of passwords on different machines, and no good way to keep them in sync. I thought of a solution for manual sync last week: One keychain per Mac. Say I have 3 systems: work, home, and other. Each system has 3 Apple keychains: work.keychain, home.keychain, and other.keychain, with each host using its own as the default. Then I can rsync work.keychain to home.keychain & other.keychain, etc. This is awkward with rsync because it’s inherently unidirectional, but keychains are small so it’s quite feasible to script.

In Tiger, I know the keychain is actually stored in memory once it’s unlocked, so it’s good to lock (unload) all keychains with “security lock-keychain -a” before updating the files — this goes in the same script. I also set mine to lock after 2 hours of inactivity, or (on those systems where I run SSHKeychain) when sleeping or activating the (locking) screen saver.

For several years, I’ve been saying Apple made a bad choice when they picked Cyrus IMAPd as the POP/IMAP server for Mac OS X Server. It’s a huge and complicated system, encompassing IMAP, POP, SSL, Sieve filtering, LMTP delivery, USENET news, clustering/proxy (Murder), pluggable authentication (SASL), etc. I cannot think of a single company outside Cupertino where it would make sense to run an enterprise mail system on Mac OS X Server, but Apple continues to add these inexplicable high-end features to its mail server, most recently XSan-based email clustering in Leopard Server.

The statement that convinced me (shortly after I had migrated to Cyrus IMAPd on Mac OS X Server 10.4 “Tiger”) that I would never choose to run Cyrus for my personal use, was the following — which I came across again today:

Installation Overview

This system should be expected to have the same order-of-magnitude installation complexity as a netnews system. Maintenance should have similar complexity, except administrators will have to deal with creation and deletion of users and will have the option of managing quotas and access control lists.

USENET news is infamously demanding and bandwidth intensive. It would be wonderful if Apple had taken Cyrus IMAPd, repackaged it (without too many changes!), and put a powerful and simple interface on top. The did this quite successfully with Apache httpd (although Server Admin breaks down on complicated configurations and has obscure bugs). Lots of people use Mac OS X Server to run websites and think it’s easy & simple. Considering the typical reactions of those same people to the httpd .conf files “under the hood”, this is a noteworthy triumph. Similarly, Time Machine provides a reasonable approximation of scheduled snapshots on a high-end NAS for do-it-yourself file recovery, with a simple interface that insulates users from the nitty-gritty of copy-on-write and hard links.

Cyrus did not get as much attention, though. Basically, Apple makes it pretty easy to create email accounts, provides a Repair button for the overall Cyrus database, and provides a Reconstruct button for individual accounts. That’s about it. Unfortunately, Apple doesn’t really document maintenance beyond “press the button and it will fix your problem”. I’ve had several serious database problems which Apple’s Repair button did not help with. Those were bad times.

Similarly, I have had problems where users could not log in, but Workgroup Manager claimed their accounts were usable. I eventually discovered that resetting passwords with passwd works sometimes, and re-setting passwords in Workgroup Manager works consistently, but when I asked Apple about it, the eventual response was basically, “Yes, that’s bad; you should restore your accounts from your recent Open Directory export.” Not a good answer.

It doesn’t help that Apple’s SpamAssassin and ClamAV installations are broken, as these result in more spam and slower deliveries.

So why am I planning to migrate to Cyrus IMAPd on CentOS 5.1? Well, I’d really like to just copy my 5gb mail directory to the new system and have my clients not notice the difference. Eudora doesn’t handle (IMAP) change well — renaming a single IMAP directory can force it to download all messages again, and various other things can cause Eudora to lose date stamps on sent mail, or message state information (when it gets disassociated from the actual message on the IMAP server). If I can make Cyrus work, I’ll be very happy, and if I can’t I’ll try Dovecot (Red Hat’s default) or Courier (which I hear is also good).

Also, I know it can work, and I have a rough model to work from on my Tiger Server, but if I wasn’t using Cyrus already I would stay away from it, as I wish Apple had done.

Update 2008/01/04: I tried again with a bulk (manufactured/pressed, not burned ont a DVD-R/DVD+R) DVD, and it worked fine. In retrospect, it seems likely to be drive deterioration, as I installed several betas from DL DVD+Rs I burned.

This is odd. I have a 1.5GHz 15″ PowerBook G4 (3.5 years old), running Leopard, which I want to reinstall. I have tried booting from two different Leopard DVDs I burned (both DVD+R DL, since I can’t find any DVD-R DL media) from legit Apple ISOs. It won’t boot from either, and often if I insert one of these DVDs while it’s running, the DVD drive chugs a bit and spits the DVD out. Sometimes, however, it reads the DVD — I can run the “Install Mac OS X” app (which just sets the startup disk and reboots), but not boot from disc.

Nothing in the logs.

Hardware Overview:

Model Name: PowerBook G4 15″
Model Identifier: PowerBook5,4
Processor Name: PowerPC G4 (1.1)
Processor Speed: 1.5 GHz
Number Of CPUs: 1
L2 Cache (per CPU): 512 KB
Memory: 1 GB
Bus Speed: 167 MHz
Boot ROM Version: 4.8.6f0
Serial Number: ****

When the disk was mounted, Apple System Profiler showed:

MATSHITA DVD-R UJ-825:

Firmware Revision: DAM5
Interconnect: ATAPI
Burn Support: Yes (Apple Shipping Drive)
Cache: 2048 KB
Reads DVD: Yes
CD-Write: -R, -RW
DVD-Write: -R, -RW, +R, +RW
Write Strategies: CD-TAO, CD-SAO, DVD-DAO
Media:
Type: DVD-ROM
Blank: No
Erasable: No
Overwritable: No
Appendable: No

Over Christmas, I updated Dad’s backup (SuperDuper is great), and upgraded to Leopard. It failed miserably — in exactly the same way as my own first Leopard upgrade failed, although I didn’t know what was going on back then. There wasn’t any documentation about the problem then, but now Apple describes a closely related issue:

Mac OS X 10.5: Unable to log in after an upgrade install

Issue or symptom

You may not be able to log in with a user account that has a password of 8 or more characters and was originally created in Mac OS X 10.2.8 or earlier, after performing an upgrade installation of Mac OS X 10.5 Leopard (the default installation type).

I do indeed use a password longer than 8 characters. At least on my own system, the accounts were not created under or before 10.2.8. On my father’s system, the accounts may date back that far, but his password was not longer, and Apple’s suggested workaround did not work either.

On my own upgrade, I installed Leopard, and was unable to log in with my (known correct) password, or my root password. I booted from DVD and was able to see my home directory, but there was no information on how to fix Leopard accounts (and really not much information on Leopard accounts at all) at that time. Reset Password from DVD didn’t work, and neither did passwd. I reinstalled from scratch and restored my home directory.

For Dad, I didn’t have time to do that, so I created a new account with a different username and real name, and swapped his old home directory with the new (basically empty) one. This took about 5 minutes, compared to several hours spent unsuccessfully trying to fix his old account. Somehow during the upgrade, his account was disabled, and I was unable to re-enable it. I booted from the Leopard DVD, and the Reset Password tool said it reset his password, but did not. I booted into my own admin account, and used passwd, which gave me a Directory Services account disabled error. The only references to that error Google has to that error code are copies of the manual page, which lists the error code but not a way to enable such an account. I even updated to get the Login & Keychain update, but it didn’t help.

In the interim, Apple has documented that Leopard stores accounts as .plist files in /var/db/dslocal/nodes/Default/users/, which is very helpful — it makes it easy to do things like change UIDs, which I need to do periodically. On the other hand, those files point into other places for some information, such as the Kerberos KDC (Key Distribution Center) for actual passwords. I don’t know enough Kerberos to feel comfortable creating an identity for his account, as should have automatically happened during the upgrade (before Leopard, non-Server versions of Mac OS X don’t include a KDC, and they store passwords differently). I considered pointing his account to the KDC identity for a new account with the right password, but this seemed fragile, so I went with the new account, which seems to have worked reasonably well.

FUBAR!

In Leopard, Tab completion in bash doesn’t immediately append trailing slashes to symbolic links that point to directories. When I complained about this change to Apple, I was told it was user configurable, and I should just configure the old behavior. It took me a while to actually find the solution — partially because it isn’t within bash itself, and partially because it was quite a nuisance but not a serious problem.

For example, ~/www is a symlink to /Volumes/www, and I cd to directories below it quite frequently. I’m in the habit of typing “cd w[Tab]/pu[Tab]“, which should expand to “cd www/public_html“. This broke in Leopard — I needed an extra Tab to get the / — otherwise I’d end up with the ugly and non-functional “cd wwwpu“. Anyway, the fix is:

echo "set mark-symlinked-directories on" >> ~/.inputrc

Now Tab completion works the way I want it to again. Thanks, stylishpants!

For over a year now, I’ve been following the development of Mac OS X Server 10.5 Leopard and testing betas, and anticipating upgrading reppep.com from Tiger Server on a dual 1.25GHz Power Mac G4 to Leopard Server on a dual 2GHz Power Mac G5. Over the weekend I had a change of plans, though.

Although I support Mac OS X Server at Rockefeller, I don’t recommend it for most requirements, as Linux compares favorably for transparency (some of the MOSXS internals are unique and poorly documented), server software compatibility (although Macs are quite good here too), and price/features at the low end. A Core Duo Mac mini has plenty of juice to saturate our 768kbps/3mbps DSL circuit, but adding a couple drives more than doubles its price, and Apple’s software RAID is quite broken; Linux software RAID is apparently quite good; I might eventually switch to hardware RAID. An Xserve is a great piece of hardware, but it’s a bit exotic and I can get a fast generic PC cheaper; I don’t want all the high-end features for a box that sits in our apartment.

Additionally, I’ve read perhaps 600 pages of docs on Leopard Server, and had at another 400-1500 yet to go. This is an investment I was finding hard to justify. The migration process is quite complicated, and Apple doesn’t support migrating accounts from a Tiger system to a Leopard system — I don’t want to do an upgrade. I could clone the G4 to the G5 and upgrade it there, but I prefer to handle upgrades as scratch installations with manual migration of applications, so I know exactly what’s been done. A lot of this is masked by upgrade procedures.

As part of this, I’ve decided to invest a bit more time in learning RHEL5 — we have a couple systems at Rockefeller, but not much in production yet, and now seems like a good time to dig in some more.

Fortunately, all the services I’ve been using on reppep.com are available on Linux (and FreeBSD), so aside from another incredibly inconvenient password change cycle (for which it is arguably time anyway), the switch should be largely transparent to reppep.com users, although I still have plenty of research to do.

A brief timeline of reppep.com

1. 1999: I left the National Audubon Society, and bought the Power Mac 7300 with accelerator card I’d been using there. I set it up with LinuxPPC and Apache, and started offering free web hosting to friends & family. LinuxPPC was eventually discontinued.
2. I upgraded from LinuxPPC to Yellow Dog Linux, which was better than LinuxPPC, but had serious flaws.
3. 2001: I was working on a couple remote FreeBSD machines (as admin of the Info-Mac server, and a user on the Apache Software Foundation userhost), and decided to learn more; I bought a cheap Celeron PC and installed FreeBSD 4.3 (IIRC); I upgraded through about v5.1 and a Pentium 4 (giving the Celeron box to the Info-Mac Archive, where it became the Info-Mac server for a while). I learned a lot about FreeBSD and UNIX in general, but eventually realized I was investing more time learning FreeBSD than I could justify. The best thing about FreeBSD is not a technical feature, but rather that the user community is so rich with knowledge. Reading the FreeBSD-STABLE list was amazing, as there was so much depth, freely shared with the community. While running on FreeBSD, I added mail services to the web services I had been offering. Note: Disruptions to personal email service are much worse than problems with personal web service.
4. 2005: It became clear that I needed anti-spam, so I began researching SpamAssassin. While I was figuring out how to build the SMTP sandwich, with a public untrusted Postfix listener on port 25 & 587, and a filter, and then a listener on a high port like 10025 to accept and deliver mail to actual users, I installed a beta of Mac OS X Server 10.4 “Tiger”, which had the whole thing implemented, plus ClamAV as a bonus. I started testing heavily before the release, and switched to MOSXS 10.4 shortly after it was finalized. It’s been very good, but as time has passed, I’ve had more and more problems. In particular, Apple chose to use Cyrus as an IMAP/POP server, and Cyrus is complicated, but Apple ignores the complexity; this can make troubleshooting impossible. The SpamAssassin installation is slightly broken; it’s a bit too old to offer the newer SpamAssassin self-upgrade mechanism. Server Admin is great, but has a bunch of bugs around SSL certificates, some of which destroy the certificates. Blojsom was nice, but Apple’s installation was very unstable; I eventually moved my blog to WordPress hosted externally.
5. 2008: I intend to switch to CentOS 5.1, which is basically a (legal) no-charge clone of Red Hat Enterprise Linux 5.1. This should make future upgrades a bit more straightforward, as I won’t have to deal with Apple’s Open Directory (OpenLDAP); it will also give me a bit more experience with RHEL5, which is a better investment for my time than Leopard Server.

I take a lot of pictures of Julia, and every year we make holiday photo albums (normally from iPhoto); last year we got 6.

I just went through December 2006’s photos, picking 5. Now I have 2,400 that made the initial cut from January through November 2007 to review. There are also 47 Julia took this year to check out.

It’s a big job! The books tend to be a bit longer than the base 20 pages, but we like them.

ARD includes a very handy script called kickstart (/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart), to configure the Remote Desktop agent, which is also what Leopard’s Screen Sharing uses. This is important because Murphy says that you will always first need to connect to a recently installed machine and only then discover the ARD agent is off. With the kickstart agent, you can configure user access to Remote Desktop through an ssh connection, and turn the agent on.

Unfortunately, it never worked for me. I have tried to use kickstart on at least 4 separate occasions (always on Tiger systems), and it never did what I wanted. Tonight, I used it on a 10.5.1 system, and in about 5 minutes I had access (manually tunneled through ssh, no less). It would have been faster if the kickstart command was simple (it’s somewhat involved), or if I wasn’t determined to configure access controls before turning on ARD. It’s easy to configure ARD access via System Preferences:Sharing, but bad practice to enable services without access control configured.

Hoo-rah!

To learn about kickstart, use sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -help. If WordPress won’t let you read that whole line, try copying it into another program. Apple’s Apple Remote Desktop Administrator’s Guide includes some helpful examples.

We also use an UID 0 account, which doesn’t appear in System Preferences:Sharing, so I tend to create the account, set the UID, remember ARD, and curse as I discover I can no longer enable ARD access to that account without restoring the UID — quite a nuisance. Since local accounts are now stored in .plist files, adding our UID 0 account and giving it ARD access should both be much easier now.

I use Parallels Desktop for Mac to run the Action Request System (Remedy) for trouble ticket tracking at work. They have a webapp, but it’s not really usable.

A couple weeks ago, about when I upgraded my work desktop to Leopard, Parallels broke. I couldn’t connect to the Remedy server, or our voicemail system. I don’t really think about Parallels networking, but it’s all virtual so normal troubleshooting is unavailable. Basically there’s a fake DHCP server (or two) inside Parallels for the VMs, and I had very little visibility into why it was doing the wrong thing. I reinstalled Parallels but hadn’t spent much time on it, since I don’t use Remedy heavily.

It turns out I needed to re-set Parallels from Bridged to Shared networking mode, whereby it uses the Mac as a NAT server. The NAT alleviates many of my concerns about running Windows. But how & why did that setting get changed in the first place??