Extra Pepperoni

I just submitted this to Apple’s bug/request tracking system. It’s #1 on my wish list…

Please allow iPhone Safari to open web archives. One of my main apps on the Treo is reading Plucker ebooks, which are basically pre-downloaded partial web sites. I spend 1-2 hours per day doing this on the subway, out of cellular/Wi-Fi footprint.

Since the iPhone already has Safari, it just needs the ability to open Safari web archives, and Safari on Mac/Windows would need the ability to build web archives from CLI or AppleScript. The key options for building ebooks are: link depth (follow 0 or more links before stopping), directory or site not to go outside, and file types to fetch (don’t bother with Java and other unusable types, but for some sites images are just ads). For bonus points, allow the user to specify max width & height, and scale any large images to these dimensions when building the archive.

http://plkr.org/

Doh! Apple reminded me that I requested this January 19th, after the iPhone was preannounced. Here’s hoping…

I give up on iSync. Last week it tanked again, and I cleared out all the Mac bits I could find and restarted from just my PowerBook (as well as I could), but it kept bloating and not finishing. On the third attempt, Conduit Manager started crashing during iSync.

Eventually I wiped my Treo, which is not an acceptable fix for Apple’s data corruption. Now I’m trying to get everything back in sync.

I thought it was going to work, after a half-hour iSync run, but CM crashed again. What a load.

I think I just have to accept that Apple hasn’t solved this problem (despite their claims), and switch to rsync updating my calendar, Address Book, and bookmarks from my PowerBook with rsync. So much for the (fantasy) of being able to update my calendar on home & work Macs.

Crud. Crud. Crud.

Conduit Manager keeps crashing on iSync. Now I’m wiping iCal and reverting. Yuck.

I had to wipe the Palm, but after Reverting iCal to a current backup, Conduit Manager miraculously stopped crashing (after 4 crashes, so it was a surprise).

Apple intends to renew my .Mac membership in 47 days, so that gives me a timeframe to wean myself from it. Murphy says I will do so, and then the iPhone or Leopard will have a great .Mac feature that sucks me back in…

Last week my PowerBook ran out of free space. After cleaning up and rebooting a couple times, I realized it was because ~/Library/Application\ Support/SyncServices/ was bloating again. I cleared it out, re-enabled iSync (deleting this folder disabled the iSync Palm HotSync conduit), and ran out of space. I deleted again & left iSync off, because I didn’t have time to fight with it.

Tonight, I again cleared my .Mac sync data (unregistered all clients) and iSync history, rebooted, did a .Mac sync, and re-enabled Palm synching. Perhaps half an hour later, the Treo 650 just finished synching “for the first time” and SyncServices is 1.4gb. That was a “Fast sync”! No good.

pepper@pepperbook:~$ du -hs Library/Application\ Support/SyncServices/
1.4G    Library/Application Support/SyncServices/

Loading “Apple”
    Conduit “iSync Conduit” version 3.0.0
    Sync type is Fast
iSync Conduit starting 2007/06/26 12:30:55 AM
OK iSync Conduit

.Mac sync is still spinning in the menu bar, and SyncServices is now up to 1.5gb.

I wrote a long article for about SSL/TLS, attempting to explain it to a lay audience. I wrote another piece for admins on how to use CA.pl, which TidBITS didn’t pick up, and a third piece about a couple shell scripts I wrote to help run a Certificate Authority, called cert.command & sign.command. I hope you find them interesting and useful.

• TidBITS: Securing Communications with SSL/TLS: A High-Level Overview

• Full Series (3 parts, with scripts): Securing Communications with SSL/TLS

Time-Warner Cable was here again today. They have called at least half a dozen times since their last visit in May, twice to confirm appointments we hadn’t made, several times during dinner to schedule appointments (the last time he was unable to actually schedule the appointment, though, as his computer wasn’t working), and consistently failed to make promised follow-up calls.

Today they just showed up and buzzed, without the automated appointment-confirmation call yesterday — fortunately we were around. When they arrived, we couldn’t get channels 3, 6, 8, or 10 on either TV.

They ran new wire from Song (next door) over the building and back down, separating us from a neighbor who was apparently on the same coax. When they left, our upstairs (non-HD) TV & TiVo seemed fine, but our downstairs TV was still showing video artifacts and occasionally buzzing loudly, although we were at least getting picture on all channels. Apparently the run is so long they need an amplifier, but didn’t have one. So tomorrow someone should be back to install an amplifier (supposed to take 10 minutes), which should fix our downstairs reception. We’ll see if this also raises the signal:noise threshold, which the phone rep I spoke to said was a hard rule, but both field technicians have said was meaningless.

The difference between 32-bit and 64-bit has come up again recently, this time around Mac OS X 10.5 “Leopard”, which is going to be more 64-bit than Mac OS X 10.4 “Tiger”, but not quite as 64-bit as originally claimed. The difference between 32-bit and 64-bit for processors and operating systems is not simple, but it’s also not too complicated. So here’s my attempt at an explanation.

A 32-bit processor has full support for working with 32-bit values. In the simplest terms, this might be a 32-bit unsigned integer — a number between 0 and 4,294,967,296 (2^32 - 1). They can add, subtract, multiply, divide, and perform various other computations which form the building blocks of modern “computation”, including things that don’t seem to have any connection to mathematics, such as word processing and working with images (or audio, or video).

The predecessors of 16-bit processors worked with 16-bit quantities. Their successors (which include the majority of processors currently sold) are 64-bit capable. They are backwards compatible, meaning they can also perform the same 32-bit operations as their predecessors.

The first advantage of 64-bit processors is that certain operations are faster, because they can be handled directly instead of as multiple smaller operations. Obviously both 32-bit and 64-bit processors are adequate for adding 1 + 1, and there’s no 64-bit advantage there, but what about adding 100,000,000,000 + 400,000,000,000? These are both larger than 4,294,967,296, so this addition would typically require more than one operation (clock cycles) for a 32-bit processor, but a 64-bit processor could do it in one operation.

This would be a substantial win for speed, but the mainstream 32-bit processors (Intel’s Pentium & early Core models, the PowerPC G4 & G5, and AMD’s Athlon & Opteron) all offer a range of 128-bit Single Instruction Multiple Data operations. These “SIMD” instructions cater to the most common “multimedia” operations, such as Photoshop filters, audio encoding/decoding, and video encoding/decoding. For things which are be handled by the SIMD units (MMX/SSE on Intel, AltiVec/Velocity Engine on PowerPC) 64-bit operations aren’t any faster. For things which are not well suited to SIMD but are larger than 32 bits, there is still a benefit from 64-bit processing.

http://en.wikipedia.org/wiki/SIMD

The other major advantage to 64-bit processors is memory addressing. A 32-bit processor can directly address 4 gigabytes of memory. 4gb is pretty good, but some of the Mac/Windows/Linux 32-bit memory “address space” is reserved for special purposes, such as I/O devices (disks, network cards, etc.). This is why Apple claimed early Intel Core MacBooks and MacBook Pros only supported up to 3gb of RAM. They could hold more, but some of it wasn’t usable, and Apple didn’t want to deal with the complaints from people who bought 4gb RAM and were only able to use 3.5gb. Current MacBooks, MacBook Pros, and Mac Pros are all 64-bit clean, and can fully take advantage of 4gb+ of RAM. There are workarounds for large memory addressing on 32-bit processors, but Mac OS X doesn’t support them.

Large memory addressing is particularly important for high-end servers and heavy-duty media workstations, where 4gb+ is quite useful.

One reason there is misinformation about the benefits of 64-bitness is simply that it’s complicated — there are mitigating factors all around. Some benefits are available with a 64-bit processor and OS, but others are unlocked only with supporting chipset (the reason some Core 2 MacBooks and MacBook Pros still didn’t fully support 4gb), and others require application upgrades as well (this is why Carbon APIs not being upgraded to 64-bit is a problem for some people).

http://www.carbondev.com/site/?page=64-bit+Carbon

We were trying to figure out what was wrong with communications between our load balancers and Oracle Application Server this week, and the F5 rep whipped out ssldump. With the SSL key, ssldump can decrypt captured traffic from tcpdump just like Wireshark (Ethereal) does for unencrypted traffic; apparently it can also do live display of SSL traffic like Wireshark, but I haven’t tried.

Good stuff!

John Gruber (Daring Fireball) gave a presentation at the SoHo Apple Store tonight. I might’ve been annoyed it was a repeat of his C4[0] presentation, except I wasn’t at C4 so I hadn’t heard it. The rest of the audience seemed suitably impressed — Apple brought extra chairs, and there were still a bunch of people sitting on the floor.

Afterwards, I tagged along to a yummy Vietnamese restaurant. We left when all the unoccupied chairs had been placed on the tables around us, only to discover a giant (empty) drum of MSG outside the front door. This sparked a brief but lively discussion of whether MSG is in fact as bad for you (us) as people once claimed, with no real resolution.

I liked Gruber’s description of Jonas fuzzing, “I need a hole.”

I just sat with (not quite on) a civil jury this week. I was Alternate #2, which meant I was the same as the other jurors until the very end, but while they deliberated I (and Alternate #1) sat next door; we were the same except in the jury room for final deliberations. Apparently in federal court, alternates participate in deliberation the same as other jurors, which would have made it feel like less of a waste of time.

To my pleasant surprise, the jury selection pool sported an open wireless network and a bunch of PC “terminals” so we could surf while cooling our heels before being empaneled. Unfortunately, apparently nobody in the area knew the PCs’ Windows passwords, so logging out was a big no-no, but I brought a laptop and didn’t really care. The actual jury rooms were not online, but we didn’t spend much time waiting there, except during the actual deliberations.

Fortunately the trial was relatively simple and fast — we started Monday with jury selection, heard a half-day of testimony on Tuesday and again on Wednesday, heard closing arguments and (they) deliberated on Thursday, and heard the verdict read out and got our jury service receipts Thursday. This last part is important, as the receipts keep us from being called back to jury duty for (at least) 8 years — not bad!

In 2000 a music studio was burglarized, and most of the equipment was stolen, including a Harrison MR2 56-channel console. We later heard parts of it were found (apparently in 2003) in boxes, and in a dumpster somewhere. There was a previous lawsuit on this insurance claim in 2006, and there (and perhaps in a negotiated settlement) all issues were settled except the insurance value of this audio recording console. Our only charge as a jury was to determine the value of the console, so the insurance company could reimburse for the loss. It was obvious we were missing a lot of the backstory and case history.

I was amazed by the discrepancies in expert opinions. The plaintiff’s expert claimed the console was worth $75-$85k in 2000, while the defense expert claimed $7,500 as its value. Both claimed (credibly) to be quite familiar with the market for this piece of equipment. I wonder how much of it is markup and attitude — “I can buy one that could be fixed up for $X.” is not the same as “I can put a price tag on this, reading $Y.” The actual purchase price (6 months before the burglary) was $28,000, so we all felt somewhat at sea — no two pieces of information corroborated each other.

This sucks, Beavis!

I had to change the school’s panel password (which will only be changed again in 24 days when I hand over the reins to my successor, who will change it again), plus my personal panel password, 3 shell passwords, 12 list passwords, this blog’s password, and possibly the Analog passswords, and what will I forget?

Crud. Crud. Crud.

DreamHost’s next failure is not telling us what the spam links look like. I can’t read the source of every page on the site, but if they would tell us what to look for, I could grep for the suspect sites quite easily. I actually found them elsewhere (see the Andy Hagan link below) — no thanks to DH, whose screw-up I’m now attempting to fix.

Adding injury to injury, I just got the cheery/goofy montly DreamHost email, with no mention of the hack, even though they must have been dealing with the break-in when the message was sent.

To make things worse, the status page where they promise to post updates on this incident doesn’t even mention it! This is 2 1/2 hours after they sent me email, and they still haven’t come clean in public. On the other hand, oscandy.com had a posting about it a couple days earlier http://www.oscandy.com/hacking/454-dreamhost-hosting-platform-hacked.

From a couple other blogs, this may have shown up a week ago:

• http://www.adamfortuna.com/2007/05/31/you-could-be-hacked-too/
• http://tropicalseo.com/2007/andyhaganscom-hacked-can-you-help-me-out/

To: "Chris Pepper"
From: DreamHost Security Team
Subject: [reppep 11988754] URGENT: FTP Account Security Concerns...
Date: Tue,  5 Jun 2007 18:52:42 -0700 (PDT)

Hello -

This email is regarding a potential security concern related to your  
'reppep' FTP account.

We have detected what appears to be the exploit of a number of  
accounts belonging to DreamHost customers, and it appears that your  
account was one of those affected.

We're still working to determine how this occurred, but it appears  
that a 3rd party found a way to obtain the password information  
associated with approximately 3,500 separate FTP accounts and has  
used that information to append data to the index files of customer  
sites using automated scripts (primarily for search engine  
optimization purposes).

Our records indicate that only roughly 20% of the accounts accessed -  
less than 0.15% of the total accounts that we host - actually had  
any changes made to them. Most accounts were untouched.

We ask that you do the following as soon as possible:

1. Immediately change your FTP password, as well as that of any other  
accounts that may share the same password. We recommend the use of  
passwords containing 8 or more random letters and numbers. You may  
change your FTP password from the web panel ("Users" section, "Manage  
Users" sub-section).

2. Review your hosted accounts/sites and ensure that nothing has been  
uploaded or changed that you did not do yourself. Many of the  
unauthorized logins did not result in changes at all (the intruder  
logged in, obtained a directory listing and quickly logged back out)  
but to be sure you should carefully review the full contents of your  
account.

Again, only about 20% of the exploited accounts showed any  
modifications, and of those the only known changes have been to site  
index documents (ie. 'index.php', 'index.html', etc - though we  
recommend looking for other changes as well).

It appears that the same intruder also attempted to gain direct  
access to our internal customer information database, but this was  
thwarted by protections we have in place to prevent such access.  
Similarly, we have seen no indication that the intruder accessed  
other customer account services such as email or MySQL databases.

In the last 24 hours we have made numerous significant behind-the- 
scenes changes to improve internal security, including the discovery  
and patching to prevent a handful of possible exploits.

We will, of course, continue to investigate the source of this  
particular security breach and keep customers apprised of what we  
find. Once we learn more, we will be sure to post updates as they  
become available to our status weblog:

      http://www.dreamhoststatus.com/

Thank you for your patience. If you have any questions or concerns,  
please let us know.

- DreamHost Security Team

Whoops! I wrote Markdown.cgi so I could easily preview Markdown content in BBEdit, but 1.2 broke this. As a fix, instead of using .markdown for source and .text to see the HTML output, v1.3 goes back to using .text files, and now ?markdown appended to the URL returns the Markdown source.

I considered allowing additional query arguments, but Apple’s sh and expr string matching is quite limited, and I don’t want to make the whole thing any slower or more complicated than necessary. Fortunately, it would be easy to change the ‘magic’ query string. Just change the 'markdown' literal on line 7.

I also moved the downloadable (.txt) script, to make the older versions available, and so I can avoid pointing people to old versions.

http://www.reppep.com/~pepper/code/Markdown.cgi/

My old desktop (PMG5 2×2GHz) is now at home, and I am installing Mac OS X Server 10.5 “Leopard” on it. I’ve done it at least half a dozen times already, sometimes finding bugs and often finding that I want to redo it for one reason or another, typically having to do with rearranging our home network. This is all since the latest beta was seeded, so I haven’t had to reinstall to upgrade the beta, although that will happen as well.

When you boot the Mac OS X Server DVD, in addition to showing the normal graphical installer, it additionally starts sshd and Apple’s ARD (VNC) server. You can log in via ssh as root with the machine serial number (first 8 characters, although the documentation incorrectly says ‘digits’). I haven’t tested non-Server, but for heavy-duty workstation deployments you’d probably want to be cloning images anyway, and Apple provides different tools for that. I did a bunch of seaerching for details on the procedure, but the only thing I found is Apple’s CLI admin guide.

http://images.apple.com/server/pdfs/Command_Line_v10.4_2nd_Ed.pdf

Unfortunately, Apple does not make their Server Admin Tools compatible with the previous version of the OS, so in a single-test-system scenario, we’re stuck with running these nice client-server admin clients on the server. I mostly do it via ARD rather than sit in the “server room” in our basement after the initial install. Better if I can pop the DVD in, reboot, and do everything remotely. Since I’m using a third-party hacked firmware on my Linksys WRT54G which provides static DHCP, I get to skip the step of finding which IP the PMG5 acquired (Apple provides a tool for this, or an nmap ping scan would work if everything else was static) — DHCP automatically assigns the correct IP, which is also listed in /etc/hosts. Now I am starting to work through the CLI installation process, which resembles the documented Tiger Server CLI installation process.

Unfortunately, my notes on the process have to remain private, since Leopard is under NDA.