Extra Pepperoni

Steve Jobs announced that the iPhone 3G would be “Twice as fast. Half the price”.

Ever since Om Malik’s interview with AT&T’s Ralph de la Vega, people have been noticing and commenting on the fact that this ignores the mandatory 2-year AT&T contract, and in fact iPhone 3Gs will generally cost more, thanks to the higher monthly fees.

Unfortunately, Amy doesn’t want my original $400 iPhone — perhaps a friend who can’t justify the 3G charges will buy it for $100.

The $300 16gb iPhone 3G will be worth the money for me — I spend a significant amount of time each day waiting for pages to load, and still take a Treo 650 & Bluetooth GPS unit on driving trips. But I’m disappointed in Apple for choosing a clearly misleading catchphrase for a product which doesn’t need deceit and customer confusion to sell well.

I don’t use SMS much, but I do sometimes, and I don’t want to worry about the astronomical per-message costs, so I like the $5/month flat rate plan. And I certainly want the $30/month unlimited 3G data plan.

Fortunately, I’m now able to drop back from the 900-minute/month plan I upgraded to, down to the base 450-minute/month plan, which will save $20/month, and nicely offset the additional $15/month for unlimited 3G & 200 SMS.

Now that AppleCare has failed me, and the iPhone isn’t as much of a hardware investment, and I don’t walk outdoors across campus (drops on carpet are much less destructive than on asphalt or concrete outside), I’ve decided not to purchase AppleCare or a case for my new iPhone (my 11-month-old plastic incase protector is falling apart, and kept the iPhone from fitting in any dock). I like the idea of leaving the iPhone charging in its dock overnight, rather than lying on a night table.

So with the new iPhone, I’ll save $70 on a 2-year AppleCare contract and $30 for the case. This is enough to pay for MobileMe service. Hopefully it will be solid, as opposed to the current .Mac service, based on the unreliable iSync.

I wonder how much turn-by-turn GPS with spoken directions will cost on the iPhone. I know TomTom and Garmin are quite interested, and Google Maps can do real-time driving GPS without spoken directions — I don’t know what the iPhone options will be, though.

I have a couple large questions. First, how well will MobileMe work? Second, how much will turn-by-turn GPS with spoken directions cost? Hardware GPS units are in the several-hundred-dollar range, while Google offers free or cheap GPS with directions but no speech. I’m looking forward to seeing what is available using the iPhone SDK.

I’ll have a much faster iPhone (and probably OpenSSH — hooray!), and next time an attractive upgrade rolls around, I won’t have $500 invested in the previous generation.

To sum up, I’ll save $20/m on extra minutes, and pay an additional $15/m on 3G data & SMS. I’ll save $100 on protection, and pay $100/year on MobileMe. If things don’t change over the next 2 years, I’ll end up paying $40 more, which isn’t bad, but also isn’t “Half the price.”

When Amy mentioned to Joyce (of Scarce) that she now has a blog, Joyce was amazed and impressed at how cutting-edge Amy is. There’s definitely a geographical factor here, because at my picnic earlier the same day, we figured out that of the 6 adults and Julia present (all Brooklynites), every single one of us has a blog.

Devjani’s is firewalled. Julia’s Journal runs on hand-crafted HTML rather than blogging software, but that’s because it dates back to mid-2002; I will move it over at some point. Sharon has two. In addition to Extra Pep, I edit Securosis.

• http://www.mscrochety.com/
• http://ceolaf.blogspot.com/
• http://cmss1971.wordpress.com/
• http://typewrittenteacup.blogspot.com/
• http://twhos.blogspot.com/

Being the compulsive sort, it bugged me whenever I missed an episode of a TV show I watched (I used to watch a lot of TV; now not much). Similarly, it bothered me that I didn’t have complete sets of the comics I read — they were both hard to find and expensive, especially since I almost never started at the beginning.

Inspired by Ernie Cline, I’ve recently been watching Airwolf. It hasn’t aged well, and was never great storytelling, but it’s still enjoyable. And it’s nice to see as a coherent whole over weeks, rather than scattered across years with commercial interruptions. I’m in the middle of season 2, and will skip season 4 (I don’t think I ever saw it, fortunately); don’t know about season 3. Perhaps I’ll watch The Fall Guy next!

Nowadays, with the Internet, back issues of comic books are pretty easy to find. I’ve completed a few series that were missing issues, such as Badger crossovers, Dynamo Joe, and Tailgunner Jo. I’d love to collect various other series, but a full run of X-Men would be prohibitive — both in terms of money and time to read them all!

I was pleased to discover Marvel made several of their more popular titles available to GIT, who released them on DVD. Unfortunately, the license was terminated in favor of Marvel’s online service, but some DVDs are still available. James gave me Ghost Rider for my birthday, and despite some aggravations (they photographed the open comic books, so there’s dead space around the corners, and didn’t bother to split left & right pages, so it’s too awkward to read in single-page portrait mode) which make the comic harder to read than it should be, I’m enjoying the old Ghost Rider issues. It’s amazing what a loser Johnny Blaze originally was — he’s an idiot (sloppy writing), a coward, a regretful devil dealer, and not really faster or more skillful than gang members. As time has gone on, and Marvel has super-sized its characters, Ghost Rider and his cycle have gotten faster, stronger, less human, and ironically much more innocent.

At Wheaton, I helped found the Progressive Alliance, a student political club. I don’t remember most of the members (in fact I no longer recall the names of most of my classmates), but Kirsten Cappy was one of the heads — one of two co-presidents, if I recall correctly — and Steve Amster (a good friend to both of us) got me involved.

As the nerdiest Progressive, I ended up laying out The Serious Shit in PageMaker. Articles were of course always late, so I remember having to shorten articles I’d just stretched out to fill space, in order to fit post-deadline content onto the page (issues were one to two pages, letter or legal sized).

The Shit was posted on the bathroom stall doors, where we had a guaranteed audience with time to read. I don’t recall much more about it, although if Jason Snell revives my old 210mb hard drive, I might get some old issues back — unless they’re on my 6 even older 44mb SyQuest cartridges.

The other thing I recall about TPA & TSS is that my mother convinced me that if I listed “Progressive Alliance” as an activity on my resume, people would decide I was a Communist and not hire me. I don’t remember if I took her suggestion and called it “The Humanist Alliance”, or simply left it out entirely. There was never any question of listing The Serious Shit on the resume — I never interviewed for a job where that would have been a plus.

Fortunately, after my first job at Rockefeller University, I had more relevant things to put on my resume, so the Progressive Alliance dilemma quickly became a non-issue.

We got a Razor for Amy and me, so we can scoot with Julia. It’s fun, but apparently for robust kids, as the handlebars are too low for a grownup, but it’s rated up to 180 pounds. The handlebars bug me, though. They have a label which reads:

Caution: this moves when used. Exercise caution & common sense when riding.

Okay, I understand that a liability lawyer made them put that label on the scooter. But anyone who needs to be told that a Razor scooter moves, is going to be unable to exercise common sense when riding.

Joyce (Raskin) White is a friend of ours from the neighborhood — Julia and her daughter Sydney are a couple days apart in age and were best friends when they lived in Brooklyn. A few years ago Joyce, Matt, and Sydney moved to Boston, and we were all sad. Before Brooklyn, Joyce was in a fairly successful rock band named Scarce, but they broke up after a brain injury took Chick Graining (lead singer) out of commission.

A couple years ago, Joyce started writing a book about her experiences growing up as a female rocker, called Aching to Be: A Girl’s True Rock and Roll Story. Amy edited the book, and we’ve been waiting to see Scarce perform ever since.

Tonight they played at Union Hall, just down the street, and we finally got to watch Joyce rock out. It was most excellent, and I got a mess of pictures.

I was quite impressed by Joel’s description of the hiring process, and we’ve been doing a lot of interviewing for System Admins lately. I put together a list of standard questions to ask during interviews, which has been quite helpful in judging a) how much technical knowledge people have, and b) (just as important) how good a match they are for the skills void we were trying to fill at the time. Here they are, for the next person who needs to perform a similar exercise.

1. How many systems does your team manage (Linux, Solaris, Windows, etc.)?
2. How large is your team?
3. Which OS are you most comfortable/familiar with?
4. Which Linux flavors are you most comfortable/familiar with?
5. Which Red Hat versions are you familiar with?
6. Are you familiar with kernel programming or configuration?
7. Have you done any custom packaging or kickstarting?
8. Have you used or managed Sun JumpStart?
9. How much experience do you have with Sendmail?
10. … NetWorker? Version? Managing backups, or just configuring clients?
11. … LDAP? Brand & version? LDIF or just querying?
12. … firewalls (iptables, ipf, etc.)?
13. … network administration (Cisco, sniffing, etc.)?
14. … Apache httpd?
15. … Tomcat & Java?
16. … EMC (Clariion, PowerPath)?
17. … shell scripting, and with which shells?
18. … perl scripting?
19. … Veritas VM/FS? Versions?
20. … Veritas Cluster, or other HA? Versions?
21. … snapshots? In which products?
22. … load balancing
23. … Oracle (as SA, not DBA)?
24. … HPC?
25. Please briefly explain the difference between RAID 1 and 5. What are layered RAID levels, and when are they appropriate?
26. What sizable projects have you done recently?
27. Why are you leaving your current employer / did you leave your last employer?
28. Please give specific examples of some routine tasks you’ve performed recently.
29. Have you done systems specification and design (servers, multi-server configurations)?
30. Have you worked with customers directly, or primarily with/for other IT personnel?

It didn’t make sense to publish a list of questions when I was involved in the interviewing process, but now that I’m leaving Rockefeller and no longer interviewing UNIX Admins for them, I can post my sample questions.

The Pretties

Inspired by When data center cabling becomes art from Andrew T Laurence & Chuck Goolsbee’s pics of Digital Forest, I took some photos of Rockefeller’s new data center. We’ve been planning out various scenarios for 5 years at this point, but we finally moved most of our systems in this month. Note that the network guys (mostly Eric) took care to run cables connecting to ports on the left half of each device in from the left, and come in from the right for ports on the right. This makes more work for them in preparation, since one cannot simply plug a cable into a free port, but makes things look prettier, and also reduces cable snarling.

More Connectivity, Please

Since we first started discussing data center plans, I’ve been saying we need more connectivity. The new DC has 48 patches per 42U rack, and some of the new racks are indeed running out of ports before they run out of vertical space. In our racks 2U is used for patch panels and 2 cables control APC managed power strips, so we have 40U and 46 patch ports for servers. Our Linux servers have Ethernet, serial console, & KVM; Suns have Ethernet & console; Windows have Ethernet & KVM. In the worst case, 40 1U Linux servers need 120 connections, but we only have 46 available. If the rack is full of 2U Suns & Windows servers, we’re okay with 6 ‘extra’, available for dual-connected servers or whatever. As we get more dense, we begin to run out of ports.

Blades

Blades are no better — their chassis tend to blow out the power budget because they’re even more dense than 1Us (although they do get more servers per rack), and with all the redundancy they still require a lot of cabling. For a reasonable IBM BladeCenter, we need 4 x 2 for GE switches (FC cables don’t go in these patch panels). Then 2 x 2 for (Ethernet & KVM) for management modules per chassis = 12 ports for 7U. For our new HP c7000 chassis with basic networking, we have 16 GE ports, 2 GE console ports, 2 OA Ethernet ports, and 2 2 OA serial ports (again, ignoring the fiber-optic GE ports): 22 ports in 10U. I’m sure somewhere HP has demo chassis, filled them with fully-connected GE switch modules: (9 x 8 + 4 = 74 patches) & (4 x 8 = 32 fiber-optic ports) = 106 cables total (not counting power connections — 6 in our case). In 10U — 1/4 of a rack — insane!

Update 2008/2/5: Eric pointed out I was wrong about the ports — the Cisco switches have 8 uplink ports, 4 of which are either fiber-optic or copper (you can see they’re 17-20 in the photo); the other 4 copper ports seem intended for cross-linking to the other switch. So the max copper patch count remains, but the the fiber connections would be instead, rather than in addition, and we may fully connect our 2 switches with only 8 GE uplinks rather than 16 going out of the chassis.

Glenn hung up his Tweets a few days ago, which makes this an apropos time to ponder Twitter and whether it’s worthwhile.

• Micro-blogging.
• Super chat status line messaging.
• Perhaps the easiest way to flex your vanity, Web 2.0 style.
• A chat room without walls.
• A remarkably uninformative way to exercise your vanity, since one has no idea how many people might read a tweet.
• A particularly 21st-century game of one-upsmanship — # followers vs. # following.
• A pleasant diversion / horrendous distraction.
• An excellent way to broadcast information, including emergency notifications, although its lack of pervasiveness limits what it’s good for.
• Access to lazyweb.

I went out tonight to see Gavin perform (accordion, piano, & backing vocals) with Anna Copa Cabanna at Jalopy, a performance space and instrument repair shop so old-timey and rustic they had a wooden bolt to lock the bathroom door and a wooden box sink.

Getting there was surprisingly difficult. I called a local car service, and they told me it would be 10 minutes. In 6, the car was outside; when I went out, she complained that I’d kept a woman waiting, and spent much of the drive complaining about the car we were in. Then she drove 7 blocks south to 9th Street, and asked me how to get there. Since the driver didn’t know the way, I used Google Maps on the iPhone to give her directions. As we got to the Fort Hamilton Parkway, she began to tell me that Columbia Street (which Jalopy is on) didn’t exist on the other (north) side of the Parkway; I insisted at least half a dozen times, and guided her around the entrance — insisting to me all along that we couldn’t get to the north side of the Parkway, until we saw the place, right where Google claimed it was. To get home I called Eastern, and they were 30% cheaper without arguing or needing directions.

I got some very dark pictures (no flash).

It was a very good show, although as a friend of Gavin’s I didn’t really appreciate the harassment of “Smitty” that’s part of their shtick. On the other hand, I wouldn’t have appreciated him treating her that way either…

Lots of strange songs about New York and America, by a fascinating and deliberately somewhat crazy outsider (Australian import). I was particularly impressed that they managed to do “Beauty Bar” as a real punk song — on tambourine & accordion.

I also enjoyed Royal Pine, but was a bit freaked out when they played a song I recognized — “Pearl Polly Adler”. This is inexplicable because I listen to perhaps a couple dozen songs I don’t know each year (largely from James), so recognizing a ‘new’ song that’s not in my iTunes library was a (pleasant) source of shock and confusion. Apparently I found it while surfing YouTube recently — something I also very rarely do, as opposed to effectively never before the iPhone. The Pearl Polly Adler video looks familiar, but I can’t be sure it’s not all delusional deja vu.

Anyway, I enjoyed that song while racking my brain for where I had heard it before, and “Stone Cold Mamacita” even more, although I enjoyed the rougher and tougher live version more than the recording on Huasteca.

I take a lot of pictures of Julia, and every year we make holiday photo albums (normally from iPhoto); last year we got 6.

I just went through December 2006’s photos, picking 5. Now I have 2,400 that made the initial cut from January through November 2007 to review. There are also 47 Julia took this year to check out.

It’s a big job! The books tend to be a bit longer than the base 20 pages, but we like them.

I was walking down the street this morning, burning a piece of paper with some old passwords on it, and holding the box of matches I had used to light it. A woman saw me, and said “Hi. Gimme a match?” I got out a match and prepared to light it for her. Before I could strike flame, the woman leaned over to my burning password paper and lit a cigarette from it, then said “Thank you.”

There I was, standing on the street, thinking “Smoking’s bad, mm-kay,” and wondering why she asked for a match when she wanted a light (yes, I know, I cannot turn off being an editor), and thinking this was probably actually not the first time someone’s lit a cigarette from a burning password, but it’s still unusual.

Rockefeller University, where I work, was one of the original UNIX sites. In 1975, Mel Ferentz held what was apparently the second UNIX users group meeting (it is not clear if he was one of the organizers of the first meeting, in 1974). Mel went on to build USENIX out of those meetings. He moved on to Rockefeller University soon after those first meetings; just before I started at RU Computing Services, Mel stepped down as Director of RUCS, and moved on to develop Internet2 at NYSERNET.

Last week, Mark Kowitz left RU IT (RUCS after a name change), where he had worked for 23 years. Mark met his wife, Robin, in RUCS over 20 years ago. I met Amy there too, when I started in 1992 (I left in 1995, and Amy left in 1996; I came back; she has not). While cleaning out his papers, Mark found some old documentation on booting UNIX on the PDP-11/70, VAX 11/750, and VAX 11/780, and passed it along to me. Mark doesn’t remember whether he or Mel wrote the documentation, but it is visibly classic UNIX documentation (distinctive fonts and layout).

You can see some more about booting PDP-11 UNIX (in emulation) at Ancient UNIX, 8bitsunplugged.org Digital Archeology, and Amit Singh’s GBA UNIX.

To give you some idea of how much water there has since been under this particular bridge, UNIX was first developed on a DEC PDP-7 in 1969. Digital Equipment Corporation was basically bought by Compaq, which itself was later acquired by HP. This version of UNIX contains Western Electric license statement; UNIX was createdat Bell Labs, which was jointly owned by Western Electric and AT&T. Bell Labs was later absorbed into AT&T, spun out as part of Lucent, and merged with Alcatel to become part of Alcatel-Lucent.

AT&T split off UNIX into UNIX System Labs, which was later bought by Novell. Novell sold much of the UNIX business to Santa Cruz Operation, which sold its UNIX rights and the “SCO” name to Caldera. SCO changed its name to Tarantella and Caldera transformed itself from a Linux company into a UNIX company named “SCO Group”. Alas, Caldera didn’t make money either way, and eventually sued the world — IBM, Novell, various of its own customers, etc.

Along the way, several BSDs were created to provide an alternative to AT&T’s UNIX, later providing a family of excellent UNIX-based operating systems (including the core of Mac OS X). In contrast, Linux was launched in 1991 by Linux Torvalds, born in 1969, the same year as UNIX.

Those little pages are quite a time capsule!

Another paper, by Dennis Richie: http://cm.bell-labs.com/cm/cs/who/dmr/cacm.html.

Just finished Harry Potter (thanks, James!). We’re looking forward to reading them to Julia in a few years.

The Super-Tent/IT Pavilion/Big Top/Big House fronts on the main RU parking lot, at the other end of which is the 66th St Gate. Except that after we moved in, they walled in the lot and started digging:

They still haven’t started on Smith Hall, though, which makes me wonder why we couldn’t still be in a proper building now. In the meantime, the main campus entrance and driveway are closed, along with the parking lot, under which a new electrical vault will be built. Getting around campus is much more complicated now than 6 months ago. This is especially true for IT, moving equipment around the tent, as the pathways and steps around the periphery don’t quite work for carts.

Our new main data center is nearing completion. It was previously our backup/disaster recovery site, so needed a lot of build-out to fit the rest of our servers. The swap from the older/smaller UPS system to the newer/larger one will be tricky, as several live servers will be switched over while running. Later we get to swap systems end-for-end across campus, so the primaries are in the primary DC, once their current location becomes the DR site. Needless to say, most of our systems are not redundant, so there will be a bunch of minor disruptions.

Stu, who is overseeing the build-out of what will largely be ‘his’ DC, showed me why Cat6A cabling is so much thicker (and thus harder to work with) than old-school Cat5 UTP (”Unshielded Twisted Pair”) — the internal copper wiring is twisted around itself many more times to reduce interference, and the whole thing is cradled by a plastic framework shaped like a plus sign. This framework is twisted as well, so as the Cat6A cables lay next to each other in cable trays, the individual conductor strands don’t align with neighboring Cat6A cables, again helping to avoid signal transference between what should be independent connections. The idea is that in 10 years, when everybody is demanding 10GE connections, we’ll be able to simply re-patch uplinks into 10GE switch ports as needed. Otherwise the rewiring would be painful for individual machines, and impossibly disruptive to do in bulk.

Unfortunately, the heavier-duty Cat6A is also heavier and bulkier, thus significantly harder to work with and slower to run. Each of the 24 new 42U racks is getting 48 runs, from 2 1U patch panels in each rack, back to 6 patch panels (96 connections) in each of the new network racks, where switches and other Cat5-based gear, such as terminal servers and KVM switches, will go. This is new 1,152 runs in addition to the slightly older stuff at the South end of the room, which is still our DR site during this construction.

My question is: How long will it be before we need more than 48 connections in a rack? Our non-blade Linux servers tend to have 3 Cat5 connections: Ethernet, serial console, and KVM; Windows systems don’t need serial consoles, so they get 2. A rack of 1U Linux servers maxes out at 40 1U servers and 120 Cat5 connections, which just won’t fly here. 8 2U Linux servers (24 connections) and 12 Windows servers (another 24 connections) fill a rack, meaning as time goes on and we are again someday tight for space, we might run out of network connections sooner. At that point we could put a KVM server in every third rack and reclaim a lot of cabling for Ethernet, but it violates our model of having everything run patched to the switch racks. We’ll see what the world looks like when we actually get there…

I discovered yesterday that they’re also simultaneously digging up the driveway between Founders Hall and Flexner — not sure why, but it looks like pipe-laying for plumbing.

Update According to Stu, this is actually conduit for electrical wiring, from the vault under our parking lot up through to an electrical switching station in Flexner.

Many more RU photographs are up at http://www.reppep.com/~pepper/album/ru/

John Gruber (Daring Fireball) gave a presentation at the SoHo Apple Store tonight. I might’ve been annoyed it was a repeat of his C4[0] presentation, except I wasn’t at C4 so I hadn’t heard it. The rest of the audience seemed suitably impressed — Apple brought extra chairs, and there were still a bunch of people sitting on the floor.

Afterwards, I tagged along to a yummy Vietnamese restaurant. We left when all the unoccupied chairs had been placed on the tables around us, only to discover a giant (empty) drum of MSG outside the front door. This sparked a brief but lively discussion of whether MSG is in fact as bad for you (us) as people once claimed, with no real resolution.

I liked Gruber’s description of Jonas fuzzing, “I need a hole.”

Rich has been told to stop blogging about techonology. This is a shame, as he had worthwhile things to say.

Having very little information on what happened, I have to assume it’s a blanket policy intended to protect Gartner’s intellectual property, by reducing competition from non-Gartner IP (such as public blogs). I wonder how bad the backlash will be. This is aside from the fact that Rich was a) careful not to post Gartner content and b) not shy about mentioning what you could get if you were a Gartner client.

It’s a pity.

http://securosis.com/2007/01/16/securosis-will-no-longer-cover-technology/

I’m writing online documentation for a site that has substantial left-side nav in a sidebar: http://www.xowave.com/. We want to be able to hide the sidebar, perhaps to build a smaller tarball of the docs, or to make more room for large images on smaller screens, or to save space on crowded pages.

Bjorn (the developer) made the sidebar conditional a few days ago, and I just enhanced it this morning to provide a user-accessible knob (in footer.incl , so on available on every normal page) to flip it on and off. Additionally, with “wget --user-agent=printme“, we can whack the whole site without nav. I don’t actually want to do this, but it’s a nice feature.

I actually tested with something like “curl --user-agent printme URL | grep -i agent“, using the URL of a special test.shtml page that basically just contained <!--#printenv -->. It was very handy for figuring out what the server thought of my requests.

footer.incl contains this snippet:

<!--#if expr="$QUERY_STRING = printme" -->
    <a href="<!--#echo var="SCRIPT_URI" -->">Restore navigation sidebar</a>
<!--#else -->
    <a href="<!--#echo var="SCRIPT_URI" -->?printme">Hide navigation sidebar</a>
<!--#endif -->

And head.incl wraps the sidebar code in:

<!--#if expr="$sidebar = hidden || $QUERY_STRING = printme || $HTTP_USER_AGENT = printme"-->