Extra Pepperoni

To content | To menu | To search

Sunday, July 10 2011

4th St NiteOwls, July 1 2011, Barbes

The NiteOwls played Barbes. Unfortunately I missed the beginning.

  • Stayin' Alive
  • It's No Fun
  • Crosspatch
  • When Day Is Done
  • Crazy about My Baby and My Baby Is Crazy about Me
  • Viper
  • The Porter's Love Song to a Chambermaid
  • Wish I Were Twins
  • Two Sleepy People
  • Busy Bee

Saturday, June 18 2011

Mermaid Parade 2011

Bjorn and I met up at the Mermaid Parade this year. We met a couple of Bjorn's friends, who met some more of their friends, who met and made more friends -- it was a social snowball. Mariah & Danielle looked great, so people kept stopping them to take their picture, or to have pictures taken with them -- including the Sea Rabbit and its human creator.

The whole thing felt very much like part of a Brooklyn continuum. We saw crazy colorful animal dude again -- Bjorn & Mariah see him around Manhattan, and I saw both last week and last year at Brooklyn Pride. And the lime green Charger reappeared the next day on 7th Ave, in the 7th Avenue Street Fair.

I took 941 photos (a personal record) and whittled them down to 281 at Flickr.

Tuesday, March 29 2011

AT&T iPad 3G details

We will be taking a couple short trips this year, and now that I have a cellular capable iPad it's time to figure out how I'll have Internet access. TidBITS has covered the 'new' AT&T Personal Hotspot feature on iPhones, and the trade-offs between iPad 3G service vs. connecting through an iPhone hotspot). I don't travel much, so I prefer AT&T's proratable service over Verizon's monthly service. I got the AT&T 3G iPad, rather than the Verizon 3G model or the WiFi-only model.

iPhone: Wireless Features

I thought I would link my iPad to our AT&T family plan, so I could start and stop service with myAT&T.app, but normal iPhone service (including our family plan) is under AT&T's 'postpaid' model, where each month they bill us for minutes/texts/features actually used. The prorated 3G service is prepaid, and doesn't appear to link up with iPhone service or be manageable through myAT&T.app.

Our first trip will be a long weekend with family. In Settings.app, under Cellular Data, I enter my billing info and sign up for the 250mb/30-day plan for $14.99. Hopefully that isn't prorated out to 8.33mbytes/day! The second trip will be over 8 days so I'll sign up for the $25 2gb/30-day plan. Signing up automatically activates autorenewal, so each time I will stop autorenewal after I sign up, and terminate service manually when we get back home. Apparently you can register for AT&T 3G service even without an active contract or WiFi service -- even without a contract the iPad has access to an AT&T (or Verizon) captive network where you can just sign up -- but I don't intend to test this.

iPad: Cellular Data Account

Supposedly http://att.com/ipadlanding provides a web interface to manage iPad 3G plans, but it won't accept my (postpaid iPhone) credentials, and I don't see any reason to register again.

Had I chosen to use Personal Hotspot on the iPhone instead of iPad 3G service, I could have changed from my current "DataPro 2 GB for iPhone" service to "DataPro 4GB for iPhone" (which enables Personal Hotspot) directly in myAT&T.app. Alternatively, an AT&T rep offered to schedule my changes to 4gb/tethering and back to 2gb (no tethering) on whatever dates I prefer, over the phone.

Thursday, March 17 2011

Robin Aigner in Boozin' & Bruisin' at Jalopy

I went to Jalopy again to see Robin Aigner. Unfortunately her site had the wrong time, so I showed up a couple hours early, came back home, and went back again! The show was a benefit for the Music Maker Relief Fund, put on by The Whistlin' Wolves (who joined most of the performances, along with David Bennet Cohen on piano).

It was funny to hear Old Plank Road performed by someone other than The Magpie, although I heard Benjoya mentioned a few times. I sat next to Sheriff Uncle Bob and recognize he was dressed for the evening from his sheriff's star (and tie with more stars), but didn't realize until he went onstage that he was a performer.

I took some photos.


Thursday, March 3 2011

Parenting in the Age of the Internet

Chuck Joiner ran a panel discussion at Macworld on Parenting in the Age of the Internet -- I heard about it through Tonya, one of the panelists. They had a bunch of suggestions, several new to me.

  • Keep hard rules as few and flexible as possible. You cannot expect a child to abide by a multi-page contract which doesn't make sense to them.
  • Recognize the same behaviors in yourself.
    • Model how to use computers and the Internet safely.
    • Be aware of how much time you spend using technology instead of focused on family. This is more of an issue with older kids, but young kids are already learning about engagement and disengagement. Several panelists commented on this, and it resonated for me.
  • Pay attention to game ratings, but try the games yourself first. I am pretty good about this, although as Julia gets more engaged in more complicated games which I am uninterested in, it will be harder.
  • Limit computer/game time (we also limit TV time).
  • Use parental controls.
    • The ones built into Mac OS X are pretty good, although not very deep.
    • The ones built into Windows Live are apparently more flexible.
    • Unfortunately, when Julia was using an Ubuntu netbook, I discovered that the Linux controls are lacking and poor. Partially this is due to a philosophical issue -- Linux hackers don't want to restrict anyone's freedom. They make the valid argument that parents should supervise their kids, but failed to consider that we have other things to do, and would like parental controls to help enforce policies, so kids can use computers without parents constantly watching over their shoulders.
    • You can get much more sophisticated controls over web surfing with a (Linux) router/filter. We don't need this yet, although we might someday. Network filters also work for devices (such as iPads & smartphones) without built-in restrictions.
    • OpenDNS provides a very easy-to-use filter that blocks a lot of crud. I hadn't thought about OpenDNS for this purpose, but it's an excellent idea.
    • Blocking should not be a hard stop. Instead make it an opportunity to discuss the rules and the site that was blocked -- perhaps there is a good reason to visit it. This makes a lot of sense.
  • Email is a very useful microcosm for the Internet. It's easy to keep track of and to limit to a small trusted subset of correspondents, and provides natural & strong motivation for typing and other computer skills.
    • Gmail offers delegation. We use another service with full parental/administrative access.
  • There was considerable discussion of how kids SMS incessantly -- often to the exclusion of making voice calls. Julia does not text yet but does like talking on the phone, so this isn't a problem for us yet.
  • One panelist required their children to accept their Facebook friend requests and provide their passwords. I am uncomfortable with password sharing but it does make sense here. Requiring young kids to accept parental friend requests is an obvious prerequisite for using Facebook. Of course, a clever kid can use custom controls to limit what their parents see even if they're friends.
  • The panelists didn't talk much about privacy.
    • We're clear that Julia doesn't have any privacy from us on the Internet. Now this is easy, but it will bother her as she grows up, and we don't know where independence begins -- we'll have to figure it out as we go.
    • The panelists didn't mention this, but explaining this lack of privacy is essential -- spying on kids who expect privacy destroys trust and makes it a battle between kids and parents. And there's no way to win this battle over the long term anyway.
  • A couple people talked about the importance of computer placement & access. This is why Julia doesn't have an old iPhone (no way to keep track of usage), and why the laptops & iPad she uses stay in common areas of our apartment -- she uses the iPad where we ca monitor her.
  • Don't forget parental/family/sibling privacy -- kids need to consider who else is affected by their posts & sharing.
  • Tonya pointed out that some kids who grow up without TV find that a problem for socialization. We had not encountered or considered this.
  • You cannot expect children not to make mistakes, or to make the same decisions they (or you) would as adults. Instead try to make them aware of the dangers, and if they don't care about the real-life consequences (such as Facebook profile review & Googling as part of job hiring processes), you may have to impose more immediate consequences (family rules & punishments) which are sufficiently real to have an impact on behavior.

Saturday, February 5 2011

Freddy's Bar, Grand Reopening, February 2011

Freddy's Bar has reopened in a new location. The new place is larger and nicer, although they still need to sort some things out -- staff kept walking through the stage during the performances. To celebrate the occasion, Freddy's hosted 3 bands on opening night: The Magpie, Brute Force, and Les Sans Culottes. Josh and I were there to see Gavin play in both The Magpie and Les Sans Culottes.

Many folks were snapping away with DSLRs and phones, and one fellow had a serious looking video camera, on a tripod, atop a raised platform. Fortunately for us all, Freddy's had the lights on.



  1. Biguine/Biguine
  2. Cormaine
  3. Copra Luca
  4. If You Want to Know Who We Are
  5. March Hora
  6. Old Plank Road
  7. Tania
  8. Mineola Rag
  9. Mysteriouse
  10. D Minor Bulgar

Brute Force

I didn't take any video of Brute Force.

Les Sans Culottes

  1. Magic Baguette
  2. Monsters from the Id
  3. Sur la Plage
  4. Allo Allo
  5. Je Suis Content
  6. Gangsteur*
  7. Jour du Velo
  8. Boots
  9. F.U.C. Something
  10. Chaussures
  11. Hot Bird
  12. Le Biz
  13. Sartre & the Lobster*
  14. Ou Est ou Est*
  15. SOS Elefants
  16. Telefon Douche

LSC Encore

  1. Shuba Duba Luba
  2. ???
  3. Ecole du Merde (for Virpi)

*: written by Gavin

Monday, January 31 2011

Isilon Cluster

Our old bulk storage is Apple Xserve RAIDs. They are discontinued and service contracts are expiring, so we have been evaluating small-to-medium storage options for some time. Our more modern stuff is a mix of Solaris 10 (ZFS) on Sun X4500/X4540 chassis (48 * 1tb SATA; discontinued), and Nexsan SATABeasts (42 SATA drives, either 1tb or 2tb) attached to Linux hosts, with ext3 filesystems. We are not buying any more Sun hardware or switching to FreeBSD for ZFS, and ext4 does not yet support filesystems over 16tb. Breaking up a nice large array into a bunch of 16tb filesystems is annoying, but moving (large) directories between filesystems is really irritating.

We eventually decided on a 4-node cluster of Isilon IQ 32000X-SSD nodes. Each ISI36 chassis is a 4U (7" tall) server with 24 3.5" drive bays on the front and 12 on the back. In our 32000X-SSD models, bays #1-4 are filled with SSDs (apparently 100gb each, currently usable only for metadata) and the other 32 bays hold 1tb SATA drives, thus the name. Each of our nodes has 2 GE ports on the motherboard and a dual-port 10GE card.

Isilon's OneFS operating system is based on FreeBSD, with their proprietary filesystem and extra bits added. Their OneFS cluster file system is cache coherent: inter-node lookups are handled over an InfiniBand (DDR?) backend, so any node can serve any request; most RAM on the nodes is used as cache. Rather than traditional RAID 5 or 6, the Isilon cluster stripes data 'vertically' across nodes, so it can continue to operate despite loss of an entire node. This means an Isilon cluster must consist of at least 3 matching nodes, just like a RAID5 must consist of at least 3 disks. Unfortunately, this increases the initial purchase cost considerably, but cost per terabyte decreases as node count grows, and the incremental system administration burden per node is much better than linear.

Routine administration is managed through the web interface, although esoteric options require the command line. Isilon put real work into the Tab completion dictionaries. This is quite helpful when exploring the command line interface, but the (zsh based) completions are not complete -- neither are the --help messages nor the manual pages, unfortunately.

There are many good things about Isilon.


  • Single filesystem & namespace. This sounds minor but is essential for coping with large data sets. Folders can be arbitrarily large and all capacity is available to all users/shares, subject to quotas.
  • Cost per terabyte decreases with node count, as parity data becomes a smaller proportion of total disk capacity.
  • Aggregate performance increases with node count -- total cache increases, and number of clients per server is reduced.
  • Administration burden is fairly flat with cluster growth.
  • The FlexProtect system (based on classic RAID striping-with-parity and mirroring, but between nodes rather than within nodes/shelves) is flexible and protects against whole-node failure.
  • NFS and CIFS servers are included in the base price.
  • Isilon's web UI is reasonably simple, but exposes significant power.
  • The command line environment is quite capable, and Tab completion improves discoverability.
  • Quotas are well designed, and flexible enough to use without too much handholding for exceptions.
  • Snapshots are straightforward and very useful. They are comparable to ZFS snapshots -- much better than Linux LVM snapshots (ext3 does not support snapshots directly).
  • The nodes include NVRAM and battery backup for safe high-speed writes.
  • Nodes are robust under load. Performance degrades predictably as load climbs, and we don't have to worry about pushing so hard the cluster falls over.
  • Isilon generally handles multiple network segments with aplomb.
  • The storage nodes provide complete services -- they do not require Linux servers to front-end services, or additional high availability support.
  • The disks are hot swap, and an entire chassis can be removed for service without disrupting cluster services.
  • Because the front end is gigabit Ethernet (or 10GE), an Isilon storage cluster can serve an arbitrarily large number of clients without expensive fibre channel HBAs and switches.

And, of course, some things are less good.


  • Initial/minimum investment is high: 3 matching nodes, 2 InfiniBand switches, and licenses.
  • Several additional licenses are required for full functionality.
  • Isilon is not perfectionistic about the documentation -- in fact, the docs are incomplete.
  • Isilon is not as invested in the supporting command-line environment as I had hoped.
  • The round-robin load balancing works by delegating a subdomain to the Isilon cluster. Organizationally, this might be complicated.
  • CIFS integration requires AD access for accounts. This might also be logistically difficult.
  • Usable capacity is unpredictable and varies based on data composition.
  • There are always two different disk utilization numbers: actual data size, and including protection. This is confusing compared to classic RAID, where users only see unique data size.
  • There is no good way for users to identify which node they're connected to. This is possible but awkward for administrators to determine, but it is generally not worth going beyond the basic web charts.
  • Support can be frustrating.
    • We often get responses from many people on the same case, and rehashing the background repeatedly wastes time.
    • Some reps are very good; but some are poor, with wrong answers, pointless instructions, and a disappointing lack of knowledge about the technology and products.
    • We are frequently asked for system name & serial number, and asked to upload a status report with isi_gather_info -- even when this is all already on file.
    • Minor events trigger email asking if we need help, even when we're in the middle of scheduled testing.
  • The cluster is built of off-the-shelf parts, and the integration is not always complete. For instance, we are not alerted of problems with an InfiniBand switch, because things like a faulted PSU are not visible to the nodes and not logged.
  • Many commands truncate output to 80 columns -- even when the terminal is wider. To see full output add -w.
  • When the system is fully up, the VGA console does not show a prompt. This makes it harder to determine whether a node has booted successfully.
  • There is only one bit of administrative access control: when users log in, they either have access to the full web interface and command-line tools, or they don't. There is no read-only or 'operator' mode.
  • Running out of space (or even low on space) is apparently dangerous.
  • One suggestion was to reserve one node's worth of disks as free space, so the whole cluster can run with a dead node. In a 4-node configuration, reserving 25% of raw space for robustness (in addition to 25% for parity) would mean 50% utilization at best, which is generally not feasible. In fairness, it is rare for a storage array to even attempt to work around a whole shelf failure, but most (non-Isilon) storage shelves are simple enclosures with fewer and simpler failure modes...
  • SmartConnect is implemented as a DNS server, but it's incomplete -- it only responds to A record requests, which causes errors when programs like host attempt other queries.
  • The front panels are finicky. The controls are counterintuitive, the LED system is prone to bizarre (software) failure modes, and removing the front panel to access the disks raises an obscure but scary alert.


  • On Isilon nodes, use du -Sl to get size without protection overhead. On Linux clients, use du --apparent-size.
  • Client load balancing is normally managed via DNS round robin, with the round robin addresses automatically redistributed in case of a node failure. This is less granular and balanced than you'd get from a full load balancer, but much simpler.
  • EMC has bought Isilon. I'm not sure what the impact will be, but I am not confident this will be a good thing over the long term.
  • In BIND (named), subdomain delegation is incompatible with forwarding. Workaround: Add forwarders {}; to zone containing Isilon NS record.


  • All that said, we are getting more Isilon storage -- it seems like the best fit for our requirements.

Monday, January 24 2011

Canon Vixia HF S20

I got a Sony video camera to take video of Julia, but an 8+ year old video camera is useless -- both the iPhone and the Canon Rebel T1i take better (and higher resolution) video, but the iPhone has no zoom and neither has high-quality audio. I have been filming and photographing friends' bands for a while now, and am getting a bit more serious about it. Unfortunately, musicians are allergic to light, so it's always an adverse environment for any kind of camera. It was time to find a video camera that can handle low light, with good/flexible audio.

I looked into Panasonic cameras briefly, but they're in the middle of updating their whole lineup, so their new models aren't available yet, and they don't have information on the old (discontinued) models online. After a great deal of searching, reading, comparing, and pondering, I decided on the Canon Vixia HF S20.


  • According to most reports it works relatively well in low light.
  • It has a 3.5mm stereo mic jack.
  • It has Canon's Mini Advanced Shoe
  • 1080i (1920*1080) resolution, at 60 fields per second.
  • In "PF30" mode the camera produces 30p video, which is what I want for iMovie. Many other cameras cannot handle this, although iMovie 11 is less picky about what footage it can handle. PF24 (recorded at 24p) appears fine as well.
  • Still photography up to 3264*2456 (16:9, in video mode) or 3264*1840 (4:3, in still mode).
  • 32gb flash onboard.
  • 2 SDHC slots. I have a firmware file that should upgrade to SDXC (different filesystem with the same physical characteristics, for cards of 64gb and larger), but haven't tried it -- I don't have any SDXC media to test against. The camera does have a "Relay Recording" mode so video can overflow from onboard to slot A and then slot B, although I may never need this. I might not even use the onboard flash memory if my iMac's SD(HC) slot is much faster than the camera's USB port.
    • To import directly from SD cards (should be faster, and enables the camera to charge simultaneously), connect it to a flash reader, launch iMovie, choose Import from Camera..., and select the SD reader (my iMac's is called "Apple Internal Memory Card Reader" from the Camera: popup
  • 10x zoom -- less than some other cameras but still quite useful.
  • Popup flash/video light. Handy, although I'm not sure if this will ever be useful.

I used the S20 at a very dark show with the DM-100, and got usable (though decidedly grainy) video with clear sound.


  • The included Canon BP-808 battery is rated at 55 minutes.
  • I bought a BP-819 with twice the capacity, rated at 105 minutes.
  • The S20 includes 32gb RAM, sufficient for 175 minutes of MXP (24mbps, 1920*1080) video.


  • The battery is frankly puny -- at highest quality, the built-in flash lasts more than 3 times as long as the puny battery. With the BP-819 as well, though, I should be able to record for 3 hours -- enough for any normal concert. For long events, I might need to bring the AC adaptor and plug the camera in. Unfortunately the camera can either run off AC current or charge the battery, but not both.
  • The S20 is overly complicated. It has 5 main modes:
    1. Dual Shot -- automatic mode, with both video and stills available, but no menu access at all.
    2. Video recording
    3. Photography
    4. Video playback
    5. Photo playback
  • Unfortunately Dual Shot mode does not allow overriding most of the defaults. This includes 60i frame rate (rather than the 30p or 24p I'd prefer), and won't even let me hide most of the onscreen status indicators.
  • Each menu command is linked to one or more non-auto modes. So to review all the settings, you must work through 3 menus in each of 4 modes.
  • The playback modes require choosing either photos or videos from one of the 3 possible sources (onboard flash, flash A, or flash B).
  • The menus are needlessly complicated. For example, when I'm shooting in "Dual Shot" mode, the steps to delete all photos from flash B are (this is different than the procedure to completely initialize flash B):
    1. Push the Camera/Review button.
    2. Push the Swap Playback(??) button (play arrow on 2 rectangles, surrounded by 2 curving arrows).
    3. Push B.
    4. Push Photos.
    5. Push Checkmark.
    6. Push Select All.
    7. Push Edit.
    8. Push Delete.
    9. Push Yes.
    10. (Wait for deletion.)
    11. Push OK.
    12. To shoot again, push Camera/Review again.
  • The touchscreen is not very sensitive. This was very annoying before I got used to it, but even now not all my presses register, and I worry about joggling the camera if I use the onscreen controls while filming.
  • No lens cap! Stupid! I cannot just throw a glass lens (or even a glass filter) in a bag and let it get scratched with every jostle. Fortunately it's 58mm, so finding caps isn't difficult, but I'm not about to pay $9 to Canon for a piece of plastic that should have been in the box anyway.
  • The dedicated low light mode is odd. It makes everything very blurry, as if it's using double-length exposures for pairs of frames. Not usable with any kind of motion. If I wanted to capture a still as video, it might serve, but the still camera would make more sense there.
  • The camera doesn't appear to work for streaming to a computer -- it only recognizes USB in 'playback' modes, not 'shooting' (recording) modes. This is a pity, as it means it can't work with iChat. I have a built-in iSight, but if I ever wanted to potcast, a better camera that isn't fixed into the monitor might be useful.
  • No viewfinder. This is a negative, but one I'm comfortable with. Canon's S21 adds a viewfinder and bumps the onboard RAM from 32gb to 64gb, but these are not worth an additional $235.


All things considered, I am happy with the camera. The touchscreen isn't very good and the menus are downright lousy, but I am able to mostly ignore them now that it's set up. I would have liked something simpler, but to get the odd combination of features I wanted, I needed a higher end and more complicated camera. Fortunately I can simply ignore most of the irrelevant capabilities (Direct burning of DVDs, onboard editing & effects, playlists, onboard creation of SD video from HD footage, etc.).

Friday, January 21 2011

Molly DNA at Pianos

Molly Mae and crew performed at Pianos, with a couple appearances by Latex Lily. Princess Sunshine opened, Amazing Amy contorted, and MDNA jammed with Squeezerock after their set. I got a bunch of decent photos.

  • Princess Sunshine opened with a rant on "Happily Ever after". Then she played accordion and ukelele and sang a couple songs about men.

Next, Molly Does Not Approve played a set:

  1. Cigarette Girl
  2. Free
  3. Find My Clitoris
  4. Canção Brasileira
  5. Pinup Girl
  6. Social Worker
  7. Baby

After Squeezerock's set the bands recombined.

  1. Hey Good Lookin' (extended jam with Julz-A -- unfortunately the lights went out partway through)
  2. Fortify (jam with Squeezerock; contortions by Amazing Amy)

Videos will be up on YouTube next week.

Sunday, December 26 2010

Brooklyn Blizzard!

We got back as the much-anticipated blizzard was beginning. It snowed all afternoon and plenty stuck. Lots of shoveling!


Friday, December 10 2010

4th Street Nite Owls at Two Boots, December 2010

Blow, man, blow!

Tonight Gavin played clarinet in the 4th Street Nite Owls at Two Boots. The music was good and the photos & video (56:10) came out pretty well, considering the poor light -- although Gavin spent most of the show hiding in shadow.

I missed the beginning of the show, but saw most of the first set:

  1. Makin' Whoopee
  2. Sweet Sue
  3. Have a Little Dream on Me
  4. Louisiana Fairytale
  5. Jerry the Junker

and the whole second set:

  1. It's No Fun
  2. Stayin' Alive
  3. Nagasaki
  4. Crosspatch
  5. When Day Is Done
  6. Crazy about My Baby (and My Baby's Crazy about Me)
  7. I Wish I Were Twins


The whole show (large & slow):

Wednesday, December 8 2010

This week's fiasco: Avocent

Today I racked a 1U Avocent LCD tray with 16-port IP-accessible KVM switch (ECS17KMM16-001). What a fiasco! We got it last week, but it took me several days and 3 phone calls before I was told to just ignore the bad spots in the documentation and work around them.


  • The Avocent Tech Support website requires registration, but doesn't let me register in Safari/Mac.
  • In Firefox/Mac, I apparently completed the registration process. Not sure because I just got a blank screen, and I am now unable to login.
  • The Services & Support area of avocent.com doesn't include manuals. I registered thinking they were behind that, but it turns out manuals are public -- just hidden under Resources.
  • Their explanation of password criteria is wrong -- it's alphanumeric, not alphabetical:

The password must be 5-16 characters and contain alphabetical characters of mixed case and at least one number.


  • It is impossible to find manuals by part number. Our "ECS17KMM16" LCD/KVM is not visible until "LCD Tray" is selected.
  • For some reason, there are 3 installation PDFs for our ECS on the website. The one for the LCD Tray doesn't mention the AutoView module at all, and the components have different part and serial numbers. This causes confusion with Avocent Tech Support, as they never seem to be able to find the first model & serial number I provide. They do not mention each other, even to say which steps must occur first and which last.

LCD Tray

  • The tray has a part number, a model number, and a serial number. The KVM switch has its own part and serial numbers. The KVM numbers are apparently useless to Avocent Technical Support, and they weren't able to look up the tray serial number either.
  • The installation instructions mention the notched ends go to the rear, but the photo doesn't show notches.
  • The LCD Tray instructions leave out a required cage nut, required to install the "Cable Channel". After I had racked the tray, I realized I would need to remove it and one of the rails to install this cage nut. But see below.
  • The instructions are quite insistent about using the "Spacer" (with warnings that things will break and fall apart otherwise), but do not identify what it actually looks like. The drawing of the spacer has no detail and doesn't match anything in the package. I found an otherwise apparently useless metal bar, but it did not fit as described or pictured.
  • The CMA (cable mounting arm) instructions say to remove and discard a screw from one of the slides, but not that it must be the right side, or that the screw is actually required to attach the CMA.

Push the LCD Tray into the rack, and then tighten the four rear slide-rail bracket screws. Remove the rail-adjustment screw that is closest to the rear of the rack from the outer slide-rail bracket and discard. Loosen the Velcro® straps on the cable retractor to allow free and smooth movement of the cable retractor arm. Align the cable-management arm (CMA) to the outer slide-rail bracket, and use the removed screw to attach the CMA.

AutoView 1016

  • The display is vertically truncated. This happens both when connected through the LCD tray (which does not allow me to vertically compress the display to fit, only to choose whether I want to see the top or bottom of the display) and via TCP/IP.
  • After I followed the LCD Tray instructions to install the rails, I discovered the AutoView requires an additional 2 cage nuts which are not mentioned (one is required for cable management even without the AutoView). I had to remove the tray, remove both rails, and add 2 more cage nuts for the AutoView to attach to. At least I didn't install the rails & tray, remove the tray and a rail, install the non-AutoView Cable Channel, install the tray a second time, and then discover I had to remove everything again.
  • There is mention of adjusting the front and back rails to fit the LCD Tray, but none of the fact that they must actually be adjusted to fit the AutoView switch -- which did not fit after I had adjusted the rear rails for the LCD Tray.
  • The AutoView's mounting flanges protrude several inches behind the rack flanges, which required me to remove an entire PDU to install the AutoView. This was only possible because it's a new rack.
  • The first step of 590-1012-501A is missing entirely -- removal of 2 silver screws from each side of the AutoView.
  • The first step pictured in 590-1012-501A is wrong. It shows attachment of 3 screws per side, but there are only 2 screw holes. I thought I had the wrong parts, but was eventually told this is apparently just a documentation error.
  • I was unable to find usage instructions for the AutoView in the package.
  • Password criteria are too restrictive. Basic punctuation should be allowed. We had to create yet another unique password just for this KVM, because it wouldn't accept the correct password.

The bright side

  • It looks like Avocent has recently decided all KVMs sold in the US should be IP accessible. Good move!

Saturday, November 13 2010

Mac OS X Time Machine: Multiple FAIL

My MacBook Pro's hard drive died last week. I got a new 250gb drive from Alex and replaced the dead one. Good thing I use Time Machine and a Time Capsule, I thought...

FAIL #1: The Mac OS X 10.6 Snow Leopard installer has an option to incorporate a Time Machine backup, so I mounted my Time Capsule. Unfortunately, Apple apparently hardwired the shared Data volume into this client. Of course, I chose to put my backups on my personal pepper volume, not the common Data volume accessible to other users! The Snow Leopard Installer absolutely refused to let me mount pepper or restore from Time Machine. I checked, and the backup is a 750gb .sparsebundle package on a 1tb Time Capsule, so I couldn't just move it over to the volume the Installer was fixated on -- not enough space.

FAIL #2: I copied the whole .sparsebundle backup to an external 1tb disk, but the Installer couldn't see the backup. It saw the disk, but couldn't find the backup. I tried putting it into a Backup folder, but no joy. Fortunately the Installer points out that you can also restore from Migration Assistant after installation (necessary if you're restoring to a Mac other than the one which made the backup).

So I cursed a little, did a regular install, and rebooted.

FAIL #3: Apple's welcome app (loud music & whizzy graphics) can restore Time Machine backups, but again couldn't see my pepper volume or restore from my hard disk. So I had to walk through creating my account again.

FAIL #4: After logging in, I launched /Applications/Utilities/Migration Assistant (I know most people don't even know it exists, much less where, although it's easy enough to find if you know what you're looking for), and told it I wanted to restore from a Time Machine backup. Again, it was unable to see either my network or local backups, but once I double-clicked the .sparsebackup file it mounted as "Time Machine Backups" and Migration Assistant saw it. Confusingly, Migration Assistant showed the new volume as 'prowler ("prowler")', which doesn't match its name on the Desktop.

Volume names are confusing!

Migration Assistant immediately complained that I was logged in as user pepper, and my backup included a user pepper, and it cannot replace the active account. I won't call this a failure because it's a legitimately intractable problem, but I used "sudo passwd root", logged in as root, and reran the migration -- this is hardly reasonable to expect of users.

This is a fine Catch-22!

Let's hope Apple fixes the Time Machine restore functionality in Mac OS X 10.7 Lion, because this is awful.

Sunday, October 31 2010

Dances of Vice: Phantasmagorey II

Dances of Vice is apparently a series of shows in NYC. This was their second http://dancesofvice.com/phantasmagorey.html event. The show started with the Ghost Train Orchestra (quite good), then a bunch of dance/performance numbers, an audience "fashion show" decided by trivia questions, and finally a fashion show by Purevile.

I got some video of the orchestra and a dance number:

Phantasmagorey II

Wednesday, October 20 2010

2 Months of Tech FAIL

Several months ago, my MacBook Pro stopped joining the work WiFi network. This had worked for a while, but then stopped. I don't bring it in much, so it took me a while to realize it was a consistent problem. After a bunch of poking and prodding, we realized it only affected my account. So I used Ethernet at work, rather than recreate my home directory from scratch. MOSX FAIL.

Then it refused to boot entirely. I sent it in to Apple, who informed me I had one of a bad batch of video cards, which they replaced. ATI FAIL.

When I got it back the case was a bit bent and the optical drive didn't work. So I sent it back and they bent the case mostly back into shape and replaced the SuperDrive. I still cannot burn DVD-DL media, but I have decided this is a more general problem (which affects Mac Pros in the office too). AppleCare & SuperDrive FAIL.

Then my 24" LCD started flickering when connected to the MBP. This was annoying enough to interfere with getting work done, but hard to reproduce (schlepping a 15" MBP to the Apple Store is a nuisance, but bringing a 24" LCD monitor and waiting for an intermittent problem to reappear was non-viable). While trying to figure this out, I noticed that it also wasn't auto-detecting connection/disconnection of the LCD monitor, and this was easy to replicate booted from a fresh 10.6.4 install. I brought it into Apple, and it didn't reproduce with their monitor. Frustrating! So I reinstalled and spent a week manually copying over the few bits I really needed from a backup of my old home directory. I thought my problems were due to the replacement video card, but this was apparently double OS FAIL.

A few weeks later, www.reppep.com/mail.reppep.com started hanging. Eventually I realized the 2-year-old Inspiron was dead and bought a new server entirely, which is running now (although it occasionally has periods of unexplained high load). Linux mdadm didn't work at all, although I suspect this was due to underlying hardware problems. I'll need to switch back to a mirrored configuration later... Dell & mdadm FAIL.

Then we had a couple floods. After over a century of working basically well, major sewer FAIL.

The new reppep.com is PCIe based, so it needed a new GE NIC, which keeps inexplicably losing its connection to the network. The new card and GE switch arrived today, but were stolen from the lobby of our building: NIC(?) & lock FAIL.

To make things more 'interesting', our Speakeasy DSL dropped a few times. I called Speakeasy, who told me the circuit was up and fine -- clearly not true -- and that my problems were due to the Linux iptables firewall (laughable, but with no Internet I found myself unable to laugh). At the same time an AirPort Extreme failed and refused to reset. I eventually got it to reset and reconfigured, and then our Time Capsule (which is relatively new, having been replaced when it died on schedule of bad caps) died. AirPort + Time Capsule + DSL FAIL.

Today at work my ~~18-month-old Mac Pro died -- apparently the power supply just stopped supplying power. Hopefully it will be up soon, once I get a replacement. PS FAIL.

Despite all this I should acknowledge that the Compaq Evo 510 SFF I bought several years ago ran fine until I retired it last month, and Amy's 2gb/2GHz MacBook has been fine (modulo some calendar problems, which were pure software). Our iPad and my iPhone 4 have also been fine. And our many hard drives have been okay. So not everything is failing -- it just feels like it.

Here's hoping the story is over, rather than still evolving.

Tuesday, October 19 2010

BIND: Beware forwarders with subdomains

Yesterday I set up a subdomain for an Isilon cluster -- they use the subdomain to handle load balancing across nodes. We were all surprised to discover that the forwarders option in named.conf) prevents subdomain delegation -- thensrecord for the subdomain was completely ignored. Once I removed theforwarders` clause, the subdomain worked.

Normally, forwarders in the options block of named.conf is handy -- it informs named of local resolvers, so named can get answers from a nearby/fast resolver, rather than going all the way up to the international root servers and working its way back down.

Sunday, October 10 2010

Les Sans Culottes & Persephone's Bees at The Rock Shop

Les Sans Culottes & Persephone's Bees played The Rock Shop. Show was excellent (and for a change I recognized most the music Gavin was playing), and photos are up.

Tuesday, October 5 2010

Well, THAT was unpleasant -- reppep.com postmortem

Last week, reppep.com (Dell Inspiron 530 desktop, a couple years old and running CentOS 5) stopped responding to email requests. It serves a bunch of websites and a few email accounts, but the email service is much more important. The key disks were a pair of 750gb disks mirrored with mdadm; I also have 3 1tb disks for data.

I discovered that logging in locally restored responsiveness, at least for a while. Unfortunately there was nothing I could do to bring it back from work. I was in the middle of a cluster build at work and busy with some projects at home, so I left it for a few days. I noticed some panics on the console and messages about resynching /dev/md3 (swap) and /dev/md6 (/home). Those should clear themselves, but I always wonder: with a discrepancy among 2 mirrored disks, how do you decide which to trust? If one disk completely fails it's clear, but in this case despite a heat warning, smartmontools stubbornly claimed neither disk had serious problems. I kept it staggering along for a few days, until one day after a particularly long bout of responsiveness and a complaint from Amy, I gave up on waiting it out or finding a solid indication of what was wrong.

I tried pulling one of the 750gb disks, hoping it would run off the good(?) submirror, but [warning: details get fuzzy at this point] it kept complaining about /dev/md3 sync not completing (with an implication it was just stuck waiting for /dev/md6 to sync, but perhaps the system just wasn't staying up long enough to resync the 634gb of /home), and additionally I got out-of-memory crashes. I had bought the system with 1gb RAM and configured 4gb of swap. After starting the mail system, Apache httpd, openfire Jabber server, CrashPlan backup service, etc., the system exceeded 1gb, and with swap offline it was killing processes and crashing. I bought and installed a couple 1gb DIMMs (it's convenient to have Staples a couple blocks away!). I saw USB / IRQ errors, which suggested irqpoll (which can apparently slow the system down, but was worth trying), so I added it to the kernel arguments, but still no stability. I tried running off the other 750gb submirror instead, but that didn't help.

I bought a new 1tb disk, figuring I'd use it to replace the 750gb disk with the heat warning. But the system kept crashing, the same way. I tried pairing the 1tb with the other 750gb, and got the same crashes. To avoid the crashing sync process, I used The --zero-superblock argument to mdadm (syntax is a bit tricky) to remove the RAID metadata, and changed the partition types from RAID to regular Linux filesystems. Finally I installed CentOS 5.5 afresh on the 1tb disk and disconnected the rest of the disks and all USB except the keyboard and mouse: more panics, including the IRQ errors. At this point, it was apparent that my 2-year-old Dell was curdled.

ns2.reppep.com is a Compaq Evo 510 SFF (EOL 8 years ago). It's perfectly adequate as a BIND slave server, but I've been planning to replace it with a plug computer or netbook for a while, to stop wasting power.

The Evo has a single PATA drive bay, but I have USB cases. Unfortunately, as I began to configure it I noticed it only has 256mb of RAM! That's fine for BIND, but not my email system. I could spend $100 on RAM for this ancient computer, but that seemed silly. Instead I bought an HP Pavilion P6610F, which so far seems fine. It has a quad-core Athlon, which may be irrelevant because its main purpose is to serve web & email up a 1.5mbps uplink (6mbps down), or might be handy for HandBrake or other stuff. It came with 4gb RAM, so the $100 I spent on the Dell was wasted. That's one of the more purely irritating aspects of this whole misadventure.

Installing CentOS on the HP was easy. With RPMforge, installing the mail system was straightforward (much easier than building amavisd-new, clamav, and all their dependencies manually, as I did a couple years ago for the Dell). Unfortunately, users did not see old mail until I realized that I was using the wrong reconstruct syntax for cyrus-imapd (Cyrus can use . or / as a path delimiter, and although chk_cyrus uses . as a delimiter on my system, reconstruct requires /, and doesn't complain when provided the wrong syntax -- I kept running reconstruct and wondering why it didn't recover mail! Thanks to the helpful info-cyrus@ list members!

openfire was trivial -- I just installed the RPM and copied /opt/openfire to the new disk. Apache was quick & easy too -- putting my configuration back was simple, then I had to install mod_ssl and a few PHP modules for Dotclear. MySQL was easy -- I just put the files back, and didn't have to test my automysqlbackup dumps.

Unfortunately, the HP only has a single 10/100 Ethernet port (and WiFi, but who cares on a Linux server?). The Dell was PCI based, so I ordered a new PCIe GE card for the HP; fortunately GE cards are cheap, so the only aggravation is waiting for it. Ironically/sadly, Staples (who sold me the HP) only has 2 GE cards in the store -- both the PCI GA311 I already have -- meaning they don't have any GE options for the HP they sold me.

My other irritation is that this Dell died -- so badly -- after 2 years. Obviously the Evo is much more robust, as have been most of my computers.

This whole unpleasant experience reminded me (painfully) that grub is very poor at dealing with mirrored boot disks. It tends to try booting the wrong disk, in various iterations. The grub command always assumes there is a single boot disk, and simply doesn't support redundancy well. With real hardware mirroring this would all be out of grub's control or visibility, but that's rare on desktops (most 'RAID' support on desktops is just fakeraid. Fortunately the HP's BIOS lets me choose which SATA disk to boot from, and that becomes /dev/sda, so I was able to get grub working (with a few false starts).

Now mail is back with all mail recovered, all websites are online, I have Jabber back, and things seem copacetic. As soon as I get the PCIe GE card and rid of the flood abatement hardware, I can restore the high-speed connection to our home LAN and reconnect my data drives...

Wednesday, September 22 2010

PC Pro Confusion on iOS vs. Flash & Linux

I read Tom Arah's Apple vs Adobe: some surprising statistics this morning, and was confused by a couple things. First, I guess I'm not surprised that there are more iPhones + iPod touches + iPads than Linux computers on the Internet, but can Linux share (including Android) really be < 1.1%?!?

The linked report claims 0.85% for 'Linux' + 0.09% for Android 2.1 + 0.05% for Android 1.6 + 0.03% for Android 2.2: about 1.02% total. That's shocking -- I thought Linux was much more common, although I guess the large number of Internet-enabled (smart)phones has diluted the Linux desktop market share a bit in recent years.

But the money quote, for me:

Moreover there’s little doubt that, if they were given any say in the matter, the overwhelming majority of that 1.1% would choose to see Flash and Silverlight content (presumably including those who choose to block Flash content by default in their desktop browsers but still install the player).

  1. Aside from browser creators, the only significant chunk of people who actually install Flash are Linux desktop/laptop users. Users never install Flash, or its share would be much lower than 97%!
  2. On the desktop, the trend is very clear: users put a lot of effort into avoiding Flash content. I have used 3 different Flash blockers and find them invaluable. Tom acknowledges this. His statement that users "still install the player" is crap. The best you can say for Adobe is that most users do not delete the Flash player.
  3. Tom's claim that "the overwhelming majority" of users would install Flash if they could is completely unjustified. I don't want to see Flash ads, videos, or games on my iPhone or iPad -- I'd much rather have h.264 videos (no ads!), and simply skip Flash ads and games on my mobile devices. On the Mac I use another plugin to see h.264 YouTube videos instead of Flash.

People who are aware of this struggle understand that h.264's recent growth has largely been at the expense of Flash video, and because iOS doesn't support it. If the iPhone & iPad supported Flash, we'd be watching (or trying to watch) videos in Adobe's broken mobile Flash player.

So what I get from this article: Although iOS/h.264 are clearly in the ascent -- at the expense of Linux/Flash -- it's too early to say the race is over, and Tom Arah either wasn't thinking very clearly, or deliberately warped a few facts to kinda-sorta support his agenda.

FWIW, it would be nice if kids could run Flash games on the iPad, but there are so many good (free) games that this isn't really a problem. I've gotten several Poisson Rouge iPhone apps, and I see a bunch of Club Penguin apps, so it's not a big problem.

PS-The reasons Google supports Flash on Android are pretty clear -- it's not because Google & Adobe are the good guys, and Apple and the Motion Picture Experts Group (MPEG) are evil villains.

  1. h.264 is more open than Flash.
  2. Before the Apple/Adobe tiff, I believe (but cannot confirm) that Google stated that Android would not support Flash.
  3. Google and its handset partners use Flash support on Android as a feature differentiator against the Apple iPhone.
  4. Google markets Android as open and embracing of Flash, while they paint Apple as closed and restrictive. But previously Google & Apple were allied in embracing the openness of h.264 video.
  5. Flash on mobiles means Android users will eventually get to see YouTube ads (once the mobile Flash player can handle them). Win! (for iOS users)

Wednesday, August 18 2010

SystemImager & SALI

We use SystemImager to maintain (rebuild) our small HPC clusters. Conceptually it's very simple:

  1. Build a node (the 'golden client') just the way you want it.
  2. si_prepareclient: Run rsyncd on the node, accessible to the 'image server'.
  3. si_getimage on the image server copies the entire node into a directory, and analyzes it to produce a script that will recreate the image (with exclusions for files which should differ between nodes).
  4. si_updateclient on a target node fetches the script from the image server; the script configures the target (disk partitioning, etc.) and fetches the image contents, making the target match the golden client.
  5. If the node is dead or brand-new, there's a DHCP/PXE/TFTP process for bootstrapping far enough to run the script and then match the golden client.

Once the SI system is all set up, it's quick & easy to rebuild nodes. Unfortunately there are several complications:

  • The DHCP & TFTP dependencies are somewhat complicated, so bringing up SI without breaking anything is tricky. TFTP & pxelinux are not terribly well documented.
  • The "Latest Stable Release" is SystemImager 4.0.2 from December 2007. One of the key components of SystemImager is a generic kernel & Linux initrd (initial RAMdisk) which include a default set of drivers. But the release is so old that it cannot handle current hardware. There are several newer development versions but they're not fully baked and choosing between them is confusing.
  • SI doesn't yet support grub2 or ext4, which are required for large disks (GPT partition tables).

The workaround I got from the very helpful folks on sisuite-users@ was to use SALI, a modern kernel/initrd pair for SystemImager. Unfortunately SALI's a bit different -- in the process of adding grub2 support, they broke compatibility with the scripts that SI generates. Here's a quick recap of the steps I used (mostly from sisuite-users@) to use SALI:

  • Drop the 2 SALI files into the TFTP directory (normally /var/lib/tftpboot/ or /tftpboot/).
  • Specify the SALI files in /var/lib/tftpboot/pxelinux.cfg/default or equivalent.
  • Add a couple lines to /etc/dhcpd.conf.
  • Set SCRIPTNAME= in pxelinux.cfg/default.
  • In the script created by SI:
    • Change DISK_SIZE entries to "DISK_SIZE=$(get_disksize $DISK0)".
    • Remove -v1 from mkswap arguments.
    • Add -I 128 to mke2fs for the /boot FS.
    • Remove "-o defaults" from mount commands.
    • SystemImager's final line in the script is "shutdown -r now", which fails on SALI. Use reboot until SALI 1.3, which should support shutdown.
  • On our newer cluster, SALI does bizarre things with console redirection. I had to type into the (virtual VGA) console, while output appeared on the serial console. The serial console recognized and echoed my input, but did not execute it.
  • (Not SALI related): Make sure the scripts (normally in /var/lib/systemimager/scripts) are executable -- SI left mine non-executable for some reason.

- page 2 of 18 -