Extra Pepperoni

To content | To menu | To search

Tag - computers

Entries feed - Comments feed

Wednesday, December 31 2008

Sun Fire X4500, Thumper

We have a couple X4500 "Thumper" servers here. An X4500 is basically a standard dual-Opteron chassis, with 48 SATA drive bays added on. This provides a raw capacity of 48tb with 1tb drives, or over 30tb usable in a realistic RAID environment. I love the informative labels on the outside (I haven't yet popped the cover to see the internals):

  1. If you open the cover, you must close it within 60 seconds to avoid frying components.
  2. If you lift the chassis, you will bend the sheet metal. I am told after removing all 48 drives and both power supplies, it's still difficult for one person to lift the empty chassis.

Solaris cannot yet boot from RAIDZ, so it's common to install the OS onto a single disk, mirror it with Sun Volume Manager (DiskSuite), and then use the 46 remaining disks for RAID and hot spares. RAIDZ works better with < 10 drives in a stripe, and can be used with single parity (RAIDZ1) or double (RAIDZ2).

A reasonable configuration would be 2 hot spares, 4 9-disk RAIDZ sets, and an 8-disk RAIDZ set -- all RAIDZ1 or RAIDZ2, because mixing different types of drives of protection levels in a single ZFS pool is not recommended. A higher performance configuration would use smaller RAIDZ sets, or even ZFS mirroring. 4 9-disk and 1 8-disk RAIDZ1 stripes provide 4 * 8 + 7 = 39 drives of capacity. With RAIDZ2 it's 4 * 7 + 6 = 34 drives usable. Of course, '1tb' drives don't provide a real 1tb usable, because drive manufacturers use base 10 and operating systems use base 2. The formatted capacity of our '1tb' drives is 931gb, so we would max out at 39 * 931gb = 36tb usable. Impressive, but not quite the 48tb advertised.

My current problem: Only 2 of the 48 disks are bootable, and their naming within Solaris is not consistent. In the list below, 46 disks report as ATA-HITACHI, but c4t0 & c4t4 report as DEFAULT. I'm pretty sure I should just mirror onto c4t4, but not enough to proceed with putting the server into production without verification. I've found docs across the Internet that refer to c5t0 & c5t4, as well as c6t0 & c6t4, so it's not as simple as it really should be.

"Which disks can I boot from?" is a fundamental question for a system that cannot boot from any attached disk, but when I called 800-USA-4-Sun yesterday, I explained my issue to a very nice gentleman who told me to pick the first two disks from format output. That's apparently wrong for an X4500, and after I explained the issue further he started finding and reading the same docs I had been. A particular favorite is "Upgrading to ILOM 2.0.2.5 Changes Controller IDs" in the Sun Fire X4500 Server Product Notes, which refers to a hd command not present on my fresh Solaris 10 10/08 installation or the pre-installed Solaris 10 11/06 that came on the other Thumper. I also found Important Solaris OS Installation and Bootable Hard Disk Drive Guidelines in the Product Notes, which says to use the first two disks returned by cfgadm -al -- c0t0 & c0t1 on this system. That's not right, although the instructions are for use within the Solaris installer.

Today I called back, and was eventually told there is only one person in who would have the answer to my question, but he's busy on another call!

This makes the "okay-to-remove" LEDs on the drives essential -- with a RAIDZ1 set, if one drive goes bad and I accidentally remove a good drive in the same set, at best the whole pool will go offline (and no data will be lost).

bash-3.00# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0

Sunday, December 28 2008

Interview Oddity

I have wondered if an interviewer would see this blog or my homepage, or my Twitter feed, and today it happened. Bobby Brill, at NYU, not only had a copy of my resume, but he also had a copy of my System Admin Interview Questions from Rockefeller. Goldman and a few other financial companies I interviewed with a year ago used a very different interview format, but they used the same format, which makes me think they all copy from each other.

I later met with a couple people who would be teammates at NYU (whom I knew socially already), and they mentioned my interview questions as well. Alas, I didn't get to sail through purely on knowing those answers, but I'm glad they're doing someone some good, at least for entertainment.


Note: I wrote this post in November, but didn't post it immediately -- I wanted to wait until the interview process was over.

Saturday, December 27 2008

DNS Glue Records

I had an obscure DNS problem, and EasyDNS pointed me in the right direction.

When looking up ns3.reppep.com, I was getting 66.92.104.200, which is wrong (that address is actually ns1.reppep.com) from some DNS servers, but not others. I confirmed that the record was correct in the reppep.com zone (hosted by EasyDNS), and forced a serial number update, but some servers kept returning the wrong address.

It turns out that years ago, when VeriSlime managed the DNS registration for reppep.com, they picked up the wrong address (it might have been correct when they picked up the IP, but hasn't been for years). When I transferred the domain registration (which is separate from the DNS service -- basically authoritative nameservers) to Dreamhost, the stale glue data remained. The solution is to have Dreamhost update the glue record for ns3.reppep.com.

You can see the glue with the whois command, e.g., "whois ns3.reppep.com. It should agree with "host ns3.reppep.com", but doesn't yet (still waiting on Dreamhost to get it together).

New Job: MSKCC

I just realized I hadn't posted my new job here.

My post-Goldman job hunt was mercifully brief. As of December 8, 2008, I work at Memorial Sloan-Kettering Cancer Center, within the Sloan-Kettering Institute (SKI is the research arm, as opposed to Memorial Hospital). I'm in the Bioinformatics Core of the Computational Biology Center.

The office is a block away from Rockefeller University, and I work with several other ex-Rockefeller people. The work itself is similar in many ways, and I much prefer it to GS. My first project (aside from figuring out my way around) is sorting out some Sun Thumpers, so I'm doing hands-on ZFS -- which I had read about but not really used before.

iPhone Users: Boycott Clickthroughs

Interstitial ads are annoying on desktop web browsers, but they're much more intrusive on handheld devices like iPhones. I have decided not to click through such ads on the iPhone, and to bail when I come across them, even when this means not reading content I want.

This is in the (likely vain) hope of convincing the people who control those websites that handheld users (at least) won't put up with that garbage.

I encourage you to join the boycott! Maybe we can get half a dozen people and really turn some heads, just like the Alice's Restaurant Anti-Massacre Movement.

Monday, December 22 2008

boxee

After a mention by Kevin van Haaren, I decided to try boxee, a multiplatform open source media sharing project. Specifically, I was interested in running boxee on our Apple TV, and pointing it at a share on a Linux server, since the Apple TV's 160gb hard drive is perpetually full, and I spend a significant amount of time moving video on and off via iTunes. Additionally, boxee adds support for additional video formats, and facilitates access to Internet content such as hulu and RSS video feeds (ONN FTW, although I was already getting that through iTunes).

atvusb-creator

My first problem was some confusion in the atvusb-creator, which builds a "patchstick" by copying the appropriate files onto a flash drive. Booting an Apple TV from the patchstick adds boxee to the existing Apple TV software, and provides additional menu items and ssh access.

I was distracted by the bright blue Choose a DMG button. In Mac OS X, blue buttons are supposed to identify the default action, so I kept trying to find a suitable .dmg file to feed it, with no luck. I saw a few warnings that the creator only works with some flash drives, so thought mine were incompatible. After I watched a video on creating the patchstick, I realized this button was a herring -- it is apparently unneeded. Once I skipped straight to Create Using ->, I got a patchstick. Hooray!


Today, I got Boxee installed. It was pretty simple:

  1. Pull the plug.
  2. Plug the patchstick into the back.
  3. Turn on the Apple TV.

I watched Linux boot messages, then saw a message telling me no errors were detected, and to reboot the Apple TV. When it came back, I had 2 extra menu items: XBMC/Boxee & Software Menu. The former includes a (highly recommended) Update command (but after updating I am still offered the same set of updates), a Boxee command which gives me an inescapable black screen, and an XBMC command that lets me configure and use up the Xbox Media Center (which Boxee is built upon). Unfortunately, XBMC doesn't see the videos, music, & photos iTunes synched over, so there's no content yet.

Ah, well. That's progress, at least.

Thursday, December 18 2008

Google's 404 Service: Apache httpd Recipe

I just registered EP and my home site with Google Webmaster Tools, and discovered they offer a 404 (page not found) service for replacing your existing not-found page with Google links which should include useful suggestions from your site. At the moment I just get a search box with the non-existent page name pre-filled, which is still a useful improvement.

I had the default Apache 404, and it's fine but it's not particularly useful or friendly, so I set up the Google alternative. Somewhat to my surprise, Google doesn't offer Apache httpd instructions for actually creating the 404 page, so here's my 3-step recipe for httpd 2.2.

1) Create a new page.

Mine is /var/www/error/404-google.html, accessible directly at http://www.reppep.com/error/404-google.html.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
        "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
    <title>404: Page Not Found</title>
</head>
<body>

<script type="text/javascript">
  var GOOG_FIXURL_LANG = 'en';
  var GOOG_FIXURL_SITE = 'http://www.reppep.com/';
</script>
<script type="text/javascript" 
    src="http://linkhelp.clients.google.com/tbproxy/lh/wm/fixurl.js"></script>

</body>
</html>

2) Enable the Google 404 in httpd.conf.

ErrorDocument 404 /error/404-google.html

If you already have a custom 404 defined, disable it.

3) Reload the server configuration.

[root@inspector ~]# apachectl graceful

Thursday, December 11 2008

hb.sh (HandBrake Wrapper) Updated for v0.9.3

As threatened, I have an update for my HandBrake wrapper script. On a Mac or Linux system, the script examines one or more DVD folders (I rip mine with MacTheRipper), and converts the DVD MPEG2 content to MP4 files suitable for Apple TV (by default) or iPhone (with an iphone argument, e.g., hb.sh iphone). It's easy to hack up for different preferences, and will require changing some variables to match your preferred media folder layout. Thanks to Brian Beardmore for the original GPL script.

The required customizations are these two lines:

inputSearchDir="$HOME/tivo-inspector/input/$mySuffix"
outputDir="$HOME/tivo-inspector/output"

The main difference in v1.0.7 of the script is that it uses HandBrake 0.9.3's new improved presets, rather than hard-coding my custom settings. I've found the new presets give excellent quality in considerably less time.

If you have trouble compiling the Linux CLI, see my earlier post about v0.9.3 (I didn't try the GUI).

You can download hb.sh v1.0.7 (recommended), or copy & paste, but might have to fix line wrapping & remove blog cruft if you do.


#!/bin/bash

# hb.sh uses HandBrakeCLI to convert whole DVDs automatically.
# Copyright (C) 2007  Brian Beardmore
# Copyright (C) 2008  Chris Pepper

#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

# This script requires HandBrakeCLI <http://handbrake.fr/>.

# This script searches the entire input directory directory tree looking
# for TS_VIDEO directories looking for DVDs to encode.  The output mp4
# filename is the DVD volume name.  When multiple tracks are to encoded 
# from a DVD, the output filename is appended with the track number.
# For example, 'Monsters, Inc.' DVD has both standard and wide screen tracks
# so there are 'Monsters, Inc.1.mp4' & 'Monsters, Inc.10.mp4' files created.
# For addition information on this script contact the script's original author,
# Brian Beardmore at medfk(at)realisticsoftware(dot)com or
# http://www.realisticsoftware.com
# Or contact Chris Pepper <http://www.extrapepperoni.com/category/television/>.

# Examples
# $ ./appletv.sh
#   [ encodes all DVDs mounted and encodes any tracks longer than 60min ]
#   [ to the mp4 folder in the users Movies folder ]
# $ appletv.sh -i ~/Movies -o toPod -b 1000 -w 576 --minGetTime 50
#   [ encodes all DVDs found in the users Movies folder and encodes any ]
#   [ tracks longer than 50min, with a 1000kb/s bitrate and scales the output ]
#   [ movie to a width of 576 pixels and puts the resulting mp4 files in the ]
#   [ toPod folder in the current directory that the command was executed from ]

# Syntax: HandBrakeCLI [options] -i <device> -o <file>
# 
# ### General Handbrake Options------------------------------------------------
# 
#     -h, --help              Print help
#     -u, --update            Check for updates and exit
#     -v, --verbose <#>       Be verbose (optional argument: logging level)
#     -C, --cpu               Set CPU count (default: autodetected)
#     -Z. --preset <string>   Use a built-in preset. Capitalization matters, and
#                             if the preset name has spaces, surround it with
#                             double quotation marks
#     -z, --preset-list       See a list of available built-in presets
# 
# ### Source Options-----------------------------------------------------------
# 
#     -i, --input <string>    Set input device
#     -t, --title <number>    Select a title to encode (0 to scan only,
#                             default: 1)
#     -L, --longest           Select the longest title
#     -c, --chapters <string> Select chapters (e.g. "1-3" for chapters
#                             1 to 3, or "3" for chapter 3 only,
#                             default: all chapters)
# 
# ### Destination Options------------------------------------------------------
# 
#     -o, --output <string>   Set output file name
#     -f, --format <string>   Set output format (avi/mp4/ogm/mkv, default:
#                             autodetected from file name)
#     -m, --markers           Add chapter markers (mp4 and mkv output formats only)
#     -4, --large-file        Use 64-bit mp4 files that can hold more than
#                             4 GB. Note: Breaks iPod, PS3 compatibility.
#     -O, --optimize          Optimize mp4 files for HTTP streaming
#     -I, --ipod-atom         Mark mp4 files so 5.5G iPods will accept them
# 
# ### Video Options------------------------------------------------------------
# 
#     -e, --encoder <string>  Set video library encoder (ffmpeg,xvid,
#                             x264,theora default: ffmpeg)
#     -x, --x264opts <string> Specify advanced x264 options in the
#                             same style as mencoder:
#                             option1=value1:option2=value2
#     -q, --quality <float>   Set video quality (0.0..1.0)
#     -Q, --cqp               Use with -q for CQP instead of CRF
#     -S, --size <MB>         Set target size
#     -b, --vb <kb/s>         Set video bitrate (default: 1000)
#     -2, --two-pass          Use two-pass mode
#     -T, --turbo             When using 2-pass use the turbo options
#                             on the first pass to improve speed
#                             (only works with x264, affects PSNR by about 0.05dB,
#                             and increases first pass speed two to four times)
#     -r, --rate              Set video framerate (5/10/12/15/23.976/24/25/29.97)
#                             Be aware that not specifying a framerate lets
#                             HandBrake preserve a source's time stamps,
#                             potentially creating variable framerate video
# 
# ### Audio Options-----------------------------------------------------------
# 
#     -a, --audio <string>    Select audio track(s), separated by commas
#                             More than one output track can be used for one
#                             input.
#                             ("none" for no audio, "1,2,3" for multiple
#                              tracks, default: first one)
#     -E, --aencoder <string> Audio encoder(s) (faac/lame/vorbis/ac3) 
#                             ac3 meaning passthrough
#                             Separated by commas for more than one audio track.
#                             (default: guessed)
#     -B, --ab <kb/s>         Set audio bitrate(s)  (default: 160)
#                             Separated by commas for more than one audio track.
#     -6, --mixdown <string>  Format(s) for surround sound downmixing
#                             Separated by commas for more than one audio track.
#                             (mono/stereo/dpl1/dpl2/6ch, default: dpl2)
#     -R, --arate             Set audio samplerate(s) (22.05/24/32/44.1/48 kHz)
#                             Separated by commas for more than one audio track.
#     -D, --drc <float>       Apply extra dynamic range compression to the audio,
#                             making soft sounds louder. Range is 1.0 to 4.0
#                             (too loud), with 1.5 - 2.5 being a useful range.
#                             Separated by commas for more than one audio track.
#     -A, --aname <string>    Audio track name(s),
#                             Separated by commas for more than one audio track.
# 
# ### Picture Settings---------------------------------------------------------
# 
#     -w, --width <number>    Set picture width
#     -l, --height <number>   Set picture height
#         --crop <T:B:L:R>    Set cropping values (default: autocrop)
#     -Y, --maxHeight <#>     Set maximum height
#     -X, --maxWidth <#>      Set maximum width
#     -p, --pixelratio        Store pixel aspect ratio in video stream
#     -P, --loosePixelratio   Store pixel aspect ratio with specified width
#           <MOD:PARX:PARY>   Takes as optional arguments what number you want
#                             the dimensions to divide cleanly by (default 16)
#                             and the pixel ratio to use (default autodetected)
#     -M  --color-matrix      Set the color space signaled by the output
#           <601 or 709>      (Bt.601 is mostly for SD content, Bt.709 for HD,
#                              default: set by resolution)
# 
# ### Filters---------------------------------------------------------
# 
#     -d, --deinterlace       Deinterlace video with yadif/mcdeint filter
#           <YM:FD:MM:QP>     (default 0:-1:-1:1)
#            or
#           <fast/slow/slower>
#     -5, --decomb            Selectively deinterlaces when it detects combing
#           <MO:ME:MT:ST:BT:BX:BY>     (default: 1:2:6:9:80:16:16)
#     -9, --detelecine        Detelecine (ivtc) video with pullup filter
#                             Note: this filter drops duplicate frames to
#                             restore the pre-telecine framerate, unless you
#                             specify a constant framerate (--rate 29.97)
#           <L:R:T:B:SB:MP>   (default 1:1:4:4:0:0)
#     -8, --denoise           Denoise video with hqdn3d filter
#           <SL:SC:TL:TC>     (default 4:3:6:4.5)
#            or
#           <weak/medium/strong>
#     -7, --deblock           Deblock video with pp7 filter
#           <QP:M>            (default 5:2)
#     -g, --grayscale         Grayscale encoding
# 
# ### Subtitle Options------------------------------------------------------------
# 
#     -s, --subtitle <number> Select subtitle (default: none)
#     -U, --subtitle-scan     Scan for subtitles in an extra 1st pass, and choose
#                             the one that's only used 10 percent of the time
#                             or less. This should locate subtitles for short
#                             foreign language segments. Best used in conjunction
#                             with --subtitle-forced.
#     -F, --subtitle-forced   Only display subtitles from the selected stream if
#                             the subtitle has the forced flag set. May be used in
#                             conjunction with --subtitle-scan to auto-select
#                             a stream if it contains forced subtitles.
#     -N, --native-language   Select subtitles with this language if it does not
#           <string>          match the Audio language. Provide the language's
#                             iso639-2 code (fre, eng, spa, dut, et cetera)

# Revision history:
# 1.0.7, 2008/12 -- Update for HandBrake 0.9.3, and use presets.
# 1.0.6 -- never released.
# 1.0.5, 2008/08 -- Require bash, per ebb.
# 1.0.4, 2008/06 -- Use different suffices so different flavors can coexist.
# 1.0.3, 2008/05 -- Add basic argument processing, so "appletv" and "iphone" as argument #1 produce different output.
# 1.0.2, 2008/05 -- hacks by Pepper to use HandBrakeCLI and optimize for AppleTV.
# 1.0.1, 2008/02/24 -- hacks by Pepper to use HandBrakeCLI and optimize for iPhone.
# 0.20070329.0 - initial release


#############################################################################
# globals

# const global variables
scriptName=`basename "$0"`
scriptVers="1.0.7"
scriptPID=$$
saveDVDinfo=1       # the DVD track info is also saved as .txt file when set
skipDuplicates=1    # if this option is off, overwrite existing files
E_BADARGS=65

if [[ $# = 0 ]]
 then
  myArgs='$HANDBRAKE_ARGS -Z AppleTV'
elif [[ $1 = "appletv" ]]
 then
  myArgs='$HANDBRAKE_ARGS -Z AppleTV'
  mySuffix=AppleTV
elif [[ $1 = "AppleTV" ]]
 then
  myArgs='$HANDBRAKE_ARGS -Z AppleTV'
  mySuffix=AppleTV
elif [[ $1 = "iphone" ]]
 then
  myArgs='$HANDBRAKE_ARGS -Z "iPhone & iPod Touch"'
  mySuffix=iPhone
elif [[ $1 = "iPhone" ]]
 then
  myArgs='$HANDBRAKE_ARGS -Z "iPhone & iPod Touch"'
  mySuffix=iPhone
else
 echo "Unknown format -- aborting!"
 exit 1
fi

# set the global variables to defaults
toolName="HandBrakeCLI"
toolPath="$HOME/bin/$toolName"
toolTrackArgs="-t 0"
toolArgs="-v"
inputSearchDir="$HOME/tivo-inspector/input/$mySuffix"
outputDir="$HOME/tivo-inspector/output"
minTrackTime="3"    # in minutes


# Pepper's sometime options:
# -s 1 # first subtitle
# -a 2 # second audio track ??
# For example (best within screen): HANDBRAKE_ARGS="-s1 -t11" iphone.sh


#############################################################################
# functions

parseProcessInArgs()
{
    if [ -z "$1" ]; then
        return
    fi

    toolArgs=""

    while [ ! -z "$1" ]
    do
        case "$1" in
            -h) displayUsageExit ;;
            --help) displayUsageExit ;;
            -i) inputSearchDir=$2
                shift ;;
            --input) inputSearchDir=$2
                shift ;;
            -o) outputDir=$2
                shift ;;
            --output) outputDir=$2
                shift ;;
            --minGetTime) minTrackTime=$2
                shift ;;
            *) toolArgs="$toolArgs $1" ;;
        esac

        shift
    done
}

verifyFindCLTool()
{
    # attempt to find the HandBrakeCLI if the script toolPath is not good
    if [ ! -x "$toolPath" ];
    then
        toolPathTMP=`PATH=.:/Applications:/:/usr/bin:/usr/local/bin:$HOME:$PATH which $toolName | sed '/^[^\/]/d' | sed 's/\S//g'`

        if [ ! -z $toolPathTMP ]; then 
            toolPath=$toolPathTMP
        fi
    fi  
}

displayUsageExit()
{
    echo "Usage: $scriptName [options]"
    echo ""
    echo "    -h, --help              Print help"
    echo "    -i, --input <string>    Set input directory to process all DVDs in it (default: /Volumes/)"
    echo "    -o, --output <string>   Set output directory for all output files (default: ~/Movies/mp4/)"
    echo "    --minGetTime <number>   Set the minimum time (mins) of the track/s to encode (default: 60)"

    if [ -x "$toolPath" ];
    then
        echo "   $toolName possible options"
        hBrakeHelp=`$toolPath --help 2>&1`
        hBrakeHelpPt=`printf "$hBrakeHelp" | egrep -v '( --input| --output| --help|Syntax: |^$)'`
        printf "$hBrakeHelpPt\n"
    else
        echo "    The options available to HandBrakeCLI except -o  and -i"
        if [ -e "$toolPath" ];
        then
            echo "    ERROR: $toolName command tool is not set up to execute"
            echo "    ERROR: attempting to use tool at $toolPath"
        else
            echo "    ERROR: $toolName command tool could not be found"
            echo "    ERROR: $toolName can be installed in ./ /usr/local/bin/ /usr/bin/ ~/ or /Applications/"
        fi
    fi

    echo ""

    exit $E_BADARGS
}

getTrackListLongerThan()
{
    # Two input arguments are are need. 
    #   arg1 is the time in minutes selector
    #   arg2 is the raw text stream from the track 0 call to HandBrake
    #   returns: a list of track numbers of tracks longer than the selector

    if [ $# -lt 2 ]; then
        return ""
    fi

    minTime="$1"
    shift
    allTrackText="$*"
    aReturn=""

    trackList=`eval "echo \"$allTrackText\" | egrep '(^\+ title |\+ duration\:)' | sed -e 's/^[^+]*+ //'g -e 's/title \([0-9]*\):/\1-/'g -e 's/duration: //'g"`

    trackNumber=""
    for aline in $trackList
    do
        trackLineFlag=`echo $aline | sed 's/[0-9]*-$/-/'`
        if [ $trackLineFlag = "-" ];
        then
            trackNumber=`echo $aline | sed 's/\([0-9]*\)-/\1/'`
        else
            set -- `echo $aline | sed -e 's/(^[:0-9])//g' -e 's/:/ /g'`
            if [ $3 -gt 29 ];
            then let trackTime=($1*60)+$2+1
            else let trackTime=($1*60)+$2
            fi

            if [ $trackTime -gt $minTime ];
            then aReturn="$aReturn $trackNumber"
            fi
        fi
    done

    echo "$aReturn"
}

makeFullPath()
{
    aReturn=""
    currentPath=`pwd`

    if [ $# -gt 0 ]; then
        inPath="$*"

        # put full path in front of path if needed
        aReturn=`echo "$inPath" | sed -e "s!~!$currentPath/!" -e "s!^./!$currentPath/!" -e "s!^\([^/]\)!$currentPath/\1!" -e "s!^../!$currentPath/../!"`

        # remove ../ from path - only goes 4 deep
        aReturn=`echo "$aReturn" | sed -e 's!/[^\.^/]*/\.\./!/!g' | sed -e 's!/[^\.^/]*/\.\./!/!g' | sed -e 's!/[^\.^/]*/\.\./!/!g' | sed -e 's!/[^\.^/]*/\.\./!/!g'`

        # cleanup by removing //
        aReturn=`echo "$aReturn" | sed -e 's!//!/!g'`
    fi

    echo "$aReturn"
}

isPIDRunning()
{
    aResult=0

    if [ $# -gt 0 ]; then
        txtResult="`ps ax | egrep \"^[ \t]*$1\" | sed -e 's/.*/1/'`"
        if [ -z "$txtResult" ];
        then aResult=0
        else aResult=1
        fi
    fi

    echo $aResult
}

#############################################################################
# MAIN SCRIPT

# initialization functions
verifyFindCLTool
parseProcessInArgs $*
# see if the output directory needs to be created
if [ ! -e $outputDir ]; then
    mkdir -p "$outputDir"
fi

# sanity checks
if [[ ! -x $toolPath || ! -d $inputSearchDir || ! -d $outputDir || -z "$toolArgs" ]]
 then

  if [[ ! -x $toolPath ]]
   then echo "ERROR: $toolPath is not executable!"
  fi
  if [[ ! -d $inputSearchDir ]]
   then echo "ERROR: $inputSearchDir is not a valid input directory!"
  fi
  if [[ ! -d $outputDir ]]
   then echo "ERROR: $outputDir is not a valid output directory!"
  fi
  if [[ ! -z "$toolArgs" ]]
   then echo "ERROR: $toolArgs is unset!"
  fi

    displayUsageExit
fi

# fix input and output paths to be full paths
inputSearchDir=`makeFullPath $inputSearchDir`
outputDir=`makeFullPath $outputDir`

# display the basic setup information
echo "$scriptName v$scriptVers"
echo "  Start: `date`"
echo "  Input directory: $inputSearchDir"
echo "  Output directory: $outputDir"
echo "  Minimum get track time: $minTrackTime mins"
echo "  Tool path: $toolPath"
echo "  Tool args: $toolArgs"
echo "  My args: $myArgs"
echo "  - - - - - - - - - - - - - - - -"

# find all the DVD videos in the input search directory tree
# spaces in file path temporarily become /008 and paths are separated with spaces
dvdTSVidList=`find $inputSearchDir -name VIDEO_TS -print0 | tr ' ' '\007' | tr '\000' ' '`

# process each DVD video found
for dvdTSDir in $dvdTSVidList
do
    # correct the tmp char back to spaces in the DVD file paths
    dvdTSDir=`echo $dvdTSDir | tr '\007' ' '`

    # get the DVD's name and path to root of the DVD
    dvdVolPath=`dirname "$dvdTSDir"`
    dvdName=`basename "$dvdVolPath"`
    dvdNameALNUM=`basename "$dvdVolPath" | sed 's/[^[:alnum:]^-^_]//g'`

    # display information
    echo "  * Processing DVD '$dvdName'"

    # create tmp link to the dvdVolPath to workaround a problem that
    # the HandBrakeCLI tool has a problem with spaces in the input
    # file paths in a script
    tmpNoSpacePath="/tmp/dvdVol-$dvdNameALNUM-$scriptPID"
    ln -s "$dvdVolPath" $tmpNoSpacePath

    # get the track list information from the DVD
    cmd="$toolPath -i $tmpNoSpacePath $toolTrackArgs /dev/null 2>&1"
    dvdTrackInfo=`eval $cmd`
    # save the DVD info
    outputFilePath="$outputDir/${dvdName}.txt"
    if [ $saveDVDinfo -eq 1 ]; then
        if [[ ! -e  $outputFilePath || skipDuplicates -eq 0 ]]; then
            echo "$dvdTrackInfo" | egrep '[ \t]*\+' > "$outputFilePath"
        fi
    fi
    # get the track number of tracks which are longer then the time desired
    trackFetchList=`getTrackListLongerThan $minTrackTime "$dvdTrackInfo"`
    if [ ! -z "$trackFetchList" ];
    then
        echo "   Will encode the following tracks: `echo $trackFetchList | sed 's/ /, /g'` "
    else
        echo "   No tracks on this DVD are longer than the minimum track time setting"
    fi

    trackCount=`echo $trackFetchList | wc -w`
    for aTrack in $trackFetchList
    do
        if [ $trackCount -gt 1 ]
            then outputFilePath="$outputDir/${dvdName}-${aTrack}.$mySuffix.m4v"
            else outputFilePath="$outputDir/${dvdName}.$mySuffix.m4v"
        # .m4v is important for AppleTV
        fi
        cmd="$toolPath $myArgs -i $tmpNoSpacePath $toolArgs -t $aTrack -o \"$outputFilePath\" > /tmp/${dvdNameALNUM}Results.txt 2>&1"

        if [[ ! -e  $outputFilePath || skipDuplicates -eq 0 ]];
        then
            # simple command execution
            #ripResult=`eval $cmd`

            # background command execution with some status

            echo "Command is: $cmd"

            eval $cmd &
            cmdPID=$!
            while [ `isPIDRunning $cmdPID` -eq 1 ]; do
                cmdStatusTxt="`tail -n 1 /tmp/${dvdNameALNUM}Results.txt | grep 'Encoding: '`"
                if [ ! -z "$cmdStatusTxt" ]; then
                    echo -n "$cmdStatusTxt"
                fi
                sleep 1s
            done
            echo ""
            wait $cmdPID

        else
            echo "   Output file SKIPPED because it ALREADY EXISTS"
        fi

        if [ -e /tmp/${dvdNameALNUM}Results.txt ]; then
            rm /tmp/${dvdNameALNUM}Results.txt
        fi
    done

    rm $tmpNoSpacePath
done

echo "  - - - - - - - - - - - - - - - -"
echo "  End: `date`"

exit 0

Wednesday, December 3 2008

New 24" Monitor

Inspired by Black Friday, I bought a Samsung 2433BW 24" LCD monitor. It's 1920*1200 -- basically a 1080p panel, but with DVI & VGA instead of HDMI & speakers. The thing is huge -- 75% more pixels than my old 1280*1024 pivoting Samsung (I should have bought 1600*1200 instead of a 912T). This seems to be as high a resolution as I can get, without going up to the much more expensive Dual-Link DVI 30" LCDs.

A few notes:

  • It's warm! I can feel the heat radiating in the corner of the loft. I noticed this with a 1600*1200 at Rockefeller a year ago. It surprises me that LCD panels radiate that much heat.
  • It's huge! It makes the 15" LCD on my MacBook Pro look sad (it's 77% larger).
  • The stand stinks. It was almost impossible to assemble without plastic lubricant, and Samsung phone support acknowledged the problem but had no useful advice. The stand is also low.
  • No dead pixels! I tested with ScreenQuery, a discontinued app for showing test patterns; I tried a few others, but they didn't work as well as the free ScreenQuery.

Monday, December 1 2008

Dotclear

Dotclear is a slick new blogging package. It offers multiple blogs and native SSL support. The software and documentation are still actively being translated from French to English, and rather slim, but usable -- and the developers are actively working on it.

I spent some time getting it set up, and it looks quite good so far. Now that I have a working installation, I'm going to try moving my movie reviews into DC, and setting up a new blog for interesting links.

Links

Configuration

  • Multiblog Configuration, Method #2
  • cd /var/www/dotclear2 && mkdir cache public && chown apache:root cache public && chmod g+w cache public
  • Make sure public_theme & public_path are right.
  • mkdir /var/www/dotclear2/public && chown apache:root /var/www/dotclear2/public && chmod g+w /var/www/dotclear2/public
  • mkdir /var/empty/empty

Post-Launch

  • Enable robots for each blog: Search engines robots policy
  • Enable XML/RPC per blog.

Wednesday, November 26 2008

HandBrake v0.9.3

HandBrake 0.9.3 is out, with many improvements. Unfortunately, the Linux binary is now built for Ubuntu. The Linux build instructions are a bit sparse. I needed jam & yasm RPMs from http://rpmfind.net/ I didn't find the list of HandBrake's Linux dependencies until later.

Note: In v0.9.3, make drives both configure & jam. jam drives wget, and my .wgetrc was somehow preventing the downloads from actually working (user-agent rejected by http://download.m0k.org/?):

pepper@inspector:~$ cat wgetrc.disabled 
reject = gif,txt
noclobber = on
robots = off
tries = 2
user-agent = Mozilla/4.77 (Macintosh; U; PPC)
#background = on
#recursive = on
quiet = on

Additionally, CentOS was missing the unversioned libbz2.so (arguably a bug), so I needed a symlink: "cd /usr/local/lib ; ln -s /usr/lib64/libbz2.so.1 libbz2.so". Reported to CentOS, since I haven't verified it on Red Hat, but I'm sure it's the same.

Tips for HandBrake debugging, and seeking help.

This got me HandBrakeCLI (I don't care about the Linux GUI -- I use the Mac GUI; and only do batch processing on Linux). Now to reconvert a few things!

PS-See also my hacked HandBrake wrapper script -- I expect to post an update which uses the new presets after I've tested a bit.

Sunday, November 23 2008

Put the Drupal down!

I got Drupal 6.6 installed, and then someone suggested Acquia Drupal (more bundled modules, and it integrates with Acquia's brand-new network services and support). So I installed that. Not too bad, but definitely more involved than my original hosted one-click WordPress install.

It's doing SSL and I have Apache mapping a couple vhosts to the main Drupal installation per http://drupal.org/getting-started/6/install/multi-site, but I don't want to deal with a mix of shared and unique MySQL tables across sites.

Drupal's blogging is pretty basic, and it's not immediately obvious how to promote vhosts to top-level virtual hosts (probably mod_rewrite, which seems pretty central to Drupal). I'm not sure how easy it would be to hide the non-blog parts of Drupal I'm not interested in, and there's no WYSIWYG built-in. I like to write raw Markdown or HTML, but one of my goals is to make it easy to host blogs for friends & family, who generally want WYSIWIG. I found YUI Editor, but Yahoo is pretty confused about it today, and the instructions are lacking.

So for now, it looks like I'm going to set up 3 more personal WP blogs, and hope that in a year or two either WP-MU gets better SSL support, or Movable Type becomes more attractive. I guess I should try DotClear first, as they tell me it handle logins for multiple blogs through a shared site.

Monday, November 10 2008

Apparently I'm Not Going to Use Textpattern

I've been looking at migrating from WordPress to Movable Type, but been stalled for a while. Recently, Kevin van Haaren mentioned Textpattern, and I was impressed by its simplicity. Unfortunately, it doesn't yet look sufficiently mature. Among other things, Textpattern cannot import from a WordPress WXR file. Movable Type can (with some massaging), even though WordPress generates invalid XML.

Textpattern can only import from a live WP database, and I don't want my test server to have access to the live server. I set it up for a test, and Textpattern threw several types of errors.

Additionally, I couldn't see Textpattern's default comment, even before importing. The import caused me to abort before I got to investigate this one, though.

So back to MT, which has annoying limitations around blog management, but no dealbreakers so far.

Sunday, November 9 2008

yum Killed Email, Again!

I ran "yum update" before bed last night to update my CentOS (RHEL) 5.2 system. Unfortunately, I didn't notice that email stopped flowing. Cyrus IMAPd was running, but no new mail was coming in. postfix was accepting mail, but new mail wasn't reaching Cyrus. Melissa let me know that mail wasn't working, and I discovered that again, the update had broken amavisd-new.

yum & rpm don't coexist well with cpan, and cpan itself has lots of trouble with Compress::Zlib. After a bunch of poking around, I eventually got cpan to reinstall (or just properly recognize?) Compress::Zlib, and got amavisd-new restarted; then postfix delivered the outstanding mail.

The whole cpan dance was prolonged and complicated by long cpan timeouts attempting to contact FTP mirrors -- presumably due to firewall restrictions that I never noticed because I avoid FTP. After a bunch of futzing with "o conf urrllist", I expunged the FTP mirrors and replaced them with HTTP mirrors; I was then eventually able to reinstall the necessary modules.

Thursday, November 6 2008

On the Job Market

I got caught up in Goldman's layoffs Wednesday. I hope to have a new job before the severance runs out, but if you are (or know someone who is) looking for a Linux/Solaris admin in the NYC area, please let me know.

My interests include writing & documentation, networking, Mac OS X, and open source.

My resume is online.

I actually wish this had happened a day earlier. It was a big comedown from the Obama victory the night before; in the reverse order, winning back the country would have been a nice counter to the layoff news.

Wednesday, November 5 2008

AT+T Cannot Forward a Phone Number

I need a new phone number for a few months, to forward to my cell phone. Apparently AT&T has an institutional blind spot around forwarding, though. I called AT&T Wireless, and had to explain myself several times (over 25 minutes). She was unable to help, so transferred me to her manager, who could not help either. The Wireless manager forwarded me to AT&T Long Distance (800 222-0300).

After much repetitive explanation, LD said they were unable to help me because I wasn't asking about a long distance number. LD forwarded me to AT&T Local Service (800 288-2747).

The Local sales rep said she(?) was unable to help, and that I needed to speak with AT&T Wireless (since, after all, I want to forward to an AT&T iPhone). I explained that I did not want to speak to Wireless, as they had already explained they can only forward my cell phone to a different number, rather than forwarding a new number to my cellphone. She asked if I had been helped to my satisfaction (part of the script), and I explained I had not been helped yet, and didn't even know who she was about to pass me along to. She apologized, assured me the next group would be able to help, ignored me as I asked (twice) who she was transferring me to, and transferred me back to Wireless. Obviously she wanted to be rid of me, and no longer cared if I was helped.

The next Wireless rep understood better, at least. She checked with 4 managers, and confirmed the Wireless group does not offer forwarding into cellphones. When asked, she passed me on to Wireless Business Services (800 999-5445) -- unfortunately it was Wireless Business Services, and they were confused by my number not being tied to a business account.

Eventually the BWS rep offered me a new wireless contract for $10/month, with a $175 early termination fee any time between 30 days and 2 years. This would get me a new number, and could be forwarded to my existing iPhone. $240 seems like a lot for something I only expect to need for a few months. Interestingly, the residential Wireless reps had mentioned this, but seemed rather confused about it -- among other things, they said the termination fee only applied within the first 6 months. $60 for 1-6 months seems pretty good in comparisoin!

Not an impressive performance, considering I spent over an hour on the phone with Customer Service.

Now I'm looking at the various $10/month web-based forwarding / phone management services. Recommendations welcome (during November)!

Tuesday, November 4 2008

Flash: Broken, Not Just Evil

Flash sucks even worse than I thought. The Never Ask Again checkbox is not honored in Flash 9.0 r124 (under 10.5.5/Safari 3.1.2), which breaks Flash videos (pretty much Flash's only real reason for being, aside from advertising spyware). I reported this to Adobe, but don't expect much.

Note that I end up with dozens of stacked dialogs, and they pop back up about one per second, so hulu clearly won't work until Adobe fixes this bug, or I give up and turn on Flash cookies (which I have no intention of doing).


******BUG******

Concise problem statement: Flash does not honor setting to not store cookies or local data.

Steps to reproduce bug:

  1. Visit http://www.macromedia.com/support/documentation/en/flashplayer/help/settings_manager03.html
  2. Set Global Storage Settings to None.
  3. Check Never Ask Again.
  4. Uncheck "Allow third-party Flash content to store data on your computer".
  5. Restart Safari.
  6. Visit http://www.hulu.com/, or http://widgets.nbc.com/cscallback/urlexchange/ (a Sarah Palin SNL clip), or http://www.nytimes.com/interactive/2008/11/04/us/politics/20081104_ELECTION_WORDTRAIN.html

Results: Many many annoying Flash dialogs asking for permission to store data (1 on NYTimes; many on the videos, which interrupt playback).

Expected results: When I check Never Ask Again, I should get NO dialogs asking to store Flash cookies, and no Flash cookies should be stored.

In addition, I unchecked "Store common Flash components to reduce download times." But when I relaunched Safari, it was unchecked. I relaunched a second time, and this setting stayed checked. Never Ask Again was still correctly checked, but I still kept getting obnoxious cookie prompts from hulu.


Update: Leonard Rosenthol informed me that Flash 10 is out. It's a bit hard to find, but does appear to solve the problem. I upgraded, relaunched Safari, reset the option, relaunched Safari, and no longer get those stupid prompts. Hooray! It's odd that Adobe hides it so effectively (it's not even on Adobe's main update page), and the built-in Flash updater doesn't offer it; unfortunately Apple hasn't delivered it to Mac OS X users yet.


Update 2 -- 2008/12/25: Disabling local data (cookies and local caching) prevents Flash videos at Flickr and other sites from playing (I just get a black rectangle of the right size -- no change or controls). YouTube/Google video is not affected, though. I'm not re-enabling Flash cookies, though.

Sunday, October 19 2008

Flash Suckage: Eat Your Cookies!

Update 2004/11/08 (Obama Day): It's worse thane I thought. Not only are these features and controls hidden and almost impossible to find, but the controls don't actually work.

Flash Cookies: The Silent Privacy Killer disturbed me. Then I visited the Flash Player Manager page, saw all the sites that were tracking me via Flash cookies, and felt really freaked. I disabled Flash cookies, and encourage you to do so as well. I'm not yet sure how well it worked, as I just checked and saw an unwelcome Yahoo cookie, apparently added after I attempted to disable them.

While people are aware of HTTP cookies, I had never heard of Flash cookies before. HTTP cookies are legitimately required by various sites, but I don't believe Flash cookies serve any user purpose, and they don't seem to be required for any reason I care about, so I don't expect problems from clearing and disabling them.

Tuesday, September 30 2008

Outlook Annoyances: September 2008 Update

I've been using Outlook heavily for a while now, so I have found many flaws. I realize Outlook 2007 fixes some of these, and includes its own issues. Note that I have complained about most of these here -- this post is a consolidation & update.

Filtering

  • Outlook Options (program-wide preferences) is not available from message windows -- only from the mailbox viewer window. This is true in Thunderbird/Mac too; presumably copied from the MS model. Dumb.
  • Outlook's filtering isn't very good. It is not flexible enough -- no booleans & very limited criteria available.
  • Two filters that match the same message will typically "copies" the message to two different different mailboxes. This is confusing -- why does the first match move (from Inbox) while the second copies? There is a "stop filtering" action, but turning it on for each message is a waste of time and precious filter ruleset bits.
  • I hit the accursed 32kb rules limit. Despite this post, I am capped at 32kb total in rules (client-side + server-side). Apparently this limit is raised to 256kb if the client is Outlook 2007 and the server is Exchange 2007. Someday, perhaps.
  • Some filter actions are handled on the server, while others (particularly flagging) are only performed on the client. This complicates rules unnecessarily, and creates inconsistencies if the PC is shut down or logged out. PCs need to reboot relatively frequently for patching and updates.
  • I thought it was cool that Outlook checks group memberships automatically when filtering. Unfortunately this cannot be turned off, so it's quite awkward to filter messages sent to a list separately from messages sent to a member of the list.
  • The pop-up window for new mail appears before filters run, so I am notified of new mail. I don't want to see.
  • If I click on such a notification, Outlook shows me the message, marks it as read, and then filters it.
  • I have several filters that match on Subject strings, but Outlook only offers "contains" (no "starts with" or "does not start with"), so I cannot specify original messages, and distinguish from replies/forwards.
  • The filter area shows a list of criteria with checkmarks at the top with blue underlines under the keywords. The bottom shows the same labels, with the same blue underlines. But at the bottom they're "links" to dialog boxes for entering the criteria, while at the top the same "links" aren't clickable. Way to misuse a visual cue, and do it in the most confusingly inconsistent way possible!
  • Oh, and the rules dialogs are all modal, so once I start creating a rule, I cannot open candidate messages for comparison against the rule.

Display & Windowing

  • I find the Reading pane indispensable, because Outlook is slow to load messages into new windows.
  • Inbox does not appear at the top of the mailbox list!
  • Outlook keeps forgetting my favorite mailboxes.
  • When I relaunch (or launch on another PC), Outlook loses my view settings.
  • Outlook does not remember open messages (including drafts) across launches. In fairness, Eudora is the only client I know that does this -- apparently Notes does too.
  • I can make Outlook sort a mailbox with the newest messages at the bottom (the default is newest-at-top), but then when I click the mailbox, Outlook selects the bottom (newest) message in a mailbox, whereas I want to read oldest first.
  • Outlook can allow commas as address delimiters (which is what they are in the actual mail messages, as part of the SMTP standard), but then I cannot type names, because we have autocomplete disabled and Outlook doesn't recognize a correct "Last, First" recipient when it's set to allow commas as delimiters (even though Outlook uses them once I click Check Names).
  • I tried to keep my mailbox "caught up" or "clean" (all messages read). Outlook doesn't mark a message read until I deselect it. This meant that when done, to have it stop showing that last message as unread, I had to select something else.
  • When reading a bunch of new messages, and Outlook selected one I didn't want to read, I had to click another message to get the Mark Unread menu command available; then scroll up to the top and click on the first message to continue.
  • Now, after a few problems where Outlook marked a message read before I was ready, I have it set to never mark messages read when I read them in the Reading pane. As a result, I spend more time than I really want manually marking messages read, but overall it works better this way.
  • I can collapse a thread to easily mark the whole thing as read, but then I have to expand it again before Refresh will hide it from my fresh search mailbox.
  • Refresh (F5) doesn't work. I have refreshed as many as 10 times without having the list of messages become correct.
  • When I delete multiple messages and some are out of view, Outlook seems to select the top message in the current window. The "next message" does not depend on where the scroll thumb is! Ironically, iTunes/Mac has a similar bug.
  • In Conversation view (which would be a lot more useful if it didn't waste 2 messages worth of space per "Conversation"), if I use down-arrow to select the next conversation, it expands the conversation instead. Use left-arrow/right-arrow to collapse/expand conversations -- they aren't needed for mailbox navigation! At least Control-KP+ expands all conversations.
  • I cannot find a good way to sort threads by date; I'd like every thread (perhaps every thread with new messages) grouped together, with the messages in each thread sorted internally by date, and the threads sorted by date (typically of the first message). In Outlook I can group "Conversations" by Subject: or group by From: line, but new messages/threads keep showing up at the top of the mailbox. This makes it very difficult to work through recent mail in chronological order, as not only do new things keep popping up at the top of the list, but additionally existing threads move up when someone replies to them. I find myself constantly working down from the top, towards some unmarked line which divides "stuff I've at least initially processed" from "earlier stuff I haven't yet seen" (much of which has to be deleted); when I finish scaning the second group, the invisible line divides "stuff I've at least initially processed" from "older stuff I've at least initially processed, which definitely needs attention at some point". Not a good way to work, but the best I've been able to manage in Outlook so far.
  • I have figured out more what's wrong with Refresh. First, I have to hit F5 repeatedly to make Outlook clear more and more read messages from unread-only views; second, collapsed conversations are not cleared; I have to expand them out and then hit F5 again. This is particularly annoying because Outlook has such a strong tendency to always keep one message selected and thus read (although it's not marked read, so I cannot simply mark it unread; I have to mark it read, then mark it unread, and then make sure Outlook doesn't preview it again), so it's quite difficult to reorganize a mailbox and get to a "clean" view (only new messages/threads) without losing some messages which Outlook insisted on selecting/previewing/marking read while rearranging.
  • Sometimes when I delete messages, Outlook selects the next message down (which is correct, given that I view messages in ascending date order). Other times it selects the top message in the mailbox, which is only the right thing to do if it's the last message in the mailbox. This inconsistent behavior may be connected to whether any off-screen messages are selected, but that shouldn't matter. I shouldn't have to wonder where the selection will go, or try not to select messages across more than one screenful at a time, or rush through selecting and deleting mail or collapsing threads, for fear of a new message coming in, removing my selection, and selecting, previewing, and (almost) marking one of those messages read, before I had a chance to delete, mark, or collapse it. This means that if client-side filters are active, the user must wait after launching Outlook, until it's finished filing messages into the current mailbox, as new messages will constantly disrupt the selection until Outlook is finished running client-side filters. Even if Outlook has been running a while, it's easy to select a few messages for processing, be interrupted by new mail, deal with whatever was previewed and start selecting again, be interrupted again, and have to deal with the second undesired selection/preview before attempting to return to manual selection for managing email. Amazingly frustrating, and a great way to 'lose' unread mail.
  • With a multi-monitor setup, the best way to use Outlook is with the mailbox filling one display and the attached preview pane covering most of another display. Unfortunately, as I select different mailboxes, the preview switches off. Each mailbox remembers its own preview arrangement, which is important because I generally only read messages (via preview) in my "fresh" search pseudo-mailbox. In other mailboxes, clicking a message shouldn't mark it read, so preview (the "Reader pane") is a bad thing everywhere except in "fresh" (and often in "fresh", as well!). Aggravating matters, sometimes (mostly when I lock the PC) Outlook turns off the preview and shrinks the mailbox to subtract the preview area. This leaves the mailbox covering one display but not extending onto the next. When I switch back to a mailbox with preview on ("fresh"), the preview reappears one character wide, rather than covering most of the secondary display as before. Very aggravating -- I now use one window for "fresh" and one for other mailboxes, and gave up on windows larger than one display.
  • In Outlook, it's impossible to determine directly what mailbox a message is in from the message window. I find myself searching across all mailboxes by title (using LookOut, which is no longer supported and not standard) -- fortunately LookOut identifies the containing mailbox, although it often cannot open the messages it finds!
  • It's also impossible to open a message in its own window from the reading pane; the workaround is to Shift-Tab back into the mailbox window and then hit Enter to open the message window from there. Dumb.
  • I often want to delete a collapsed thread ("Conversation"). Outlook insists on expanding it first, which wastes time and often results in unread messages appearing and then being deleted -- disconcerting, as it gives the impression I'm losing important (unread) mail. Worse, Outlook cannot mark a thread unread without expanding it, which moves the selection into the thread and marks that one or two messages read when deselected (unless the selection lasts a second or less, as I had set Outlook to mark messages read after a second, because I cannot eliminate the delay, and above one second it doesn't automatically mark short messages which I read quickly as read; I had to go back and mark them read manually later). If I had just read a new message in threaded mode, and wanted to mark it unread, I had to either hit Control-Q to mark it read or move to another message and back (assuming I've had the current message previewed for at least a second), mark it unread (Control-U), then hit left-arrow to collapse the thread.
  • Worse was marking a whole thread unread (more common) from the keyboard. Then I had to collapse the thread to implicitly select the whole thing, hit Control-U to mark the whole thread read (implicitly expanding it), and then hit left-arrow within a second to collapse it again before Outlook decides I've read a message in preview. Now that I mark everything read manually, I still need 4 keystrokes, but at least the last one isn't racing the clock.
  • When I delete a message, Outlook immediately selects (and in "fresh" previews) another message. When I'm reading mail, this is generally what I want, so I can deal with the next message. When I'm trying to delete or file mail, it means Outlook automatically starts the process of dealing with another next message, and unless I'm very quick marks it read as soon as I deselect, so I must decide what to do about the new selection. This makes it harder to stop reading mail in the current mailbox, as every time I complete an action, Outlook picks the "next" message and engages me in dealing with it; stopping without losing unread status on a message I haven't actually read yet requires contortions. When I know I'm about to stop, I tend to deal with a message or thread and then hit Control-up-arrow to jump to the top of the mailbox, which should be the first message I read (so already marked read), but is often a new message that's come in recently; I then have to decide on and handle that before I can move on to another mailbox or activity.
  • Normally, when a thread ("Conversation") is collapsed, Outlook deselects its messages. Sometimes (unpredictably) it still shows the preview for a hidden message, which breaks the Control-Q Control-U left-arrow dance, and I have to instead hit Control-Q Control-U left-arrow up-arrow to get a collapsed unread thread.
  • F5 (Refresh) doesn't clear collapsed conversations; this is annoying. On the other hand, sometimes messages disappear immediately upon being marked read, which means I don't even get a chance to mark them unread; they're effectively just gone. I have no idea what triggers the second problem; fortunately it's not that common, as it tends to result in losing mail -- often mail I was saving for later attention.
  • If I have a message which has been previewed for over a second, I know it's effectively read (it will be marked as such as soon as I deselect it, unless I drag it into another mailbox first). It would be good if I could use Control-U to tell Outlook not to mark this message read as soon as it gets deselected, but instead I have to mark it read, then mark it unread, and then move away within a second -- before the preview timer marks it mostly-read again.
  • Outlook cannot select multiple mailboxes at one time, which is ungood; on the other hand, it makes an effort to be helpful -- when I select a mailbox, it kinda-sorta move the selection into the message list (which is pretty reliably what the user really wants, since you can't do much with mailboxes except delete or move them). It's all a bit confusing.
  • The flag column is at the right side of mailbox list windows, and cannot be moved. I'd like to move it left, to be alongside the other important columns, but cannot.
  • In general, there are many customization options for mailbox display, but the controls are all over the place, so hard to find and confusing to use.

Things I like

  • Outlook can show me mailing list (group) membership. It's called "Outlook Properties" in the menu, despite being maintained on the Exchange side, but after I got over thinking that couldn't be the right place, this is quite handy.
  • The setting to view all received email as plain text is amazingly useful; there's a handy option at the top of each message window to see formatting.

Searching

  • There are at least 4 different types of Find. Control-F in some places, F3 in others, F4 in others, and LookOut (now Live Search). Sometimes I hit Control-F to Find, and Outlook forwards a copy of the message instead.
  • I cannot Find unless the selection is in the right part of the message window. Amazing!

Super-Modal

  • I have to use View Options to see full headers (in the Message Options window), but while that window is open, the main Outlook program is visible but completely unresponsive. Alt-Tab doesn't show Outlook windows, and I cannot access the others until I close. I thought Outlook had crashed, until I realized it was accessible again after I dismissed Message Options.
  • Similarly, emptying Deleted Items (which must happen every time I quit) locks up the whole program until done. In fairness, other programs empty the trash modally, but don't necessarily force it at quit.
  • Every single time I Logout, Windows tells Outlook to quit. Outlook starts emptying my Deleted Items mailbox, and I watch the race. Then I normally cancel the logout so Outlook can finish -- it's almost never fast enough to complete the operation before the Windows timer runs out. This means automated reboots, as for patching, are crashing Outlook while it's in the middle of modifying my mailbox on a regular basis. I guess this doesn't cause huge amounts of grief because everything in Deleted Items is expendable, but lame lame lame.

Keyboard

  • I found a couple pages of Outlook keyboard shortcuts (the online help lists shortcuts too). Many are standard Windows shortcuts, but a few were new and useful for me.
  • Now that I use Thunderbird at home and Outlook at work, it strikes me how similar Thunderbird is, even extending to some misfeatures (design flaws, not quite bugs), such as over-using the Esc key. Thunderbird has a few notable improvements, such as being able to mark messages Read and Unread with the M key, instead of Ctl-Q/Ctl-U, or S to flag messages (stored as an IMAP tag; this shows up in Eudora as Label 15).
  • In both Outlook and Thunderbird, Esc closes message windows; this is inconsistent with all other full windows, which are closed with Command-W/Control-W, and makes messages feel particularly ephemeral. In Outlook, when I open a received message and hit the Space bar to scroll to the next page (which works in every other email client and browser I know), it instead inserts spaces at the beginning of the received message, which is completely absurd.

Misc

  • AutoCorrect absolutely would not let me type "SAs" (System Administrators) until I killed it.

Sunday, September 28 2008

Windows Annoyances: September 2008 Update

I've been using Windows regularly for a while now. I have many gripes. I realize Vista and Office 2007 fix some of these, but I'm pretty sure they'd be aggravatingly slow on my desktop PC, where Windows XP isn't. Note that I have complained about most of these here -- this post is a consolidation & update.

Windows

  • In the Alt-Tab task switcher, I cannot Hide (Command-H), Quit (Command-Q), or click an application's icon to switch directly to it. This is aggravated by the fact that icons in the switcher often correspond to windows rather than on the Mac, where they correspond to applications (each with one or more windows), so there are many more icons to Tab through, and often several indistinguishable windows (4 Firefox windows generate 4 identical unlabeled icons; so do 4 open messages in Outlook). Considering Windows has had this feature for longer than Apple, it's shockingly lame.
  • I cannot hide the current application (window) from the keyboard (Command-H), or Hide Others (Command-Option-H).
  • I want to drag icons onto applications. The Start bar, Windows' equivalent of the Dock, doesn't accept dragged files -- Microsoft knows it should because it pops up a message when I try, but they didn't make it work.
  • I want to drag proxy icons from document windows into other windows (typically Outlook messages). Proxy icons seem like such a minor thing, but Apple introduced them years ago, and I feel the lack in XP keenly.

Text (UltraEdit, UltraCompare Lite, & kate)

  • I miss BBEdit. UltraEdit is very good, but BBEdit has many advantages, and I haven't yet encountered any UE advantages (although presumably there are some). I use UE and vi daily, and kate periodically -- when it's faster than vi but not worth bringing the files back to my desktop, or pointing UE to my account on a UNIX box.
  • kate is broken in several ways, although I'm not sure how much to blame Exceed. Specifically, Copy & Paste are unreliable (sometimes they both work, sometimes only one direction works, and sometimes neither works!). Shift-selection from the keyboard doesn't work. Tip: In Exceed, assign the left modifier keys to X11 instead of Windows.
  • I really miss the bbedit command, particularly for opening local and sftp files from the command line. I miss being able to bbedit in a loop!
  • In Windows, Control-Backspace works in some places, but inserts non-printing ('box') characters in others.
  • I see that UltraEdit offers 4 different flavors of Find & Replace: Plain, perl regex, UNIX regex, and UltraEdit regex. This seems crazy to me -- I consider anything that's not 100% backward-compatible with PCRE a bug, but I am not selling to a population of users who live in MS Word. BBEdit's is based on PCRE.
  • BBEdit's Find Differences is much better than either UE's (both files must be in a rooted window, and UltraCompare Lite is confusing and awkward). kate uses kompare, which is pretty weak. Neither kompare nor UltraCompare is very well integrated. BBEdit v9 makes major improvements to Find Differences.
  • UltraCompare Lite (like UltraEdit) displays documents inside a root window, which makes comparing across 2 monitors a pain, since the root window must be manually sized to cover 2 displays.
  • UltraCompare Lite doesn't have BBEdit's (new) character-level comparison, which I've wanted for years.
  • UltraCompare cannot compare unsaved windows.
  • UltraCompare's merge functionality is very confusing -- there are lots of obscure buttons, and some simply don't work. Manually copying from one document to another is a pain, as it requires a switch to UltraEdit (the UC docs aren't editable) and a re-compare (leaving the original one open) to get a fresh comparison.

Internet Explorer 7

  • I can increase the font size, but if I do the window grows beyond the window size -- there's some brain-dead full-page magnification, rather than the classic (and useful) text size increase.
  • If I increase the font size, selection highlights no longer align with the text I'm dragging over. It's bizarre.
  • In PDF tabs, Control-W doesn't work; I find this extremely aggravating, but fortunately Command-W works fine in Safari (even with the Acrobat plug-in).
  • I still hit Control-L to select the URL for copying, and IE7 still fails to do it, bringing up a blank URL entry dialog, instead of selecting the URL in the current window as Safari & Firefox do. I shouldn't need to use Alt-D instead to select the current URL. That Control-L window has no reason for being.

Misc

  • MS Office Communicator flashes in the Start bar, many times, whenever someone types to me (unless that window is already in the foreground). It's very distracting -- on the Mac I tend to turn this off, or set it to only once. For some reason it annoys me more on Windows.
  • When someone sends me a message in Communicator, the "buddy list" window's icon flashes in the Alt-Tab task switcher, rather than the updated conversation.
  • I open a bunch of programs whenever I log in, so I created a batch file to launch several applications and PuTTY sessions. Unfortunately, it launches one application and then stops. In a UNIX (Mac) shell, I'd use a trailing & on each command, so the script could continue to the next one without waiting, but I was unable to find a working equivalent for .bat files.
  • The inevitable overnight reboots are quite disruptive. They make me appreciate the Mac more, where this kind of thing is less common, and the system (Finder) makes an effort to resume where I left off.

- page 1 of 11