Extra Pepperoni

To content | To menu | To search

Tag - Linux

Entries feed - Comments feed

Wednesday, March 12 2008

Extra Pepperoni Re-Hosted

After DreamHost's breach 8 months ago, I was aggravated at their poor handling of the situation, but willing to give them the benefit of the doubt, and still happy with their low prices and flexible services.

With the new bad news and worse confirmation (still with poor incident handling), though, it's time to get out of dodge.

I have moved Extra Pepperoni back onto my own hardware. I started blogging on Apple's Blojsom install, but gave up on Tiger Server for Blojsom (and Mailman) because the services kept silently shutting down, leaving me to notice they were disabled days or weeks later (no fault of Blojsom or Mailman -- Apple didn't do a good job porting SpamAssassin either). Bringing up a WordPress blog and mailing lists at DreamHost was easy and cheap, but that's no good if they are unsafe.

I'll look at moving a couple very light-duty Mailman lists off DH next, but the lists are so lightly used I'm not too concerned. There just isn't any confidential information on the mailing lists, aside from their tiny subscriber lists.

Ah, well. I now know much more about WordPress and MySQL than I cared too, but the setup wasn't too bad. I hadn't realized how many customizations and tweaks I made to WordPress until it came time to recreate them on my own system:

  1. Almost Spring theme (included by DreamHost); with minor hack
  2. PHP Markdown Extra; with minor hack
  3. MySQL admin UI
  4. WP-DB-Backup (DH included one, which I'm no longer using)
  5. mod_rewrite for permalinks
  6. Admin-SSL, with "Shared SSL" tweak, integrated into my existing SSL site (meaning EP is available through two different "sites", and I have to keep the Apache configurations reconciled)
  7. Twitter
  8. WP-Cache (DH standard)
  9. Akismet anti-spam registration
  10. Technorati pinger (came over automatically with the DB).
  11. Fix for widget.php to use legal JavaScript tag.

Tuesday, February 19 2008

reppep.com Migrated

On Feb 19, 2008, I shut down the old reppep.com server, which ran Mac OS X 10.4 "Tiger" Server, and replaced it with a new (cheaper and faster) PC running Linux. Unfortunately, the password formats are incompatible, so I apologize to app reppep users for the disruption.

Please call me if you have an account on reppep.com and haven't received your password already, or find anything not working right.

I switched from Apple's jabberd to Openfire, which doesn't use the UNIX system accounts, so let me know if you want a chat account (compatible with iChat & GTalk).


[Done] I forgot SquirrelMail address books -- should be able to bring those over too.


  • Firewall problem fixed. SMTP MX issue fixed.
  • Virus filtering problem fixed.
  • Webmail certificate fixed.
  • Quota problem fixed.
  • Virtual domains for email fixed.

As of 5pm, I don't know anything that doesn't work (aside from SquirrelMail address books) [fixed Thursday].

Thanks for your patience!


As of 10:30 on the 20th, things seem to be working. Something's screwy with amavisd-new's quarantine, but mail is going through. I reinstalled Openfire, and chat seems okay under the correct hostname/certificate name now (will try signing it as ca.reppep.com later).

Good timing -- the optical drive on the old server died tonight.

I have distributed all the new temporary passwords, so any users having trouble logging in should let me know.

Markdown.cgi is still broken, but I'm the only person who uses it here, so I'll get to it.


On Thursday the 21st, I found a problem with amavisd-new -- it had quarantined 32,000 messages in a single directory, and was stuck (apparently ext3 doesn't support more than 32,000 files in a directory). I cleared it out and finally managed to disable quarantine, which wasn't as easy as it should have been, and the backlog of messages have been delivered as of 9:15pm.

At 11pm, I fixed an issue preventing SMTP AUTH from working properly, which was interfering with sending email to non-reppep addresses.

Saturday, December 8 2007

Upgrading from Tiger Server to Linux

For over a year now, I've been following the development of Mac OS X Server 10.5 Leopard and testing betas, and anticipating upgrading reppep.com from Tiger Server on a dual 1.25GHz Power Mac G4 to Leopard Server on a dual 2GHz Power Mac G5. Over the weekend I had a change of plans, though.

Although I support Mac OS X Server at Rockefeller, I don't recommend it for most requirements, as Linux compares favorably for transparency (some of the MOSXS internals are unique and poorly documented), server software compatibility (although Macs are quite good here too), and price/features at the low end. A Core Duo Mac mini has plenty of juice to saturate our 768kbps/3mbps DSL circuit, but adding a couple drives more than doubles its price, and Apple's software RAID is quite broken; Linux software RAID is apparently quite good; I might eventually switch to hardware RAID. An Xserve is a great piece of hardware, but it's a bit exotic and I can get a fast generic PC cheaper; I don't want all the high-end features for a box that sits in our apartment.

Additionally, I've read perhaps 600 pages of docs on Leopard Server, and had at another 400-1500 yet to go. This is an investment I was finding hard to justify. The migration process is quite complicated, and Apple doesn't support migrating accounts from a Tiger system to a Leopard system -- I don't want to do an upgrade. I could clone the G4 to the G5 and upgrade it there, but I prefer to handle upgrades as scratch installations with manual migration of applications, so I know exactly what's been done. A lot of this is masked by upgrade procedures.

As part of this, I've decided to invest a bit more time in learning RHEL5 -- we have a couple systems at Rockefeller, but not much in production yet, and now seems like a good time to dig in some more.

Fortunately, all the services I've been using on reppep.com are available on Linux (and FreeBSD), so aside from another incredibly inconvenient password change cycle (for which it is arguably time anyway), the switch should be largely transparent to reppep.com users, although I still have plenty of research to do.

A brief timeline of reppep.com

  1. 1999: I left the National Audubon Society, and bought the Power Mac 7300 with accelerator card I'd been using there. I set it up with LinuxPPC and Apache, and started offering free web hosting to friends & family. LinuxPPC was eventually discontinued.
  2. I upgraded from LinuxPPC to Yellow Dog Linux, which was better than LinuxPPC, but had serious flaws.
  3. 2001: I was working on a couple remote FreeBSD machines (as admin of the Info-Mac server, and a user on the Apache Software Foundation userhost), and decided to learn more; I bought a cheap Celeron PC and installed FreeBSD 4.3 (IIRC); I upgraded through about v5.1 and a Pentium 4 (giving the Celeron box to the Info-Mac Archive, where it became the Info-Mac server for a while). I learned a lot about FreeBSD and UNIX in general, but eventually realized I was investing more time learning FreeBSD than I could justify. The best thing about FreeBSD is not a technical feature, but rather that the user community is so rich with knowledge. Reading the FreeBSD-STABLE list was amazing, as there was so much depth, freely shared with the community. While running on FreeBSD, I added mail services to the web services I had been offering. Note: Disruptions to personal email service are much worse than problems with personal web service.
  4. 2005: It became clear that I needed anti-spam, so I began researching SpamAssassin. While I was figuring out how to build the SMTP sandwich, with a public untrusted Postfix listener on port 25 & 587, and a filter, and then a listener on a high port like 10025 to accept and deliver mail to actual users, I installed a beta of Mac OS X Server 10.4 "Tiger", which had the whole thing implemented, plus ClamAV as a bonus. I started testing heavily before the release, and switched to MOSXS 10.4 shortly after it was finalized. It's been very good, but as time has passed, I've had more and more problems. In particular, Apple chose to use Cyrus as an IMAP/POP server, and Cyrus is complicated, but Apple ignores the complexity; this can make troubleshooting impossible. The SpamAssassin installation is slightly broken; it's a bit too old to offer the newer SpamAssassin self-upgrade mechanism. Server Admin is great, but has a bunch of bugs around SSL certificates, some of which destroy the certificates. Blojsom was nice, but Apple's installation was very unstable; I eventually moved my blog to WordPress hosted externally.
  5. 2008: I intend to switch to CentOS 5.1, which is basically a (legal) no-charge clone of Red Hat Enterprise Linux 5.1. This should make future upgrades a bit more straightforward, as I won't have to deal with Apple's Open Directory (OpenLDAP); it will also give me a bit more experience with RHEL5, which is a better investment for my time than Leopard Server.

Wednesday, June 20 2007

Daring Fireball Visits NYC

John Gruber (Daring Fireball) gave a presentation at the SoHo Apple Store tonight. I might've been annoyed it was a repeat of his C4[0] presentation, except I wasn't at C4 so I hadn't heard it. The rest of the audience seemed suitably impressed -- Apple brought extra chairs, and there were still a bunch of people sitting on the floor.

Afterwards, I tagged along to a yummy Vietnamese restaurant. We left when all the unoccupied chairs had been placed on the tables around us, only to discover a giant (empty) drum of MSG outside the front door. This sparked a brief but lively discussion of whether MSG is in fact as bad for you (us) as people once claimed, with no real resolution.

I liked Gruber's description of Jonas fuzzing, "I need a hole."

Tuesday, June 5 2007

Markdown.cgi v1.3: Fixing the Markdown Source

Whoops! I wrote Markdown.cgi so I could easily preview Markdown content in BBEdit, but 1.2 broke this. As a fix, instead of using .markdown for source and .text to see the HTML output, v1.3 goes back to using .text files, and now ?markdown appended to the URL returns the Markdown source.

I considered allowing additional query arguments, but Apple's sh and expr string matching is quite limited, and I don't want to make the whole thing any slower or more complicated than necessary. Fortunately, it would be easy to change the 'magic' query string. Just change the 'markdown' literal on line 7.

I also moved the downloadable (.txt) script, to make the older versions available, and so I can avoid pointing people to old versions.

http://www.reppep.com/~pepper/code/Markdown.cgi/

Tuesday, May 29 2007

Markdown.cgi v1.2

Almost immediately after I announced Markdown.cgi, Adam pointed out that it was now impossible to get the source of a Markdown file, since the CGI was automatically rendering the files to XHTML.

To fix this, I have renamed my Markdown source files from .text to .markdown, and made the CGI look for .markdown files, instead of using the .text filename supplied in the URL. This minor change means the source is now available as .markdown, while the HTML version is available as .text. The Apache configuration does not have to change at all.

Additionally, I updated the comments to mention that Markdown.pl requires blank lines between block elements, so one should follow the initial title line.

The new version of Markdown.cgi is v1.2. You can download and rename Markdown.txt, but here it is for reference:

#!/bin/sh
# Markdown v1.2 2007/05/28
# Build an HTML page (with headers) from Markdown.pl output.

# v1.2: Source is .markdown, available without modification.
# Access as .text to get HTML version.

INCLUDES=/home/web/www.reppep.com/include
REALFILE=`dirname $PATH_TRANSLATED`/`basename $PATH_TRANSLATED .text`.markdown
TITLE=`head -1 $REALFILE | cut -f2`


echo "Content-type: text/html"
echo

cat $INCLUDES/head1.incl
echo $TITLE
cat $INCLUDES/head2.incl

# If we already have an H1, don't insert one.
if ! grep --silent '^# ' $REALFILE
 then
  echo -n '<h1>'
  echo -n $TITLE
  echo '</h1>'
  echo
fi

/usr/local/bin/Markdown.pl < $REALFILE

cat $INCLUDES/foot.incl

exit 0

# http://www.extrapepperoni.com/category/computers/markdown/
# http://daringfireball.net/projects/markdown/

# To use, copy Markdown.cgi (this wrapper, which you may have to rename
# from Markdown.txt) and Markdown.pl (from Daring Fireball) into your
# cgi-bin/ and make them executable
# ("chmod +x Markdown.cgi Markdown.pl"), set the correct path for
# INCLUDES below, and install head1.incl (HTML header up to <title>),
# head2.incl (HTML header starting with </title>), and foot.incl in
# that directory.

# Markdown.cgi reads the page's title from the first line, starting
# after the first tab and ending before the second.
# Your document's title should be inside an HTML comment, set off by tabs.
# Follow the title line with a blank line (Markdown.pl requires blank
# lines between block elements).
# The title line contains 5 parts:
# 1) the HTML comment open delimiter (less-bang-dash-dash)
# 2) a tab
# 3) the title text
# 4) a tab
# 5) the HTML comment close delimiter (dash-dash-greater)
# For example (not counting the # on the next line):
#<!--   Markdown.cgi: A Simple Wrapper for Markdown.pl  -->

# Add the following to your Apache httpd configuration
# (likely httpd.conf or a virtual host .conf file):
#   AddHandler markdown .text
#   Action markdown /cgi-bin/Markdown.cgi
#   AddType text/html .text
#   ScriptAlias /cgi-bin/ /home/web/www.reppep.com/cgi-bin/
#   AddType text/html .pl

Thursday, May 24 2007

Markdown.cgi: Markdown in Apache httpd

I've written a couple articles for TidBITS since they started using John Gruber's Markdown format, and despite actually liking HTML as a writing format, I was impressed with Markdown's simplicity and efficiency (no <p>s are a big time savings!).

So I installed PHP Markdown Extra here on Extra Pepperoni, and got hooked on writing in Markdown. Unfortunately, there's no Markdown plug-in for plain Apache -- lots of ways to parse Markdown in your blogging software or wiki or CMS, but I want to be able to write a .text file and serve it up 'directly' from Apache on www.reppep.com.

Update: Markdown.cgi has been updated. Check my Markdown category for the latest.

Markdown is designed to run as a simple filter, so it's well suited to drop-in installation in a lot of places, without having to build customized versions for a particular application's APIs. There are several implementations -- the original Perl script, as well as versions in PHP (which I use in WordPress, slightly hacked), Python, Ruby, JavaScript, etc. http://markdown.infogami.com/ keeps a list.

Since I couldn't find an Apache handler (plug-in) or a CGI for Markdown, I wrote a very simple wrapper for Gruber's Markdown.pl. Conceptually, my wrapper spits out an HTML header, uses Markdown.pl to render the requested page as (X)HTML, and then appends an (X)HTML footer. The reality is slightly more complicated, due to the vagaries of figuring out the document's title, and conditionally inserting it back into the output as an <H1> tag. Even so, the whole thing is under 60 lines, mostly whitespace and comments.

Markdown.cgi also solves a problem which has wasted a significant amount of my time. BBEdit's built-in Preview tool can use Markdown.pl to generate HTML, which it then passes to WebKit for previewing in a formatted window. But if you put the pages on a real active website, BBEdit has another feature I really like, whereby it will actually calculate a live URL for the page to be previewed, request that URL from the web server, and preview that instead.

This is great, but if you're writing Markdown, BBEdit shows you the unrendered Markdown code (as served up by the web server), instead of rendering the Markdown file from disk. To make matters worse, BBEdit's Preview is live in real time, but continuously re-rendering a Markdown document as you type makes BBEdit stall badly on my 1.5GHz PBG4, so I've stopped using it as a live preview. I instead trained myself to use Markdown.pl as a UNIX script, which I manually trigger to generate a temporary HTML document. I then view this document in BBEdit's live Preview. Among other things, I frequently found myself editing the scratch HTML document, and having to copy my changes back to the Markdown source. Yuck.

Now that my web servers can serve up Markdown .text documents in HTML format, I can skip that whole mess, and go back to previewing .text documents (using Safari or BBEdit's live Preview) with server-side HTML conversion, seeing exactly what surfers see, as Siegel intended.

Tip: If you're reading about or trying out Markdown, don't read the syntax page -- skip to the simple crib sheet on the Dingus page. It's much simpler (and shorter!).


To implement this, I added some lines to my Apache httpd (1.3) configuration, inside the main vhost block:

Action markdown /cgi-bin/Markdown.cgi
AddHandler markdown .text
AddType text/html .text
ScriptAlias /cgi-bin/ /home/web/www.reppep.com/cgi-bin/
AddType text/html .pl

Here is Markdown.cgi, although I had to rename this copy Markdown.txt so you can download it:

#!/bin/sh
# Markdown v1.1.1 2007/05/24
# Build an HTML page (with headers) from Markdown.pl output.

INCLUDES=/home/web/www.reppep.com/include
TITLE=`head -1 $PATH_TRANSLATED | cut -f2`
echo "Content-type: text/html"
echo

cat $INCLUDES/head1.incl
echo $TITLE
cat $INCLUDES/head2.incl

# If we already have an H1, don't insert one.
if ! grep --silent '^# ' $PATH_TRANSLATED
 then
  echo -n '<h1>'
  echo -n $TITLE
  echo '</h1>'
  echo
fi

/usr/local/bin/Markdown.pl < $PATH_TRANSLATED

cat $INCLUDES/foot.incl

exit 0

# http://www.extrapepperoni.com/2007/05/24/markdowncgi-using-markdown-in-apache-httpd/
# http://daringfireball.net/projects/markdown/

# To use, copy Markdown.cgi (this wrapper, which you may have to rename
# from Markdown.txt) and Markdown.pl (from Daring Fireball) into your
# cgi-bin/ and make them executable
# ("chmod +x Markdown.cgi Markdown.pl"), set the correct path for
# INCLUDES below, and install head1.incl (HTML header up to <title>),
# head2.incl (HTML header starting with </title>), and foot.incl in
# that directory.

# Markdown.cgi reads the page's title from the first line, starting
# after the first tab and ending before the second.
# Your document's title should be inside an HTML comment, set off by tabs.
# The title line contains 5 parts:
# 1) the HTML comment open delimiter (less-bang-dash-dash)
# 2) a tab
# 3) the title text
# 4) a tab
# 5) the HTML comment close delimiter (dash-dash-greater)
# For example (not counting the # on the next line):
#<!--   Markdown.cgi: A Simple Wrapper for Markdown.pl  -->

# Add the following to your Apache httpd configuration
# (likely httpd.conf or a virtual host .conf file):
#   AddHandler markdown .text
#   Action markdown /cgi-bin/Markdown.cgi
#   AddType text/html .text
#   ScriptAlias /cgi-bin/ /home/web/www.reppep.com/cgi-bin/
#   AddType text/html .pl

My header and footer are dead simple, but easy to replace with something more sophisticated.

head1.incl:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
    <title>

head2.incl:

    </title>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
</head>
<body>

foot.incl:

<hr />

<p align="center"><small><a href="./">home</a></small></p>

</body>
</html>