Klotzbach & Gray Discontinue December Hurricanes Prediction

via Russ Roberts:

Two top U.S. hurricane forecasters, famous across Deep South hurricane country, are quitting the practice of making a seasonal forecast in December because it doesn’t work.
William Gray and Phil Klotzbach say a look back shows their past 20 years of forecasts had no predictive value.

I love the intellectual honesty this must have taken. I wish practitioners of software development could look back and evaluate what doesn’t work, but that’s challenging from both a pragmatic and human-nature standpoint.

pretenseofknowledge Dec 26 2011

mogenerator 1.24

What’s New:

  • [FIX] Was incorrectly using -mutableSetValueForKey: for ordered relationships instead of -mutableOrderedSetValueForKey:. bug 75 (Martin Schürrer)

  • [NEW] Now generates Mike Ash-style constant structures for attributes, relationships and fetched properties. This allows you to write code like [obj valueForKey:PersonMOAttributes.age]. Tip: you’ll need to enable ARC generation (--template-var arc=true) if you’re using ARC. (Daniel Tull)

  • [NEW] --base-class-force option, for specifying a base class even if the model file’s entities don’t specify one. (Joe Carroll)

  • [NEW] PONSO: NSSet-based templates, improved inverse relationship logic and plug memory leak. (Tyrone Trevorrow)

  • [FIX] PONSO: Added import for super entity in machine headers. (Tyrone Trevorrow)

  • [FIX] Migrate from deprecated -[NSString initWithContentsOfFile:] and fix a MiscMerge warning where an immutable object was assigned to a mutable ivar. (Joshua Smith)

mogenerator Dec 8 2011

Death and Resurrection of an SSD

Summary: SSDs live fast, die young, and pretend to be OK even while they’re dying. Don’t use one without awesome backups.

And sometimes, they come back from the dead.

※ ※ ※

On November 30 2010, I received my first SSD: a 240 GB OWC Mercury Extreme Pro.

On Thursday, November 10 2011, the drive “died”.

I claimed on twitter there was no warning: there weren’t any I/O errors logged to Console.app > All Messages (a standard technique to recognize a drive going bad). Looking back on it now, there were hints: Alfred corrupting its SQLite database, EyeTV losing its schedules and a recording, pbs (Pasteboard Server) crashing, mds throwing a hissy-fit (not uncommon) and finally a kernel panic (an uncommon event).

Last week I came back from lunch to discover my machine frozen. It was still pingable, but everything that touched the disk locked up. I held down the power key to force a hard reboot.

My machine bounced back, but with kernel BootCache warnings in the Console log. After a bit of googling, I decided to restart the machine in Safe Mode, which I understood would rebuild the BootCache. Turns out it also runs fsck, putting up a nice little progress bar. It was taking a long time, so I went to the gym. I came back an hour later, and the progress bar was where I left it: around 30%.

Uh-oh.

I booted off my nightly SuperDuper backup and launched Disk Utility.

My internal SSD fell off the bus: wouldn’t even appear on the device list. My SSD was gone.

I powered down my MacBook Pro and prepared to yank the drive for replacement from OWC. I didn’t expect any issues, they’ve happily replaced two traditional failed drives for me in the past. On a hunch, after I yanked the battery, I counted to down 10 and plugged the machine back in.

It successfully booted off the SSD.

So my SSD was “back”. My guess is the drive firmware simply turns off its SATA connection when it gets backed into an unrecoverable corner. Removing power seemed to “unlock” the drive.

This is kind of a worst-case scenario, since I didn’t trust the drive anymore but it seemed to be working and OWC may not want to replace it.

fsck came back with a couple of invalid inodes, and indicated successful repair. Still not trusting it, I tried a traditional way to force drive failure: a reformat with writing zeros. ~45 minutes later, the drive mounts sans any reported trouble.

If this was a traditional drive, I might have started to trust it again. However, I know about an extra trick some SSDs have up their sleeve: block-level de-duplication.

So I wrote a small C program that fills a file or device with random data. Note to Unix pedants: I know I could have done this with shell commands or your $FAVORITE_LANGUAGE, but I wanted to get close to the kernel on this one and reduce variables for ease of reproduction.

Random data defeats deduping, and I ran my program with parameters to fill my SSD. I went to bed.

This morning I discovered my SSD, which happily survived a complete filling with zeros, failed when I tried to fill it with random data. It fell off the bus again, and wouldn’t show up in Disk Utility.

I removed the drive from the MacBook Pro and plugged it into my Mac Pro via a drive toaster and reran the stress test.

It passed. I reran it again. It passed again.

My SSD had resurrected itself.

I have since reinstalled the SSD and am happily using it again. I’ve also rewritten and enhanced my initial C program into a better, faster one I’ve entitled stressdrive. It now passes my SSD with flying colors:

$ sudo ./stressdrive /dev/rdisk0
blockSize: 512
blockCount: 468862128
speedScale: 16x
scaled blockSize: 8192
scaled blockCount: 29303883
writing random data to /dev/rdisk0
writing 100% (block 29303002 of 29303883)
1779f30a231c1d07c578b0e4ee49fde159210d95 : SHA-1 of written data
verifying written data
reading 100% (block 29302306 of 29303883)
1779f30a231c1d07c578b0e4ee49fde159210d95 : SHA-1 of read data
SUCCESS

My current hypothesis is that my SSD wore out a flash block and attempted to mark the block as bad and recruit a fresh block from its overprovisioning reserve. This path has a bug, causing the controller to panic. Maybe the supposedly fresh block also had issues, maybe a few of them did. I’m thinking restarting the SSD by removing power helped it make progress in the recovery until it succeeded.

I did have the SSD mysteriously drop off my internal bus again today right before the stressdrive test, so I’m keeping an eye on it — I may not be out of the woods yet.

By the way, through all of this, my SSD’s SMART status has remained “Verified”. Ugh.

This week two friends of mine with a Sandforce controller also had SSD failures similar to mine, where the drive fell off the bus. At least one had the same experience as me where the drive was able to “resurrect” itself and pass “surface scans” (whatever those are). Anecdotes aren’t data, but there you go.

I should also mention OWC has been a champ, proactively finding my original order with them and emailing me when I originally mentioned my failure on twitter. They’ve offered to replace my drive, but I’m keeping it for now. What can I say, running a fast drive that may die at any moment makes me feel alive.

sysadmin ssd Dec 4 2011

Nearly Hygienic C Macros via __COUNTER__

A few months back Benjamin Stiglitz taught me that it’s finally possible to build practically-hygienic macros with modern plain ol’ C #defines. Here’s his example:

#define MIN_PASTE(A,B) A##B
#define MIN_IMPL(A,B,L) ({ \
    __typeof__(A) MIN_PASTE(__a,L) = (A); \
    __typeof__(B) MIN_PASTE(__b,L) = (B); \
    MIN_PASTE(__a,L) < MIN_PASTE(__b,L) ? MIN_PASTE(__a,L) : MIN_PASTE(__b,L); \
})
#define MIN(A,B) MIN_IMPL(A,B,__COUNTER__)

So this C code:

int a = 11;
int b = 42;
int c = MIN(a, b);

preprocesses into this:

int a = 11;
int b = 42;
int c = ({
    __typeof__(a) __a0 = (a);
    __typeof__(b) __b0 = (b);
    __a0 < __b0 ? __a0 : __b0;
});

Here’s how __COUNTER__ works:

A new predefined macro __COUNTER__ has been added. It expands to sequential integral values starting from 0. In conjunction with the ## operator, this provides a convenient means to generate unique identifiers.

It’s still not gensym-quality, since the generated variable name can shadow other variable names:

int a = 11;
int b = 42;
int __a0 = a * b; // uh oh
int c = MIN(a, b);
assert(__a0 == 462); // passes though

This is contrived, since you probably aren’t going to prefix your variable names with double-underscores. That said, this code does compile and execute correctly, although enabling -Wshadow will reveal all is not well.

__COUNTER__ is supported by GGC 4.3, clang and even MSVC.

Nov 18 2011

Copy OmniFocus Task Outline Text to Clipboard

Here’s a quick script that will take the selected tasks in OmniFocus 1.9.4 and put a simple space-indented text representation on the clipboard:

tell application "OmniFocus"
    tell first document window of front document
        set selectedItems to selected trees of content
        if ((count of selectedItems) < 1) then
            error "Please first select a task"
        end if
        set output to ""
        repeat with itemItr in selectedItems
            set output to output & my printTree(itemItr, 0)
        end repeat
    end tell
end tell
set the clipboard to output

on printTree(node, indent)
    set output to ""
    repeat indent times
        set output to output & "    "
    end repeat
    set output to output & name of node & (ASCII character 13)
    tell application "OmniFocus"
        repeat with treeItr in trees of node
            set treeItr to treeItr as any
            set output to output & my printTree(treeItr, indent + 1)
        end repeat
    end tell
    return output
end printTree

applescript omnifocus Nov 8 2011

Scott Adams on Systems

Scott Adams:

The trick is that I didn’t have an exercise goal in the first place, so I couldn’t have failed to reach it. What I do have is an exercise system, and I was completely successful at the system. My philosophy is that losers have goals and winners have systems.

Building systems is far more effective than merely aiming at goals.

Oct 22 2011

Swap Word Instances in BBEdit

Sometimes I want to swap all instances of “foo” with “bar”. That is, I want every instance of “foo” turned into “bar” and vice-versa.

This may sound odd, but I encounter often it enough that I formed a strategy long ago, three replacement operations:

  1. Replace “foo” with something unique (a sentinal) in the document. I often use a bullet (•) for this.

  2. Replace “bar” with “foo”.

  3. Finally replace the sentinel string with “bar”.

Encountering this again tonight, I decided to finally automate the three-operation replacement:

set uuid to "1DB2FC05-1011-4467-BA2C-A8F3A8B530BC"
set findPasteboard to do shell script "pbpaste -pboard find"
set findPasteboard to findPasteboard & "↔"
display dialog "Swap a↔b" default answer findPasteboard
set swapSet to split of (text returned of result) by "↔"
set term1 to item 1 of swapSet
set term2 to item 2 of swapSet
tell application "BBEdit"
    tell text window 1
        replace term1 using uuid
        replace term2 using term1
        replace uuid using term2
    end tell
end tell

to split of textToSplit by splitter
    set oldTextItemDelimiters to AppleScript's text item delimiters
    if the class of splitter is list then
        set AppleScript's text item delimiters to splitter
    else
        set AppleScript's text item delimiters to {splitter}
    end if
    set theResult to text items of textToSplit
    set AppleScript's text item delimiters to oldTextItemDelimiters
    return theResult
end split

I now use a hard-coded UUID instead of a bullet for my sentinel, but the idea is the same.

AppleScript’s simple display dialog doesn’t allow two separate text input fields, so I delimit the input with ↔. Hacky.

applescript bbedit Sep 24 2011

Switch Between Tabs and Spaces in BBEdit

Here’s a script I wrote to make it easy to switch between tab-based and space-based indention in BBEdit 10:

tell application "BBEdit"
    tell text window 1
        if expand tabs then
            set currentMode to "Spaces (" & tab width & ")"
        else
            set currentMode to "Tabs"
        end if
        display dialog "Current Mode: " & currentMode & ".
Enter new mode (blank for tabs):" default answer "4"
        set newTabWidth to text returned of result as number
        if newTabWidth = 0 then
            set expand tabs to false
        else
            set expand tabs to true
            set tab width to newTabWidth
        end if
    end tell
end tell

Indention control in BBEdit is painful. Whether tabs or spaces are used is in the Edit > Text Options sheet (confusingly named “Auto-expand tabs”) and the amount of spaces inserted in space-indention mode in stuck to the bottom of the Fonts panel (accessible via View > Text Display > Show Fonts).

This script makes it one action to figure out the current mode and modify it if necessary.

applescript bbedit Sep 21 2011

Apps I Love: DTerm

DTerm is a free app that uses the Accessibility API to figure out the current document you’re looking at. When you hit a hotkey (mine is bound to command-shift-return), it pops up a window with a shell command line with the working directory already set to the frontmost document’s folder.

DTerm works awesomely with Finder, BBEdit, TextMate, Xcode, GitX and a bunch of other Mac apps.

Protip: instead of typing return to execute your DTerm command, type command-return to open a new Terminal.app window with your command. Great if you plan a few follow-up commands or want colored output.

appsilove Sep 21 2011

Google Storage Economics

Scott McNulty / @blankbaby:

You can add 1 TB of storage to your Google account for $256 a year. Who knew? bit.ly/pDO7aS

At first glance it seems excessive that Google would charge you $256/year for an extra 1TB of storage when raw 1TB drives run ~$60-80 on Amazon today.

But you need to keep in mind a lot of extra factors:

  • Redundancy. You’ll need at least two drives to match google’s data safety. Probably more like three with offsite rotation.

  • Energy. Those Amazon drives are bare. Spin them for a year and see what it costs you on your electricity bill. Not to mention that i5 machine you have wrapped around it.

  • Bandwidth. Data’s gotta hit the platters, and that will go over wires Google pays for. Both ways.

  • Humans. Maintaining all of Google’s servers and hard drives takes world-leading sysadmin skills and raw headcount. Part of that $256/year are humans watching monitors at 3:30 in the morning on your behalf.

All that said, I’m not saying purchasing an extra 1TB/year for $256 is a good deal, I’m just saying it’s not as bad a deal as it may seem initially.

Related.

Sep 17 2011