David Hancock

Where the parentheses go… (and seq)

I have a dickens of a time remembering how to parenthesize a pipeline of commands so they can be timed as a group. Here’s a snippet I read on the net today (from a post about reducing Python’s memory footprint):

time (for i in $(seq 1 100); do python -c "pass"; done)

So, parentheses around everything to be timed. The seq command work on my Linux box (/usr/bin/seq) but not my Macbook.

Better one-liner for ssh keys

Useful update to previous one-liner for adding ssh keys (letter to Linux Journal August 2009). Instead of using ssh and cat, simply use ssh-copy-id. For example:

ssh-copy-id -i id_rsa user@somehost

Will not only append your id_rsa.pub key to authorized_keys on somehost, it will ensure that the permissions on .ssh and authorized_keys are correct.

Fixing Entourage inbox

My Entourage inbox stopped updating! Everything else (sent items, junk folder, etc.) was updating fine. The lazyweb led me to the following fix: Empty the Inbox cache.

Right-click Inbox folder, select Folder Properties. “Empty Cache” is in the middle of the General pane. Read the warnings, then click Empty.

This empties your inbox folder, which then gets repopulated from the Exchange server on the next sync.

A few programmer aphorisms

Seen on a software-architect’s blog:

Think Once — Code Twice

Some thoughts for the day

* "Quick And Dirty == Guaranteed Rework"
* "He Who Codes First Loses"
* "Think Once -- Code Twice"
* "Admin's Law: It's Always Permissions"
* "Programmer's Law: If it's not permissions, it's the path"
* "If it seems hard, you're doing it wrong"
* "One-Off == The First of Many"
* "Requirements Translation: Never == Rarely, Always == Mostly"
* "Things Change: Generalize and Parameterize"

http://slott-softwarearchitect.blogspot.com/2009/06/think-once-code-twice.html

Nice one-liners from LJ Hack and /

Kyle Rankin writes a column for LInux Journal called Hack and /. He’s got a couple nice ssh-based one-liners in the June 2009 issue, which I’m listing here for myself and others to find later.

Put public key on remote server

This appends your public key to the authorized keys list on the remote server. This saves scp-ing it, ssh-ing, and appending it. Nice.

ssh user@server.whatever.tld "cat >> ~/.ssh/authorized_keys2" < ~/.ssh/id_rsa.pub

Image disk to file on remote server

This example assumes /dev/sda is the disk you want to image.

sudo dd if=/dev/sda | ssh user@server.whatever.tld “cat > /path/to/images/sda-image.img”

Clean, not necessarily quick. You can image directly to another disk if you ssh as root and cat to a device instead of a file.

Restore image from remote server

ssh user@server.whatever.tld "cat /path/to/images/sda-image.img" | sudo dd of=/dev/sda

As in the backup, you can do this directly from a device on the remote (I think).

Lightning Talks

Hello from PyCon Lightning Talks. Jehiah will be presenting txLoadBalancer.

Some PostgreSQL reminders

From Chander Ganesan via email:

/etc/init.d/postgresql reload

This rereads postgresql.conf, and new backend processes get the new settings. Existing backends are unchanged.

alter database db_name set log_min_duration_statement 250;

Immediately causes all backends to log statements taking longer than 250 seconds to run (assuming logging is active already).

alter database db_name reset log_min_duration_statement;

Immediately resets to use the default value from postgresql.conf.

We are planning to use CSV logging for our main PostgreSQL database, rotating hourly and loading the CSVs into another database for analysis. But we would only need to see ALL statements if we’re troubleshooting. In normal operations, we’d want to see only long-running queries (candidates for tuning).

Being able to “poke” a new value for the minimum duration logged makes this very easy to change on the fly.

Saving a couple idioms

Things I’ve written down from articles lately, don’t want to lose them:

Nice awk example (count number of users of each shell):

awk -F: '{x[$7]+=1} END {for(z in x) {print z, x[z]}}' /etc/passwd

Nonobvious (to dumb me, at least) use of slicing in Python:

def reverse(str):
    return str[::-1]

print reverse('Bite me')

Using fileinput Python module:

import fileinput
for line in fileinput.input(): # cmd line files or stdin
    print fileinput.filelineno(), fileinput.filename(), line

for line in fileinput.input("test.txt", inplace=1):
    print fileinput.filelineno(), fileinput.filename(), line

Second example overwrites lines of test.txt with your munged lines.

From Linux Journal letters, a nice way to use text in compressed files without needing to know they’re compressed, or what they’re compress with…

function do_cat()
{
    local CAT
    CAT=cat
    case "$1" in
        *.gz) CAT=zcat;;
        *.bz2) CAT=bzcat;;
    esac
    ${CAT} "$1"
}

function smart_cat()
{
    local i
    for i in "$@"; do
        do_cat "$i"
    done
}

Put these in your shell profile, and you can do stuff like:

smart_cat *log* | grep 'somethinginteresting'

and it won’t matter if they’re gzipped, bzip2ed, or plain text. Plugging in another compression tool is a matter of adding a branch in the case statement in do_cat.