Easy Command Line Apps with Argparse

argparse, which will be included in Python's Standard Library when 2.7 is released, is a great command line parsing library. It's both easier to use and more capable than optparse -- a fine combination! Here's a quick-and-dirty example of creating a script that supports commands, using argparse.

#!/usr/bin/env python
import sys

class Command(object):
    """Base class for commands.
    """
    # optional command-specific args
    args = []
    def execute(self, args):
        raise NotImplementedError()

def arg(*args, **kw):
    # sugar for arg declaration
    return (args, kw)

def commands():
    """Return list of Command sub classes from current module.
    """
    mod = sys.modules[__name__]
    classes = []
    for attr in [getattr(mod, name) for name in dir(mod)]:
        if Command in getattr(attr, '__bases__', []):
            classes.append(attr)
    return classes

def register_commands(arg_parser):
    """Register all subclasses of Command.
    """
    cmd_parsers = arg_parser.add_subparsers()
    for cmd in commands():
        # use first line of docstring as help
        help = (cmd.__doc__ or 'no help').strip().splitlines()[0]
        cmd_parser = cmd_parsers.add_parser(cmd.__name__, help=help)
        # add optional command-specific arguments
        for (args, kw) in cmd.args:
            cmd_parser.add_argument(*args, **kw)
        # This is really, really important!
        # Without it we won't know which command to execute.
        cmd_parser.set_defaults(command_class=cmd)

Define some commands:

class Date(Command):
    """show current date
    """
    args = [
        arg('format', default=['%Y-%m-%d'], nargs='*',
            help='strftime-compatiable format'),
    ]
    def execute(self, args):
        import datetime
        print datetime.date.today().strftime(' '.join(args.format))


class Echo(Command):
    """echo input to stdout
    """
    args = [
        arg('-n', dest='newline', default=True,
            action='store_false',
            help='do not output the trailing newline'),
        arg('strings', metavar='string', nargs='*'),
    ]
    def execute(self, args):
        sys.stdout.write(' '.join(args.strings))
        if args.newline:
            sys.stdout.write('\n')

Finally, glue it together with argparse.

if __name__ == '__main__':
    import argparse
    arg_parser = argparse.ArgumentParser()
    # really just to demonstrate a global option
    arg_parser.add_argument('-v', '--verbose',
        action='store_true', help='example global option')
    # register commands
    register_commands(arg_parser)
    # parse args including which command to run
    args = arg_parser.parse_args(sys.argv[1:])
    # create instance of command
    command = args.command_class()
    if args.verbose:
        print 'running', command
    # run it
    command.execute(args)

Let's try it! First, let's get a listing of commands:

orutherfurd@laptop:~/tmp$ ./argparse_commands.py --help
usage: argparse_commands.py [-h] {Date,Echo} ...

positional arguments:
  {Date,Echo}
    Date       show current date
    Echo       echo input to stdout

optional arguments:
  -h, --help     show this help message and exit
  -v, --verbose  example global option

then help for a specific one:

orutherfurd@laptop:~/tmp$ ./argparse_commands.py Echo --help
usage: argparse_commands.py Echo [-h] [-n] [string [string ...]]

positional arguments:
  string

optional arguments:
  -h, --help  show this help message and exit
  -n          do not output the trailing newline

lastly, let's run one:

orutherfurd@laptop:~/tmp$ ./argparse_commands.py Date
2010-04-29

orutherfurd@laptop:~/tmp$ ./argparse_commands.py Date %b %d, %Y
Apr 29, 2010

While cursory, this demonstrates how easy it is to create flexible command line apps using argparse. We supplied the Command boilerplate, but argparse took care of all argument parsing and made command dispatching a piece of cake.

Tagged with: python argparse

posted on the April 29th, 2010 at 4:19 p.m.

Merging in Mercurial: A Practical Primer

Merging, and more specifically conflict resolution, in Mercurial can be a sticking point for new and casual users. The fetch extension can decrease some repetitive merging, but sooner or later conflicts will occur. For example, if you have a project with a web developer and designer collaborating in parallel on a task, at some point you'll have conflicting changes. While it's great that Mercurial now comes with sample configuration settings for a variety of merge tools, they're mostly GUI-based, which isn't going to help when logged-in to a headless Linux server. Ever try walking a casual user, over instant messenger, through the process of resolving their conflicts in a 3-pane Vim window? Mercurial's internal:merge merger is a simple alternative. It inserts good, old-fashioned conflict markers. You use whichever editor you like to resolve the conflicts, and you tell Mercurial when everything is resolved. We'll walk through this process starting with creating and conflict and finishing with resolving it.

Setup a New Repository

To begin, we'll create an example repository:

$ hg init hello
$ cd hello
$ echo "print 'hello world'" > hello.py
$ python hello.py
hello world
$ hg ci --config ui.username='Able <able@example.com>' -A -m 'getting started'
adding hello.py

Let's see what we've got:

$ hg log -p
changeset:   0:e15827879a00
tag:         tip
user:        Able <able@example.com>
date:        Wed Apr 21 19:09:02 2010 -0400
summary:     getting started

diff --git a/hello.py b/hello.py
new file mode 100755
--- /dev/null
+++ b/hello.py
@@ -0,0 +1,1 @@
+print 'hello world'

Create Conflicting Changes

Now that we've got a base version, let's create a conflict. Pretending to be two other users, we'll create conflicting changes to Able's file. First, Baker will add a line with the current date and time.

$ cd ..
$ hg clone hello hello_baker
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd hello_baker
$ echo print \"baker says: it is now $(date)\" >> hello.py
$ hg ci --config ui.username="Baker <baker@example.com>" -m "baker's time"
$ python hello.py
hello world
baker says: it is now Wed Apr 21 19:20:34 EDT 2010

Second, Charlie will do the same but with a different date & time.

$ cd ..
$ hg clone hello hello_charlie
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd hello_charlie
$ echo print \"charlie says: it is now $(date)\" >> hello.py
$ hg ci --config ui.username="Charlie <charlie@example.com>" -m "charlie's time"
$ python hello.py
hello world
charlie says: it is now Wed Apr 21 19:22:15 EDT 2010

Now we've got 3 repositories with 3 different versions:

  A
 / \
B   C
  1. Able's original 1-line file
  2. Baker's version, which adds a line to Able's
  3. Charlie's version, which ads a different line to Able's

Pull Changes & Resolve Conflicts

Able's in charge of consolidating the changes, which we know will conflict. He starts with Baker's:

$ cd ../hello
$ hg pull -u ../hello_baker
pulling from ../hello_baker
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg tip
changeset:   1:786404a3c8b9
tag:         tip
user:        Baker <baker@example.com>
date:        Wed Apr 21 19:20:45 EDT 2010
summary:     baker's time

At this point, Able and Baker both have the same repository. Both consist of 2 changesets: Able's original and Baker's addition. When Able pulls from Charlie, he's going to get a conflict, because a human must resolve the conflicting changes made to line 2 of hello.py.

$ hg pull -u ../hello_charlie
pulling from ../hello_charlie
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
not updating, since new heads added
(run 'hg heads' to see heads, 'hg merge' to merge)

hg heads lists the changesets that require merging:

$ hg heads
changeset:   2:8c0dc6a6e5fe
tag:         tip
parent:      0:e15827879a00
user:        Charlie <charlie@example.com>
date:        Wed Apr 21 19:22:25 EDT 2010
summary:     charlie's time

changeset:   1:786404a3c8b9
user:        Baker <baker@example.com>
date:        Wed Apr 21 19:20:45 EDT 2010
summary:     baker's time
$ hg merge --config ui.merge=internal:merge
merging hello.py
warning: conflicts during merge.
merging hello.py failed!
0 files updated, 0 files merged, 0 files removed, 1 files unresolved
use 'hg resolve' to retry unresolved file merges or 'hg update -C' to abandon

With internal:merge, Mercurial inserts conflict markers showing the conflicting portions. You can pick one, the other, or neither.

$ cat hello.py
print 'hello world'
<<<<<<< local
print "charlie says: it is now Wed Apr 21 19:22:15 EDT 2010"
=======
print "baker says: it is now Wed Apr 21 19:20:34 EDT 2010"
>>>>>>> other

In this case, Able doesn't like Baker or Charlie's solution and does things his own way:

$ hg diff
diff --git a/hello.py b/hello.py
--- a/hello.py
+++ b/hello.py
@@ -1,2 +1,2 @@
 print 'hello world'
-print "charlie says: it is now Wed Apr 21 19:22:15 EDT 2010"
+print 'able says: use "date" to check the time'

$ hg ci --config ui.username='Able <able@example.com>' -m 'use date!'
abort: unresolved merge conflicts (see hg resolve)

Oops, it's easy to forget about hg resolve! While Able resolve the situation, Mercurial needs to be informed. This is a good time to mention hg resolve plays several roles. In addition to marking files as resolved, it can list files needed resolution, and mark files as unresolved. Let's mark all files as resolved and then commit:

$ hg resolve -m -a
$ hg ci --config ui.username='Able <able@example.com>' -m 'use date!'

Able's committed the merge. Notice this changeset has two parents -- the two heads have been merged into one. For refugees from other VCS, having to commit a merge can be a surprise, but it's great! It means until you've committed the merge you haven't actually committed yourself to anything. You're free to experiment, redo the merge if things don't go smoothly, etc... and since your work was committed before you began merging your original work is also safe.

$ hg tip
hg tip
changeset:   3:fa1b367d9adb
tag:         tip
parent:      2:8c0dc6a6e5fe
parent:      1:786404a3c8b9
user:        Able <able@example.com>
date:        Wed Apr 21 19:24:15 2010 -0400
summary:     use date!

To use internal:merge all the time, add it to your ~/.hgrc.

[ui]
merge=internal:merge

More "internals"

Here are few more merge tools Mercurial has up its sleeves:

  • internal:prompt: have Mercurial ask you whether "keep local" or "take other" -- used by default for unmergable files (e.g. images)
  • internal:other: take other version -- useful if you need to merge and want to accept the other version as-is
  • internal:local: keep local version -- useful if you want to merge and keep your version as-is

Tips & Tricks

hg glog (graphlog) is an extension that shows a text-based graphical representation of history.

$ hg glog -r -3:

@    changeset:   3:fa1b367d9adb
|\   tag:         tip
| |  parent:      2:8c0dc6a6e5fe
| |  parent:      1:786404a3c8b9
| |  user:        Able <able@example.com>
| |  date:        Wed Apr 21 19:24:15 2010 -0400
| |  summary:     use date!
| |
| o  changeset:   2:8c0dc6a6e5fe
| |  parent:      0:e15827879a00
| |  user:        Charlie <charlie@example.com>
| |  date:        Wed Apr 21 19:22:25 EDT 2010
| |  summary:     charlie's time
| |
o |  changeset:   1:786404a3c8b9
|/   user:        Baker <baker@example.com>
|    date:        Wed Apr 21 19:20:45 EDT 2010
|    summary:     baker's time
|

You can browse your repository using Mercurial's built-in web server, which has graphical log view.

$ hg server -v -a 127.0.0.0 -p 8888
listening at http://localhost:8888/ (bound to 127.0.0.1:8888)

and open http://127.0.0.1:8888/ in your browser.

Other Resources

Tagged with: mercurial

posted on the April 22nd, 2010 at 2:05 p.m.

django-easyurls: Making it easier to read and write Django URLs

By making assumptions, like year is usually a 4-digit number and id 1 or more digits, django-easyurls takes much of the repetition out of defining URLs, using a syntax that's shorter and easier to read.

Compare the following:

# standard
urlpatterns += patterns('django.views.generic.date_based',
    url(r'^(?P<year>\d{4})/(?P<month>[a-z]{3})/(?P<day>\w{1,2})/(?P<slug>[-\w]+)/$',
        'object_detail', info_dict),
    url(r'^(?P<year>\d{4})/(?P<month>[a-z]{3})/(?P<day>\d{1,2})/$',
        'archive_day',   info_dict),
    url(r'^(?P<year>\d{4})/(?P<month>[a-z]{3})/$', 'archive_month', info_dict),
    url(r'^(?P<year>\d{4})/$', 'archive_year',  info_dict),
)

# use easyurls, and let it generate the regex for you
from easyurls import regex as p
urlpatterns += patterns('django.views.generic.date_based',
    url(p('<year>/<month:mon>/<day>/<slug>'), 'object_detail', info_dict),
    url(p('<year>/<month:mon>/<day>'),        'archive_day',   info_dict),
    url(p('<year>/<month:mon>'),              'archive_month', info_dict),
    url(p('<year>'),                          'archive_year',  info_dict),
)

These two sets of URL patterns are functionally equivalent -- the same regex is passed to url() in both cases, but the second is shorter and clearer. Why keep repeating that year is a 4-digit number, month is 3 letters, and day a 1 or 2 digit number? Also, since 99% of the time one wants the pattern to start with '^', and end with '/$', why keep repeating it? If you don't want them, you can say so.

django-easyurls works by defining names for patterns and generating regular expressions for you. By default, the name of the captured variable is the name of the pattern. This can be overriden, as is done above where the "mon" pattern is used for "month", instead of the default \d{1,2}.

Here's a list of the default patterns:

>>> from easyurls import regex as p
>>> for name in sorted(p.patterns):
...     print '%5s: %s' % (name,p.patterns[name])
  day: \d{1,2}
   id: \d+
  mon: [a-z]{3}
month: \d{1,2}
    n: \d+
 slug: [\w-]+
  tag: \w+
 year: \d{4}

To use a different name for a pattern, or different pattern for a name, add the pattern after the name, prefixing the pattern with ":".

# default for month is \d{1,2}
>>> print p('<month>')
^(?P<month>\d{1,2})/$

# using [a-z]{3} for month
>>> print p('<month:mon>')
^(?P<month>[a-z]{3})/$

# using [a-z]{3} for mmm
>>> print p('<mmm:mon>')
^(?P<mmm>[a-z]{3})/$

It's easy to add new or override existing patterns:

>>> p['yy'] = r'\d{2}'
>>> p['mm'] = r'\d{2}'
>>> p['dd'] = r'\d{2}'

>>> print p('<year:yy>/<month:mm>/<day:dd>')
^(?P<year>\d{2})/(?P<month>\d{2})/(?P<day>\d{2})/$

By default, if no pattern is found, \d+ is assumed.

>>> print p('releases/<project_id>')
^releases/(?P<project_id>\d+)/$

For flexibility, you can always use a regular expression.

# regex for unknown "zip_code"
>>> print p('zip/<zip_code:\d{5}>')
^zip/(?P<zip_code>\d{5})/$

# override slug, allowing "."
>>> print p('<slug:[\w-.]+>')
^(?P<slug>[\w-.]+)/$

For demonstration, and testing, purposes here's how prepending and appending or '^', '/', and '$' is handled:

>>> print p('')
^$
>>> print p('foo$')
^foo$
>>> print p('foo/')
^foo/$
>>> print p('/')
^/$

Prepending of '^' and appending of '/' and '$' can be disabled.

>>> p('foo', anchor=False, terminate=False, append_slash=False)
'foo'

Getting django-easyurls

django-easyurls lives in a mercurial repository on Bitbucket, available here: http://bitbucket.org/orutherfurd/django-easyurls/. Alternatively, you can download the single file easyurls.py, which is all you need.

Tagged with: annoucement django

posted on the March 23rd, 2009 at 2:23 p.m.

Breads to Bake

Scali from Bread cetera. SteveB often has good videos, too.

Flaxseed Rye, based on Hamelman's "Flaxseed Bread". As always, Susan's pictures are enticing!

While on the topic of ryes, I've recently done a couple of Hamelman's: "70 Percent Rye with a Rye Soaker and Whole-Wheat Flour" and "Volkornbrot with Flaxseeds" -- both of which were about as serious as they sound, but very good. Angus was particularly fond of the of the Volkornbrot, always requesting a piece to go along with his usual breakfast.

Tagged with: bread

posted on the March 19th, 2009 at 1:11 p.m.

Flatbread Recipes to Try

Some flatbread recipes to try. I'm still very much in search of a great flour tortilla recipe. The one I use from Crust and Crumb is good, but not great. I've stumbled across several recipes that say lard is the key.

Tagged with: bread

posted on the March 15th, 2009 at 12:06 p.m.

Released FirstMate 0.3.1 plugin for jEdit

  • Released FirstMate 0.3.1 plugin for jEdit. Fixes 2 bugs related to apostrophe handling for auto-paired characters.

Tagged with: firstmate announcement jedit

posted on the June 23rd, 2008

New Vimposter Releases

Tagged with: announcement vimposter jedit

posted on the December 8th, 2007

Added Links Section

Tagged with: announcement

posted on the November 25th, 2007

This Site is Now Powered By Django

  • This site is now powered by Django.

Tagged with: announcement

posted on the September 3rd, 2007

jEdit Plugin Releases and New and Updated jEdit Modes

Released Tags 2.0.2 plugin for jEdit

  • Released Tags 2.0.2 plugin for jEdit. Updated for 4.3pre3.
  • Released MoinMoin 0.3.4 plugin for jEdit. Updated for MoinMoin 1.5.

Tagged with: moinmoin announcement jedit tags

posted on the April 14th, 2006

New Version of MoinMoin Plugin for jEdit

Tagged with: moinmoin announcement jedit

posted on the March 16th, 2006

Django Edit Mode for jEdit

Tagged with: announcement jedit

posted on the August 12th, 2005

New Vimposter Release

  • Updated Vimposter plugin to work with (and require) jEdit 4.3pre3 (not yet released). I probably won't do a new release until 4.3pre3 is released, unless I get requests, but the code's in the repository.
  • New build of new jEdit Launcher. Includes a GUI for editing settings, instead of having to edit the registry directly.

Tagged with: announcement vimposter jedit

posted on the July 20th, 2005

New EditorScheme Release and Build of New jEdit Launcher

  • New build of new jEdit Launcher, with better logging of errors and handling of missing 'Working directory setting'.
  • EditorScheme plugin 1.0, 'DarkMine' scheme from Matt Gilbert.

Tagged with: announcement jedit editorscheme

posted on the May 3rd, 2005