Parameters, options and flags for Python scripts

Theory and practice

Summary

While there are various ways of making a script configurable in Python, argparse is a very popular option. Unfortunately, it's also a cryptic one.

This article provides ready to copy/paste snippets for the common use cases, and explains how they work for when you will ask ChatGPT about it.

In short:

import argparse

parser = argparse.ArgumentParser(
    description="Description of the program"
)

parser.add_argument(
    "positional_argument", 
    type=str, 
    help="Required first command parameter"
)

parser.add_argument(
    "-f",
    "--flag",
    action="store_true",
    help="Will be either True or False",
)

parser.add_argument(
    "-o",
    "--option,
    help="Attach a value to the name 'option'",
)

args = parser.parse_args()

print(
  args.positional_argument,
  args.flag,
  args.option
)

I put that here for me as well, as ChatGPT is blocked by some of my clients, and I'm lazy.

This is where ChatGPT is awesome

The more you dev, the more you need to make scripts, and therefore, make them configurable. Now, for project maintenance, I use doit, which I wrote about a few weeks before.

However, for the rest, I still do that the old fashion way, which in Python means 90% of the time using argparse or click.

argparse is in the stdlib, so this is what people will use when they don't want to pip install stuff.

But argparse doc and API suck.

They are so bad I have a permanent file with an argparse snippets I used to copy/paste from, every time I wrote a script because I could never remember or find out quickly how to use it.

Well, that's what I used to do. Now I ask ChatGPT, it's amazing for these kinds of things, and never gets tired of repeating.

Yet, ChatGPT has a limitation: you need to know what to ask, and understand the answer to be able to use it.

This article will help you do just that.

Creating an argparse parser

Since we don't want to process sys.argv manually, we are going to use a parser. Let's create a script "hello_wizard.py":

import argparse
parser = argparse.ArgumentParser(description="Hogwash reveal")
args = parser.parse_args()

If we now run:

python hello_wizard.py

It will appear to do nothing.

But it will in fact do many things:

• Generate documentation for the script.

• Define a -h and a --help flags to display the documentation.

• Analyze anything you might write after python hello_wizard.py.

• Match it to existing parameters, options and flags.

• The run the code associated to those parameters, options and flags.

For now, this means you can type python hello_wizard.py -h or python hello_wizard.py --help and it will display the documentation:

$ python hello_wizard.py --help
usage: hello_wizard.py [-h]

Hogwash reveal

options:
  -h, --help  show this help message and exit

Positional parameters

Positional parameters are parameters that are recognized depending of where they are.

E.G: if you do mkdir my_project, it will create a directory "my_project" where you are. mkdir is the command, but my_project is a positional parameter. The command knows it's the directory to create because it's the first thing that comes after it.

Let's add one to our script, your name:

import argparse

parser = argparse.ArgumentParser(description="Hogwash reveal")

# Define the positional parameter.
parser.add_argument("name", type=str, help="Your name")

# This finds if "name" has been passed to your script or not
args = parser.parse_args()

# All found values are attached to args as attributes
print(f"You're a hazard {args.name}")

If we use -h again, can see it's now part of the doc:

$ python hello_wizard.py -h
usage: hello_wizard.py [-h] name

Hogwash reveal

positional arguments:
  name        Your name

options:
  -h, --help  show this help message and exit

If you run the script without any parameters, it gives you an error message:

$ python hello_wizard.py
usage: hello_wizard.py [-h] name
hello_wizard.py: error: the following arguments are required: name

But if we pass a name, it will print it back:

$ python hello_wizard.py Mary
You're a hazard Mary

Positional parameters are often required, but we can add optional ones.

Let's add a second one:

import argparse

parser = argparse.ArgumentParser(description="Hogwash reveal")

parser.add_argument("name", type=str, help="Your name")

# The order matters: only a value coming after "name" will be
# considered the value "age" in the command line, because 
# the add_argument() for "name" came before this one.
parser.add_argument("age", type=int, nargs="?", help="Your age")

args = parser.parse_args()

if args.age is not None:
    print(f"Now that you are {args.age}, I can reveal your true nature")

print(f"You're a hazard {args.name}")

The script will work as before, if I just pass the name. But now, optionally, you can pass the age. In this case we get:

python hello_wizard.py Mary 19
Now that you are 19, I can reveal your true nature
You're a hazard Mary

The optionality comes from nargs="?". This tells how many times this argument is expected. It's a very obscure notation:

• ? means the argument is expected zero or one time.

• + means it's expected one or more times.

• * means zero, one or more.

It's similar to regexes, so of course, it's as clear as mud.

We also changed type to pass int. If we pass an age that is not a number, it will give us an error:

$ python hello_wizard.py Mary old
usage: hello_wizard.py [-h] name [age]
script.py: error: argument age: invalid int value: 'old'

And of course, all that will be reported in the help:

$ python hello_wizard.py -h
usage: hello_wizard.py [-h] name [age]

Hogwash reveal

positional arguments:
  name        Your name
  age         Your age

options:
  -h, --help  show this help message and exit

Note that age is noted "[age]". Using brackets is the convention to say it's optional.

Flags

A flag is a parameter with a name, and is either on or off. It is found, not by position, but by this name.

E.G: -h and --help are flags.

Flags can have a short form, with one dash and one letter (like -h), or a long form with two dashes and several letters (like --help).

Some commands use a different convention (tar has short flags without dash, find has long options with a single dash, some Windows use using slashes instead of dashes).

But the world has generally agreed on using this convention nowadays.

Let's add one flag:

import argparse

parser = argparse.ArgumentParser(description="Hogwash reveal")

parser.add_argument("name", type=str, help="Your name")

parser.add_argument("age", type=int, nargs="?", help="Your age")

# Flag position doesn't matter.
# The can omit either the long or the short form, but you 
# need at least one
parser.add_argument(
    "-m",
    "--is-main-character",
    action="store_true",
    help="The person is the main character",
)

args = parser.parse_args()

if args.age is not None:
    print(f"Now that you are {args.age}, I can reveal your true nature")
print(f"You're a hazard {args.name}")

# Dashes can't be used in python variable names, 
# so the attribute is named using underscores.
# If you have only a short form, this would use it instead.
if args.is_main_character:
    print("Also, here's a pile of cash")

You'll note that we do everything using add_argument, which makes the whole thing confusing. You really have to know argparse to understand which one does what.

The help message gains an "options" section:

python hello_wizard.py -h
usage: hello_wizard.py [-h] [-m] name [age]

Hogwash reveal

positional arguments:
  name        Your name
  age         Your age

options:
  -h, --help            show this help message and exit
  -m, --is-main-character
                        The person is the main character

It's because flags are a specific case of options, which we will see later on.

And of course, you can now use your flag:

$ python hello_wizard.py Mary --is-main-character
You're a hazard Mary
Also, here's a pile of cash

$ python hello_wizard.py Mary -m 19
usage: hello_wizard.py [-h] [-m] name [age]
hello_wizard.py: error: unrecognized arguments: 19

$ python hello_wizard.py --is-main-character Mary  19
Now that you are 19, I can reveal your true nature
You're a hazard Mary
Also, here's a pile of cash

Options

An option is a parameter with a name and a value. Like a flag, it is found, not by position, but by this name. But unlike a flag, an option has a value attached to it. In fact, a flag is just a special option.

The Python command has the -c option, which let you execute code directly from the cmd. E.G:

python -c "print('Wingarda Leviosum')"
Wingarda Leviosum

The option print('Wingarda Leviosum') is the value that comes with -c, and you can't use -c without passing any value, that would make no sense.

We can add an option to our script to make everything uppercase or lowercase:

import argparse


def speak(*texts, situation=None):
    text = " ".join(texts)
    if situation == "sequel":
        print(text.upper())
    elif situation == "parsertongue":
        print(text.lower())
    else:
        print(text)


parser = argparse.ArgumentParser(description="Hogwash reveal")

parser.add_argument("name", type=str, help="Your name")

parser.add_argument("age", type=int, nargs="?", help="Your age")

parser.add_argument(
    "-m",
    "--is-main-character",
    action="store_true",
    help="The person is the main character",
)

parser.add_argument(
    "-s",
    "--situation,
    help="Decide if the script scream or whisper",
)


args = parser.parse_args()

situation = args.situation

if args.age is not None:
    speak(
        f"Now that you are {args.age}, I can reveal your true nature",
        situation=situation,
    )
speak(f"You're a hazard {args.name}", situation=situation)

if args.is_main_character:
    speak("Also, here's a pile of cash", situation=situation)

And you got what you need:

$ python hello_wizard.py Mary --situation sequel
YOU'RE A HAZARD MARY
$ python hello_wizard.py Mary --situation parsertongue
you're a hazard mary

Some script will use an = sign to separate the option’s name from its value, like this: --situation=parsertongue. argparse accepts both forms.

Organizing your script

Putting everything like is at the root of the script is fine for a short story of code. If you think you are going full novel, then a better organization will make the whole thing more manageable:

import argparse


def speak(*texts, situation=None):
    text = " ".join(texts)
    if situation == "sequel":
        print(text.upper())
    elif situation == "parsertongue":
        print(text.lower())
    else:
        print(text)


# Parsing the arguments is separated into its own function
# This means the rest of the code doesn't need to know 
# where the data comes from
def parse_args():
    parser = argparse.ArgumentParser(description="Hogwash reveal")

    parser.add_argument("name", type=str, help="Your name")

    parser.add_argument("age", type=int, nargs="?", help="Your age")

    parser.add_argument(
        "-m",
        "--is-main-character",
        action="store_true",
        help="The person is the main character",
    )

    parser.add_argument(
        "-s",
        "--situation",
        help="Decide if the script scream or whisper",
    )

    return parser.parse_args()


# This function is now simpler, and simply accepts regular parameters
# It's easier to understand, and to test.
def hello(name, age=None, situation=None, is_main_character=False):
    if age is not None:
        speak(
            f"Now that you are {age}, I can reveal your true nature",
            situation=situation,
        )
    speak(f"You're a hazard {name}", situation=situation)

    if is_main_character:
        speak("Also, here's a pile of cash", situation=situation)


# We allow the script to run as such, but also to be imported
if __name__ == "__main__":
    args = parse_args()
    hello(args.name, args.age, args.situation, args.is_main_character)

Of course, this can be improved a lot. We can sculpt the functions into something with docs and types. We can add sub-commands and arguments validation. And also get data from environment variables or a configuration file.

For another article, maybe.