IPython: Magic Commands, Tab Completion, and Faster Prototyping

Quick answer: IPython is a far better interactive Python shell than the default REPL. It adds tab completion, ? and ?? for instant documentation and source, _ for the last result, and magic commands like %timeit, %run, and %%writefile that compress a whole debugging session into a few keystrokes.

Part 9 of our Python series, closing Module 2. The control flow article covers the loops you will be experimenting with.

Why leave the default REPL

The stock >>> prompt gives you one feature: a calculator with memory. Professional prototyping needs more — checking what attributes an object has, reading a function’s signature without leaving the shell, timing two implementations, and reloading edited code without losing your session. IPython is the shell that makes all of that routine, and it is the engine underneath Jupyter, so learning it pays off twice.

Install and start

pip install ipython
ipython

Your prompt becomes In [1]:, and every input gets a number — which matters for recalling results.

Tab completion and introspection

The two features you will use sixty times a day:

In [1]: import json

In [2]: json.du      # press TAB -> completes to json.dumps

Type json.du and hit Tab: IPython completes the name, and on a second Tab lists every candidate. That is how you discover an API instead of grepping documentation.

Object discovery works the same way:

In [3]: data = {"a": 1, "b": 2}
In [4]: data.        # TAB lists get, items, keys, pop, update, values...

data.<TAB> shows the methods. data.ke<TAB> completes to keys.

? and ??: documentation without a browser

In [5]: json.dump?
Signature: json.dump(obj, fp, *, skipkeys=False, ensure_ascii=True, ...)
Docstring: Serialize obj as a JSON formatted stream to fp...

A single ? prints the signature and docstring. A double ?? prints the full source code when IPython can find it — invaluable for a third-party function whose behaviour surprises you.

You can also star a namespace:

In [6]: json.*dump*?
json.dump
json.dumps

That is a wildcard search across names, something the plain REPL cannot do at all.

History and previous results

In and Out are real objects, and _ is shorthand for the last output:

In [7]: 21 * 2
Out[7]: 42

In [8]: _ + 8          # _ is the last output
Out[8]: 50

In [9]: _7            # result of input 7
Out[9]: 42

In [10]: %history 5   # last 5 inputs

That beats re-typing an expression, and _N lets you pull any earlier result by its input number — extremely useful when you computed something six steps ago and lost the variable name.

Magic commands that earn their keep

Magic commands start with % (single line) or %% (whole cell).

In [11]: %timeit sum(range(1000))
27.4 us +- 0.6 us per loop (mean +- std. dev. of 7 runs, 10000 loops each)

%timeit runs the statement many times and reports a distribution. Use it before every optimisation claim — guessing is how you make code slower.

In [12]: %run organize.py        # execute a file in this namespace
In [13]: %ls                     # list files (works on Windows too)
In [14]: %cd reports             # change directory
In [15]: %who                    # what names exist here
In [16]: %load_ext autoreload
In [17]: %autoreload 2           # reload edited modules automatically

%run is the one you will use most: it executes a script as if you typed it, so its variables and functions land in your session for inspection afterwards. %autoreload 2 at the top of a session means editing a module file and re-calling its function picks up your changes without restarting the kernel.

Writing a file from inside the shell:

In [18]: %%writefile greet.py
   ...: def greet(name):
   ...:     return f"Hello, {name}!"
   ...:
Writing greet.py

Complete executable example

# ipython_workflow.py — a realistic prototyping session, reproduced as a script.
# Run it as a script to see the outputs; run the same lines in IPython to feel the workflow.

import timeit

def naive_square_sum(values):
    """Append in a loop — the obvious implementation."""
    out = []
    for v in values:
        out.append(v * v)
    return out

def comprehension_square_sum(values):
    """List comprehension — the idiomatic implementation."""
    return [v * v for v in values]

values = list(range(1000))

naive = timeit.timeit(lambda: naive_square_sum(values), number=1000)
idiom = timeit.timeit(lambda: comprehension_square_sum(values), number=1000)

print(f"loop        : {naive:.4f}s")
print(f"comprehension: {idiom:.4f}s")
print(f"speedup      : {naive / idiom:.2f}x faster")

# in IPython the same lines are one keystroke apart:
#   %timeit naive_square_sum(values)
#   %timeit comprehension_square_sum(values)

assert naive_square_sum(values) == comprehension_square_sum(values)
print("results identical:", True)

# history intuition: in IPython, `_` is the last output and `_N` is output N.
last = naive / idiom
print(f"last result reusable via _ -> {last:.2f}")

Line by line: timeit.timeit accepts a callable and runs it number times, returning total seconds; comparing both implementations proves the comprehension is not only shorter but measurably faster; the assert guards against a “faster” version that computes something different; and the trailing comment shows the two-line IPython equivalent you would actually type.

Common mistakes and edge cases

• Using IPython syntax in a .py file — %timeit and ? are not Python. They work in IPython and Jupyter only; a script using them fails on python script.py.
• Assuming %run isolates variables — it executes in your namespace, so a script defining config overwrites your config. Rename or use %run -i deliberately.
• Trusting a single %timeit run — it reports mean and standard deviation; a huge deviation means run it again, ideally with -n and -r to control loops and repeats.
• Forgetting to restart after edits — without autoreload, stale modules cause “my fix isn’t working” confusion. Enable %autoreload 2 or restart the kernel.
• Tab completion appears dead — some terminals bind Tab themselves. Install/configure pyreadline3 on Windows or use the newer prompt_toolkit-based shell (the default in current IPython) in a modern terminal.

Key takeaways and challenge

• Tab completion and ?/?? replace context-switching to documentation.
• _, _N, and %history recover anything you computed earlier.
• %timeit measures; %run executes; %autoreload 2 keeps modules fresh.

Challenge: open IPython, %timeit three implementations of the same task (a loop, a comprehension, and map with a lambda), and write down which wins and by how much. Then run %history -g dict to search every command you have ever typed containing dict — that is your own personal cheat sheet.

This closes Module 2. Module 3 next: web scraping with Beautiful Soup and numerical work with NumPy. Want one-to-one help getting ramped in Python? Ampersand Academy offers hands-on training.