IPython: beyond plain Python

When executing code in IPython, all valid Python syntax works as-is, but IPython provides a number of features designed to make the interactive experience more fluid and efficient.

First things first: running code, getting help

In the notebook, to run a cell of code, hit Shift-Enter. This executes the cell and puts the cursor in the next cell below, or makes a new one if you are at the end. Alternately, you can use:

  • Alt-Enter to force the creation of a new cell unconditionally (useful when inserting new content in the middle of an existing notebook).
  • Control-Enter executes the cell and keeps the cursor in the same cell, useful for quick experimentation of snippets that you don't need to keep permanently.
In [5]:
print "Hi"
Hi

Getting help:

In [1]:
?

Typing object_name? will print all sorts of details about any object, including docstrings, function definition lines (for call arguments) and constructor details for classes.

In [28]:
import collections
collections.namedtuple?
In [29]:
collections.Counter??
In [30]:
*int*?

An IPython quick reference card:

In [31]:
%quickref

Tab completion

Tab completion, especially for attributes, is a convenient way to explore the structure of any object you’re dealing with. Simply type object_name.<TAB> to view the object’s attributes. Besides Python objects and keywords, tab completion also works on file and directory names.

In [44]:
collections.

The interactive workflow: input, output, history

In [48]:
2+10
Out[48]:
12
In [49]:
_+10
Out[49]:
22

You can suppress the storage and rendering of output if you append ; to the last cell (this comes in handy when plotting with matplotlib, for example):

In [50]:
10+20;
In [51]:
_
Out[51]:
22

The output is stored in _N and Out[N] variables:

In [24]:
_16 == Out[16]
Out[24]:
True

And the last three have shorthands for convenience:

In [25]:
print 'last output:', _
print 'next one   :', __
print 'and next   :', ___
last output: True
next one   : 22
and next   : 12
In [29]:
In[17]
Out[29]:
u'_16 == Out[16]'
In [30]:
_i
Out[30]:
u'In[17]'
In [31]:
_ii
Out[31]:
u'In[17]'
In [32]:
print 'last input:', _i
print 'next one  :', _ii
print 'and next  :', _iii
last input: _ii
next one  : _i
and next  : In[17]
In [36]:
%history -n 1-5
   1: %lsmagic
   2: %magic
   3: %timeit range(10)
   4:
%%timeit
range(10)
range(100)
   5:
for i in range(10):
    print 'i',
    %timeit range(i*100)

Exercise

Write the last 10 lines of history to a file named log.py.

Accessing the underlying operating system

In [8]:
!pwd
/home/fperez/ipython/tutorial/notebooks
In [40]:
files = !ls
print "My current directory's files:"
print files
My current directory's files:
['A quick tour of the IPython notebook.ipynb', 'Cell Magics.ipynb', 'Display control.ipynb', 'figs', 'First steps.ipynb', 'Frontend-Kernel Model.ipynb', 'IPython - beyond plain Python.ipynb', 'myscript.py', 'P01 Overview and Architecture.ipynb', 'P10 Direct Interface.ipynb', 'P25 Parallel Magics.ipynb', 'P30 LoadBalancing.ipynb', 'P35 All Together Now.ipynb', 'P51 Example - Remote Iteration.ipynb', 'P55 Working with MPI.ipynb', 'P99 Summary.ipynb', 'PZ Performance.ipynb', 'Script Magics.ipynb', 'soln', 'text_analysis.py', 'Z Callbacks.ipynb']
In [41]:
!echo $files
[A quick tour of the IPython notebook.ipynb, Cell Magics.ipynb, Display control.ipynb, figs, First steps.ipynb, Frontend-Kernel Model.ipynb, IPython - beyond plain Python.ipynb, myscript.py, P01 Overview and Architecture.ipynb, P10 Direct Interface.ipynb, P25 Parallel Magics.ipynb, P30 LoadBalancing.ipynb, P35 All Together Now.ipynb, P51 Example - Remote Iteration.ipynb, P55 Working with MPI.ipynb, P99 Summary.ipynb, PZ Performance.ipynb, Script Magics.ipynb, soln, text_analysis.py, Z Callbacks.ipynb]
In [42]:
!echo {files[0].upper()}
A QUICK TOUR OF THE IPYTHON NOTEBOOK.IPYNB

Beyond Python: magic functions

The IPyhton 'magic' functions are a set of commands, invoked by prepending one or two % signs to their name, that live in a namespace separate from your normal Python variables and provide a more command-like interface. They take flags with -- and arguments without quotes, parentheses or commas. The motivation behind this system is two-fold:

  • To provide an orthogonal namespace for controlling IPython itself and exposing other system-oriented functionality.

  • To expose a calling mode that requires minimal verbosity and typing while working interactively. Thus the inspiration taken from the classic Unix shell style for commands.

In [2]:
%magic

Line vs cell magics:

In [3]:
%timeit range(10)
1000000 loops, best of 3: 195 ns per loop
In [4]:
%%timeit
range(10)
range(100)
1000000 loops, best of 3: 812 ns per loop

Line magics can be used even inside code blocks:

In [6]:
for i in range(5):
    size = i*100
    print 'size:',size, 
    %timeit range(size)
size: 0 10000000 loops, best of 3: 152 ns per loop
size: 100 1000000 loops, best of 3: 642 ns per loop
size: 200 1000000 loops, best of 3: 1.1 µs per loop
size: 300 100000 loops, best of 3: 1.68 µs per loop
size: 400 100000 loops, best of 3: 2.65 µs per loop

Magics can do anything they want with their input, so it doesn't have to be valid Python:

In [9]:
%%bash
echo "My shell is:" $SHELL
echo "My memory status is:"
free
My shell is: /bin/bash
My memory status is:
             total       used       free     shared    buffers     cached
Mem:       7992652    4763784    3228868          0     418440    1909192
-/+ buffers/cache:    2436152    5556500
Swap:      8059900          0    8059900

Another interesting cell magic: create any file you want locally from the notebook:

In [13]:
%%file test.txt
This is a test file!
It can contain anything I want...

And more...
Overwriting test.txt
In [14]:
!cat test.txt
This is a test file!
It can contain anything I want...

And more...

Let's see what other magics are currently defined in the system:

In [10]:
%lsmagic
Available line magics:
%alias  %alias_magic  %autocall  %automagic  %bookmark  %cd  %clear  %colors  %config  %connect_info  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %install_default_config  %install_ext  %install_profiles  %killbgscripts  %less  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %lsmagic  %macro  %magic  %man  %more  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %run  %save  %sc  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%bash  %%capture  %%file  %%javascript  %%latex  %%perl  %%prun  %%pypy  %%python  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit

Automagic is ON, % prefix IS NOT needed for line magics.

Running normal Python code: execution and errors

Notonly can you input normal Python code, you can even paste straight from a Python or IPython shell session:

In [61]:
>>> # Fibonacci series:
... # the sum of two elements defines the next
... a, b = 0, 1
>>> while b < 10:
...     print b
...     a, b = b, a+b
1
1
2
3
5
8
In [62]:
In [1]: for i in range(10):
   ...:     print i,
   ...:     
0 1 2 3 4 5 6 7 8 9

And when your code produces errors, you can control how they are displayed with the %xmode magic:

In [63]:
%%file mod.py

def f(x):
    return 1.0/(x-1)

def g(y):
    return f(y+1)
Writing mod.py

Now let's call the function g with an argument that would produce an error:

In [85]:
import mod
mod.g(0)
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-85-a54c5799f57e> in <module>()
      1 import mod
----> 2 mod.g(0)

/home/fperez/ipython/tutorial/notebooks/mod.py in g(y)
      4 
      5 def g(y):
----> 6     return f(y+1)

/home/fperez/ipython/tutorial/notebooks/mod.py in f(x)
      1 
      2 def f(x):
----> 3     return 1.0/(x-1)
      4 
      5 def g(y):

ZeroDivisionError: float division by zero
In [67]:
%xmode plain
mod.g(0)
Traceback (most recent call last):

  File "<ipython-input-67-5a5bcec1553f>", line 2, in <module>
    mod.g(0)

  File "mod.py", line 6, in g
    return f(y+1)

  File "mod.py", line 3, in f
    return 1.0/(x-1)

ZeroDivisionError: float division by zero
Exception reporting mode: Plain
In [68]:
%xmode verbose
mod.g(0)
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-68-81967cfaa0c3> in <module>()
      1 get_ipython().magic(u'xmode verbose')
----> 2 mod.g(0)
        global mod.g = <function g at 0x30daa28>

/home/fperez/ipython/tutorial/notebooks/mod.py in g(y=0)
      4 
      5 def g(y):
----> 6     return f(y+1)
        global f = <function f at 0x30da5f0>
        y = 0

/home/fperez/ipython/tutorial/notebooks/mod.py in f(x=1)
      1 
      2 def f(x):
----> 3     return 1.0/(x-1)
        x = 1
      4 
      5 def g(y):

ZeroDivisionError: float division by zero
Exception reporting mode: Verbose

The default %xmode is "context", which shows additional context but not all local variables. Let's restore that one for the rest of our session.

In [69]:
%xmode context
Exception reporting mode: Context

Plotting in the notebook

This imports numpy as np and matplotlib's plotting routines as plt, plus setting lots of other stuff for you to work interactivel very easily:

In [56]:
%pylab inline
Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.kernel.zmq.pylab.backend_inline].
For more information, type 'help(pylab)'.
In [80]:
x = np.linspace(0, 2*np.pi, 300)
y = np.sin(x**2)
plt.plot(x, y)
plt.title("A little chirp")
f = gcf()  # let's keep the figure object around for later...

The IPython kernel/client model

In [82]:
%connect_info
{
  "stdin_port": 54348, 
  "ip": "127.0.0.1", 
  "hb_port": 59258, 
  "key": "70a6e436-050a-4397-ba89-06b3fd67cac6", 
  "shell_port": 50951, 
  "transport": "tcp", 
  "iopub_port": 35145
}

Paste the above JSON into a file, and connect with:
    $> ipython <app> --existing <file>
or, if you are local, you can connect with just:
    $> ipython <app> --existing kernel-3ef84b8f-3688-4a33-80c7-87bfaa512b64.json 
or even just:
    $> ipython <app> --existing 
if this is the most recent IPython session you have started.
In [83]:
%qtconsole
In [ ]: