IDL Magics for the IPython Notebook

by eric

The IPython Notebook combines code, documentation, and computational results in one package that’s easy to share.  It’s proving a great way to teach, as notebooks are easy for the instructor to write and for students to modify.  Notebooks also provide seamless integration with other programming environments through extensions providing “magic functions“: short code prefixes beginning with % or %% that call other interpreters, like R or Octave.

Since many astronomers who might want to transition to Python already know IDL, I wanted to provide an %idl magic function for IPython so they could easily call existing code.  The difficult interfacing between IDL and Python was already done by Anthony Smith‘s pIDLy package.  (As a bonus, it supports the free GDL interpreter as well.)  I adapted the Octave magic code to provide %idl magic functions with consistent syntax.  You can find the code on github as ipython-idlmagic.

The demonstration notebook below is available on github as well, or you can view it with nbviewer.

Installation

To begin, we install pIDLy:

pip install pidly

Then we install idlmagic:

In [ ]:
%install_ext https://raw.github.com/ebellm/ipython-idlmagic/master/idlmagic.py

Usage

When starting a new notebook, we load the magic:

In [1]:
%load_ext idlmagic
IDL not found, using GDL

(I am using GDL rather than IDL on this computer. idlmagic will first look for the idl interpreter on the search path and and fall back to gdl if needed.)

Line magics

The %idl magic enables one-line execution of IDL commands in the IPython interpreter or notebook:

In [2]:
%idl print, findgen(5)
      0.00000      1.00000      2.00000      3.00000      4.00000

(Note that the %idl line magic fails with TypeError: coercing to Unicode: need string or buffer, dict found in current release versions of IPython (0.13.2 and below) due to a known bug; the github development version of IPython works as expected.)

Cell magics

Multi-line input can be entered with the %%idl cell magic:

In [3]:
%%idl
x = findgen(5)
y = x^2.
; comments are supported
print, $ ; as are line continuations
mean(y)
% Compiled module: MEAN.
      6.00000

Passing variables between Python and IDL

The mechanisms for passing variables to and from IDL are based on those in the built-in %R and %octave magics.

Variables may be pushed from Python into IDL with %idl_push:

In [4]:
msg = '  padded   string   '
import numpy as np
arr = np.arange(5)
In [5]:
%idl_push msg arr
In [6]:
%%idl
print, strcompress(msg,/REMOVE_ALL)
print, reverse(arr)
paddedstring
              4                     3                     2
              1                     0

Similarly, variables can be pulled from IDL back to Python with %idl_pull:

In [7]:
%idl arr += 1
In [8]:
%idl_pull arr
In [9]:
arr
Out[9]:
array([1, 2, 3, 4, 5])

Variables can also be pushed and pulled from IDL inline using the -i (or --input) and -o (or --output) flags:

In [10]:
Z = np.array([1, 4, 5, 10])
In [11]:
%idl -i Z -o W W = sqrt(Z)
In [12]:
W
Out[12]:
array([ 1.        ,  2.        ,  2.23606801,  3.1622777 ])

Plotting

Inline plots are displayed automatically by the IPython notebook. IDL Direct graphics are used. The optional -s width,height argument (or --size; default: 600,375) specifies the size of the resulting png image.

In [13]:
%%idl -s 400,400
plot,findgen(10),xtitle='X',ytitle='Y'
% Compiled module: WRITE_PNG.

Known issues and limitations

  • The %idl line magic fails with TypeError: coercing to Unicode: need string or buffer, dict found in current release versions of IPython (0.13.2 and below) due to a known bug; the github development version of IPython works as expected.
  • Only one plot can be rendered per cell
  • Processing for possibly unused plot output slows execution
  • Scalar variables from IDL may be returned as single-element Numpy arrays

The plotting capabilities are rather kludgy due to IDL’s old-school graphics routines.  I opted to implement Direct Graphics, which are the lowest common denominator supported by IDL and GDL.  Since I have to initialize the device before the plot call and close it afterwards, the %idl magic can only produce one plot output per notebook cell.  The plotting code produces overhead on non-plotting lines as well, unfortunately; I chose syntactic simplicity over execution speed for the time being.

Stay tuned to see it in action!