Summary of what we have covered

  • Simple remote execution of functions, or statements
  • Distribution of data
  • Parallel map
  • Multiplexed execution
  • Load-Balanced task farming
  • Asynchronous task submission and result retrieval

And further things we have not covered

  • Functional Task Dependencies
  • Configuring the IPython cluster (working with profiles, SGE, MongoDB, etc.)
  • Tuning the IPython Cluster (TaskScheduler.hwm, custom serialization, etc.)
  • Inter-engine communication with PyZMQ or MPI
  • decorators for parallel and remote functions
  • IPython %magics for some parallel execution
  • Result caching, and task metadata
  • The TaskResult Database
  • Task resubmission
  • Queue monitoring and management

See the IPython.parallel docs for more detailed coverage of these topics, and check out our examples.

In [ ]: