AMMPI

Active Messages over MPI

Abstract

Active Messages (AM) is a lightweight messaging protocol used to optimize network communications with an emphasis on reducing latency by removing software overheads associated with buffering and providing applications with direct user-level access to the network hardware [1]. AM provides low-level asymmetric network messaging primitives which have come into popular use as a low-level substrate in the implementations of higher-level parallel languages and systems, for example Split-C, Titanium, and others [3], [4].

Many implementations of AM are highly hardware-specific, and in particular they usually require the support of high-performance, non-commodity "smart" network interfaces such as Myrinet [5] (a noteable exception being AM-UDP [6]). This unfortunately presents a problem when trying to run AM-based software on systems that have commodity network interface hardware, or network interfaces for which no AM implementation is readily available. This first part of this project attempts to bridge that gap by providing an AM-2 implementation that runs on MPI version 1.1 [1], a standard high-performance networking layer that has been widely implemented on a number of parallel systems, and is often carefully tuned by the vendor for optimal performance. We seek to provide a compatibility layer that will allow AM-based systems to quickly get up and running on virtually any MPi-enabled platform, and turn a careful eye towards maintaining the high performance provided by the MPI layer.

AMMPI is a production-quality AM layer that provides a number of features in addition to the standard AM-2 API, including a helper library for SPMD job startup and an API to access built-in networking utilization and performance statistics. It has been successfully ported to a number of parallel architectures, and porting to new architectures generally requires minimal effort. We present performance numbers for various popular systems.

References

[1]    Active Messages - 2 Specification. https://www2.eecs.berkeley.edu/Pubs/TechRpts/1996/5768.html (archived PDF)
[2]    Message Passing Interface (MPI) Forum home page. http://www.mpi-forum.org/
[3]    Split-C. https://en.wikipedia.org/wiki/Split-C
[4]    Titanium home page. http://titanium.cs.berkeley.edu
[5]    Myrinet home page. http://www.myricom.com/
[6]    AMUDP home page. https://gasnet.lbl.gov/amudp

Slides

Bonachea, D. "AMMPI: Active Messages over MPI - Quick Overview", unpublished. (ppt)

The measurements are based on version 0.2, which is available in the download archive.

Software

Download AMMPI, version 2.10

An implementation of the AM-2 spec over MPI that should run on any MPI 1.1 or newer implementation on any standard POSIX system (including Mac OSX and MS Windows with Cygwin).
README with ChangeLog
Browse the full archive

Archived versions

Note: AMMPI is now included as a component in the GASNet communication system, which provides a high-performance one-sided communication layer used as a compilation target for several Partitioned Global Address Space languages (including UPC and Titanium).

Questions or comments, email Dan Bonachea