Plug-In-Plug-Out / Programming Interface for (Afferent Stream) Processing Objects

PiPo is an extremely simple plugin API for modules processing streams of multi-dimensional data such as audio, audio descriptors, or gesture and motion data. The current version of the interface is limited to unary operations. Each PiPo module receives and produces a single stream. The elements of a stream are time-tagged or regularly sampled scalars, vectors, or two-dimensional matrices.

Context and Motivation

PiPo has been developed in the context of designing interactive audio applications in Max/MSP and on different platforms integrating applications prototyped in Max/MSP. In this context, we constantly use a set of rather simple modules forfilteringtransformingextracting, and segmenting streams of audio and/or motion capture data. The same modules are used to process incoming real-time streams dedicated to the control of other audiovisual processes and for the automatic annotation of pre-recorded or abstract data streams serving as content for resynthesis or recognition.

PiPo is designed to allow for easily applying the same set of operators on streams of different origines, different scheduling, and on different platforms.

PiPo picks up on many ideas that already contributed to the development of FTM & Co and especially the Gabor and MnM libraries as well as others that came up while developing and using FTM & Co.

PiPo has been developed in the framework of the MuBu project and the underlying formalization of streams inherits the principal features of the the MuBu track data structure representing recorded data streams.

The focussed formalization of data streams and its extremely simple C/C++ implementation also should allow for very easily recycling PiPo modules and related code fragments in any context.

Current Project State

A set of Max/MSP externals is freely distributed with MuBu for Max via the IRCAM Forum.

The available Max/MSP PiPo host externals include:

  • mubu.process, an external processing streams recorded or loaded into MuBu tracks
  • pipo and pipo~, externals without any dependencies processing Max lists and audio input, both outputting lists

The PiPo API and an example project for developers as well as Max/MSP users is distributed on GitHub. The current version 0.2 of the API consist of a single header file for modules and about 50 lines of code.


PiPo has been developed to be make things simple for users and developers.

User Level Features

  • Easy integration and customization of stream processing modules
  • Suited for filteringtransformationextraction, and segmentation algorithms
  • Real-time and offline processing
  • Applying to audio and control streams (for whom this distinction still makes sense)

Developer Level Features

  • Fast and easy implementation of modules and reuse of code for different contexts
  • C/C++ API defined by a single header file without additional dependencies
  • Core API as single abstract class defining a small set of virtual methods

PiPo Stream Attributes

PiPo streams are a sequences of frames characterized by a set of attributes. A PiPo module defines the attributes of its output stream when receiving the attributes of the input stream.

Each module can configure its internal state depending on the attributes of the input stream (e.g. memory allocation and pre-calculated state variables) before propagating its output stream attributes to the next module.

This way, the attributes of the input stream are propagated through a series of PiPo modules before starting the actual stream processing.

In summary, a PiPo stream is described by the following attributes:

  • a boolean representing whether the elements of the stream are time-tagged
  • frame rate (highest average rate for time-tagged streams)
  • lag of the output stream relative to the input
  • frame width (also number of channels or matrix columns)
  • frame size (or number of matrix rows)
  • labels (for the frame channels or columns)
  • a boolean representing whether the frames have a variable size (respecting the given frame size as maximum)
  • extent of a frame in the given domain (e.g. duration or frequency range)
  • maximum number of frames in a block exchanged between two modules

PiPo Module Parameters

The PiPo SDK comes with a template class PiPo::Attr that permits to define scalar, enum, or variable or fixed size vector attributes of a pipo module that are exposed to the host environment. Since certain parameter changes may also change the attributes of a module’s output stream, PiPo provides a mechanism for signaling these changes through the following modules to the processing environment (i.e. the PiPo host).

For MAX/MSP PiPo includes a binding to extend a PiPo class to a Max/MSP external that then declares the module’s parameters as Max/MSP attributes and implement the required setters and getters.


The PiPo API consists of an abstract class of a few virtual methods for propagating stream attributes (see above), frames, and additional processing control through a series of modules:

  • Propagating stream attributes
  • Propagating frames
  • Reset stream processing
  • Finalize stream processing
  • Propagate the change of a parameter requiring redefining the output stream attributes

An example xcode project is distributed here.

PiPo Hosts

PiPo hosts have to provide the following functionalities:

  • Providing an input stream
  • Providing a (terminal) extension of the PiPo base class to receive the output stream
  • Connecting a series of modules (or a single module) with the terminal receiver module
  • Configuring a series of modules by propagating the input stream attributes into the first module
  • Driving the stream processing by propagating the frames of the input stream into the first module
  • Controlling the stream processing by propagating PiPo control methods such as reset and finalize
  • Making sure that all PiPo methods are called respecting mutual exclusion

For now, there is no API that would support the implementation of PiPo hosts (see section Current Project State above for available hosts and code examples).


Help patch of pipo~, a simple PiPo host for Max/MSP receiving an audio stream and producing a stream of lists