PiCo: High-Performance Data-Analytics Pipelines in C++

PiCo (Pipeline Composition) is an open-source C++11 header-only DSL for high-performance data analytics, featuring low latency, high throughput, and minimal memory footprint on multi-core platforms.

Take a look to the word-count code to see how easy is writing a PiCo pipeline!

Report bugs + get help

https://github.com/alpha-unito/pico/issues/new

Build and run tests

The following steps require cmake >=3.1 as build system.

Get PiCo:

git clone https://github.com/alpha-unito/pico.git

The current implementation is based on FastFlow as runtime system. Get it and a link it:

cd pico
git clone https://github.com/fastflow/fastflow.git
ln -s fastflow/ff .

⛑️ A better solution for including FastFlow as dependency is under development!

Build and run tests and examples: (from pico root directory)

mkdir build && cd build
cmake .. -DPICO_ENABLE_UNIT_TEST=ON
cmake --build .
ctest

Use PiCo in your code

Good news! PiCo is header-only, you do not need to build/link any library to use it in your code. Just include PiCo headers at the beginning of your source file:

#include "pico/pico.hpp"

and use good ol' compiler flags to include PiCo (and FastFlow) when compiling your app:

git clone https://github.com/alpha-unito/pico.git
cd pico
git clone https://github.com/fastflow/fastflow.git
ln -s fastflow/ff .
g++ -I/path/to/pico/include -Iff app.cc

⛑️ A modern CMake-based solution for linking PiCo (with its dependencies) is under development!

Examples

The examples folder contains some proof-of-concept applications, showing the PiCo user experience:

• word-count: PiCo pipelines 101 + visualizing application graphs
• stock-market: batch vs stream pipelines
• page-rank: iterative pipelines

PiCo Team

Maurizio Drocco maurizio.drocco@pnnl.gov (maintainer)
Claudia Misale c.misale@ibm.com (creator + co-maintainer)
Alberto Riccardo Martinelli alberto.martinelli@edu.unito.it (co-maintainer)

Contributors

Marco Aldinucci aldinuc@di.unito.it (boss)
Massimo Torquati torquati@di.unipi.it (FastFlow maintainer)
Guy Tremblay tremblay.guy@uqam.ca (DSL wizard)

How to cite PiCo

C. Misale, M. Drocco, G. Tremblay, A. R. Martinelli, and M. Aldinucci, "PiCo: High-Performance Data Analytics Pipelines in Modern C++," Future Generation Computer Systems, Volume 87, 2018.