.. _collectors-overview:

Collectors Overview
===================

.. automodule:: perun.collect

.. _collectors-list:

Supported Collectors
--------------------

Perun's tool suite currently contains the following three collectors:

  1. :ref:`collectors-trace` (authored by **Jirka Pavela**), collects running times of C/C++
     functions along with the size of the structures they were executed on. E.g. this collects
     resources such that function ``search`` over the class ``SingleLinkedList`` took 100ms on
     single linked list with one million elements. :ref:`collectors-trace-examples` shows
     concrete examples of profiles generated by :ref:`collectors-trace`

  2. :ref:`collectors-memory` (authored by **Radima Podola**), collects specifications of
     allocations in C/C++ programs, such as the type of allocation or the full call trace.
     :ref:`collectors-memory-examples` shows concrete generated profiles by
     :ref:`collectors-memory`.

  3. :ref:`collectors-time`, collects overall running times of arbitrary commands. Internally
     implemented as a simple wrapper over ``time`` utility

All of the listed collectors can be run from command line. For more information about command line
interface for individual collectors refer to :ref:`cli-collect-units-ref`.

Collector modules are implementation independent (hence, can be written in any language) and only
requires simple python interface registered within Perun. For brief tutorial how to create and
register new collectors in Perun refer to :ref:`collectors-custom`.

.. _collectors-trace:

Trace Collector
~~~~~~~~~~~~~~~

.. automodule:: perun.collect.trace

.. _collectors-trace-cli:

Overview and Command Line Interface
"""""""""""""""""""""""""""""""""""

.. click:: perun.collect.trace.run:trace
   :prog: perun collect trace

.. _collectors-trace-examples:

Examples
""""""""

.. literalinclude:: /../examples/complexity-sll-short.perf
    :language: json
    :linenos:
    :emphasize-lines: 2,7,12,43,47-51

The above is an example of profiled data for the simple manipulation with program with single
linked list. Profile captured running times of three functions---``SLList_init`` (an
initialization of single linked list), ``SLList_destroy`` (a destruction of single linked list) and
``SLList_search`` (search over the single linked list).

Highlighted lines show important keys and regions in the profile, e.g. the :preg:`origin`,
:preg:`collector-info` or :pkey:`resources`.

.. image:: /../examples/complexity-scatter-with-models.*

The :ref:`views-scatter` above shows the example of visualization of trace profile. Each
points corresponds to the running time of the ``SLList_search`` function over the single linked
list with ``structure-unit-size`` elements. Elements are further interleaved with set of models
obtained by :ref:`postprocessors-regression-analysis`. The light green line corresponds to `linear`
model, which seems to be the most fitting to model the performance of given function.

.. image:: /../examples/complexity-bars.*

The :ref:`views-bars` above shows the overall sum of the running times for each
``structure-unit-size`` for the ``SLList_search`` function. The interpretation highlights that the
most of the consumed running time were over the single linked lists with 41 elements.

.. image:: /../examples/complexity-flow.*

The :ref:`views-flow` above shows the trend of the average running time of the ``SLList_search``
function depending on the size of the structure we execute the search on.

.. _collectors-memory:

Memory Collector
~~~~~~~~~~~~~~~~

.. automodule:: perun.collect.memory

Overview and Command Line Interface
"""""""""""""""""""""""""""""""""""

.. click:: perun.collect.memory.run:memory
   :prog: perun collect memory

.. _collectors-memory-examples:

Examples
""""""""

.. literalinclude:: /../examples/memory-short.perf
    :language: json
    :linenos:
    :emphasize-lines: 2,7,12,22,26-56

The above is an example of profiled data on a simple binary, which makes several minor allocations.
Profile shows a simple allocation followed by deallocation and highlights important keys and
regions in the `memory` profiles, e.g. the :preg:`origin`, :preg:`collector-info` or
:pkey:`resources`

.. image:: /../examples/memory-flow.*

The :ref:`views-flow` above shows the mean of allocated amounts per each allocation site (i.e.
``uid``) in stacked mode. The stacking of the means clearly shows, where the biggest allocations
where made during the program run.

.. image:: /../examples/memory-flamegraph.*

The :ref:`views-flame-graph` is an efficient visualization of inclusive consumption of resources.
The width of the base of one flame shows the bottleneck and hotspots of profiled binaries.

.. image:: /../examples/memory-heapmap.*

The :ref:`views-heapmap` shows the address space through the time (snapshots) and visualize the
fragmentation of memory allocation per each allocation site. The `heap map` aboe shows the
difference between allocations using lists (purple), skiplists (pinkish) and standard vectors
(blue). The map itself is interactive and displays details about individual address cells.

.. image:: /../examples/memory-heatmap.*

`Heat map` is a mode of heap map, which aggregates the allocations over all of the snapshots and
uses warmer colours for address cells, where more allocations were performed.

.. _collectors-time:

Time Collector
~~~~~~~~~~~~~~

.. automodule:: perun.collect.time

Overview and Command Line Interface
"""""""""""""""""""""""""""""""""""

.. click:: perun.collect.time.run:time
   :prog: perun collect time

.. _collectors-time-examples:

Examples
""""""""

.. literalinclude:: /../examples/time-short.perf
    :language: json
    :linenos:
    :emphasize-lines: 2,7,12-18,24-28

The above is an example of profiled data using the `time` wrapper with important regions and keys
highlighted. The given command was profiled two times.

.. _collectors-custom:

Creating your own Collector
---------------------------

.. currentmodule:: perun.utils

New collectors can be registered within Perun in several steps. Internally they can be implemented
in any programming language and in order to work with Perun requires three phases to be specified
as given in :ref:`collectors-overview`---``before()``, ``collect()`` and ``after()``. Each new
collector requires a interface module ``run.py``, which contains the three functions and, moreover,
a cli API for Click_.

You can register your new collector as follows:

    1. Run ``perun utils create collect mycollector`` to generate a new modules in
       ``perun/collect`` directory with the following structure. The command takes a predefined
       templates for new collectors and creates ``__init__.py`` and ``run.py`` according to the
       supplied command line arguments (see :ref:`cli-utils-ref` for more information about
       interface of ``perun utils create`` command)::

        /perun
        |-- /collect
            |-- /mycollector
                |-- __init__.py
                |-- run.py
            |-- /trace
            |-- /memory
            |-- /time
            |-- __init__.py

    2. First, implement the ``__init__.py`` file, including the module docstring with brief
       collector descriptions and definitions of constants that are used for automatic setting of
       profiles (namely the :preg:`collector-info`) which has the following structure:

    .. literalinclude:: /_static/templates/collectors_init.py
        :language: python
        :linenos:

    3. Next, implement the ``run.py`` module with ``collect()`` function, (optionally with
       ``before()`` and ``after()``). The ``collect()`` function should do the actual collection of
       the profiling data over the given configuration. Each function should return the integer
       status of the phase, the status message (used in case of error) and dictionary including
       params passed to additional phases and 'profile' with dictionary w.r.t :ref:`profile-spec`.

    .. literalinclude:: /_static/templates/collectors_run.py
        :language: python
        :linenos:

    4. Additionally implement the command line interface function in ``run.py``, named the same as
       your collector. This function will is called from command line as ``perun collect
       mycollector`` and is based on Click_ library.

    .. literalinclude:: _static/templates/collectors_run_api.py
        :language: python
        :linenos:
        :diff: _static/templates/collectors_run.py

    5. Finally register your newly created module in :func:`get_supported_module_names` located in
       ``perun.utils.__init__.py``:

    .. literalinclude:: _static/templates/supported_module_names_collectors.py
        :language: python
        :linenos:
        :diff: _static/templates/supported_module_names.py

    6. Preferably, verify that registering did not break anything in the Perun and if you are not
       using the developer installation, then reinstall Perun::

        make test
        make install

    7. At this point you can start using your collector either using ``perun collect`` or using the
       following to set the job matrix and run the batch collection of profiles::

        perun config --edit
        perun run matrix

    8. If you think your collector could help others, please, consider making `Pull Request`_.

.. _Pull Request: https://github.com/tfiedor/perun/pull/new/develop
.. _Click: http://click.pocoo.org/5/