path: root/Documentation/admin-guide/mm/damon/usage.rst

.. SPDX-License-Identifier: GPL-2.0

===============
Detailed Usages
===============

DAMON provides below interfaces for different users.

- *Special-purpose DAMON modules.*
  :ref:`This <damon_modules_special_purpose>` is for people who are building,
  distributing, and/or administrating the kernel with special-purpose DAMON
  usages.  Using this, users can use DAMON's major features for the given
  purposes in build, boot, or runtime in simple ways.
- *DAMON user space tool.*
  `This <https://github.com/damonitor/damo>`_ is for privileged people such as
  system administrators who want a just-working human-friendly interface.
  Using this, users can use the DAMON’s major features in a human-friendly way.
  It may not be highly tuned for special cases, though.  For more detail,
  please refer to its `usage document
  <https://github.com/damonitor/damo/blob/next/USAGE.md>`_.
- *sysfs interface.*
  :ref:`This <sysfs_interface>` is for privileged user space programmers who
  want more optimized use of DAMON.  Using this, users can use DAMON’s major
  features by reading from and writing to special sysfs files.  Therefore,
  you can write and use your personalized DAMON sysfs wrapper programs that
  reads/writes the sysfs files instead of you.  The `DAMON user space tool
  <https://github.com/damonitor/damo>`_ is one example of such programs.
- *Kernel Space Programming Interface.*
  :doc:`This </mm/damon/api>` is for kernel space programmers.  Using this,
  users can utilize every feature of DAMON most flexibly and efficiently by
  writing kernel space DAMON application programs for you.  You can even extend
  DAMON for various address spaces.  For detail, please refer to the interface
  :doc:`document </mm/damon/api>`.

.. _sysfs_interface:

sysfs Interface
===============

DAMON sysfs interface is built when ``CONFIG_DAMON_SYSFS`` is defined.  It
creates multiple directories and files under its sysfs directory,
``<sysfs>/kernel/mm/damon/``.  You can control DAMON by writing to and reading
from the files under the directory.

For a short example, users can monitor the virtual address space of a given
workload as below. ::

    # cd /sys/kernel/mm/damon/admin/
    # echo 1 > kdamonds/nr_kdamonds && echo 1 > kdamonds/0/contexts/nr_contexts
    # echo vaddr > kdamonds/0/contexts/0/operations
    # echo 1 > kdamonds/0/contexts/0/targets/nr_targets
    # echo $(pidof <workload>) > kdamonds/0/contexts/0/targets/0/pid_target
    # echo on > kdamonds/0/state

Files Hierarchy
---------------

The files hierarchy of DAMON sysfs interface is shown below.  In the below
figure, parents-children relations are represented with indentations, each
directory is having ``/`` suffix, and files in each directory are separated by
comma (",").

.. parsed-literal::

    :ref:`/sys/kernel/mm/damon <sysfs_root>`/admin
    │ :ref:`kdamonds <sysfs_kdamonds>`/nr_kdamonds
    │ │ :ref:`0 <sysfs_kdamond>`/state,pid,refresh_ms
    │ │ │ :ref:`contexts <sysfs_contexts>`/nr_contexts
    │ │ │ │ :ref:`0 <sysfs_context>`/avail_operations,operations,addr_unit,
    │ │ │ │   pause
    │ │ │ │ │ :ref:`monitoring_attrs <sysfs_monitoring_attrs>`/
    │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
    │ │ │ │ │ │ │ intervals_goal/access_bp,aggrs,min_sample_us,max_sample_us
    │ │ │ │ │ │ nr_regions/min,max
    │ │ │ │ │ │ :ref:`probes <damon_usage_sysfs_probes>`/nr_probes
    │ │ │ │ │ │ │ 0/weight
    │ │ │ │ │ │ │ │ filters/nr_filters
    │ │ │ │ │ │ │ │ │ 0/type,matching,allow,path
    │ │ │ │ │ │ │ │ │ ...
    │ │ │ │ │ │ │ ...
    │ │ │ │ │ :ref:`targets <sysfs_targets>`/nr_targets
    │ │ │ │ │ │ :ref:`0 <sysfs_target>`/pid_target,obsolete_target
    │ │ │ │ │ │ │ :ref:`regions <sysfs_regions>`/nr_regions
    │ │ │ │ │ │ │ │ :ref:`0 <sysfs_region>`/start,end
    │ │ │ │ │ │ │ │ ...
    │ │ │ │ │ │ ...
    │ │ │ │ │ :ref:`schemes <sysfs_schemes>`/nr_schemes
    │ │ │ │ │ │ :ref:`0 <sysfs_scheme>`/action,target_nid,apply_interval_us
    │ │ │ │ │ │ │ :ref:`access_pattern <sysfs_access_pattern>`/
    │ │ │ │ │ │ │ │ sz/min,max
    │ │ │ │ │ │ │ │ nr_accesses/min,max
    │ │ │ │ │ │ │ │ age/min,max
    │ │ │ │ │ │ │ :ref:`quotas <sysfs_quotas>`/ms,bytes,reset_interval_ms,
    │ │ │ │ │ │ │     effective_bytes,goal_tuner,
    │ │ │ │ │ │ │     fail_charge_num,fail_charge_denom
    │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
    │ │ │ │ │ │ │ │ :ref:`goals <sysfs_schemes_quota_goals>`/nr_goals
    │ │ │ │ │ │ │ │ │ 0/target_metric,target_value,current_value,nid,path
    │ │ │ │ │ │ │ :ref:`watermarks <sysfs_watermarks>`/metric,interval_us,high,mid,low
    │ │ │ │ │ │ │ :ref:`{core_,ops_,}filters <sysfs_filters>`/nr_filters
    │ │ │ │ │ │ │ │ 0/type,matching,allow,memcg_path,addr_start,addr_end,damon_target_idx,min,max
    │ │ │ │ │ │ │ :ref:`dests <damon_sysfs_dests>`/nr_dests
    │ │ │ │ │ │ │ │ 0/id,weight
    │ │ │ │ │ │ │ :ref:`stats <sysfs_schemes_stats>`/nr_tried,sz_tried,nr_applied,sz_applied,sz_ops_filter_passed,qt_exceeds,nr_snapshots,max_nr_snapshots
    │ │ │ │ │ │ │ :ref:`tried_regions <sysfs_schemes_tried_regions>`/total_bytes
    │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age,sz_filter_passed
    │ │ │ │ │ │ │ │ │ probes
    │ │ │ │ │ │ │ │ │ │ 0/hits
    │ │ │ │ │ │ │ │ │ │ ...
    │ │ │ │ │ │ │ │ ...
    │ │ │ │ │ │ ...
    │ │ │ │ ...
    │ │ ...

.. _sysfs_root:

Root
----

The root of the DAMON sysfs interface is ``<sysfs>/kernel/mm/damon/``, and it
has one directory named ``admin``.  The directory contains the files for
privileged user space programs' control of DAMON.  User space tools or daemons
having the root permission could use this directory.

.. _sysfs_kdamonds:

kdamonds/
---------

Under the ``admin`` directory, one directory, ``kdamonds``, which has files for
controlling the kdamonds (refer to
:ref:`design <damon_design_execution_model_and_data_structures>` for more
details) exists.  In the beginning, this directory has only one file,
``nr_kdamonds``.  Writing a number (``N``) to the file creates the number of
child directories named ``0`` to ``N-1``.  Each directory represents each
kdamond.

.. _sysfs_kdamond:

kdamonds/<N>/
-------------

In each kdamond directory, three files (``state``, ``pid`` and ``refresh_ms``)
and one directory (``contexts``) exist.

Reading ``state`` returns ``on`` if the kdamond is currently running, or
``off`` if it is not running.

Users can write below commands for the kdamond to the ``state`` file.

- ``on``: Start running.
- ``off``: Stop running.
- ``commit``: Read the user inputs in the sysfs files except ``state`` file
  again.  Monitoring :ref:`target region <sysfs_regions>` inputs are also be
  ignored if no target region is specified.
- ``update_tuned_intervals``: Update the contents of ``sample_us`` and
  ``aggr_us`` files of the kdamond with the auto-tuning applied ``sampling
  interval`` and ``aggregation interval`` for the files.  Please refer to
  :ref:`intervals_goal section <damon_usage_sysfs_monitoring_intervals_goal>`
  for more details.
- ``commit_schemes_quota_goals``: Read the DAMON-based operation schemes'
  :ref:`quota goals <sysfs_schemes_quota_goals>`.
- ``update_schemes_stats``: Update the contents of stats files for each
  DAMON-based operation scheme of the kdamond.  For details of the stats,
  please refer to :ref:`stats section <sysfs_schemes_stats>`.
- ``update_schemes_tried_regions``: Update the DAMON-based operation scheme
  action tried regions directory for each DAMON-based operation scheme of the
  kdamond.  For details of the DAMON-based operation scheme action tried
  regions directory, please refer to
  :ref:`tried_regions section <sysfs_schemes_tried_regions>`.
- ``update_schemes_tried_bytes``: Update only ``.../tried_regions/total_bytes``
  files.
- ``clear_schemes_tried_regions``: Clear the DAMON-based operating scheme
  action tried regions directory for each DAMON-based operation scheme of the
  kdamond.
- ``update_schemes_effective_quotas``: Update the contents of
  ``effective_bytes`` files for each DAMON-based operation scheme of the
  kdamond.  For more details, refer to :ref:`quotas directory <sysfs_quotas>`.

If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.

Users can ask the kernel to periodically update files showing auto-tuned
parameters and DAMOS stats instead of manually writing
``update_tuned_intervals`` like keywords to ``state`` file.  For this, users
should write the desired update time interval in milliseconds to ``refresh_ms``
file.  If the interval is zero, the periodic update is disabled.  Reading the
file shows currently set time interval.

``contexts`` directory contains files for controlling the monitoring contexts
that this kdamond will execute.

.. _sysfs_contexts:

kdamonds/<N>/contexts/
----------------------

In the beginning, this directory has only one file, ``nr_contexts``.  Writing a
number (``N``) to the file creates the number of child directories named as
``0`` to ``N-1``.  Each directory represents each monitoring context (refer to
:ref:`design <damon_design_execution_model_and_data_structures>` for more
details).  At the moment, only one context per kdamond is supported, so only
``0`` or ``1`` can be written to the file.

.. _sysfs_context:

contexts/<N>/
-------------

In each context directory, four files (``avail_operations``, ``operations``,
``addr_unit`` and ``pause``) and three directories (``monitoring_attrs``,
``targets``, and ``schemes``) exist.

DAMON supports multiple types of :ref:`monitoring operations
<damon_design_configurable_operations_set>`, including those for virtual address
space and the physical address space.  You can get the list of available
monitoring operations set on the currently running kernel by reading
``avail_operations`` file.  Based on the kernel configuration, the file will
list different available operation sets.  Please refer to the :ref:`design
<damon_operations_set>` for the list of all available operation sets and their
brief explanations.

You can set and get what type of monitoring operations DAMON will use for the
context by writing one of the keywords listed in ``avail_operations`` file and
reading from the ``operations`` file.

``addr_unit`` file is for setting and getting the :ref:`address unit
<damon_design_addr_unit>` parameter of the operations set.

``pause`` file is for setting and getting the :ref:`pause request
<damon_design_execution_model_and_data_structures>` parameter of the context.

.. _sysfs_monitoring_attrs:

contexts/<N>/monitoring_attrs/
------------------------------

Files for specifying attributes of the monitoring including required quality
and efficiency of the monitoring are in ``monitoring_attrs`` directory.
Specifically, three directories, ``intervals``, ``nr_regions`` and ``probes``
exist in this directory.

Under ``intervals`` directory, three files for DAMON's sampling interval
(``sample_us``), aggregation interval (``aggr_us``), and update interval
(``update_us``) exist.  You can set and get the values in micro-seconds by
writing to and reading from the files.

Under ``nr_regions`` directory, two files for the lower-bound and upper-bound
of DAMON's monitoring regions (``min`` and ``max``, respectively), which
controls the monitoring overhead, exist.  You can set and get the values by
writing to and reading from the files.

For more details about the intervals and monitoring regions range, please refer
to the Design document (:doc:`/mm/damon/design`).

.. _damon_usage_sysfs_monitoring_intervals_goal:

contexts/<N>/monitoring_attrs/intervals/intervals_goal/
-------------------------------------------------------

Under the ``intervals`` directory, one directory for automated tuning of
``sample_us`` and ``aggr_us``, namely ``intervals_goal`` directory also exists.
Under the directory, four files for the auto-tuning control, namely
``access_bp``, ``aggrs``, ``min_sample_us`` and ``max_sample_us`` exist.
Please refer to  the :ref:`design document of the feature
<damon_design_monitoring_intervals_autotuning>` for the internal of the tuning
mechanism.  Reading and writing the four files under ``intervals_goal``
directory shows and updates the tuning parameters that described in the
:ref:`design doc <damon_design_monitoring_intervals_autotuning>` with the same
names.  The tuning starts with the user-set ``sample_us`` and ``aggr_us``.  The
tuning-applied current values of the two intervals can be read from the
``sample_us`` and ``aggr_us`` files after writing ``update_tuned_intervals`` to
the ``state`` file.

.. _damon_usage_sysfs_probes:

contexts/<N>/monitoring_attrs/probes/
-------------------------------------

A directory for registering :ref:`data attributes monitoring
<damon_design_data_attrs_monitoring>` probes.

In the beginning, this directory has only one file, ``nr_probes``.  Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``.  Each directory represents each monitoring probe.

In each probe directory, one directory, ``filters`` exists.  The directory
contains files for installing filters for the probe, that is used to determine
the data attribute for the probe.

Each probe directory also contains ``weight`` file.  Reading from and writing
to the file gets and sets the :ref:`attributes-only monitoring
<damon_design_attrs_only_monitoring>` weight for the attribute of the probe.

In the beginning, ``filters`` directory has only one file, ``nr_filters``.
Writing a number (``N``) to the file creates the number of child directories
named ``0`` to ``N-1``.  Each directory represents each filter and works in a
way similar to that for :ref:`DAMOS filter <sysfs_filters>`.  When the filter
``type`` is ``memcg``, ``path`` file acts as ``memcg_path`` for :ref:`DAMOS
filter <sysfs_filters>`.

.. _sysfs_targets:

contexts/<N>/targets/
---------------------

In the beginning, this directory has only one file, ``nr_targets``.  Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``.  Each directory represents each monitoring target.

.. _sysfs_target:

targets/<N>/
------------

In each target directory, two files (``pid_target`` and ``obsolete_target``)
and one directory (``regions``) exist.

If you wrote ``vaddr`` to the ``contexts/<N>/operations``, each target should
be a process.  You can specify the process to DAMON by writing the pid of the
process to the ``pid_target`` file.

Users can selectively remove targets in the middle of the targets array by
writing non-zero value to ``obsolete_target`` file and committing it (writing
``commit`` to ``state`` file).  DAMON will remove the matching targets from its
internal targets array.  Users are responsible to construct target directories
again, so that those correctly represent the changed internal targets array.


.. _sysfs_regions:

targets/<N>/regions
-------------------

In case of ``fvaddr`` or ``paddr`` monitoring operations sets, users are
required to set the monitoring target address ranges.  In case of ``vaddr``
operations set, it is not mandatory, but users can optionally set the initial
monitoring region to specific address ranges.  Please refer to the :ref:`design
<damon_design_vaddr_target_regions_construction>` for more details.

For such cases, users can explicitly set the initial monitoring target regions
as they want, by writing proper values to the files under this directory.

In the beginning, this directory has only one file, ``nr_regions``.  Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``.  Each directory represents each initial monitoring target region.

If ``nr_regions`` is zero when committing new DAMON parameters online (writing
``commit`` to ``state`` file of :ref:`kdamond <sysfs_kdamond>`), the commit
logic ignores the target regions.  In other words, the current monitoring
results for the target are preserved.

.. _sysfs_region:

regions/<N>/
------------

In each region directory, you will find two files (``start`` and ``end``).  You
can set and get the start and end addresses of the initial monitoring target
region by writing to and reading from the files, respectively.

Each region should not overlap with others.  ``end`` of directory ``N`` should
be equal or smaller than ``start`` of directory ``N+1``.

.. _sysfs_schemes:

contexts/<N>/schemes/
---------------------

The directory for DAMON-based Operation Schemes (:ref:`DAMOS
<damon_design_damos>`).  Users can get and set the schemes by reading from and
writing to files under this directory.

In the beginning, this directory has only one file, ``nr_schemes``.  Writing a
number (``N``) to the file creates the number of child directories named ``0``
to ``N-1``.  Each directory represents each DAMON-based operation scheme.

.. _sysfs_scheme:

schemes/<N>/
------------

In each scheme directory, nine directories (``access_pattern``, ``quotas``,
``watermarks``, ``core_filters``, ``ops_filters``, ``filters``, ``dests``,
``stats``, and ``tried_regions``) and three files (``action``, ``target_nid``
and ``apply_interval_us``) exist.

The ``action`` file is for setting and getting the scheme's :ref:`action
<damon_design_damos_action>`.  The keywords that can be written to and read
from the file and their meaning are same to those of the list on
:ref:`design doc <damon_design_damos_action>`.

The ``target_nid`` file is for setting the migration target node, which is
only meaningful when the ``action`` is either ``migrate_hot`` or
``migrate_cold``.

The ``apply_interval_us`` file is for setting and getting the scheme's
:ref:`apply_interval <damon_design_damos>` in microseconds.

.. _sysfs_access_pattern:

schemes/<N>/access_pattern/
---------------------------

The directory for the target access :ref:`pattern
<damon_design_damos_access_pattern>` of the given DAMON-based operation scheme.

Under the ``access_pattern`` directory, three directories (``sz``,
``nr_accesses``, and ``age``) each having two files (``min`` and ``max``)
exist.  You can set and get the access pattern for the given scheme by writing
to and reading from the ``min`` and ``max`` files under ``sz``,
``nr_accesses``, and ``age`` directories, respectively.  Note that the ``min``
and the ``max`` form a closed interval.

.. _sysfs_quotas:

schemes/<N>/quotas/
-------------------

The directory for the :ref:`quotas <damon_design_damos_quotas>` of the given
DAMON-based operation scheme.

Under ``quotas`` directory, seven files (``ms``, ``bytes``,
``reset_interval_ms``, ``effective_bytes``, ``goal_tuner``, ``fail_charge_num``
and ``fail_charge_denom``) and two directories (``weights`` and ``goals``)
exist.

You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and
``reset interval`` in milliseconds by writing the values to the three files,
respectively.  Then, DAMON tries to use only up to ``time quota`` milliseconds
for applying the ``action`` to memory regions of the ``access_pattern``, and to
apply the action to only up to ``bytes`` bytes of memory regions within the
``reset_interval_ms``.  Setting both ``ms`` and ``bytes`` zero disables the
quota limits unless at least one :ref:`goal <sysfs_schemes_quota_goals>` is
set.

You can set the goal-based effective quota auto-tuning algorithm to use, by
writing the algorithm name to ``goal_tuner`` file.  Reading the file returns
the currently selected tuner algorithm.  Refer to the design documentation of
:ref:`automatic quota tuning goals <damon_design_damos_quotas_auto_tuning>` for
the background design of the feature and the name of the selectable algorithms.
Refer to :ref:`goals directory <sysfs_schemes_quota_goals>` for the goals
setup.

You can set the action-failed memory quota charging ratio by writing the
numerator and the denominator for the ratio to ``fail_charge_num`` and
``fail_charge_denom`` files, respectively.  Reading those files will return th