.. _components:

===================
Libervia Components
===================

Libervia can act as an XMPP server component, which can be seen as a generic plugin for
XMPP servers.

This page explains which components are available and how to use them.

Running a component
===================

To run a component, you'll need to know its *entry point*, which is the name of the import
name of plugin managing it. The entry point to use will be specified in the component
installation documentation.

Components are linked to a Libervia profile in the same way as normal clients. You could
set them manually, but most of time you can use the simplified configuration.

You'll also have to declare the component on your XMPP server, this is a server dependent
step and you'll have to check your server documentation for details. You'll have to
specify a **shared secret** (can also be named simply *password*) that must be set both on
the XMPP server and as the XMPP password of the Libervia profile.

Here is a list of relevant documentation for most common servers:

ejabberd
  https://docs.ejabberd.im/admin/configuration/listen-options/

MongooseIm
  https://esl.github.io/MongooseDocs/latest/configuration/listen/#xmpp-components-listenservice

OpenFire
  use the web-based admin panel

Prosody
  https://prosody.im/doc/components

Tigase
  https://docs.tigase.net/en/latest/Tigase_Administration/Components/_Components.html#external-component-configuration

Simplified Component Configuration
==================================

Most user will want to use this simplified configuration. To set up a component, you
simply need to create a configuration section with ``[component.<entry_point>]`` where
you'll set the ``jid`` and ``secret`` options. Everything will be automatically configured
on backend startup, and the component will start automatically.

settings
--------

In ``[component.<entry_point>]``:

.. autopydantic_settings:: libervia.backend.tools.config.ComponentSettings

example
-------

To set up the ``Pubsub`` component, you check the entry point in its documentation, and
find that it's ``pubsub``, then you simply put in your configuration (usually
``libervia.toml``):

.. sourcecode:: toml

  [component.pubsub]
  jid = "pubsub.example.org"
  secret = "your_shared_secret"

Manual Component Configuration
==============================

In some rare case, you may want to manually create component profiles (e.g.; if you want
to use the same component multiple times).


On Libervia, setup is done with Libervia CLI's :ref:`profile create <li_profile_create>`
command.

You'll usually want to have the component to start automatically when the backend
is started, for this you must unset the profile password (not to be confused with the XMPP
password which is the one also set on the server configuration) with ``-p ""`` and set
auto-connection with ``-A``.

You'll specify the XMPP password (also named *shared secret* in `XEP-0144`_ terminology)
with ``-x <your_shared_secret>`` and the JID to use with ``-j
<component_subdomain>.<server.tld>``.

The component entry point is specified with ``-C <entry_point>``.

.. _XEP-0144: https://xmpp.org/extensions/xep-0114.html

example
-------

Louise wants to run an ActivityPub gateway on her server ``example.org`` with the JID
``ap.example.org``. The shared secret is ``xmpp_rocks`` and she wants the component to
start automatically with the backend, thus she doesn't set a profile password. The
entry-point for ActivityPub component is ``ap-gateway``, and she wants to use the same
name for the profile. To do this, she enters the following command::

  $ li profile create ap-gateway -j ap.example.org -p "" -x xmpp_rocks -C ap-gateway -A

The component will then be started next time Libervia Backend is launched. If Louise
wants to connect it immediately, she can use::

  $ li profile connect -cp ap-gateway

Available Components
====================

Below is a list of currently available components in Libervia, and instructions on what
they do and how to use them.


File Sharing
------------

**entry_point:** ``file-sharing``

File Sharing component manage the hosting of user files. Users can upload file there using
either `Jingle File Transfer`_ or `HTTP File Upload`_.

There is no limit to the size of files which can be uploaded, but administrators can set a
quota to limit the space that can be used.

Files can be retrieved using `File Information Sharing`_, and deleted using `Ad-Hoc Commands`_.

Files can be shared with a public HTTP link, or made available only to a specified list of
entities (JIDs). Permissions can be set through Ad-Hoc Commands.

.. _Jingle File Transfer: https://xmpp.org/extensions/
.. _HTTP File Upload: https://xmpp.org/extensions/xep-0363.html
.. _File Information Sharing: https://xmpp.org/extensions/xep-0329.html
.. _Ad-Hoc Commands: https://xmpp.org/extensions/xep-0050.html

Configuration
~~~~~~~~~~~~~

All options are to be set in ``[components.file_sharing]`` section.

``http_upload_port``
  port to use for HTTP File Upload

  **default**: 8888

``http_upload_connection_type``
  either ``http`` or ``https``.

  **default**: ``https``

  Note that HTTP Upload should always be ``https`` to end-user, the ``http`` option is to
  be used only if you use a HTTP server as a proxy, and this server is already set for
  TLS.

``http_upload_public_facing_url``
  must be set to the URL that end-user will see. Notably useful if the component is behind
  a proxy.

  **default**: ``https://<component host>:<http_upload_port``

``quotas_json``
  a JSON object indicating quotas to use for users. The object can have 3 keys:

  ``admins``
    quotas to use for administrators (i.e. profiles set in ``admins_list``)

  ``users``
    quotas to use for normal users (i.e. non admin profiles)

  ``jids``
    per-jid specific quotas. The value is a JSON object where key is a user bare jid and
    value is a quota.

  Quotas can be either ``null`` for unlimited space, or a size value (`SI prefixes and
  binary prefixes`_ can be used).

  example::

    quotas_json = {
      "admins": null,
      "users": "50 Mio",
      "jids": {"pierre@example.org": "1 Gio"}
    }

  .. _SI prefixes and binary prefixes: https://en.wikipedia.org/wiki/Octet_(computing)#Unit_multiples


ActivityPub Gateway
-------------------

**entry_point:** ``ap-gateway``

This gateway will provide a bidirectional gateway between XMPP and `ActivityPub`_ (or AP
below). That means that user from XMPP will be able to follow actors or comments messages
from any software compatible with ActivityPub protocol, and vice versa.

.. _ActivityPub: https://activitypub.rocks/

.. note::

  this component is mostly tested with Prosody as XMPP server reference, and Mastodon as
  AP server reference, but it should work with any XMPP or AP server.

The component launches a HTTP server (necessary to communicate with AP software). This
server needs to handle HTTP requests made at paths ``/.well-known/webfinger`` and ``/_ap``
(or the ``ap_path`` set in configuration, see below). If the component is not directly
facing internet (e.g. integrated in an existing website though a proxy), you'll have to
redirect the requests made to those path to the HTTP server (i.e. to component host at the
port set at ``http_port``, see configuration below). Please check your HTTP server
documentation to find how this must be done.


Configuration
~~~~~~~~~~~~~

All options are to be set in ``[components.ap_gateway]`` section.

.. _ap-actor-from-xmpp:

How to Address an AP Actor from XMPP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When addressing an ActivityPub actor from XMPP, you must use a JID corresponding to the
actor. The domain part of the JID correspond to the gateway JID (the one set in gateway
profile), while the local part (before the ``@``) is used to specify the AP actor.

`XEP-0106`_ (JID Escaping) is used to indicate the AP actor identifier, thus the ``@``
must be escaped with ``\40``.

.. _XEP-0106: JID Escaping

**example**

If Louise wants to talk to Pierre which is on the ``example.net`` AP server, she can use
her XMPP AP gateway which is at ``ap.example.org``. Pierre AP's actor handle is
``pierre@example.net``, Louise can access it via the JID
``pierre\40example.net@ap.example.org`` (we call it *virtual JID* of the AP actor).

Of course, this is a bit cumbersome to do by hand, it is expected that XMPP clients will
do the (un)escaping automatically for end-user, in a way that Louise could enter
``pierre@example.net`` directly, with an indicator to show that this is an ActivityPub
actor identifier rather than an XMPP JID.


.. _xmpp-node-from-ap:

How to Address an XMPP Entity from AP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To access an XMPP entity, it is a little bit more complicated for 2 reasons:

- XMPP use a wider range of allowed characters than most AP implementations [#AP_chars]_.
- to get XMPP items, we need 2 data: the entity JID, and a pubsub node


However, Libervia AP gateway tries to make it as user friendly as possible, thus it works
like this:

- in the most common case, one just wants to access the personal blog of a user, and basic
  ASCII characters (with possibly ``-``, ``_`` or ``.``) are used. in this case, the XMPP
  JID can be directly used as AP actor handle
- when a pubsub node needs to be specified it is most of time with a pubsub JID which has
  not user part (something like ``pubsub.example.org``). In this case, the pubsub node can
  be used as AP actor handle's user part, Libervia will use XMPP discovery to know that
  it's a pubsub service. So if you want to access the blog named ``xmpp_news`` at
  ``pubsub.example.org``, you can use the handle ``xmpp_news@pubsub.example.org`` (be sure
  that the domain ``pubsub.example.org`` links to the Libervia AP gateway HTTP server)
- if you want to use a specific node with an entity which already has a user part, then a
  special encoding must be used, where ``---`` (three dashes) are used to separate node
  from entity: ``some_node--some_user@example.org``
- if you need to use special characters, then you'll have to use ``___`` followed by the
  special encoding (see below).

The encoding is explained in the documentation of the following method:

.. automethod:: libervia.backend.plugins.plugin_comp_ap_gateway.APGateway.get_jid_and_node


.. [#AP_chars] Most if not all AP implementations use webfinger `acct` URI as a de-facto
   standard to manage user-friendly handles (something like ``user@example.org``). Those
   handles, according to `RFC7565`_, should
   manage a wide variety of characters thanks to the support of percent-encoding.

   Unfortunately, at least one of the most-used AP implementation (Mastodon, which is used
   a reference implementation for this gateway), only uses a limited subset of allowed
   characters. In addition, Mastodon needs an associated handle [#m_wf]_ thus an alternate
   way to encode characters had to be found.

   An issue has been opened to improve this situation on Mastodon bug tracker:
   https://github.com/mastodon/mastodon/issues/17222

   .. _RFC7565: https://datatracker.ietf.org/doc/html/rfc7565
   .. [#m_wf] https://docs.joinmastodon.org/spec/webfinger/

**example**

If Pierre wants to talk to Louise, he can directly use the JID which is the same as the AP
actor identifier, i.e. ``louise@example.org`` (note that on AP software, a ``@`` prefix is
often required, thus Pierre will look for ``@louise@example.org``).

.. note::

   The actor endpoint can also be used directly in AP software (in the example above, it
   would be by default ``https://example.org/_ap/actor/louise%40example.org``).

.. _ap-ad-hoc:

Ad-Hoc Commands
~~~~~~~~~~~~~~~

The gateway supports `XEP-0050 (Ad-Hoc Commands)`_. For now, only the following node is
provided:

``https://libervia.org/ap_gateway/xmpp_jid_node_2_ap_actor``
  Convert XMPP JID and Node to corresponding virtual actor. Node is optional.

.. _XEP-0050 (Ad-Hoc Commands): https://xmpp.org/extensions/xep-0050.html

Getting AP Message from XMPP
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To retrieve ActivityPub messages of an actor from an XMPP client with blogging
capabilities (like Libervia or Movim), just use the associated JID as explained in
:ref:`ap-actor-from-xmpp`. The actor messages (from ``outbox`` collection) should appear
as regular XMPP blog items.

.. note::

   Due to limitation of `ActivityStream Collection Paging`_, the conversion from XMPP
   `RSM`_ requests is inneficient beyond first or last page. This problem is avoided if
   anybody subscribe to the gateway node (i.e. follow the AP actor), as the collection
   will then be cached, and efficiently delivered.

   .. _ActivityStream Collection Paging: https://www.w3.org/TR/activitystreams-core/#h-paging
   .. _RSM: https://xmpp.org/extensions/xep-0059.html

Getting XMPP Items from ActivityPub
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To get XMPP items from an ActivityPub implementation, just use the handle as explained at
:ref:`xmpp-node-from-ap` (often handle searches are triggered with a ``@`` before the
handle in AP implementations, e.g.: ``@louise@example.org``).

.. note::

   Some AP implementations such as Mastodon don't retrieve existing items, but only keep
   new ones once an actor is followed. That means that you won't see any message published
   before your entity is followed. Other implementations may work differently.


.. _ap-xmpp-threads-conversion:

ActivyPub to XMPP Discussion Threads Conversion
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By default, each (micro)blog publication converted from ActivityPub to XMPP Pubsub is
associated with a comment node (see `XEP-0277 comments`_) to allow user to post a reply on
any on them. This result in a "tree-like" comments threading model, which is similar to
what you can see on popular website such as Reddit, Hacker News, or LinuxFr.

However, some XMPP clients may not play nicely with this kind of threading model. To work
around this, the ``comments_max_depth`` option allows to limit the maximum number of
thread. It is an integer value which indicate how many comments nodes may exist for a root
post. If set to ``1``, only one comment node will be made, and ActivityPub items below
this level will be moved to this single XMPP pubsub comment node.

The default value of ``0`` means unlimited max depth.

An example can probably make it more clear. Imagine that you have a message ``root
message``, with a comment to it named ``1``, a comment to ``1`` named ``2`` and so on
until ``5``. With ``comments_max_depth`` set to ``0``, you'll have one comment node per
item, resulting in following threads model::

  root message ┑
               ┕ 1 ┑
                   ┕ 2 ┑
                       ┕ 3 ┑
                           ┕ 4 ┑
                               ┕ 5

With ``comments_max_depth`` set to ``2``, only 2 nodes will be created, and items below
depth ``2`` will be put on the same level::

  root message ┑
               ┕ 1 ┑
                   ┝ 2
                   ┝ 3
                   ┝ 4
                   ┕ 5

This way, admins can configure the model which suits best the clients which is expected to
be mainly used on the instance.

.. _XEP-0277 comments: https://xmpp.org/extensions/xep-0277.html#comments

Publishing an Item
~~~~~~~~~~~~~~~~~~

To publish a new item (e.g. a blog post), you just need to publish normally on your own
PEP/pubsub node, AP actors following you will be notified. To reply to an AP item, just
publish to the corresponding pubsub node managed by the gateway. This is transparent for
AP and XMPP end users.

For instance, if Pierre has posted an interesting message on his AP server, and Louise
wants to reply to it, she just use a client to reply on the comments node of this message,
this will be delivered as an AP object to Pierre's AP server.

On the other hand, if Louise is publishing a new blog post on her XMPP server, Pierre will
receive corresponding AP object because he's following her. If Pierre answer using his AP
client, the corresponding message will be published on the comments node of the message
that Louise has initially published.

Following, Subscribing and Cache
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When you try to access an uncached AP collection from XMPP (e.g. blog posts), a best
effort is done to translate XMPP pagination (which uses `XEP-0059 (Result Set
Management)`_) to the less powerful `AP Collection Paging`_. This is inefficient due to
technical limitations (page size can't be specified in AP, there is not standard way to
request item after or before a specific ID, implementations may not implement reverse
pagination).

That's one of the reason why whenever possible, collections are cached locally. Once
cached, it's easier to return items according to complex requests.

However, to cache correctly an AP collection, you need to keep it in sync, and thus to
receive update when something change (e.g. a new blog item is published).

In AP, this is done by following an actor, in XMPP this correspond to a node subscription.

When you subscribe to a blog node managed by this gateway, this will be translated to a
*follow* activity on AP side, and vice versa.

When an AP actor is followed, its *outbox* collection (i.e. message published), are
automatically cached, and will be updated when events will be received. That means that
you can use pubsub cache search on followed actors, e.g. to retrieve all items about a
specific topic or published at specific time range.

Reciprocally, unsubscribing from a node will *unfollow* the corresponding AP actor.

If an AP actor is following or unfollowing an actor mapping an XMPP entity, they nodes
will be subscribed to or unsubscribed from.

All subscriptions are made public as specified by `XEP-0465 (Pubsub Public Subscriptions)`_.

.. _XEP-0059 (Result Set Management): https://xmpp.org/extensions/xep-0059.html
.. _AP Collection Paging: https://www.w3.org/TR/activitystreams-core/#h-paging
.. _XEP-0465 (Pubsub Public Subscriptions): https://xmpp.org/extensions/inbox/pubsub-public-subscriptions.html

Following/Followers Collections and Public Pubsub Subscription
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The AP *following* collection is mapped to `XEP-0465 (Pubsub Public Subscriptions)`_.

In the same spirit, the AP *followers* collection correspond to public subscribers to the
microblog node.

Because AP doesn't send any event when *following* or *followers* collections are
modified, those collections can't be cached, and thus the translation to public pubsub
subscriptions is done as best as possible given the constraints.

.. _ap-message_delivery:

Messages Delivery
~~~~~~~~~~~~~~~~~

The gateway can convert AP publications to either XMPP pubsub items (using `XEP-0277
(Microblogging over XMPP)`_ when suitable) or to XMPP messages (`<message>` stanzas, i.e.
the ones used for instant messaging). Of course it also converts in a similar way in the
other direction (XMPP → ActivityPub).

A received AP item will be converted to an XMPP pubsub item if any of the following
conditions is fulfilled:

  - it is addressed to the special `*public* collection`_
  - it is addressed to a local *followers* collection

A received AP item will be converted to an XMPP message if all the following conditions
are fulfilled:

  - it is **not** addressed to the special *public* collection
  - it is **not** addressed to a any local *followers* collection.

In other words, if an AP item is addressed directly to one or more local users, is not
public and is not addressed to a *followers* collection, it will be converted to an XMPP
message, otherwise it will be converted to an XMPP pubsub item.

The behaviour is symmetric, thus if you send an XMPP message it will be converted to an AP
item which will be only addressed to your recipient. If you write using XMPP pubsub, the
suitable followers collections of the target AP actor will be automatically added, and the
item will have the special *public* collection added.

.. note::

   ActivyPub doesn't do any kind of `end-to-end encryption`_, it is **not** the same level of
   privacy as XMPP.

   Messages will be at minimum readable by the administrators of the AP gateway and of the
   ActivyPub server of the destinee. Be sure to understand that before sending messages.

   If you need more privacy, you need to use either XMPP only (without the ActivityPub
   gateway) with clients implementing correctly end-to-end encryption, or an other
   software. Depending of the level of pricacy required, you may want to use
   authentication by a separated channel, a client which has been audited, encryption by
   default, etc. This is beyond the scope of this AP gateway documentation.

   In short, don't use ActivityPub if you need a high level of privacy. This is a current
   protocol limitation, and there is nothing that this gateway can do about this until
   ActivityPub and its implementations evolve.

.. _XEP-0277 (Microblogging over XMPP): https://xmpp.org/extensions/xep-0277.html
.. _*public* collection: https://www.w3.org/TR/activitypub/#public-addressing
.. _end-to-end encryption: https://en.wikipedia.org/wiki/End-to-end_encryption

Message/Item Retraction
~~~~~~~~~~~~~~~~~~~~~~~

If you retract a pubsub item (e.g. a blog post), that is followed by ActivityPub actors,
a suitable `Delete` activity will be emitted. In other words, the ActivityPub servers will
be notified of the retraction.

Similarly if an AP actor is deleting an item, a suitable `XEP-0060 retraction event`_ will
be sent to subscribers.

In the case of direct messages (see :ref:`ap-message_delivery`), `XEP-0424 (Message
Retraction)`_ is used in both reception (AP ``Delete`` activity is converted to XEP-0424
and vice versa).

All of this should be transparent to users as long as their clients support the mentioned
XEPs.

.. note::

   When retracting a message/item, a retraction request (or equivalent AP "Delete"
   activity) is sent, so the other end clients **may** delete the item.

   However, there is no way to be sure that the item will be actually deleted: once
   something is sent on the network, any recipient can see it, copy it, share it, etc.
   There is no way to forbid that, and this is true for software, decentralized or not
   (and not only ActivityPub or XMPP).

   Be sure to understand that before sending anything sensitive.

.. note::

   When deleting an item on ActivityPub, the item is often replaced by a "tombstone", and
   it's not possible to send a new item with the same ID. Some software (it's at least the
   case with Mastodon), will silently reject the item, and thus people won't see it.

   This is a different behaviour from XMPP where you can publish a pubsub item, retract
   it, and publish a new item with the same ID.

   Thus if you retract an item, be sure to publish any new one with new and unique ID,
   otherwise the item may not appear to ActivityPub users (and as the item may be rejected
   silently, you may not be notified).

   However this should not be a problem in the vast majority of cases, as most XMPP client
   implementing pubsub or pubsub service will generate automatically unique IDs for new
   items.

.. _XEP-0060 retraction event: https://xmpp.org/extensions/xep-0060.html#publisher-delete
.. _XEP-0424 (Message Retraction): https://xmpp.org/extensions/xep-0424.html

Blocking an User
~~~~~~~~~~~~~~~~

You can block AP actors using :ref:`its virtual JID <ap-actor-from-xmpp>` in the same way
as for any XMPP entity, by using `XEP-0191 (Blocking Command)`_.

This is done at the XMPP server level and your server must support it. From client side,
many clients support it, and you can use Libervia CLI to do it from command-line (see
:ref:`libervia-cli_blocking`).

.. _XEP-0191 (Blocking Command): https://xmpp.org/extensions/xep-0191.html

Identity and Avatar
~~~~~~~~~~~~~~~~~~~

The gateway does the conversion between XMPP identity and AP actor metadata.

XMPP "identity" is actually a compilation of data coming from various locations and in
particular `XEP-0292 (vCard4 Over XMPP)`_. In those data, vCard's ``nickname`` (first
found) is used as AP ``name`` field, and vCard's ``note`` is used as AP's ``summary``
(it's generally a self-description of users).

For avatars, `XEP-0084 (User Avatar)`_ is used (and actor's ``icon`` field is used on AP
side). Avatars are downloaded the first time that they are requested, and kept in cache
for future requests.

Even if XEP-0292 and XEP-0084 are the main XEPs used, Libervia checks various locations,
and other extensions like `XEP-0054 (vcard-temp)`_ may be used as fallback.

.. _XEP-0292 (vCard4 Over XMPP): https://xmpp.org/extensions/xep-0292.html
.. _XEP-0084 (User Avatar): https://xmpp.org/extensions/xep-0084.html
.. _XEP-0054 (vcard-temp): https://xmpp.org/extensions/xep-0054.html

Mentions
~~~~~~~~

Mentions are converted by the gateway under some conditions explained below.

When receiving an AP item which converts to a pubsub item (i.e. not a direct message), for
any local user mentioned (using a tag of type ``Mention``) or directly specified in a
target field (``to``, ``bto``, ``cc`` or ``bcc``), a `XEP-0372 (References)`_ type mention
will be emitted.

This is not done for direct message as the recipients are already specified, and sending
reference is most probably not desired for a private message.

In the direction XMPP 🠞 ActivityPub, it's more complicated because XMPP sends references
separately from the pubsub item, while ActivityPub links them in the same item.

If a XEP-0372 reference targeting an AP actor is received and anchor a pubsub item, the
item will be looked after in cache. If found, it will be converted to an AP item, a
mention will be added, and the item will be delivered to the mentioned entity.

This only works if the mentioned actor is on a server which has not already received the
original item, because otherwise the AP implementation will most probably ignore the new
item which has the same ID.

It is hard to avoid sending first published item, then the same item with the reference,
because XEP-0372 references are received after the published items.

To work around that, when an XMPP blog item is received, its body is scanned to find
``@user@server.tld`` type mention, and if found a mention is automatically added to the
resulting AP item.

It is not ideal to parse the body, but that's an acceptable trade-off to make mention
working. This behaviour is activated by default, but can be deactivated by setting
``auto_mentions`` option to ``false``. Auto mentions are only used for pubsub items, and
not for direct messages for the same reason as mention are not checked in AP direct
messages (see above).

.. note::

   For auto mentions, it's the AP actor handle which must be used, not the corresponding
   virtual JID.

   If you want to mention ``louise@example.org`` then you must use directly
   ``@louise@example.org``, and NOT something like
   ``@louise\40example.org@ap.example.net``.

   To mention XMPP entities, your client must use XEP-0372 references directly.

.. _XEP-0372 (References): https://xmpp.org/extensions/xep-0372.html

Repeat/Share/Reboost
~~~~~~~~~~~~~~~~~~~~

The repeat feature (also named "share", or "boost") which consist of republishing an item
of interest under our own account to make it visible to our subscribers, is done on the
XMPP side with `XEP-0277 § Repeating a Post`_, and on the AP side it's an ``Announce``
activity. Conversion is done in both directions and should be transparent to user.

.. _XEP-0277 § Repeating a Post: https://xmpp.org/extensions/xep-0277.html#repeat

Noticed/Like
~~~~~~~~~~~~

AP's ``Like`` activity is converted on the XMPP side to "`XEP-0470 (Pubsub
Attachments)`_". ``Like`` is converted to ``noticed`` attachment, meaning that it used in
to indicate that something has been seen and taken into account, without really indication
if the content is liked or disliked (see XEP-0470 § Foreword: `"noticed" instead of "like"
or "favourite"`_ for the rational).

As usual, conversion is done in both ways.

.. _XEP-0470 (Pubsub Attachments): https://xmpp.org/extensions/xep-0470.html
.. _XMPP Council: https://xmpp.org/about/xmpp-standards-foundation/
.. _"noticed" instead of "like" or "favourite": https://xmpp.org/extensions/xep-0470.html#noticed-foreword

Reactions
~~~~~~~~~

Reactions are not specified in `base ActivityPub specification`_. However `Pleroma`_, a
software implemeting ActivityPub, implements them using a non standardised (yet?)
``EmojiReact`` activity. Liberva AP Gateway use it to handle reactions.

On the XMPP side, `XEP-0470 (Pubsub Attachments)`_ is once again used.

Libervia also implements `XEP-0444 (Message Reactions)`_ for messages, but because Pleroma
doesn't handle reactions for direct messages, this is not used in the Libervia AP gateway.
If in the future an AP implementation uses reactions on direct messages, it will be
straightforward to add it in the AP gateway.

.. _base ActivityPub specification: https://www.w3.org/TR/activitypub/
.. _Pleroma: https://pleroma.social/
.. _XEP-0444 (Message Reactions): https://xmpp.org/extensions/xep-0444.html

Events
~~~~~~

This gateway manages events. On the XMPP side, the ``Events`` protoXEP is used, on AP side
``Event`` objects are used. `Mobilizon`_ is used as reference implementation here.

.. note::

   The ``Events`` protoXEP has been proposed but not published or reviewed by XMPP council
   yet. The current version is availale at https://github.com/xsf/xeps/pull/1206 and
   hopefully it will eventually become an official experimental XEP. The code and this
   documentation will be updated as the standard evolves.

To retrieve AP events from XMPP, use the corresponding virtual JID as explained in
:ref:`ap-actor-from-xmpp` with the event node (which is ``urn:xmpp:events:0``).

*example:*

To retrieve events from AP actor with the handle ``somebody@mobilizon.example``, throught
the ``ap.example.org`` gateway, the ``somebody\40mobilizon.example@ap.example.org`` JID
can be used with the ``urn:xmpp:events:0`` node.


To retrieve XMPP events from AP, you need to the virtual actor corresponding to the JID
of the pubsub/PEP service with the event node (starting with ``urn:xmpp:events:0``), as
explained in :ref:`xmpp-node-from-ap`. You can use :ref:`ap-ad-hoc` to easily convert your
JID/node combination to a virtual AP actor. Note that the resulting virtual actor is
unaesthetic due to the presence of event node and the constraints explained in
:ref:`xmpp-node-from-ap`.

To retrieve the link to the XMPP item from AP, you can use the ID from the generated AP
object. The ID can also be constructed by hand by using the URL
``https://<ap_gateway_public_domain>/_ap/<virtual_actor>/<event_item_id>``.

*example:*

to retrieve from AP the XMPP event at pubsub service ``pubsub.example.net`` on node
``urn:xmpp:events:0/party`` with item ID ``picnic_abd1``, and using the AP gateway with
public url ``https://ap.example.net``, the following link can be used::

  https://ap.example.net/_ap/___urn.3Axmpp.3Aevents.3A0.2Fparty---pubsub.2eexample.2enet@ap.tazar3.int/picnic_abd1

If you paste it in an AP implementation featuring events (e.g. Mobilizon), the event
should appear and you should be able to comment it and participate to it.

Event comments are supported, the node to use is indicated in the event XMPP item as
specified in the protoXEP, this should be transparent to end-user.

Participation to an event is supported through the RSVP mechanism on XMPP and with
``Join`` and ``Leave`` activities on AP. Note that XMPP allows more nuances with its RSVP
mechanism (it's possible to say if someone will "maybe" attend an event, and to request
any kind of extra informations) while AP only indicates if one will attend or not an
event.


.. _Mobilizon: https://joinmobilizon.org/

Using the Component (for developers)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Publication of AP items can be tested using the following method (with can be accessed
through the ``ap_send`` bridge method, client is then replaced by the ``profile`` name, as
last argument):

.. automethod:: libervia.backend.plugins.plugin_comp_ap_gateway.APGateway.publish_message

The method can be used either with CLI's :ref:`debug bridge method
<li_debug_bridge_method>` or with any D-Bus tool like ``qdbus`` or ``d-feet`` (only if you
use the D-Bus bridge).

example
~~~~~~~

On its ``example.net`` Mastodon instance, Pierre has published a message with the id
``https://example.net/@pierre/106986412193109832``. To send a reply to this message,
Louise can use the following command::

  $ li debug bridge method -c ap_send '"{\"node\": \"https://example.net/@pierre/106986412193109832\", \"content\": \"A lille hello from XMPP\"}","pierre\\40example.net@ap.example.org", "louise"'

Note the double escaping, one for the shell argument, and the other to specify JSON
object.


Audio/Video Conferences
-----------------------

**entry_point:** ``conferences``

This component integrates a service to run multiparty audio/video conferences over XMPP.
The service is called a "Selective Forwarding Unit" (SFU), and it's main role is to
distribute participants' audio and video streams.

The component currently integrates the `Galène project`_, as it is a complete, simple, and
easy to install service. Other projects may be integrated in the future if the need
arises.

Please ensure that Galène is already installed on your device.

If the ``galene`` executable is not in your ``PATH`` environment variable, you can specify its
location using the ``galene_path`` option (see below).

Note that Galène will be automatically started and stopped by the component, you don't
have to start it yourself.

Configuration
~~~~~~~~~~~~~

All options are to be set in the ``[component conferences]`` section.

``galene_path``
  The path to the Galène executable, if it is not in the `PATH` environment variable.

  **default**: Look for ``galene`` in the ``PATH``

``http_port``
  The port used by the Galène HTTP server.

  **default**: 9443

.. _Galène project: https://galene.org/


Email Gateway
-------------

**entry_point:** ``email-gateway``

.. note::

  This component is **work in progress** and under heavy development.

This gateway links the email network and XMPP in both directions. It allows sending and
receiving emails using XMPP clients.

The gateway uses an existing email account configured using IMAP4 and SMTP. IMAP4 is used
for reception and the server must support IMAP4 rev1 protocol with the "IDLE" extension
for real-time notifications. SMTP is used for sending emails.

To use the gateway, one must register through `XEP-0100 (Gateway Interaction)`_. Once
registered, connection to the IMAP server is started immediately, and new emails in the
``INBOX`` mailbox are received as XMPP messages.

For the moment, mailboxes are used in read-only mode, meaning that the read status and
other attributes are not changed, and messages will appear normally in other MUA (email
clients).

To send a message, just write to the corresponding JID. Email addresses are mapped to JIDs
using `XEP-0106 (JID Escaping)`_: the email is escaped and used as the user-part of the
gateway JID.

Multicast addressing (i.e. the ``To``, ``CC`` and ``BCC`` email headers or their
`XEP-0033 (Extended Stanza Addressing)`_ XMPP equivalent) are supported and will be converted from and to XMPP. Other
metadata like ``Reply-To`` are also supported. You can use
:ref:`libervia-cli_message_send` to specify those metadata when sending a message.

Some headers are converted between XMPP and email, such as the ``Keywords`` or the
``Importance`` ones. On XMPP side, `XEP-0131 (Stanza Headers and Internet Metadata)`_ is
used.


Attachment Management
~~~~~~~~~~~~~~~~~~~~~

Email attachments are downloaded and stored by the gateway. Attachments can be managed
individually, independent of the email message itself. They can be retrieved and deleted
from the gateway (using `XEP-0498 (PubSub File Sharing)`_).

.. note::

   **Current Limitation:** Attachments are partially handled; while they can be retrieved
   and deleted, they are not yet sent with the message. This functionality will be added
   as development progresses.

End-to-End Encryption
~~~~~~~~~~~~~~~~~~~~~

The gateway supports end-to-end encryption through the Gateway Relayed Encryption
specification. This works by indicating to client how to format and encrypt the message
(in technical terms, it's by telling it to use MIME and OpenPGP) before sending it to the
gateway. This only works with clients implementing this specification.

Autocrypt
~~~~~~~~~

The gateway relays "Autocrypt" headers, which is a way to share public keys without
relaying them to a key server. In layman's terms, it's a way to exchange encryption data
to be able to encrypt messages automatically. However, while better than not using
end-to-end encryption, this method is not as secure as proper management of encryption
keys and authentication of the peer we're talking to.

Please refer to `Wikipedia's Autocrypt page`_ for more details, in particular on its
security model


.. note::

   Gateway Relayed Encryption is not yet an official XEP, it will be evaluated by XMPP
   council to see if it gets an official number in coming weeks.

Mailing Lists
~~~~~~~~~~~~~

Emails parts of mailing lists are detected and handled separately. They are converted to
Pubsub blog nodes: the list identifier is used as the pubsub node, and each topic is added
as a `XEP-0277 (Microblogging over XMPP)`_ item. Replies to a specific topic (i.e. in a
thread) are set as comments to the corresponding item.

If a title has some words between square brackets (``[like][this] Within the title``)
they are removed and added as blog tags (aka categories).

Only the JID associated to the recipient of the mailing list can access the pubsub node.
It will be possible to make a mailing list public in the future.

.. _email-gateway-message-cleaning:

Message Cleaning
~~~~~~~~~~~~~~~~

Emails often contain extraneous content that can make them harder to read. For example,
replying to an email in a client typically includes the entire previous message as a
quote. Similarly, mailing list software often appends signatures with
subscription/unsubscription instructions.

To improve readability, the email gateway automatically cleans up mailing list messages
during conversion. First, all text-only messages are converted to XHTML for consistency.
Then, it identifies and tags these “noisy” elements with a CSS class named ``noise``. This
allows your XMPP client to selectively hide or visually distinguish these parts of the
message.

The original content is preserved; the ``noise`` class simply provides a way for your
client to present the message more clearly. This means you can still view the full
original message if desired. The gateway attempts to intelligently detect noisy elements,
but it’s possible for some content to be incorrectly identified.



.. _XEP-0033 (Extended Stanza Addressing): https://xmpp.org/extensions/xep-0033.html
.. _XEP-0100 (Gateway Interaction): https://xmpp.org/extensions/xep-0100.html
.. _XEP-0106 (JID Escaping): https://xmpp.org/extensions/xep-0106.html
.. _XEP-0131 (Stanza Headers and Internet Metadata): https://xmpp.org/extensions/xep-0131.html
.. _XEP-0277 (Microblogging over XMPP): https://xmpp.org/extensions/xep-0277.html
.. _XEP-0498 (Pubsub File Sharing): https://xmpp.org/extensions/xep-0498.html
.. _Wikipedia's Autocrypt page: https://en.wikipedia.org/wiki/Autocrypt

Libervia Pubsub
---------------

**entry_point:** ``pubsub``

.. note::

  This component is **work in progress** and under heavy development. It is the rewrite on
  top of Libervia Backend of the independent "Libervia Pubsub" component, and has not yet
  full feature parity with it.

Libervia Pubsub is a feature-full XMPP Pubsub service that can also be used as a
`XEP-0163 (Personal Eventing Protocol)`_ service (simply named "PEP"). Pubsub/PEP is a
major part of XMPP, and this implementation aims to be the state of the art with all
important features available. It is the reference implementation for Libervia, which means
that by using it you know that you'll have all pubsub related features needed to fully
enjoy Libervia capabilities.


.. _XEP-0163 (Personal Eventing Protocol): https://xmpp.org/extensions/xep-0163.html

Features
~~~~~~~~

Libervia Pubsub aims to be a feature-full pubsub implementation. In addition to base
PEP/Pubsub service, it implements:

`XEP-0059 (Result Set Management)`_
    Handle pagination on large result sets.

`XEP-0355 (Namespace Delegation)`_ and `XEP-0356 (Privileged Entity)`_
    Can be used to replace a server's internal PEP implementation.

`XEP-0496 (Pubsub Node Relationships)`_
    Build a node hierarchy with parent/children nodes and linked nodes. Useful to organise
    nodes, change permissions, manage or delete automatically related nodes (e.g. a blog
    post and its comments), implement forums, etc.

`XEP-0499 (Pubsub Extended Discovery)`_
    Explore hierarchies of nodes as created with XEP-0496.

History
~~~~~~~

Libervia Pubsub is a reimplementation on top of Libervia Backend of the former standalone
`pubsub component <https://repos.goffi.org/libervia-pubsub/>`__ of the same name, which
was previously called "SàT Pubsub". "SàT Pubsub" was itself a fork of `Idavoll`_, a Pubsub
component made by Ralph Meijer

Acknowledgement
~~~~~~~~~~~~~~~

This component has been possible thanks to the work of Ralph Meijer on `Wokkel`_ and
`Idavoll`_. Ralph is also one of the authors of `XEP-0060 (Pubsub)`_. Other people made
that possible: the other authors of XEP-0060 (Peter Millard and Peter Saint-Andre), all
the awesome folks working or having worked on XMPP, and Adrien Cossa who worked on
Libervia and built the foundations to implement RSM (`XEP-0059 (Result Set Management)`_)
and MAM (`XEP-0313 (Message Archiving XEP)`_ and `XEP-0442 (Predefined Namespace for PEP
Nodes)`_). A big thanks to them!

The reimplementation of Libervia Pubsub into Libervia Backend has been possible thanks to
a `NLnet`_/`NGI0 Commons Fund`_, for the "`Serverless and Metadata Reduction`_ for XMPP"
project.

.. _XEP-0060 (Pubsub): https://xmpp.org/extensions/xep-0060.html
.. _Wokkel: https://github.com/ralphm/wokkel
.. _Idavoll: https://github.com/ralphm/idavoll
.. _XEP-0059 (Result Set Management): https://xmpp.org/extensions/xep-0059.html
.. _XEP-0313 (Message Archiving XEP): https://xmpp.org/extensions/xep-0313.html
.. _XEP-0355 (Namespace Delegation): https://xmpp.org/extensions/xep-0355.html
.. _XEP-0356 (Privileged Entity): https://xmpp.org/extensions/xep-0356.html
.. _XEP-0442 (Predefined Namespace for PEP Nodes): https://xmpp.org/extensions/xep-0442.html
.. _XEP-0496 (Pubsub Node Relationships): https://xmpp.org/extensions/xep-0496.html
.. _XEP-0499 (Pubsub Extended Discovery): https://xmpp.org/extensions/xep-0499.html
.. _NGI0 Commons Fund: https://nlnet.nl/commonsfund/

.. _NLnet: https://nlnet.nl/
.. _Serverless and Metadata Reduction: https://nlnet.nl/project/ServerlessXMPP/

Prosody Configuration
---------------------

Libervia PubSub can work with any XMPP server (which supports components), but if you want
to use it as your PEP service, you need a server which supports XEP-0355 and XEP-0356.

Below you'll find the instructions to use Libervia PubSub as a PEP service with Prosody:

-  add these two lines at the end of your ``prosody.cfg.lua`` file, adapting them to your
  XMPP server domain (virtual host) and selecting a password of your choice:

.. sourcecode:: lua

    Component "pubsub.<xmpp_domain>"
            component_secret = "<password>"

-  there are extra steps to enable the micro-blogging feature with Prosody. Please follow
  the installation and configuration instructions that are given on these pages:

   - https://modules.prosody.im/mod_delegation.html
   - https://modules.prosody.im/mod_privilege.html

.. note::

   On recent prosody installation, instead of manually cloning and linking it, you can use
   the `Prosody Plugin Installer`_.

To keep your modules up to date, we recommend to clone the full modules repository and
then to symlink them like that:

.. sourcecode:: shell

    % cd /path/to/install_dir
    % hg clone https://hg.prosody.im/prosody-modules
    % cd /path/to/prosody_plugins
    % ln -sf /path/to/install_dir/prosody-modules/mod_delegation ./
    % ln -sf /path/to/install_dir/prosody-modules/mod_privilege ./

Of course, you have to adapt ``/path/to/install_dir`` to the directory where you want to
install the modules, and ``/path/to/prosody_plugins`` to the directory where prosody
modules are installed (hint: check ``prosodyctl about`` to find the latter). The ``ln``
commands may have to be run as root depending on your installation.

Once your symlinks are set, to update the modules we just need to type this:

.. sourcecode:: shell

    % cd /path/to/install_dir/prosody-modules
    % hg pull -u

Here is an example of how your ``prosody.cfg.lua`` should look like with
``mod_delegation`` and ``mod_privilege`` activated:

.. sourcecode:: lua

    [...]
    modules_enabled = {
                  [...]
                  "delegation";
                  "privilege";
    }
    [...]
    VirtualHost "<xmpp_domain>"
      privileged_entities = {
        ["pubsub.<xmpp_domain>"] = {
          roster = "get";
          message = "outgoing";
          presence = "roster";
          iq = {
            ["http://jabber.org/protocol/pubsub"] = "set";
          };
        },
      }
      delegations = {
          ["urn:xmpp:mam:2"] = {
            filtering = {"node"};
            jid = "pubsub.<xmpp_domain>";
          },
          ["http://jabber.org/protocol/pubsub"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["http://jabber.org/protocol/pubsub#owner"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["https://salut-a-toi/protocol/schema:0"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["https://salut-a-toi.org/spec/pubsub_admin:0"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["urn:xmpp:pam:0"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["urn:xmpp:delegation:2:bare:disco#info:*"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["urn:xmpp:delegation:2:bare:disco#items:*"] = {
            jid = "pubsub.<xmpp_domain>";
          },
          ["jabber:iq:private"] = {
            jid = "pubsub.<xmpp_domain>";
          },
      }

    Component "pubsub.<xmpp_domain>"
       component_secret = "<password>"
       modules_enabled = {"delegation", "privilege"}

Of course, you still have to replace and adapt to your own settings.

.. _XEP-0355: https://xmpp.org/extensions/xep-0355.html
.. _XEP-0356: https://xmpp.org/extensions/xep-0356.html


.. _Prosody Plugin Installer: https://prosody.im/doc/plugin_installer

