Chamadas

Importante

This page details the principle for Jami accounts. For SIP accounts, the SIP protocol is used.

Vamos fazer uma chamada em Jami!

Lado da demônio

When creating a call between two peers, Jami mainly uses known protocols such as ICE, SIP, or TLS. However, to make it distributed, the process of creating a call is a bit different. To summarize, when someone wants to contact one of their contacts, this is what they will do:

1. Search the contact presence on the DHT (for more details, see Gestão de contatos).

2. Once the contact is found, send a call request, announcing the known candidates (the IP address of each network interface + relay addresses (TURN) + reflexive addresses (UPnP, public ones).

3. Wait for the response of the contact (they will respond to their known addresses).

4. Negociar a tomada através do ICE. Na verdade, duas sessões do ICE são negociadas. Uma (preferida) no TCP, uma no UDP (como um fallback).

5. Em seguida, o socket é criptografado em TLS (se TCP) ou DTLS (se UDP).

6. The contact is now able to accept or decline the call. When they accept, an ICE transport (UDP only for now) is negotiated to create 4 new sockets for the media (2 for audio, 2 for video).

7. A chamada está viva!

Candidatos de intercâmbio de ICE

Everything really starts in jamiaccount.cpp (JamiAccount::startOutgoingCall). Once both ICE objects are ready and when the contact is found via the DHT, the call request for the contact is crafted. This request contains all the information necessary for the remote ICE session defined by:

dht::IceCandidates(callvid,  blob)

where:

• callvid is a random number used to identify the call, and

• blob contains two concatenated ICE messages (IceTransport::packIceMsg in ice_transport.cpp) containing the password of the session, the ufrag, and ICE candidates like:

0d04b935
7c33834e7cf944bf0e367b42
H6e6ca382 1 UDP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 14133 typ host
H42c1g477 1 UDP 2130706431 fe80::9eb6:d0ff:fee7:1412 14133 typ host
Hc0a8027e 1 UDP 2130706431 192.168.0.123 34567 typ host
Sc0a8027e 1 UDP 1694498815 X.X.X.X 32589 typ srflx
0d04b932
7c33834e7cf944bf0e367b47
H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 50693 typ host tcptype passive
H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 9 typ host tcptype active
H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 50693 typ host tcptype passive
H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 9 typ host tcptype active
Hc0a8007e 1 TCP 2130706431 192.168.0.123 42751 typ host tcptype passive
Hc0a8007e 1 TCP 2130706431 192.168.0.123 9 typ host tcptype active
Sc0a8007e 1 TCP 1694498815 X.X.X.X 42751 typ srflx tcptype passive

and is sent via the DHT in an encrypted message for the device to hash(callto:xxxxxx) where xxxxxx is the device ID. The peer will answer at the exact same place (but encrypted for the sender device) its own dht::IceCandidates. See JamiAccount::replyToIncomingIceMsg for more details.

The ICE session is created on both sides when they have all the candidates (so for the sender, when the reply from the contact is received).

Negociação do ICE

Pending calls are managed by JamiAccount::handlePendingCallList(), which first wait for the TCP negotiation to finish (and if it fails, wait for the UDP one). The code for the ICE negotiation is mainly managed by pjproject but for Jami, the interesting part is located in ice_transport.cpp. Moreover, we add some important patches/features on top of pjproject not merged upstream for now (for example, ICE over TCP). These patches are present in contrib/src/pjproject.

Encripta a tomada de controle

Once the socket is created and managed by an IceTransport instance, it is then wrapped in a SipTransport corresponding to a TlsIceTransport. The main code is located in JamiAccount::handlePendingCall() and the wrapping is done in SipTransportBroker::getTlsIceTransport. Finally, our session is managed by TlsSession in daemon/src/security/tls_session.cpp and uses the GnuTLS library.

So, the control socket will be a TLS (1.3 if you and your peer’s GnuTLS version supports it) if a TCP socket is negotiated. If a UDP socket is negotiated instead (due to firewall restrictions/problems in the negotiation/etc.), the socket will use DTLS (still managed by the same parts).

The control socket is used to transmit SIP packets, like invites, custom messages (Jami sends the vCard of your profile on this socket at the start of the call, or the rotation of the camera), and text messages.

Artigos relacionados:

• https://jami.net/improved-video-rotation-support/

• https://jami.net/peer-to-peer-file-sharing-support-in-jami/

Soquetes de mídia

Media sockets are SRTP sockets where the key is negotiated through the TLS session previously created.

Aviso

TODO: This section is incomplete.

Arquitetura

Aviso

TODO: This section is incomplete.

Múltiplos fluxos

Since daemon version 13.3.0, multi-stream is fully supported. This feature allows users to share multiple videos during a call at the same time. In the following parts, we will describe all related changes.

PJSIP

The first part is to negotiate enough media streams. In fact, every media stream uses 2 UDP sockets. We consider three scenarios:

1. If it’s the host of a conference who wants to add media, there is nothing more to negotiate, because we already mix the videos into one stream. So, we add the new media directly to the video mixer without negotiations.

2. Se estamos em 1:1, por enquanto, como não há informações de conferência, multi-stream não é suportado.

3. Caso contrário, 2 novos soquetes são negociados para novas mídias.

To make PJSIP able to generate more sockets per ICE session, PJ_ICE_COMP_BITS was modified to \(5\) (which corresponds to \(2^5\), so \(32\) streams).

Deprecar interruptorInput, solicitação de suporteMediaChange

No daemon, a antiga API switchInput é agora DEPRECATED; igual para switchSecondaryInput:

<method name="switchInput" tp:name-for-bindings="switchInput">
    <tp:docstring>
        Switch input for the specified call
    </tp:docstring>
    <arg type="s" name="accountId" direction="in"/>
    <arg type="s" name="callId" direction="in"/>
    <arg type="s" name="input" direction="in"/>
    <arg type="b" direction="out" />
</method>

<method name="switchSecondaryInput" tp:name-for-bindings="switchSecondaryInput">
    <tp:added version="11.0.0"/>
    <tp:docstring>
        Switch secondary input for the specified conference
    </tp:docstring>
    <arg type="s" name="accountId" direction="in" />
    <arg type="s" name="conferenceId" direction="in"/>
    <arg type="s" name="input" direction="in"/>
    <arg type="b" direction="out" />
</method>

requestMediaChange substitui esta, tanto para as chamadas como para as conferências:

<method name="requestMediaChange" tp:name-for-bindings="requestMediaChange">
    <tp:added version="11.0.0"/>
    <tp:docstring>
        <p>Request changes in the media of the specified call.</p>
    </tp:docstring>
    <arg type="s" name="accountId" direction="in" />
    <arg type="s" name="callId" direction="in">
        <tp:docstring>
        The ID of the call.
        </tp:docstring>
    </arg>
    <annotation name="org.qtproject.QtDBus.QtTypeName.In2" value="VectorMapStringString"/>
    <arg type="aa{ss}" name="mediaList" direction="in">
        <tp:docstring>
        A list of media attributes to apply.
        </tp:docstring>
    </arg>
    <arg type="b" name="requestMediaChangeSucceeded" direction="out"/>
</method>

Compatibility

If a call is done with a peer where the daemon’s version is < 13.3.0, multi-stream is not enabled, and the old behavior is used (1 video only).

Stream identification

Como agora podem haver vários fluxos, cada fluxo de mídia é identificado pelo seu identificador, e o formato é “_”; por exemplo: “audio_0”, “video_2”, etc.

Rotatividade

O XML foi atualizado para adicionar o fluxo desejado:

<?xml version="1.0" encoding="utf-8" ?>
<media_control>
  <vc_primitive>
    <stream_id>{}</stream_id>
    <to_encoder>
      <device_orientation>0</device_orientation>
    </to_encoder>
  </vc_primitive>
</media_control>

Quadro-chave

O XML foi atualizado para adicionar o fluxo desejado:

<?xml version="1.0" encoding="utf-8" ?>
<media_control>
  <vc_primitive>
    <stream_id>{}</stream_id>
    <to_encoder><picture_fast_update/></to_encoder>
  </vc_primitive>
</media_control>

Atividade de voz

The XML was updated to add the required stream:

<?xml version="1.0" encoding="utf-8" ?>
<media_control>
  <vc_primitive>
    <stream_id>{}</stream_id>
    <to_encoder>
      <voice_activity>true</voice_activity>
    </to_encoder>
  </vc_primitive>
</media_control>

Conferência

As alterações refletidas estão documentadas aqui.

Cliente

Even if the back-end supports up to 32 media at the same time, except for custom clients, we currently recommend only giving the ability to share one camera and one video at the same time. The camera is controlled via the camera button, and the other media via the “Share” button.

In client-qt, the interesting part is in AvAdapter (methods like isCapturing, shareAllScreens, stopSharing). In the library’s logic, addMedia and removeMedia in the callModel directly use the requestMediaChange, and can be used as a design reference.