群群

A swarm (group chat) is a set of participants capable of resilient, decentralized communication. For example, if two participants lose connectivity with the rest of the group (e.g., during an Internet outage) but can still reach each other over a LAN or subnetwork, they can exchange messages locally and then synchronize with the rest of the group once connectivity is restored.

A swarm is defined by the following properties:

Ability to split and merge based on network connectivity.
History synchronization. Every participant must be able to send a message to the entire group.
没有中央权威,不能依赖任何服务器.
Non-repudiation. Devices must be able to verify past messages’ validity and to replay the entire history.
Perfect Forward Secrecy (PFS) is provided on the transport channels. Storage is handled by each device.

首先要与参与者共享一个同步的梅克尔树.

We identified four modes for swarms that we want to implement:

ONE_TO_ONE: A private conversation between two endpoints—either between two users or with yourself.
ADMIN_INVITES_ONLY: A swarm in which only the administrator can invite members (for example, a teacher-managed classroom).
INVITES_ONLY: A closed swarm that admits members strictly by invitation; no one may join without explicit approval.
PUBLIC: A public swarm that anyone can join without prior invitation (For example a forum).

场景

创造一个群众

勃想创造一个新的群体

Bob creates a local Git repository.
之后,他创建了一个最初签署的承诺,
- /admins 的公钥
- 他的设备证书在 ̀ /设备中
- 他的CRL在 ̀ /crls`
首先提交的哈希成为对话的ID
Bob announces to his other devices that he created a new conversation. This is done via an invite to join the group sent through the DHT to other devices linked to that account.

增加一个人

Bob adds Alice

Bob adds Alice to the repo:
- 加入邀请URI以 /invited
- 增加CRL到 /crls
Bob sends a request on the DHT.

接收邀请

Alice gets the invite to join the previously created swarm

Alice accepts the invite (if she declines, nothing happens; she will remain in the “invited” list, and will never receive any messages)
A peer-to-peer connection is established between Alice and Bob.
Alice pulls the Git repository from Bob. WARNING this means that messages require a connection, not from the DHT as it is today.
Alice validates the commits from Bob.
To validate that Alice is a member, she removes the invite from /invited directory, then adds her certificate to the /members directory
Once all commits are validated and syncronized to her device, Alice discovers other members of the group. with these peers, she will then construct the DRT with Bob as a bootstrap.

发送一个信息

Alice sends a message to Bob

Alice creates a commit message. She constructs a JSON payload containing the MIME type and message body. For example:

{
    "type": "text/plain",
    "body": "hello"
}

Alice ensure her device credentials are present. If Alice’s device certificate or its associated CRL isn’t already stored in the repository, she adds them so that other participants can verify the commit.
Alice commits to the repository (Because Jami relies primarily on commit-message metadata rather than file contents, merge conflicts are rare; the only potential conflicts would involve CRLs or certificates, which are versioned in a dedicated location).
Alice announces the commit via the DRT with a service message and pings the DHT for mobile devices (they must receive a push notification).

备注

To notify other devices, the sender transmits a SIP message with type: application/im-gitmessage-id. The JSON payload includes the deviceId (the sender’s), the conversationId and the reference (hash) of the new commit.

接收消息

Bob receives a message from Alice

Bob performs a Git pull on Alice’s repository.
All incoming commits MUST be verified by a hook.
If all commits are valid, commits are stored and displayed.Bob then announces the message via the DRT for other devices.
If any commit is invalid, pull is aborted. Alice must restore her repository to a correct state before retrying.

确认承诺

为了避免用户推出一些不必要的提交 (冲突,虚假消息等), 每个提交 (从最古老到最新的) 必须在将远程分支合并之前验证:

备注

If the validation fails, the fetch is ignored and we do not merge the branch (and remove the data), and the user should be notified.
If a fetch is too big, it’s not merged.

For each incoming commit, ensure that the sending device is currently authorized and that the issuer’s certificate exists under /members or /admins, and the device’s certificate under /devices.
Then handle one of three cases, based on the commit’s parent count:
- Merge Commit (2 parents). No further validation is required, merges are always accepted.
- Initial Commit (0 parents). Validate that this is the very first repository snapshot:
  - Admin certificate is added.
  - Device certificate is added.
  - CRLs (Certificate Revocation Lists) are added.
  - No other files are present.
- Ordinary Commit (1 parent). The commit message must be JSON with a top‑level type field. Handle each type as follows:
  - If text (or any non–file‑modifying MIME type)
    - Signature is valid against the author’s certificate in the repo.
    - No unexpected files are added or removed.
  - If vote
    - voteType is one of the supported values (e.g. “ban”, “unban”).
    - The vote matches the signing user.
    - The signer is an admin, their device is present, and not themselves banned.
    - No unexpected files are added or removed.
  - If member
    - If adds
      - Properly signed by the inviter.
      - New member’s URI appears under /invited.
      - No unexpected files are added or removed.
      - If ONE_TO_ONE, ensure exactly one admin and one member.
      - If ADMIN_INVITES_ONLY, the inviter must be an admin.
    - If joins
      - Properly signed by the joining device.
      - Device certificate added under /devices.
      - Corresponding invite removed from /invited and certificate added to /members.
      - No unexpected files are added or removed.
    - If banned
      - Vote is valid per the vote rules above.
      - Ban is issued by an admin.
      - Target’s certificate moved to /banned.
      - Only files related to the ban vote are removed.
      - No unexpected files are added or removed.
  - Fallback. If the commit’s type or structure is unrecognized, reject it and notify the peer (or user) that they may be running an outdated version or attempting unauthorized changes.

禁止设备

重要

Jami source code tends to use the terms (un)ban, while the user interface uses the terms (un)block.

Alice, Bob, Carla, Denys are in a swarm. Alice issues a ban against Denys.

In a fully peer‑to‑peer system with no central authority, this simple action exposes three core challenges:

Untrusted Timestamps: Commit timestamps cannot be relied upon for ordering ban events, as any device can forge or replay commits with arbitrary dates.
Conflicting bans: In cases where multiple admin devices exist, network partitions can result in conflicting ban decisions. For instance, if Alice can communicate with Bob but not with Denys and Carla, while Carla can communicate with Denys, conflicting bans may occur. If Denys bans Alice while Alice bans Denys, the group’s state becomes unclear when all members eventually reconnect and merge their conversation histories.
Compromised or expired devices: Devices can be compromised, stolen, or have their certificates expire. The system must allow banning such devices and ensure they cannot manipulate their certificate or commit timestamps to send unauthorized messages or falsify their expiration status.

类似的系统 (分布式组系统) 并不多,但以下是几个例子:

没有任何禁令.
信号,没有任何集团聊天的中央服务器 (EDIT:他们最近改变了这一点),

This voting system needs a human action to ban someone or must be based on the CRLs info from the repository (because we can not trust external CRLs).

删除一个设备

这就是唯一一个必须达成共识的部分,以避免对话分裂,比如如果两个成员从对话中开对方,

通过使用该设备,您可以使用该设备,以便在公共场所发现被撤销的设备,或者避免让不需要的人出现.

爱丽丝把勃带走了

重要

Alice MUST be an admin to vote.

为了做到这一点,她创建文件在 /votes/ban/members/uri_bob/uri_alice (成员可以被设备替换,或邀请邀请或管理员的管理员) 和承诺
之后她检查投票是否已解决. 这意味着50%以上的管理员同意禁止Bob (如果她独自一人,那肯定是超过50%).
如果投票得到解决,可以删除文件中的 /votes/ban,所有 Bob 的文件在 /members, /admins, /invited, /CRL, /device 中可以删除 (或只在 /device中如果它是被禁止的设备) 和Bob 的证书可以放在 /banned/members/bob_uri.crt (或 /banned/devices/uri.crt如果设备被禁止) 和提交到 repo
然后,爱丽丝通知其他用户 (除了勃)

Alice (admin) re-adds Bob (banned member)

If she votes for unbanning Bob. To do that, she creates the file in /votes/unban/members/uri_bob/uri_alice (members can be replaced by devices for a device, or invited for invites or admins for admins) and commits
之后她检查投票是否已解决. 这意味着50%以上的管理员同意禁止Bob (如果她独自一人,那肯定是超过50%).
如果投票得到解决,可以删除文件,所有 Bob 的文件在 /成员, /管理员, /邀请, /CRL 中,可以重新添加 (或只有在 /设备中如果它是未被禁止的设备)

删除一个对话

保存在 convInfos removed=time::now() (如删除Contact 保存在联系人中) 中,即对话被删除并与其他用户的设备同步
如果收到新的承诺,就会被忽视.
如果Jami开始和回复仍然存在,
Two cases: a. If no other member in the conversation we can immediately remove the repository b. If still other members, commit that we leave the conversation, and now wait that at least another device sync this message. This avoids the fact that other members will still detect the user as a valid member and still sends new message notifications.
当我们确定某人同步时,删除已删除=time::now() 并同步其他用户的设备
现在所有用户所有设备都可以删除存储库和相关文件

如何指定模式

时间不变,或者是另一场对话.所以,这些数据存储在最初的提交消息中.提交消息将是这样的:

{
    "type": "initial",
    "mode": 0,
}

目前,”模式”接受值0 (ONE_TO_ONE),1 (ADMIN_INVITES_ONLY),2 (INVITES_ONLY),3 (公众)

Processes for 1:1 chats

The goal here is to keep the old API (addContact/removeContact, sendTrustRequest/acceptTrustRequest/discardTrustRequest) to create a chat with a peer and its contact. This still implies some changes that we cannot ignore:

通过 addContact 添加联系人,然后通过 DHT 发送信托请求.

托托请求嵌入一个”对话Id”以告知同行接受请求时要克隆哪种对话
当联系人回来时, TrustRequest 再次尝试.今天情况不一样 (如果同行丢弃第一个,我们不想生成新的 TrustRequest).因此,如果一个帐户收到一个信任请求,如果与相关对话的请求被拒绝 (因为 convRequests 是同步的)

然后,当联系人接受请求时,需要一个同步时间,

删除Contact() 将删除联系人和相关的 1:1 对话 (与”删除对话”相同的过程).

复杂的场景

某些情况下,可以建立两个对话.

Alice adds Bob.
Bob accepts.
Alice removes Bob.
Alice adds Bob.

或

Alice adds Bob and Bob adds Alice at the same time, but both are not connected together.

In this case, two conversations are generated. We don’t want to remove messages from users or choose one conversation here. So, sometimes two conversations between the same members will be shown. It will generate some bugs during the transition time (as we don’t want to break API, the inferred conversation will be one of the two shown conversations, but for now it’s “ok-ish”, will be fixed when clients will fully handle conversationId for all APIs (calls, file transfer, etc)).

重要

After accepting a conversation’s request, there is a time the daemon needs to retrieve the distant repository. During this time, clients MUST show a syncing view to give informations to the user. While syncing:

ConfigurationManager::getConversations() will return the conversation’s id even while syncing.
配置管理器::对话Infos() 将返回{“同步”:”真”}}如果同步.
ConfigurationManager::getConversationMembers() will return a map of two URIs (the current account and the peer who sent the request).

谈判要求规范

交谈请求由**图<字符串,字符串>****表示,按以下键:

id: the conversation ID
from: URI of the sender
收到:时间标签
标题: (可选) 对话名称
描述: (可选)
avatar: (optional) the profile picture

对话的个人资料同步

为了能够识别,对话通常需要一些元数据,例如标题 (例如: Jami),描述 (例如:一些链接,项目是什么,等等),以及图像 (项目标志).这些元数据是可选的,但所有成员都会分享,因此需要同步并纳入请求.

在存储库中存储

对话的个人资料存储在一个经典的 vCard文件中 (/profile.vcf)

BEGIN:VCARD
VERSION:2.1
FN:TITLE
DESCRIPTION:DESC
END:VCARD

协同化

To update the vCard, a user with enough permissions (by default: =ADMIN) needs to edit /profile.vcf and will commit the file with the mimetype application/update-profile. The new message is sent via the same mechanism and all peers will receive the MessageReceived signal from the daemon. The branch is dropped if the commit contains other files or too big or if done by a non-authorized member (by default: <ADMIN).

最后显示

在同步数据中,每个设备向其他设备发送对话状态.在这种状态下,最后显示的状态被发送.然而,由于每个设备可以为每个对话拥有自己的状态,并且可能在某个时候不会有相同的最后承诺,因此需要考虑几个情况:

支持5种情况:

如果其他设备发送的最后显示是与当前显示的相同,就没有办法.
如果目前设备没有最后显示,则使用远程显示的消息.
如果最后显示的远程不在 repo 中,这意味着提交将会稍后被取回,所以缓存结果
如果遥控器已经被取回,我们检查显示的本地是之前的历史记录,以取代它
最后,如果来自同一个作者的消息被公布,

偏好

每次对话都附加了用户设置的偏好.这些偏好在用户设备上都会同步.如果用户想忽略通知,文件传输尺寸限制等,这可能是对话的颜色.目前,已识别的键是:

“color” - the color of the conversation (#RRGGBB format)
“忽略通知” - - 在此对话中忽略新的消息通知
“符号” - 定义默认的情感符号.

这些偏好存储在MapStringString包中,存储在 accountDir/conversation_data/conversationId/preferences,并且只通过SyncMsg传输到同一用户的设备中.

与偏好交互的API是:

// Update preferences
void setConversationPreferences(const std::string& accountId,
                                const std::string& conversationId,
                                const std::map<std::string, std::string>& prefs);
// Retrieve preferences
std::map<std::string, std::string> getConversationPreferences(const std::string& accountId,
                                                              const std::string& conversationId);
// Emitted when preferences are updated (via setConversationPreferences or by syncing with other devices)
struct ConversationPreferencesUpdated
{
    constexpr static const char* name = "ConversationPreferencesUpdated";
    using cb_type = void(const std::string& /*accountId*/,
                            const std::string& /*conversationId*/,
                            std::map<std::string, std::string> /*preferences*/);
};

合并冲突管理

由于两个管理员可以同时更改描述,所以在 profile.vcf上可能发生合并冲突.

应用程序

用户可以使用两个方法来获取和设置对话的元数据:

       <method name="updateConversationInfos" tp:name-for-bindings="updateConversationInfos">
           <tp:added version="10.0.0"/>
           <tp:docstring>
               Update conversation's infos (supported keys: title, description, avatar)
           </tp:docstring>
           <arg type="s" name="accountId" direction="in"/>
           <arg type="s" name="conversationId" direction="in"/>
           <annotation name="org.qtproject.QtDBus.QtTypeName.In2" value="VectorMapStringString"/>
           <arg type="a{ss}" name="infos" direction="in"/>
       </method>

       <method name="conversationInfos" tp:name-for-bindings="conversationInfos">
           <tp:added version="10.0.0"/>
           <tp:docstring>
               Get conversation's infos (mode, title, description, avatar)
           </tp:docstring>
           <annotation name="org.qtproject.QtDBus.QtTypeName.Out0" value="VectorMapStringString"/>
           <arg type="a{ss}" name="infos" direction="out"/>
           <arg type="s" name="accountId" direction="in"/>
           <arg type="s" name="conversationId" direction="in"/>
       </method>

在此, info 是一个以以下键为单位的 map<str,str>:

模式:仅阅读
标题
描述
avatar: the profile picture

重新进口账户 (链接/出口)

文件档案必须包含对话Id,以便在重新进口后能够恢复新提交的对话 (因为此时没有邀请).如果提交来参加不存在的对话,有两个可能性:

谈话的存在,在这种情况下,恶魔能够重新克隆这场谈话
对话Id是缺失的,所以恶魔要求 (通过一个信息 {{"应用程序/邀请",对话Id}}) 一个新的邀请,用户需要 (重新) 接受

重要

A conversation can only be retrieved if a contact or another device is there, else it will be lost. There is no magic.

使用的协议

关键字

为什么选择

Each conversation will be a Git repository. This choice is motivated by:

我们需要同步和订单消息. Merkle Tree是完美的结构来做到这一点,可以通过将分支合并来线性化.
它们是自然的,大量使用,很多后端,可以插入.
通过子和大量使用的加密货币可以验证承诺
如果需要,可以存储在数据库中
通过使用提交消息而不是文件来避免冲突.

我们必须验证的

git.lock 可能低
子在 libgit2
两次同时拉动?

限制

历史不能删除. 为了删除对话,设备必须离开对话并创建另一个.

然而,非永久性消息 (例如只能读取几分钟的消息) 可以通过通过DRT发送特殊消息 (如键入或阅读通知).

结构

/
 - invited
 - admins (public keys)
 - members (public keys)
 - devices (certificates of authors to verify commits)
 - banned
   - devices
   - invited
   - admins
   - members
 - votes
    - ban
        - members
            - uri
                - uriAdmin
        - devices
            - uri
                - uriAdmin
    - unban
        - members
            - uri
                - uriAdmin
 - CRLs

文件转移

This new system overhauls file sharing: the entire history is now kept in sync, so any device in the conversation can instantly access past files. Rather than forcing the sender to push files directly—an approach that was fragile in the face of connection drops and often required manual retries—devices simply download files when they need them. Moreover, once one device has downloaded a file, it can act as a host for others, ensuring files remain available even if the original sender goes offline.

议定书

发送者在对话中添加一个新的承诺,以以下格式:

value["tid"] = "RANDOMID";
value["displayName"] = "DISPLAYNAME";
value["totalSize"] = "SIZE OF THE FILE";
value["sha3sum"] = "SHA3SUM OF THE FILE";
value["type"] = "application/data-transfer+json";

在 ${data_path}/conversation_data/${conversation_id}/${file_id}中创建一个链接,其中 file_id=${commitid}_${value["时间"]}.${extension}

然后,接收器现在可以通过打开一个频道来联系托管文件的设备下载文件,以: name="data-transfer://" + conversationId + "/" + currentDeviceId() + "/" + fileId,并存储文件正在等待的信息在 ${data_path}/conversation_data/${conversation_id}/waiting

接收器将通过验证文件是否可以发送 (如果 sha3sum是正确的,如果文件存在) 接受频道.接收器将保留第一个开放的频道,关闭其他频道,并将所有输入数据写入一个文件 (与发送者相同的路径: ${data_path}/conversation_data/${conversation_id}/${file_id}).

当传输完成或道关闭时,sha3sum被验证以验证文件是否正确 (否则它被删除).如果有效,文件将从等待中删除.

如果失败,当对话的设备重新上线时,我们将以同样的方式要求所有等待文件.

Call in Swarm

想法

A swarm conversation can have multiple rendez-vous. A rendez-vous is defined by the following URI:

“accountUri/deviceId/conversationId/confId”在此描述主机的accountUri/deviceId.

宿主可以通过两种方式确定:

In the swarm metadatas. Where it’s stored like the title/desc/avatar (profile picture) of the room
或是最初的通话者.

When starting a call, the host will add a new commit to the repository, with the URI to join (accountUri/deviceId/conversationId/confId). This will be valid till the end of the call (announced by a commit with the duration to show)

接下来,每个成员都会收到电话开始的信息,

攻击?

Avoid Git bombs

备忘录

提交的时间标签可靠,因为它可以编辑. 只有用户的时间标签才能信任.

其他类型

基特操作,控制消息,文件等将使用p2p TLS v1.3链接,只有加密符号才能保证PFS.因此,每个关键都会重新进行每个新连接的谈判.

DHT (UDP)

用于发送手机消息 (触发推通知) 和启动TCP连接.

网络活动

邀请一个人

爱丽丝想邀请勃:

爱丽丝把子加入了谈话中
Alice generates an invite: { “application/invite+json” : { “conversationId”: “$id”, “members”: [{…}] }}
如果没有连接,通过DHTb.否则,Alice通过SIP频道发送
对于Bob a来说,有两个可能性.接收邀请,给客户端发出信号.b没有连接,所以永远不会接收请求,因为Alice不能知道Bob是否忽略或阻止了Alice.唯一的方法是通过新的消息再生新的邀请 (见下一场景).

发送信息的过程

爱丽丝想给勃发一个信息:

艾丽丝在备忘录中添加了一个信息,给出了身份证
如果成功,爱丽丝会收到一个消息.
两个可能性, alice 和 bob 连接,或者没有.在两种情况下都会生成一个消息: {“应用程序/im-gitmessage-id” : “{“id”:”$convId","承诺":"$commitId”,”设备Id”:”$alice_device_hash”}”}.
勃的四个可能性:a.勃没有与爱丽丝连接,所以如果他信任爱丽丝,请一个新的连接,然后去b.如果连接,请从爱丽丝那里来宣布新的消息.勃不知道这次对话.请通过DHT先获得邀请才能接受那次对话 ({“应用程序/邀请”,对话Id}) d.勃被断开 (没有网络,或者只是关闭).他不会接收新消息,但会试图同步下一次连接发生时

实施

图片/图片/图片/图片

Supported messages

Initial message

{
    "type": "initial",
    "mode": 0,
    "invited": "URI"
}

Represents the first commit of a repository and contains the mode:

enum class ConversationMode : int { ONE_TO_ONE = 0, ADMIN_INVITES_ONLY, INVITES_ONLY, PUBLIC }

and invited if mode = 0.

Text message

{
    "type": "text/plain",
    "body": "content",
    "react-to": "id (optional)"
}

Or for an edition:

{
    "type": "application/edited-message",
    "body": "content",
    "edit": "id of the edited commit"
}

呼叫

Show the end of a call (duration in milliseconds):

{
    "type": "application/call-history+json",
    "to": "URI",
    "duration": "3000"
}

Or for hosting a call in a group (when it starts)

{
    "type": "application/call-history+json",
    "uri": "host URI",
    "device": "device of the host",
    "confId": "hosted confId"
}

A second commit with the same JSON + duration is added at the end of the call when hosted.

Add a file

{
    "type": "application/data-transfer+json",
    "tid": "unique identifier of the file",
    "displayName": "File name",
    "totalSize": "3000",
    "sha3sum": "a sha3 sum"
}

totalSize is in bits,

Updating profile

{
    "type": "application/update-profile",
}

Member event

{
    "type": "member",
    "uri": "member URI",
    "action": "add/join/remove/ban"
}

When a member is invited, join or leave or is kicked from a conversation

Vote event

Generated by administrators to add a vote for kicking or un-kicking someone.

{
    "type": "vote",
    "uri": "member URI",
    "action": "ban/unban"
}

!老草稿!

备注

Following notes are not organized yet. Just some line of thoughts.

加密货币的改进.

对于一个认真的团体聊天功能,我们还需要认真的加密. 现在的设计,如果作为对话的前DHT值被盗,对话可以被解密. 也许我们需要去像双重这样的东西.

备注

A lib might exist to implement group conversations.

需要在OpenDHT中获得ECC支持

使用

增加角色?

群体聊天有两个主要的使用情况:

某种程度上,它是一个公司中的Mattermost,有私人道,有某些角色 (管理员/观众/机器人/等) 或是教育 (只有少数人活跃).
水平对话就像朋友之间的对话.

Jami will be for which one?

实施想法

对于一个用户签署一个角色的证书.

加入一个对话

只有通过直接邀请
通过链接/QR码/任何
通过一个房间名称?

我们需要什么

隐私:在群体聊天之外的成员不应该能够阅读群体中消息
传输机密:如果对集团的任何密钥受到损害,之前的消息应保持机密性 (尽可能多).
信息排序:需要按正确的顺序进行信息排序
同步:还需要尽快确保所有消息都能得到.
持久性:实际上,DHT上的消息只能持续10分钟.因为这是最好的时间计算的这种DHT.为了保持数据,节点必须每10分钟重新设置DHT的值.在节点离线时,另一个方法是让节点重新设置数据.但如果10分钟后,8个节点仍然存在,它们将执行64个请求 (而且是指数量).目前避免垃圾邮件的方法是查询.这仍然会执行64个请求,但限制最大冗余到8个节点.

其他分布式方式

需要一些调查
需要一些调查
需要一些调查

根据我们目前的工作

群体聊天可以基于我们已经在多设备上做过的同样的工作 (但这里,有群体证书).

这需要将数据库从客户端移动到恶魔中.
如果没有人连接,同步不能完成,

另一个专用 DHT

像一个DHT和一个超级用户.

What’s next for file transfers

Currently, the file transfer algorithm is based on a TURN connection (See 文件转移). In the case of a big group, this will be bad. We first need a P2P connection for the file transfer. Implement the RFC for P2P transfer.

Other problem: currently there is no implementation for TCP support for ICE in PJSIP. This is mandatory for this point (in PJSIP or homemade)

群群

场景