nips/104.md

NIP-104

E2EE Messaging using the Messaging Layer Security (MLS) Protocol

draft optional

This NIP standardizes how to use the MLS Protocol with Nostr for efficient and E2EE (end-to-end encrypted) direct and group messaging.

Context

Originally, one-to-one direct messages (DMs) in Nostr happened via the scheme defined in NIP-04. This NIP is not recommended because, while it encrypts the content of the message (provides decent confidentiality), it leaks significant amounts of metadata about the parties involved in the conversation (completely lacks privacy).

With the addition of NIP-44, we have an updated encryption scheme that improves confidentiality guarantees but stops short of defining a new scheme for doing direct messages using this encryption scheme. Hence, makes little to no difference to privacy.

Most recently, NIP-17 combines NIP-44 encryption with NIP-59 gift-wrapping to hide the encrypted direct message inside another set of events to ensure that it's impossible to see who is talking to who and when messages passed between the users. This largely solves the metadata leakage problem; while it's still possible to see that a user is receiving gift-wrapped events, you can't tell from whom and what kind of events are within the gift-wrap outer event. This gives some degree of deniability/repudiation but doesn't solve forward secrecy or post compromise security. That is to say, if a user's private key (or the calculated conversation key shared between two users used to encrypt messages) is compromised, the attacker will have full access to all past and future DMs sent between those users.

In addition, neither NIP-04 or NIP-17 attempt to solve the problem of group messages.

Why is this important?

Without proper E2EE, Nostr cannot be used as the protocol for secure messaging clients. While clients like Signal do a fantastic job with E2EE, they still rely on centralized servers and as a result can be shut down by a powerful (i.e. state-level) actor. The goal of Nostr is not only to protect against centralized entities censoring you and your communications, but also protect against the ability of a state-level actor to stop these sorts of services from existing in the first place. By replacing centralized servers with decentralized relays, we make it nearly impossible for a centralized actor to completely stop communications between individual users.

Goals of this NIP

1. Private and Confidential DMs and Group messages
   1. Private means that an observer cannot tell that Alice and Bob are talking to one another, or that Alice is part of a specific group. This necessarily requires protecting metadata.
   2. Confidential means that the contents of conversations can only be viewed by the intended recipients.
2. Forward secrecy and Post-compromise security
   1. Forward secrecy means that encrypted content in the past remains encrypted even if a key material is leaked.
   2. Post compromise security means that leaking key material doesn't allow an attacker to continue to read messages indefinitely into the future.
3. Scales efficiently for large groups
4. Allows for the use of multiple device/clients in a single conversation/group.

Why MLS?

This scheme adapts the Message Layer Security (MLS) protocol for use with Nostr. You can think of MLS as an evolution of the Signal Protocol. However, it significantly improves the scalability of encryption operations for large group messaging significantly (linear -> log), is built to accommodate federated environments, and also allows for graceful updating of ciphersuites and versions over time. In addition, it's very flexible and agnostic about the message content that is sent.

It's beyond the scope of this NIP to explain the MLS protocol but you can read more about it in it's Architectural Overview or the RFC. MLS is on track to become an internet standard under the IETF so the protocol itself is extremely well vetted and researched. This also means there is the potential for cross network messaging interoperability in the future as MLS gains more adoption.

Core MLS Concepts

From the MLS Architectural Overview:

MLS provides a way for clients to form groups within which they can communicate securely. For example, a set of users might use clients on their phones or laptops to join a group and communicate with each other. A group may be as small as two clients (e.g., for simple person to person messaging) or as large as hundreds of thousands. A client that is part of a group is a member of that group. As groups change membership and group or member properties, they advance from one epoch to another and the cryptographic state of the group evolves.

The group is represented as a tree, which represents the members as the leaves of a tree. It is used to efficiently encrypt to subsets of the members. Each member has a state called a LeafNode object holding the client's identity, credentials, and capabilities.

The MLS protocol's job is to manage and evolve the cryptographic state of a group. This includes managing the membership of a group, the cryptographic state of a group (ratchet tree, keys, and encryption/decryption/authentication of messages), and managing the evolution of the group over time.

Groups

Groups are created by their first member, who then invites one or more other members. Groups evolve over time in blocks called Epochs. New epochs are proposed via one ore more Proposal messages and then committed to via a Commit message.

Clients

The device/client pair (e.g. Primal on iOS or Coracle on web) with which a user joins the group is represented as a LeafNode in the tree. The terms Client and Member are interchangeable in this regard. It is not possible to share group state across multiple Clients. If a user joins a group from 2 separate devices, their state is separate and they will be tracked as 2 separate members of the group.

Messages

There are several different types of messages sent within a group. Some of these are control messages that are used to update the group state over time. These include Welcome, Proposal, and Commit messages. Others are the actual messages that are sent between members in a group. These include Application messages.

Messages in MLS are "framed". Meaning that they are wrapped in a data structure that includes information about the sender, the epoch, the message index within the epoch and the message content. This framing makes it possible to authenticate and decrypt messages correctly, even if they arrive out of order.

MLS is agnostic to the "content" of the messages that are sent. This is a key feature of MLS that allows for the use of MLS for a wide variety of applications.

MLS is also agnostic to the transport protocol that is used to send messages. Obviously for us, we'll be using websockets, Nostr events and relays.

The focus of this NIP

This NIP focuses on how to use Nostr to perform the Authentication Service and Delivery Service functions required by the MLS protocol. Most clients will choose to use an MLS implementation to handle keys, ratcheting, group state management, and other aspects of the MLS protocol itself. OpenMLS is the most actively developed library that implements MLS.

This NIP specifies the following:

1. A standardized way that Nostr clients should create MLS groups.
2. The required format of the MLS Credential that Nostr clients should use to represent a Nostr user in a group.
3. The structure of KeyPackage Events published to relays that allow Nostr users to be added to a group asynchronously.
4. The structure of Group Events published to relays that represent the evolution of a group's state and the contents of the messages sent in the group.

Creating groups

When creating a new group clients MUST create a random 32-byte hex-encoded group ID value. This identifier can be changed over the life of the group with an MLS re-init Proposal. See Group Messages for more information.

Clients must also ensure that the ciphersuite, capabilities, and extensions they use when creating the group are compatible with those advertised by the users they'd like to invite to the group. They can check this info via the user's published KeyPackage Events.

Changes to an MLS group are affected by first creating one or more Proposal events and then committing to a set of proposals in a Commit event. These are MLS events, not Nostr events. However, for the group state to properly evolve the Commit events (which represent a specific set of proposals - like adding a new user to the group) must be published to relays for the other group members to see. See Group Messages for more information.

MLS Credentials

A Credential in MLS is an assertion of who the user is coupled with a signing key. When constructing Credentials for MLS, clients MUST use the BasicCredential type and set the identity value as the 32-byte hex-encoded public key of the user's Nostr identity key.

A Credential also has an associated signing key. The initial signing key for a user is included in the KeyPackage event. The signing key MUST be different from the user's Nostr identity key. This signing key will be rotated over time to provide improved post-compromise security.

KeyPackage Event and Signing Keys

Each user that wishes to be reachable via MLS-based messaging MUST first publish at least one KeyPackage event. The KeyPackage Event is used to authenticate users and create the necessary Credential to add members to groups in an asynchronous way. Users can publish multiple KeyPackage Events with different parameters (supporting different ciphersuites or MLS extensions, for example).

KeyPackages SHOULD be used only once. Reuse of KeyPackage Events can lead to replay attacks. In most cases, clients that implement this NIP will manage the creation and rotation of KeyPackage Events.

The signing key (the public key included in the KeyPackage Event) is used for signing within the group that adds a new user via the KeyPackage Event. Therefore, clients implementing this NIP MUST ensure that they retain access to the private key material of the signing key for each group they are a member of.

Example KeyPackage Event

  {
    "id": <id>,
    "kind": 443,
    "created_at": <unix timestamp in seconds>,
    "pubkey": <main identity pubkey>,
    "content": "",
    "tags": [
        ["mls_protocol_version", "1.0"],
        ["ciphersuite", <MLS CipherSuite ID value e.g. "0x0001">],
        ["extensions", <An array of MLS Extension ID values e.g. "0x0001, 0x0002">],
        ["signing_key", <signing key pubkey>],
        ["client", <client name>, <handler event id>, <optional relay url>],
        ["relays", <array of relay urls>],
        ["-"]
    ],
    "sig": <signed with main identity key>
}

• The content field is not required.
• The mls_protocol_version tag is required and MUST be the version number of the MLS protocol version being used. For now, this is 1.0.
• The ciphersuite tag is the value of the MLS ciphersuite that this KeyPackage Event supports. Read more about ciphersuites in MLS.
• The extensions tag is an array of MLS extension IDs that this KeyPackage Event supports. Read more about MLS extensions.
• The signing_key tag is the signing key public key.
• The client tag helps other clients manage the user experience when they receive group invites but don't have access to the signing key.
• The relays tag identifies each of the relays that the client will attempt to publish this KeyPackage event. This allows for deletion of KeyPackage Events at a later date.
• The - tag is optional, but can be used to ensure that KeyPackage Events are only published by their authenticated author. Read more in NIP-70

Deleting KeyPackage Events

Clients MUST delete the KeyPackage Event on all the listed relays any time they successfully process a group request event for a given KeyPackage Event. Clients MAY also create a new KeyPackage Event at the same time.

If clients cannot process a Welcome message (e.g. because the signing key was generated on another client), clients MUST not delete the KeyPackage Event and SHOULD show a human-understandable error to the user.

Rotating Signing Keys

Clients MUST regularly rotate the user's signing key in each group that they are a part of. The more often the signing key is rotated the stronger the post-compromise security. This rotation is done via Proposal and Commit events and broadcast to the group via a Group Event. Read more about forward secrecy and post-compromise security inherent in MLS.

KeyPackage Relays Event

A kind: 10051 event indicates the relays that a user will publish their KeyPackage Events to. The event MUST include a list of relay tags with relay URIs.

{
  "kind": 10051,
  "tags": [
    ["relay", "wss://inbox.nostr.wine"],
    ["relay", "wss://myrelay.nostr1.com"],
  ],
  "content": "",
  //...other fields
}

Welcome Event

When a new user is added to a group via an MLS Commit message. The member who sends the Commit message to the group is responsible for sending the user being added to the group a Welcome Event. This Welcome Event is sent to the user as a NIP-59 gift-wrapped event. The Welcome Event gives the new member the context they need to join the group and start sending messages.

Clients creating the Welcome Event SHOULD wait until they have received acknowledgement from relays that their Group Event with the Commit has been received before publishing the Welcome Event.

{
   "id": <id>,
   "kind": 444,
   "created_at": <unix timestamp in seconds>,
   "pubkey": <nostr identity pubkey of sender>,
   "content": <serialized MLSMessage object>,
   "tags": [
      ["relays", <array of relay urls>],
   ],
   "sig": <NOT SIGNED>
}

• The content field is required and is a serialized MLSMessage object containing the MLS Welcome object.
• The relays tag is required and is a list of relays clients should query for Group Events.

Welcome Events are then sealed and gift-wrapped as detailed in NIP-59 before being published. Like all events that are sealed and gift-wrapped, kind: 444 events MUST never be signed. This ensures that if they were ever leaked they would not be publishable to relays.

Group Events

Group Events are all the messages that are sent within a group. This includes all "control" events that update the shared group state over time (Proposal, Commit) and messages sent between members of the group (Application messages).

Group Events are published using an ephemeral Nostr keypair to obfuscate the number and identity of group participants. Clients MUST use a new Nostr keypair for each Group Event they publish.

{
   "id": <id>,
   "kind": 445,
   "created_at": <unix timestamp in seconds>,
   "pubkey": <ephemeral sender pubkey>,
   "content": <serialized MLSMessage object>,
   "tags": [
      ["h", <group id>]
   ],
   "sig": <signed with ephemeral sender key>
}

• The content field is a tls-style serialized MLSMessage object.
• The pubkey is the hex-encoded public key of the ephemeral sender.
• The h tag is the group ID value

Application Messages

Application messages are the messages that are sent within the group by members. These are contained within the MLSMessage object. The format of these messages should be unsigned Nostr events of the appropriate kind. For example, if a user sends a text note to the group, it would be a kind: 1 event. If the user reacts to a message, it would be a kind: 7 event.

This means that once the application message has been deserialized, clients can store those events and treat them as any other Nostr event, effectively creating a private Nostr feed of the group's activity and taking advantage of all the features of Nostr.

The Nostr event MUST use the member's Nostr identity key for the pubkey field.

These Nostr events MUST remain unsigned to ensure that if they were to leak to relays they would not be published publicly.

Commit Message race conditions

The MLS protocol is resilient to almost all messages arriving out of order. However, the order of Commit messages is important for the group state to move forward from one epoch to the next correctly. Given Nostr's nature as a decentralized network, it is possible for a client to receive 2 or more Commit messages all attempting to update to a new epoch at the same time.

Clients sending commit messages MUST wait until they receive acknowledgement from at least one relay that their Group Message Event with the Commit has been received before applying the commit to their own group state.

If a client receives 2 or more Commit messages attempting to change same epoch, they MUST apply only one of the Commit messages they receive, determined by the following:

1. Using the created_at timestamp on the kind 445 event. The Commit with the lowest value for created_at is the message to be applied. The other Commit message is discarded.
2. If the created_at timestamp is the same for two or more Commit messages, the Commit message with the lowest value for id field is the message to be applied.

Clients SHOULD retain previous group state for a short period of time in order to recover from forked group state.