nostr/nips
1
0
Fork 0
mirror of https://github.com/nostr-protocol/nips.git synced 2024-12-25 09:45:51 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity

Rewriting NIP-01

Browse Source
This commit is contained in:
Vitor Pamplona 2024-01-03 11:23:53 -05:00
parent 27fef638e2
commit 0c37bd1738
1 changed files with 97 additions and 71 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

168
01.md
View File

@ -1,23 +1,23 @@
NIP-01
======
Basic protocol flow description
-------------------------------
Nostr Protocol
--------------
`draft` `mandatory`
This NIP defines the basic protocol that should be implemented by everybody. New NIPs may add new optional (or mandatory) fields and messages and features to the structures and flows described here.
This NIP defines the complete protocol that MUST be implemented by everybody. New NIPs may add new optional (or mandatory) fields, messages and features to the structures and flows described here.
## Events and signatures
## User and Events
Each user has a keypair. Signatures, public key, and encodings are done according to the [Schnorr signatures standard for the curve `secp256k1`](https://bips.xyz/340).
Each user has a keypair and is identified by their public key.
The only object type that exists is the `event`, which has the following format on the wire:
Event is only object type available. It is a hashed and signed payload with the following format:
```json
{
"id": <32-bytes lowercase hex-encoded sha256 of the serialized event data>,
"pubkey": <32-bytes lowercase hex-encoded public key of the event creator>,
"id": <32-byte lowercase hex-encoded sha256 of the serialized event data>,
"pubkey": <32-byte lowercase hex-encoded public key used to sign>,
"created_at": <unix timestamp in seconds>,
"kind": <integer between 0 and 65535>,
"tags": [
@ -25,11 +25,15 @@ The only object type that exists is the `event`, which has the following format
...
],
"content": <arbitrary string>,
"sig": <64-bytes lowercase hex of the signature of the sha256 hash of the serialized event data, which is the same as the "id" field>
"sig": <64-byte lowercase hex of the signature of the "id" field>
}
```
To obtain the `event.id`, we `sha256` the serialized event. The serialization is done over the UTF-8 JSON-serialized string (which is described below) of the following structure:
### Signature
Signatures and encodings are done according to the [Schnorr signatures standard for the curve `secp256k1`](https://bips.xyz/340).
To obtain the hash `.id`, we `sha256` the UTF-8 bytearray of a JSON-serialized string with the following structure:
```
[
@ -42,21 +46,20 @@ To obtain the `event.id`, we `sha256` the serialized event. The serialization is
]
```
To prevent implementation differences from creating a different event ID for the same event, the following rules MUST be followed while serializing:
- No whitespace, line breaks or other unnecessary formatting should be included in the output JSON.
- No characters except the following should be escaped, and instead should be included verbatim:
- A line break, `0x0A`, as `\n`
- A double quote, `0x22`, as `\"`
- A backslash, `0x5C`, as `\\`
- A carriage return, `0x0D`, as `\r`
- A tab character, `0x09`, as `\t`
- A backspace, `0x08`, as `\b`
- A form feed, `0x0C`, as `\f`
- UTF-8 should be used for encoding.
The JSON serialization MUST follow this rules:
- Minified: No whitespace, line breaks or other unnecessary formatting used
- Escape Set: ONLY the following characters MUST be escaped:
- line break, `0x0A`, as `\n`
- double quote, `0x22`, as `\"`
- backslash, `0x5C`, as `\\`
- carriage return, `0x0D`, as `\r`
- tab character, `0x09`, as `\t`
- backspace, `0x08`, as `\b`
- form feed, `0x0C`, as `\f`
### Tags
Each tag is an array of strings of arbitrary size, with some conventions around them. Take a look at the example below:
Each tag is an array of strings of arbitrary size. Their meaning is determined by the event `.kind` and defined in NIPs in this repository.
```json
{
@ -72,94 +75,119 @@ Each tag is an array of strings of arbitrary size, with some conventions around
}
```
The first element of the tag array is referred to as the tag _name_ or _key_ and the second as the tag _value_. So we can safely say that the event above has an `e` tag set to `"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"`, an `alt` tag set to `"reply"` and so on. All elements after the second do not have a conventional name.
The first element of the tag array is referred to as the tag _name_ or _key_ and the second as the tag _value_. All elements after the second do not have a conventional name.
This NIP defines 3 standard tags that can be used across all event kinds with the same meaning. They are as follows:
This NIP defines the format of 3 standard tags that can be used across all event kinds:
- The `e` tag, used to refer to an event: `["e", <32-bytes lowercase hex of the id of another event>, <recommended relay URL, optional>]`
- The `p` tag, used to refer to another user: `["p", <32-bytes lowercase hex of a pubkey>, <recommended relay URL, optional>]`
- The `a` tag, used to refer to a (maybe parameterized) replaceable event
- for a parameterized replaceable event: `["a", <kind integer>:<32-bytes lowercase hex of a pubkey>:<d tag value>, <recommended relay URL, optional>]`
- for a non-parameterized replaceable event: `["a", <kind integer>:<32-bytes lowercase hex of a pubkey>:, <recommended relay URL, optional>]`
- The `e` tag refers to an event: `["e", <32-byte lowercase hex of the id of another event>, <recommended relay URL, optional>]`
- The `p` tag refers to a pubkey: `["p", <32-byte lowercase hex of a pubkey>, <recommended relay URL, optional>]`
- The `a` tag refers to a replaceable event
- for a parameterized replaceable event: `["a", <kind integer>:<32-byte lowercase hex of a pubkey>:<d tag value>, <recommended relay URL, optional>]`
- for a non-parameterized replaceable event: `["a", <kind integer>:<32-byte lowercase hex of a pubkey>:, <recommended relay URL, optional>]`
As a convention, all single-letter (only english alphabet letters: a-z, A-Z) key tags are expected to be indexed by relays, such that it is possible, for example, to query or subscribe to events that reference the event `"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"` by using the `{"#e": "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}` filter.
All single-letter (only english alphabet letters: a-z, A-Z) key tags indexed by relays for faster queries.
### Kinds
Kinds specify how clients should interpret the meaning of each event and the other fields of each event (e.g. an `"r"` tag may have a meaning in an event of kind 1 and an entirely different meaning in an event of kind 10002). Each NIP may define the meaning of a set of kinds that weren't defined elsewhere. This NIP defines two basic kinds:
Kinds specify the meaning of an event and its tags. Tags with the same name might have entirely different meanings in diferent kinds.
- `0`: **metadata**: the `content` is set to a stringified JSON object `{name: <username>, about: <string>, picture: <url, string>}` describing the user who created the event. A relay may delete older events once it gets a new one for the same pubkey.
This NIP defines two kinds:
- `0`: **user metadata**: the `content` is set to a stringified JSON object `{name: <username>, about: <string>, picture: <url, string>}` describing the user who created the event. A relay may delete older events once it gets a new one for the same pubkey.
- `1`: **text note**: the `content` is set to the **plaintext** content of a note (anything the user wants to say). Content that must be parsed, such as Markdown and HTML, should not be used. Clients should also not parse content as those.
And also a convention for kind ranges that allow for easier experimentation and flexibility of relay implementation:
Kind ranges define storage behaviours. The current ranges are:
- for kind `n` such that `1000 <= n < 10000`, events are **regular**, which means they're all expected to be stored by relays.
- for kind `n` such that `10000 <= n < 20000 || n == 0 || n == 3`, events are **replaceable**, which means that, for each combination of `pubkey` and `kind`, only the latest event MUST be stored by relays, older versions MAY be discarded.
- for kind `n` such that `20000 <= n < 30000`, events are **ephemeral**, which means they are not expected to be stored by relays.
- for kind `n` such that `30000 <= n < 40000`, events are **parameterized replaceable**, which means that, for each combination of `pubkey`, `kind` and the `d` tag's first value, only the latest event MUST be stored by relays, older versions MAY be discarded.
| Name | Range | SHOULD retain |
| ------------- | --------------------------------- | ------------------------------------------------------ |
| Regular | ` 1000 <= kind < 10000` | All events |
| Replaceable | `10000 <= kind < 20000` OR `0, 3` | The newest event for each `pubkey` and `kind` |
| Ephemeral | `20000 <= kind < 30000` | Nothing |
| Parameterized | `30000 <= kind < 40000` | The newest event for each `pubkey`, `kind` and `d`-tag |
In case of replaceable events with the same timestamp, the event with the lowest id (first in lexical order) should be retained, and the other discarded.
In case of replaceable events with the same timestamp, the event with the lowest `.id` (first in lexical order) SHOULD be retained. Older versions MAY be kept but SHOULD not be returned on queries.
When answering to `REQ` messages for replaceable events such as `{"kinds":[0],"authors":[<hex-key>]}`, even if the relay has more than one version stored, it SHOULD return just the latest one.
## Relay Protocol
These are just conventions and relay implementations may differ.
Nostr has two main components: Clients & Relays. Users run a client to fetch/subscribe to events from one or more Relays via websockets. Relays are not expected to communicate with one another. It's the Client's responsibility to discover which relay has the event set their user wants to see. Both sides SHOULD verify the hash and the signature of each event upon receipt.
## Communication between clients and relays
Clients SHOULD open a single websocket connection to each relay and use it for all their subscriptions. Relays MAY limit number of connections from specific IP/client/etc. All messages are defined as JSON arrays.
Relays expose a websocket endpoint to which clients can connect. Clients SHOULD open a single websocket connection to each relay and use it for all their subscriptions. Relays MAY limit number of connections from specific IP/client/etc.
### Subscriptions
### From client to relay: sending events and creating subscriptions
Clients send a **subscription** message with one or more filters. The Relay MUST query its database, return all events that match the filter and keep applying the filter to all connections, returning new events as they arrive. Subscriptions stay indefinatelly open until either side closes the subscription or the connection.
Clients can send 3 types of messages, which must be JSON arrays, according to the following patterns:
#### Subscription Requests
* `["EVENT", <event JSON as defined above>]`, used to publish events.
* `["REQ", <subscription_id>, <filters1>, <filters2>, ...]`, used to request events and subscribe to new updates.
To open, update and close subscriptions, Clients MUST use the following formats:
* `["REQ", <subscription_id>, <filter1>, <filter2>, ...]`, used to request events and subscribe to new updates.
* `["CLOSE", <subscription_id>]`, used to stop previous subscriptions.
`<subscription_id>` is an arbitrary, non-empty string of max length 64 chars. It represents a subscription per connection. Relays MUST manage `<subscription_id>`s independently for each WebSocket connection. `<subscription_id>`s are not guarantueed to be globally unique.
`<subscription_id>` is an non-empty string with maximum length of 64 chars. Relays MUST manage `<subscription_id>`s independently for each WebSocket connection. `<subscription_id>`s are not globally unique. A `REQ` message on an existing subscription overwrides the previous subscription.
`<filtersX>` is a JSON object that determines what events will be sent in that subscription, it can have the following attributes:
`<filterX>` is a JSON object that determines what events will be sent in that subscription, it can have the following attributes:
```json
{
"ids": <a list of event ids>,
"authors": <a list of lowercase pubkeys, the pubkey of an event must be one of these>,
"kinds": <a list of a kind numbers>,
"#<single-letter (a-zA-Z)>": <a list of tag values, for #e — a list of event ids, for #p — a list of event pubkeys etc>,
"ids": [<id1>, <id2>, ...],
"authors": [pubkey1, pubkey2, ...],
"kinds": [kind1, kind2, ...],
"#<single-letter-tag-key (a-zA-Z)>": [tag value1, tag value2, ...],
"since": <an integer unix timestamp in seconds, events must be newer than this to pass>,
"until": <an integer unix timestamp in seconds, events must be older than this to pass>,
"limit": <maximum number of events relays SHOULD return in the initial query>
"limit": <maximum number of events to return, ordered by created_at>
}
```
Upon receiving a `REQ` message, the relay SHOULD query its internal database and return events that match the filter, then store that filter and send again all future events it receives to that same websocket until the websocket is closed. The `CLOSE` event is received with the same `<subscription_id>` or a new `REQ` is sent using the same `<subscription_id>`, in which case relay MUST overwrite the previous subscription.
A `REQ` message may contain multiple filters which are interpreted as an OR statement: events that match any of the filters are to be returned.
Filter attributes containing lists (`ids`, `authors`, `kinds` and tag filters like `#e`) are JSON arrays with one or more values. At least one of the arrays' values must match the relevant field in an event for the condition to be considered a match. For scalar event attributes such as `authors` and `kind`, the attribute from the event must be contained in the filter list. In the case of tag attributes such as `#e`, for which an event may have multiple values, the event and filter condition values must have at least one item in common.
Properties in each filter are a logical AND statement: all present properties must match for the filter to pass.
Array properties (i.e., `ids`, `authors`, `kinds` and tag filters) represent a logical OR statement. At least one of the arrays' values must match the respective field in the event to be considered a match. In the case of tag attributes such as `#e`, for which an event may have multiple values, the event and filter condition values must have at least one item in common.
The `ids`, `authors`, `#e` and `#p` filter lists MUST contain exact 64-character lowercase hex values.
The `since` and `until` properties can be used to specify the time range of events returned in the subscription. If a filter includes the `since` property, events with `created_at` greater than or equal to `since` are considered to match the filter. The `until` property is similar except that `created_at` must be less than or equal to `until`. In short, an event matches a filter if `since <= created_at <= until` holds.
The `since` and `until` properties are used to specify the time range of events returned in the subscription. An event matches the filter if `since <= created_at <= until` holds.
All conditions of a filter that are specified must match for an event for it to pass the filter, i.e., multiple conditions are interpreted as `&&` conditions.
The `limit` property operates over the stored events and is ignored afterwards.
A `REQ` message may contain multiple filters. In this case, events that match any of the filters are to be returned, i.e., multiple filters are to be interpreted as `||` conditions.
#### Receiving Events
The `limit` property of a filter is only valid for the initial query and MUST be ignored afterwards. When `limit: n` is present it is assumed that the events returned in the initial query will be the last `n` events ordered by the `created_at`. It is safe to return less events than `limit` specifies, but it is expected that relays do not return (much) more events than requested so clients don't get unnecessarily overwhelmed by data.
Relays send 3 types of messages during a subscription:
### From relay to client: sending events and notices
Relays can send 4 types of messages, which must also be JSON arrays, according to the following patterns:
* `["EVENT", <subscription_id>, <event JSON as defined above>]`, used to send events requested by clients.
* `["OK", <event_id>, <true|false>, <message>]`, used to indicate acceptance or denial of an `EVENT` message.
* `["EVENT", <subscription_id>, <JSON-serialized string of the event>]`, used to send events requested by clients.
* `["EOSE", <subscription_id>]`, used to indicate the _end of stored events_ and the beginning of events newly received in real-time.
* `["CLOSED", <subscription_id>, <message>]`, used to indicate that a subscription was ended on the server side.
All messages MUST be sent with the subscription_id initiated by the client (using the `REQ` message above).
`CLOSED` messages MUST be sent in response to a `REQ` when the relay refuses to fulfill it. It can also be sent when a relay decides to kill a subscription on its side before a client has disconnected or sent a `CLOSE`. The message MUST be a string formed by a machine-readable single-word prefix followed by a `:` and then a human-readable message.
### Broadcasting an Event
To send an event to the Relay, Clients send a broadcast message in the format:
* `["EVENT", <JSON-serialized string of the event>]`
Relays reply with an ACK message in the format of:
* `["OK", <event_id>, <accepted:true|false>, <message>]`, used to indicate acceptance (true) or denial (false) of an `EVENT` message.
`OK` messages MUST be sent in response to `EVENT` messages received from clients, they must have the 3rd parameter set to `true` when an event has been accepted by the relay, `false` otherwise. The 4th parameter MUST always be present, but MAY be an empty string when the 3rd is `true`.
If present, the message MUST be a string formed by a machine-readable single-word prefix followed by a `:` and then a human-readable message.
### Notices
Notices are warnings that might help explain or debug the behaviour of a given relay.
* `["NOTICE", <message>]`, used to send human-readable error messages or other things to clients.
This NIP defines no rules for how `NOTICE` messages should be sent or treated.
### Error Codes
- `EVENT` messages MUST be sent only with a subscription ID related to a subscription previously initiated by the client (using the `REQ` message above).
- `OK` messages MUST be sent in response to `EVENT` messages received from clients, they must have the 3rd parameter set to `true` when an event has been accepted by the relay, `false` otherwise. The 4th parameter MUST always be present, but MAY be an empty string when the 3rd is `true`, otherwise it MUST be a string formed by a machine-readable single-word prefix followed by a `:` and then a human-readable message. Some examples:
The standardized machine-readable prefixes for `OK` and `CLOSED` are: `duplicate`, `pow`, `blocked`, `rate-limited`, `invalid`, and `error` for when none of that fits.
Some examples:
* `["OK", "b1a649ebe8...", true, ""]`
* `["OK", "b1a649ebe8...", true, "pow: difficulty 25>=24"]`
* `["OK", "b1a649ebe8...", true, "duplicate: already have this event"]`
@ -169,9 +197,7 @@ This NIP defines no rules for how `NOTICE` messages should be sent or treated.
* `["OK", "b1a649ebe8...", false, "invalid: event creation date is too far off from the current time"]`
* `["OK", "b1a649ebe8...", false, "pow: difficulty 26 is less than 30"]`
* `["OK", "b1a649ebe8...", false, "error: could not connect to the database"]`
- `CLOSED` messages MUST be sent in response to a `REQ` when the relay refuses to fulfill it. It can also be sent when a relay decides to kill a subscription on its side before a client has disconnected or sent a `CLOSE`. This message uses the same pattern of `OK` messages with the machine-readable prefix and human-readable message. Some examples:
* `["CLOSED", "sub1", "duplicate: sub1 already opened"]`
* `["CLOSED", "sub1", "unsupported: filter contains unknown elements"]`
* `["CLOSED", "sub1", "error: could not connect to the database"]`
* `["CLOSED", "sub1", "error: shutting down idle subscription"]`
- The standardized machine-readable prefixes for `OK` and `CLOSED` are: `duplicate`, `pow`, `blocked`, `rate-limited`, `invalid`, and `error` for when none of that fits.
* `["CLOSED", "sub1", "error: shutting down idle subscription"]`