4.5 KiB
NIP-101
Enhancing Event Compression and Encoding Support
draft optional author:b35363
Abstract
This NIP proposes a modification to the Nostr protocol to introduce a more flexible and modular approach for message compression and encoding. It suggests extending the current WebSocket message structure to include optional encoding information, such as base64, base58, gzip64, gzip58, and plaintext. This enhancement aims to improve modularity, retro-compatibility, and the potential for future encoding schemes.
Motivation
WebSocket is a widely used protocol for real-time communication between clients and servers. While the WebSocket protocol itself (ref. LZ77) supports message compression, there are scenarios where direct WebSocket compression might not be sufficient or optimal. This NIP suggests leveraging encoding and compression techniques within the message payload to provide enhanced flexibility and compatibility.
Nostr Implementation Possibility
Extended Message Structure
The current WebSocket message structure is defined like this : [MessageType, Payload]
example:
["EVENT", <json_event_structure> ]
This NIP suggests extending the message structure to: [MessageType, Payload, EncodingInfo]
example:
["EVENT", <encoded_event_structure>, <encoding_info> ]
- MessageType: Represents the type of message, e.g., "EVENT".
- EncodingInfo: An optional field indicating the encoding used for the payload.
- Payload: The actual payload data of the message.
Note that when not provided, the default encoding is set to default, for backward compatibility.
Supported Encodings
This NIP proposes the inclusion of the following encoding identifiers:
base64: Base64 encoding.base58: Base58 encoding.gzip64: Gzip compression followed by Base64 encoding.gzip58: Gzip compression followed by Base58 encoding.plaintext: Default, uncompressed JSON payload.
Capability Exchange Message
In order to know what encoding is both supported by the client and server, we need capability informations to be synced between both actors.
Here, a new message typed CAP is introduced to facilitate the exchange of encoding capabilities between the server and client. The structure of the "CAP" message is as follows:
[ "CAP", <json_structure_encoding_caps> ]
Where Capabilities is a JSON structure containing information about supported encodings and any related configuration options.
Benefits
Modularity and Retro-Compatibility
By encoding the message payload within the message itself rather than relying solely on WebSocket-level compression, the system gains modularity. This approach also maintains retro-compatibility with existing WebSocket implementations that might not support the latest compression extensions.
Future Encoding Support
The proposed message structure opens the door for supporting additional encoding schemes beyond the initial set. Encodings like protocol buffers (protobuff) and Cap'n Proto can be added in the future without altering the core WebSocket protocol.
Flexibility and Selective Encoding
Clients and servers can now choose the most appropriate encoding for their use case. For instance,
gzip64 might be chosen when bandwidth optimization is crucial, while plaintext could be selected for minimal
processing overhead.
Example Usage
sequenceDiagram
participant client
participant relay
relay-->>client: [ "CAP", { "supportedEncodings": ["base64", "gzip64", "plaintext"] } ]
client->>relay: [ "CAP", { "selectedEncoding": "gzip64" } ]
relay-->>client: [ "EVENT", {}, "gzip64" ]
Conclusion
This NIP suggests a valuable enhancement to the WebSocket protocol by introducing an extended message structure that supports various encoding options directly within the message payload. This approach offers improved modularity, retro-compatibility, and the potential for accommodating future encoding schemes. By allowing clients and servers to selectively choose their preferred encoding, the proposed changes empower developers to optimize communication based on their specific requirements.
Implementations
...
References
LZ77
https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/ https://datatracker.ietf.org/doc/html/rfc7692 https://en.wikipedia.org/wiki/LZ77_and_LZ78