NIP-45

Event Counts

draft optional

Relays may support the verb COUNT, which provides a mechanism for obtaining event counts.

Motivation

Some queries a client may want to execute against connected relays are prohibitively expensive, for example, in order to retrieve follower counts for a given pubkey, a client must query all kind-3 events referring to a given pubkey only to count them. The result may be cached, either by a client or by a separate indexing server as an alternative, but both options erode the decentralization of the network by creating a second-layer protocol on top of Nostr.

Filters and return values

This NIP defines the verb COUNT, which accepts a subscription id and filters as specified in NIP 01 for the verb REQ. Multiple filters are OR'd together and aggregated into a single count result.

["COUNT", <subscription_id>, <filters JSON>...]

Counts are returned using a COUNT response in the form {"count": <integer>}. Relays may use probabilistic counts to reduce compute requirements. In case a relay uses probabilistic counts, it MAY indicate it in the response with approximate key i.e. {"count": <integer>, "approximate": <true|false>}.

["COUNT", <subscription_id>, {"count": <integer>}]

Whenever the relay decides to refuse to fulfill the COUNT request, it MUST return a CLOSED message.

HyperLogLog

Relays may return an HyperLogLog value together with the count, hex-encoded.

["COUNT", <subscription_id>, {"count": <integer>, "hll": "<hex>"}]

This is so it enables merging results from multiple relays and yielding a reasonable estimate of reaction counts, comment counts and follower counts, while saving many millions of bytes of bandwidth for everybody.

Algorithm

The HLL value must be calculated with a precision of 8, i.e. with 256 registers.

To compute HLL values, first initi the 256 registers to 0 each; then, for on every event to be counted,

take byte 16 of the id and use it to determine the register index;
count the number of leading zero bits in the following bytes 17..24 of the id;
if the number of leading zeros is bigger than what was previously stored in that register, overwrite it.

That is all that has to be done on the relay side, and therefore the only part needed for interoperability.

On the client side, these HLL values received from different relays can be merged (by simply going through all the registers in HLL values from each relay and picking the highest value for each register, regardless of the relay).

And finally the absolute count can be estimated by running some methods I don't dare to describe here in English, it's better to check some implementation source code (also, there can be different ways of performing the estimation, with different quirks applied on top of the raw registers).

`hll` encoding

The value hll value must be the concatenation of the 256 registers, each being a uint8 value (i.e. a byte). Therefore hll will be a 512-character hex string.

Examples

Count posts and reactions

["COUNT", <subscription_id>, {"kinds": [1, 7], "authors": [<pubkey>]}]
["COUNT", <subscription_id>, {"count": 5}]

Count posts approximately

["COUNT", <subscription_id>, {"kinds": [1]}]
["COUNT", <subscription_id>, {"count": 93412452, "approximate": true}]

Followers count with HyperLogLog

["COUNT", <subscription_id>, {"kinds": [3], "#p": [<pubkey>]}]
["COUNT", <subscription_id>, {"count": 16578, "hll": "0607070505060806050508060707070706090d080b0605090607070b07090606060b0705070709050807080805080407060906080707080507070805060509040a0b06060704060405070706080607050907070b08060808080b080607090a06060805060604070908050607060805050d05060906090809080807050e0705070507060907060606070708080b0807070708080706060609080705060604060409070a0808050a0506050b0810060a0908070709080b0a07050806060508060607080606080707050806080c0a0707070a080808050608080f070506070706070a0908090c080708080806090508060606090906060d07050708080405070708"}]

Relay refuses to count

["COUNT", <subscription_id>, {"kinds": [4], "authors": [<pubkey>], "#p": [<pubkey>]}]
["CLOSED", <subscription_id>, "auth-required: cannot count other people's DMs"]

4.2 KiB Raw Blame History