nips/66.md

NIP-66: Relay Meta and Monitoring System

draft optional

This NIP defines four event kinds, 10066, 30066, 10166 and 1066.

10066: "Self-Reported Relay Meta"

Summary

An event that contains data about a relay. These events are self-reported by relays from the owner pubkey defined in their Information Document (NIP-11). Hereinafter referred to as "Relay Meta" events

Purpose

An event that makes relays discoverable.

Schema

event.content

Should be empty.

event.tags

"Relay Meta" events have one (1) required tag.

"Relay Meta" events have two (2) non-standard, NIP-66 specific indexable tags.

Tags should be used to represent the Relay's abilities and meta at any given point in time. The "tags schema" is expressed using pseudo-types for reasons of communication and brevity, for example tagDescriptor[]. This NIP utilizes NIP-32 to expose indexable values.

Tags

• "r" == tagId[0] The relay URL. The #d tag must be included in the event.tags[] array. tagId[1] must be the relay websocket URL. The URL should be normalized

  ["d", "wss://history.nostr.watch/"] //tagId[]
  

• n: Such as clearnet, tor, i2p, cjdns, etc in the context of NIP-66 only

  [ "n", "clearnet" ]
  

• desc == tagDesc[0] The description from nip11.

  [ "desc", "A nostr relay" ]
  

• N: Supported NIPs in the context of NIP-66 only

  [ "N", "1" ]
  [ "N", "11" ]
  [ "N", "33" ]
  

• "count" == tagCount[0] Meta values for arbitrary counts related to a relay. tagCount[1] is the value expressed as the string representation of an integer or float. tagCount[2] is the key and describes the count, such as total_users or total_events. Counts should only be included when representing unique or computationally expensive counts, not ones that can be easily achieved via NIP-45 counts. There may be zero (0) to many count tags.

  ["count", "total_events", "502341"],
  ["count", "total_users", "52000"],
  ["count", "active_users_24h", "321"]
  ["count", "events_per_minute", "21.4"]
  

• "infohash" == tagInfohash[0] A SHA256 hash of the deterministically stringified NIP-11 "Info Document" JSON at tagInfohash[1]

  [ "infohash", "fj89as0n3inmcd89nefkdlsky2mfkdlsfds" ]
  

Label Tags ^NIP-32

• relay_type, for example proxy, bridge and public for example. There may be more than one type. There have not been efforts to atomize relay types at the time of writing this NIPs, so these values cannot be enumerated at this time.

  [ "L", "relay_type" ]
  [ "l", "proxy", "relay_type" ]
  [ "l", "public", "relay_type" ]
  

• ipv4/ipv6, relay ips.

  [ "L", "ipv4" ]
  [ "l", "1.1.1.1", "ipv4" ]
  [ "L", "ipv6" ]
  [ "l", "2001:db8:3333:4444:5555:6666:7777:8888"", "ipv6" ]
  

• as/asn the relay ISPs "Autonomous System (Number)"

  [ "L", "as" ]
  [ "l", "QUAD9-AS-1, CH", "as" ]
  [ "L", "asn" ]
  [ "l", "19281", "asn" ]
  

• String values from NIP-11 using dot notation, for example

  [ "L", "nip11.tags" ]
  [ "l", "sfw-only", "nip11.tags"  ]
  [ "l", "bitcoin-only", "nip11.tags"  ]
  [ "L", "nip11.language_tags" ]
  [ "l", "en", "nip11.language_tags"  ]
  [ "l", "en-419", "nip11.language_tags" ]
  

• Boolean values from NIP-11, for example

  [ "L", "nip11.limitations" ]
  [ "l", "payment_required", "nip11.limitations" ] //nip11.limitations.payment_required==true
  [ "l", "auth_required", "nip11.limitations" ] //nip11.limitations.auth_required==true
  

Geo tags ^NIP-YAGT

• NIP-66 leverages a draft NIP for geo tags, see YAGT

  [ 'G', 'countryCode' ]
  [ 'g', 'US', 'countryCode', 'alpha-2' ]
  [ 'g', 'USA', 'countryCode', 'alpha-3' ]
  [ 'g', '840', 'countryCode', 'numeric' ]
  [ 'G', 'regionCode' ]
  [ 'g', 'US-CA', 'regionCode']
  [ 'g', '9r1652whz' ]
  [ 'g', '9r1652wh' ]
  [ 'g', '9r1652w' ]
  [ 'g', '9r1652' ]
  [ 'g', '9r165' ]
  [ 'g', '9r16' ]
  [ 'g', '9r1' ]
  [ 'g', '9r' ]
  [ 'g', '9' ]
  

Topic Tags

• NIP-66 leverages topics [t tags] and optionally adds values at positions tag[2] and tag[3]. If tag[2] is set, it should be a string-integer representing the number of tags on the relay. If tag[2] is set, tag[3] should be set and be a string-integer representing the timeframe for the value set as tag[2]. This tag is used to indicate topics relevant to the relay through event analysis (as oppossed to 'tags' from NIP-11 document)

   //69 events with "meme" topic in the last hour
  ["t", "meme", "69", "3600"]
  

30066: "Relay Status" Events

Summary

30066 is a parameterized replaceable event [NIP-33], referred to as a "Relay Status" event. These events store the existence and optionally some data relating to the relay.

Purpose

To store useful, computationally expensive relay meta-data about relays. 30066 should be used to give a canonical reference to the last known functional state of a relay, to be evaluated subjectively by a client and/or user to derive conclusions from.

NIPs Used

• NIP-32 Labels
• NIP-33 Parameterized Replacable Events
• NIP-40 Expirable Events
• YAGT draft

Schema

30066 events are almost identical to 10066 except a few key differences.

Different referencial tag

• 30066 uses a d tag instead of an r tag for the normalized relay URL since it is a Parameterized-Replaceable Event

It adds some additional tags

• "rtt" == tagRtt[0] Round-trip time of websocket ping/pong in milliseconds. Example values for tagRtt[1] are open, read and write. Where open represents the round-trip time forWebsocket to open, read represents the round-trip time of a Websocket REQ (subscription) message's response, and write represents the round-trip time of a Websocket EVENT (publish) message and subsequent ok message. tagRtt[2...] are strings with the millisecond values. At a minimum tagRtt[2] should be set. When more than one value is provided, values may be treated as an array to find min, max, average and median values. There may be zero rtt tags.

  ["rtt", "open", "201", "190", "540"],
  ["rtt", "read", "35", "60", "46"],
  ["rtt", "write", "701", "497", "508"]
  

Methodology

A Relay Monitor publishes `30066`` events exclusively when a relay is operational. This approach ensures that the last known active state of the relay is maintained and recorded. Based on this data, several inferences can be drawn about the relay's status and characteristics

1. Clients and/or users can set a custom threshold to establish a cutoff timestamp for filtering events using since. This timestamp helps in identifying which relays are currently online. Selecting a lower threshold value results in a stricter criterion for relay uptime, making the filter more sensitive to brief downtimes. Conversely, choosing a higher threshold value creates a more lenient filter, allowing relays with longer downtimes to still be considered as online.

2. In determining whether a relay is 'dead,' the decision is solely at the discretion of the client or user. The are responsible for setting and applying arbitrary thresholds using until filters or post-processing to make this determination. This approach underscores that the classification of a relay as 'dead' is a subjective decision, varying according to each client's or user's assessment criteria, rather than a fixed status provided by the monitor.

3. For relay status events that have become outdated, the retained data points remain valuable. They offer insights and information about the relay's characteristics and performance, which might not be currently accessible due to the relay being offline.

Important Notes

• NIP-11 values are provided as means to filter and discover relays, however should not be used as a replacement to NIP-11.
• A particular relay's retention policy could conflict with subjective thresholds used in determination of "online" or "dead" relays and so care should be taken with regards to chosen relays by both monitors and consumers.

Examples

Minimum Requirements, "Relay was online at Date(created_at)"

{
  "id": "<eventid>",
  "pubkey": "<pubkey>",
  "created_at": "<created_at>",
  "signature": "<signature>",
  "content": "",
  "tags": [  
    ["d","wss://some.relay/"]
  ]
}

Relay is checked and online example, "Relay is online and here's some meta-data this monitor believes is accurate"

{
  "id": "<eventid>",
  "pubkey": "<monitor's pubkey>",
  "created_at": "<created_at  [some recent date ...]>",
  "signature": "<signature>",
  "content": "{}",
  "tags": [  
    ["d","wss://some.relay/"],
    ["N", "1"],
    ["N", "7"],
    ["rtt", "open", "201", "190", "540"],
    ["rtt", "read", "35", "60", "46"],
    ["rtt", "write", "701", "497", "508"],
    ["ssl", "valid", "timestamp", "timestamp"],
    ["count", "502341", "events"],
    ["count", "21.4", "events_per_minute"],
    ["L", "network"]
    ["l", "clearnet", "network"]
    ["g", "ww8p1r4t8"],
    ["g", "NL", "countryCode", "alpha-2"],
    ["L", "ipv4"],
    ["l", "1.1.1.1""ipv4" ],
    ["L", "ipv6"],
    ["l", "2001:db8:3333:4444:5555:6666:7777:8888", "ipv6"]
  ]
}

Testing Criteria

The testing criteria to determine conditions defined in event's tags may be subjective and may vary between monitors.

Limitations

The data in 30066 may be erroneous, intentionally or otherwise. Where accuracy is required, the data in 30066 events should be subscribed to by populating the authors filter array with the pubkeys of trusted monitors and where security or privacy is a concern any republished values (such as NIP-11 values) should instead be attained from the source. All data is for informational purposes and to make finding and filtering through relays through nostr a possiblity.

Appending NIP-66

Any test results that cannot be expressed through NIP-66 should be ammended to the nip following discussion and general consensus

Use Cases

1. Geographic Relay Discovery: Identify relays situated near a specific geographic location or within a particular country, facilitating localized network interactions.

2. NIP Support Filtering: Search for relays based on their support for specific Nostr Improvement Proposals (NIPs), ensuring compatibility with desired protocol features.

3. Accessibility Search: Locate relays that are free to use, helping users find cost-effective or no-cost options for their network interactions.

4. Real-Time Status Monitoring: Utilize a status client to display up-to-date statuses of various relays, providing insights into their current operational state.

5. Relay Network Analysis: Analyze connections and patterns between relays using their IP addresses, aiding in network topology understanding and security assessments.

6. Error Detection in Relay Listings: Spot and rectify erroneous entries in relay lists, ensuring the accuracy and reliability of relay data.

7. Performance Benchmarking: Compare relays based on performance metrics like round-trip times and uptime, aiding in the selection of the most efficient relays for specific needs.

8. Security and Compliance Checks: Evaluate relays for adherence to security standards and regulatory compliance, essential for users with specific security and privacy requirements.

9. Language and Content Filtering: Identify relays catering to specific languages or content types, enabling users to engage in a more targeted and relevant social networking experience.

10. Data-Driven Relay Selection: Make informed choices about which relays to connect to, based on comprehensive metadata including user counts, event frequencies, network types and more.

10166: "Relay Monitor" Events

Summary

10166 is a replacable event herein referred to as "Relay Monitor" events. These events contain information about a publisher's intent to monitor and/or aggregate relays and publish data as 30066 events.

Purpose

To provide a directory of monitors, their intent to publish, their criteria and parameters of monitoring activities.

Schema

10166 events have no required tags.

• "url" == tagUrl[0] A URL with human readable information about the monitor's activities. If not included, it is assumed these details are included in the monitor's kind 0 for the pubkey that signed the 10066 event.

• "timeout" == tagTimeout[0] The timeout values for various tests. tagTimeout[1] is the monitor's timeout in milliseconds. tagTimeout[2] describes what test the timeout is used for, for example open, read, write, info, etc. If no tagTimeout[2] is provided, it is inferred that the timeout provided applies to all tests.

  [ "timeout", "2000", "open" ]
  [ "timeout", "2000", "read" ]
  [ "timeout", "3000", "write" ]
  [ "timeout", "2000", "info" ]
  

• "ssl" == tagSsl[0] SSL certificate check. tag[1] is either valid or invalid. tag[2] is the validate start date. tag[3] is the validity end date. Can include reference via e or a to raw SSL data for indepdent validation.

  ["ssl", "valid", "timestamp", "timestamp"],
  

• "kind === tagKind[0]" The kind(s) this monitor publishes.

  [ "kind", "30066" ]
  [ "kind", "1066" ]
  

• "frequency" == tagFrequency[0] The frequency at which the monitor publishes events. A string-integer at tagFrequency[1] represents the frequency a event kind is published. The string-integer at tagFrequency[2] represents the kind bound to the frequency. If tagFrequency[2] is undefined, it is inferred the timeout applies to all events published by the monitor. A frequency should not be created for this kind [1006]. If a kind is referenced in tagFrequency that is not set in tagKind it should be disregarded.

  [ "frequency", "3600", "30066" ]
  [ "frequency", "3600", "1066" ]
  

Label Tags ^NIP-32

• checks should be a lowercase string describing the check(s) conducted by a monitor. Some examples are: websocket, nip11, ssl, dns, geo

  [ "L", "checks" ]
  [ "l", "websocket", "checks" ]
  [ "l", "nip11", "checks" ]
  [ "l", "ssl", "checks" ]
  [ "l", "dns", "checks" ]
  [ "l", "geo", "checks" ]
  

Geo tags ^NIP-YAGT

• NIP-66 leverages a draft NIP for geo tags, see YAGT

Other Requirements

• Monitors should have a published 10002 event that defines the relays the monitor publishes to.

1066: "Relay Status History" Events

Summary

1066 is a standard nostr event, referred to as a "Relay Status History" event. These events store the history of a relay at a periodic interval (defined in 10166).

Schema

Relay History events can contain any of the tags defined for kind 30066 (Including the d tag which is used to filter these events by relay). There is one new indexable tag

Special Considerations

• These events can potentially demand a lot of disk space if not used wisely.

Guidelines

Tags representing less-volatile values, for instance nip11.software vs rtt, contained with this event should only be included when their respectiive values have changed compared to the last Relay History event of the same d tag. If there have been no changes, then the event should not be published.

Use Cases

• Generate a time series from one or data-point(s) for a specific relay.
• Generate network-wide aggregated statistics