Change NIP to new scheme

41.md

@@ -4,55 +4,31 @@ NIP-41
 Stateless Unambiguous Key Invalidation
 --------------------------------------
 
-`draft` `optional` `author:fiatjaf` `author:RubenSomsen`
+`draft` `optional` `author:fiatjaf` `author:RubenSomsen` `author:optout`
 
-The idea of this NIP is that compromised keys can be invalidated in a way that prevents identity theft.
+The idea of this NIP is that compromised keys can be invalidated to prevent identity theft, in a secure way.
 
 Cryptographers must forgive me for trying to write this in a way that humans like myself can understand.
 
-## Notation
-
-Lowercase letters denote private keys, uppercase letters denote public keys. `||` means concatenate. `sk_` means "secret key", `pk_` means "public key". `G` is the generator point of the secp256k1 curve.
 
 ## Motivation and explanation
 
-Currently if a private key `A` is compromised, for example, because they chose to input it in a malicious or infected client, they can generate a new key and send events from `A` saying: "hey, this key was compromised, my new key is `B`".
+Currently if a private key `A` is compromised, for example, due to usage in a malicious or compromised client, the owner can generate a new key and send events from `A` saying: "hey, this key was compromised, my new key is `B`".
+
 
 ### Problems with the approach above
 
-This is not a **safe** way to rotate keys, because the attacker who has `a` could publish an recent with a lower `created_at` value pointing people to a new key `X`, for example, instead of `B`. And for any reader only seeing these two events it becomes at least very hard to decide which one, `X` or `B`, is the actual new key of `A`'s owner.
+This is not a **secure** way to rotate keys, because the attacker who has `a` could publish a competing event with a lower `created_at` value pointing to a new key `X` (instead of `B`). And for any reader only seeing these two events it becomes at least very hard to decide which one, `X` or `B`, is the actual new key of `A`'s owner.
 
 Even if the attacker doesn't do anything, it is still not a **scalable** way (for the lack of a better word) to do this, since it will require manual action from all readers to migrate to stop following the previous key and start following the new. The amount of manual action required grows as the number of followers of `A` grows. And they may not even see the note from `A` because they have a giant feed with hundreds of notes, or they may forget to perform the manual unfollow-refollow process and so on.
 
+
 ### A better solution
 
-If we can create a cryptographically verifiable way to _point users to a new key_ that could both _negate the possibility of identity theft_ and _allow clients to perform the unfollow/follow_ process automatically.
+If we can create a formalized communication for key rotation and a scheme whereby new keys can cryptographically verifiably verified that could both _negate the possibility of identity theft,_ _exclude the possibility of hijacked key rotation,_ and _allow clients to perform the unfollow/follow_ process automatically.
 
-The way this NIP does it is by generating a sequence of keys such that each one of these _commits_ (i.e., that hides a value in a way that it can't be changed but can be revealed later) to the following. The scheme is as follows:
+The way this NIP achieves this is by pre-generating a sequence of extended key pairs such that each one of these _commits_ (i.e., that hides a value in a way that it can't be changed but can be revealed later) to the following. Initially only the last key is published and it used for identity. In case of a compromise, the key can be rotated to the _previous_ one, and it is possible to verify that it is a genuine pre-existing key, as the compromised can can be genreated from it.
 
-```
-A = A' + hash(A'||B)
-B = B' + hash(B'||C)
-C = C' + hash(C'||D)
-D = ...
-```
-
-The idea here is that a client will pregenerate 256 keys in sequence (arbitrary number) and use the last one, which in this case will be `A`. If `A` gets compromised they can publish a special event
-
-```
-{
-  "kind": 13,
-  "tags": [
-    ["p", "A"],
-    ["hidden-key", "A'"]
-  ],
-  "content": "optional explanation",
-  "pubkey": "B"
-  "created_at": ...
-}
-```
-
-And from this event alone any client will be able to verify that `hash(A'||B) + A' = A`, or, in other words, that `hash(.tags["hidden-key"] || .pubkey) + .tags["hidden-key"] = .tags["p"]`, and that the next key in the sequence can't be anything other than `B`, which has a private key `b` that can be assumed to have not been compromised. So it is easy to just stop following `A` and start following `B`.
 
 ### Justifying the NIP title
 
@@ -60,50 +36,117 @@ The invalidation is **unambiguous**, i.e., once an invalidation event is fired f
 
 The invalidation is **stateless**, i.e., the clients who see an invalidation event don't have to know anything else, by reading that and that only they can conclude that the previous key was irrevocably compromised and revoked, and can react properly, by, for example, unfollowing the previous key and following the new one.
 
+
+### Earlier work
+
+The first version of this NIP employed a custom derivation scheme, using key hashing and arithmetics, quite similar to BIP32 but different. Employing the established BIP32 standard allows for easier implementation.
+
+
+## Notation
+
+The following abbreviations are used:
+
+- `SK` secret key (32 bytes)
+- `PK` public key
+- `ESK` extended secret key, consisting of private key part (32 bytes) and 'chain code' part (32 bytes), as defined in BIP32.
+- `EPK` extended public key, consisting of public key part (33 bytes) and 'chain code' part (32 bytes), as defined in BIP32.
+- `CC` is the chain code
+
+
 ## Implementation 
 
-If, at the time of first generating their Nostr public keys, one decides to use this NIP, they will have a chain of 256 keys that can be used in sequence, which means a user can lose up to 255 keys, which should be much more than enough for any user that is not actively trying to lose their keys.
+Generation of keys uses key generation used by Bitcoin BIP-32 'Hierarchically Deterministic' wallets 
+
+Verification operates on public keys only, taking use of the BIP-32 property that a public key derived from the corresponding public key of an extended private key is the same as the one corresponding to the derived private key.
+
 
 ### Key generation
 
 This is intended to be run on safe and trusted hardware.
 
-A (BIP-32)[https://bips.xyz/32) seed is generated by any means (probably using BIP-39 words is the best idea), then the sequence of 256 hidden keys is generated with the following paths:
+A (BIP-32)[https://bips.xyz/32) seed is generated by any means (probably using BIP-39 words is the best idea).
+The extended private keys are then derived iteratively:
 
 ```
-sk0' = m/44'/1237'/41'/0'
-sk1' = m/44'/1237'/41'/1'
+ESK_1 = f(ESK_0)
+ESK_2 = f(ESK_1)
+ESK_3 = f(ESK_2)
 ...
-sk2' = m/44'/1237'/41'/255'
+ESK_N = f(ESK_N-1)
 ```
 
-- The "root" key, i.e. the last key in the sequence, is equal to the "hidden" key for the first position: `sk0` = `sk0'`.
-- Its immediate child, i.e. the second-to-last key in the sequence is calculated as follows:
-  - calculate the public key for `sk0`: `pk0 = sk0 * G`;
-  - take the bytes for the X coordinates of that: `pk0x = getX(pk0)`;
-  - calculate the public key for `sk1'`: `pk1' = sk1' * G`;
-  - take the bytes for the X coordinates of that: `pk1x' = getX(pk1')`;
-  - concatenate the bytes `pk0x` and `pk1x'` and calculate the sha256 hash of that: `hash = sha256(pk0x || pk1x')`;
-  - add that, as a number, to the hidden private key `sk1'` (which is a number): `sk1 = sk1' + hash`;
-  - the result is the second-to-last secret key `sk1`.
-- Now proceed to calculate `sk2` using the same process above, but moving everything one number up.
-- And so on and so forth until you get to `sk255`.
-- `sk255` is the key to be used first. If that gets compromised one must move to `sk254` and so on.
+where:
+
+- the `f()` function takes the BIP32 `41`th child of the previous extended private key,
+- `ESK_0` is derived from the seed, using derivation path `m/44'/1237'/41'` (using values from NIP-06, and `41` as present NIP number)
+- N is an arbitrary number, such as 256.
+
+The user will take the key pair `(SK_N, PK_N)` of the last extended key `ESK_N`, and use them.
 
 Presumably this key generation process happens in somewhat trusted hardware by a trusted dedicated program, and from it users may copy the initial key and paste it in the normal (still trusted) client they use for day-to-day operations.
 
+Note that in the iterations _non-hardened_ derivation is used, this is in order to allow verification to proceed on the extended public keys (in case of hardened derivation this is not possible),
+
+Another way to define the extended private keys is that they are derived from the seed using the following derivation paths:
+
+```
+m/44'/1237'/41'
+m/44'/1237'/41'/41
+m/44'/1237'/41'/41/41
+m/44'/1237'/41'/41/41/41
+...
+```
+
+Note that the keys are children of each other, not siblings, as is the case e.g. for Bitcoin addresses in an HD wallet.
+
+
+### Rotation and Verification
+
+If at some point the key gets compromised, the user can publish a special event to rotate to a new identity.
+The form of the event is:
+
+```
+{
+  "kind": 13,
+  "tags": [
+    ["p", "PK_N"],
+    ["hidden-key", "CC_N-1'"]
+  ],
+  "content": "optional explanation",
+  "pubkey": "PK_N-1"
+  "created_at": ...
+}
+```
+
+From this event alone any client will be able to verify that `PK(f'(EPK(PK_N-1, CC_N-1))) = PK_N`. `f'()` here is very similar to `f()`, but it performs derivation on the public keys.
+The verified relationship means that the newly published key must have been generated _before_ the compromised key (as the latter can be generated from it), so it can be assumed to be belonging to the user and not made up by an attacker, so the clients can start using the new key.
+
+
+
 ### Reader implementation
 
-Clients can query relays for the invalidation of key `A` whenever they want by using the filter `{"#p": ["A"], "kinds": [13]}`.
+Clients can query relays for the invalidation of a key `A` whenever they want by using the filter `{"#p": ["A"], "kinds": [13]}`.
 
 The verification process for the validity of an invalidation (`kind: 13`) event was given in the explanation above.
 
+
 ## Additional comments
 
 ### This is not a general "rotation" scheme for keys
 
 This scheme is intended to make it less catastrophic when a key is compromised because it was input into a computer, phone or Nostr client softwre that turns out later to have been compromised. It isn't intended to allow people to rotate their keys every month as a routine practice, nor is it supposed to let users be reckless and give their private keys to any malware or trusted third parties. A compromised key is still a bad event, just not one of awful and unrepairable consequences if this NIP is followed and relied upon.
 
+
+### Notes on Implementation
+
+32 vs 33-byte public keys. In Nostr usually the 32-byte 'X-only' format public key is used. However, in the BIP32 scheme employed here the 33-byte 'compressed' format public key is used. The 32-byte format misses the parity. In practice this means that at verification both parity alternatives have to be checked.
+
+For derivation an 'off-the-shelve' BIP32-capable library implementation can be used directly, or an own implementation. The own implementation is recommended, for the reasons:
+
+- Due to a limitation that stems from serialization format that is not relevant here, implementations typically limit the maximum depth of a derivation path to 256, meaning that at most 253 keys could be generated in this scheme (though in practice that's probably sufficient).
+- Constructing an extended public key from its parts is needed at verification, but this is not a typical operation in the BIP32 use cases, and it cannot be easily accessed, only through some workarounds.
+
+
 ### Fallback mechanism possibilities
 
 What happens if Bob is following Carol and Carol publishes an invalidation event for her key, but Bob's client doesn't see it for any reason or doesn't support the automatic refollow mechanism for the new key? Well, in this case we are at least not worse than the current state of things, but Bob has other possibilities:
@@ -112,3 +155,4 @@ What happens if Bob is following Carol and Carol publishes an invalidation event
   - or there could be a central directory Bob can use to check from time to time if any of their followed keys has been invalidated and perform the fixes manually in his client;
   - anyone with a supporting client can be 100% sure the key was really invalidated, so they can alert others about that fact without any fear of spreading wrong information;
   - and last but not least Bob may want to verify things manually if he sees suspicious activity from Carol's key. At least he doesn't have to ask her (and then get a response from the attacker stating that "no, it's just me, Carol!").
+