BIP461: Deterministic ECDSA Signatures

bip-low-r-grinding.md

@@ -0,0 +1,243 @@
+    BIP: ?
+    Layer: Applications
+    Title: Low-R Grinding for ECDSA Signatures
+    Authors: Liam Gilligan <liamdgilligan@gmail.com>
+    Status: Draft
+    Type: Specification
+    Assigned: ?
+    License: CC0-1.0
+
+## Abstract
+
+Low-R Grinding is an algorithm for re-deriving the ECDSA nonce for a fixed key and message, under a counter, until the resulting signature's *r* value has its most significant bit unset, so that the DER encoding of *r* requires no leading null byte.
+Combined with low-*s* normalization, this bounds a DER-encoded ECDSA signature at 70 bytes, or 71 with the sighash flag appended, rather than 71 and 72.
+
+This document specifies an algorithm for computing such low-*r* signatures, and provides a reference implementation and test vectors to ensure all implementations produce byte-for-byte identical signatures.
+
+## Motivation
+
+Bitcoin encodes ECDSA signatures in a strict DER form.[^bip66]
+DER treats *r* and *s* as big-endian *signed* integers, so an integer whose most significant bit is set must be preceded by a null byte to keep it positive.
+
+A high-*s* value of a signature can be normalized through simple negation, resulting in an equally valid signature whose *s* value has its most significant bit unset. A signature's *r* value cannot be modified in a similar way: it is the x-coordinate of the nonce point, and is fixed once the nonce is chosen. Its most significant bit is set about half the time.
+Thus, about half of ECDSA signatures carry an extra padding byte when DER encoded.
+
+The only way to obtain a different *r* is to re-sign with a different nonce. On average, Low-R Grinding takes two signing operations[^geometric] and saves roughly 0.5 vbytes per signature in legacy inputs and 0.125 vbytes in v0 segwit inputs.[^optech]
+
+In 2-of-3 P2SH inputs, having at least one high-*r* signature increases the size of the input by two additional bytes, as the input script's length indicator must be three bytes instead of one. On average, Low-R Grinding will save 2.5 vbytes across such inputs.
+
+Combined with normalizing *s*, Low-R Grinding lets a signer produce ECDSA signatures for arbitrary messages with a fixed *maximum* size: 70 bytes of DER, 71 with the sighash flag. This allows for lower fees and more accurate transaction size predictions.
+
+A signer that does not perform Low-R Grinding is identifiable on-chain, since about half of its signatures will have a high *r* value. Uniform adoption avoids partitioning transactions into identifiable groups.[^review]
+
+## Specification
+
+### Notation
+
+Lowercase variables represent integers, byte arrays, or signatures:
+
+- The constant *p* refers to the field size, *0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEFFFFFC2F*.
+- The constant *n* refers to the curve order, *0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141*.
+- The constant *empty_bytestring* refers to the byte array of length zero.
+- An ECDSA signature *sig* can be represented literally by an integer pair *(r, s)*, with *0 < r, s < n*.
+
+Points on the curve with equation *y<sup>2</sup> = x<sup>3</sup> + 7* over the integers modulo *p*:
+
+- The point at infinity is the identity element of the curve group, and is the only point without X and Y coordinates.
+- The function *is_infinite(P)* returns whether or not *P* is the point at infinity.
+- *x(P)* and *y(P)* are integers in the range *0..p-1* and refer to the X and Y coordinates of a point *P* (assuming it is not infinity).
+- The constant *G* refers to the base point, for which *x(G) = 0x79BE667EF9DCBBAC55A06295CE870B07029BFCDB2DCE28D959F2815B16F81798* and *y(G) = 0x483ADA7726A3C4655DA4FBFC0E1108A8FD17B448A68554199C47D08FFB10D4B8*.
+- Addition of points refers to the usual [elliptic curve group operation](https://en.wikipedia.org/wiki/Elliptic_curve#The_group_law).
+- [Multiplication (⋅) of an integer and a point](https://en.wikipedia.org/wiki/Elliptic_curve_point_multiplication) refers to the repeated application of the group operation.
+
+Functions and operations:
+
+- *||* refers to byte array concatenation.
+- The function *int(x)*, where *x* is a 32-byte array, returns the 256-bit unsigned integer whose most significant byte first encoding is *x*.
+- The function *bytes(l, x)*, where *x* is an integer, returns the *l*-byte encoding of *x*, most significant byte first.
+- The function *bytes_le(l, x)*, where *x* is an integer, returns the *l*-byte encoding of *x*, least significant byte first.
+- The function *fill_bytes(l, x)*, where *x* is a byte, returns the length *l* byte array in which every byte is *x*.
+- The function *hmac(k, m)*, where *k* and *m* are byte arrays, returns HMAC-SHA256[^hmac] with key *k* over message *m*.
+- The function *ser_sig(r, s)* returns the DER encoding of the signature *(r, s)* required by BIP 66[^bip66]. The one-byte sighash flag is appended to the DER encoding separately, outside of the algorithms specified in this document.
+- *x<sup>-1</sup>* refers to the modular multiplicative inverse of the integer *x* modulo *n*.
+
+### Algorithms
+
+This section specifies the following algorithms:
+
+#### ECDSA Signing
+
+Input:
+
+- The secret key *sk*: a 32-byte array
+- The message hash *msghash*: a 32-byte array
+- The nonce *nonce*: a 32-byte array
+
+The algorithm *sign(sk, msghash, nonce)* is defined as:
+
+- Let *d = int(sk)*
+- Fail if *d = 0* or *d >= n*
+- Let *k = int(nonce)*
+- Fail if *k = 0* or *k >= n*
+- Let *R = k⋅G*
+- Let *r = x(R) mod n*
+- Fail if *r = 0*
+- Let *s' = k<sup>-1</sup> * (d * r + int(msghash)) mod n*
+- If *s' > (n - 1) / 2*:[^lows]
+  - Let *s = n - s'*
+- Else:
+  - Let *s = s'*
+- Fail if *s = 0*
+- Return *(r, s)*
+
+#### ECDSA Verification
+
+Input:
+
+- The signature *sig*: *(r, s)*
+- The public key *P*: a point on the curve
+- The message hash *msghash*: a 32-byte array
+
+The algorithm *verify(sig, P, msghash)* is defined as:
+
+- If *is_infinite(P)*:
+  - Return *false*
+- If *y(P)<sup>2</sup> ≠ x(P)<sup>3</sup> + 7 mod p*:
+  - Return *false*
+- Let *(r, s) = sig*
+- If *r = 0* or *r >= n*:
+  - Return *false*
+- If *s = 0* or *s >= n*:
+  - Return *false*
+- Let *w = s<sup>-1</sup>*
+- Let *u<sub>1</sub> = w * int(msghash) mod n*
+- Let *u<sub>2</sub> = w * r mod n*
+- Let *R = u<sub>1</sub>⋅G + u<sub>2</sub>⋅P*
+- If *is_infinite(R)*:
+  - Return *false*
+- If *x(R) mod n ≠ r*:
+  - Return *false*
+- Return *true*
+
+#### Nonce Derivation
+
+Input:
+
+- The secret key *sk*: a 32-byte array
+- The message hash *msghash*: a 32-byte array
+- The additional data *extra*: a byte array, possibly empty
+- The attempt counter *count*: an integer, *count >= 0*
+
+The algorithm *nonce_rfc6979(sk, msghash, extra, count)* is defined as:[^kv]
+
+- Let *h = bytes(32, int(msghash) mod n)*[^bits2octets]
+- Let *V = fill_bytes(32, 0x01)*
+- Let *K = fill_bytes(32, 0x00)*
+- Let *K = hmac(K, V || 0x00 || sk || h || extra)*
+- Let *V = hmac(K, V)*
+- Let *K = hmac(K, V || 0x01 || sk || h || extra)*
+- Let *V = hmac(K, V)*
+- Let *V = hmac(K, V)*
+- For *j = 1, ..., count*:
+  - Let *K = hmac(K, V || 0x00)*
+  - Let *V = hmac(K, V)*
+  - Let *V = hmac(K, V)*
+- Return *V*
+
+#### Low-R Grinding
+
+Input:
+
+- The secret key *sk*: a 32-byte array
+- The message hash *msghash*: a 32-byte array
+
+The algorithm *LowRSign(sk, msghash)* is defined as follows, where the loop variable *i* is the **grind counter**:
+
+- Fail if *int(sk) = 0* or *int(sk) >= n*
+- For *i = 0, ..., 2<sup>32</sup>-1*:[^geometric]
+  - If *i = 0*:
+    - Let *extra = empty_bytestring*[^ndata]
+  - Else:
+    - Let *extra = bytes_le(4, i) || fill_bytes(28, 0x00)*
+  - For *count = 0, 1, 2, ...*:[^attempt]
+    - Let *nonce = nonce_rfc6979(sk, msghash, extra, count)*
+    - Let *(r, s) = sign(sk, msghash, nonce)*; break if *sign* does not fail
+  - If *r < 2<sup>255</sup>*:
+    - Let *P = int(sk)⋅G*
+    - If *verify((r, s), P, msghash)* returns *false*, abort[^verifysign]
+    - Return *ser_sig(r, s)*
+- Fail
+
+The predicate *r < 2<sup>255</sup>* is equivalent to testing that the most significant byte of *bytes(32, r)* is below *0x80*, which is how Bitcoin Core expresses it.[^core]
+
+Other nonce derivation strategies also yield low-*r* signatures, and those signatures are equally valid and indistinguishable on-chain from the signatures specified here. A signer using another strategy implements the idea described in this document, but does not comply with this specification, and its signatures cannot be checked against the accompanying test vectors. The strategies that were considered are discussed in the Rationale.
+
+## Rationale
+
+### Why an Exact Implementation is Specified
+
+A malicious signer is free to choose its nonces, and can choose them so that its signatures leak key material.[^darkskippy] A standardized, deterministic signing algorithm removes that freedom: the signature becomes a function of the secret key and message alone, so every signer following the standard produces the same bytes, and any implementation of the standard can recompute a signer's output and compare. A signer complying with this standard therefore cannot exfiltrate secrets through nonce choice without risking detection. Note that standardization is necessary, not just determinism: an algorithm nobody else implements leaves nothing to compare against.
+
+### Why RFC 6979 with a Grind Counter
+
+Producing low-*r* signatures deterministically requires deriving the nonce deterministically. The derivation must be a function of the secret key, so that an adversary cannot compute the nonce; of the message, to prevent nonce reuse across messages; and of a grind counter, to allow the grinding process. Nonces derived under the same secret key for different messages must also be unrelated.[^noncereq]
+
+RFC 6979 provides all of these properties. It derives the nonce deterministically from the secret key and message hash, and its section 3.6 permits an additional data input to carry the grind counter. Because the derivation runs through HMAC-SHA256, distinct inputs yield nonces with no tractable relation. The RFC states what that input must satisfy, and names a counter as its example:[^rfc6979]
+
+> It suffices that the additional data k' is non-repeating (e.g., a signature counter or a monotonic clock) to ensure "random-looking" signatures are indistinguishable, in a cryptographic way, from plain (EC)DSA signatures.
+
+The grind counter is exactly such a value, and can be used as the additional data directly.
+
+### Potential Alternate Designs
+
+**A fresh random nonce on each attempt.** Drawing each candidate nonce from a cryptographically secure random source also yields unrelated nonces, and is a valid means of producing low-*r* signatures. Random nonces are not recommended because the signatures are not reproducible, which reopens the exfiltration problem described above.
+
+**Grinding for smaller *r*.** In order for a lower *r* value to save any space, it must be an entire byte smaller, and thus must have the leading 9 bits unset rather than just the leading bit. Computing such an *r* value would take 512 signing operations on average, as opposed to the two of regular Low-R Grinding.[^geometric2] Grinding for a smaller *r* was deemed too expensive to be adopted widely. Because few signers would do so, the signers that did would be identifiable on-chain.[^pr13666]
+
+**Speeding up grinding with the *λ* endomorphism.** It was suggested to use secp256k1's *λ* endomorphism to quickly compute alternative nonces.[^endomorphism] Low-R Grinding as specified takes only two point multiplications on average. The added complexity of using the *λ* endomorphism cannot meaningfully improve on that.[^endospeedup]
+
+## Backward Compatibility
+
+Signatures generated using Low-R Grinding are normal valid signatures and will verify under existing consensus rules, and thus are fully backward compatible.
+
+## Copyright
+
+This BIP is licensed under the CC0-1.0 license.
+
+[^bip66]: [BIP 66: Strict DER signatures](https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki). The corresponding `SCRIPT_VERIFY_DERSIG` flag is one of Bitcoin Core's mandatory (consensus) script verification flags.
+
+[^geometric]: Each attempt yields a low *r* with probability approximately 1/2. The probability that the first success occurs on attempt *j* is therefore 2<sup>-j</sup>, the expected number of attempts is Σ *j* · 2<sup>-j</sup> = 2, the probability that more than *j* attempts are needed is 2<sup>-j</sup>, and the loop terminates with probability 1. Bitcoin Core allots a 32-bit counter, which cannot be exhausted in practice.
+
+[^geometric2]: In general, for an event of probability *q*, the first success occurs on attempt *j* with probability *q* · (1 - *q*)<sup>j-1</sup>, so the expected number of attempts is Σ *j* · *q* · (1 - *q*)<sup>j-1</sup> = 1/*q*, with *j* ranging from 1 to infinity. Here *q* is 2<sup>-9</sup>, giving 512 signing operations on average. The threshold is nine bits rather than eight because DER restores the padding byte whenever the leading content byte has its most significant bit set: *r* must fall below 2<sup>247</sup>, not 2<sup>248</sup>.
+
+[^optech]: [Bitcoin Optech: Low-r grinding](https://bitcoinops.org/en/topics/low-r-grinding/).
+
+[^pr13666]: [Bitcoin Core PR #13666, "Always create signatures with Low R values"](https://github.com/bitcoin/bitcoin/pull/13666), released in [Bitcoin Core v0.17.0](https://bitcoincore.org/en/releases/0.17.0/). The review discussion also considered grinding for even smaller *r* values; see [this comment](https://github.com/bitcoin/bitcoin/pull/13666#issuecomment-405116700).
+
+[^review]: See, e.g., [this comment](https://github.com/bitcoin/bitcoin/pull/13666#issuecomment-406053492) and the surrounding discussion in the pull request that introduced grinding to Bitcoin Core.
+
+[^core]: [Bitcoin Core `src/key.cpp`](https://github.com/bitcoin/bitcoin/blob/master/src/key.cpp), `SigHasLowR` and `CKey::Sign`. The little-endian counter is an artifact of `CKey::Sign` writing the counter into a 32-byte entropy buffer with `WriteLE32`.
+
+[^verifysign]: Verifying the signature before leaving the signer prevents random or attacker provoked computation errors. This prevents publishing invalid signatures which may leak information about the secret key. It is recommended, but can be omitted if the computation cost is prohibitive. The preceding wording is taken from the equivalent step in the signing algorithm of [BIP 340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), which is licensed under BSD-2-Clause. Bitcoin Core performs the check unconditionally in [`CKey::Sign`](https://github.com/bitcoin/bitcoin/blob/master/src/key.cpp). Because the check cannot alter the returned signature, omitting it affects neither the test vectors nor interoperability.
+
+[^attempt]: *count* advances the RFC 6979 generator without changing *extra*, and is distinct from the grind counter *i*. libsecp256k1 increments the same counter whenever the derived nonce falls outside *[1, n-1]*, or the resulting *r* or *s* is zero; see `secp256k1_ecdsa_sign` in [libsecp256k1 `src/secp256k1.c`](https://github.com/bitcoin-core/secp256k1/blob/master/src/secp256k1.c), which retries in both cases rather than failing. Each of those conditions has probability on the order of 2<sup>-128</sup> or less, so *count* is 0 for every signature produced in practice. Because *sign* rejects an out-of-range secret key, and *LowRSign* rejects one before the loop is entered, no secret key makes every attempt fail; the inner loop terminates with probability 1.
+
+[^noncereq]: Two published signatures whose nonces stand in a relation an attacker can find will leak the secret key. Neither the intermediate attempts of a grind nor the counter that produced the successful one is published, so the constraint binds across messages rather than within a single grind loop; RFC 6979 satisfies it in both directions in any case. A fresh nonce from a cryptographically secure random source on each attempt also satisfies it, though the resulting signatures are not reproducible.
+
+[^rfc6979]: [RFC 6979: Deterministic Usage of the Digital Signature Algorithm (DSA) and Elliptic Curve Digital Signature Algorithm (ECDSA)](https://www.rfc-editor.org/rfc/rfc6979). The additional data input and the non-repetition requirement are given in [section 3.6](https://www.rfc-editor.org/rfc/rfc6979#section-3.6).
+
+[^bits2octets]: This reduction is RFC 6979's *bits2octets(h1)*, required by [section 3.2d](https://www.rfc-editor.org/rfc/rfc6979#section-3.2). It has no effect unless *int(msghash) >= n*, which for a 32-byte hash occurs with probability about 1.27 · 2<sup>-128</sup>. libsecp256k1 performs it as `msgmod32` in `nonce_function_rfc6979`; see [libsecp256k1 `src/secp256k1.c`](https://github.com/bitcoin-core/secp256k1/blob/master/src/secp256k1.c). Bitcoin Core acquired the reduction in v24.0, having fed the unreduced hash from v0.17.0 through v23.0. No signature produced in practice is affected by the difference.
+
+[^kv]: *K* and *V* are the internal state variables of the HMAC-based generator specified in [RFC 6979 section 3.2](https://www.rfc-editor.org/rfc/rfc6979#section-3.2), and carry its names. Both are 32-byte arrays rather than curve points, and *K* is unrelated to the nonce *k*.
+
+[^darkskippy]: [Dark Skippy](https://darkskippy.com/) is a method by which malicious signing device firmware embeds seed material in the nonces of otherwise valid signatures, from which an attacker who observes those signatures on-chain can recover the seed. Raised in the context of this document by Craig Raw; see [this comment](https://github.com/bitcoin/bips/pull/2224#issuecomment-5114943404).
+
+[^endomorphism]: [Suggested during review of the pull request that introduced grinding to Bitcoin Core](https://github.com/bitcoin/bitcoin/pull/13666#issuecomment-405116700); the replies that follow discuss the nonce distribution it would produce.
+
+[^endospeedup]: secp256k1 admits an endomorphism *φ* for which *φ*(*x*, *y*) = (*βx*, *y*), equivalently *φ*(*P*) = *λ*⋅*P*, where *λ* and *β* are primitive cube roots of unity modulo the group order and the field order respectively; see [libsecp256k1 `src/scalar_impl.h`](https://github.com/bitcoin-core/secp256k1/blob/master/src/scalar_impl.h), `secp256k1_const_lambda`. A single point multiplication for the nonce *k* therefore also yields the *r* values belonging to *λk* and *λ*<sup>2</sup>*k*. Since a signer must compute at least one point multiplication in any case, the endomorphism can save at most one point multiplication per signature, and obtaining that saving requires deriving the candidate points outside the signing routine, so a wallet would need its own curve arithmetic or support for the construction in the library the wallet depends on.
+
+[^lows]: [libsecp256k1 `src/ecdsa_impl.h`](https://github.com/bitcoin-core/secp256k1/blob/master/src/ecdsa_impl.h), `secp256k1_ecdsa_sig_sign`: `high = secp256k1_scalar_is_high(sigs); secp256k1_scalar_cond_negate(sigs, high);`
+
+[^ndata]: Omitting the additional data makes RFC 6979 hash a 64-byte seed rather than a 96-byte one, and the two produce different nonces. See [libsecp256k1 `src/secp256k1.c`](https://github.com/bitcoin-core/secp256k1/blob/master/src/secp256k1.c), `nonce_function_rfc6979`, which notes that the argument mixtures have distinct lengths and so cannot emulate one another; the library's test suite (`src/tests.c`, `run_rfc6979_hmac_sha256_tests`) asserts that seeds of length 64 and 65 produce different outputs. In Bitcoin Core the omission appears as a ternary that passes a null pointer, rather than the entropy buffer, on the first signing call.
+
+[^hmac]: [RFC 2104: HMAC: Keyed-Hashing for Message Authentication](https://www.rfc-editor.org/rfc/rfc2104), instantiated with SHA-256 as specified in [FIPS 180-4](https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf).