BIP442: OP_PAIRCOMMIT

[moonsettler]

OP_PAIRCOMMIT is the newest member of the LNhance family of opcodes. It provides limited vector commitment functionality in tapscript.

When evaluated, the OP_PAIRCOMMIT instruction:

• pops the top two values off the stack,
• takes the "PairCommit" tagged SHA256 hash of the stack elements,
• pushes the resulting commitment on the top of the stack.

Discussion: https://delvingbitcoin.org/t/op-paircommit-as-a-candidate-for-addition-to-lnhance/1216/12

Mail list proposal: https://groups.google.com/g/bitcoindev/c/si6ZNIkVfOw/m/29VY_YRrCgAJ

[murchandamus]

This document has a few formatting issues, please make sure that the preamble matches the BIP 2 requirements and take a look at the rich diff to see whether it looks the way you intend.

Please note that the BIPs repository also accepts markdown files.

[moonsettler]

Switched back to markdown. Header now in BIP-2 format.

[moonsettler]

The original create date of OP_PAIRCOMMIT is 2024-03-15 this is the latest revision based on feedback from Anthony Towns.
https://gist.github.com/moonsettler/d7f1fb88e3e54ee7ecb6d69ff126433b/revisions
What date should go to the header?

[jonatack]

Added a discussion link to the PR description.

[murchandamus]

According to BIP 2:

The Created header records the date that the BIP was assigned a number, […]

[murchandamus]

Has this proposal been sent to the mailing list?

[moonsettler]

Has this proposal been sent to the mailing list?

Not yet. Wanted to get it into an acceptable shape before I post it there.

Proposed to the mailing list, waiting for feedback.

[murchandamus]

I would like to see this proposal to get more review from other covenant researchers before it moves forward.

[moonsettler]

It looks like we gonna have to amend the PAIRCOMMIT BIP with some new use cases.
Turns out within certain practical limitations any computational function can be proven out in the form of a merkle tree.
The root hash of the merkle tree represents the function the leaves represent the inputs and output.
Any 32 bit arithmetic function can certainly be proven out with this method.
CAT itself with a limited set of inputs or limited input sizes can be proven out.
At this point it's an open question if this enables new behaviors not enabled by taproot MAST itself?

Special thanks to: @JeremyRubin @Ademan @bigspider

edit:
Alternatively could consider imposing specific script limits that make PAIRCOMMIT explicitly less capable than MAST itself.

[Ademan]

I think I've changed my mind a bit. We were talking about computing a merkle tree for f(u32,u32) as if it was trivial but after a quick experiment it seems like that would take hundreds of years to compute (am I being dumb here?) Instead, you can compute mul(u32,u32) -> u32 using 3 mul(u16,u16)s which is feasible to compute. The witness size is worse, ~32 * 32 * 3 = 3072 instead of 32 * 64 * 1 = 2048, but computing the tree for mul(u16,u16) is feasible using a naive algorithm on commodity hardware.

The implication of this is that where a function can be decomposed into operations on smaller inputs, PAIRCOMMIT is massively more feasible to use than encoding things into a tap tree.

[bigspider]

I think I've changed my mind a bit. We were talking about computing a merkle tree for f(u32,u32) as if it was trivial but after a quick experiment it seems like that would take hundreds of years to compute (am I being dumb here?) Instead, you can compute mul(u32,u32) -> u32 using 3 mul(u16,u16)s which is feasible to compute. The witness size is worse, ~32 * 32 * 3 = 3072 instead of 32 * 64 * 1 = 2048, but computing the tree for mul(u16,u16) is feasible using a naive algorithm on commodity hardware.

Arithmetic and bitwise operations where inputs & outputs are small enough, can already be done in Script in cheaper ways. Merkle trees as lookup tables are only interesting for functions that are either extremely complex, or where preimages/images are larger than what Script can work with.
Note that you can already do small indexed lookup tables more efficiently by just hard-coding them in Script (that is: push the table on the stack and use OP_PICK to read its entries), and these techniques are widely used (e.g. in BitVM).

The implication of this is that where a function can be decomposed into operations on smaller inputs, PAIRCOMMIT is massively more feasible to use than encoding things into a tap tree.

I think the only substantial difference is that in a Script where you need several lookups, you can do it with Merkle trees, while you can only do a single lookup with a precomputed taptree.

[moonsettler]

Proving general computation

Merkle trees can be used to prove out computation where the root of the tree
represents the function and the leaves represent the inputs and output.
There are practical limits to the entropy space for the inputs as it needs
to be iterated over and hashed up.

Currently MAST trees can cover 128 bits of entropy space, which is well over
the practical limits to iterate over and merklize. Therefore we assume this
capability does not materially extend what computations are possible to prove
out in bitcoin script. While OP_PAIRCOMMIT is not limited to a height of 128,
that should not be practically feasible to utilize.

There is a way to reduce the size of the witness for proving out computation,
by eliminating the merkle path inclusion proofs, using OP_CHECKSIGFROMSTACK
together with OP_PAIRCOMMIT. This method involves deleted key assumptions,
most likely using MPC to create an enormous amount of signatures for the stack
elements representing the inputs and the output of the function.

Is this correct? Any suggestions? @Ademan @bigspider

[moonsettler]

The implication of this is that where a function can be decomposed into operations on smaller inputs, PAIRCOMMIT is massively more feasible to use than encoding things into a tap tree.

This is the main open question I believe. does it or does it not practically expand what we can already do?
For example using PC to emulate smolCAT and using traditional methods with lookup tables could make 32 bit or even 64 bit arithmetics more feasible?

edit:
Within the 32 bit realm we can already use OP_ADD, I see little practical diff between <0x1234> <0x5678> CAT and <0x12340000> <0x5678> ADD.
And it sounds like 64 bit smolCAT would be way too expensive to generate (and also to interact with trustlessly).

(actually the above examples are wrong, because internally bitcoin script uses little endian, but should convey the point)

[Ademan]

...

Arithmetic and bitwise operations where inputs & outputs are small enough, can already be done in Script in cheaper ways. Merkle trees as lookup tables are only interesting for functions that are either extremely complex, or where preimages/images are larger than what Script can work with. Note that you can already do small indexed lookup tables more efficiently by just hard-coding them in Script (that is: push the table on the stack and use OP_PICK to read its entries), and these techniques are widely used (e.g. in BitVM).

Even u16,u16 is quite a bit larger than I think is practical as a lookup table, but the efficiency for repeated operations is constant, obviously. The lookup table is less efficient for small numbers of operations (a u8,u8 table is 16k vs 1 u8,u8 proof is 0.4k) but the merkle tree loses quickly when those operations are repeated.

The implication of this is that where a function can be decomposed into operations on smaller inputs, PAIRCOMMIT is massively more feasible to use than encoding things into a tap tree.

I think the only substantial difference is that in a Script where you need several lookups, you can do it with Merkle trees, while you can only do a single lookup with a precomputed taptree.

Right, and the key point is these merkle trees and lookup tables rapidly become infeasible to compute as the input size grows, so multiple smaller lookups is significantly more useful.

EDIT: But your point is well taken that for smaller operations they can already be better accomplished by lookup tables.

[Ademan]

...
edit: Within the 32 bit realm we can already use OP_ADD, I see little practical diff between <0x1234> <0x5678> CAT and <0x12340000> <0x5678> ADD. And it sounds like 64 bit smolCAT would be way too expensive to generate (and also to interact with trustlessly).

(actually the above examples are wrong, because internally bitcoin script uses little endian, but should convey the point)

Yeah for arbitrary 8 byte strings smolCAT seems infeasible to compute the table or merkle tree for. After a bit of conversation on IRC it could probably be feasible for arbitrary f(b[4],b[4]) -> b[8] with a custom ASIC¹ or maybe a cluster of FPGAs in a span of ~a few years but that would not be very useful for the average person.

Bit shifts over 32 bit integers seems pretty feasible though, that's f(u32,u6)->u32 (maybe save some space by special casing shift = 0). it seems like my incredibly naive, unoptimized, single-core experiment could calculate that merkle tree in ~96 hours. Of course the proof is ~1.2k and users would likely need multiple, but the lookup table for that wouldn't fit in a block anyway so maybe something new is possible?

You can also separate positive and negative shifts, and maybe break it down into multiple rounds of shifts 1-3 or something (or 1k for a proof for a constant shift)

[1]: afaik existing ASICs operate on block headers so couldn't help

[murchandamus]

Looks good to me. This reads much better than earlier versions of the document, thanks for the improvements! My comments could be addressed in a follow-up, unless you prefer to address them here in this PR.

[murchandamus]

If you take my suggestion from above, it’s no longer necessary to explain most column titles, and you can only explain the ones that need more description (feel free to adjust the wording, of course):

Suggested change

	*ChannelSc: channel script, UpdateSc: update script, UpdateW: witness size for Force Close and Contest, ForceC: cost of unilateral close, Contest: additional cost to contest, Settle: signature or CTV required for settlement.*
	¹ *witness is the same weight for both Force Close and Contest in LN-Symmetry*
	² *total cost of unilateral close transactions*

[jonatack]

"Re-work" commit e8672fc contains changes that I find less human and approachable (parts of it feel more corporate-speak or AI-generated). I propose doing a human editing review to retain the best of before/after -- am willing to go over that with you on a call this week, if you like, or submit a diff.

Once the content is stable, it would be good to see a few experienced reviews.

[jonatack]

Why was that PR closed? Suggest providing context on the status of that PR or removing it here.

[moonsettler]

Ahh, the PR was closed to not confuse anyone about the state of the LNhance repository.
When I restructured the changes regarding the 4 opcode package for better reviewability (script changes [consensus], policy and wallet changes, deployment changes [consensus], tests) it suddenly didn't fit anywhere, while originally it was an addition on top of the old 3 opcode LNhance branch.

[murchandamus]

This proposal has been sitting for a couple months about an hour work away from being published as Draft. Any chance you could take a stab at getting it ready to be merged?
I’m sure Jon would still be up to go over it with you as he offered above. If you’d rather address the proposed changes in a follow-up, the minimal changes necessary to get this ready for Draft could be done in a couple clicks.

[murchandamus]

@moonsettler, @brandonblack, @jonatack: As stated above, this is essentially ready to be published except for a few minor formatting fixes and the planned small tweaks. I can make the changes to make this document BIP 3 compliant, if you prefer to address your tweaks in a follow-up PR, but I would need an indication that this is what you want.

[moonsettler]

Should I rebase to resolve conflicts?

[murchandamus]

Should I rebase to resolve conflicts?

Yes please!

[murchandamus]

@jonatack wrote:

"Re-work" commit e8672fc contains changes that I find less human and approachable (parts of it feel more corporate-speak or AI-generated). I propose doing a human editing review to retain the best of before/after -- am willing to go over that with you on a call this week, if you like, or submit a diff.

Is this still planned? Would you be open to doing that for a follow-up PR?

@moonsettler: Would I be right that there is no outstanding planned work from your side at this time?

[moonsettler]

@jonatack wrote:

"Re-work" commit e8672fc contains changes that I find less human and approachable (parts of it feel more corporate-speak or AI-generated). I propose doing a human editing review to retain the best of before/after -- am willing to go over that with you on a call this week, if you like, or submit a diff.

Is this still planned? Would you be open to doing that for a follow-up PR?

@moonsettler: Would I be right that there is no outstanding planned work from your side at this time?

I somewhat sympathize with the aversions of @jonatack, but I also find find it hard to care as much as he does. I hoped to be in a better mental state by now than I was at the end of last year, but I'm still not back. Low on energy and distracted by other stuff. If Jon thinks it's important, I will defer to him on this matter and try to cooperate.

[murchandamus]

I’m gonna give @jonatack a couple days to respond and revisit next week.