[Lightning-dev] Fee-free rebalancing to support JIT-routing
ZmnSCPxj at protonmail.com
Wed Jul 3 06:09:31 UTC 2019
Good morning list,
As it happens, I was considering about JIT-routing by Rene Pickhardt.
And I notice that Rene has been proposing about a "fee-free rebalance" in order to better support JIT-routing.
And I have been thinking about this "fee-free rebalance" proposal.
As a review, JIT-routing allows a sort of "semi-local multipart payment".
The intuition is, that a forwarding node has better information about local channel balances, than the source node has information about remote channel balances along the route.
What JIT-routing means, more specifically, is to perform channel rebalance operations "just-in-time" for a forwarding request.
The forwarding node, knows exactly the balances in its channels, and thus can determine how best to deliver the required funds to the next hop in the onion.
So I considered, the simplest case of useful JIT-routing.
Specifically, suppose that there exist three nodes on the network, forming a tiny cyclic superhub of three members.
Let us call them ZmnSCPxj, YAijbOJA, and Rene.
Let us suppose that they all have channels to each other, of total capacity 0.1 BTC, and with each channel perfectly balanced at 0.05 BTC to each side.
Let us suppose there is a channel reserve of 0.01 BTC for each channel.
Suppose ZmnSCPxj receives a forwarding request, with the next hop being 0.06BTC to YAijbOJA.
As the useable capacity (minus channel reserve) is only 0.04BTC, ZmnSCPxj cannot facilitate this forwarding request.
Under JIT-Routing, what ZmnSCPxj does, is to route 0.02 BTC from ZmnSCPxj->Rene->YAijbOJA->ZmnSCPxj, i.e. a rebalance of capacity from the ZmnSCPxj<->Rene channel, to the ZmnSCPxj<->YAijbOJA channel.
But under the current network, this would require a fee for this rebalancing attempt.
Now, ZmnSCPxj could argue with YAijbOJA and show the forwarding request of 0.06 BTC to YAijbOJA.
As YAijbOJA also knows the current state of the ZmnSCPxj<->YAijbOJA channel, it knows this forwarding request cannot push through currently, unless the rebalance completes.
This is sufficient to convince YAijbOJA to waive its fee for transporting from the Rene<->YAijbOJA channel to the ZmnSCPxj<->YAijbOJA channel, since otherwise it would not gain the fee (or final payment if it is the payment termination point) for the ZmnSCPxj->YAijbOJA forward.
However, Rene cannot be so convinced.
After all, both ZmnSCPxj and YAijbOJA are strangely-named nodes, unlike Rene whose name is actually pronounceable.
In particular, the forwarding packet cannot even be read by Rene; and in any case, why should Rene waive the fee when it cannot benefit by doing so?
YAijbOJA can benefit since by enabling the forward request from ZmnSCPxj->YAijbOJA, it could gain an even larger fee from forwarding onwards (assuming the same base fee, and a proportional fee, the larger forwarding from ZmnSCPxj->YAijbOJA of 0.06 BTC will lead to a larger proportional fee compared to the YAijbOJA->ZmnSCPxj forward of 0.02 BTC).
Thus, it seems to me that we can argue for a fee-free forwarding, but only for the last hop in the rebalance.
Now, in particular, note that YAijbOJA should only be willing to waive the fee, if ZmnSCPxj will actually hand it a 0.06 BTC HTLC for ZmnSCPxj->YAijbOJA, in exchange for resolving the 0.02 BTC HTLC for YAijbOJA->ZmnSCPxj.
So I propose the following constructions below for this.
We need an HTLC-dependent HTLC construction.
What this means is, that YAijbOJA will offer a construction, which requires that ZmnSCPxj reveals a preimage.
Onchain, when ZmnSCPxj reveals this preimage, it is forced to claim this into an HTLC, which represents the original forwarding attempt from ZmnSCPxj->YAijbOJA.
This is needed to assure YAijbOJA that it will have an opportunity to earn fees later if it waives the fee for forwarding to ZmnSCPxj first.
So let us be more precise, and say:
orig_preimage = preimage for the original 0.06BTC forwarding from ZmnSCPxj->YAijbOJA.
orig_hash = h(orig_preimage)
rebal_preimage = preimage for the rebalance from ZmnSCPxj->Rene->YAijbOJA->ZmnSCPxj.
rebal_hash = h(rebal_preimage)
1. When the rebalance onion reaches YAijbOJA from Rene, YAijbOJA opens the onion packet.
Included in this packet is a short note from ZmnSCPxj explaining that there is an "original onion" that would forward 0.06 BTC to YAijbOJA, but given the current channel state, ZmnSCPxj cannot forward it unless this rebalance pushes through, so can YAijbOJA waive its fee?
2. Then, YAijbOJA requests for the original onion from ZmnSCPxj.
This is safe for ZmnSCPxj to send, since there is no HTLC from ZmnSCPxj->YAijbOJA yet.
YAijbOJA is not incentivized to forward this yet since it has no incoming HTLC, meaning it would lose money if it forwarded it immediately.
3. YAijbOJA validates the original onion.
If it decrypts correctly, and forwards with sufficient fee from ZmnSCPxj to YAijbOJA so that it beats the fee it is being asked to waive, or if YAijbOJA is the final payee, then it allows to continue the protocol.
4. YAijbOJA and ZmnSCPxj agree to create a 0.06 BTC output on both commitment transactions to the below SCRIPT.
0.02 BTC is gotten from YAijbOJA main output, and 0.04 BTC from ZmnSCPxj main output, to funds this contract.
OP_DUP OP_HASH160 <RIPEMD(SHA256(revocationpubkey))> OP_EQUAL
OP_SIZE 32 OP_EQUAL
# hash branch
OP_HASH160 <RIPEMD(rebal_hash)> OP_EQUALVERIFY
# timelock branch
<cltv_expiry> OP_CHECKLOCKTIMEVERIFY OP_DROP
2 <ZmnSCPxj_htlcpubkey> <YAijbOJA_htlcpubkey> 2 OP_CHECKMULTISIG
5. Note that the commitment transactions **cannot** be signed **yet**!
Both YAijbOJA first need to exchange signatures for two special transactions, HTLC-waivefee-timeout and HTLC-waivefee-success.
6. HTLC-waivefee-timeout takes the above output and spends it using the timelock branch.
Its `nLockTime` is thus equal to `cltv_expiry`.
Its witness is `<ZmnSCPxj_waivefee_timeout_signature> <YAijbOJA_waivefee_timeout_signature> 0`.
There are two versions, depending on which commitment transaction (that of ZmnSCPxj, or that of YAijbOJA) is being spent from.
Signatures for both versions must be exchanged.
It has two outputs, 0.04 going to ZmnSCPxj (revocable if from ZmnSCPxj commitment) and 0.02 going to YAijbOJA (revocable if from YAijbOJA commitment).
7. HTLC-waivefee-success takes te above output and spends it using the hash branch.
Its witness is `<ZmnSCPxj_waivefee_success_signature> <YAijbOJA_waivefee_success_signature> <rebal_preimage>`.
Again two version depending on which commitment transaction is spent.
Signatures for both versions must be exchanged.
It pays out to a single output, with script "Offered HTLC Outputs" / "Received HTLC Outputs" in existing BOLT#3, using the `orig_hash` as the `payment_hash`.
8. After exchanging the signatures for the above transactions, ZmnSCPxj and YAijbOJA can now exchange signatures for the commitment transactions as usual using `commitment_signed`, then `revoke_and_ack`.
9. When ZmnSCPxj fulfills the rebalancing HTLC, this puts the `orig_hash` HTLCs directly into the commitment transactions.
This lets YAijbOJA claim the rebaancing HTLC from Rene, and forward normally onwards.
If the `orig_hash` payment is later failed, then the entire 0.06 BTC amount is returned to ZmnSCPxj, since it has already transferred 0.02 BTC from its ZmnSCPxj<->Rene channel.
Another observation I make, is the consideration of the use of Channel Factories.
In particular, JIT-routing can instead use a factory-level operation to reorganize channel funds.
ZmnSCPxj can request a factory channel reorganization to move some funds from the ZmnSCPxj<->Rene channel to the ZmnSCPxj<->YAijbOJA channel.
This has the same effect, i.e. it allows a forwarding attempt to push through, that would not be possible without the factory-level channel reorganization.
Further, assuming only ZmnSCPxj, YAijbOJA, and Rene are in the channel factory, then it is the same: all three need to be online in order for the JIT-routing to work.
But I observed above that, in a channel rebalance using current channels (without factories) Rene cannot be convinced to waive the fee.
This points to the next observation:
1. Channel rebalances really should be free, as we might imagine channel factory reorganizations to be.
2. Factory-level channel reorganizations should charge a fee, paid by nodes that want to remove capacity, to the nodes whose channel is reduced by the removed capacity.
i.e. in the Channel Factory case, the factory-level channel reorganization should make Rene demand a fee from ZmnSCPxj in exchange for agreeing to the reorganization, because of the loss of capacity in the ZmnSCPxj<->Rene channel.
I suspect the second is true: the reduced capacity in the ZmnSCPxj<->Rene channel means that ZmnSCPxj is less likely to successfully route to Rene, due to the reduce capacity to Rene.
Thus, Rene may be disincentivized to allow the transfer of capacity *away* from ZmnSCPxj<->Rene channel without recompense.
Another thought is the below.
Suppose that in fact, YAijbOJA thinks that the capacity of the ZmnSCPxj<->YAijbOJA channel is too high on the YAijbOJA side.
And similarly, suppose Rene thinks the capacity of the Rene<->YAijbOJA channel is too high on the Rene side.
Thus, both YAijbOJA and Rene would welcome the ZmnSCPxj proposal to rebalance, as it moves the capacities.
It may be that they are so welcoming of this proposal, that they are willing to waive the fee for the rebalance.
I observe that many have already proposed "negative routing fees" in order to support rebalancing of their channels.
I also observe that routing fees are the cost used in pathfinding algorithms, and most pathfinding algorithms do not behave well with negative costs.
But it is perfectly fine to use ***zero*** routing fees, I think.
For those pathfinding algorithms that require nonzero cost, it is often easy to add a very tiny minimal cost to edges that have 0 cost.
Indeed, this is often practical to add a tiny cost to every edge traversed, whether the edge is nominally 0-cost or not.
For example, C-Lightning does this, since routes with the same fees are not equal if one route has more nodes --- more nodes are less likely to succeed in routing.
So our software today, should in practice already be quite fine with handling 0 routing fees, if the node wishes to rebalance its channel.
I also observe, from a skim of BOLT #7, the spec does ***NOT*** have any verbiage to the effect "`fee_base_msat` MUST be non-zero" or "`fee_proportional_millionths` MUST be non-zero".
Thus our spec implicitly allows, by not specifying otherwise, 0 routing fees, already, today.
Thus I think we can fix multiple problems with one solution ---
* Instead of ***negative*** routing fees, use ***zero*** routing fees if a channel has too much capacity on our side.
* Such ***zero*** routing fees also implicitly implement fee-free rebalancing, to support JIT-routing.
This requires ***no spec change***, which is a tremendously good property that JIT-routing has.
More information about the Lightning-dev