[Lightning-dev] Routemap scaling (was: Just in Time Routing (JIT-Routing) and a channel rebalancing heuristic as an add on for improved routing success in BOLT 1.0)

[ZmnSCPxj]

Good morning Ariel,

I am forking this thread as my reply is not at all related to the JIT-Routing.


Sent with ProtonMail Secure Email.

> Public nodes could advertise channels which don't actually exist directly but are instead hidden paths that the source node doesn't need to know or care about. The fees advertised for these aggregate-channels would be the aggregate fees expected from the sub-route.

Nonexistent channels (i.e. channels that do not have some backing UTXO in the Bitcoin blockchain) are not safe to propagate on the network since they are trivially spammable (i.e. can generate a large number of fake channels to waste network bandwidth).

> I think the biggest gain from this system is that the network can be more abstract. This abstraction allows all possible subsets of public nodes to be a clique since all subsets of nodes can be maximally connected with aggregate-channels as long as the entire network is well connected.
> This new property of the network could allow a source node to select a random privacy-clique and rely on payments to be routed along aggregate-channels without the source node needing to compute or even know the exact sub-routes. Futhermore, the source node could exclusively download and listen to the privacy-clique and ignore the rest of the network structure thus reducing the burden of keeping up to date network information.

Let me suggest something else.

As the LN grows, the public routemap becomes larger and larger, until keeping them in a fast in-memory data structure becomes infeasible on cheap hardware.
In all likelihood, at some point in the future, users will want to be able to limit the memory consumed by implementations for routemaps.

So, some pruning heuristic is needed, to reduce the resource usage of large routemaps.

A good pruning heuristic is "channel capacity", which can be checked onchain (the value of the UTXO backing the channel is the channel capacity).
It is good to keep channels with large capacity in the routemap, because such large channels are more likely to successfully route a payment than smaller channels.
So it is reasonable to delete channels with low capacity when the routemap memory is becoming close to full.

It seems to me that s/aggregate-channel/high-capacity-channel/g in your idea would still work.
In effect, the high-capacity channel is very likely to successfully route in either direction.
But if by chance it falls into a state where it is unable to route in one direction or other, the intermediate node has other, lower-capacity channels that it can use JIT-Routing with to improve the directionality of the high-capacity channel.
Nothing in the JIT-Routing idea requires that the rebalancing loop is small, only that a loop exists.

Nodes still need to track their direct channels (so they are implicitly always in the routemap).
But payee nodes making BOLT1 invoices could also provide `r` routes in the invoice, with the given routes being from nodes with high-capacity channels, so that even if the intermediate channels are pruned due to low capacity, it is possible to get paid.

Regards,
ZmnSCPxj