[Lightning-dev] Avoiding gossip spam: how many updates do you need?

[ZmnSCPxj]

Good morning Rusty,

As it happens, I already proposed a possible use-case for relatively-common `channel_update`: https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-July/002055.html
In the final section I mention:

> Suppose that in fact, YAijbOJA thinks that the capacity of the ZmnSCPxj<->YAijbOJA channel is too high on the YAijbOJA side.
> And similarly, suppose Rene thinks the capacity of the Rene<->YAijbOJA channel is too high on the Rene side.
>
> Thus, both YAijbOJA and Rene would welcome the ZmnSCPxj proposal to rebalance, as it moves the capacities.
> It may be that they are so welcoming of this proposal, that they are willing to waive the fee for the rebalance.
>
>I observe that many have already proposed "negative routing fees" in order to support rebalancing of their channels.
> I also observe that routing fees are the cost used in pathfinding algorithms, and most pathfinding algorithms do not behave well with negative costs.
>
> But it is perfectly fine to use ***zero*** routing fees, I think.

Briefly: if a channel has too much liquidity on your side, passively rebalance by broadcasting a `channel_update` with 0 routing fees.
This helps JIT-Routing of nearby nodes as it allows cheaper rebalances to support.

Of course, it is still desirable to rate-limit such updates.
So we can do the below policy:

1.  Maintain a "latest broadcast is zero" flag.
2.  If the channel has been >=75% in your favor for more than 10 minutes (or whatever configuration you want), and the "latest broadcast is zero" flag is cleared, set it and broadcast a 0-fee `channel_update` and set your feerate to 0.
3.  If the channel has been <75% in your favor, set your feerate to non-zero, but do not broadcast (meaning "latest broadcast is zero" flag does not change).
4.  If the channel has been <75% in your favor for more than 10 minutes, and the "latest broadcast is zero" flag is set, clear it and broacast your normal `channel_update`.

However this will probably still lead to more than a burst of 4 `channel_update`s per day.

Regards,
ZmnSCPxj


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, September 5, 2019 2:59 PM, Rusty Russell <rusty at rustcorp.com.au> wrote:

> Hi all,
>
> The next release of c-lightning will start suppressing gossip
> which comes too fast. I have implemented a simple filter which allows
> each message (node_announcement or channel_update) ONCE PER DAY on
> average, with a burst up to 4 times per day. We will also discard
> identical gossip messages (except timestamp and signature of course).
>
> If people have a reason to want more frequent updates, please tell me
> now! If this is tolerable, it will avoid some of the more obvious
> scaling issues in future.
>
> With 40,000 channels and 10,000 nodes, that caps us at about 90,000
> 12 MB per day. In practice much less.
>
> As an aside, we're implementing other gossip reductions:
>
> 1.  We'll start using gossip_queries for backfilling, rather than
>     asking three peers for a complete copy if we think we've missed
>     something.[1]
>
> 2.  We will no longer ask for the last 24 hours of gossip on startup.[1]
> 3.  We already suppressed our own duplicate gossip, and only generated
>     "disable" updates if we had to return an error.
>
> 4.  My current code has us never generating two channel_update or
>     node_announcement less than 300 seconds apart.
>
> 5.  We have a PR to suppress reply gossip: if you send us a msg, we won't
>     return it to you.
>
> 6.  My current code has us issuing "refresh" updates only once every
>     13 days instead of the current 7.
>
>     The main motivation is one user who has a rPi on a 4G plan. It's
>     painful for him to restart his node at the moment, but I am determined
>     to make it reasonable!
>
>     Cheers,
>     Rusty.
>     [1] Not yet coded, but RSN.
>
>
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev