[Lightning-dev] Quick analysis of channel_update data

[Fabrice Drouin]

Additional info on channel_update traffic:

Comparing daily backups of routing tables over the last 2 weeks shows
that nearly all channels get at least a new update every day. This
means that channel_update traffic is not primarily cause by nodes
publishing new updates when channel are about to become stale:
otherwise we would see 1/14th of our channels getting a new update on
the first day, then another 1/14th on the second day and so on.This is
confirmed by comparing routing table backups over a single day: nearly
all channels were updated, one average once, with an update that
almost always does not include new information.

It could be caused by "flapping" channels, probably because the hosts
that are hosting them are not reliable (as in is often offline).

Heuristics can be used to improve traffic but it's orhtogonal to the
problem of improving our current sync protocol.
Also, these heuristics would probaly be used to close channels to
unreliable nodes instead of filtering/delaying publishing updates for
them.

Finally, this is not just obsessing over bandwidth (though bandwidth
is a real issue for most mobile users). I'm also over obsessing over
startup time and payment UX :), because they do matter a lot for
mobile users, and would like to push the current gossip design as far
as it can go. I also think that we'll face the same issue when
designing inventory messages for channel_update messages.

Cheers,

Fabrice



On Wed, 9 Jan 2019 at 00:44, Rusty Russell <rusty at rustcorp.com.au> wrote:
>
> Fabrice Drouin <fabrice.drouin at acinq.fr> writes:
> > I think there may even be a simpler case where not replacing updates
> > will result in nodes not knowing that a channel has been re-enabled:
> > suppose you got 3 updates U1, U2, U3 for the same channel, U2 disables
> > it, U3 enables it again and is the same as U1. If you discard it and
> > just keep U1, and your peer has U2, how will you tell them that the
> > channel has been enabled again ? Unless "discard" here means keep the
> > update but don't broadcast it ?
>
> This can only happen if you happen to lose connection to the peer(s)
> which sent U2 before it sends U3.
>
> Again, this corner case penalizes flapping channels.  If we also
> ratelimit our own enables to 1 per 120 seconds, you won't hit this case?
>
> > But then there's a risk that nodes would discard channels as stale
> > because they don't get new updates when they reconnect.
>
> You need to accept redundant updates after 1 week, I think.
>
> Cheers,
> Rusty.