[Lightning-dev] General question on routing difficulties

Pedro Moreno Sanchez pmorenos at purdue.edu
Tue Nov 21 15:37:48 UTC 2017


my name is Pedro Moreno-Sanchez and I am a PhD student at the computer
science department at Purdue. I would like to bring to your attention a
novel routing algorithm suitable for the Lightning Network (LN) that I
have been working on with my supervisor Prof. Aniket Kate (Purdue
University) and my co-workers Stefanie Roos and Prof. Ian Goldberg
(University of Waterloo).

Our approach is called SpeedyMurmurs, a routing algorithm for
decentralized payment networks such as the LN. SpeedyMurmurs uses an
embedding-based approach, meaning that the algorithm assigns meaningful
coordinates to nodes that enable efficient and effective discovery of
payment paths.  In a nutshell, SpeedyMurmurs creates a spanning tree by
means of a Breadth-First Search and then associates a coordinate to each
node depending on its position in the tree. A path from the sender to
the receiver is then calculated in a flexible manner, with each
intermediate node choosing the next node in the path as a function of
its neighbors' coordinates, available funds and closeness to the
receiver. To account for topology changes (e.g., a new channel is
created), the routing information is locally updated by only those
affected nodes in the network.

We have simulated several configurations of SpeedyMurmurs using real
data from the Ripple network and compared it with other routing
algorithms available in the literature. Our simulation results show that
SpeedyMurmurs is able to find paths at about twice faster, reduces the
communication overhead by at least a factor of 2 and maintains a similar
or higher payment success ratio. Our simulation framework is open source
and we believe that it might be of independent interest for this
community to test this and any other alternative protocols that you
might have in mind. If you are interested, we are happy to extend on this.

Finally, we also show that SpeedyMurmurs achieves the privacy notions of
interest in the LN. In particular, SpeedyMurmurs achieves value privacy,
i.e., the total value of a transaction remains hidden, as well as sender
and receiver privacy, i.e., the identities of the two transacting nodes
remain hidden from the adversary.

You can find all the details in the draft of our paper [1]. The final
version of this work will appear at NDSS 2018 conference [2]. We would
be glad to hear any question and feedback from you and are open to carry
out further collaborations if this line of work is of interest for you.

Best regards,
Pedro, Stef, Aniket and Ian.

[1] https://arxiv.org/abs/1709.05748
[2] https://www.ndss-symposium.org/

On 11/17/17 8:09 PM, Saravanan Vijayakumaran wrote:
> Hi Christian,
> Are there any open source simulators available for trying different
> routing strategies? Or even a simulator for the Lightning network as a
> whole?
> Regards
> sarva
> On Fri, Nov 17, 2017 at 8:00 PM, Christian Decker
> <decker.christian at gmail.com <mailto:decker.christian at gmail.com>> wrote:
>     Oh yeah, my mail tool destroyed that mail quite expertly :-)
>     The footnotes were 
>     [1]
>     https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md
>     <https://secure-web.cisco.com/1wkWGu7k6qIvsw8KxbTL8XXIpbTYjcgzOohVSUjpLJpNW0_r00YaFlguhX0pSCEAg2qU5kztXF4bxEpLbMz-RLAz9KTBvE0lh3kFGUjL5qke6yx8EcYvhHQQSttWjRX5HOt69vu8suXd7AhjEweVxeBFhvptINqjBarDx7woqCa14ZgWZdMk0dAt45Lnu_w1wjgn3j5sD7tBo187MGXR94eapimiMFjXySj70GeP1yiEA8rP0NUQ5CSXme2wQy-spVW_SLQpvkAQ01NlXUjK-ufQw_APez6C73Qx0bFh_9F-CPhKhhvM3tSs6IGNEM63aXMVeti2Ci0R5Xc15tvcT9gxpC32bNetja5ber6wbIHLbI9FWviQ63cWaNwhedQRN/https%3A%2F%2Fgithub.com%2Flightningnetwork%2Flightning-rfc%2Fblob%2Fmaster%2F07-routing-gossip.md>
>     [2]
>     https://medium.com/@rusty_lightning/lightning-routing-rough-background-dbac930abbad
>     <https://secure-web.cisco.com/1UG1OaEI1KFpQ4prjxrhN1Wsxs0P1Tco0MXQz0xltJZQjwI-sNi98eXMz-gW4qOQd2jJ4i0uktvL8CH-9RrmEg3GkxHfcxjnjxY_hlLP-ctXOYMSk4BFbySy7vD5xWivimHIfMHtr13ffgEoFLItUgoajxUe7tnkchPN_P5OZ_FOzYdpqW_UDdgWW0_VOsccR5yt1vh0MRyVxO2B2ua8k4NmbFgTmht6hxUlXDsXOsOSGDHm1WO5VNrRbUcGeTPpdBMx0xeyZ9FTMTBCIAMOZ6UEb_eAX3X1iVIFkP_MPtuJcp0q8t9Tk_UBs-dHVRjyYcCsnetXYNI_mEsdtyg63aLXuJE1pMLb8-eamWaFfklVo-w9N9F0XTbmbkgGfcWU4/https%3A%2F%2Fmedium.com%2F%40rusty_lightning%2Flightning-routing-rough-background-dbac930abbad>
>     [3]
>     https://www.tik.ee.ethz.ch/file/a20a865ce40d40c8f942cf206a7cba96/Scalable_Funding_Of_Blockchain_Micropayment_Networks%20(1).pdf
>     <https://secure-web.cisco.com/1XOlJdRo0gtzmPhNfVlEpMsrVBAI6BSjdDawPogEDIwYDdva2BSx4H8F9B3E6bkVIqA7ByFEF85qVjJ775leFwE54p5G6-wHH4Cio0p9sYLJ14-NHwcwvYQ--zdI8hdAyjGQbcLltVFAmorMaTlHq4FGI1CmxlwiUYgH1tjZn3UAHOu5xm5pLVi6KTb9WsJvuJsOBJhLfRLWGcAhVbjRXuV8b3x_G4ybOg1CQYC9ZVq3RJCPnNgQ3BN3a5ZuzW4veOE_dgi80FEy1x7a8spH-TV-cb_fey6ud-S25AWQ1PbctVS5zgQ4Ki4XYkR5igotNGGbWhACevBJfU1Hxqk90_g-DgQtS9_e_UX_FsY-yAjw/https%3A%2F%2Fwww.tik.ee.ethz.ch%2Ffile%2Fa20a865ce40d40c8f942cf206a7cba96%2FScalable_Funding_Of_Blockchain_Micropayment_Networks%2520%281%29.pdf>
>     We will eventually move away from the hash function based approach
>     in favor of something that allows us to decorrelate hops in a route.
>     We have indeed started writing down some of the ideas at least for
>     Lightning in the project's wiki [4], but they're definitely not
>     fleshed out.
>     [4] https://github.com/lightningnetwork/lightning-rfc/wiki/Brainstorming
>     <https://secure-web.cisco.com/1XqVZXht8sW8tdbLxwEMjoU-hxxV6OJwpKqc2OudZy26Le21yvOIBPwqizepwLi9TBeEV2BMDc-nCiKpj3eryi59jqvoZcBRrSQSVt9Qrq8pIxNSvhIMlG4cRd3lnj17JT9mDRRt0lS51C_9gpryV6qFqdiROdyJeTKfqUGmnvPo3isfeUbC_TOOfWLDV16jYA38ytCfTOryyDvJgJdYw7ArAUEMg10jNv8lV9aTARBOcOmgLjqJt0ktecsUpUCfIVAQlJEvtAAbWAUKwMoXg6MpIQEA4NE1ATntmwjLGl4IqQEqRAGxkGxWI8yQDL74yPPTIQmGTxs_JXE6YYMrhuD93GR1kyUJkOAH3Z_5nL1bsr3ifW31PJtUckQNnwY8e/https%3A%2F%2Fgithub.com%2Flightningnetwork%2Flightning-rfc%2Fwiki%2FBrainstorming>
>     On Fri, Nov 17, 2017 at 3:10 PM Benjamin Mord <ben at mord.io
>     <mailto:ben at mord.io>> wrote:
>         "I think this is exactly the right venue to discuss these kinds
>         of issue..." - you are probably right! My bad.
>         Christian, thank you for your knowledgable reply. The footnotes
>         did not come through on my end, I am especially interested in
>         [3]. Do you have a link? I am thrilled to hear of a
>         Bitcoin-compatible revive alternative! :)
>         Are we keeping an inventory somewhere of the cryptographic
>         primitives being used in lightning and the specific assumptions
>         being made about them (e.g. preimage resistance vs collision
>         resistance and such)? One project I have not yet found but
>         believe we need across the entire cryptocurrency community, is a
>         (wiki-style?) inventory of unproven mathematical assumptions
>         (e.g. hardness of discrete logarithm) and/or cryptographic
>         primitives, cataloged in terms of the cryptocurrency
>         technologies which require them. Such a resource could help the
>         community respond more quickly, comprehensively, and
>         transparently to the inevitable cryptanalytic surprises that
>         will pop up over time (especially from the quantum cryptanalytic
>         area, but even the classical cryptanalytic community as well).
>         Related, I believe the ideal end state would be to only assume
>         existence of a preimage-resistant hash function, and to code
>         such that one function could be quickly swapped with another and
>         thus update entire system. I'm not sure if that is a realistic
>         goal, but here is my first attempt to move in that direction in
>         case it is of interest to lightning. It is hard to imagine it
>         would be a new idea, although I have not yet found the precedent:
>         http://ben.mord.io/p/delayed-chained-key-revelation-dckr.html
>         <http://secure-web.cisco.com/1ewGQFfIxw1QZwneD3sDbhSSWP-YTmBwOYe529E7_zeYZZADnASbspvBAftPFXX6ZxJI2l2-8-xEqdpkmFg3fEIfkfRBYN8oZ8Z0HpeHh73MnT3Zi3M8GUs8SGMww38ZPnzsc7xlt7H5KFlLMcCsTWIgEtq4roZHDkYasanNeViP_UA3DIod7A281fNvWnQ1mnLs6d8WN_uFx1diU_xr-EoMab3wyANozirDj1gZ2_yPBn6S8tePYMkTkhIZlLz_BlvIfk9_KCOOiUcWxDTtCq9KdvoqEzStC04Z6q8xS5rtSnmK9GZZxv30yrgT8eWA4eOzce6AA3m0WEClNzbzANgi11GXBVde9pNiUIRcLDNw/http%3A%2F%2Fben.mord.io%2Fp%2Fdelayed-chained-key-revelation-dckr.html>
>         Thanks,
>         Ben
>         On Nov 17, 2017 8:04 AM, "Christian Decker"
>         <decker.christian at gmail.com <mailto:decker.christian at gmail.com>>
>         wrote:
>             On Thu, Nov 16, 2017 at 5:01 PM Benjamin Mord <ben at mord.io
>             <mailto:ben at mord.io>> wrote:
>                 Ivan,
>                 That is mostly false, but with bits of truth sprinkled
>                 in. Contact me at ben at mord.io <mailto:ben at mord.io> for
>                 further discussion so we tread lightly on the lists'
>                 email inboxes. 
>             I think this is exactly the right venue to discuss these
>             kinds of issue,
>             so please don't move the conversation somewhere else :-)
>             Routing is still very much in flux, we have a minimally
>             viable routing
>             protocol in the spec [1]. It is minimal in the sense that we
>             just push
>             the entire network's topology to the edges, which can then
>             locally
>             compute routes. This is effectively a source-routed network,
>             which
>             matches the requirements of the onion routing protocol we
>             use for
>             privact as well. But this does not mean that this is
>             protocol is set in
>             stone. We are actively working on finding better solutions
>             to the
>             problem of finding routes across a vast network of millions
>             if not
>             billions of nodes. Distance vector routing such as BGP uses
>             may be one
>             option like Ben suggested.
>             For now the network can easily scale to about 1 million
>             channels [2]
>             even on very limited devices, Upgrading to another protocol
>             at a later
>             point in time is trivial, since none of the routing
>             information is
>             consensus critical. We have all the extension points built
>             in to allow
>             future extensibility.
>                 But briefly: scale-capable routing protocols are
>                 possible as demonstrated by IP and thus by the internet
>                 itself. As for centralizing flow through small number of
>                 liquidity providers, yes that does seem economically
>                 probable, at least unless / until off-chain channel
>                 rebalancing mechanism (like the recently proposed
>                 "revive" protocol) come about. Bitcoin script is not
>                 currently revive-capable but Ethereum is, so either
>                 Bitcoin revive could be enabled via two-way pegged
>                 sidechain protocol with Ethereum, or even better, by a
>                 purpose-built (yet still not Turing-complete) extension
>                 to Bitcoin script itself in the future.
>             As a matter of fact, Conrad and I just published a similar
>             technique for
>             off-chain channel rebalancing and fund re-allocation based
>             solely on
>             Bitcoin [3] (major props to Conrad for the excellent
>             writeup!). The
>             flexibility in Bitcoin exists.
>             As for the hubs everybody is assuming will form, I don't
>             think they're
>             as likely to form. Creating such a hub is extremely costly
>             since it'll
>             have to allocate sufficient funds to cover the maximum
>             imbalance of all
>             of its channels ahead of time. Then the fees must cover the
>             opportunity
>             cost of allocating all of those funds to channels instead of
>             investing
>             them somewhere else. On top of that the funds will not be
>             moved alot
>             since they serve only a small number of endpoints connected
>             through
>             those channels, this compounds the problem of having high
>             fees. The high
>             fees make the hub channels a really bad choice for your
>             payments, after
>             all you were looking for small fees for your payments,
>             right? It opens
>             up an opportunity for nodes to open bypasses that grab some
>             of the
>             traffic and associated fees from the expensive hub.
>             All of that being said, we should be careful about our
>             predictions on
>             how the topology will look, I added some counter arguments to a
>             hub-and-spoke network forming, but nobody can really be sure
>             about
>             what'll happen.
>                 In either case the lightning network seems a key first
>                 step, and even were off-chain payment rebalancing not
>                 possible for some odd reason, the lightning network
>                 seems extremely valuable and scaleable - regardless
>                 because the centralization you speak is not one that
>                 affects safety of the money supply itself, and these
>                 centralized hubs would be more dispensable / swappable
>                 versus the mining centralization risk that people more
>                 often talk about in Bitcoin. Lightning network
>                 centralization, even if it persisted somehow despite
>                 revive and future concepts, would not be an existential
>                 risk.
>             Rebalancing is definitely possible, even without [3], you can
>             disincentivize the use of a channel until they have been
>             rebalanced. For
>             long term imbalance, opening another channel may be the best
>             option
>                 As for transaction fees, the idea is only channel setup
>                 / tear down are required greatly reducing fees. Yes if
>                 txin fees were millions of dollars then people could not
>                 practically penalize fraud, but that is unlikely. Even
>                 if txin fees made fraud claims marginally unprofitable
>                 (yet practical) that would still be ok - the judicial
>                 systems of most countries prove that people go beyond
>                 self-interest when sufficiently ticked, a fact of human
>                 psychology which in turn creates the incentives that
>                 support honest business. (Also please be aware I'm not a
>                 lightning code contributor, so that team might also be
>                 doing more to address already than I thought to mention
>                 above.)
>             This is open to speculation as well. We hope to reduce the
>             load on the
>             on-chain network sufficiently to allow timely on-chain
>             settlements. By
>             aggregating payments off-chain we can also aggregate the
>             fees and then
>             use them to pay on-chain fees. So don't consider the
>             on-chain fees for
>             your channels as your sole loss, they are paid for by
>             payments you
>             forward. Ultimately this should encourage participants to
>             open channels
>             that support the network as a whole, not just themselves. We are
>             building automations that should take care of this, the user
>             won't have
>             to do anything to improve the network topology.
>             Cheers,
>             Christian
>     _______________________________________________
>     Lightning-dev mailing list
>     Lightning-dev at lists.linuxfoundation.org
>     <mailto:Lightning-dev at lists.linuxfoundation.org>
>     https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>     <https://secure-web.cisco.com/1rKvOL6YDFMOjqgsWHIPJzPSwJMPOfbibptuquR_1TsVdIQrLf1NxCVE-s5WE1ZYrdP24xdimGxHK9mO5lyt539Tkw2lzYa9rwS9--thSiY_hjklJDMsHrV19nNF8fOU35MvLMeISx2brsJSH0K4MNQpctvdWWImDsvS4rwweaXJRgtUfd1uSxbqCGdjJ7KTTOlWCdUehIfdcJtlBoiUGIy2yKJMqn7uLJT4tXQQWbGZAou9uWgwGJmHnUk4oW05-otW_6u-nrcT1jhnhPGcP6x6_k8EMv1PaCs5w0baDgHhZ09GvGpSFr9Wmk8cd69ob8sEcl6xXHuFHTt1GHMh6KfMOc11pAXgM6FsY6B35bjHVVOw0DiIz7n_a8w-RLxc-/https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Flightning-dev>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://secure-web.cisco.com/1rKvOL6YDFMOjqgsWHIPJzPSwJMPOfbibptuquR_1TsVdIQrLf1NxCVE-s5WE1ZYrdP24xdimGxHK9mO5lyt539Tkw2lzYa9rwS9--thSiY_hjklJDMsHrV19nNF8fOU35MvLMeISx2brsJSH0K4MNQpctvdWWImDsvS4rwweaXJRgtUfd1uSxbqCGdjJ7KTTOlWCdUehIfdcJtlBoiUGIy2yKJMqn7uLJT4tXQQWbGZAou9uWgwGJmHnUk4oW05-otW_6u-nrcT1jhnhPGcP6x6_k8EMv1PaCs5w0baDgHhZ09GvGpSFr9Wmk8cd69ob8sEcl6xXHuFHTt1GHMh6KfMOc11pAXgM6FsY6B35bjHVVOw0DiIz7n_a8w-RLxc-/https%3A%2F%2Flists.linuxfoundation.org%2Fmailman%2Flistinfo%2Flightning-dev

More information about the Lightning-dev mailing list