[Lightning-dev] proposal for Lightning Network improvement proposals

Olaoluwa Osuntokun laolu32 at gmail.com
Mon Jul 23 19:41:36 UTC 2018


> My concerns are mainly about assignment of values for types and feature
> bits, and how we can test new features without conflicting with other
> developers.

There're enough feature bits that one can likely just pick a random high
number, and start to roll out their experiment in the wild. If a collision
_does_ happen, then your new feature/protocol will likely detect the
incompatibility (example invalid message from your PoV), disconnect, and
move on. Local feature bits of course, are much more lax than say, global
feature bits.

> For example, looking at the current BOLT#9, it appears bits 8/9 are the
next
> free bits available for use,

There's no strict requirement that feature bits be assigned in order.
They've been assigned in this order so far, as there really haven't been
many new additions, or experiments in the wild.

> however, I find that other developers have also opted to use the same
bits:
> https://github.com/rgb-org/spec/blob/master/04-lightning-network.md

A simple courtesy to other developers would simply be to announce on the
list that one is running an experiment with a new feature bit (bit 1493 or
w/e) such that collisions in the wild are less likely. The authors of "RGB"
to my knowledge have never posted on the list to "claim" their experimental
feature bit. Also looking at the spec, their LN integration is more or less
a hand wave over the actual implementations details, so really just a high
level design sketch.

-- Laolu


On Mon, Jul 23, 2018 at 12:15 PM Mr Hodlen <hodlen at outlook.com> wrote:

> Hi, new guy here. I've also been a bit confused about the process from
> just
> reading the BOLTs.
>
> I'm working on an implementation of the BOLTs with the intention of
> experimenting with new features.
> My concerns are mainly about assignment of values for types and feature
> bits,
> and how we can test new features without conflicting with other
> developers.
>
> For example, looking at the current BOLT#9, it appears bits 8/9 are the
> next
> free bits available for use, so if I were implementing a new optional
> feature,
> I would pick the next available bits,
> however, I find that other developers have also opted to use the same
> bits:
> https://github.com/rgb-org/spec/blob/master/04-lightning-network.md
>
> I think issue of requiring centralized number assignment can be easily
> avoided
> for experimental features by using a unique identifier for the feature
> instead,
> such as a hash of the feature name.
> This should make it simpler for independent developers to try new features
> before creating new pull requests for a BOLT assignment.
>
> Here's an extremely rough draft of the idea. The intent is that once an
> experiment has been tested and is succesful, it can move towards being
> standardized as a BOLT. Is it worth adding something like this?
>
> --- BOLT #(-1): Experimental Features  [tentatively named]
>
> Overview
>
> The experimental features protocol is intended for testing protocol
> features
> which are not necessarily intended to become standard BOLTs.
>
> * Lightning base protocol messages:
>
> ** experiment
>
> A message for arbitrary protocol implementaion with a unique ID for each
> feature supported.
>
>   type: <??>
>   data:
>     [32: experiment_name_hash]
>     [2: payloadlen]
>     [payloadlen: payload]
>
> `experiment_name_hash` is be a SHA256 of the feature name, in ASCII,
> concatenated with a good source of randomness to avoid any conflict if any
> two
> developers chose the same name.
>
> `payloadlen` is the length of the remaining message, which MUST equal the
> length of the outer message -34 bytes.
>
> `payload` is the payload of the experiment message, which should have the
> same
> format as a regular Lightning message described in BOLT #1.
>
> A sending node:
>     - MUST send the `init_experiments` message with `experiment_name_hash`
> set
> to 0 before any other `experiment` message is sent.
>     - MUST NOT send an experiment message with any non-zero
> `experiment_name_hash` if the receiver has not already declared support
> for
> the feature_hash.
>
> A receiving node:
>     - If experiment_name_hash is zero:
>         - Treat the payload as the same format of lightning message, ie, a
> 16-
> bit big endian experiment_type followed by the data.
>     - If the experiment_name_hash is not recognized or disabled, MUST
> ignore
> the message. (SHOULD also fail the channels?)
>
> Rationale:
>
> The base lightning layer allows extension through feature bits and new odd
> message types, but these require centralized assignment by the Lightning
> community.
> This creates a chicken and egg problem for developers testing experimental
> features: In order to be assigned numbers, we need to standarize our
> feature
> through the BOLT process,
> but we might not wish to attempt standardization until our experimental
> features have been tested.
> Without standardization, different teams working on different experimental
> features may cause conflicts if different clients are treating feature
> bits or
> message types differently.
>
> By moving all experimental features to a new message where they are
> wrapped in
> a unique feature name, this eradicates chance of conflicting
> implementations.
>
> Additionally, this message can serve as a generic transport mechanism
> between
> any two lightning nodes who have agreed to support the
> expierment_name_hash,
> as there is no restriction on the format of the payload. This may make it
> possible to serve e.g: HTTP over Lightning.
>
>
> * General experiment messages:
>
> If `experiment_name_hash` in the experiment message is 0, treat its
> payload as
> on of the following messages:
>
> ** init_experiments message
>
> Informs a peer of features supported by the client.
>
>   1. experiment_type: 16
>   2. data:
>       * [2: eflen]
>       * [eflen*32: experiment_name_hashes]
>
> A sending node:
>    * MUST send the `init_experiments` message before any other
> `experiment`
> message for each connection.
>    * SHOULD send the `experiment_name_hash` for any features supported and
> set
> to enabled in their software client.
>
> A receiving node:
>    * For each experiment_name_hash:
>       * If the hash is unknown or 0: Ignore the feature
>       * If the hash is known: SHOULD enable the feature for communication
> with
> this peer.
>
> ** experiment_error message
>
>  experiment_type: 17
>  data:
>     [32: channel_id]
>     [32: experiment_name_hash]
>     [2: len]
>     [len: data]
>
> For all messages before funding_created: Must use temporary_channel_id in
> lieu
> of channel_id.
>
> A sending node:
>    * If error is critical, should also send the regular lightning `error`
> message from BOLT #1
>    * If the error is not specific to any channel: set channel_id to 0.
>
> A receiving node
>    * If experiment_name_hash is unknown:
>       - MUST fail the channel.
>    * If channel_id is 0
>       - MUST fail all the channels
>
> Rationale
>
> This message is not intended to replace `error` for criticial errors, but
> is
> provided for additional debugging information
> related to the experimental feature being used.
> A client may decide whether or not it can recover from such errors
> individually per experimental feature, which may include aborting channels
> and
> the connection.
>
> TODO: Define gossip/query messages related to nodes/channels which support
> features by experiment_hash_name.
>
> ---EOF
>
>
> On Sunday, 22 July 2018 13:32:02 BST Olaoluwa Osuntokun wrote:
> > No need to apologize! Perhaps this confusion shows that the current
> process
> > surrounding creating/modifying/drafting BOLT documents does indeed need
> to
> > be better documented. We've more or less been rolling along with a pretty
> > minimal process among the various implementations which I'd say has
> worked
> > pretty well so far. However as more contributors get involved we may need
> > to add a bit more structure to ensure things are transparent for
> newcomers.
> >
> > On Sun, Jul 22, 2018, 12:57 PM René Pickhardt <
> r.pickhardt at googlemail.com>
> >
> > wrote:
> > > Sorry did not realized that BOLTs are the equivalent - and aparently
> many
> > > people I spoke to also didn't realize that.
> > >
> > > I thought BOLT is the protocol specification and the bolts are just the
> > > sections. And the BOLT should be updated to a new version.
> > >
> > > Also I suggested that this should take place for example within the
> > > lightning rfc repo. So my suggestion was not about creating another
> place
> > > but more about making the process more transparent or kind of filling
> the
> > > gap that I felt was there.
> > >
> > > I am sorry for spaming mailboxes with my suggestion just because I
> didn't
> > > understand the current process.
> > >
> > >
> > > Olaoluwa Osuntokun <laolu32 at gmail.com> schrieb am So., 22. Juli 2018
> > >
> > > 20:59:
> > >> We already have the equiv of improvement proposals: BOLTs.
> Historically
> > >> new standardization documents are proposed initially as issues or PR's
> > >> when
> > >> ultimately accepted. Why do we need another repo?
> > >>
> > >> On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> > >>
> > >> lightning-dev at lists.linuxfoundation.org> wrote:
> > >>> Hey everyone,
> > >>>
> > >>> in the grand tradition of BIPs I propose that we also start to have
> our
> > >>> own LIPs (Lightning Network Improvement proposals)
> > >>>
> > >>> I think they should be placed on the github.com/lightning account
> in a
> > >>> repo called lips (or within the lightning rfc repo) until that will
> > >>> happen
> > >>> I created a draft for LIP-0001 (which is describing the process and
> is
> > >>> 95%
> > >>> influenced by BIP-0002) in my github repo:
> > >>>
> > >>> https://github.com/renepickhardt/lips  (There are some open Todos
> and
> > >>> Questions in this LIP)
> > >>>
> > >>> The background for this Idea: I just came home from the bitcoin
> munich
> > >>> meetup where I held a talk examining BOLT. As I was asked to also
> talk
> > >>> about the future plans of the developers for BOLT 1.1 I realized
> while
> > >>> preparing the talk that many ideas are distributed within the
> community
> > >>> but
> > >>> it seems we don't have a central place where we collect future
> > >>> enhancements
> > >>> for BOLT1.1. Having this in mind I think also for the meeting in
> > >>> Australia
> > >>> it would be nice if already a list of LIPs would be in place so that
> the
> > >>> discussion can be more focused.
> > >>> potential LIPs could include:
> > >>> * Watchtowers
> > >>> * Autopilot
> > >>> * AMP
> > >>> * Splicing
> > >>> * Routing Protcols
> > >>> * Broadcasting past Routing statistics
> > >>> * eltoo
> > >>> * ...
> > >>>
> > >>> As said before I would volunteer to work on a LIP for Splicing
> (actually
> > >>> I already started)
> > >>>
> > >>> best Rene
> > >>>
> > >>>
> > >>> --
> > >>> https://www.rene-pickhardt.de
> > >>>
> > >>> Skype: rene.pickhardt
> > >>>
> > >>> mobile: +49 (0)176 5762 3618 <+49%20176%2057623618>
> > >>> _______________________________________________
> > >>> Lightning-dev mailing list
> > >>> Lightning-dev at lists.linuxfoundation.org
> > >>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> > >
> > > Am 22.07.2018 20:59 schrieb "Olaoluwa Osuntokun" <laolu32 at gmail.com>:
> > >
> > > We already have the equiv of improvement proposals: BOLTs. Historically
> > > new standardization documents are proposed initially as issues or PR's
> > > when
> > > ultimately accepted. Why do we need another repo?
> > >
> > > On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> > >
> > > lightning-dev at lists.linuxfoundation.org> wrote:
> > >> Hey everyone,
> > >>
> > >> in the grand tradition of BIPs I propose that we also start to have
> our
> > >> own LIPs (Lightning Network Improvement proposals)
> > >>
> > >> I think they should be placed on the github.com/lightning account in
> a
> > >> repo called lips (or within the lightning rfc repo) until that will
> > >> happen
> > >> I created a draft for LIP-0001 (which is describing the process and is
> > >> 95%
> > >> influenced by BIP-0002) in my github repo:
> > >>
> > >> https://github.com/renepickhardt/lips  (There are some open Todos and
> > >> Questions in this LIP)
> > >>
> > >> The background for this Idea: I just came home from the bitcoin munich
> > >> meetup where I held a talk examining BOLT. As I was asked to also talk
> > >> about the future plans of the developers for BOLT 1.1 I realized while
> > >> preparing the talk that many ideas are distributed within the
> community
> > >> but
> > >> it seems we don't have a central place where we collect future
> > >> enhancements
> > >> for BOLT1.1. Having this in mind I think also for the meeting in
> > >> Australia
> > >> it would be nice if already a list of LIPs would be in place so that
> the
> > >> discussion can be more focused.
> > >> potential LIPs could include:
> > >> * Watchtowers
> > >> * Autopilot
> > >> * AMP
> > >> * Splicing
> > >> * Routing Protcols
> > >> * Broadcasting past Routing statistics
> > >> * eltoo
> > >> * ...
> > >>
> > >> As said before I would volunteer to work on a LIP for Splicing
> (actually
> > >> I already started)
> > >>
> > >> best Rene
> > >>
> > >>
> > >> --
> > >> https://www.rene-pickhardt.de
> > >>
> > >> Skype: rene.pickhardt
> > >>
> > >> mobile: +49 (0)176 5762 3618 <+49%20176%2057623618>
> > >> _______________________________________________
> > >> Lightning-dev mailing list
> > >> Lightning-dev at lists.linuxfoundation.org
> > >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180723/9524b234/attachment-0001.html>


More information about the Lightning-dev mailing list