[Lightning-dev] type,len,value standard

ZmnSCPxj ZmnSCPxj at protonmail.com
Thu Nov 15 07:39:19 UTC 2018

Good morning Conner,

> > For a sequence of `type,len,value`, the `type`s must be in ascending order
> > -- not explicitly accepted or rejected. It would be easier to check
> > uniqueness > (the previous rule we accepted) here for a naive parser (keep
> > track of some "minimum allowed type" that initializes at zero, check current
> > type >= this, update to current type + 1) if `type`s are in ascending order.
> Yep ascending makes sense to me, for the reasons you stated.

We could bikeshed this and point out that descending would make just as much sense and would work just as well, especially if we want to waste precious electrons on debating this.

> > 1, `type` - one byte or two?
> I'd lean towards one, if a message has 256 optional fields, it might be time to
> consider a new message type altogether.

This rationale seems sensible.

> > 3.  `type` - does "it's OK to be odd" apply? i.e. if an even `type` that is
> >     not known is found, crash and burn. But intent of this system is for future
> >     expansion for optional fields, so...?
> >
> Perhaps this depends on context:
> -   for gossip messages, I think the primary concern is not breaking signature
>     validation, and that these would need to remain optional for backwards
>     compatibility.
> -   for link-level messages, we have a little more control. I imagined the fields
>     would be gated by feature bit negotiation, and deviating from
>     unsupported/required would result in being disconnected.

Then I suppose gossip messages could always just use odd type/key, there would still be 128 of those with a single byte, and at least we would have some sort of consistency, which could help if our software factors out some consistency checks for all t-l-v.

> > 5.  `len` - one byte or two? I believe we tend to use two bytes for various
> >     lengths.
> >
> Maybe varint? One byte is not enough for all lengths, but two seems excessive
> for uint8 or even uint32.

Given that messages are currently only up to 65536 bytes total, is that not a bit much?

> > 6.  BOLT - I propose making a separate BOLT for `type,len,value`, which other
> >     messages and so on simply refer to.
> >
> Indeed, are you thinking we'd use this to add new fields proposed in 1.1?

It would also be good to have a single place to describe the new scheme.
I also imagine that we would do the sensible thing and create a new type/module/class/whatever for t-l-v data structures (a mapping from u16 to binary blobs).
Having a separate BOLT document would help us spec and verify this in a reasonably separate manner.

> In addition to the above, do we also want to flesh out what sub-TLV structures
> would look like? Or perhaps that isn't necessary, if we can continue adding more
> root-level keys.

No, I imagine at the t-l-v level we see only binary blobs for the value, and separate parts of our software would understand what the actual meanings of those blobs are.


More information about the Lightning-dev mailing list