[bitcoin-dev] [BIP Draft] Datastream compression of Blocks and Transactions

[Rusty Russell]

Gavin Andresen via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org>
writes:
> On Wed, Dec 2, 2015 at 1:57 PM, Emin Gün Sirer <
> bitcoin-dev at lists.linuxfoundation.org> wrote:
>
>> How to Do It
>>
>> If we want to compress Bitcoin, a programming challenge/contest would be
>> one of the best ways to find the best possible, Bitcoin-specific
>> compressor. This is the kind of self-contained exercise that bright young
>> hackers love to tackle. It'd bring in new programmers into the ecosystem,
>> and many of us would love to discover the limits of compressibility for
>> Bitcoin bits on a wire. And the results would be interesting even if the
>> final compression engine is not enabled by default, or not even merged.
>>
>
> I love this idea. Lets build a standardized data set to test against using
> real data from the network (has anybody done this yet?).

https://github.com/rustyrussell/bitcoin-corpus

It includes mempool contents and tx receipt logs for 1 week across 4
nodes.  I vaguely plan to update it every year.

A more ambitious version would add some topology information, but we
need to figure out some anonymization strategy for the data.

Cheers,
Rusty.