[Bitcoin-development] Scalability issues

[Michael Grønager]

Hi Steve,

45-90 minutes - note that its numbers from March/April, so a bit longer today, but far, far away from the 12 hours. 

I am using libcoin and the bitcoind build based on this. Libcoin is based on the Satoshi client, but refactured to use an async concurrency model. I also did a minor tweeks to the db parameters. It has earlier been tested up against Satoshi bitcoin where on some OS'es it performs similarly (at least on some linuxes) and on some faster (e.g. mac). 

What is your CPU load during a block download ? (both initially/up to the point where verification sets in and after). The initial download is typically disk I/O bound, the verification stage CPU bound, though I lean to believe that even there it is disk I/O bound (at least on my system ~50% CPU load). What should be better in libcoin is the concurrency model. The Satoshi client uses a pure reentrant mutexes model, that is not generally believed to motivate the best coding practice nor performance, you might end up without the concurrency you initially strived for *). As mentioned earlier libcoin uses a pure async concurrency model (and so does libbitcoin btw). 

I would like to stress again that these numbers will depend largely on the system running the test - I would call my laptop a bit over the average today (MB Pro, 2.66Ghz i7 dual core, 8GBRAM, 512GB SSD). But again 12 hours - I only reach such numbers on some of my VPS'es (linode 1024) that are known for notoriously slow disk I/O. (here I have a few % CPU load during the verification indicating indeed that the disk i/o is the culprit).

Cheers,

Michael


*) I like this Dave Butenhof quote: "The biggest of all the big problems with recursive mutexes is that they encourage you to completely lose track of your locking scheme and scope. This is deadly. Evil. It's the "thread eater". You hold locks for the absolutely shortest possible time. Period. Always. If you're calling something with a lock held simply because you don't know it's held, or because you don't know whether the callee needs the mutex, then you're holding it too long. You're aiming a shotgun at your application and pulling the trigger. You presumably started using threads to get concurrency; but you've just PREVENTED concurrency."




On 23/07/2012, at 17:54, steve wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Michael,
> 
> On 23/07/2012 10:00, Michael Grønager wrote:
>> I get a full blockchain from scratch in 45 minutes on my laptop,
>> /M
>> 
> Hang on a sec, in 45 minutes you can download the entire chain from
> the genesis block?
> 
> I have been doing extensive testing in this area and would love to
> know what is special about your setup (I have never had the entire
> chain in under 12 hours, infact it is normally closerto 24.) I have an
> extensive setup of test machines, everything from e4300 to phenom2x6
> to i5's.
> 
> as an example on an amd e-450 with 4gb ram, and approx 3gb/s internet
> connection it took 2 hours to sync the last 5 days.
> 
> Maybe i am missing something important...
> 
> Any additional information that you could provide to help me with
> testing would be really appreciated.
> 
> cheers,
> 
> steve
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iQEcBAEBAgAGBQJQDXO4AAoJEFvEB9dQFvtQxdcH/ieqQkyDCg8mKeOa6CqsWaS6
> fhoeny3Ke2b/CsvhYmsThCvntN9volIqR2CTn5tkHiVwG9OmlxyHZcNpN0ZTHhK5
> lsfLap/Y0QpiysXpV4Bu7Z4Hwp9jnhOP74TshT305r2pX6EGXPQ0CrlHqlIry/X/
> vNcunUclliou+KjL7EHcY50GH5wDpqJAjlNyF97Lj9YiPrAC9vahGwWdxkbCYtG+
> KUuWGBKMMdHuMAgcQh7nI9q0WT3k/gzRQtuC2kf+v0wvQhaGlTVkku4uanhpuw4p
> 99blRF3/SfWimGuQgsm6wT3Y7dk+z8MFHLb6XGPxmgV9+gF+TWNczfU3GRzfcXw=
> =CQkI
> -----END PGP SIGNATURE-----
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development