[Bitcoin-ml] Solving the difficulty adjustment problem by asking the opposite question

Troy Benjegerdes hozer at hozed.org
Tue Aug 29 19:20:44 UTC 2017


What about something like the following:

//Checks for 'hardcoded' block timestamps
bool AcceptBlockTimestamp(CValidationState &state, CBlockIndex* pindexPrev, const CBlockHeader *pblock)
{
	int64_t time_allow = -30;
	int64_t time_warn = MINIMUM_BLOCK_SPACING;
	int64_t delta = pblock->GetBlockTime() - pindexPrev->GetBlockTime();
	int nHeight = pindexPrev->nHeight + 1;

	if (nHeight > minimum_time_fork_2){
		time_allow = 30;
	}
	
	if (delta < time_warn){
		printf("WARNING blocktime nHeight %d time_allow %" PRId64" time_warn %" PRId64" time delta %" PRId64"\n", nHeight, time_allow, time_warn, delta);
	}

	if (nHeight >= minimum_time_fork_2) {
		if (delta <= time_allow) // see above, from first hard limit
			return state.Invalid(error("AcceptBlock(height=%d) : block time delta %" PRId64" too short", nHeight, delta));
	}
	if (nHeight >= minimum_time_fork) { /* don't forward these */
		if (delta <= MINIMUM_BLOCK_SPACING)
			return state.DoS(10, (error("AcceptBlock(height=%d) : block time delta %" PRId64" too short", nHeight, delta)));
	}
	return true;	
}

In actual practice it has worked with both Catcoin and Grantcoin over the
course of years. Yes, you are correct that you cannot go 'back in time' to
validate timestamps on a chain download, however you can still trust certain
assumptions about historical hashrate, the compiled-in checkpoints, and the
(assumption/fact) that miners have an incentive to use accurate block
timestamps or their blocks will not be forwarded or built on by other miners.

This could initially be implemented as a soft-fork in which only a couple of
mining pools need to deploy, in which they only build on blocks which are:

* no more than XX seconds in the future
* at least YY seconds after the previous block timestamp

Mining pools could be incentivized to deploy such a thing even if it resulted
in slighty lower short term revenue because they risk building blocks on a
'non-standard' chain that **exchanges** consider to have invalid timestamps.

Finally, I'd say instead of theorizing about this, we test it with real money
on the table, which could be a testnet, or my preference, which is a real live
fork of BCH with a new short-adjustment difficulty adjustment (maybe using a
better fixed-point PID algorithm modeled after the one in Catcoin, which is,
regrettably, implemented using floating-point), and some agreements by the
idea's backers to exchange test-fork-coins for real BCH coins for a limited
test period (say 1-2 months)

Opponents of the proposal could then put real money on the table to both
short the test chain, and implement attacks (or bounties for attack code).

Otherwise we're just making a lot of noise and arguing, and the greatest
thing BCH did was let the market sort out what's real between the big
blockers and segwit crowd. You did something instead of just posturing,
so let's follow the good example and try some different time-based reward
strategies, as well as difficulty adjustment mechanisms.

On Mon, Aug 28, 2017 at 01:56:49AM +0200, Amaury Séchet via bitcoin-ml wrote:
> To the contrary, if coins being unstable is too damaging, they hurt their
> bottom line. And if they are not so damaging, then this problem is not that
> urgent and there is no need to freak out.
> 
> Back to the proposal, considering timestamp are unreliable and can't be
> checked ex post facto, how is that secure in any way ?
> 
> Le 28 août 2017 1:53 AM, "Jan Nightingale via bitcoin-ml" <
> bitcoin-ml at lists.linuxfoundation.org> a écrit :
> 
> > I don't think another solution is possible at all.
> >
> > In >1-chain case all pure difficulty adjustments can only produce a binary
> > result: more profitable than other chains or not, taking into account the
> > entire coin maturity time, predicted difficulty changes, etc. If the result
> > is more profitable, all profit-maximizing power jumps. After the coin stops
> > being profitable, they jump out, and the only way to save the chain is for
> > some miner(s) to mine at a loss.
> > Stabilizing the block issuance by manipulating the difficulty is thus
> > IMPOSSIBLE. New strategies can only work temporarily until a new exploiting
> > strategy is developed.
> >
> > Why do simple strategies sort-of work in altcoins under similar situations
> > and without node enforcement? The most likely answer seems to be that
> > miners just aren't interested enough to start exploiting them, in other
> > words the particular mining market is very far from being efficient,
> > possibly due to the relatively small size. Alternatively, the majority is
> > unwilling to engage in hostile practices.
> >
> > Unfortunately none of this is true for sha256 mining, and Bitcoin Cash can
> > expect a lot of hostility and attacks.
> >
> > It's of course possible that the chain manages to survive without attacks
> > until it becomes the biggest one, but that's a risky bet to make.
> >
> >
> > -------- Original Message --------
> > Subject: Re: [Bitcoin-ml] Solving the difficulty adjustment problem by
> > asking the opposite question
> > Local Time: August 27, 2017 12:04 AM
> > UTC Time: August 26, 2017 10:04 PM
> > From: erik.beijnoff at gmail.com
> > To: Jan Nightingale <turifex at protonmail.ch>, Bitcoin and related network
> > protocol discussion <bitcoin-ml at lists.linuxfoundation.org>
> >
> > I think this is an interesting proposal that shouldn???t be dismissed too
> > easily.
> >
> > I also think that the possibilities for a miner to shift timestamps is
> > fairly limited due to network acceptance. It would probably be a one time
> > movement in one direction after which the timestamp will be firmly planted
> > at one end of the acceptable range.
> >
> > However, and this is probably a big issue, this is such a big deviation
> > from something tried and tested with Bitcoin etc. that the implications of
> > it would need to be tested over a longer time than we have. Therefore I
> > think it should be included in a list of suggestions, elaborated on, but to
> > switch to it is probably a too big deviation from what is currently
> > accepted by the network.
> >
> > Much appreciated that you propose a new way to attack the issue though.
> >
> > /Erik
> >
> >
> > > On 26 Aug 2017, at 22:08, Jan Nightingale via bitcoin-ml <
> > bitcoin-ml at lists.linuxfoundation.org> wrote:
> > >
> > > Additional points:
> > >
> > > Scenario - miners trying to manipulate difficulty/time:
> > > A miner mines a block that"s 30 second into the future with difficulty
> > 1. It"s not accepted by the network now, but it"s going to be 30 seconds
> > from now.
> > > I can mine on top of his chain, creating a t+60 block. Which means
> > waiting that long until it"s considered valid, risking an orphan. Or I can
> > orphan his block mining with difficulty 2 and wait "just" 30 seconds.
> > Orphaning is a much safer proposition.
> > >
> > > But then another miner comes, and also mines with difficulty 2, but with
> > the current timestamp, orphaning my block, as it"s earlier even though the
> > difficulty is equal.
> > >
> > > (1) it"s much easier to lower the timestamp in comparison to increasing
> > the difficulty
> > > (2) Mining a block with a future timestamp gives competing miners time
> > to mine something with at least equal difficulty but lower timestamp, so
> > lower timestamp makes an orphan much less likely.
> > > (3) As time=reward, the more in the future a block is, the more
> > incentive other miners have to orphan it.
> > >
> > > Due to these incentives timestamps should turn out to be as close to the
> > current time as possible.
> > >
> > > The difficulty increase process would continue until subsequent miners
> > decide that the expected profitability of mining on top of an existing
> > block is higher than the expected profitability of trying to orphan. Or
> > more likely, seeing the futility of too low difficulty miners are just
> > going to put in something reasonable, with low probability of getting
> > orphaned.
> > >
> > > Is there going to be more wasted hash power?
> > > I don"t see a fundamental reason why. There are going to be more
> > orphans, yes. However, the hashpower waste shouldn"t be much worse than the
> > current one. Current waste is invisible - as it"s much harder to create a
> > block. The only person who knows about stale shares is a miner and his
> > pool. With faster blocks it"s going to become globally visible, but actual
> > levels should be comparable.
> > >
> > > I admit it"s likely at the beginning there"s going to be an additional
> > hash power waste as miners are likely to try various difficulty strategies
> > to learn what works best.
> > >
> > >> -------- Original Message --------
> > >> Subject: [Bitcoin-ml] Solving the difficulty adjustment problem by
> > asking the opposite question
> > >> Local Time: August 26, 2017 5:48 PM
> > >> UTC Time: August 26, 2017 3:48 PM
> > >> From: bitcoin-ml at lists.linuxfoundation.org
> > >> To: bitcoin-ml at lists.linuxfoundation.org <bitcoin-ml at lists.
> > linuxfoundation.org>
> > >>
> > >> Hi,
> > >> I think the current approach - changing the difficulty based on past
> > block times - is impossible to do right in the presence of >1 chains with
> > the same POW. The 2-chain case is not even close to the worst one - what if
> > there are 3? A realistic short-term possibility! In the future, one can
> > imagine dozens of sha256 chains.
> > >>
> > >> Consider:
> > >> If one chain is more profitable, ALL profit-maximizing miners should
> > switch.
> > >> After difficulty drops enough so that the chain stops being profitable,
> > again, ALL profit-maximizing miners switch.
> > >> What this means: there"s no solution. No matter what the difficulty
> > algorithm is, enormous oscillations of hash power are unavoidable. It"s a
> > rabbit hole of introducing ever more complex strategies fixing the
> > particular exploitation methods of previous methods.
> > >>
> > >> Non-profit maximizing miners are impossible to model, as there are
> > infinite possible incentives. There"s also additional complexity arising
> > due to miners" switching lag - but again, imo that"s impossible to model as
> > it depends on the particular mining operation.
> > >>
> > >> Even if the 2-chain case could be solved this way (I"m skeptical) I
> > don"t believe it"s going to work in the >2 case, making the system fragile
> > and requiring another change.
> > >>
> > >> For this reason, I propose going to the bare fundamentals and turning
> > the difficulty system on its head. To quote Tom Harding "The combined
> > BTC+BCC system is very complex. I think more ideas need to be considered as
> > well.". The idea is radical but imo the only approach that could actually
> > work.
> > >>
> > >> What are the fundamentals in my opinion?
> > >> - we want X new coins per Y minutes according to the inflation schedule
> > - $desiredInflationPer10Minutes;
> > >> - we want the transaction capacity to be limited, equivalent to
> > $currentBlockLimit per Y minutes.
> > >>
> > >> Currently, Bitcoin, and to my knowledge all other POW coins, work by
> > observing (1) reducing the relation between those variables to a constant,
> > reducing the variables in the system to one - block time, and
> > >> (2) observing the block time on the live network and correcting the
> > difficulty in the desired direction.
> > >>
> > >> The system works because of an ephemeral network consensus - nodes
> > refusing to accept a block that"s too far in the future.
> > >>
> > >> The proposal:
> > >> - Instead of per block, make the rewards directly connected to time. A
> > block with a timestamp 600 seconds more than the previous one gets
> > $desiredInflationPer10Minutes; a block with 300 seconds more gets
> > $desiredInflationPer10Minutes/2 [0].
> > >> - Same for the block size limit - a block 300 seconds after the last
> > one gets $currentBlockLimit/2, 150 seconds $currentBlockLimit/4 etc [0].
> > >> - No minimum difficulty - miners can mine whatever difficulty they
> > want, however, the most valid chain rule stays the same - the highest
> > cumulative difficulty.
> > >> - To prevent excessive flooding with empty blocks, consecutive blocks
> > have to be at least 10 seconds apart. That"s completely arbitrary, 10
> > seconds just seem sensible to me.
> > >>
> > >> The desired outcome:
> > >> A stable equilibrium is reached and all oscillations are dampened.
> > Miners have the incentive to estimate the amount of competing hash power at
> > the moment and set their difficulty high enough so that there"s a
> > sufficiently low chance of getting orphaned, but low enough so that they
> > are producing blocks.
> > >>
> > >> A miner setting his difficulty too low will find out he"s wasting hash
> > power as all or most of his blocks are getting orphaned.
> > >> A miner setting his difficulty too high will find out he"s wasting hash
> > power as with a lower difficulty he could get more rewards that wouldn"t be
> > orphaned.
> > >>
> > >> As a result, the system self-corrects very fast, incentivizing miners
> > to use variables that are impossible to include in the pure past-block time
> > difficulty adjustment: current ratio of block rewards compared to other
> > chains and amount of competing hash power, possibly more.
> > >>
> > >> What about block time? Is that a problem? In my opinion, no - on the
> > contrary!
> > >> - a block header is only 80 bytes, so even one block every 10 seconds
> > means 4800B-80B=4720B of "waste" per 10 minutes - that"s nothing. [1]
> > >> - block confirmations get to be almost continuous. There"s some initial
> > period, let"s say 30 seconds, in which there"s a high risk of orphans - but
> > after that, confirmation strength grows very fast, potentially making
> > on-chain payments safe enough for even large in-person shop payments! More
> > useful as cash.
> > >> - there are probably going to be empty blocks even in the presence of
> > unconfirmed transactions, but note that the block size limit is defined the
> > same way - so there"s no capacity lost! Imo there"s no reason to prefer one
> > 8MB block per 10 minutes to, for example, 8x1MB in the same 10 minutes.
> > >>
> > >> Minor considerations:
> > >> - node DOS protection - a node accepting blocks under this scheme could
> > find itself DOSed with lots of spurious low-difficulty chains. This problem
> > already exists but would be exacerbated. However, that"s not a consensus
> > problem, so it can be dealt much more arbitrarily - by estimating
> > "reasonable" difficulty from external sources or other nodes.
> > >> [0] - there"s a very minor problem arising due to integer division
> > leading to loss of lowest bits. It could be ignored, leading to minusculely
> > lower inflation and/or size. Instead, I think accumulating the errors is
> > the right approach - ie. lost block reward appears as soon as it fits into
> > the uint64 amount, similar for size.
> > >> [1] - the $blockLimit per 10 minutes could be computed independently of
> > headers
> > >>
> > >> Regards,
> > >> Jan Nightingale
> > >
> > > _______________________________________________
> > > bitcoin-ml mailing list
> > > bitcoin-ml at lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-ml
> >
> >
> >
> > _______________________________________________
> > bitcoin-ml mailing list
> > bitcoin-ml at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-ml
> >
> >

> _______________________________________________
> bitcoin-ml mailing list
> bitcoin-ml at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-ml



More information about the bitcoin-ml mailing list