[Bitcoin-development] BIP 38 NFC normalisation issue

[Andreas Schildbach]

I think generally control-characters (such as \u0000) should be
disallowed in passphrases. (Even the use of whitespaces is very
questionable.)

I'm ok with allowing pile-of-poo's. On mobile phones there is keyboards
just containing emoticons -- why not allow those? Assuming NFC works of
course.


On 07/15/2014 03:07 PM, Eric Winer wrote:
> I don't know for sure if the test vector is correct NFC form.  But for
> what it's worth, the Pile of Poo character is pretty easily accessible
> on the iPhone and Android keyboards, and in this string it's already in
> NFC form (f09f92a9 in the test result).  I've certainly seen it in
> usernames around the internet, and wouldn't be surprised to see it in
> passphrases entered on smartphones, especially if the author of a
> BIP38-compatible app includes a (possibly ill-advised) suggestion to
> have your passphrase "include special characters".
> 
> I haven't seen the NULL character on any smartphone keyboards, though -
> I assume the iOS and Android developers had the foresight to know how
> much havoc that would wreak on systems assuming null-terminated strings.
>  It seems unlikely that NULL would be in a real-world passphrase entered
> by a sane user.
> 
> 
> On Tue, Jul 15, 2014 at 8:03 AM, Mike Hearn <mike at plan99.net
> <mailto:mike at plan99.net>> wrote:
> 
>     [+cc aaron]
> 
>     We recently added an implementation of BIP 38 (password protected
>     private keys) to bitcoinj. It came to my attention that the third
>     test vector may be broken. It gives a hex version of what the NFC
>     normalised version of the input string should be, but this does not
>     match the results of the Java unicode normaliser, and in fact I
>     can't even get Python to print the names of the characters past the
>     embedded null. I'm curious where this normalised version came from.
> 
>     Given that "pile of poo" is not a character I think any sane user
>     would put into a passphrase, I question the value of this test
>     vector. NFC form is intended to collapse things like umlaut control
>     characters onto their prior code point, but here we're feeding the
>     algorithm what is basically garbage so I'm not totally surprised
>     that different implementations appear to disagree on the outcome.
> 
>     Proposed action: we remove this test vector as it does not represent
>     any real world usage of the spec, or if we desperately need to
>     verify NFC normalisation I suggest using a different, more realistic
>     test string, like Zürich, or something written in Thai.
> 
> 
> 
>     Test 3:
> 
>       * Passphrase ϓ␀𐐀💩 (\u03D2\u0301\u0000\U00010400\U0001F4A9; GREEK
>         UPSILON WITH HOOK <http://codepoints.net/U+03D2>, COMBINING
>         ACUTE ACCENT <http://codepoints.net/U+0301>, NULL
>         <http://codepoints.net/U+0000>, DESERET CAPITAL LETTER LONG I
>         <http://codepoints.net/U+10400>, PILE OF POO
>         <http://codepoints.net/U+1F4A9>)
>       * Encrypted key:
>         6PRW5o9FLp4gJDDVqJQKJFTpMvdsSGJxMYHtHaQBF3ooa8mwD69bapcDQn
>       * Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF
>       * Unencrypted private key (WIF):
>         5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4
>       * /Note:/ The non-standard UTF-8 characters in this passphrase
>         should be NFC normalized to result in a passphrase
>         of0xcf9300f0909080f09f92a9 before further processing
> 
> 
> 
> 
>     ------------------------------------------------------------------------------
>     Want fast and easy access to all the code in your enterprise? Index and
>     search up to 200,000 lines of code with a free copy of Black Duck
>     Code Sight - the same software that powers the world's largest code
>     search on Ohloh, the Black Duck Open Hub! Try it now.
>     http://p.sf.net/sfu/bds
>     _______________________________________________
>     Bitcoin-development mailing list
>     Bitcoin-development at lists.sourceforge.net
>     <mailto:Bitcoin-development at lists.sourceforge.net>
>     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> 
> 
> 
> 
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> 
> 
> 
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>