19
$\begingroup$

If you run this code on 11.2

    hmacF[method_, message_, key_] := Module[

  {keyLen, dkey, opad, ipad, blocksize},

  blocksize =
   If[method === "SHA384" || method === "SHA512",
    128,
    64
    ];

  keyLen = StringLength[key];

  dkey = If[keyLen > blocksize,
    IntegerString[Hash[key, method], 16],
    StringPadRight[key, blocksize, FromCharacterCode[0]]
    ];

  {opad, ipad} = 
   FromCharacterCode[
      BitXor[ToCharacterCode@
        StringRepeat[FromCharacterCode[FromDigits[#, 16]], blocksize],
        ToCharacterCode@dkey]] & /@ {"5c", "36"};

  IntegerString[
   Hash[
    StringJoin[opad, 
     ExportString[
      IntegerDigits[Hash[StringJoin[ipad, message], method], 256], 
      "Binary"]],
    method],
   16, blocksize]
  ]

On something we know the answer to https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md

hmacF["SHA256","symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559","NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

you get the correct answer

(*c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71*)

Run the same code on 11.3 and you get

(*88fdd5c8c2226ff025e290b37dec36d4bc6f84affb9928950fd865f9a0fd475e*)

Why the difference?? 11.3 is not the correct HMAC result.

Improve this question
$\endgroup$
6
  • $\begingroup$ Well break the function down and find out which piece changed between versions... it's likely a single function giving a different output, my guess is Hash... $\endgroup$ Commented Mar 11, 2018 at 1:38
  • $\begingroup$ I tested it, 11.2 on desktop, 11.3 on cloud, and yes, Hash seems to have changed in some manner? If you think there is a bug, minimize it and report the issue to [email protected]. $\endgroup$ Commented Mar 11, 2018 at 1:44
  • $\begingroup$ Looking at the Hash documentation page, there does seem to have been a handful of changes to the function for 11.3. Perhaps some of these adversely affected your code. $\endgroup$ Commented Mar 11, 2018 at 1:46
  • $\begingroup$ @user6014 I did read the documentation. None of the changes to Hash should have changed the output, hence my post. In your opinion, is it a bug? Does my function need to be written differently in 11.3? There should be no reason why the function needs to change. $\endgroup$ Commented Mar 11, 2018 at 2:00
  • 3
    $\begingroup$ Here is the minimal example demonstrating the difference: Hash[ExportString[{217, 73, 200, 195, 231, 110, 83, 36, 139, 50, 62, 251, 63, 221, 85, 205, 192, 34, 244, 161, 198, 42, 132, 228, 7, 215, 112, 249, 161, 228, 204, 113}, "Binary"], "SHA256"] $\endgroup$ Commented Mar 11, 2018 at 2:04

1 Answer 1

Reset to default
22
$\begingroup$

Indeed, Hash has changed for 11.3 in several ways.

The one relevant here is that strings are now hashed according to their UTF-8 representation.

As a simple example of the difference, consider the following string, which has only one character.

c = FromCharacterCode[217]; (* "\[CapitalUGrave]" *)

Its UTF-8 representation consists of the two (decimal) bytes {195, 153}, so in version 11.3 the hash would be computed from those two bytes, while in 11.2 the hash would be computed from the single (decimal) byte 217.

Then the two corresponding hash results would be different, namely

(* new hash value *)
Hash[c, "SHA256"]
Hash[StringToByteArray[c], "SHA256"] 
Hash[ByteArray[{195, 153}], "SHA256"]

(*115129295529739515318262490854329296079790445441298973994137838100166123599005*)
(* or as a hex string: *)
(* fe88df3f083e08f99f796b5ba7f6eedbe14d45241cf5e0944caf17f55ac2349d *)

which agrees with various online hash calculators, e.g. http://new.md5calc.com/hash-calc/sha256/%C3%99 and 

(* old hash value *)
Developer`LegacyHash[c, "SHA256"]
Hash[StringToByteArray[c, "ISO8859-1"], "SHA256"]
Hash[ByteArray[{217}], "SHA256"]

(* 11345241613194184164858110515603144575751235911333201083896508434490695406828*)
(* or as a hex string: *)
(* 19152ddfba193b5b09fcb80d1bba5248f36027c06e81670db5a7146fb654d4ec *)

where all inputs were evaluated with 11.3.0.

The hmacF implementation uses strings in a somewhat awkward way as data containers. It is easier to take advantage of several new Hash enhancements instead, like operating on byte arrays and returning the hashes in different formats.

For example,

hmacFnew[method_, message_, key_] := 
 Module[{dkey, dmsg, opad, ipad, blocksize}, 
  blocksize = If[method === "SHA384" || method === "SHA512", 128, 64];
  {dkey, dmsg} = StringToByteArray /@ {key, message};
  If[Length[dkey] > blocksize, dkey = Hash[dkey, method, "ByteArray"]];
  dkey = Normal[dkey];
  If[Length[dkey] < blocksize, dkey = PadRight[dkey, blocksize, 0]];
  {opad, ipad} = ByteArray[BitXor[dkey, ConstantArray[#, blocksize]]]& /@ {92,54};
  Hash[Join[opad,Hash[Join[ipad,dmsg], method, "ByteArray"]], method, "HexString"]
]

hmacFnew["SHA256", \
"symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.\
1&recvWindow=5000&timestamp=1499827319559", \
"NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)

If you don't feel like modifying your code, another way to get the previous behavior is to use Developer`LegacyHash, say

Block[{Hash = Developer`LegacyHash}, 
 hmacF["SHA256", 
  "symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=\
0.1&recvWindow=5000&timestamp=1499827319559", 
  "NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)
Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ I was on the way to posting a similar answer. Just one comment, the correct way of padding a key: PadRight[Normal@ If[keyLen > blocksize, Hash[StringToByteArray@key, method, "ByteArray"], StringToByteArray@key], blocksize, 0]; $\endgroup$ Commented Mar 11, 2018 at 2:45
  • $\begingroup$ Thanks @swish, I was just going by OP's code. I hope it looks correct now. $\endgroup$ Commented Mar 11, 2018 at 2:57
  • 1
    $\begingroup$ That part would be more elegant if PadRight and BitXor accepted byte arrays, but unfortunately they don't. $\endgroup$ Commented Mar 11, 2018 at 5:28
  • $\begingroup$ Thank you! This was really throwing me for a loop. $\endgroup$ Commented Mar 11, 2018 at 13:50

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.