Posts by saladhorn

saladhorn · mORMot 1

ab wrote:

pointer(aKey)^ is clearly correct.
Look at the asm.

I get the same asm, I agree that it should be correct.
BUT when I run it without my change, it doesn't give the right tag, with the change it gives the correct tag.
Edit: and now I can't replicate it. Uugh! I take my question back

saladhorn · mORMot 1

ab wrote:

Yes, it is as expected, because only PKCS7 padding is really supported in the x86_64 AVX code.
Perhaps try to use zero-padding for the AEAD data.

I'll just disable AVX at runtime if the ciphertext or plaintext is not a multiple. That solves my problem. I will not receive padded data, so I need to support that case anyway, so its easier to only implement that solution. Edit: on my very simple benchmark its takes 150% of the time compared to the accelerated one, but we don't think that encryption will be the restricting bottleneck.

Could you please also take a look at the constructor to TAesAbstract that takes a TBytes as input. I had to modify it as follows to get the correct key. Without this change it never set the correct key.

constructor TAesAbstract.Create(const aKey: TBytes);
begin
---  Create(pointer(aKey)^, length(aKey) shl 3);
+++  Create(pointer(@(aKey[0]))^, length(aKey) shl 3);
end;

saladhorn · mORMot 1

The following code works as expected on both Win32 and Win64. More complete example: https://pastebin.com/pQZarm0H
Without disabling daAesGcmAvx Win64 code throws an exception in Decrypt line 6388 in mormot.crypt.core.pas

DisabledAsm := DisabledAsm + [ daAesGcmAvx ];
aes := TAesGcm.Create(pointer(key)^, 128);
try
  aes.IV := PHash128(iv)^;
  aes.MacSetNonce(true, dummy, aad);
  aes.Decrypt(pointer(cipher), pointer(uncipher), 68);
  var a := aes.AesGcmFinal(tag, 12);
  if not a then
    raise Exception.Create('Tag does not match');
finally
  aes.Free;
end;

saladhorn · mORMot 1

ab wrote:

If the tag is not what you expect, why bother making it on Win64?

We run all our code on Win64 platform. That is the default choice.

The encrypted data comes from another source that is not under our control. If we can not calculate the correct tag, then decryption at the other end fails. If we cannot validate their encrypted packages, we can not use the data they send. That is why getting the correct tag is important.

At the moment the other side does not employ ANY padding, which means they encrypt 68 bytes of data and send those. We can not request the other side to use PKCS7 padding. This means that the 64-bit version of the library is not usable by us BECAUSE it runs a code-path that throws an error when the cleartext/ciphertext is not a multiple of 128-bits. The 32-bit codepath has a fallback path that accounts for this possibility.

On Win64

TAesGcmAbstract.Decrypt -> TAesGcm.AesGcmProcess ->
	if Count and AesBlockMod <> 0 then
		raise ESynCrypto.CreateUtf8('%.Encrypt/Decrypt should use PKCS7', [self]);

On Win32

TAesGcmAbstract.Decrypt -> TAesGcm.AesGcmProcess -> TAesGcmEngine.Decrypt
	// generic process in dual steps

As it is now I can get the Win32 version of mORMot to create a ciphertext+tag combination that will be validated by other libraries, and the ciphertext+tag created by the external party will be validated by mORMot. If I switch to Win64 version of mORMot I cannot validate the external parties ciphertext+tag.

saladhorn · mORMot 1

I have created a new example to try and show my tests: https://pastebin.com/QvVXcYYp

If I run on Win64, the second attempt throws an error. In Win32 it gives the tag I expected from the start.

The following code returns the expected tag (albeit 16 byte length because I haven't updated to the latest version) in Win32 version. BUT in Win64 it throws an exception.

Edit:

FillZero(tag);
aes := TAesGcm.Create(pointer(key)^, 128);
try
  aes.IV := PHash128(iv)^;
  aes.MacSetNonce(true, dummy, aad);
  aes.Decrypt(pointer(cipher), pointer(uncipher), 68);
  var a := aes.AesGcmFinal(tag);
// a is TRUE in Win32
finally
  aes.Free;
end;

saladhorn · mORMot 1

ab wrote:

I already tried all other code paths.
The dual pass pure pascal version gives the same tag version - and it should for sure.

They do for me as well BUT the "Win32" version allows me to run Encrypt() with a 68 byte length. The "Win64" version forces me to have a 80 byte length OR use EncryptPkcs7.

When I pad manually with Pkcs7 padding, I get the same result as you noted above $4276 ... also in the other languages. I do not think there is a bug in the code here.

What in my mind is remaining is non-padded input.

In "Win32" mode I fall back to use the TAesGcmEngine.internal_crypt(ptp, ctp: PByte; ILen: PtrUInt) procedure, this never gets called in "Win64" mode, because an error is raised indicating that I should use the padded version. (path via line 6406 in AesGcmProcess )

What is the exact uncipher text previous to the :12 truncation?

0FC00000010C07E6091A0103272E5AFF8880020209060001190900FF01010206090C
07E6091A0103230000FF8880115A1749CB251D1749CB251D1749CB251D1749CB251D

This is the cleartext. This is 68 bytes long.

saladhorn · mORMot 1

saladhorn wrote:

I will try to investigate some more to see if I can get the same tag in the other languages.

I was able to reproduce the tag in node.js by extending the cleartext with PKCS7 padding.

cleartext =   Buffer.from("0fc00000010c07e6091a0103272e5aff8880020209060001190900ff01010206090c07e6091a0103230000ff8880115a1749cb251d1749cb251d1749cb251d1749cb251d0c0c0c0c0c0c0c0c0c0c0c0c", "hex")
...
tag2 = ciph.getAuthTag()
// <Buffer 42 76 4e 1c 60 93 c7 42 03 4a d8 70>

I think that explains how the different tags are arrived at. It still leaves me with the problem that I have non-aligned cipher and cleartexts, where the ciphertext is not, and can't be because I do not control the sender, padded with Pkcs7.

In order to test this theory further, I started trying to force it down another code path. Just uncommenting the check doesn't work. But when I changed to Win32, which on my machine causes TAesGcmEngine.Encrypt to be run. Here there is a code-path that handles non-aligned cleartext. Starting on line 4814 in mormot.crypto.core

begin
  // generic process in dual pass
  internal_crypt(ptp, ctp,iLen);
  internal_auth(ctp, ILen, txt_ghv, atx_cnt);
end;

So at least we have identified why we see the results we see.

Can this backup path be added to the "Win64" (I know that is probably the wrong name) version as well?

saladhorn · mORMot 1

ab wrote:

When I try with both OpenSSL and mORMot native code, I got the same result, everything is OK, there is no padding problem with aad, and aad is taken into account (if I change it, the tag changes).
https://gist.github.com/synopse/b7051b5 … f4a930af9a
But your values come into 42764E1C6093C742034AD870AAC9D589 as tag, which matches not what you expect...

Huge thanks for the code and the assistance!
I receive the same value as you, so I will assume that there is several problems with my code.

Since both mORMot native (TAesGcm) and OpenSSL (TAesGcmOsl) give the same result tag (42764E1C6093C742034AD870AAC9D589), I don't know what to say.

I would however expect them to give the same tag as the other implementations in other languages. That is the whole point of the tag.

I will try to investigate some more to see if I can get the same tag in the other languages.

How should I use the mORMot library to decode an encrypted piece of data that is not multiple 128-bit blocks, and where the cleartext was not padded with PKCS7? aes.Decrypt will not accept the incorrect length, and aes.DecryptPkcs7 throws an error saying that it has an invalid input length of 68, as according to protocol the tag is the last 12 bytes.

cipher := hextobin('b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843')

saladhorn · mORMot 1

ab wrote:

Your MacSetNonce() seems not correct.
You use the wrong parameter: AAD should not be in RandomNonce but in Associated rawbytestring.
But I guess this is not enough.
Try to use 16 bytes of AAD, and compare the results.

I tried that, and the output is independent of the value of Associated. I get the same checktag with an empty string, as with 3 bytes, as with 16 bytes, as with 17 bytes. I can not change the value of checktag3 by manipulating Associated.
It is the same value as checktag2, which is the case when no AAD is used.

The same is true when I used MacAndCrypt, here the checktag is also independent of Associated.

// Use MacSetNonce instead of AesGcmAAD
mAES3.MacSetNonce(False, IV256, Convert(aad));
rawciphertext := mAES3.EncryptPkcs7(Convert(cleartext));
mAES3.AesGcmFinal(checktag3);
...

...
mAES5.MacSetNonce(False, IV256, '');
rawciphertext := mAES5.EncryptPkcs7(Convert(cleartext));
mAES5.AesGcmFinal(checktag5);
...

checktag3 and checktag5 are the same value. This is strange to me.

saladhorn · mORMot 1

ab wrote:

The GMAC works on 16 bytes blocks by definition.
My guess is that AAD of 17 bytes length is not handled the same.

From what I could find the AAD is not limited in any way in size, it must be geq 0 bits.
https://datatracker.ietf.org/doc/html/r … ection-5.2
https://nvlpubs.nist.gov/nistpubs/legac … 00-38d.pdf

NIST wrote:

• len(A) ≤ 2^64-1;

I don't know what python does, but mORMot uses an internal buffer to GMAC only 16 bytes blocks.

Is that correct?

I tried to replicate my findings in Javascript also : https://pastebin.com/LpK9U6Nu it seems to generate the same tag at least.

Node.JS also https://pastebin.com/y6XjseT4

All of them give the same tag as the Python code, but different to mORMot

saladhorn · mORMot 1

ab wrote:

Try what I wrote above.

https://pastebin.com/TZeKm5H7

I have tried all four variants. None of them give me the checktag that I am expecting when I run it in AesGcmFinal.

saladhorn · mORMot 1

For Delphi Code: https://pastebin.com/wnrCBrAx
For Python Code: https://pastebin.com/ef3HzMG5

saladhorn · mORMot 1

I hope this is clearer.
Four cases: (mORMot, Python) cross (with AAD, without AAD)

# CASE: ciphertext  tag
Python_with:  b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843  e9ceff7675f00b0e217b7620f4baf679
Python_without: b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843  72a227be7c5a8b0ef71aafa13f1407af
mORMot_with:   B662A493A5DFDBCCC1DC832271BAE416945F2E0474D102D2C7941FCD50C678534083E5D1520AE04C3038A281D176B6B2A1CE6E15FE861F4689B7FE7909F309908A40F843  D91A96D469394742D52B01F16167245F 
mORMot_without: B662A493A5DFDBCCC1DC832271BAE416945F2E0474D102D2C7941FCD50C678534083E5D1520AE04C3038A281D176B6B2A1CE6E15FE861F4689B7FE7909F309908A40F843  D91A96D469394742D52B01F16167245F

Both mORMot versions give the same tags as each other, independent of AAD.
Both are different to the Python code. Python code gives different tags in each case.

saladhorn · mORMot 1

ab wrote:

But we did not explicitly validate AAD pre-cipher data in this test. This is why I asked you to make comparison without AAD.

I can not generate the original input without AAD, but a few posts up you can see the output of the different attempts. I'm happy to provide any more.

Are you using Delphi Win32 or Win64? Does mORMot use the flagAVX asm of x86_64, or the pascal version?

Win64. fGcm.flags contains flagAVX is that is what you are asking.

I tested also with Win32, and there it gGcm.flags is flagCLMUL

saladhorn · mORMot 1

ab wrote:

Try using EncryptPkcs7() instead of Encrypt().

That gives the same ciphertext, but still a different tag. Is there a different function that AesGcmFinal that I should be using, or a different AesGcmAad?

saladhorn · mORMot 1

ab wrote:

Is the ciphered text the same?
It may be a padding issue...

Yes, modulo some extra bits in mORMot because I could not find out a way to encrypt only the actual cleartext which is 68 bytes, I add another 12 bytes of zeroes to pad it.

Try perhaps to achieve the same value with python and mORMot with no AAD information. Or with AAD in mulitple of 16 bytes.

I have been trying to do that, and I do agree that it seems to be something with the padding, but I am very unsure what to do about it: (alignment with tabs)

Orig: 		b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843 e9ceff7675f00b0e217b7620
RawPy:		b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843                          withtag e9ceff7675f00b0e217b7620f4baf679		
RawPy_noAAD:	b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843                          withtag 72a227be7c5a8b0ef71aafa13f1407af
PadPy: 		b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843 c2e2f9e5005c3cd6128c2e00 withtag 90782181971ecfeca1f1066bbc4b6d69
PadPy_noAAD: 	b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843 c2e2f9e5005c3cd6128c2e00 withtag 0b14f9499eb44fec7790dfea77e59cbf
mORMot:		B662A493A5DFDBCCC1DC832271BAE416945F2E0474D102D2C7941FCD50C678534083E5D1520AE04C3038A281D176B6B2A1CE6E15FE861F4689B7FE7909F309908A40F843 D8885BFF0E82B9086F69283B withtag F67E34BDD52334E0DFEF8399479F2CFB
mORMot_noAAD:	B662A493A5DFDBCCC1DC832271BAE416945F2E0474D102D2C7941FCD50C678534083E5D1520AE04C3038A281D176B6B2A1CE6E15FE861F4689B7FE7909F309908A40F843 D8885BFF0E8293086F69283B withtag F82C7601236116B70688335BEAFE89C7

Perhaps the AAD has some padding problem.

Unfortunately the form of the AAD and IV are given by the protocol that this message comes from. I am not free to select them myself.

saladhorn · mORMot 1

ab wrote:

IIRC the default MAC length is 16 with TAesGcm.

Agreed, that is what Python returns in the default version also. The "mac_len = 12" parameter is only needed to silence some errors, the first 12 bytes match.

Isn't your tag just truncated from 16 bytes to 12 bytes?

In Python yes, in mormot I don't get the same ones at all. They are completely different.

0E 75 CA B1 98 F5 13 FA 76 AF 65 9E BF 6F F5 B8

saladhorn · mORMot 1

I am trying to replicate Python code with mormot2. I want to decode a AES-GCM encrypted message, and verify its authenticity.

The message like this: b662a493a5dfdbccc1dc832271bae416945f2e0474d102d2c7941fcd50c678534083e5d1520ae04c3038a281d176b6b2a1ce6e15fe861f4689b7fe7909f309908a40f843e9ceff7675f00b0e217b7620
The last 12 bytes (e9ceff7675f00b0e217b7620) are the tag I am looking for.

In Python the tag is generated by the following code:

from Crypto.Cipher import AES # pyCryptoDome
cipher = AES.new(key, AES.MODE_GCM, nonce=iv, mac_len=12)
cipher.update(aad)
genciphertext, tag = cipher.encrypt_and_digest(plaintext)

I am trying to do the same thing in Delphi with the following code, but am not receiving the expected tag. I would appreciate a pointer in the correct direction to continue looking. The code generates the expected ciphertext, but with padding at the end, but if I enter the length of the cleartext

Am I wrong to expect the same tag, or is there some step that I have forgotten?

 var KeyArray: TBytes := [$11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11, $11];

 var iv: TBytes := [$4D, $45, $54, $00, $00, $00, $00, $01, $00, $00, $00, $00];
 var mmIV: TAESBlock;
 FillChar(mmIv, 16, 0);
 System.Move(iv[0], mmIv, System.Length(iv) );

 var cleartext: TBytes := [$0f, $c0, $00, $00, $01, $0c, $07, $e6, $09, $1a, $01, $03, $27, $2e, $5a, $ff, $88, $80, $02, $02, $09, $06, $00, $01, $19, $09, $00, $ff, $01, $01, $02, $06, $09, $0c, $07, $e6, $09, $1a, $01, $03, $23, $00, $00, $ff, $88, $80, $11, $5a, $17, $49, $cb, $25, $1d, $17, $49, $cb, $25, $1d, $17, $49, $cb, $25, $1d, $17, $49, $cb, $25, $1d];
 var aad : TBytes := [$30, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33, $33];

 var ciphertext: TBytes;
 setlength(ciphertext, 80); // 80 is nearest multiple of 16

 var mAES := TAESGCM.Create(KeyArray);
 mAES.IV := mmIv;
 mAES.AesGcmAad(@aad[0], System.Length(aad));
 mAES.Encrypt(@cleartext[0], @ciphertext[0], System.Length(ciphertext));

 var checktag : TAESBlock;
 mAES.AesGcmFinal(checktag);

In a future step I would also want to run the code corresponding to the decryption step also, there the tag verification is currently also the problem.

cipher = AES.new(key, AES.MODE_GCM, nonce=iv, mac_len=12)
cipher.update(aad)
data = cipher.decrypt_and_verify(ciphertext[:-12], ciphertext[-12:]);

mORMot Open Source

#1 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-30 18:38:07

#2 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-30 17:56:06

#3 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-29 19:30:37

#4 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-29 10:19:04

#5 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-29 07:56:49

#6 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-28 11:48:53

#7 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-27 19:36:02

#8 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-27 18:17:14

#9 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-27 12:36:42

#10 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-27 10:44:04

#11 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-27 07:59:01

#12 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 14:37:09

#13 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 14:05:45

#14 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 13:00:50

#15 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 10:35:53

#16 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 10:09:30

#17 Re: mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 08:51:56

#18 mORMot 1 » AES-GCM-128 - calculation of tag not as expected » 2022-09-26 03:28:56

Board footer