#1 2020-10-08 12:00:54

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Hi,
I have a system that is compiled on Delphi 7 using mORMOt (last commit 1dee1951016b9ab7 in 2020-03-29).
The system receives an XML which has some PDF encoded in Base64. These PDF are stored in a TDocVariantData structure. Later, it is read in some form to show PDF's for user.
To show a PDF I need to save as a file, in temporary directory. My issue is that I cannot decode Base64 just using Base64Bin() function, but it works using the (old) Synapse lib. But I would like to rip of this lib for use just mORMot.

Consider this:

var
  i: Integer;
  fn: TFileName;
  pdf: RawByteString;
begin
  {...}
  { docs is a TDocVariantData...}
  for i := 0 to docs.Count-1 do
  begin
    {find the correct PDF selected by user...}
    pdf := docs._[i].S['pdf'];
    // FileFromString(DecodeBase64(pdf), fn, True); //<< this works
    FileFromString(Base64ToBin(pdf), fn, True); // << this does not
    ShellExecute(fn); // starts Adobe PDF
  end;
end;

- synacode.DecodeBase64() from Synapse reads perfectly
- XML looks like UTF8
- I've tried to make some conversions, like UTF8ToString / StringToUF8, SynUnicodeTo ... etc, but in vain.

Any ideas?

best regards.

Last edited by mdbs99 (2020-10-08 18:28:34)

Offline

#2 2020-10-08 15:33:09

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,700
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Could you put the input text into a gist so that we reproduce it?

Offline

#3 2020-10-08 18:15:00

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

ab wrote:

Could you put the input text into a gist so that we reproduce it?

I couldn't send the private PDF so, I asked the other part to make a fake one.

Here https://gist.github.com/mdbs99/fb6ddf23 … 2fff7a82f7

The structure is:

<Parametros>
  <docs>
    <doc>
      <nome>filename</nome>
      <pdf>base64</pdf>
    ...
</Parametros>

Offline

#4 2020-10-08 18:32:35

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,700
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

I guess this is because you have line feeds in the text.
Our units expect no line feed (#13#10) in the input content.

Offline

#5 2020-10-08 18:48:17

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

The other system, made in Java, uses a known lib (Apache, I think) to encode in Base64... So, I think it's a valid parser and because that I cannot ask them to change it. sad
Is it possible to implement a workaround in mORMot?

Offline

#6 2020-10-08 19:03:44

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,700
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

You can use StringReplaceAll() before the parsing.

Offline

#7 2020-10-08 19:08:12

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Of course I've test and it still doesn't working. sad

        pdf := StringReplaceAll(docs._[i].S['pdf'], #13#10, '');
        //FileFromString(DecodeBase64(pdf), fn, True);
        FileFromString(Base64ToBin(pdf), fn, True);

Offline

#8 2020-10-08 19:13:12

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

If it can helps, here an example how Synapse do this parser -> https://github.com/marado/synapse/blob/ … e.pas#L172

Offline

#9 2020-10-08 19:37:39

macfly
Member
From: Brasil
Registered: 2016-08-20
Posts: 374

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

I think this replace will be safer if you consider only  #13 as a line break as well.

 pdf := docs._[i].S['pdf'];
 pdf := StringReplaceAll(pdf, #13#10, '');
 pdf := StringReplaceAll(pdf, #13, '');

NOTE: The order matters. You should replace #13#10 first, and then #13

Offline

#10 2020-10-08 19:46:49

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,700
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Or just call

  pdf := TrimControlChars(pdf);

Offline

#11 2020-10-08 20:03:52

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

That is it, TrimControlChars(), worked.
I didn't know about this function thank you.

But, just thinking, if the data that I've provided is considered valid, shouldn't we add a TrimControlChars() call in Base64 functions before conversion?
I've asked the guy and we tested using Java.util (built-in) and Apache commons. Both add line feed in the output...

Last edited by mdbs99 (2020-10-08 20:04:22)

Offline

#12 2020-10-08 20:29:15

mdbs99
Member
From: Rio de Janeiro, Brazil
Registered: 2018-01-20
Posts: 139
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

macfly wrote:

I think this replace will be safer if you consider only  #13 as a line break as well.

 pdf := docs._[i].S['pdf'];
 pdf := StringReplaceAll(pdf, #13#10, '');
 pdf := StringReplaceAll(pdf, #13, '');

NOTE: The order matters. You should replace #13#10 first, and then #13

Thank you too. But I chose Arnaud's tip because it's cleaner.

Offline

#13 2020-10-08 20:37:51

macfly
Member
From: Brasil
Registered: 2016-08-20
Posts: 374

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Has many functions that simplify everything in SynCommons and I always take a look.

But I didn't really know about this TrimControlChars function.

But based on what it does the name shouldn't be RemoveControlChars? smile

Offline

#14 2020-10-08 20:41:17

ab
Administrator
From: France
Registered: 2010-06-21
Posts: 14,700
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Line feeds are usual in base-64, but the problem is that they can be almost everywhere.
So it is not possible to pre-allocate the output buffer if there are some line feeds.

Our conversion routines expect pure base-64 characters, and pre-compute the resulting buffer size.
To be fair, our main usage was binary storage - e.g. for Blobs - as JSON string.
For our use case, our routines are very optimized and efficient. And also safer, since they detect any unexpected/invalid char in the input, whereas DecodeBase64() doesn't.

If you want to handle external base-64 content, as you need, just process the input as TrimControlChars().

Offline

#15 2020-10-08 21:21:20

mpv
Member
From: Ukraine
Registered: 2012-03-24
Posts: 1,571
Website

Re: Issue using Base64ToBin(), but it works using synacode.DecodeBase64()

Linebrakes in base64 It's already explained in this topic https://synopse.info/forum/viewtopic.php?id=5373
Link to the the universal convertor is already there. @ab,  may be we  add it to the trunk (we can even replace existed implementation)

Offline

Board footer

Powered by FluxBB