mORMot Open Source

Are you a fool? These properties are not assigned; they are read-only!

Strange, I tried direct conversion and it doesn’t work

uses mormot.core.text, mormot.core.base;

function ParseCommandLine(const param: string; out acount: Integer): PPWideChar;
var
par:TRawUtf8DynArray;
j:integer;
begin
par:=CsvToRawUtf8DynArray(param, ' -');
acount:=Length(par);
for j:=1 to acount-1 do
par[j]:=Trim('-'+par[j]));
Result:=PPWideChar(par);
end;

Yeah, I think I did it and it works well, but I don’t like that there are a lot of transformations

uses mormot.core.text, mormot.core.base;

function ParseCommandLine(const param: string; out acount: Integer): PPWideChar;
var
par:TRawUtf8DynArray;
arg:array of PWideChar;
j:integer;
begin
par:=CsvToRawUtf8DynArray(param, ' -');
acount:=Length(par);
SetLength(arg, acount);
arg[0]:=PWideChar(wideString(Trim(par[0])));
for j:=1 to acount-1 do
arg[j]:=PWideChar(wideString(Trim('-'+par[j])));
Result:=PPWideChar(arg);
end;

Maybe it will be useful to someone

I also noticed that when using c.Args[0]); an error is thrown (

No, I need to parse it so that each parameter is in an array

Example:

params:string:='-o file.txt --y -v -t 1';

after parse

array[0]= -o file.txt
array[1]= --y
array[2]= -v
array[3]= -t 1

And I don’t really know how to do this

Thanks a lot, but for example I need to use my own user agent and proxy ?

I figured it all out, but I didn’t understand one thing: does it automatically determine the type of proxy HTTP, SOCKS4, SOCKS5 ?

procedure TForm2.Button1Click(Sender: TObject);
var
H:TSimpleHttpClient;
begin
H:=TSimpleHttpClient.Create(False);
h.UserAgent:='.....................';
h.Proxy:=????;
h.Request('https://', 'GET');
Memo1.Lines.Add(h.Body);
h.Free;
end;

Yes, indeed, I myself noticed that sha256-sha512 works perfectly and quickly in the mORMot library !

Hello, I forgot to answer, I think I did as you advised and now the results are normal:

function MyHasher(crc: cardinal; buf: PAnsiChar; len: cardinal): cardinal;
var
Hasher:Thasher;
begin
Hasher:=cryptCrc32(caMd5);
SetBitPtr(buf, 2);
Result:=Hasher(crc, buf, len);
end;

Now there are about 12-15 duplicates in file displacement seems to help a lot, I would like to somehow transfer SynBloomFilter to 64-bit hashes; in theory, there will be even fewer collisions

Hello, here are the test results for the standard TSynBloomFilter hashing and our own:

Hashing out of the box:

BloomFilter:=TSynBloomFilter.Create(100000000,0.00001);

Hashing with your own function:

function MyHasher(crc: cardinal; buf: PAnsiChar; len: cardinal): cardinal;
begin
Result:=BobJenkinsHash(buf[0], len, MurMurHash3(buf[0], len, crc xor len));
end;

BloomFilter:=TSynBloomFilter.Create(100000000,0.00001, MyHasher);

As you can see, standard hashing is much worse, both tests were carried out on the same file without duplicates

Okay, here the debate can actually drag on for a long time, I would like to try something like SHA256, SHA512 So I can do it like this:

uses
HlpHashFactory // module HashLib4Pascal

function MyHasher(crc: cardinal; buf: PAnsiChar; len: cardinal): cardinal;
begin
Result:=THashFactory.TCrypto.CreateSHA3_256.ComputeUntyped(buf, len).GetInt32;
end;

BloomFilter:=TSynBloomFilter.Create(100000000,0.00001, MyHasher);

Can I do something similar using just mORMot? Because HashLib4Pascal hashes very slowly

Even any one hash from the MOrmot set is significantly inferior to BobJenkinsHash or xxHash32 please check this, is there any hashing error ?
I really don’t understand why, in fact, cryptographic hashes are suddenly inferior to the old non-cryptographic hashes ?

Thank you very much for your great desire to help, what I don’t understand is how to use sha256 or sha512 for TSynBloomFilter for the newest version mORMot2 )?

I have just started to understand the new BloomFilter arsenal and I still don’t understand how to specify sha256 , sha512 in it

BloomFilter:=TSynBloomFilter.Create(100000000,0.00001, SHA3_512)); // NOT )

Tell me how I can find Merkle Tree in mORMot lib ?

Hello, I’m very glad that you are also interested in this question,
It seems that I was able to achieve almost 0% collisions with the hybrid use of hashes:

function crc32c(crc: cardinal; buf: PAnsiChar; len: cardinal): cardinal;
begin
Result:=BobJenkinsHash(buf[1], len, xxHash32(buf[1], len, crc)); // Spring4D module
end;

In mormot.core.search.pas:

uses
  classes,
  sysutils,
  mormot.core.base,
  mormot.core.os,
  mormot.core.rtti,
  mormot.core.unicode,
  mormot.core.text,
  mormot.core.buffers,
  mormot.core.datetime,
  mormot.core.data,
  mormot.core.json,
  System.Generics.Defaults, // Add this
  Spring.Hash; // And This (Spring4D Module)

.......

{ ****************** Bloom Filter Probabilistic Index }

{ TSynBloomFilter }


function crc32c(crc: cardinal; buf: PAnsiChar; len: cardinal): cardinal;
begin
Result:=BobJenkinsHash(buf[1], len, xxHash32(buf[1], len, crc)); // Spring4D module
end;

const
  BLOOM_VERSION = 0;
  BLOOM_MAXHASH = 32; // only 7 is needed for 1% false positive ratio

And I create TSynBloomFilter like this:

BloomFilter:=TSynBloomFilter.Create(100000000,0.000001); // If you put less 0.000001 then there will be collisions Seems 0.000001 this is the limit =(

Please check this as well and report the results !

Hello, I mean that there is a very large number of collisions in a file that has no duplicates; it finds more than 1000 duplicates
Converting to the RawByteString type changes absolutely doesn't change anything

Hail, mORMot version 1, well look, I need, for example, from a file of 100 gigabytes in size to clear duplicate lines using TSynBloomFilter, I read the file line by line using TStreamReader I can do that ?
p.s Delphi XE2