mORMot Open Source

softx

Member

Registered: 2016-07-05

Posts: 3

Problem Exception class ERangeError with message 'TDynArray SetLength

I have a problem whit dynarrayhashed  Exception class ERangeError with message 'TDynArray SetLength(TMyMapDynArray,41217735) size concern'.

i use the example to work whit big cant of data 85.000.000 of text
type
  TMyMap = record
    Key: string;
    Value: string;
  end;
  TMyMapDynArray = array of TMyMap;

var
  Map: TMyMap;
  Maps: TMyMapDynArray;
  MapW: TDynArrayHashed;
begin
      conteo:=85000000;
     MapW.Init(TypeInfo(TMyMapDynArray),Maps,nil, nil, nil,@conteo);
     SetLength(Maps,conteo);
     MapW.Capacity := conteo;
but in 33.000.000 or more have this exception  Exception class ERangeError with message 'TDynArray SetLength(TMyMapDynArray,41217735) size concern'.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 13,165

Website

Re: Problem Exception class ERangeError with message 'TDynArray SetLength

Array size is too big so is a concern.

softx

Member

Registered: 2016-07-05

Posts: 3

Re: Problem Exception class ERangeError with message 'TDynArray SetLength

thanks for response  im put {$R-} to dont read range checks

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 13,165

Website

Re: Problem Exception class ERangeError with message 'TDynArray SetLength

There is a limitation in TDynArray:

  if NeededSize>1024*1024*512 then // max allowed memory block is 512MB
    raise ERangeError.CreateFmt('TDynArray SetLength(%s,%d) size concern',
      [PShortString(@PTypeInfo(ArrayType).NameLen)^,NewLength]);

So you want to allocate more than 512MB of dynamic array.

This is on purpose.
With such huge number of items, a dynamic array may not be a good candidate, since a single deletion may be very slow.
A linked list may perform better - and use more memory...
Or you should try to use a database, instead. You can have SQLite3 database with a huge number of indexed fields, and still have very good performance.

Anyway.
I've increased the size check to 1GB, and only for 32-bit executables.
See http://synopse.info/fossil/info/25ac1a235b

softx

Member

Registered: 2016-07-05

Posts: 3

Re: Problem Exception class ERangeError with message 'TDynArray SetLength

Thanks for the change works