mORMot2 API Reference

mormot.core.buffers.pas unit

Purpose: Framework Core Low-Level Memory Buffer Process
- this unit is a part of the Open Source Synopse mORMot framework 2, licensed under a MPL/GPL/LGPL three license - see LICENSE.md

1.1. Units used in the mormot.core.buffers unit

Unit Name	Description
mormot.core.base	Framework Core Shared Types and RTL-like Functions
mormot.core.os	Framework Core Low-Level Wrappers to the Operating-System API
mormot.core.rtti	Framework Core Low-Level Cross-Compiler RTTI Definitions
mormot.core.text	Framework Core Low-Level Text Processing
mormot.core.unicode	Framework Core Low-Level Unicode UTF-8 UTF-16 Ansi Conversion

1.2. mormot.core.buffers class hierarchy

mormot.core.buffers class hierarchy

1.3. Objects implemented in the mormot.core.buffers unit

Objects	Description
EAlgoCompress	Exception raised by TAlgoCompress classes
EBufferException	Exception raised during buffer processing
EFastReader	Exception raised during TFastReader decoding
EStreamRedirect	Exception raised during TStreamRedirect processing
TAlgoCompress	Abstract low-level parent class for generic compression/decompression algorithms
TAlgoCompressWithNoDestLen	Abstract class storing the plain length before calling compression API
TAlgoRle	Implement RLE compression as a TAlgoCompress class
TAlgoRleLZ	Implement our SynLZ compression with RLE preprocess as a TAlgoCompress class
TAlgoSynLZ	Implement our fast SynLZ compression as a TAlgoCompress class
TBufferedStreamReader	TStream with an internal memory buffer
TBufferWriter	This class can be used to speed up writing to a file or a stream
TFakeWriterStream	A fake TStream, which will just count the number of bytes written
TFastReader	Safe decoding of a TBufferWriter content from an in-memory buffer
TMemoryMapText	Able to read a UTF-8 text file using memory map
TMultiPart	Used by MultiPartFormDataDecode() to return one item of its data
TNestedStreamReader	TStream allowing to read from some nested TStream instances
TProgressInfo	Information about the progression of a process, e.g. for TStreamRedirect
TRawByteStringBuffer	Store one RawByteString content with an associated length
TRawByteStringGroup	Store several RawByteString content with optional concatenation
TRawByteStringGroupValue	Item as stored in a TRawByteStringGroup instance
TRawByteStringHeap	Thread-safe reusable set of constant RawByteString/RawUtf8 instances
TStreamRedirect	An abstract pipeline stream able to redirect and hash read/written content
TStreamRedirectCrc32c	TStreamRedirect with crc32c 32-bit checksum
TStreamRedirectHasher	TStreamRedirect with 32-bit THasher checksum
TValueResult	Kind of result returned by FromVarBlob() function

1.3.1. TValueResult

TValueResult = record

Kind of result returned by FromVarBlob() function

Len: PtrInt;

Value length (in bytes)

Ptr: PAnsiChar;

Start of data value

1.3.2. EAlgoCompress

EAlgoCompress = class(ESynException)

Exception raised by TAlgoCompress classes

1.3.3. TAlgoCompress

TAlgoCompress = class(TObject)

Abstract low-level parent class for generic compression/decompression algorithms
- will encapsulate the compression algorithm with crc32c hashing
- all Algo* abstract methods should be overriden by inherited classes
- don't inherit from TSynPersistent since we don't need any of it

constructor Create; virtual;

Will register AlgoID in the global list, for Algo() class methods
- no need to free this instance, since it will be owned by the global list
- raise a EAlgoCompress if the class or its AlgoID are already registered
- you should never have to call this constructor, but define a global variable holding a reference to a shared instance

destructor Destroy; override;

Finalize this algorithm

class function Algo(Comp: PAnsiChar; CompLen: integer): TAlgoCompress; overload;

Get the TAlgoCompress instance corresponding to the AlgoID stored in the supplied compressed buffer
- returns nil if no algorithm was identified

class function Algo(const Comp: RawByteString): TAlgoCompress; overload;

Get the TAlgoCompress instance corresponding to the AlgoID stored in the supplied compressed buffer
- returns nil if no algorithm was identified

class function Algo(const Comp: TByteDynArray): TAlgoCompress; overload;

Get the TAlgoCompress instance corresponding to the AlgoID stored in the supplied compressed buffer
- returns nil if no algorithm was identified

class function Algo(aAlgoID: byte): TAlgoCompress; overload;

Get the TAlgoCompress instance corresponding to the supplied AlgoID
- returns nil if no algorithm was identified
- stored content is identified as TAlgoSynLZ

class function Algo(Comp: PAnsiChar; CompLen: integer; out IsStored: boolean): TAlgoCompress; overload;

Get the TAlgoCompress instance corresponding to the AlgoID stored in the supplied compressed buffer
- returns nil if no algorithm was identified
- also identifies "stored" content in IsStored variable

function AlgoCompress(Plain: pointer; PlainLen: integer; Comp: pointer): integer; virtual; abstract;

This method will compress the supplied data

function AlgoCompressDestLen(PlainLen: integer): integer; virtual; abstract;

Get maximum possible (worse) compressed size for the supplied length

function AlgoDecompress(Comp: pointer; CompLen: integer; Plain: pointer): integer; virtual; abstract;

This method will decompress the supplied data

function AlgoDecompressDestLen(Comp: pointer): integer; virtual; abstract;

This method will return the size of the decompressed data

function AlgoDecompressPartial(Comp: pointer; CompLen: integer; Partial: pointer; PartialLen, PartialLenMax: integer): integer; virtual; abstract;

This method will partially and safely decompress the supplied data
- expects PartialLen <= result < PartialLenMax, depending on the algorithm

function AlgoHash(ForceHash32: boolean; Data: pointer; DataLen: integer): cardinal; overload;

Computes the digital signature of the buffer, or Hash32() if specified

function AlgoHash(Previous: cardinal; Data: pointer; DataLen: integer): cardinal; overload; virtual;

Computes by default the crc32c() digital signature of the buffer

function AlgoName: TShort16;

Returns the algorithm name, from its classname
- e.g. TAlgoSynLZ->'synlz' TAlgoLizard->'lizard' nil->'none' TAlgoDeflateFast->'deflatefast'

function Compress(Plain: PAnsiChar; PlainLen: integer; CompressionSizeTrigger: integer = 100; CheckMagicForCompressed: boolean = false; BufferOffset: integer = 0): RawByteString; overload; virtual;

Compress a memory buffer with crc32c hashing to a RawByteString

function Compress(const Plain: RawByteString; CompressionSizeTrigger: integer = 100; CheckMagicForCompressed: boolean = false; BufferOffset: integer = 0): RawByteString; overload;

Compress a memory buffer with crc32c hashing to a RawByteString

function Compress(Plain, Comp: PAnsiChar; PlainLen, CompLen: integer; CompressionSizeTrigger: integer = 100; CheckMagicForCompressed: boolean = false): integer; overload; virtual;

Compress a memory buffer with crc32c hashing
- supplied Comp buffer should contain at least CompressDestLen(PlainLen) bytes

function CompressDestLen(PlainLen: integer): integer;

Get maximum possible (worse) compressed size for the supplied length
- including the crc32c + algo 9 bytes header

function CompressToBytes(Plain: PAnsiChar; PlainLen: integer; CompressionSizeTrigger: integer = 100; CheckMagicForCompressed: boolean = false): TByteDynArray; overload;

Compress a memory buffer with crc32c hashing to a TByteDynArray

function CompressToBytes(const Plain: RawByteString; CompressionSizeTrigger: integer = 100; CheckMagicForCompressed: boolean = false): TByteDynArray; overload;

Compress a memory buffer with crc32c hashing to a TByteDynArray

function Decompress(const Comp: TByteDynArray): RawByteString; overload;

Uncompress a RawByteString memory buffer with crc32c hashing

function Decompress(const Comp: RawByteString; out PlainLen: integer; var tmp: RawByteString; Load: TAlgoCompressLoad = aclNormal): pointer; overload;

Uncompress a RawByteString memory buffer with crc32c hashing
- returns nil if crc32 hash failed, i.e. if the supplied Comp is not correct
- returns a pointer to the uncompressed data and fill PlainLen variable, after crc32c hash
- avoid any memory allocation in case of a stored content - otherwise, would uncompress to the tmp variable, and return pointer(tmp) and length(tmp)

function Decompress(Comp: PAnsiChar; CompLen: integer; out PlainLen: integer; var tmp: RawByteString; Load: TAlgoCompressLoad = aclNormal): pointer; overload;

Uncompress a RawByteString memory buffer with crc32c hashing
- returns nil if crc32 hash failed, i.e. if the supplied Data is not correct
- returns a pointer to an uncompressed data buffer of PlainLen bytes
- avoid any memory allocation in case of a stored content - otherwise, would uncompress to the tmp variable, and return pointer(tmp) and length(tmp)

function Decompress(const Comp: RawByteString; Load: TAlgoCompressLoad = aclNormal; BufferOffset: integer = 0): RawByteString; overload;

Uncompress a RawByteString memory buffer with crc32c hashing

function DecompressBody(Comp, Plain: PAnsiChar; CompLen, PlainLen: integer; Load: TAlgoCompressLoad = aclNormal): boolean; virtual;

Decode the content of a memory buffer compressed via the Compress() method
- PlainLen has been returned by a previous call to DecompressHeader()

function DecompressHeader(Comp: PAnsiChar; CompLen: integer; Load: TAlgoCompressLoad = aclNormal): integer; virtual;

Decode the header of a memory buffer compressed via the Compress() method
- validates the crc32c of the compressed data (unless Load=aclNoCrcFast), then return the uncompressed size in bytes, or 0 if the crc32c does not match
- should call DecompressBody() later on to actually retrieve the content

function DecompressPartial(Comp, Partial: PAnsiChar; CompLen, PartialLen, PartialLenMax: integer): integer; virtual;

Partial decoding of a memory buffer compressed via the Compress() method
- returns 0 on error, or how many bytes have been written to Partial
- will call virtual AlgoDecompressPartial() which is slower, but expected to avoid any buffer overflow on the Partial destination buffer
- some algorithms (e.g. Lizard) may need some additional bytes in the decode buffer, so PartialLenMax bytes should be allocated in Partial^, with PartialLenMax > expected PartialLen, and returned bytes may be > PartialLen, but always <= PartialLenMax

function FileCompress(const Source, Dest: TFileName; Magic: cardinal; ForceHash32: boolean = false; ChunkBytes: Int64 = 128 shl 20; WithTrailer: boolean = false): boolean; virtual;

Compress a file content using this compression algorithm
- source file is split into ChunkBytes blocks (128 MB by default) for fast in-memory compression of any file size, then compressed and including checksums of Source/Dest data
- it is not compatible with StreamCompress format, which has no chunking
- you should specify a Magic number to be used to identify the compressed file format
- follow the FileSynLZ() deprecated function format, if ForceHash32=true so that Hash32() is used instead of the AlgoHash() of this instance
- may be overriden to support a standard file layout (e.g. AlgoGZ)

class function FileIsCompressed(const Name: TFileName; Magic: cardinal): boolean; virtual;

Returns TRUE if the supplied file name is a compressed file, matching the Magic number as supplied to FileCompress() function
- follow the FileIsSynLZ() deprecated function format
- expects the compressed data to be at file beginning (not appended)
- may be overriden to support a standard file layout (e.g. AlgoGZ)

function FileUnCompress(const Source, Dest: TFileName; Magic: cardinal; ForceHash32: boolean = false): boolean; virtual;

Uncompress a file previously compressed via FileCompress()
- you should specify a Magic number to be used to identify the compressed file format
- follow the FileUnSynLZ() deprecated function format, if ForceHash32=true so that Hash32() is used instead of the AlgoHash() of this instance
- may be overriden to support a standard file layout (e.g. AlgoGZ)

function StreamCompress(Source: TStream; const DestFile: TFileName; Magic: cardinal; ForceHash32: boolean = false; WithTrailer: boolean = true; ChunkBytes: PtrInt = 4 shl 20): Int64; overload;

Compress a Stream content using this compression algorithm into a file
- just a wrapper around the overloaded StreamCompress() method

function StreamCompress(Source, Dest: TStream; Magic: cardinal; ForceHash32: boolean = false; WithTrailer: boolean = true; ChunkBytes: PtrInt = 4 shl 20): Int64; overload;

Compress a Stream content using this compression algorithm
- source Stream may be read and compressed by ChunkBytes = 4MB chunks
- a 32-bit Magic number identifies the compressed content chunks
- WithTrailer would finish the Dest output with a trailer block to locate the position of the compressed data, to be used e.g. when it is appended
- follow the StreamSynLZ() deprecated function format, if ForceHash32=true and WithTrailer=true so that Hash32() is used instead of AlgoHash()

function StreamComputeLen(P: PAnsiChar; Len: PtrUInt; Magic: cardinal): integer;

Compute the length of a given StreamCompress() buffer from its trailer
- allows to replace an existing appended content, for instance
- expects StreamCompress(WithTrailer=true) format

function StreamUnCompress(Source, Dest: TStream; Magic: cardinal; ForceHash32: boolean = false): boolean; overload;

Uncompress a Stream previously compressed via StreamCompress()
- you should specify a Magic number to be used to identify the compressed Stream format
- if Source is not positioned at compressed data beginning, a trailer is searched at the end of the Source stream to get the proper location
- returns true on success, and false on decoding error - but some chunks may have been decompressed in Dest even if false is returned

function StreamUnCompress(const Source: TFileName; Magic: cardinal; ForceHash32: boolean = false): TMemoryStream; overload;

Uncompress a File previously compressed via StreamCompress() as TStream
- you should specify a Magic number to be used to identify the compressed Stream format
- follow the StreamUnSynLZ() deprecated function format, if ForceHash32=true so that Hash32() is used instead of the AlgoHash() of this instance
- if the compressed data is not at Source file beginning, a trailer is searched at the end of the Source content to get the proper location

function StreamUnCompress(Source: TStream; Magic: cardinal; ForceHash32: boolean = false): TMemoryStream; overload;

Uncompress a Stream previously compressed via StreamCompress()
- return nil on decompression error, or a new TMemoryStream instance
- follow the StreamUnSynLZ() deprecated function format, if ForceHash32=true so that Hash32() is used instead of the AlgoHash() of this instance

function TryDecompress(const Comp: RawByteString; out Dest: RawByteString; Load: TAlgoCompressLoad = aclNormal): boolean;

Uncompress a RawByteString memory buffer with crc32c hashing
- returns TRUE on success

class function UncompressedSize(const Comp: RawByteString): integer;

Quickly validate a compressed buffer content, without uncompression
- extract the TAlgoCompress, and call DecompressHeader() to check the hash of the compressed data, and return then uncompressed size
- returns 0 on error (e.g. unknown algorithm or incorrect hash)

procedure Decompress(Comp: PAnsiChar; CompLen: integer; out result: RawByteString; Load: TAlgoCompressLoad = aclNormal; BufferOffset: integer = 0); overload;

Uncompress a memory buffer with crc32c hashing

property AlgoFileExt: TFileName read fAlgoFileExt;

The usual file extension of this algorithm
- e.g. '.synlz' or '.synz' or '.synliz' for SynLZ, Deflate or Lizard

property AlgoHasForcedFormat: boolean read fAlgoHasForcedFormat;

If this algorithm does not supports our custom storage format
- e.g. AlgoGZ set true and only supports plain buffers and files methods and would raise EAlgoCompress when stream methods are used

property AlgoID: byte read fAlgoID;

Contains a genuine byte identifier for this algorithm
- 0 is reserved for stored, 1 for TAlgoSynLz, 2/3 for TAlgoDeflate/Fast (in mormot.core.zip.pas), 4/5/6 for TAlgoLizard/Fast/Huffman (in mormot.lib.lizard.pas), 7/8 for TAlgoRleLZ/TAlgoRle, 9/10 for limited TAlgoGZ/TAlgoGZFast (in mormot.core.zip.pas)

1.3.4. TAlgoSynLZ

TAlgoSynLZ = class(TAlgoCompress)

Implement our fast SynLZ compression as a TAlgoCompress class
- please use the AlgoSynLZ global variable methods instead of the deprecated SynLZCompress/SynLZDecompress wrapper functions

constructor Create; override;

Set AlgoID = 1 as genuine byte identifier for SynLZ

function AlgoCompress(Plain: pointer; PlainLen: integer; Comp: pointer): integer; override;

Compress the supplied data using SynLZ

function AlgoCompressDestLen(PlainLen: integer): integer; override;

Get maximum possible (worse) SynLZ compressed size for the supplied length

function AlgoDecompress(Comp: pointer; CompLen: integer; Plain: pointer): integer; override;

Decompress the supplied data using SynLZ

function AlgoDecompressDestLen(Comp: pointer): integer; override;

Return the size of the SynLZ decompressed data

function AlgoDecompressPartial(Comp: pointer; CompLen: integer; Partial: pointer; PartialLen, PartialLenMax: integer): integer; override;

Partial (and safe) decompression of the supplied data using SynLZ

1.3.5. TAlgoCompressWithNoDestLen

TAlgoCompressWithNoDestLen = class(TAlgoCompress)

Abstract class storing the plain length before calling compression API
- some libraries (e.g. Deflate or Lizard) don't provide the uncompressed length from its output buffer - inherit from this class to store this value as ToVarUInt32, and override the RawProcess abstract protected method

function AlgoCompress(Plain: pointer; PlainLen: integer; Comp: pointer): integer; override;

Performs the compression, storing PlainLen and calling protected RawProcess

function AlgoDecompress(Comp: pointer; CompLen: integer; Plain: pointer): integer; override;

Performs the decompression, retrieving PlainLen and calling protected RawProcess

function AlgoDecompressDestLen(Comp: pointer): integer; override;

Return the size of the decompressed data (using FromVarUInt32)

function AlgoDecompressPartial(Comp: pointer; CompLen: integer; Partial: pointer; PartialLen, PartialLenMax: integer): integer; override;

Performs the decompression, retrieving PlainLen and calling protected RawProcess

1.3.6. TAlgoRleLZ

TAlgoRleLZ = class(TAlgoCompressWithNoDestLen)

Implement our SynLZ compression with RLE preprocess as a TAlgoCompress class
- SynLZ is very good with JSON or text, but not so efficient when its input has a lot of padding (e.g. a database file, or unoptimized raw binary)
- this class would make a first pass with RleCompress() before SynLZ
- if RLE has no effect during compression, will fallback to plain SynLZ with no RLE pass during decompression

constructor Create; override;

Set AlgoID = 7 as genuine byte identifier for RLE + SynLZ

function AlgoCompressDestLen(PlainLen: integer): integer; override;

Get maximum possible (worse) compressed size for the supplied length

1.3.7. TAlgoRle

TAlgoRle = class(TAlgoCompressWithNoDestLen)

Implement RLE compression as a TAlgoCompress class
- if RLE has no effect during compression, will fallback to plain store

constructor Create; override;

Set AlgoID = 8 as genuine byte identifier for RLE

function AlgoCompressDestLen(PlainLen: integer): integer; override;

Get maximum possible (worse) compressed size for the supplied length

1.3.8. EFastReader

EFastReader = class(ESynException)

Exception raised during TFastReader decoding

1.3.9. TFastReader

TFastReader = object(TObject)

Safe decoding of a TBufferWriter content from an in-memory buffer
- raise a EFastReader exception on decoding error (e.g. if a buffer overflow may occur) or call OnErrorOverflow/OnErrorData event handlers

CustomVariants: pointer;

When used to unserialize variants, stores options for TDocVariant creation
- contains a PDocVariantOptions reference pointer as defined in the mormot.core.data unit

Last: PAnsiChar;

The last position in the buffer

OnErrorData: TOnFastReaderErrorData;

Use this event to customize the ErrorData process

OnErrorOverflow: TOnFastReaderErrorOverflow;

Use this event to customize the ErrorOverflow process

P: PAnsiChar;

The current position in the memory

Tag: PtrInt;

Some opaque value, e.g. a version number to define the binary layout

function CopySafe(Dest: pointer; DataLen: PtrInt): boolean;

Copy data from the current position, and move ahead the specified bytes
- this version won't call ErrorOverflow, but return false on error
- returns true on read success

function EOF: boolean;

Returns TRUE if the current position is the end of the input stream

function Next(DataLen: PtrInt): pointer;

Returns the current position, and move ahead the specified bytes

function Next2: cardinal;

Read the next 2 bytes from the buffer as a 16-bit unsigned value

function Next2BigEndian: cardinal;

Read the next 2 bytes from the buffer as a 16-bit big-endian value

function Next4: cardinal;

Read the next 4 bytes from the buffer as a 32-bit unsigned value

function Next8: Qword;

Read the next 8 bytes from the buffer as a 64-bit unsigned value

function NextByte: byte;

Read the next byte from the buffer

function NextByteEquals(Value: byte): boolean;

Consumes the next byte from the buffer, if matches a given value

function NextByteSafe(dest: pointer): boolean;

Read the next byte from the buffer, checking

function NextSafe(out Data: pointer; DataLen: PtrInt): boolean;

Returns the current position, and move ahead the specified bytes

function PeekVarInt32(out value: PtrInt): boolean;

Try to read the next 32-bit signed value from the buffer
- don't change the current position

function PeekVarUInt32(out value: PtrUInt): boolean;

Try to read the next 32-bit unsigned value from the buffer
- don't change the current position

function ReadCompressed(Load: TAlgoCompressLoad = aclNormal; BufferOffset: integer = 0): RawByteString;

Retrieve some TAlgoCompress buffer, appended via Write()
- BufferOffset could be set to reserve some bytes before the uncompressed buffer

function ReadVarRawUtf8DynArray(var Values: TRawUtf8DynArray): PtrInt;

Retrieved RawUtf8 values encoded with TFileBufferWriter.WriteRawUtf8DynArray
- returns the number of items read into Values[] (may differ from length(Values))

function ReadVarUInt32Array(var Values: TIntegerDynArray): PtrInt;

Retrieved cardinal values encoded with TBufferWriter.WriteVarUInt32Array
- Values[] will be resized only if it is not long enough, to spare heap
- returns decoded count in Values[], which may not be length(Values)
- wkFakeMarker will return -count and the caller should make the decoding

function ReadVarUInt64Array(var Values: TInt64DynArray): PtrInt;

Retrieved Int64 values encoded with TBufferWriter.WriteVarUInt64DynArray
- Values[] will be resized only if it is not long enough, to spare heap
- returns decoded count in Values[], which may not be length(Values)

function RemainingLength: PtrUInt;

Returns remaining length (difference between Last and P)

function VarBlob: TValueResult; overload;

Read the next pointer and length value from the buffer

function VarBlobSafe(out Value: TValueResult): boolean;

Read the next pointer and length value from the buffer
- this version won't call ErrorOverflow, but return false on error
- returns true on read success

function VarInt32: integer;

Read the next 32-bit signed value from the buffer

function VarInt64: Int64;

Read the next 64-bit signed value from the buffer

function VarShortString: ShortString;

Read the next ShortString value from the buffer

function VarString: RawByteString; overload;

Read the next RawByteString value from the buffer

function VarString(CodePage: integer): RawByteString; overload;

Read the next RawByteString value from the buffer

function VarUInt32: cardinal;

Read the next 32-bit unsigned value from the buffer

function VarUInt32Safe(out Value: cardinal): boolean;

Read the next 32-bit unsigned value from the buffer
- this version won't call ErrorOverflow, but return false on error
- returns true on read success

function VarUInt64: QWord;

Read the next 64-bit unsigned value from the buffer

function VarUtf8: RawUtf8; overload;

Read the next RawUtf8 value from the buffer

function VarUtf8Safe(out Value: RawUtf8): boolean;

Read the next RawUtf8 value from the buffer
- this version won't call ErrorOverflow, but return false on error
- returns true on read success

procedure Copy(Dest: pointer; DataLen: PtrInt);

Copy data from the current position, and move ahead the specified bytes

procedure ErrorData(const msg: shortstring); overload;

Raise a EFastReader with "Incorrect Data: ...." error message

procedure ErrorData(const fmt: RawUtf8; const args: array of const); overload;

Raise a EFastReader with "Incorrect Data: ...." error message

procedure ErrorOverflow;

Raise a EFastReader with "Reached End of Input" error message

procedure Init(Buffer: pointer; Len: PtrInt); overload;

Initialize the reader from a memory block

procedure Init(const Buffer: RawByteString); overload;

Initialize the reader from a RawByteString content

procedure VarBlob(out result: TValueResult); overload;

Read the next pointer and length value from the buffer

procedure VarBlob(out Value: TSynTempBuffer); overload;

Copy the next VarBlob value from the buffer into a TSynTempBuffer

procedure VarNextInt(count: integer); overload;

Fast ignore the next count VarUInt32/VarInt32/VarUInt64/VarInt64 values
- don't raise any exception, so caller could check explicitly for any EOF

procedure VarNextInt; overload;

Fast ignore the next VarUInt32/VarInt32/VarUInt64/VarInt64 value
- don't raise any exception, so caller could check explicitly for any EOF

procedure VarUtf8(out result: RawUtf8); overload;

Read the next RawUtf8 value from the buffer

1.3.10. EBufferException

EBufferException = class(ESynException)

Exception raised during buffer processing

1.3.11. TBufferWriter

TBufferWriter = class(TObject)

This class can be used to speed up writing to a file or a stream
- big speed up if data is written in small blocks
- also handle optimized storage of any integer/Int64/RawUtf8 values
- use TFileBufferReader or TFastReader for decoding of the stored binary

constructor Create(aClass: TStreamClass; BufLen: integer = 4096); overload;

Initialize the buffer, using an owned TStream instance
- parameter could be e.g. TMemoryStream or TRawByteStringStream
- use Flush then TMemoryStream(Stream) to retrieve its content, or FlushTo if TRawByteStringStream was used
- Write() fails over 800MB (_STRMAXSIZE) for a TRawByteStringStream

constructor Create(aClass: TStreamClass; aTempBuf: pointer; aTempLen: integer); overload;

Initialize with a specified buffer and an owned TStream
- use a specified external buffer (which may be allocated on stack), to avoid a memory allocation
- aClass could be e.g. TMemoryStream or TRawByteStringStream

constructor Create(const aStackBuffer: TTextWriterStackBuffer); overload;

Initialize with a stack-allocated 8KB of buffer
- destination stream is an owned TRawByteStringStream - so you can call FlushTo to retrieve all written data
- Write() fails over 800MB (_STRMAXSIZE) for a TRawByteStringStream
- convenient to reduce heap presure, when writing a few KB of data

constructor Create(aStream: TStream; aTempBuf: pointer; aTempLen: integer); overload;

Initialize with a specified buffer and an existing TStream instance
- use a specified external buffer (which may be allocated on stack), to avoid a memory allocation

constructor Create(aFile: THandle; BufLen: integer = 65536); overload;

Initialize the buffer, and specify a file handle to use for writing
- define an internal buffer of the specified size

constructor Create(aStream: TStream; BufLen: integer = 65536); overload;

Initialize the buffer, and specify a TStream to use for writing
- define an internal buffer of the specified size

constructor Create(const aFileName: TFileName; BufLen: integer = 65536; Append: boolean = false); overload;

Initialize the buffer, and specify a file to use for writing
- define an internal buffer of the specified size
- would replace any existing file by default, unless Append is TRUE

destructor Destroy; override;

Release internal TStream (after AssignToHandle call)
- warning: an explicit call to Flush is needed to write the data pending in internal buffer

function DirectWritePrepare(maxlen: PtrInt; var tmp: RawByteString): PAnsiChar;

Allows to write directly to a memory buffer
- caller should specify the maximum possible number of bytes to be written
- then write the data to the returned pointer, and call DirectWriteFlush
- if len is bigger than the internal buffer, tmp will be used instead

function DirectWriteReserve(maxlen: PtrInt): PByte;

Allows to write directly to a memory buffer
- caller should specify the maximum possible number of bytes to be written
- len should be smaller than the internal buffer size (not checked)

function Flush: Int64;

Write any pending data in the internal buffer to the stream
- after a Flush, it's possible to call FileSeek64(aFile,....)
- returns the number of bytes written between two FLush method calls

function FlushAndCompress(nocompression: boolean = false; algo: TAlgoCompress = nil; BufferOffset: integer = 0): RawByteString;

Write any pending data, then call algo.Compress() on the buffer
- if algo is left to its default nil, will use global AlgoSynLZ
- features direct compression from internal buffer, if stream was not used
- BufferOffset could be set to reserve some bytes before the compressed buffer
- raise an exception if internal Stream is not a TRawByteStringStream

function FlushTo: RawByteString;

Write any pending data, then create a RawByteString from the content
- raise an exception if internal Stream is not a TRawByteStringStream

function FlushToBytes: TBytes;

Write any pending data, then create a TBytes array from the content
- raise an exception if the size exceeds 800MB (_DAMAXSIZE)

procedure CancelAll; virtual;

Rewind the Stream to the position when Create() was called
- note that this does not clear the Stream content itself, just move back its writing position to its initial place

procedure DirectWriteFlush(len: PtrInt; const tmp: RawByteString);

Finalize a direct write to a memory buffer
- by specifying the number of bytes written to the buffer

procedure DirectWriteReserved(pos: PByte);

Flush DirectWriteReserve() content

procedure Write(const Text: RawByteString); overload;

Append some length-prefixed UTF-8 text at the current position
- will write the string length (as VarUInt32), then the string content, as expected by the FromVarString() function
- is just a wrapper around WriteVar()

procedure Write(Data: pointer; DataLen: PtrInt); overload;

Append some data at the current position
- will be inlined as a MoveFast() most of the time

procedure Write1(Data: byte);

Append 1 byte of data at the current position

procedure Write2(Data: cardinal);

Append 2 bytes of data at the current position

procedure Write2BigEndian(Data: cardinal);

Append 2 bytes of data, encoded as BigEndian, at the current position

procedure Write4(Data: integer);

Append 4 bytes of data at the current position

procedure Write4BigEndian(Data: integer);

Append 4 bytes of data, encoded as BigEndian, at the current position

procedure Write8(Data8Bytes: pointer);

Append 8 bytes of data at the current position

procedure WriteBinary(const Data: RawByteString);

Append some content at the current position
- will write the binary data, without any length prefix

procedure WriteI64(Data: Int64);

Append 8 bytes of 64-bit integer at the current position

procedure WriteN(Data: byte; Count: integer);

Append the same byte a given number of occurrences at the current position

procedure WriteRawUtf8Array(Values: PPtrUIntArray; ValuesCount: integer);

Append a RawUtf8 array of values, from its low-level memory pointer
- handled the fixed size strings array case in a very efficient way

procedure WriteRawUtf8DynArray(const Values: TRawUtf8DynArray; ValuesCount: integer);

Append the RawUtf8 dynamic array
- handled the fixed size strings array case in a very efficient way

procedure WriteShort(const Text: ShortString);

Append some UTF-8 encoded text at the current position
- will write the string length (as VarUInt32), then the string content
- is just a wrapper around WriteVar()

procedure WriteStream(aStream: TCustomMemoryStream; aStreamSize: integer = -1);

Append a TStream content
- is StreamSize is left as -1, the Stream.Size is used
- the size of the content is stored in the resulting stream

procedure WriteVar(var Item: TTempUtf8); overload;

Append some TTempUtf8 text content prefixed by its encoded length
- will also release any memory stored in Item.TempRawUtf8

procedure WriteVar(Data: pointer; DataLen: PtrInt); overload;

Append some content (may be text or binary) prefixed by its encoded length
- will write DataLen as VarUInt32, then the Data content, as expected by FromVarString/FromVarBlob functions

procedure WriteVarInt32(Value: PtrInt);

Append an integer value using 32-bit variable-length integer encoding of the by-two complement of the given value

procedure WriteVarInt64(Value: Int64);

Append an integer value using 64-bit variable-length integer encoding of the by-two complement of the given value

procedure WriteVarUInt32(Value: PtrUInt);

Append a cardinal value using 32-bit variable-length integer encoding

procedure WriteVarUInt32Array(const Values: TIntegerDynArray; ValuesCount: integer; DataLayout: TBufferWriterKind);

Append cardinal values (NONE must be negative!) using 32-bit variable-length integer encoding or other specialized algorithm, depending on the data layout
- could be decoded later on via TFastReader.ReadVarUInt32Array

procedure WriteVarUInt32Values(Values: PIntegerArray; ValuesCount: integer; DataLayout: TBufferWriterKind);

Append cardinal values (NONE must be negative!) using 32-bit variable-length integer encoding or other specialized algorithms, depending on the data layout
- could be decoded later on via TFastReader.ReadVarUInt32Array

procedure WriteVarUInt64(Value: QWord);

Append an unsigned integer value using 64-bit variable-length encoding

procedure WriteVarUInt64DynArray(const Values: TInt64DynArray; ValuesCount: integer; Offset: boolean);

Append UInt64 values using 64-bit variable length integer encoding
- if Offset is TRUE, then it will store the difference between two values using 64-bit variable-length integer encoding (in this case, a fixed-sized record storage is also handled separately)
- could be decoded later on via TFastReader.ReadVarUInt64Array

procedure WriteXor(New, Old: PAnsiChar; Len: PtrInt; crc: PCardinal = nil);

Append "New[0..Len-1] xor Old[0..Len-1]" bytes
- as used e.g. by ZeroCompressXor/TSynBloomFilterDiff.SaveTo

property BufferPosition: PtrInt read fPos;

The current position in the internal buffer

property Stream: TStream read fStream;

The associated writing stream

property Tag: PtrInt read fTag write fTag;

Simple property used to store some integer content

property TotalWritten: Int64 read GetTotalWritten;

Get the byte count written since last Flush

1.3.12. TMultiPart

TMultiPart = record

Used by MultiPartFormDataDecode() to return one item of its data

1.3.13. TMemoryMapText

TMemoryMapText = class(TObject)

Able to read a UTF-8 text file using memory map
- much faster than TStringList.LoadFromFile()
- will ignore any trailing UTF-8 BOM in the file content, but will not expect one either

constructor Create(aFileContent: PUtf8Char; aFileSize: integer); overload;

Read an UTF-8 encoded text file content
- every line beginning is stored into LinePointers[]
- this overloaded constructor accept an existing memory buffer (some uncompressed data e.g.)

constructor Create(const aFileName: TFileName); overload;

Read an UTF-8 encoded text file
- every line beginning is stored into LinePointers[]

constructor Create; overload; virtual;

Initialize the memory mapped text file
- this default implementation just do nothing but is called by overloaded constructors so may be overriden to initialize an inherited class

destructor Destroy; override;

Release the memory map and internal LinePointers[]

function LineContains(const aUpperSearch: RawUtf8; aIndex: integer): boolean; virtual;

Returns TRUE if the supplied text is contained in the corresponding line

function LineSize(aIndex: integer): integer;

Retrieve the number of UTF-8 chars of the given line
- warning: no range check is performed about supplied index

function LineSizeSmallerThan(aIndex, aMinimalCount: integer): boolean;

Check if there is at least a given number of UTF-8 chars in the given line
- this is faster than LineSize(aIndex)<aMinimalCount for big lines

procedure AddInMemoryLine(const aNewLine: RawUtf8); virtual;

Add a new line to the already parsed content
- this line won't be stored in the memory mapped file, but stay in memory and appended to the existing lines, until this instance is released

procedure AddInMemoryLinesClear; virtual;

Clear all in-memory appended rows

procedure SaveToFile(FileName: TFileName; const Header: RawUtf8 = '');

Save the whole content into a specified file
- including any runtime appended values via AddInMemoryLine()
- an optional header text can be added to the beginning of the file

procedure SaveToStream(Dest: TStream; const Header: RawUtf8);

Save the whole content into a specified stream
- including any runtime appended values via AddInMemoryLine()

property Count: integer read fCount;

The number of text lines

property FileName: TFileName read fFileName write fFileName;

The file name which was opened by this instance

property LinePointers: PPointerArray read fLines;

Direct access to each text line
- use LineSize() method to retrieve line length, since end of line will NOT end with #0, but with #13 or #10
- warning: no range check is performed about supplied index

property Lines[aIndex: integer]: RawUtf8 read GetLine;

Retrieve a line content as UTF-8
- a temporary UTF-8 string is created
- will return '' if aIndex is out of range

property Map: TMemoryMap read fMap;

The memory map used to access the raw file content

property Strings[aIndex: integer]: string read GetString;

Retrieve a line content as RTL string type
- a temporary RTL string is created (after conversion for UNICODE Delphi)
- will return '' if aIndex is out of range

1.3.14. TProgressInfo

TProgressInfo = object(TObject)

Information about the progression of a process, e.g. for TStreamRedirect
- can also compute user-level text information from raw numbers

Context: RawUtf8;

Optional process context, e.g. a download URI, used for logging/progress

CurrentSize: Int64;

Number of bytes for the processed size

Elapsed: Int64;

Number of milliseconds elasped since process beginning

ExpectedSize: Int64;

Number of bytes for the final processed size
- may equal 0 if not known

OnLog: TSynLogProc;

Can be assigned from TSynLog.DoLog class method for low-level logging
- at least at process startup and finish, and every second (ReportDelay)

OnProgress: TOnInfoProgress;

Can be assigned to a TOnInfoProgress callback for high-level logging
- at least at process startup and finish, and every second (ReportDelay)

Percent: integer;

Percentage of CurrentSize versus ExpectedSize
- equals 0 if ExpectedSize is 0

PerSecond: PtrInt;

Number of bytes processed per second

ProcessedSize: Int64;

How many bytes have processed yet

Remaining: Int64;

Number of milliseconds remaining for full process, as estimated
- equals 0 if ExpectedSize is 0
- is just an estimation based on the average PerSecond speed

ReportDelay: Int64;

Number of milliseconds between each DoReport notification
- default is 1000, i.e. to notify OnLog/OnProgress every second

function DoReport(Sender: TObject; ReComputeElapsed: boolean): boolean;

Update the computed fields according to the curent state
- will be updated only every ReportDelay ms (default 1000 = every second)
- return false and compute nothing if ReportDelay has not been reached
- optionally call OnLog and OnProgress callbacks

function GetProgress: RawUtf8;

Retrieve the current status as simple text
- ready to be displayed on the console, e.g. as a single short line with no CRLF during process, and eventually with full information and ending CRLF

procedure DoAfter(Sender: TObject; ChunkSize: Int64);

Can be called

procedure DoStart(Sender: TObject; SizeExpected: Int64; const Ident: string);

Initialize the information for a new process
- once expected size and ident are set, caller should call DoAfter()

procedure Init;

Initialize the information, especially start the timing

procedure SetExpectedSize(SizeExpected, Position: Int64);

Called during process to setup ExpectedSize/ExpectedWrittenSize fields

1.3.15. EStreamRedirect

EStreamRedirect = class(ESynException)

Exception raised during TStreamRedirect processing

1.3.16. TStreamRedirect

TStreamRedirect = class(TStreamWithPosition)

An abstract pipeline stream able to redirect and hash read/written content
- can be used either Read() or Write() calls during its livetime
- hashing is performed on the fly during the Read/Write process
- it features also a callback to mark its progress
- can sleep during Read/Write to reach a LimitPerSecond average bandwidth

constructor Create(aRedirected: TStream; aRead: boolean = false); reintroduce; virtual;

Initialize the internal structure, and start the timing
- before calling Read/Write, you should set the Redirected property or specify aRedirected here - which will be owned by this instance
- if aRead is true, ExpectedSize is set from aRedirected.Size

destructor Destroy; override;

Release the associated Redirected stream

function GetHash: RawUtf8; virtual;

Return the current state of the hash as lower hexadecimal
- by default, will return '' meaning that no hashing algorithm was set

class function GetHashFileExt: RawUtf8; virtual;

Current algorithm name as file/url extension, e.g. '.md5' or '.sha256'
- by default, will return '' meaning that no hashing algorithm was set

class function GetHashName: RawUtf8;

Current algorithm name, from GetHashFileExt, e.g. 'md5' or 'sha256'

class function HashFile(const FileName: TFileName; const OnProgress: TOnStreamProgress = nil): RawUtf8;

Apply the internal hash algorithm to the supplied file content
- could be used ahead of time to validate a cached file

function Read(var Buffer; Count: Longint): Longint; override;

Update the hash and redirect the data to the associated TStream
- also trigger OnProgress at least every second
- will raise an error if Write() (or Append) have been called before

function Write(const Buffer; Count: Longint): Longint; override;

Update the hash and redirect the data to the associated TStream
- also trigger OnProgress at least every second
- will raise an error if Read() has been called before

procedure Append;

Update the hash of the existing Redirected stream content
- ready to Write() some new data after the existing

procedure Ended;

Notify end of process
- should be called explicitly when all Read()/Write() has been done

class procedure NotifyEnded( const OnStream: TOnStreamProgress; const OnInfo: TOnInfoProgress; const Fmt: RawUtf8; const Args: array of const; Size, StartedMs: Int64);

Notify a TOnStreamProgress callback that a process ended
- create a fake TStreamRedirect and call Ended with the supplied info

class procedure ProgressInfoToConsole(Sender: TObject; Info: PProgressInfo);

Can be used as TOnInfoProgress callback writing into the console

class procedure ProgressStreamToConsole(Sender: TStreamRedirect);

Can be used as TOnStreamProgress callback writing into the console

procedure Terminate;

Could be set from another thread to abort the streaming process
- will raise an exception at the next Read()/Write() call

property Context: RawUtf8 read fInfo.Context write fInfo.Context;

Optional process context, e.g. a download URI, used for logging/progress

property Elapsed: Int64 read fInfo.Elapsed;

Number of milliseconds elasped since beginning, as set by Read/Write

property ExpectedSize: Int64 read fInfo.ExpectedSize write SetExpectedSize;

You can specify a number of bytes for the final Redirected size
- will be used for the callback progress - could be left to 0 for Write() if size is unknown

property LimitPerSecond: PtrInt read fLimitPerSecond write fLimitPerSecond;

Can limit the Read/Write bytes-per-second bandwidth used, if not 0
- sleep so that PerSecond will keep close to this LimitPerSecond value

property OnInfoProgress: TOnInfoProgress read fInfo.OnProgress write fInfo.OnProgress;

Optional TOnInfoProgress callback triggered during Read/Write
- at least at process startup and finish, and every second / ReportDelay

property OnLog: TSynLogProc read fInfo.OnLog write fInfo.OnLog;

Can be assigned from TSynLog.DoLog class method for low-level logging

property OnProgress: TOnStreamProgress read fOnStreamProgress write fOnStreamProgress;

Optional TOnStreamProgress callback triggered during Read/Write
- at least at process startup and finish, and every second / ReportDelay

property Percent: integer read fInfo.Percent;

Percentage of Size versus ExpectedSize
- equals 0 if ExpectedSize is 0

property PerSecond: PtrInt read fInfo.PerSecond;

Number of bytes processed per second, since initialization of this instance

property ProcessedSize: Int64 read fInfo.ProcessedSize;

How many bytes have passed through Read() or Write()
- may not equal Size or Position after an Append - e.g. on resumed download from partial file

property Progress: RawUtf8 read GetProgress;

The current progression as text, as returned by ProgressStreamToConsole

property Redirected: TStream read fRedirected write fRedirected;

Specify a TStream to which any Read()/Write() will be redirected
- this TStream instance will be owned by the TStreamRedirect

property Remaining: Int64 read fInfo.Remaining;

Number of milliseconds remaining for full process, as set by Read/Write
- equals 0 if ExpectedSize is 0
- is just an estimation based on the average PerSecond speed

property ReportDelay: Int64 read fInfo.ReportDelay write fInfo.ReportDelay;

Number of milliseconds between each notification
- default is 1000, i.e. notify OnLog/OnProgress/OnInfoProgress every second

property TimeOut: Int64 read fTimeOut write fTimeOut;

Read/Write will raise an exception if not finished after TimeOut milliseconds

1.3.17. TStreamRedirectHasher

TStreamRedirectHasher = class(TStreamRedirect)

TStreamRedirect with 32-bit THasher checksum

1.3.18. TStreamRedirectCrc32c

TStreamRedirectCrc32c = class(TStreamRedirectHasher)

TStreamRedirect with crc32c 32-bit checksum

1.3.19. TFakeWriterStream

TFakeWriterStream = class(TStream)

A fake TStream, which will just count the number of bytes written

1.3.20. TNestedStreamReader

TNestedStreamReader = class(TStreamWithPositionAndSize)

TStream allowing to read from some nested TStream instances

destructor Destroy; override;

Finalize the nested TStream instance

function ForText: TRawByteStringStream;

Get the last TRawByteStringStream, or append a new one if needed

function NewStream(Stream: TStream): TStream;

Append a nested TStream instance
- you could use a TFileStreamEx here for efficient chunked reading

function Read(var Buffer; Count: Longint): Longint; override;

Will read up to Count bytes from the internal nested TStream

function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; override;

Overriden method to call Flush on rewind, i.e. if position is set to 0

function Write(const Buffer; Count: Longint): Longint; override;

This TStream is read-only: calling this method will raise an exception

procedure Append(const Content: RawByteString);

Append some text or content to an internal TRawByteStringStream
- is the easy way to append some text or data to the internal buffers

procedure Flush; virtual;

You should call this method before any Read() call
- is also called when you execute Seek(0, soBeginning)

1.3.21. TBufferedStreamReader

TBufferedStreamReader = class(TStreamWithPositionAndSize)

TStream with an internal memory buffer
- can be beneficial e.g. reading from a file by small chunks

constructor Create(const aSourceFileName: TFileName; aBufSize: integer = 65536); reintroduce; overload;

Initialize a source file and the internal buffer

constructor Create(aSource: TStream; aBufSize: integer = 65536); reintroduce; overload;

Initialize the source TStream and the internal buffer
- will also rewind the aSource position to its beginning, and retrieve its size

destructor Destroy; override;

Finalize this instance and its buffer

function Read(var Buffer; Count: Longint): Longint; override;

Will read up to Count bytes from the internal buffer or source TStream

function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64; override;

Overriden method to flush buffer on rewind

function Write(const Buffer; Count: Longint): Longint; override;

This TStream is read-only: calling this method will raise an exception

1.3.22. TRawByteStringGroupValue

TRawByteStringGroupValue = record

Item as stored in a TRawByteStringGroup instance

1.3.23. TRawByteStringGroup

TRawByteStringGroup = object(TObject)

Store several RawByteString content with optional concatenation

Count: integer;

How many items are currently stored in Values[]

LastFind: integer;

Naive but efficient cache for Find()

Position: integer;

The current size of data stored in Values[]

Values: TRawByteStringGroupValueDynArray;

Actual list storing the data

function AsBytes: TByteDynArray;

Return all content as a single TByteDynArray

function AsText: RawByteString;

Return all content as a single RawByteString
- will also compact the Values[] array into a single item (which is returned)

function Equals(const aAnother: TRawByteStringGroup): boolean;

Compare two TRawByteStringGroup instance stored text

function Find(aPosition: integer): PRawByteStringGroupValue; overload;

Returns a pointer to Values[] containing a given position
- returns nil if not found

function Find(aPosition, aLength: integer): pointer; overload;

Returns a pointer to Values[].Value containing a given position and length
- returns nil if not found

function FindAsText(aPosition, aLength: integer): RawByteString; overload;

Returns the text at a given position in Values[]
- text should be in a single Values[] entry

procedure Add(const aItem: RawByteString); overload;

Add a new item to Values[]

procedure Add(const aAnother: TRawByteStringGroup); overload;

Add another TRawByteStringGroup to Values[]

procedure Add(aItem: pointer; aItemLen: integer); overload;

Add a new item to Values[]

procedure AddFromReader(var aReader: TFastReader);

Add another TRawByteStringGroup previously serialized via WriteString()

procedure AppendTextAndClear(var aDest: RawByteString);

Append stored information into another RawByteString, and clear content

procedure Clear;

Clear any stored information

procedure Compact;

Compact the Values[] array into a single item
- is also used by AsText to compute a single RawByteString

procedure FindAsText(aPosition, aLength: integer; out aText: RawByteString); overload;

Returns the text at a given position in Values[]
- text should be in a single Values[] entry

procedure FindAsVariant(aPosition, aLength: integer; out aDest: variant);

Returns the text at a given position in Values[] via RawUtf8ToVariant()
- text should be in a single Values[] entry
- explicitly returns null if the supplied text was not found

procedure FindMove(aPosition, aLength: integer; aDest: pointer);

Copy the text at a given position in Values[]
- text should be in a single Values[] entry

procedure FindWrite(aPosition, aLength: integer; W: TTextWriter; Escape: TTextWriterKind = twJsonEscape; TrailingCharsToIgnore: integer = 0);

Append the text at a given position in Values[], JSON escaped by default
- text should be in a single Values[] entry

procedure FindWriteBase64(aPosition, aLength: integer; W: TTextWriter; withMagic: boolean);

Append the blob at a given position in Values[], Base64 encoded
- text should be in a single Values[] entry

procedure RemoveLastAdd;

Low-level method to abort the latest Add() call
- warning: will work only once, if an Add() has actually been just called: otherwise, the behavior is unexpected, and may wrongly truncate data

procedure Write(W: TTextWriter; Escape: TTextWriterKind = twJsonEscape); overload;

Save all content into a TJsonWriter instance

procedure WriteBinary(W: TBufferWriter); overload;

Save all content into a TBufferWriter instance

procedure WriteString(W: TBufferWriter);

Save all content as a string into a TBufferWriter instance
- storing the length as WriteVarUInt32() prefix

1.3.24. TRawByteStringHeap

TRawByteStringHeap = class(TObject)

Thread-safe reusable set of constant RawByteString/RawUtf8 instances
- all RawByteString will be constant (RefCnt=-2) with some max length
- maintain internally its own TLockedList O(1) set of instances
- warning: any call to New() should manually be followed by one Release()

constructor Create(aMaxLength: integer);

Initialize the internal cache for a given maximum length

destructor Destroy; override;

Release all cached instances

function Clean: PtrInt;

Release the RawByteString instances in the cache bin
- keep any existing New() instances intact
- returns how many memory has been released to the heap

procedure New(var aDest: RawByteString; aText: PUtf8Char = nil; aTextLen: TStrLen = 0; aCodePage: integer = CP_RAWBYTESTRING); overload;

Return a new RawByteString of a given length, with refcount=-2
- if aTextLen is its default 0, MaxLength will be used
- returned from its internal cache, unless aTextLen>MaxLength, which will allocate a regular RawByteString from heap

procedure New(var aDest: RawUtf8; aText: PUtf8Char; aTextLen: TStrLen); overload;

Return a new RawUtf8 of a given length, with refcount = -2

procedure Release(var aDest: pointer); overload;

Put back a RawByteString acquired from NewUtf8() into the internal cache

procedure Release(var aDest: RawByteString); overload;

Put back a RawByteString acquired from New() into the internal cache

procedure Release(var aDest: RawUtf8); overload;

Put back a RawUtf8 acquired from New() into the internal cache

property Count: integer read fOne.Count;

How many New() calls are currently active

property MaxLength: TStrLen read fMaxLength;

The maximum length() of RawByteString returned by New()

1.3.25. TRawByteStringBuffer

TRawByteStringBuffer = object(TObject)

Store one RawByteString content with an associated length
- to be used e.g. as a convenient reusable memory buffer

function Buffer: pointer;

A convenient wrapper to pointer(fBuffer) for direct Buffer/Len use

function Capacity: PtrInt;

How many bytes are currently allocated in the Buffer

function Extract(Dest: pointer; Count: PtrInt): PtrInt;

Move up to Count bytes from the internal Buffer into another place
- returns how many bytes were available to be copied into Dest^
- then remove the copied bytes from the internal Buffer/Len storage

function ExtractAt(var Dest: PAnsiChar; var Count: PtrInt; var Pos: PtrInt): PtrInt;

Move up to Count bytes from the internal Buffer into another place
- returns how many bytes were available to be copied into Dest^
- don't move any byte, but just update the given Pos index

function TryAppend(P: pointer; PLen: PtrInt): boolean;

Add some UTF-8 buffer content to the Buffer, without resizing it

procedure Append(const Text: array of RawUtf8); overload;

Add some UTF-8 string(s) content to the Buffer, resizing it if needed

procedure Append(Ch: AnsiChar); overload;

Just after Append/AppendShort, append one single character

procedure Append(const Text: RawUtf8); overload;

Add some UTF-8 string content to the Buffer, resizing it if needed

procedure Append(P: pointer; PLen: PtrInt); overload;

Add some UTF-8 buffer content to the Buffer, resizing it if needed

procedure Append(Value: QWord); overload;

Add some number as text content to the Buffer, resizing it if needed

procedure AppendCRLF;

Just after Append/AppendShort, append a #13#10 end of line

procedure AppendShort(const Text: ShortString);

Add some UTF-8 shortstring content to the Buffer, resizing it if needed

procedure AsText(out Text: RawUtf8; Overhead: PtrInt = 0; UseMainBuffer: boolean = false);

Retrieve the current Buffer/Len content as RawUtf8 text
- with some optional overhead for faster reallocmem at concatenation
- won't force Len to 0: caller should call Reset if done with it
- UseMainBuffer=true will return a copy of fBuffer into Text

procedure Clear;

Release/free the internal Buffer storage

procedure Remove(FirstBytes: PtrInt);

Similar to delete(fBuffer, 1, FirstBytes)

procedure Reserve(MaxSize: PtrInt); overload;

Ensure the internal Buffer has at least MaxSize bytes
- also reset the internal Len to 0

procedure Reserve(const WorkingBuffer: RawByteString); overload;

Use a specified string buffer as start

procedure Reset;

Set Len to 0, but doesn't clear/free the Buffer itself

property Len: PtrInt read fLen write fLen;

How many bytes are currently used in the Buffer

1.4. Types implemented in the mormot.core.buffers unit

1.4.1. PRawByteStringBuffer

PRawByteStringBuffer = ^TRawByteStringBuffer;

Pointer reference to a TRawByteStringBuffer

1.4.2. PRawByteStringGroup

PRawByteStringGroup = ^TRawByteStringGroup;

Pointer reference to a TRawByteStringGroup

1.4.3. TAlgoCompressLoad

TAlgoCompressLoad = ( aclNormal, aclSafeSlow, aclNoCrcFast );

Define the implementation used by TAlgoCompress.Decompress()

1.4.4. TBufferWriterKind

TBufferWriterKind = ( wkUInt32, wkVarUInt32, wkVarInt32, wkSorted, wkOffsetU, wkOffsetI, wkFakeMarker );

Available kind of integer array storage, corresponding to the data layout of TBufferWriter
- wkUInt32 will write the content as "plain" 4 bytes binary (this is the prefered way if the integers can be negative)
- wkVarUInt32 will write the content using our 32-bit variable-length integer encoding
- wkVarInt32 will write the content using our 32-bit variable-length integer encoding and the by-two complement (0=0,1=1,2=-1,3=2,4=-2...)
- wkSorted will write an increasing array of integers, handling the special case of a difference of similar value (e.g. 1) between two values - note that this encoding is efficient only if the difference is mainly < 253
- wkOffsetU and wkOffsetI will write the difference between two successive values, with detection of any constant difference (unsigned or signed)
- wkFakeMarker won't be used by WriteVarUInt32Array, but to notify a custom encoding

1.4.5. TEmoji

TEmoji = ( eNone, eGrinning, eGrin, eJoy, eSmiley, eSmile, eSweat_smile, eLaughing, eInnocent, eSmiling_imp, eWink, eBlush, eYum, eRelieved, eHeart_eyes, eSunglasses, eSmirk, eNeutral_face, eExpressionless, eUnamused, eSweat, ePensive, eConfused, eConfounded, eKissing, eKissing_heart, eKissing_smiling_eyes, eKissing_closed_eyes, eStuck_out_tongue, eStuck_out_tongue_winking_eye, eStuck_out_tongue_closed_eyes, eDisappointed, eWorried, eAngry, ePout, eCry, ePersevere, eTriumph, eDisappointed_relieved, eFrowning, eAnguished, eFearful, eWeary, eSleepy, eTired_face, eGrimacing, eSob, eOpen_mouth, eHushed, eCold_sweat, eScream, eAstonished, eFlushed, eSleeping, eDizzy_face, eNo_mouth, eMask, eSmile_cat, eJoy_cat, eSmiley_cat, eHeart_eyes_cat, eSmirk_cat, eKissing_cat, ePouting_cat, eCrying_cat_face, eScream_cat, eSlightly_frowning_face, eSlightly_smiling_face, eUpside_down_face, eRoll_eyes, eNo_good, oOk_woman, eBow, eSee_no_evil, eHear_no_evil, eSpeak_no_evil, eRaising_hand, eRaised_hands, eFrowning_woman, ePerson_with_pouting_face, ePray );

Map the first Unicode page of Emojis, from U+1F600 to U+1F64F
- naming comes from github/Markdown :identifiers:

1.4.6. TFileBufferWriter

TFileBufferWriter = TBufferWriter;

Deprecated alias to TBufferWriter binary serializer

1.4.7. THashFile

THashFile = function(const FileName: TFileName): RawUtf8;

Prototype of a file hashing function, returning its hexadecimal hash
- match HashFileCrc32c() below, HashFileCrc32() in mormot.core.zip, and HashFileMd5/HashFileSha* in mormot.crypt.secure functions signature

1.4.8. TLogEscape

TLogEscape = array[0..511] of AnsiChar;

512 bytes buffer to be allocated on stack when using LogEscape()

1.4.9. TMimeType

TMimeType = ( mtUnknown, mtPng, mtGif, mtTiff, mtJpg, mtBmp, mtDoc, mtPpt, mtXls, mtHtml, mtCss, mtJS, mtXIcon, mtFont, mtText, mtSvg, mtXml, mtWebp, mtManifest, mtJson, mtOgg, mtMp4, mtMp2, mtMpeg, mtH264, mtWma, mtWmv, mtAvi, mtGzip, mtWebm, mtRar, mt7z, mtZip, mtBz2, mtPdf, mtSQlite3, mtXcomp, mtDicom );

Some of the best-known mime types
- subset of the whole IANA list which can be quite huge (>1500 items)

1.4.10. TMultiPartDynArray

TMultiPartDynArray = array of TMultiPart;

Used by MultiPartFormDataDecode() to return all its data items

1.4.11. TOnFastReaderErrorData

TOnFastReaderErrorData = procedure(const fmt: RawUtf8; const args: array of const) of object;

Event signature to customize TFastReader.ErrorData notification

1.4.12. TOnFastReaderErrorOverflow

TOnFastReaderErrorOverflow = procedure of object;

Event signature to customize TFastReader.ErrorOverflow notification

1.4.13. TOnInfoProgress

TOnInfoProgress = procedure(Sender: TObject; Info: PProgressInfo) of object;

Callback definition to notify some TProgressInfo process
- see e.g. TStreamRedirect.ProgressInfoToConsole

1.4.14. TOnStreamCreate

TOnStreamCreate = function(const FileName: string; Mode: cardinal): TStream of object;

Optional callback as used e.g. by THttpClientSocketWGet.OnStreamCreate

1.4.15. TOnStreamProgress

TOnStreamProgress = procedure(Sender: TStreamRedirect) of object;

TStreamHasher.Write optional progression callback
- see Sender properties like Context/Size/PerSecond and ExpectedSize (which may be 0 if the download size is unknown)
- see e.g. TStreamRedirect.ProgressStreamToConsole

1.4.16. TRawByteStringGroupValueDynArray

TRawByteStringGroupValueDynArray = array of TRawByteStringGroupValue;

Items as stored in a TRawByteStringGroup instance

1.4.17. TStreamRedirectClass

TStreamRedirectClass = class of TStreamRedirect;

Meta-class of TStreamRedirect hierarchy

1.4.18. TTextWriterHtmlEscape

TTextWriterHtmlEscape = set of ( heHtmlEscape, heEmojiToUtf8);

Tune AddHtmlEscapeWiki/AddHtmlEscapeMarkdown wrapper functions process
- heHtmlEscape will escape any HTML special chars, e.g. & into &
- heEmojiToUtf8 will convert any Emoji text into UTF-8 Unicode character, recognizing e.g. :joy: or :) in the text

1.5. Constants implemented in the mormot.core.buffers unit

1.5.1. ALGO_SAFE

ALGO_SAFE: array[boolean] of TAlgoCompressLoad = ( aclNormal, aclSafeSlow);

Used e.g. as when ALGO_SAFE[SafeDecompression] for TAlgoCompress.Decompress

1.5.2. JSON_BASE64_MAGIC_C

JSON_BASE64_MAGIC_C = $b0bfef;

UTF-8 encoded \uFFF0 special code to mark Base64 binary content in JSON
- Unicode special char U+FFF0 is UTF-8 encoded as EF BF B0 bytes
- as generated by BinToBase64WithMagic() functions, and expected by the TExtractInlineParameters decoder
- used e.g. when transmitting TDynArray.SaveTo() content

1.5.3. JSON_BASE64_MAGIC_QUOTE_C

JSON_BASE64_MAGIC_QUOTE_C = ord('"') + cardinal(JSON_BASE64_MAGIC_C) shl 8;

'"' + UTF-8 encoded \uFFF0 special code to mark Base64 binary in JSON

1.5.4. JSON_BASE64_MAGIC_QUOTE_S

JSON_BASE64_MAGIC_QUOTE_S: string[4] = '"'#$ef#$bf#$b0;

'"' + UTF-8 encoded \uFFF0 special code to mark Base64 binary in JSON
- defined as a ShortString constant to be used as:

 AddShorter(JSON_BASE64_MAGIC_QUOTE_S);

1.5.5. JSON_BASE64_MAGIC_S

JSON_BASE64_MAGIC_S: string[3] = #$ef#$bf#$b0;

1.5.6. LOG_MAGIC

LOG_MAGIC = $ABA51051;

The "magic" number used to identify .log.synlz compressed files, as created by EventArchiveSynLZ / EventArchiveLizard callbacks

1.5.7. MIME_TYPE

MIME_TYPE: array[TMimeType] of RawUtf8 = ( '', 'image/png', 'image/gif', 'image/tiff', JPEG_CONTENT_TYPE, 'image/bmp', 'application/msword', 'application/vnd.ms-powerpoint', 'application/vnd.ms-excel', HTML_CONTENT_TYPE, 'text/css', 'text/javascript', 'image/x-icon', 'font/woff', TEXT_CONTENT_TYPE, 'image/svg+xml', XML_CONTENT_TYPE, 'image/webp', 'text/cache-manifest', JSON_CONTENT_TYPE, 'video/ogg', 'video/mp4', 'video/mp2', 'audio/mpeg', 'video/H264', 'audio/x-ms-wma', 'video/x-ms-wmv', 'video/x-msvideo', 'application/gzip', 'video/webm', 'application/x-rar-compressed', 'application/x-7z-compressed', 'application/zip', 'application/bzip2', 'application/pdf', 'application/x-sqlite3', 'application/x-compress', 'application/dicom');

The known mime types text representation

1.5.8. PLURAL_FORM

PLURAL_FORM: array[boolean] of RawUtf8 = ( '', 's');

Can be used to append to most English nouns to form a plural
- as used by the Plural() function

1.5.9. SYNLZTRIG

SYNLZTRIG: array[boolean] of integer = ( 100, maxInt);

CompressionSizeTrigger parameter SYNLZTRIG[true] will disable then SynLZCompress() compression

1.6. Functions or procedures implemented in the mormot.core.buffers unit

Functions or procedures	Description
AddHtmlEscapeMarkdown	Convert minimal Markdown text into proper HTML
AddHtmlEscapeWiki	Convert some wiki-like text into proper HTML
Append999ToBuffer	Fast add text conversion of 0-999 integer value into a given buffer
AppendBuffersToRawUtf8	Fast add some characters to a RawUtf8 string
AppendBufferToBuffer	Fast add some characters from ane buffer into another buffer
AppendBufferToRawByteString	Just redirect to mormot.core.text Append(...) overloads
AppendCharOnceToRawUtf8	Fast add one character to a RawUtf8 string, if not already present
AppendRawUtf8ToBuffer	Fast add some characters from a RawUtf8 string into a given buffer
AppendUInt32ToBuffer	Fast add text conversion of a 32-bit signed integer value into a given buffer
AreUrlValid	Checks if the supplied UTF-8 text values don't need URI encoding
AsciiToBaudot	Convert some ASCII-7 text into binary, using Emile Baudot code
AsciiToBaudot	Convert some ASCII-7 text into binary, using Emile Baudot code
Base32ToBin	Conversion from Base32 encoded text into a binary string
Base32ToBin	Conversion from Base32 encoded text into a binary string
Base58ToBin	Conversion from Base58 encoded text into a binary string
Base58ToBin	Conversion from Base58 encoded text into a binary string
Base58ToBin	Conversion from Base58 encoded text into a binary buffer
Base64Decode	Direct low-level decoding of a Base64 encoded buffer
Base64Encode	Raw function for efficient binary to Base64 encoding
Base64EncodeTrailing	Raw function for efficient binary to Base64 encoding of the last bytes
Base64MagicCheckAndDecode	Check and decode '\uFFF0base64encodedbinary' content into binary
Base64MagicCheckAndDecode	Check and decode '\uFFF0base64encodedbinary' content into binary
Base64MagicCheckAndDecode	Check and decode '\uFFF0base64encodedbinary' content into binary
Base64MagicDecode	Just a wrapper around Base64ToBin() for in-place decode of JSON_BASE64_MAGIC_C '\uFFF0base64encodedbinary' content into binary
Base64MagicToBlob	Convert a Base64-encoded content into binary hexadecimal ready for SQL
Base64MagicTryAndDecode	Decode '\uFFF0base64encodedbinary' or 'base64encodedbinary' into binary
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBin	Fast conversion from Base64 encoded text into binary data
Base64ToBinLength	Retrieve the expected undecoded length of a Base64 encoded buffer
Base64ToBinLengthSafe	Retrieve the expected undecoded length of a Base64 encoded buffer
Base64ToBinSafe	Fast conversion from Base64 encoded text into binary data
Base64ToBinSafe	Fast conversion from Base64 encoded text into binary data
Base64ToBinSafe	Fast conversion from Base64 encoded text into binary data
Base64ToBinSafe	Fast conversion from Base64 encoded text into binary data
Base64ToBinTrim	Conversion from Base64 encoded text into binary data, ignoring spaces
Base64ToUri	Conversion from any Base64 encoded value into URI-compatible encoded text
Base64uriDecode	Direct low-level decoding of a Base64-URI encoded buffer
Base64uriEncode	Low-level conversion from a binary buffer into Base64-like URI-compatible encoded text
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBin	Fast conversion from Base64-URI encoded text into binary data
Base64uriToBinLength	Retrieve the expected undecoded length of a Base64-URI encoded buffer
BaudotToAscii	Convert some Baudot code binary, into ASCII-7 text
BaudotToAscii	Convert some Baudot code binary, into ASCII-7 text
BinToBase32	Conversion from a binary buffer into Base32 encoded text as RawUtf8
BinToBase32	Conversion from a binary buffer into Base32 encoded text as RawUtf8
BinToBase32	Conversion from a binary buffer into Base32 encoded text buffer
BinToBase32Length	Compute the length resulting of Base32 encoding of a binary buffer
BinToBase58	Conversion from a binary buffer into Base58 encoded text as TSynTempBuffer
BinToBase58	Conversion from a binary buffer into Base58 encoded text as RawUtf8
BinToBase58	Conversion from a binary buffer into Base58 encoded text as RawUtf8
BinToBase64	Fast conversion from binary data into prefixed/suffixed Base64 encoded UTF-8 text
BinToBase64	Fast conversion from binary data into Base64 encoded UTF-8 text
BinToBase64	Fast conversion from binary data into Base64 encoded UTF-8 text
BinToBase64Length	Retrieve the expected encoded length after Base64 process
BinToBase64Line	Fast conversion from binary into prefixed/suffixed Base64 with 64 chars per line
BinToBase64Short	Fast conversion from a small binary data into Base64 encoded UTF-8 text
BinToBase64Short	Fast conversion from a small binary data into Base64 encoded UTF-8 text
BinToBase64uri	Fast conversion from a binary buffer into Base64-like URI-compatible encoded text
BinToBase64uri	Fast conversion from binary data into Base64-like URI-compatible encoded text
BinToBase64uriLength	Retrieve the expected encoded length after Base64-URI process
BinToBase64uriShort	Fast conversion from a binary buffer into Base64-like URI-compatible encoded ShortString
BinToBase64WithMagic	Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)
BinToBase64WithMagic	Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)
BinToBase64WithMagic	Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)
BinToHumanHex	Generate some 'xx:xx:xx:xx' output buffer with left and right margins
BinToHumanHex	Generate some 'xx:xx:xx:xx' output buffer with left and right margins
BinToSource	Generate some pascal source code holding some data binary as constant
BinToSource	Generate some pascal source code holding some data binary as constant
BinToSource	Generate some pascal source code holding some data binary as constant
BlobToBytes	Create a TBytes from TEXT-encoded blob data
BlobToRawBlob	Fill a RawBlob from TEXT-encoded blob data
BlobToRawBlob	Fill a RawBlob from TEXT-encoded blob data
BlobToRawBlob	Fill a RawBlob from TEXT-encoded blob data
BlobToStream	Create a memory stream from TEXT-encoded blob data
BytesToRawByteString	Creates a RawByteString memory buffer from a TBytes content
EmojiFromDots	Conversion of github/Markdown :identifiers: into UTF-8 Emoji sequences
EmojiFromDots	Low-level conversion of github/Markdown :identifiers: into UTF-8 Emoji sequences
EmojiFromText	Recognize github/Markdown compatible text of Emojis
EmojiParseDots	Low-level parser of github/Markdown compatible text of Emojis
EmojiToDots	Conversion of UTF-8 Emoji sequences into github/Markdown :identifiers:
EmojiToDots	Low-level conversion of UTF-8 Emoji sequences into github/Markdown :identifiers:
EscapeBuffer	Low-level fast conversion from binary data to escaped text
EscapeToShort	Fill a ShortString with the (hexadecimal) chars of the input text/binary
EscapeToShort	Fill a ShortString with the (hexadecimal) chars of the input text/binary
FromVarBlob	Retrieve pointer and length to a variable-length text/blob buffer
FromVarInt32	Convert a 32-bit variable-length integer buffer into an integer
FromVarInt64	Convert a 64-bit variable-length integer buffer into a Int64
FromVarInt64Value	Convert a 64-bit variable-length integer buffer into a Int64
FromVarString	Retrieve a variable-length text buffer
FromVarString	Retrieve a variable-length UTF-8 encoded text buffer in a temporary buffer
FromVarString	Retrieve a variable-length UTF-8 encoded text buffer in a temporary buffer
FromVarString	Safe retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8
FromVarString	Retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8
FromVarString	Retrieve a variable-length text buffer
FromVarString	Retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8
FromVarUInt32	Convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt32	Convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt32Big	Convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt32High	Convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt32Safe	Safely convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt32Up128	Convert a 32-bit variable-length integer buffer into a cardinal
FromVarUInt64	Convert a 64-bit variable-length integer buffer into a UInt64
FromVarUInt64	Convert a 64-bit variable-length integer buffer into a UInt64
FromVarUInt64Safe	Safely convert a 64-bit variable-length integer buffer into a UInt64
GetJpegSize	Fast guess of the size, in pixels, of a JPEG memory buffer
GetMimeContentType	Retrieve the MIME content type from its file name or a supplied binary buffer
GetMimeContentTypeFromBuffer	Retrieve the MIME content type from a supplied binary buffer
GetMimeContentTypeFromExt	MtUnknown mtPng mtGif mtTiff mtJpg mtBmp mtDoc mtPpt mtXls mtHtml mtCss mtJS RFC 9239 mtXIcon mtFont RFC 8081 mtText mtSvg mtXml mtWebp mtManifest mtJson mtOgg RFC 5334 mtMp4 RFC 4337 6381 mtMp2 mtMpeg RFC 3003 mtH264 RFC 6184 mtWma mtWmv mtAvi mtGzip mtWebm mtRar mt7z mtZip mtBz2 mtPdf mtSQlite3 mtXcomp mtDicom retrieve the MIME content type from its file name
GetMimeContentTypeFromMemory	Retrieve the MIME content type from a supplied binary buffer
GetMimeContentTypeHeader	Retrieve the HTTP header for MIME content type from a supplied binary buffer
GetMimeTypeFromExt	Retrieve the MIME content type from its file extension text (without '.')
GetStreamBuffer	Retrieve the memory buffer of a TCustomMemoryStream/TRawByteStringStream
GotoNextVarInt	Jump a value in the 32-bit or 64-bit variable-length integer buffer
GotoNextVarString	Jump a value in variable-length text buffer
HashFile	Compute the 32-bit default hash of a file content
HashFileCrc32c	Compute the crc32c checksum of a given file
HtmlEscapeMarkdown	Escape some Markdown-marked text into HTML
HtmlEscapeWiki	Escape some wiki-marked text into HTML
IncludeTrailingUriDelimiter	Ensure the supplied URI contains a trailing '/' charater
IsBase64	Check if the supplied text is a valid Base64 encoded stream
IsBase64	Check if the supplied text is a valid Base64 encoded stream
isBlobHex	Return true if the TEXT is encoded as SQLite3 BLOB literals (X'53514C697465' e.g.)
IsContentCompressed	Retrieve if some content is compressed, from a supplied binary buffer
IsContentTypeCompressible	Recognize e.g. 'text/css' or 'application/json' as compressible
IsContentTypeCompressibleU	Recognize e.g. 'text/css' or 'application/json' as compressible
IsStreamBuffer	Check if class is a TCustomMemoryStream/TRawByteStringStream
IsUrlValid	Checks if the supplied UTF-8 text don't need URI encoding
LogEscape	Fill TLogEscape stack buffer with the (hexadecimal) chars of the input binary
LogEscapeFull	Returns a text buffer with the (hexadecimal) chars of the input binary
LogEscapeFull	Returns a text buffer with the (hexadecimal) chars of the input binary
MultiPartFormDataAddField	Encode a field in a multipart array
MultiPartFormDataAddFile	Encode a file in a multipart array
MultiPartFormDataDecode	Decode multipart/form-data POST request content into memory
MultiPartFormDataEncode	Encode multipart fields and files
MultiPartFormDataNewBound	Used e.g. by MultiPartFormDataEncode and THttpMultiPartStream.Add
Plural	Write count number and append 's' (if needed) to form a plural English noun
RawBlobToBlob	Creates a TEXT-encoded version of blob data from a memory data
RawBlobToBlob	Creates a TEXT-encoded version of blob data from a RawBlob
RawByteStringArrayConcat	Fast concatenation of several AnsiStrings
RawByteStringToBytes	Creates a TBytes from a RawByteString memory buffer
ResourceSynLZToRawByteString	Creates a RawByteString memory buffer from an SynLZ-compressed embedded resource
ResourceToRawByteString	Creates a RawByteString memory buffer from an embedded resource
SameFileContent	Compare two files by content, reading them by blocks
ToVarInt32	Convert an integer into a 32-bit variable-length integer buffer
ToVarInt64	Convert a Int64 into a 64-bit variable-length integer buffer
ToVarString	Convert a RawUtf8 into an UTF-8 encoded variable-length buffer
ToVarUInt32	Convert a cardinal into a 32-bit variable-length integer buffer
ToVarUInt32Length	Return the number of bytes necessary to store a 32-bit variable-length integer
ToVarUInt32LengthWithData	Return the number of bytes necessary to store some data with a its 32-bit variable-length integer length
ToVarUInt64	Convert a UInt64 into a 64-bit variable-length integer buffer
UrlDecode	Decode a UrlEncode() URI encoded parameter into its original value
UrlDecode	Decode a UrlEncode() URI encoded parameter into its original value
UrlDecodeCardinal	Decode a specified parameter compatible with URI encoding into its original cardinal numerical value
UrlDecodeDouble	Decode a specified parameter compatible with URI encoding into its original floating-point value
UrlDecodeExtended	Decode a specified parameter compatible with URI encoding into its original floating-point value
UrlDecodeInt64	Decode a specified parameter compatible with URI encoding into its original Int64 numerical value
UrlDecodeInteger	Decode a specified parameter compatible with URI encoding into its original integer numerical value
UrlDecodeName	Decode a UrlEncodeName() URI encoded network name into its original value
UrlDecodeName	Decode a UrlEncodeName() URI encoded network name into its original value
UrlDecodeNeedParameters	Returns TRUE if all supplied parameters do exist in the URI encoded text
UrlDecodeNextName	Decode a URI-encoded Name from an input buffer
UrlDecodeNextNameValue	Decode the next Name=Value&.... pair from input URI
UrlDecodeNextValue	Decode a URI-encoded Value from an input buffer
UrlDecodeValue	Decode a specified parameter compatible with URI encoding into its original textual value
UrlDecodeVar	Decode a UrlEncode/UrlEncodeName() URI encoded string into its original value
UrlEncode	Encode supplied parameters to be compatible with URI encoding
UrlEncode	Encode a string as URI parameter encoding, i.e. ' ' as '+'
UrlEncode	Encode a string as URI parameter encoding, i.e. ' ' as '+'
UrlEncode	Append a string as URI parameter encoding, i.e. ' ' as '+'
UrlEncode	Append a string as URI parameter encoding, i.e. ' ' as '+'
UrlEncodeName	Encode a string as URI network name encoding, i.e. ' ' as %20
UrlEncodeName	Encode a string as URI network name encoding, i.e. ' ' as %20
UrlEncodeName	Append a string as URI network name encoding, i.e. ' ' as %20
UrlEncodeName	Append a string as URI network name encoding, i.e. ' ' as %20

1.6.1. AddHtmlEscapeMarkdown

procedure AddHtmlEscapeMarkdown(W: TTextWriter; P: PUtf8Char; esc: TTextWriterHtmlEscape = [heEmojiToUtf8]);

Convert minimal Markdown text into proper HTML
- see https://enterprise.github.com/downloads/en/markdown-cheatsheet.pdf
- convert all #13#10 into ..., *..* into .., **..** into .., `...` into <code>...</code>, backslash espaces \\ \* \_ and so on, [title](http://...) and detect plain http:// as <a href=...>
- create unordered lists from trailing * + - chars, blockquotes from trailing > char, and code line from 4 initial spaces
- as with default Markdown, won't escape HTML special chars (i.e. you can write plain HTML in the supplied text) unless esc is set otherwise
- only inline-style links and images are supported yet (not reference-style); tables aren't supported either

1.6.2. AddHtmlEscapeWiki

procedure AddHtmlEscapeWiki(W: TTextWriter; P: PUtf8Char; esc: TTextWriterHtmlEscape = [heHtmlEscape, heEmojiToUtf8]);

Convert some wiki-like text into proper HTML
- convert all #13#10 into ..., *..* into .., +..+ into .., `..` into <code>..</code>, and http://... as <a href=http://...>
- escape any HTML special chars, and Emoji tags as specified with esc

1.6.3. Append999ToBuffer

function Append999ToBuffer(Buffer: PUtf8Char; Value: PtrUInt): PUtf8Char;

Fast add text conversion of 0-999 integer value into a given buffer
- warning: it won't check that Value is in 0-999 range
- up to 4 bytes may be written to the buffer (including trailing #0)

1.6.4. AppendBuffersToRawUtf8

procedure AppendBuffersToRawUtf8(var Text: RawUtf8; const Buffers: array of PUtf8Char);

Fast add some characters to a RawUtf8 string
- faster than Text := Text+RawUtf8(Buffers[0])+RawUtf8(Buffers[0])+...
- will handle up to 64 Buffers[] - raise an ESynException on too many Buffers

1.6.5. AppendBufferToBuffer

function AppendBufferToBuffer(Buffer: PUtf8Char; Text: pointer; Len: PtrInt): PUtf8Char;

Fast add some characters from ane buffer into another buffer
- warning: the Buffer should contain enough space to store the Text, otherwise you may encounter buffer overflows and random memory errors

1.6.6. AppendBufferToRawByteString

procedure AppendBufferToRawByteString(var Content: RawByteString; const Buffer; BufferLen: PtrInt); overload;

Just redirect to mormot.core.text Append(...) overloads

1.6.7. AppendCharOnceToRawUtf8

procedure AppendCharOnceToRawUtf8(var Text: RawUtf8; Ch: AnsiChar);

Fast add one character to a RawUtf8 string, if not already present
- avoid a temporary memory allocation of a string, so faster alternative to

 if (Text<>'') and (Text[length(Text)]<>Ch) then Text := Text + ch;

1.6.8. AppendRawUtf8ToBuffer

function AppendRawUtf8ToBuffer(Buffer: PUtf8Char; const Text: RawUtf8): PUtf8Char;

Fast add some characters from a RawUtf8 string into a given buffer
- warning: the Buffer should contain enough space to store the Text, otherwise you may encounter buffer overflows and random memory errors

1.6.9. AppendUInt32ToBuffer

function AppendUInt32ToBuffer(Buffer: PUtf8Char; Value: PtrUInt): PUtf8Char;

Fast add text conversion of a 32-bit signed integer value into a given buffer
- warning: the Buffer should contain enough space to store the text, otherwise you may encounter buffer overflows and random memory errors

1.6.10. AreUrlValid

function AreUrlValid(const Url: array of RawUtf8): boolean;

Checks if the supplied UTF-8 text values don't need URI encoding
- returns TRUE if all its chars of all strings are non-void plain ASCII-7 RFC compatible identifiers (0..9a..zA..Z-_.~)

1.6.11. AsciiToBaudot

function AsciiToBaudot(const Text: RawUtf8): RawByteString; overload;

Convert some ASCII-7 text into binary, using Emile Baudot code
- as used in telegraphs, covering #10 #13 #32 a-z 0-9 - ' , ! : ( + ) $ ? @ . / ; charset, following a custom static-huffman-like encoding with 5-bit masks
- any upper case char will be converted into lowercase during encoding
- other characters (e.g. UTF-8 accents, or controls chars) will be ignored
- resulting binary will consume 5 (or 10) bits per character
- reverse of the BaudotToAscii() function
- the "baud" symbol rate measurement comes from Emile's name ;)

1.6.12. AsciiToBaudot

function AsciiToBaudot(P: PAnsiChar; len: PtrInt): RawByteString; overload;

1.6.13. Base32ToBin

function Base32ToBin(const base32: RawUtf8): RawByteString; overload;

Conversion from Base32 encoded text into a binary string
- RFC4648 Base32 is defined as upper alphanumeric without misleading 0O 1I 8B
- returns '' if input was not valid Base32 encoded

1.6.14. Base32ToBin

function Base32ToBin(B32: PAnsiChar; B32Len: integer): RawByteString; overload;

Conversion from Base32 encoded text into a binary string
- RFC4648 Base32 is defined as upper alphanumeric without misleading 0O 1I 8B
- returns '' if input was not valid Base32 encoded

1.6.15. Base58ToBin

function Base58ToBin(B58: PAnsiChar; B58Len: integer; var Dest: TSynTempBuffer): integer; overload;

Conversion from Base58 encoded text into a binary buffer
- Bitcoin' Base58 was defined as alphanumeric chars without misleading 0O I1
- Base58 is much slower than Base64, performing in O(n^2) instead of O(n), and should not be used on big buffers
- returns the number of decoded chars encoded into Dest.buf
- caller should call Dest.Done once it is finished with the output binary

1.6.16. Base58ToBin

function Base58ToBin(B58: PAnsiChar; B58Len: integer): RawByteString; overload;

Conversion from Base58 encoded text into a binary string
- Bitcoin' Base58 was defined as alphanumeric chars without misleading 0O I1
- Base58 is much slower than Base64, and should not be used on big buffers
- returns '' if input was not valid Base58 encoded

1.6.17. Base58ToBin

function Base58ToBin(const base58: RawUtf8): RawByteString; overload;

1.6.18. Base64Decode

function Base64Decode(sp, rp: PAnsiChar; len: PtrInt): boolean;

Direct low-level decoding of a Base64 encoded buffer
- here len is the number of 4 chars chunks in sp input
- deprecated low-level function: use Base64ToBin/Base64ToBinSafe instead

1.6.19. Base64Encode

procedure Base64Encode(rp, sp: PAnsiChar; len: cardinal);

Raw function for efficient binary to Base64 encoding
- just a wrapper around Base64EncodeMain() + Base64EncodeTrailing()

1.6.20. Base64EncodeTrailing

procedure Base64EncodeTrailing(rp, sp: PAnsiChar; len: cardinal);

Raw function for efficient binary to Base64 encoding of the last bytes
- don't use this function, but rather the BinToBase64() overloaded functions

1.6.21. Base64MagicCheckAndDecode

function Base64MagicCheckAndDecode(Value: PUtf8Char; var Blob: RawByteString): boolean; overload;

Check and decode '\uFFF0base64encodedbinary' content into binary
- this method will check the supplied value to match the expected JSON_BASE64_MAGIC_C pattern, decode and set Blob and return TRUE

1.6.22. Base64MagicCheckAndDecode

function Base64MagicCheckAndDecode(Value: PUtf8Char; ValueLen: integer; var Blob: RawByteString): boolean; overload;

Check and decode '\uFFF0base64encodedbinary' content into binary
- this method will check the supplied value to match the expected JSON_BASE64_MAGIC_C pattern, decode and set Blob and return TRUE

1.6.23. Base64MagicCheckAndDecode

function Base64MagicCheckAndDecode(Value: PUtf8Char; var Blob: TSynTempBuffer; ValueLen: integer = 0): boolean; overload;

Check and decode '\uFFF0base64encodedbinary' content into binary
- this method will check the supplied value to match the expected JSON_BASE64_MAGIC_C pattern, decode and set Blob and return TRUE

1.6.24. Base64MagicDecode

procedure Base64MagicDecode(var ParamValue: RawUtf8);

Just a wrapper around Base64ToBin() for in-place decode of JSON_BASE64_MAGIC_C '\uFFF0base64encodedbinary' content into binary
- input ParamValue shall have been checked to match the expected pattern

1.6.25. Base64MagicToBlob

procedure Base64MagicToBlob(Base64: PUtf8Char; var result: RawUtf8);

Convert a Base64-encoded content into binary hexadecimal ready for SQL
- returns e.g. X'53514C697465'

1.6.26. Base64MagicTryAndDecode

function Base64MagicTryAndDecode(Value: PUtf8Char; ValueLen: integer; var Blob: RawByteString): boolean;

Decode '\uFFF0base64encodedbinary' or 'base64encodedbinary' into binary
- same as Base64MagicCheckAndDecode(), but will detect and ignore the magic and not require it

1.6.27. Base64ToBin

function Base64ToBin(const base64: RawByteString; bin: PAnsiChar; binlen: PtrInt ; nofullcheck: boolean = true ): boolean; overload;

Fast conversion from Base64 encoded text into binary data
- returns TRUE on success, FALSE if base64 does not match binlen
- nofullcheck is deprecated and not used any more, since nofullcheck=false is now processed with no performance cost

1.6.28. Base64ToBin

function Base64ToBin(sp: PAnsiChar; len: PtrInt; var data: RawByteString): boolean; overload;

Fast conversion from Base64 encoded text into binary data
- is now just an alias to Base64ToBinSafe() overloaded function
- returns false and data='' if sp/len buffer was invalid

1.6.29. Base64ToBin

function Base64ToBin(sp: PAnsiChar; len: PtrInt): RawByteString; overload;

Fast conversion from Base64 encoded text into binary data
- is now just an alias to Base64ToBinSafe() overloaded function
- returns '' if sp/len buffer was not a valid Base64-encoded input

1.6.30. Base64ToBin

function Base64ToBin(base64, bin: PAnsiChar; base64len, binlen: PtrInt ; nofullcheck: boolean = true ): boolean; overload;

1.6.31. Base64ToBin

function Base64ToBin(const s: RawByteString): RawByteString; overload;

Fast conversion from Base64 encoded text into binary data
- is now just an alias to Base64ToBinSafe() overloaded function
- returns '' if s was not a valid Base64-encoded input

1.6.32. Base64ToBin

function Base64ToBin(sp: PAnsiChar; len: PtrInt; var Blob: TSynTempBuffer): boolean; overload;

Fast conversion from Base64 encoded text into binary data
- returns TRUE on success, FALSE if sp/len buffer was invvalid

1.6.33. Base64ToBinLength

function Base64ToBinLength(sp: PAnsiChar; len: PtrInt): PtrInt;

Retrieve the expected undecoded length of a Base64 encoded buffer
- here len is the number of bytes in sp

1.6.34. Base64ToBinLengthSafe

function Base64ToBinLengthSafe(sp: PAnsiChar; len: PtrInt): PtrInt;

Retrieve the expected undecoded length of a Base64 encoded buffer
- here len is the number of bytes in sp
- will check supplied text is a valid Base64 encoded stream

1.6.35. Base64ToBinSafe

function Base64ToBinSafe(sp: PAnsiChar; len: PtrInt; out data: TBytes): boolean; overload;

Fast conversion from Base64 encoded text into binary data
- will check supplied text is a valid Base64 encoded stream

1.6.36. Base64ToBinSafe

function Base64ToBinSafe(sp: PAnsiChar; len: PtrInt): RawByteString; overload;

Fast conversion from Base64 encoded text into binary data
- will check supplied text is a valid Base64 encoded stream

1.6.37. Base64ToBinSafe

function Base64ToBinSafe(const s: RawByteString): RawByteString; overload;

Fast conversion from Base64 encoded text into binary data
- will check supplied text is a valid Base64 encoded stream

1.6.38. Base64ToBinSafe

function Base64ToBinSafe(sp: PAnsiChar; len: PtrInt; var data: RawByteString): boolean; overload;

Fast conversion from Base64 encoded text into binary data
- will check supplied text is a valid Base64 encoded stream

1.6.39. Base64ToBinTrim

function Base64ToBinTrim(const s: RawByteString): RawByteString;

Conversion from Base64 encoded text into binary data, ignoring spaces
- returns '' if s was not a valid Base64-encoded input once spaces are trimmed
- consider PemToDer() from mormot.crypt.secure if you need to read PEM content

1.6.40. Base64ToUri

procedure Base64ToUri(var base64: RawUtf8);

Conversion from any Base64 encoded value into URI-compatible encoded text
- warning: will modify the supplied base64 string in-place
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.41. Base64uriDecode

function Base64uriDecode(sp, rp: PAnsiChar; len: PtrInt): boolean;

Direct low-level decoding of a Base64-URI encoded buffer
- the buffer is expected to be at least Base64uriToBinLength() bytes long
- returns true if the supplied sp[] buffer has been successfully decoded into rp[] - will break at any invalid character, so is always safe to use
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'
- you should better not use this, but Base64uriToBin() overloaded functions

1.6.42. Base64uriEncode

procedure Base64uriEncode(rp, sp: PAnsiChar; len: cardinal);

Low-level conversion from a binary buffer into Base64-like URI-compatible encoded text
- you should rather use the overloaded BinToBase64uri() functions

1.6.43. Base64uriToBin

function Base64uriToBin(const s: RawByteString): RawByteString; overload;

Fast conversion from Base64-URI encoded text into binary data
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.44. Base64uriToBin

function Base64uriToBin(base64, bin: PAnsiChar; base64len, binlen: PtrInt): boolean; overload;

1.6.45. Base64uriToBin

function Base64uriToBin(const base64: RawByteString; bin: PAnsiChar; binlen: PtrInt): boolean; overload;

1.6.46. Base64uriToBin

function Base64uriToBin(sp: PAnsiChar; len: PtrInt): RawByteString; overload;

1.6.47. Base64uriToBin

function Base64uriToBin(sp: PAnsiChar; len: PtrInt; var bin: RawByteString): boolean; overload;

1.6.48. Base64uriToBin

function Base64uriToBin(sp: PAnsiChar; len: PtrInt; var temp: TSynTempBuffer): boolean; overload;

Fast conversion from Base64-URI encoded text into binary data
- caller should always execute temp.Done when finished with the data
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.49. Base64uriToBinLength

function Base64uriToBinLength(len: PtrInt): PtrInt;

Retrieve the expected undecoded length of a Base64-URI encoded buffer
- here len is the number of bytes in sp
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.50. BaudotToAscii

function BaudotToAscii(Baudot: PByteArray; len: PtrInt): RawUtf8; overload;

Convert some Baudot code binary, into ASCII-7 text
- reverse of the AsciiToBaudot() function
- any uppercase character would be decoded as lowercase - and some characters may have disapeared outside of a-z 0-9 - ' , ! : ( + ) $ ? @ . / ; range
- the "baud" symbol rate measurement comes from Emile's name ;)

1.6.51. BaudotToAscii

function BaudotToAscii(const Baudot: RawByteString): RawUtf8; overload;

1.6.52. BinToBase32

procedure BinToBase32(Bin: PByteArray; Dest: PAnsiChar; BinLen: PtrInt; b32enc: PAnsiChar); overload;

Conversion from a binary buffer into Base32 encoded text buffer
- default b32enc is RFC4648 upper alphanumeric without misleading 0O 1I 8B

1.6.53. BinToBase32

function BinToBase32(Bin: PAnsiChar; BinLen: PtrInt): RawUtf8; overload;

Conversion from a binary buffer into Base32 encoded text as RawUtf8
- RFC4648 Base32 is defined as upper alphanumeric without misleading 0O 1I 8B

1.6.54. BinToBase32

function BinToBase32(const Bin: RawByteString): RawUtf8; overload;

Conversion from a binary buffer into Base32 encoded text as RawUtf8
- RFC4648 Base32 is defined as upper alphanumeric without misleading 0O 1I 8B

1.6.55. BinToBase32Length

function BinToBase32Length(BinLen: cardinal): cardinal;

Compute the length resulting of Base32 encoding of a binary buffer
- RFC4648 Base32 is defined as upper alphanumeric without misleading 0O 1I 8B

1.6.56. BinToBase58

function BinToBase58(const Bin: RawByteString): RawUtf8; overload;

Conversion from a binary buffer into Base58 encoded text as RawUtf8
- Bitcoin' Base58 was defined as alphanumeric chars without misleading 0O I1
- Base58 is much slower than Base64, performing in O(n^2) instead of O(n), and should not be used on big buffers

1.6.57. BinToBase58

function BinToBase58(Bin: PAnsiChar; BinLen: integer): RawUtf8; overload;

1.6.58. BinToBase58

function BinToBase58(Bin: PAnsiChar; BinLen: integer; var Dest: TSynTempBuffer): integer; overload;

Conversion from a binary buffer into Base58 encoded text as TSynTempBuffer
- Bitcoin' Base58 was defined as alphanumeric chars without misleading 0O I1
- Base58 is much slower than Base64, performing in O(n^2) instead of O(n), and should not be used on big buffers
- returns the number of encoded chars encoded into Dest.buf
- caller should call Dest.Done once it is finished with the output text

1.6.59. BinToBase64

function BinToBase64(Bin: PAnsiChar; BinBytes: integer): RawUtf8; overload;

Fast conversion from binary data into Base64 encoded UTF-8 text

1.6.60. BinToBase64

function BinToBase64(const s: RawByteString): RawUtf8; overload;

Fast conversion from binary data into Base64 encoded UTF-8 text

1.6.61. BinToBase64

function BinToBase64(const data, Prefix, Suffix: RawByteString; WithMagic: boolean): RawUtf8; overload;

Fast conversion from binary data into prefixed/suffixed Base64 encoded UTF-8 text
- with optional JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)
- may use AVX2 optimized asm (10GB/s) on FPC x86_64

1.6.62. BinToBase64Length

function BinToBase64Length(len: PtrUInt): PtrUInt;

Retrieve the expected encoded length after Base64 process

1.6.63. BinToBase64Line

function BinToBase64Line(sp: PAnsiChar; len: PtrUInt; const Prefix: RawUtf8 = ''; const Suffix: RawUtf8 = ''): RawUtf8;

Fast conversion from binary into prefixed/suffixed Base64 with 64 chars per line

1.6.64. BinToBase64Short

function BinToBase64Short(Bin: PAnsiChar; BinBytes: integer): ShortString; overload;

Fast conversion from a small binary data into Base64 encoded UTF-8 text

1.6.65. BinToBase64Short

function BinToBase64Short(const s: RawByteString): ShortString; overload;

Fast conversion from a small binary data into Base64 encoded UTF-8 text

1.6.66. BinToBase64uri

function BinToBase64uri(const s: RawByteString): RawUtf8; overload;

Fast conversion from binary data into Base64-like URI-compatible encoded text
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.67. BinToBase64uri

function BinToBase64uri(Bin: PAnsiChar; BinBytes: integer): RawUtf8; overload;

Fast conversion from a binary buffer into Base64-like URI-compatible encoded text
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.68. BinToBase64uriLength

function BinToBase64uriLength(len: PtrUInt): PtrUInt;

Retrieve the expected encoded length after Base64-URI process
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'

1.6.69. BinToBase64uriShort

function BinToBase64uriShort(Bin: PAnsiChar; BinBytes: integer): ShortString;

Fast conversion from a binary buffer into Base64-like URI-compatible encoded ShortString
- in comparison to Base64 standard encoding, will trim any right-sided '=' unsignificant characters, and replace '+' or '/' by '_' or '-'
- returns '' if BinBytes void or too big for the resulting ShortString

1.6.70. BinToBase64WithMagic

procedure BinToBase64WithMagic(Data: pointer; DataLen: integer; var Result: RawUtf8); overload;

Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)

1.6.71. BinToBase64WithMagic

function BinToBase64WithMagic(const data: RawByteString): RawUtf8; overload;

Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)

1.6.72. BinToBase64WithMagic

function BinToBase64WithMagic(Data: pointer; DataLen: integer): RawUtf8; overload;

Fast conversion from binary data into Base64 encoded UTF-8 text with JSON_BASE64_MAGIC_C prefix (UTF-8 encoded \uFFF0 special code)

1.6.73. BinToHumanHex

function BinToHumanHex(Data: PByte; Len: integer; PerLine: integer = 16; LeftTab: integer = 0; SepChar: AnsiChar = ':'): RawUtf8; overload;

Generate some 'xx:xx:xx:xx' output buffer with left and right margins
- used e.g. by ParsedToText() to output X509 public key content in PeerInfo

1.6.74. BinToHumanHex

procedure BinToHumanHex(W: TTextWriter; Data: PByte; Len: integer; PerLine: integer = 16; LeftTab: integer = 0; SepChar: AnsiChar = ':'); overload;

Generate some 'xx:xx:xx:xx' output buffer with left and right margins

1.6.75. BinToSource

function BinToSource(const ConstName, Comment: RawUtf8; const Data: RawByteString; PerLine: integer = 16; const Suffix: RawUtf8 = ''): RawUtf8; overload;

Generate some pascal source code holding some data binary as constant

1.6.76. BinToSource

function BinToSource(const ConstName, Comment: RawUtf8; Data: pointer; Len: integer; PerLine: integer = 16; const Suffix: RawUtf8 = ''): RawUtf8; overload;

Generate some pascal source code holding some data binary as constant
- can store sensitive information (e.g. certificates) within the executable
- generates a source code snippet of the following format:

 const
   // Comment
   ConstName: array[0..2] of byte = (
     $01, $02, $03);

1.6.77. BinToSource

procedure BinToSource(Dest: TTextWriter; const ConstName, Comment: RawUtf8; Data: pointer; Len: integer; PerLine: integer = 16); overload;

 const
   // Comment
   ConstName: array[0..2] of byte = (
     $01, $02, $03);

1.6.78. BlobToBytes

function BlobToBytes(P: PUtf8Char): TBytes;

Create a TBytes from TEXT-encoded blob data
- blob data can be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.) or or Base64 encoded content ('\uFFF0base64encodedbinary') or plain TEXT

1.6.79. BlobToRawBlob

procedure BlobToRawBlob(P: PUtf8Char; var result: RawBlob; Len: integer = 0); overload;

Fill a RawBlob from TEXT-encoded blob data
- blob data can be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.) or or Base64 encoded content ('\uFFF0base64encodedbinary') or plain TEXT

1.6.80. BlobToRawBlob

function BlobToRawBlob(P: PUtf8Char; Len: integer = 0): RawBlob; overload;

Fill a RawBlob from TEXT-encoded blob data
- blob data can be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.) or or Base64 encoded content ('\uFFF0base64encodedbinary') or plain TEXT

1.6.81. BlobToRawBlob

function BlobToRawBlob(const Blob: RawByteString): RawBlob; overload;

Fill a RawBlob from TEXT-encoded blob data
- blob data can be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.) or or Base64 encoded content ('\uFFF0base64encodedbinary') or plain TEXT

1.6.82. BlobToStream

function BlobToStream(P: PUtf8Char): TStream;

Create a memory stream from TEXT-encoded blob data
- blob data can be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.) or or Base64 encoded content ('\uFFF0base64encodedbinary') or plain TEXT
- the caller must free the stream instance after use

1.6.83. BytesToRawByteString

procedure BytesToRawByteString(const bytes: TBytes; out buf: RawByteString);

Creates a RawByteString memory buffer from a TBytes content

1.6.84. EmojiFromDots

procedure EmojiFromDots(P: PUtf8Char; W: TTextWriter); overload;

Low-level conversion of github/Markdown :identifiers: into UTF-8 Emoji sequences

1.6.85. EmojiFromDots

function EmojiFromDots(const text: RawUtf8): RawUtf8; overload;

Conversion of github/Markdown :identifiers: into UTF-8 Emoji sequences

1.6.86. EmojiFromText

function EmojiFromText(P: PUtf8Char; len: PtrInt): TEmoji;

Recognize github/Markdown compatible text of Emojis
- for instance 'sunglasses' text buffer will return eSunglasses
- returns eNone if no case-insensitive match was found

1.6.87. EmojiParseDots

function EmojiParseDots(var P: PUtf8Char; W: TTextWriter = nil): TEmoji;

Low-level parser of github/Markdown compatible text of Emojis
- supplied P^ should point to ':'
- will append the recognized UTF-8 Emoji if P contains e.g. :joy: or :)
- will append ':' if no Emoji text is recognized, and return eNone
- will try both EMOJI_AFTERDOTS[] and EMOJI_RTTI[] reference set
- if W is nil, won't append anything, but just return the recognized TEmoji

1.6.88. EmojiToDots

procedure EmojiToDots(P: PUtf8Char; W: TTextWriter); overload;

Low-level conversion of UTF-8 Emoji sequences into github/Markdown :identifiers:

1.6.89. EmojiToDots

function EmojiToDots(const text: RawUtf8): RawUtf8; overload;

Conversion of UTF-8 Emoji sequences into github/Markdown :identifiers:

1.6.90. EscapeBuffer

function EscapeBuffer(s: PAnsiChar; slen: integer; d: PAnsiChar; dmax: integer): PAnsiChar;

Low-level fast conversion from binary data to escaped text
- non printable characters will be written as $xx hexadecimal codes
- will be #0 terminated, with '...' characters trailing on dmax overflow
- ensure the destination buffer contains at least dmax bytes, which is always the case when using LogEscape() and its local TLogEscape variable

1.6.91. EscapeToShort

function EscapeToShort(source: PAnsiChar; sourcelen: integer): ShortString; overload;

Fill a ShortString with the (hexadecimal) chars of the input text/binary

1.6.92. EscapeToShort

function EscapeToShort(const source: RawByteString): ShortString; overload;

Fill a ShortString with the (hexadecimal) chars of the input text/binary

1.6.93. FromVarBlob

function FromVarBlob(Data: PByte): TValueResult;

Retrieve pointer and length to a variable-length text/blob buffer

1.6.94. FromVarInt32

function FromVarInt32(var Source: PByte): integer;

Convert a 32-bit variable-length integer buffer into an integer
- decode negative values from cardinal two-complement, i.e. 0=0,1=1,2=-1,3=2,4=-2...

1.6.95. FromVarInt64

function FromVarInt64(var Source: PByte): Int64;

Convert a 64-bit variable-length integer buffer into a Int64

1.6.96. FromVarInt64Value

function FromVarInt64Value(Source: PByte): Int64;

Convert a 64-bit variable-length integer buffer into a Int64
- this version won't update the Source pointer

1.6.97. FromVarString

procedure FromVarString(var Source: PByte; var Value: RawUtf8); overload;

Retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8

1.6.98. FromVarString

procedure FromVarString(var Source: PByte; var Value: RawByteString; CodePage: integer); overload;

Retrieve a variable-length text buffer
- this overloaded function will set the supplied code page to the AnsiString

1.6.99. FromVarString

function FromVarString(var Source: PByte): RawUtf8; overload;

Retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8

1.6.100. FromVarString

function FromVarString(var Source: PByte; SourceMax: PByte): RawUtf8; overload;

Safe retrieve a variable-length UTF-8 encoded text buffer in a newly allocation RawUtf8
- supplied SourceMax value will avoid any potential buffer overflow

1.6.101. FromVarString

function FromVarString(var Source: PByte; SourceMax: PByte; var Value: RawByteString; CodePage: integer): boolean; overload;

Retrieve a variable-length text buffer
- this overloaded function will set the supplied code page to the AnsiString and will also check for the SourceMax end of buffer
- returns TRUE on success, or FALSE on any buffer overload detection

1.6.102. FromVarString

function FromVarString(var Source: PByte; SourceMax: PByte; var Value: TSynTempBuffer): boolean; overload;

Retrieve a variable-length UTF-8 encoded text buffer in a temporary buffer
- caller should call Value.Done after use of the Value.buf memory
- this overloaded function will also check for the SourceMax end of buffer, returning TRUE on success, or FALSE on any buffer overload detection

1.6.103. FromVarString

procedure FromVarString(var Source: PByte; var Value: TSynTempBuffer); overload;

Retrieve a variable-length UTF-8 encoded text buffer in a temporary buffer
- caller should call Value.Done after use of the Value.buf memory
- this overloaded function would include a trailing #0, so Value.buf could be parsed as a valid PUtf8Char buffer (e.g. containing JSON)

1.6.104. FromVarUInt32

function FromVarUInt32(var Source: PByte; SourceMax: PByte; out Value: cardinal): boolean; overload;

Convert a 32-bit variable-length integer buffer into a cardinal
- will call FromVarUInt32() if SourceMax=nil, or FromVarUInt32Safe() if set
- returns false on error, true if Value has been set properly

1.6.105. FromVarUInt32

function FromVarUInt32(var Source: PByte): cardinal; overload;

Convert a 32-bit variable-length integer buffer into a cardinal
- fast inlined process for any number < 128
- use overloaded FromVarUInt32() or FromVarUInt32Safe() with a SourceMax pointer to avoid any potential buffer overflow
- use FromVarUInt32Big() is the content is likely to be >= 128

1.6.106. FromVarUInt32Big

function FromVarUInt32Big(var Source: PByte): cardinal;

Convert a 32-bit variable-length integer buffer into a cardinal
- this version could be called if number is likely to be > $7f, so if inlining the first byte won't make any benefit

1.6.107. FromVarUInt32High

function FromVarUInt32High(var Source: PByte): cardinal;

Convert a 32-bit variable-length integer buffer into a cardinal
- this version must be called if Source^ has already been checked to be > $7f

1.6.108. FromVarUInt32Safe

function FromVarUInt32Safe(Source, SourceMax: PByte; out Value: cardinal): PByte;

Safely convert a 32-bit variable-length integer buffer into a cardinal
- slower but safer process checking out of boundaries memory access in Source
- SourceMax is expected to be not nil, and to point to the first byte just after the Source memory buffer
- returns nil on error, or point to next input data on successful decoding

1.6.109. FromVarUInt32Up128

function FromVarUInt32Up128(var Source: PByte): cardinal;

Convert a 32-bit variable-length integer buffer into a cardinal
- used e.g. when inlining FromVarUInt32()
- this version must be called if Source^ has already been checked to be > $7f

 result := Source^;
 inc(Source);
 if result>$7f then
   result := (result and $7F) or FromVarUInt32Up128(Source);

1.6.110. FromVarUInt64

function FromVarUInt64(var Source: PByte; SourceMax: PByte; out Value: Qword): boolean; overload;

Convert a 64-bit variable-length integer buffer into a UInt64
- will call FromVarUInt64() if SourceMax=nil, or FromVarUInt64Safe() if set
- returns false on error, true if Value has been set properly

1.6.111. FromVarUInt64

function FromVarUInt64(var Source: PByte): QWord; overload;

Convert a 64-bit variable-length integer buffer into a UInt64

1.6.112. FromVarUInt64Safe

function FromVarUInt64Safe(Source, SourceMax: PByte; out Value: QWord): PByte;

Safely convert a 64-bit variable-length integer buffer into a UInt64
- slower but safer process checking out of boundaries memory access in Source
- SourceMax is expected to be not nil, and to point to the first byte just after the Source memory buffer
- returns nil on error, or point to next input data on successful decoding

1.6.113. GetJpegSize

function GetJpegSize(jpeg: PAnsiChar; len: PtrInt; out Height, Width, Bits: integer): boolean; overload;

Fast guess of the size, in pixels, of a JPEG memory buffer
- will only scan for basic JPEG structure, up to the StartOfFrame (SOF) chunk
- returns TRUE if the buffer is likely to be a JPEG picture, and set the Height + Width + Bits variable with its dimensions - but there may be false positive recognition, and no waranty that the memory buffer is a valid JPEG
- returns FALSE if the buffer does not have any expected SOI/SOF markers

1.6.114. GetMimeContentType

function GetMimeContentType(Content: pointer; Len: PtrInt; const FileName: TFileName = ''; const DefaultContentType: RawUtf8 = BINARY_CONTENT_TYPE; Mime: PMimeType = nil): RawUtf8;

Retrieve the MIME content type from its file name or a supplied binary buffer
- will first check for known file extensions, then inspect the binary content
- return the MIME type, ready to be appended to a 'Content-Type: ' HTTP header
- default is DefaultContentType or 'application/octet-stream' (BINARY_CONTENT_TYPE) or 'application/fileextension' if FileName was specified
- see @http://en.wikipedia.org/wiki/Internet_media_type for most common values

1.6.115. GetMimeContentTypeFromBuffer

function GetMimeContentTypeFromBuffer(Content: pointer; Len: PtrInt; const DefaultContentType: RawUtf8; Mime: PMimeType = nil): RawUtf8;

Retrieve the MIME content type from a supplied binary buffer
- inspect the first bytes, to guess from standard known headers
- return the MIME type, ready to be appended to a 'Content-Type: ' HTTP header
- returns DefaultContentType if the binary buffer has an unknown layout

1.6.116. GetMimeContentTypeFromExt

function GetMimeContentTypeFromExt(const FileName: TFileName; FileExt: PRawUtf8 = nil): TMimeType;

MtUnknown mtPng mtGif mtTiff mtJpg mtBmp mtDoc mtPpt mtXls mtHtml mtCss mtJS RFC 9239 mtXIcon mtFont RFC 8081 mtText mtSvg mtXml mtWebp mtManifest mtJson mtOgg RFC 5334 mtMp4 RFC 4337 6381 mtMp2 mtMpeg RFC 3003 mtH264 RFC 6184 mtWma mtWmv mtAvi mtGzip mtWebm mtRar mt7z mtZip mtBz2 mtPdf mtSQlite3 mtXcomp mtDicom retrieve the MIME content type from its file name

1.6.117. GetMimeContentTypeFromMemory

function GetMimeContentTypeFromMemory(Content: pointer; Len: PtrInt): TMimeType;

Retrieve the MIME content type from a supplied binary buffer

1.6.118. GetMimeContentTypeHeader

function GetMimeContentTypeHeader(const Content: RawByteString; const FileName: TFileName = ''; Mime: PMimeType = nil): RawUtf8;

Retrieve the HTTP header for MIME content type from a supplied binary buffer
- just append HEADER_CONTENT_TYPE and GetMimeContentType() result
- can be used as such:

  Call.OutHead := GetMimeContentTypeHeader(Call.OutBody,aFileName);

1.6.119. GetMimeTypeFromExt

function GetMimeTypeFromExt(const Ext: RawUtf8): TMimeType;

Retrieve the MIME content type from its file extension text (without '.')

1.6.120. GetStreamBuffer

function GetStreamBuffer(S: TStream): pointer;

Retrieve the memory buffer of a TCustomMemoryStream/TRawByteStringStream
- returns nil if the instance is not of those classes

1.6.121. GotoNextVarInt

function GotoNextVarInt(Source: PByte): pointer;

Jump a value in the 32-bit or 64-bit variable-length integer buffer

1.6.122. GotoNextVarString

function GotoNextVarString(Source: PByte): pointer;

Jump a value in variable-length text buffer

1.6.123. HashFile

function HashFile(const FileName: TFileName; Hasher: THasher = nil): cardinal; overload;

Compute the 32-bit default hash of a file content
- you can specify your own hashing function if DefaultHasher is not what you expect

1.6.124. HashFileCrc32c

function HashFileCrc32c(const FileName: TFileName): RawUtf8;

Compute the crc32c checksum of a given file
- this function maps the THashFile signature

1.6.125. HtmlEscapeMarkdown

function HtmlEscapeMarkdown(const md: RawUtf8; esc: TTextWriterHtmlEscape = [heEmojiToUtf8]): RawUtf8;

Escape some Markdown-marked text into HTML
- just a wrapper around AddHtmlEscapeMarkdown() process

1.6.126. HtmlEscapeWiki

function HtmlEscapeWiki(const wiki: RawUtf8; esc: TTextWriterHtmlEscape = [heHtmlEscape, heEmojiToUtf8]): RawUtf8;

Escape some wiki-marked text into HTML
- just a wrapper around AddHtmlEscapeWiki() process

1.6.127. IncludeTrailingUriDelimiter

function IncludeTrailingUriDelimiter(const URI: RawByteString): RawByteString;

Ensure the supplied URI contains a trailing '/' charater

1.6.128. IsBase64

function IsBase64(const s: RawByteString): boolean; overload;

Check if the supplied text is a valid Base64 encoded stream

1.6.129. IsBase64

function IsBase64(sp: PAnsiChar; len: PtrInt): boolean; overload;

Check if the supplied text is a valid Base64 encoded stream

1.6.130. isBlobHex

function isBlobHex(P: PUtf8Char): boolean;

Return true if the TEXT is encoded as SQLite3 BLOB literals (X'53514C697465' e.g.)

1.6.131. IsContentCompressed

function IsContentCompressed(Content: pointer; Len: PtrInt): boolean;

Retrieve if some content is compressed, from a supplied binary buffer
- returns TRUE, if the header in binary buffer "may" be compressed (this method can trigger false positives), e.g. begin with most common already compressed zip/gz/gif/png/jpeg/avi/mp3/mp4 markers (aka "magic numbers")

1.6.132. IsContentTypeCompressible

function IsContentTypeCompressible(ContentType: PUtf8Char): boolean;

Recognize e.g. 'text/css' or 'application/json' as compressible

1.6.133. IsContentTypeCompressibleU

function IsContentTypeCompressibleU(const ContentType: RawUtf8): boolean;

Recognize e.g. 'text/css' or 'application/json' as compressible

1.6.134. IsStreamBuffer

function IsStreamBuffer(S: TStream): boolean;

Check if class is a TCustomMemoryStream/TRawByteStringStream

1.6.135. IsUrlValid

function IsUrlValid(P: PUtf8Char): boolean;

Checks if the supplied UTF-8 text don't need URI encoding
- returns TRUE if all its chars are non-void plain ASCII-7 RFC compatible identifiers (0..9a..zA..Z-_.~)

1.6.136. LogEscape

function LogEscape(source: PAnsiChar; sourcelen: integer; var temp: TLogEscape; enabled: boolean = true): PAnsiChar;

Fill TLogEscape stack buffer with the (hexadecimal) chars of the input binary
- up to 512 bytes will be escaped and appended to a local temp: TLogEscape variable, using the EscapeBuffer() low-level function
- you can then log the resulting escaped text by passing the returned PAnsiChar as % parameter to a TSynLog.Log() method
- the "enabled" parameter can be assigned from a process option, avoiding to process the escape if verbose logs are disabled
- used e.g. to implement logBinaryFrameContent option for WebSockets

1.6.137. LogEscapeFull

function LogEscapeFull(const source: RawByteString): RawUtf8; overload;

Returns a text buffer with the (hexadecimal) chars of the input binary
- is much slower than LogEscape/EscapeToShort, but has no size limitation

1.6.138. LogEscapeFull

function LogEscapeFull(source: PAnsiChar; sourcelen: integer): RawUtf8; overload;

Returns a text buffer with the (hexadecimal) chars of the input binary
- is much slower than LogEscape/EscapeToShort, but has no size limitation

1.6.139. MultiPartFormDataAddField

function MultiPartFormDataAddField(const FieldName, FieldValue: RawUtf8; var MultiPart: TMultiPartDynArray; const ForcedContentType: RawUtf8 = ''): boolean;

Encode a field in a multipart array
- FieldName: field name of the part
- FieldValue: value of the field
- Multipart: where the part is added
- consider THttpMultiPartStream from mormot.net.client for huge file content

1.6.140. MultiPartFormDataAddFile

function MultiPartFormDataAddFile(const FileName: TFileName; var MultiPart: TMultiPartDynArray; const Name: RawUtf8 = ''; const ForcedContentType: RawUtf8 = ''): boolean;

Encode a file in a multipart array
- FileName: file to encode
- Multipart: where the part is added
- Name: name of the part, is empty the name 'File###' is generated
- consider THttpMultiPartStream from mormot.net.client for huge file content

1.6.141. MultiPartFormDataDecode

function MultiPartFormDataDecode(const MimeType, Body: RawUtf8; var MultiPart: TMultiPartDynArray): boolean;

Decode multipart/form-data POST request content into memory
- following RFC 1867
- decoded sections are appended to MultiPart[] existing array

1.6.142. MultiPartFormDataEncode

function MultiPartFormDataEncode(const MultiPart: TMultiPartDynArray; var MultiPartContentType, MultiPartContent: RawUtf8; Rfc2388NestedFiles: boolean = false): boolean;

Encode multipart fields and files
- only one of them can be used because MultiPartFormDataDecode must implement both decodings
- MultiPart: parts to build the multipart content from, which may be created using MultiPartFormDataAddFile/MultiPartFormDataAddField
- MultiPartContentType: variable returning

 Content-Type: multipart/form-data; boundary=xxx

where xxx is the first generated boundary
- MultiPartContent: generated multipart content
- Rfc2388NestedFiles will force the deprecated nested "multipart/mixed" format
- consider THttpMultiPartStream from mormot.net.client for huge file content

1.6.143. MultiPartFormDataNewBound

function MultiPartFormDataNewBound(var boundaries: TRawUtf8DynArray): RawUtf8;

Used e.g. by MultiPartFormDataEncode and THttpMultiPartStream.Add

1.6.144. Plural

function Plural(const itemname: ShortString; itemcount: cardinal): ShortString;

Write count number and append 's' (if needed) to form a plural English noun
- for instance, Plural('row',100) returns '100 rows' with no heap allocation

1.6.145. RawBlobToBlob

function RawBlobToBlob(const RawBlob: RawBlob): RawUtf8; overload;

Creates a TEXT-encoded version of blob data from a RawBlob
- TEXT will be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.)

1.6.146. RawBlobToBlob

function RawBlobToBlob(RawBlob: pointer; RawBlobLength: integer): RawUtf8; overload;

Creates a TEXT-encoded version of blob data from a memory data
- same as RawBlob, but with direct memory access via a pointer/byte size pair
- TEXT will be encoded as SQLite3 BLOB literals (X'53514C697465' e.g.)

1.6.147. RawByteStringArrayConcat

function RawByteStringArrayConcat(const Values: array of RawByteString): RawByteString;

Fast concatenation of several AnsiStrings

1.6.148. RawByteStringToBytes

procedure RawByteStringToBytes(const buf: RawByteString; out bytes: TBytes);

Creates a TBytes from a RawByteString memory buffer

1.6.149. ResourceSynLZToRawByteString

procedure ResourceSynLZToRawByteString(const ResName: string; out buf: RawByteString; Instance: TLibHandle = 0);

Creates a RawByteString memory buffer from an SynLZ-compressed embedded resource
- returns '' if the resource is not found
- this method would use SynLZDecompress() after ResourceToRawByteString(), with a ResType=PChar(10) (i.e. RC_DATA)
- you can specify a library (dll) resource instance handle, if needed

1.6.150. ResourceToRawByteString

procedure ResourceToRawByteString(const ResName: string; ResType: PChar; out buf: RawByteString; Instance: TLibHandle = 0);

Creates a RawByteString memory buffer from an embedded resource
- returns '' if the resource is not found
- warning: resources size may be rounded up to alignment
- you can specify a library (dll) resource instance handle, if needed

1.6.151. SameFileContent

function SameFileContent(const One, Another: TFileName): boolean;

Compare two files by content, reading them by blocks

1.6.152. ToVarInt32

function ToVarInt32(Value: PtrInt; Dest: PByte): PByte;

Convert an integer into a 32-bit variable-length integer buffer
- store negative values as cardinal two-complement, i.e. 0=0,1=1,2=-1,3=2,4=-2...

1.6.153. ToVarInt64

function ToVarInt64(Value: Int64; Dest: PByte): PByte;

Convert a Int64 into a 64-bit variable-length integer buffer

1.6.154. ToVarString

function ToVarString(const Value: RawUtf8; Dest: PByte): PByte;

Convert a RawUtf8 into an UTF-8 encoded variable-length buffer

1.6.155. ToVarUInt32

function ToVarUInt32(Value: cardinal; Dest: PByte): PByte;

Convert a cardinal into a 32-bit variable-length integer buffer

1.6.156. ToVarUInt32Length

function ToVarUInt32Length(Value: PtrUInt): PtrUInt;

Return the number of bytes necessary to store a 32-bit variable-length integer
- i.e. the ToVarUInt32() buffer size

1.6.157. ToVarUInt32LengthWithData

function ToVarUInt32LengthWithData(Value: PtrUInt): PtrUInt;

Return the number of bytes necessary to store some data with a its 32-bit variable-length integer length

1.6.158. ToVarUInt64

function ToVarUInt64(Value: QWord; Dest: PByte): PByte;

Convert a UInt64 into a 64-bit variable-length integer buffer

1.6.159. UrlDecode

function UrlDecode(U: PUtf8Char): RawUtf8; overload;

Decode a UrlEncode() URI encoded parameter into its original value

1.6.160. UrlDecode

function UrlDecode(const s: RawUtf8): RawUtf8; overload;

Decode a UrlEncode() URI encoded parameter into its original value

1.6.161. UrlDecodeCardinal

function UrlDecodeCardinal(U: PUtf8Char; const Upper: RawUtf8; var Value: cardinal; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original cardinal numerical value
- UrlDecodeCardinal('offset=20&where=LastName%3D%27M%C3%B4net%27','OFFSET=',O,@Next) will return Next^='where=...' and O=20
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.162. UrlDecodeDouble

function UrlDecodeDouble(U: PUtf8Char; const Upper: RawUtf8; var Value: double; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original floating-point value
- UrlDecodeDouble('price=20.45&where=LastName%3D%27M%C3%B4net%27','PRICE=',P,@Next) will return Next^='where=...' and P=20.45
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.163. UrlDecodeExtended

function UrlDecodeExtended(U: PUtf8Char; const Upper: RawUtf8; var Value: TSynExtended; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original floating-point value
- UrlDecodeExtended('price=20.45&where=LastName%3D%27M%C3%B4net%27','PRICE=',P,@Next) will return Next^='where=...' and P=20.45
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.164. UrlDecodeInt64

function UrlDecodeInt64(U: PUtf8Char; const Upper: RawUtf8; var Value: Int64; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original Int64 numerical value
- UrlDecodeInt64('offset=20&where=LastName%3D%27M%C3%B4net%27','OFFSET=',O,@Next) will return Next^='where=...' and O=20
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.165. UrlDecodeInteger

function UrlDecodeInteger(U: PUtf8Char; const Upper: RawUtf8; var Value: integer; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original integer numerical value
- UrlDecodeInteger('offset=20&where=LastName%3D%27M%C3%B4net%27','OFFSET=',O,@Next) will return Next^='where=...' and O=20
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.166. UrlDecodeName

function UrlDecodeName(const s: RawUtf8): RawUtf8; overload;

Decode a UrlEncodeName() URI encoded network name into its original value
- only parameters - i.e. after '?' - should replace spaces by '+'

1.6.167. UrlDecodeName

function UrlDecodeName(U: PUtf8Char): RawUtf8; overload;

Decode a UrlEncodeName() URI encoded network name into its original value
- only parameters - i.e. after '?' - should replace spaces by '+'

1.6.168. UrlDecodeNeedParameters

function UrlDecodeNeedParameters(U, CsvNames: PUtf8Char): boolean;

Returns TRUE if all supplied parameters do exist in the URI encoded text
- CsvNames parameter shall provide as a CSV list of names
- e.g. UrlDecodeNeedParameters('price=20.45&where=LastName%3D','price,where') will return TRUE

1.6.169. UrlDecodeNextName

function UrlDecodeNextName(U: PUtf8Char; out Name: RawUtf8): PUtf8Char;

Decode a URI-encoded Name from an input buffer
- decoded value is set in Name out variable
- returns a pointer just after the decoded name, after the '='
- returns nil if there was no name=... pattern in U

1.6.170. UrlDecodeNextNameValue

function UrlDecodeNextNameValue(U: PUtf8Char; var Name, Value: RawUtf8): PUtf8Char;

Decode the next Name=Value&.... pair from input URI
- Name is returned directly (should be plain ASCII 7-bit text)
- Value is returned after URI decoding (from %.. patterns)
- if a pair is decoded, return a PUtf8Char pointer to the next pair in the input buffer, or points to #0 if all content has been processed
- if a pair is not decoded, return nil

1.6.171. UrlDecodeNextValue

function UrlDecodeNextValue(U: PUtf8Char; out Value: RawUtf8): PUtf8Char;

Decode a URI-encoded Value from an input buffer
- decoded value is set in Value out variable
- returns a pointer just after the decoded value (may points e.g. to

0 or '&') - it is up to the caller to continue the process or not

1.6.172. UrlDecodeValue

function UrlDecodeValue(U: PUtf8Char; const Upper: RawUtf8; var Value: RawUtf8; Next: PPUtf8Char = nil): boolean;

Decode a specified parameter compatible with URI encoding into its original textual value
- UrlDecodeValue('select=%2A&where=LastName%3D%27M%C3%B4net%27','SELECT=',V,@Next) will return Next^='where=...' and V='*'
- Upper should be already uppercased, and end with a '=' character
- if Upper is not found, Value is not modified, and result is FALSE
- if Upper is found, Value is modified with the supplied content, and result is TRUE

1.6.173. UrlDecodeVar

procedure UrlDecodeVar(U: PUtf8Char; L: PtrInt; var result: RawUtf8; name: boolean);

Decode a UrlEncode/UrlEncodeName() URI encoded string into its original value
- name=false for parameters (after ?), to replace spaces by '+'

1.6.174. UrlEncode

procedure UrlEncode(W: TTextWriter; Text: PUtf8Char; TextLen: PtrInt); overload;

Append a string as URI parameter encoding, i.e. ' ' as '+'

1.6.175. UrlEncode

procedure UrlEncode(W: TTextWriter; const Text: RawUtf8); overload;

Append a string as URI parameter encoding, i.e. ' ' as '+'

1.6.176. UrlEncode

function UrlEncode(Text: PUtf8Char): RawUtf8; overload;

Encode a string as URI parameter encoding, i.e. ' ' as '+'

1.6.177. UrlEncode

function UrlEncode(const NameValuePairs: array of const; TrimLeadingQuestionMark: boolean = false): RawUtf8; overload;

Encode supplied parameters to be compatible with URI encoding
- parameters must be supplied two by two, as Name,Value pairs, e.g.

 url := UrlEncode(['select','*','where','ID=12','offset',23,'object',aObject]);

- parameters names should be plain ASCII-7 RFC compatible identifiers (0..9a..zA..Z_.~), otherwise their values are skipped
- parameters values can be either textual, integer or extended, or any TObject
- TObject serialization into UTF-8 will be processed with ObjectToJson()

1.6.178. UrlEncode

function UrlEncode(const Text: RawUtf8): RawUtf8; overload;

Encode a string as URI parameter encoding, i.e. ' ' as '+'

1.6.179. UrlEncodeName

procedure UrlEncodeName(W: TTextWriter; Text: PUtf8Char; TextLen: PtrInt); overload;

Append a string as URI network name encoding, i.e. ' ' as %20
- only parameters - i.e. after '?' - should replace spaces by '+'

1.6.180. UrlEncodeName

procedure UrlEncodeName(W: TTextWriter; const Text: RawUtf8); overload;

Append a string as URI network name encoding, i.e. ' ' as %20
- only parameters - i.e. after '?' - should replace spaces by '+'

1.6.181. UrlEncodeName

function UrlEncodeName(const Text: RawUtf8): RawUtf8; overload;

Encode a string as URI network name encoding, i.e. ' ' as %20
- only parameters - i.e. after '?' - should replace spaces by '+'

1.6.182. UrlEncodeName

function UrlEncodeName(Text: PUtf8Char): RawUtf8; overload;

Encode a string as URI network name encoding, i.e. ' ' as %20
- only parameters - i.e. after '?' - should replace spaces by '+'

1.7. Variables implemented in the mormot.core.buffers unit

1.7.1. AlgoRle

AlgoRle: TAlgoCompress;

Run-Length-Encoding (RLE) compression as a TAlgoCompress class
- if RLE has no effect during compression, will fallback to plain store

1.7.2. AlgoRleLZ

AlgoRleLZ: TAlgoCompress;

SynLZ compression with RLE preprocess as a TAlgoCompress class
- SynLZ is not efficient when its input has a lot of identical characters (e.g. a database content, or a raw binary buffer) - try with this class which is slower than AlgoSynLZ but may have better ratio on such content

1.7.3. AlgoSynLZ

AlgoSynLZ: TAlgoCompress;

Our fast SynLZ compression as a TAlgoCompress class
- please use this global variable methods instead of the deprecated SynLZCompress/SynLZDecompress wrapper functions

1.7.4. Base64DecodeMain

Base64DecodeMain: function(sp, rp: PAnsiChar; len: PtrInt): boolean;

Raw function for efficient Base64 to binary decoding of the main block
- don't use this function, but rather the Base64ToBin() overloaded functions
- on FPC x86_64, detect and use AVX2 asm for very high throughput (9GB/s)

1.7.5. Base64EncodeMain

Base64EncodeMain: function(rp, sp: PAnsiChar; len: cardinal): integer;

Raw function for efficient binary to Base64 encoding of the main block
- don't use this function, but rather the BinToBase64() overloaded functions
- on FPC x86_64, detect and use AVX2 asm for very high throughput (11GB/s)

1.7.6. EMOJI_AFTERDOTS

EMOJI_AFTERDOTS: array['('..'|'] of TEmoji;

To recognize simple :) :( :| :/ :D :o :p :s characters as smilleys

1.7.7. EMOJI_RTTI

EMOJI_RTTI: PShortString;

Low-level access to TEmoji RTTI - used when inlining EmojiFromText()

1.7.8. EMOJI_TAG

EMOJI_TAG: array[TEmoji] of RawUtf8;

Github/Markdown compatible tag of Emojis, including trailing and ending :
- e.g. ':grinning:' or ':person_with_pouting_face:'

1.7.9. EMOJI_TEXT

EMOJI_TEXT: array[TEmoji] of RawUtf8;

Github/Markdown compatible text of Emojis
- e.g. 'grinning' or 'person_with_pouting_face'

1.7.10. EMOJI_UTF8

EMOJI_UTF8: array[TEmoji] of RawUtf8;

The Unicode character matching a given Emoji, after UTF-8 encoding

1.7.11. LogCompressAlgo

LogCompressAlgo: TAlgoCompress;

Define how files are compressed by TSynLog.PerformRotation
- as used within mormot.core.log.pas unit, and defined in this unit to be available wihout any dependency to it (e.g. in compression units)
- assigned to AlgoSynLZ by default for .synlz which is the fastest for logs
- you may set AlgoGZFast from mormot.core.zip.pas to generate .gz standard files during TSynLog file rotation (with libdeflate if available)
- you may set AlgoLizardFast or AlgoLizardHuffman as non-standard alternatives (default AlgoLizard is much slower and less efficient on logs)
- if you set nil, no compression will take place during rotation
- note that compression itself is run in the logging background thread

1.7.12. LogCompressAlgoArchive

LogCompressAlgoArchive: function(aAlgo: TAlgoCompress; aMagic: cardinal; const aOldLogFileName, aDestinationPath: TFileName): boolean;

Internal wrapper function used by TSynLogArchiveEvent handlers to compress and delete older .log files using our proprietary FileCompress format for a given algorithm
- as used within mormot.core.log.pas unit, and defined in this unit to be available wihout any dependency to it (e.g. in compression units)
- called by EventArchiveLizard/EventArchiveSynLZ to implement .synlz/.synliz archival