mORMot Open Source

esmondb

Member

From: London

Registered: 2010-07-20

Posts: 299

FTS5 records

TSQLRecordFTS5 is causing an error in createMissingTables. The simple tokenizer seems to have been removed from FTS

see:
https://sqlite.org/fts5.html#tokenizers

esmondb

Member

From: London

Registered: 2010-07-20

Posts: 299

Re: FTS5 records

in TSQLRecord.GetSQLCreate in mORMot.pas changing

tokenizer := 'simple';

to

if Props.Kind = rFTS5 then
  tokenizer := 'unicode61'
else
  tokenizer := 'simple';

should fix it.
Also in the comments it mentions 'unicode64' a few times - I assume that's a typo.

esmondb

Member

From: London

Registered: 2010-07-20

Posts: 299

Re: FTS5 records

Incidentally TSQLRecordFTS5Unicode61 saves a few % in DB size compared to TSQLRecordFTS4 with the simple tokenizer.