You are not logged in.
Pages: 1
Ok. Many thanks.
Hi
I have some really big files. Usually they stay in 1-6GB range. But sometimes these file can be even up to 14GB large.
Each file has several millions of entries like this (this is a simplified example):
string_about_100_to_1000_chars a_short_string number1 number2 number3 ... number10 <--------- there is no more than 2KB of data on each entry
I need to:
1. display all rows in a TStringGrid (this is easy because I can only load from disk the data associated with the visible rows)
2. to sort the data (when the user click a column header).
My questions are:
1. can I use Big Table for this
2. if yes, can you please provide a skeleton example and some basic concepts on how to start (to store my data into the table for 'fast' sorting on computers with 2-4GB RAM)?
2. will it work also on Lazarus?
Many thanks
I agree with Ab.
Delphi XE is really slow and buggy. It will take some time until they will release Update 2, 3, 4 (and maybe 5) to fix all those bugs.
Hi.
There is some way (email, forum, twitter, newslist, etc) to find out when a new version of BigTable (or other library) is released?
>when all AddField() methods have been called
I imagined that but I had to be sure
Thanks.
If I add more fields, I have to call Table.AddFieldUpdate after EACH AddField? Or just once, after all AddField calls is ok?
Wow. This is great . Thanks.
Hi.
I have a question: When to use tfoIndex, tfoUnique, tfoCaseInsensitive?
Let's say I want to store in the table two strings, some binary data and some integers. Something like this:
fdName := Table.AddField('Name' , tftWinAnsi, [tfoIndex] );
fdBase := Table.AddField('Base' , tftWinAnsi, [] );
fdNoOf := Table.AddField('NoOf' , tftInt32 , [] );
When I read the table back, I need only few of its columns (the entire column) - I just read the columns in memory and process the data. So I don't do searches in the table. Is it ok is none of the fields have the tfoIndex and tfoUnique attribute set?
Also, if I set to tfoCaseInsensitive attribute, it will work faster (usually CaseInsensitive strings are faster, right)?
Thanks
> Some more simple sample is needed...
Yes. It took me quite a while until a figure out basic stuff (how to create/add/update records, etc).
Anyway, now that I figured out most of these basic stuffs, the library is great.
I will continue to test it. If I find other strange stuff I let you know.
Thanks.
There might be another issue. This time in the example provided by you in your blog. The record obtained using the variant code is not 1 but 0. The ID obtained with the non-variant code is 1. Maybe I have done something wrong. Also maybe there is another bug. Just take a look.
...
Table.UpdateToFile;
{ VERIFY - READ RECORD: }
ReadRec:= Table.RecordGet(aRecID);
assert(ReadRec.ID= aRecID);
{ a) CLASSIC }
s:= ReadRec.GetFieldValue(fdBases);
Form1.lblString.caption:= s;
Assert(s= ctTestStr);
{ b) VARINAT }
Vari:= Table.VariantGet(aRecID);
vID:= vari.ID;
assert(vari.Base= ctTestStr);
Assert(vID= aRecID); // <-- this fails
...
------------------------
I have also a suggestion.
The example (TestBigTable) provided by you is EXTREMELY difficult to follow because it is a function that contina lots of other functions. It will be nice if the example will be provided in a separated file and without that function-in-function structure.
Thanks.
Where (which page) will be the update released?
Hi.
I want to create a table and and to write to it records. Each record contains a string and an integer:
Table:= TSynBigTableRecord.Create('MyFirstDB.DB','Tabel1');
TRY
fdBases := Table.AddField ('Base' , tftWinAnsi, [tfoIndex] );
fdNoOfBas := Table.AddField( 'NoOfBase', tftInt32 , [tfoIndex, tfoUnique] );
Table.AddFieldUpdate;
Tomorrow I may add a boolean field also:
procedure TForm1.WriteExtraFields;
Table:= TSynBigTableRecord.Create('MyFirstDB.DB','Tabel1');
TRY
Table.AddField ('bool', tftBoolean);
Table.AddFieldUpdate;
But forget tomorrow. I want to read the table today, while the third field was not yet added. But the program crashes here:
{ Retrieve fields }
aStr:= ReadRec.GetFieldValue(fdBases);
aInt:= ReadRec.GetFieldValue(fdNoOfBas);
bol := ReadRec.GetFieldValue(fdBoolean); <-------- Crash here.
How do I test if a field exists before reading it. There is something like: "if Table.FieldExists then DoStuff" ?
----------------------
EDIT:
I also have problems updating the current records when adding the new column:
procedure TForm1.WriteExtraFields;
VAR
i: Integer;
Table: TSynBigTableRecord;
CurRec: TSynTableData;
NewField: TSynTableFieldProperties;
begin
{ Create table }
Table:= TSynBigTableRecord.Create('MyFirstDB.DB','Tabel1');
TRY
{ Define an extra fields/column to be added to the table }
NewField:= Table.AddField ('bool', tftBoolean);
Table.AddFieldUpdate;
for i:= 1 to 10 DO
begin
{ GET CUR RECORD }
CurRec.Init(Table.Table);
CurRec:= Table.RecordGet(i);
{ EDIT RECORD }
CurRec.SetFieldValue(NewField, TRUE);
{ WRITE BACK RECORD }
if NOT Table.RecordUpdate(CurRec) <-------- Error here. It returns false
then Log.AddError('Error adding record');
end;
Table.UpdateToFile;
FINALLY
FreeAndNil(Table);
END;
end;
Hi.
I discovered recently your web site. Lots of nice articles!!
I have also found the fast JPEG decoder and intended to use it immediately, but lucky me I have found a post on this forum where you say that it won't with progressive JPEG files. Well, 90% of existing jpg files are progressive (which is the only way they should be!) so basically your library cannot be used in the real world. Or there is a trick?
Pages: 1