mORMot Open Source

lideshi

Member

Registered: 2012-11-24

Posts: 8

Application to read various number of fields in Big Table Files

Hi there,

I am trying to create an application to read Big Table File that could have various number of fields.

for example:

Say one Big Table file is created with these fields:

  Table.AddField('Name',tftWinAnsi,[tfoIndex, tfoUnique]);
  Table.AddField('Country',tftUTF8,[tfoIndex]);

In the future more fields were added and now the file consist of :

  Table.AddField('Name',tftWinAnsi,[tfoIndex, tfoUnique]);
  Table.AddField('Country',tftUTF8,[tfoIndex]);
  Table.AddField('Price',tftInt32,[tfoIndex]);
  Table.AddField('Low_Price',tftInt32,[tfoIndex]);

My plan of attack was that since I can figure out number of fields by using rec.Table.FieldCount.
I can loop through rec.Table.Field[tIndex].Name. to get the field names.  Then by passing
in the field name to rec.GetFieldValue(...) to get the value.  The only problem I found out
is that Table.AddField is not indexed in the order it was called.  Now I can't predict what
Field will be at Index 0 for example.  This is important if I want to display the data in
TListView where I need certain data to be displayed at specific column.

Ultimately what I want is for my application to read any number of fields from the Big Table Field.

Could someone please help out.

Thanks,
Jason

lideshi

Member

Registered: 2012-11-24

Posts: 8

Re: Application to read various number of fields in Big Table Files

I think I can make this work the way I wanted If I could make it so that the first call to Table.AddField will have index 0 and increment from there.
Any suggestion on how to make this happen?  Maybe there should a flag to allow you to do it.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 13,165

Website

Re: Application to read various number of fields in Big Table Files

The field order is the one on file layout itself.
So changing the order expect changing the file layout.

BigTable core has to be modified to allow this.
By now, for performance reasons, new fields are added at the end of existing row data.

I suspect the easiest is to use a lookup table in your UI application just to reorder the fields in the expected order.

lideshi

Member

Registered: 2012-11-24

Posts: 8

Re: Application to read various number of fields in Big Table Files

By now, for performance reasons, new fields are added at the end of existing row data.

Then why is it that when I call AddField in the following order

Table.AddField('Name',tftWinAnsi,[tfoIndex, tfoUnique]);
Table.AddField('Country',tftUTF8,[tfoIndex]);
Table.AddField('Price',tftInt32,[tfoIndex]);
Table.AddField('Low_Price',tftInt32,[tfoIndex]);

I don't get Name for Field Index 0, Country for Field Index 1, Price for Field Index 2 and Low_Price for Field Index 3 ? 

Am I misunderstanding your quote when you say new fields are added at the end of existing row data?

Thanks,
Jason

Last edited by lideshi (2012-11-26 04:26:07)

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 13,165

Website

Re: Application to read various number of fields in Big Table Files

Indexed fields are added left side, that's why they are added in order.

See TSynTable.AddField() code.

lideshi

Member

Registered: 2012-11-24

Posts: 8

Re: Application to read various number of fields in Big Table Files

I just tested this:

  Table := TSynBigTableRecord.Create('MyTestTable','TableName');
  Table.AddField('Price',tftInt32,[tfoIndex]);
  Table.AddField('Country',tftUTF8,[tfoIndex]);
  Table.AddField('Name',tftWinAnsi,[tfoIndex, tfoUnique]);
  Table.AddField('Low_Price',tftInt32,[tfoIndex]);
  Table.AddFieldUpdate;

  rec.Init(Table.Table);

  for tIndex := 0 to rec.Table.FieldCount - 1 do
    Memo1.Lines.Add(rec.Table.Field[tIndex].Name)

I see the following result:

Low_Price
Price
Country
Name

Shouldn't it be in this order if it's added to the left? :

Low_Price
Name
Country
Price

Anything I did wrong here?

Last edited by lideshi (2012-11-26 13:58:12)

lideshi

Member

Registered: 2012-11-24

Posts: 8

Re: Application to read various number of fields in Big Table Files

I modified the code in TSynTable.AddField to be the following:

function TSynTable.AddField(const aName: RawUTF8;
  aType: TSynTableFieldType; aOptions: TSynTableFieldOptions): TSynTableFieldProperties;
var aSize: Integer;
begin
  result := nil;

  aSize := FIELD_FIXEDSIZE[aType];

  if (self=nil) or (aSize=0) or IsRowID(pointer(aName)) or
     not FieldNameValid(pointer(aName)) or (GetFieldFromName(aName)<>nil) then
    exit;
  result := TSynTableFieldProperties.Create;

  if fAddedField=nil then
    fAddedField := TList.Create;

  fAddedField.Add(result);
  result.Name := aName;
  result.FieldType := aType;

  if tfoUnique in aOptions then
    Include(aOptions,tfoIndex); // create an index for faster Unique field

  result.FieldNumber := fField.Add(result);

  AfterFieldModif; // set Offset,FieldNumber,FieldSize fFieldVariableIndex/Offset
end;

And in AfterFieldModif method I commented out

 //assert(Offs>=0); 

Now I am getting the result that I wanted.  When I tested my code again:

  Table := TSynBigTableRecord.Create('MyTestTable','TableName');
  Table.AddField('Price',tftInt32,[tfoIndex]);
  Table.AddField('Country',tftUTF8,[tfoIndex]);
  Table.AddField('Name',tftWinAnsi,[tfoIndex, tfoUnique]);
  Table.AddField('Low_Price',tftInt32,[tfoIndex]);
  Table.AddFieldUpdate;

  rec.Init(Table.Table);

  for tIndex := 0 to rec.Table.FieldCount - 1 do
    Memo1.Lines.Add(rec.Table.Field[tIndex].Name)

I am getting the result:

Price
Country
Name
Low_Price

My only concern now is that with the code modification I made, do you think I could have broken something else?

Thanks again!

lideshi

Member

Registered: 2012-11-24

Posts: 8

Re: Application to read various number of fields in Big Table Files

I don't think my way of solving the issue is efficient.  Can you provide your opinion on how I should tackle this problem. 

Do you think it's possible to view the BigTable File without knowing the fields' name or type ? and display it all out?

If you could provide a simple code example I would greatly appreciated.

Thanks.

ab

Administrator

From: France

Registered: 2010-06-21

Posts: 13,165

Website

Re: Application to read various number of fields in Big Table Files

What is wrong with using a lookup table at the client level, without touching the main library?