You are not logged in.
The ColumnDataSize indicates the number of bytes that remain to read, if data is truncated. But previous logic was based on the assumption that it indicates the actual bytes count in the received buffer fColData.
My fix is not optimized and I suggested this as a quick fix. Actually, it works only because Copy function ignore size parameter if it grater then string size.
If BLOB data is more then 64 bytes, function RetrieveBlob returns data with garbage (part of data + null bytes + another part of data + more nulls etc.).
It seems that bug in function TODBCStatement.ColumnBlob (SynDBODBC.pas). I comment buggy usage of ColumnDataSize and add one line to fix a bug:
function TODBCStatement.ColumnBlob(Col: integer): RawByteString;
var res: TSQLDBStatementGetCol;
offs: integer;
begin
res := GetCol(Col,ftBlob);
case res of
colNull: result := '';
colWrongType: ColumnToTypedValue(Col,ftBlob,result);
else
with fColumns[Col] do begin
result := copy(fColData[Col],1,ColumnDataSize);
offs := 0;
while res=colDataTruncated do begin
//inc(offs,ColumnDataSize);
res := GetColNextChunk(Col);
if ColumnDataSize<=0 then
break;
//SetLength(result,offs+ColumnDataSize);
//move(pointer(fColData[Col])^,PByteArray(result)^[offs],ColumnDataSize);
result := result + copy(fColData[Col],1,ColumnDataSize); // < bug fix >
end;
end;
end;
end;Bug reproduced on MySQL 5.5 and PostgreSQL 9.3 via ODBC drivers.
I just googled that JavaScript bit-level operations are limited to 32 bit. You can use some library to work with 64-bit integers in JS, for example: http://google.github.io/closure-library … _Long.html
Take a look to the mORMot sources (SynCommons.pas):
// bits: S=0..5 M=6..11 H=12..16 D=17..21 M=22..25 Y=26..38
// size: S=6 M=6 H=5 D=5 M=4 Y=12
// i.e. S<64 M<64 H<32 D<32 M<16 Y<4096: power of 2 -> use fast shl/shr
procedure TTimeLogBits.From(Y, M, D, HH, MM, SS: cardinal);
begin
inc(HH,D shl 5+M shl 10+Y shl 14-(1 shl 5+1 shl 10));
Value := SS+MM shl 6+Int64(HH) shl 12;
end;Why I should use different strings in Update() and UpdateField() methods? I think that behaviour in this methods must be the same.
Besides that, with SQLite3 or DBMS, UpdateField() works fine, error only with MongoDB, and if I wish use the same code with SQLite3 and MongoDB, I should use \\ or \ depends on the DB backend? I think that it's bad idea.
In any case, I decided not to use UpdateField() method in my code anymore.
New bug: if we try update text field with "\" symbol it's erase this symbol:
VStr := StringToUTF8('Test string\with slash\with 2 slashes');
VRes := VClientDB.UpdateField(TTestDoc, VDoc.ID, 'Field1', [VStr]);TSQLRestServerDB(023D6D40).URI(PUT root/TestDoc?setname=Field1&set=%22Test+string%5Cwith+slash%5Cwith+2+slashes%22&wherename=RowID&where=1 inlen=0)
{collection:"test.TestDoc",opCode:"opUpdate",requestID:7,selector:{_id:1},update:{$set:{Field1:"Test stringwith slashwith 2 slashes"}}}Test code:
uses
mORMot,
mORMotSQLite3,
mORMotMongoDB,
SynMongoDB,
SynSQLite3Static,
SynCommons;
type
TTestDoc = class(TSQLRecord)
private
FField1: RawUTF8;
FField2: Int64;
published
property Field1: RawUTF8 read FField1 write FField1;
property Field2: Int64 read FField2 write FField2;
end;
procedure TestUpdateField;
var
VRes: Boolean;
VStr: RawUTF8;
VDoc: TTestDoc;
VModel: TSQLModel;
VClient: TMongoClient;
VDatabase: TMongoDatabase;
VClientDB: TSQLRestClientDB;
begin
VClient := TMongoClient.Create('127.0.0.1', 27017);
try
VDatabase := VClient.Open('test');
VModel := TSQLModel.Create([TTestDoc]);
try
VClientDB := TSQLRestClientDB.Create(VModel, nil, ':memory:', TSQLRestServerDB);
try
if not StaticMongoDBRegisterAll(VClientDB.Server, VDatabase) then begin
Exit;
end;
VClientDB.Server.CreateMissingTables;
VDoc := TTestDoc.Create;
try
VStr := StringToUTF8('Test string\with slash');
VDoc.FField1 := VStr;
VDoc.FField2 := 123456;
if not (VClientDB.Add(VDoc, True) > 0) then begin
Assert(False);
Exit;
end;
VStr := StringToUTF8('Test string\with slash\with 2 slashes');
VRes := VClientDB.UpdateField(TTestDoc, VDoc.ID, 'Field1', [VStr]);
Assert(VRes);
VRes := VClientDB.Retrieve(VDoc.ID, VDoc);
Assert(VRes);
Assert(VDoc.FField1 = VStr);
Assert(VDoc.FField2 = 123456);
finally
VDoc.Free;
end;
finally
VClientDB.Free;
end;
finally
VModel.Free;
end;
finally
VClient.Free;
end;
end;In same time, method TSQLRestClient.Update() is save correct text with all slashes.
With MongoDB as a backend, TSQLRestClient.UpdateField() generate bad request and always replace whole document:
{collection:"MyDB.MyColl",opCode:"opUpdate",requestID:41,selector:{_id:1},update:{MyField:1}}Correct request should use $set keyword:
{collection:"MyDB.MyColl",opCode:"opUpdate",requestID:41,selector:{_id:1},update:{$set:{MyField:true}}}Please, fix it.
Thanks a lot.
I test it with mongod 2.6, 3.0.4 and with MongoLab and it works! ab, you’re awesome.
Auto-detect authentication mechanism by server version is a good idea, but to be on the safe side, сan you make option to use MONGODB-CR with MongoDB 3.xx?
Bad news: MongoDB 3.x has 5 (!) auth mechanisms, and MONGODB-CR that I implement is deprecated. Now default is SCRAM-SHA-1 and MongoLab use it.
I found implementations of all mechanisms in one place in PyMongo driver: auth.py and SCRAM is terrible.
Here is implementation of TMongoClient.OpenAuth method:
uses
SynCrypto;
...
function TMongoClient.OpenAuth(const DatabaseName, UserName,
PassWord: RawUTF8): TMongoDatabase;
var res,bson: variant;
err,nonce,key: RawUTF8;
begin
if (self=nil) or (UserName='') or (PassWord='') then
result := nil else begin
result := TMongoDatabase(fDatabases.GetObjectByName(DatabaseName));
if result=nil then begin // not already opened -> try now from primary host
if not fConnections[0].Opened then begin
fConnections[0].Open;
// step 1
bson := BSONVariant('{getnonce:1}');
err := fConnections[0].RunCommand(DatabaseName,bson,res);
if err<>'' then
raise EMongoException.CreateUTF8('%.OpenAuth("%") step1 error: %',[self,DatabaseName,err]);
// step 2
nonce := _Safe(res)^.GetValueOrRaiseException('nonce');
key := MD5(nonce + UserName + MD5(UserName + ':mongo:' + PassWord));
bson := BSONVariant('{authenticate:1,user:?,nonce:?,key:?}',[],[UserName,nonce,key]);
err := fConnections[0].RunCommand(DatabaseName,bson,res);
if err<>'' then
raise EMongoException.CreateUTF8('%.OpenAuth("%") step2 error: %',[self,DatabaseName,err]);
end;
result := TMongoDatabase.Create(Self,DatabaseName);
fDatabases.AddObject(DatabaseName,result);
end;
end;
if result=nil then
raise EMongoException.CreateUTF8('%.Open("%") unknown DB',[self,DatabaseName]);
end;Tested on localhost with MongoDB 2.6 and Delphi 2007.
Based on MongoDB docs: Implement Authentication in a Driver
Simple call aMessarts.Free after use?
If I need insert item into sorted array, I can do it like this:
VDynArr.Add(I);
VDynArr.Sort;but I think that it's not a very optimized and I suggest to do it differently:
if not VDynArr.FastLocateSorted(I, VNewIndex) then begin
VDynArr.Insert(VNewIndex, I);
end;After this Insert operation, array will still be sorted.
Implementation of FastLocateSorted is (tested on Delphi 2007):
type
TDynArrayExt = object(TDynArray)
public
function FastLocateSorted(const Elem; out Index: Integer): boolean;
end;
function TDynArrayExt.FastLocateSorted(const Elem; out Index: Integer): boolean;
var n, L, i, cmp: integer;
P: PAnsiChar;
begin
result := False;
Index := -1; // return -1 if not Sorted
if fSorted and (@fCompare<>nil) then begin
P := fValue^;
L := 0;
n := Count - 1;
while L <= n do begin
i := (L+n) shr 1;
cmp := fCompare(P[cardinal(i)*ElemSize],Elem);
if cmp=0 then begin
L := i;
result := True;
Break;
end else
if cmp<0 then
L := i+1 else
n := i-1;
end;
// if result is True - return the index of existing Elem
// if result is False - return the index where to insert
Index := L;
end;
end;...and small test:
procedure Test;
var
I, VIndex, VNewIndex, VCount: Integer;
VIntArr: TIntegerDynArray;
VDynArr: TDynArrayExt;
begin
VDynArr.InitSpecific(TypeInfo(TIntegerDynArray), VIntArr, djInteger, @VCount);
for I := 1 to 6 do begin
VDynArr.Add(I);
end;
VDynArr.Sorted := True;
I := 3;
VIndex := VDynArr.Find(I);
if VIndex >= 0 then begin
VDynArr.Delete(VIndex);
VDynArr.Sorted := True;
if not VDynArr.FastLocateSorted(I, VNewIndex) then begin
Assert(VNewIndex >= 0);
Assert(VNewIndex = VIndex);
VDynArr.Insert(VNewIndex, I);
VDynArr.Sorted := True;
end;
end;
end;I think it will be useful to see this functional in TDynArray. Or maybe you know other way to do it?
+1 for Zip64
BBackSoon
Try this: https://gist.github.com/
Gist is a simple way to share snippets and pastes with others. All gists are Git repositories, so they are automatically versioned, forkable and usable from Git.
Fixed. Thanks.
Hi. It seems, that "EchoToConsole" not work for me:
program SynLogTest;
{$APPTYPE CONSOLE}
uses
SysUtils, SynLog, SynCommons;
procedure DoTest;
begin
with TSynLog.Family do begin
Level := LOG_VERBOSE;
EchoToConsole := LOG_VERBOSE;
end;
TSynLog.Add.Log(sllDebug, StringToUTF8('Test message'));
TSynLog.Add.Flush(True);
end;
begin
try
Writeln(SynCommons.SYNOPSE_FRAMEWORK_FULLVERSION); // 1.18.1048 FTS3
DoTest;
Readln;
except
on E:Exception do
Writeln(E.Classname, ': ', E.Message);
end;
end.Log file is created and contains test message, but console output is empty.
Tested on Delphi 2007 and XE2.
ab
Thanks, bug fixed.
Hi, I try save some object to JSON, like this (Delphi 2007):
type
TMyClass = class
private
FText: string;
published
property Text: string read FText write FText;
end;
...
var
m: TMyClass;
s: RawUTF8;
begin
m := TMyClass.Create;
m.Text := 'One-Один'; // chars in win-1251 encoding (Russian)
s := ObjectToJson(m, [woStoreClassName]);
...
end;and result is:
{"ClassName":"TMyClass","Text":"One-"}all Russian chars just disappeared!
Then I take a look to the mORMot sources and found that in string conversions (in my case string type is tkLString) used AnsiStringCodePage property. And this CodePage is always 65535. I talk about this part of code in mORMot.pas:
procedure TJSONSerializer.WriteObject(Value: TObject; Options: TTextWriterWriteObjectOptions);
...
{$ifdef FPC}tkAString,{$endif} tkLString: begin
HR(P);
Add('"');
P^.GetLongStrProp(Value,tmp);
AddAnyAnsiString(tmp,twJSONEscape,P^.PropType^.AnsiStringCodePage); // AnsiStringCodePage = 65535 ($FFFF)
Add('"');
end;I think that problem in TSynAnsiConvert.Engine function (SynCommons.pas). When I modify it and set to return CurrentAnsiConvert if CodePage is 65535, all my code starts work as expected.
class function TSynAnsiConvert.Engine(aCodePage: cardinal): TSynAnsiConvert;
var i: integer;
begin
if SynAnsiConvertList=nil then begin
GarbageCollectorFreeAndNil(SynAnsiConvertList,TObjectList.Create);
CurrentAnsiConvert := TSynAnsiConvert.Engine(GetACP);
WinAnsiConvert := TSynAnsiConvert.Engine(CODEPAGE_US) as TSynAnsiFixedWidth;
UTF8AnsiConvert := TSynAnsiConvert.Engine(CP_UTF8) as TSynAnsiUTF8;
end;
-- if aCodePage=0 then begin
++ if (aCodePage=0) or (aCodePage=65535) then begin
result := CurrentAnsiConvert;
exit;
end;
with SynAnsiConvertList do
for i := 0 to Count-1 do begin
result := List[i];
if result.CodePage=aCodePage then
exit;
end;
if aCodePage=CP_UTF8 then
result := TSynAnsiUTF8.Create(CP_UTF8) else
if aCodePage=CP_UTF16 then
result := TSynAnsiUTF16.Create(CP_UTF16) else
if IsFixedWidthCodePage(aCodePage) then
result := TSynAnsiFixedWidth.Create(aCodePage) else
result := TSynAnsiConvert.Create(aCodePage);
SynAnsiConvertList.Add(result);
end;What do you think?