You are not logged in.
Pages: 1
Hi,
First af all, I have to say thanks for this great artifact called mORMot!
I'm following it for a while, waiting the right project to use it, and finally the day is come!
I started to develop the backend, and I have some questions/suggestion requests on various topics, so I will ask them one by one in the hoping to help someone like me approaching for the first time to mORMot.
I followed your advice in this article
http://blog.synopse.info/post/2012/03/2 … ed-service
to return a given file (a PDF in my case) setting
Result.Header := HEADER_CONTENT_TYPE + 'application/pdf';
and it works wonderfully!
Now I have to implement a 'symmetric' method, to upload a binary file to the mORMot server.
Wich is the preferred/optimized way to do it? I mean, there is a way to avoid the Base64 encoding and save bandwidth also uploading?
Offline
The best is to use a simple POST HTTP method with the content as body.
In fact, this is how our RESTful ORM model handle BLOB fields.
But sadly it won't work with interface-based services, only with method-based services.
It could make sense to add support for incoming raw content for interface-based services, e.g. if the only incoming parameter of a method is a "const InputBody: TServiceCustomAnswer" record, then it may contain the incoming body, without expecting the input body to be a JSON array of parameters.
You may create a feature request ticket for this, in http://synopse.info/fossil/tktnew
Note that there is no support to MIME-PART encoding yet in our framework.
Offline
In my project I use method to upload (MIME-PART encoding) content from Javascript:
procedure TSettingsServer.UploadContent(Ctxt: TSQLRestServerURIContext);
var
FBody : RawUTF8;
boundary : string;
ContentType : String;
Header : TStrings;
Decoder : TMimeDecoder;
i : Integer;
...
begin
...
ContentType := UTF8ToString(FindIniNameValue(pointer(Ctxt.Call.InHead),'CONTENT-TYPE: '));
Header := TStringList.Create;
try
ExtractHeaderFields([';'], [' '], PChar(ContentType), Header, False, False);
Boundary := Header.Values['boundary'];
finally
Header.Free;
end;
FBody := Ctxt.Call.InBody;
Decoder := TMimeDecoder.Create;
try
Decoder.AllContent := (FBody);
Decoder.Boundary := StringToAnsi7(boundary);
Decoder.UploadPath := DefaultWorkPlace + Format('%s\%d\%s\%s\', [sType, FWPID, SGuid, SFolder]);
if not ForceDirectories(Decoder.UploadPath) then
begin
Ctxt.Error(FormatUtf8('Unable to create folder: %', [Decoder.UploadPath]));
exit;
end
else
Decoder.ProcessMimeData(Length(FBody));
finally
Decoder.Free;
end;
end;
Source of TMimeDecoder class:
unit ProcessMime;
interface
uses System.Classes, System.SysUtils, Web.HTTPApp, SynCommons;
Type
TMimeDecoder = class
private
Data : RawByteString;
function ReadMultipartRequest(const Boundary: RawByteString;
ARequest: RawByteString; var AHeader: TStrings; var Data: RawByteString): RawByteString;
public
AllContent, Boundary: RawByteString;
UploadPath : String;
procedure ProcessMimeData(ContentLength: Cardinal);
end;
implementation
function TMimeDecoder.ReadMultipartRequest(const Boundary: RawByteString;
ARequest: RawByteString; var AHeader: TStrings; var Data: RawByteString): RawByteString;
var
Req, RHead: RawByteString;
i: Integer;
begin
Result := '';
AHeader.Clear;
Data := '';
if (Pos(Boundary, ARequest) < Pos (Boundary + '--', ARequest))
and (Pos(Boundary, ARequest) = 1) then
begin
Delete(ARequest, 1, Length(Boundary) + 2);
Req := Copy(ARequest, 1, Pos(Boundary, ARequest) - 3);
Delete(ARequest, 1, Length(Req) + 2);
RHead := Copy(Req, 1, Pos(#13#10#13#10,Req)-1);
Delete(Req, 1, Length(RHead) + 4);
AHeader.Text := RHead;
for i := 0 to AHeader.Count - 1 do
if Pos(':', AHeader.Strings[i]) > 0 then
AHeader.Strings[i] := Trim(Copy(AHeader.Strings[i], 1,
Pos(':', AHeader.Strings[i])-1)) + '=' + Trim(Copy(AHeader.Strings[i],
Pos(':', AHeader.Strings[i])+1, Length(AHeader.Strings[i]) -
Pos(':', AHeader.Strings[i])));
Data := Req;
Result := ARequest;
end
end;
procedure TMimeDecoder.ProcessMimeData(ContentLength: Cardinal);
var
Header, HList: TStrings;
FileName : String;
begin
if ContentLength = Length(AllContent) then
while Length(AllContent) > Length('--' + Boundary + '--' + #13#10) do
begin
Header := TStringList.Create;
HList := TStringList.Create;
try
AllContent := ReadMultipartRequest('--' + Boundary, AllContent,
Header, Data);
ExtractHeaderFields([';'], [' '],
PChar(Header.Values['Content-Disposition']), HList, False, True);
if (Header.Values['Content-Type'] <> '') and (Data <> '') then
begin
FileName := Utf8ToAnsi(ExtractFileName(HList.Values['filename']));
FileFromString(Data, UploadPath + FileName, true);
end
finally
Header.Free;
HList.Free;
end;
end;
end;
end.
Offline
It would be great to create a topic with links to the software created using mORMot.
Offline
Indeed!
Sounds like if more and more projects are using our little rodent...
The cross-platform clients did increase its popularity, as far as I found out.
And when you compare with alternatives (DataSnap, or even the TMS Business Subscription) about features, implementation patterns or performance, we may honestly say that we can be proud of our Open Source solution!
Feel free to create a dedicated forum thread (as we did for third-party samples).
I will stick the topic to the "mORMot Framework" category of this forum.
We may also include the list in the documentation, for reference.
Offline
Thanks for such a quick reply!
@ab: if I undestend well your proposal to add support for incoming raw content for interface-based services, then there will be one and only one TServiceCustomAnswer parameter, right?
If so, how the interface service will be able to properly handle such data, without any other input parameters such, say, an ID Customer to link the content to?
@DigDiver: thanks for sharing your code!
I'm not yet able to undestend well this 'low-level' use of mORMt... guess I have to dig a little under the surface!
Anyway thanks for pointing me in this direction!
Offline
..and about the links to the softwares created with mORMot, I will add a link to mine with pleasure!
In fact, another question I have is how I can mention mORMot in my application(s)? Something like 'powered by mORMot' and a link?
Offline
@CycleSoft
Good point.
We may allow other parameters in addition to TServiceCustomAnswer record (perhaps renamed TServiceCustomRequest ) but expecting them to be encoded at the URI level.
In all cases, this would work only for TSQLRestRoutingREST kind of routing, in its ExecuteSOAByInterface method.
Note that DigDiver code uses a method-based service, not an interface-based service here.
Offline
@CycleSoft
Good point.
We may allow other parameters in addition to TServiceCustomAnswer record (perhaps renamed TServiceCustomRequest ) but expecting them to be encoded at the URI level.
In all cases, this would work only for TSQLRestRoutingREST kind of routing, in its ExecuteSOAByInterface method.
My knowledge of mORMot (and HTTP-world in general) is too superficial to be meaningful, but my impression is that if to implement this feature you have a hard time and limitations, maybe this is not the right direction and there is a cleaner solution?
A newbie (like me) will easily be tricked, I don't even know what 'TSQLRestRoutingREST kind of routing' is
If you already are using the 'method' approach for the BLOBs, there will be a good reason, so I prefer learn something new and adopt your approach.
That said, can you only point me in the right direction? Can I find a start point in the documentation or is better to search your source code?
Note that DigDiver code uses a method-based service, not an interface-based service here.
OK!
Offline
Just take a look at the "26 - RESTful ORM" sample, which implement a BLOB uploading/download mechanism, using a method-based service.
Offline
Just to 'close' the thread...
I successfully built a method service using POST requests as hinted by Arnaud, and with the very valuable help of DigDiver that shared his TMimeDecoder class that I have slightly modified ( with Delphi XE4, ExtractHeaderFields returns 'values' in the Strings parameter with still the double quotes round them, so the filename field is not usable 'as-is', not sure how it works for you DigDiver)
Again thanks to you guys!
Now I have to be sure that my implementation is thread-safe, but this is worth another..... well... thread
Offline
many thanks to all of you.
but i have another issue:
the memory of server kept increasing after each Binary Data was posted to the server, even if the server did not handle it at all.
Offline
In moRMot, memory blocks coming from HTTP is handled with RawByteString (or RawUTF8) variables, so there should not be any memory leak.
Unless your code still refers to the incoming data.
The framework is tested for any memory leak, and is known to run for weeks in production, without any memory increase.
I suspect there is a memory leak in your code.
For instance, you create a temporary stream on your side, without releasing it.
Enable memory leak reporting in Delphi, and find out what is wrong.
Offline
i compiled "13 - StandAlone JSON SQL server" without changing any thing, and run the JSONSQLServer.exe, then used the code below to post data to the server, not post of REST, but post of URL, the memory of server kept increasing after each data was posted to the server.
procedure TForm8.Button11Click(Sender: TObject);
var
tmpstream:TMemoryStream;
ab:boolean;
HTTP: THTTPSend;
url:string;
slist:TStringlist;
begin
tmpstream := TMemoryStream.Create;
HTTP := THTTPSend.Create;
slist:=TStringlist.Create;
try
url := 'http://localhost:888/root';
tmpstream.LoadFromFile('c:\rtl70.bpl');
tmpstream.Position := 0;
ab := HttpPostFile(url,'abc','def',tmpstream,slist);
... ...
finally
tmpstream.Free;
HTTP.Free;
slist.free;
end;
end;
Last edited by profh (2014-09-19 09:37:41)
Offline
i did it because i need to post data from browser, but REST methord can not be used in browser as usual .
Offline
Weird.
There is no memory leak when running our extensive regression tests.
I doubt there is a memory leak - what does exist is that the memory increases a little for the first requests, until the thread pool is completely used and memory is stabilized.
Which version are you using? Current is 1.18.264.
What does FastMM4 state, when the server is run with EnableMemoryLeakReporting conditional?
Offline
i used the latest version, and i enabled EnableMemoryLeakReporting, but no memory leak reported.
Offline
please note that i just post data to server from client, and the server did not handle the data at all.
it seems that the server will be attacked by continuing data whatever the client postes.
Last edited by profh (2014-09-19 14:09:17)
Offline
it is strange, and it really bothers me for a little while.
Offline
We send gigabytes of files in production without problems. Our clients is browser what use this technics .
Is memory usage continue increasing after 100,200,300 requests?
Offline
you are absolutely right, after certain of requests the memory usage is not continue increasing.
and thank you so much.
Offline
got it.
after a little while, without request, the memory usage of server will decrease.
thanks again.
Offline
Pages: 1