You are not logged in.
Pages: 1
QuickLZ's site (quicklz.com) says:
QuickLZ is the world's fastest compression library, reaching 308 Mbyte/s per core
how SynLZ compares?
plaintext:
Why Use Fossil? There are plenty of open-source version control systems available on the internet these days. What makes Fossil worthy of attention? Bug Tracking And Wiki - In addition to doing distributed version control like Git and Mercurial, Fossil also supports
zip compressed: (fastest)
PK a§#@~®Op´
whyusefossil3.txteŽ1nAE¯ò¸JŠ
:ÐÖfÇa-Æ+ÛÚÛÇ+…*…ëý÷†iÅÕßê.õ€ËÄÆ ¼¥r‹ú]¸í\»Œ'›‹6ŒÚ´ÂW~8èIRéVÆÄlc{r¿Ðê{4³ÿñR‹TH
E$0·øêw\ŒÆYÚŸ-`Y°Ã©J'-„PÝò"&·\þÙU™G PŽœÙÆnBõã
§ê
ïË'þPK a§#@~®Op´
whyusefossil3.txtPK ? ã
synlz compressed:
Ç Why Use Fossil? There aAwplenty o f open-source version control sy └stems available æ╞the internet !ΓQw ┴days. What makesE
wort±╢æFatt1Ʊÿ? B Çug Tracking And Wiki - In addit±ÿ@ to do≥üdistributedlike Git aüfMe rcurial,F
also supports
What I say is, on synlz compressed data a few plain text is visible (especially those few from the start of file?) on the top of compressed data. This is a concern for very small text files for which whole plaintext is visible.
In the case of zip compression (even with fastest mode) the plaintext is never visible no matter how small the source file is.
Is there any way you can adopt this in synlz?
the Thread Pool facility is available only if I am going to use a HTTP based class (say THttpServer)
What if I need thread pool when I am using only direct socket class? (i.e TCrtSocket)
Can you provide a class/function to achieve this?
In your code, you mentioned that Synopse is faster than indy or synapse. I think all these three are blocking sockets.
/// Fast low-level Socket implementation
// - direct access to the OS (Windows, Linux) network layer
// - use Open constructor to create a client to be connected to a server
// - use Bind constructor to initialize a server
// - use direct access to low level Windows or Linux network layer
// - use SockIn and SockOut (after CreateSock*) to read or write data
// as with standard Delphi text files (see SendEmail implementation)
// - if app is multi-threaded, use faster SockSend() instead of SockOut^
// for direct write access to the socket; but SockIn^ is much faster than
// SockRecv() thanks to its internal buffer, even on multi-threaded app
// (at least under Windows, it may be up to 10 times faster)
// - but you can decide whatever to use none, one or both SockIn/SockOut
// - our classes are much faster than the Indy or Synapse implementation
TCrtSocket = class
public
/// initialized after Open() with socket
Sock: TSocket;
...
...
In ICS (commet from ICS code included below), it is claimed that it is of very high performance due to its asynchronous non-blocking behaviour.
How your socket's performance compare with ICS, mainly when using multiple connections (sockets) to achieve better speed? (like a download manager using 10 connections to download a file at top speed; instead of using a single connection like browsers do)
ICS's multi connection and your's multi thread approach (say 10 threads to DL a file) both achieve more or less same speed?
Comments from ICS code:
{
About multithreading and event-driven:
TWSocket is a pure asynchronous component. It is non-blocking and
event-driven. It means that when you request an operation such as connect,
the component start the operation your requested and give control back
immediately while performing the operation in the background automatically.
When the operation is done, an event is triggered (such as
OnSessionConnected if you called Connect).
This asynchronous non-blocking behaviour is very high performance but a
little bit difficult to start with. For example, you can't call Connect and
immediately call SendStr the line below. If you try, you'll have an
exception triggered saying you are not connected. Calling connect will start
connection process but will return long before connection is established.
Calling SendStr at the next line will not work because the socket is not
connected yet. To make it works the right way, you have to put your SendStr
in the OnSessionConnected event.
The asynchronous operation allows you to do several TCP/IP I/O
simultaneously. Just use as many component as you need. Each one will
operate independently of the other without blocking each other ! So you
basically don't need multi-threading with TWSocket, unless YOUR processing
is lengthy and blocking.
If you have to use multithreading, you have two possibilities:
1) Create your TWSocket from your thread's Execute method
2) Attach a TWSocket to a given thread using ThreadAttach.
In both cases, you must set MultiThreaded property to TRUE.
If you don't use one of those methods, you'll end up with a false
multithreaded program: all events will be processed by the main tread !
For both methods to work, you MUST have a message loop withing your thread.
Delphi create a message loop automatically for the main thread (it's in
the Forms unit), but does NOT create one in a thread ! For your convenience,
TWSocket has his own MessageLoop procedure. You can use it from your thread.
Sample program MtSrv uses first method while ThrdSrv uses second method.
Sample program TcpSrv is much the same as ThrdSrv but doesn't use any
thread. You'll see that it is able to server a lot of simultaneous clients
as well and it is much simpler.
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
...
Oh no! I shouldn't have underestimated your amount of research
If subtraction is used alternatively (or in a some more advanced manner),
and that sum too used in the result, then your Hash32() would have
even less collisions? May be also a cumulative sum of s3 would be of use?
s3 := 0;
...
inc(s1,P^[0]);
inc(s3, P^[0]);
inc(s2,s1);
inc(s1,P^[1]);
dec(s3,P^[1]);
inc(s2,s1);
inc(s1,P^[2]);
inc(s3,P^[2]);
inc(s2,s1);
inc(s1,P^[3]);
dec(s3,P^[3]);
inc(s2,s1);
inc(PtrUInt(P),16);
In http://burtleburtle.net/bob/c/checksum.c , the author (Bob Jenkins)
returns 8 integers as checksum in hash(), so very less chance for collisions?.
Likewise, in your Hash32 (may be you have a better name like BouchezHash()),
if you return multiple integers (say s2,s3) , may be it becomes even less in collisions.
(I don't know why you would mix s2 and s1, losing the fine value of s2 in the result)
May I know the significance of "result := s1 xor (s2 shl 16);"
I think s2 is the one that avoids the collions more (as far as s1, s1 would be same
even if all the bytes were swapped in any combination). So, why not use s2 alone?
JFYI:
Bob Jenkins home page: http://burtleburtle.net/bob/
He also has a hash called lookup3 (http://burtleburtle.net/bob/c/lookup3.c),
64bit: http://burtleburtle.net/bob/c/lookup8.c
which he claims faster than crc32 (http://burtleburtle.net/bob/hash/examhash.html)
and claims 0 collisions.
His ISAAC has been ported in many languages including Pascal/Delphi
(http://home.netsurf.de/wolfgang.ehrhardt/misc_en.html) by
Wolfgang Ehrhardt another Delphi/Speed expert like you
(http://home.netsurf.de/wolfgang.ehrhardt/)
I never imagined there would be this much stuff to work with to achieve speed.
About the Hash32() code, s1 and s2 would never have the same value, because there is a inc(s2,s1) every two instructions. Use the step by step debugger, and you'll see how it works.
My mad. It became clear that s2 is cumulative sum when I manually gone through the loop just for 2 iterations.
If you try to achieve best speed, take a look at some recent reference documentation.
You've some very good reference material available at http://www.agner.org/optimize
I don't know assembly and have no plans to learn
BTW, While you are there..., why I have to worry?
I tried reading data by DWORDs with DJB, but still Hash32 is faster than DJB (approx 2.5 times). The reason: it seems add is faster than bit shift and Hash32 does only add (inc())
// only to test performance; few bytes at end not used in hash
function DjbHash(data: pointer; len: integer): cardinal;
type
PCardinal = ^cardinal;
function SubHash(P: PCardinal; L: integer): cardinal;
var
i: cardinal;
begin
result := 5381;
if p = nil then
Exit;
// few bytes at end discarded, as it wouldn't affect performance test
for i := 1 to L div sizeof(cardinal) do begin
result := ((result shl 5)+result) + p^;
inc(p);
end;
end;
begin
result := SubHash(data, len);
end;
My understanding correct?
PS: I observed performance with difference in QueryPerformanceCounter value before and after calling the hash function. I think it is fair.
PS2: How s1 and s2 differ?
s1 := 0;
s2 := 0;
for i := 1 to L shr 4 do begin // 16 bytes (4 DWORD) by loop - aligned read
inc(s1,P^[0]);
inc(s2,s1);
inc(s1,P^[1]);
inc(s2,s1);
inc(s1,P^[2]);
inc(s2,s1);
inc(s1,P^[3]);
inc(s2,s1);
inc(PtrUInt(P),16);
end;
In your Hash32() code, after the end of first(main) loop s1 and s2 differ (it should by purpose I think). But by looking at the code plainly it seems both s1 and s2 would have the same value. Where do I overlook?
Thanks.
I did 2 votes; I think you are #1 in the list. Really???
BTW, excellent work regarding your Delphi stuff.
Are you any super human?
Pages: 1