Mqtt server based on mormot's network,give some suggestion pls

songshuang · 2021-02-06 12:14:39

rewrite the unit mormot.net.http or TRestHttpServer?
i am learning the demo :websocket. i try to study the framework of websockt.
please give some suggestions!

ab · 2021-02-06 15:45:48

You could reuse the WebSockets client/server to implement the MQTT protocol in top of it.
Just encode and decode the WS binary frames, as expected by the protocol.

There are some existing MQTT implementation around, e.g. https://github.com/ZiCog/TMQTTClient/bl … t/MQTT.pas

mpv · 2021-02-06 15:55:04

MQTT is implemented in libcurl - see https://curl.se/docs/mqtt.html. So if limitations listed there is not a problem for you, we already have an MQTT in mORMot by using a low-level SynCurl unit.

Last edited by mpv (2021-02-06 15:55:44)

songshuang · 2021-02-07 05:57:10

What I care is the maximum number of connections,
because mqtt is long term connection,I noticed that acceped is served by TSynThreadPoolSubThread. a keeplived connected client is served by THttpServerResp. THttpServerResp is a thread , means a clent is a thread.
I noticed that THttpServerResp is asynchronous wating for received:pending := fServerSock.SockReceivePending(50); ....
I am thinking:
how many threads in a server?
many many client served by a thread pool? how to readlin?
The frame of the network serverd many client is my noticed!
mqtt's protocol is simple to me.
mormot's interface is complex to me
based frame units:https://github.com/songshuang8/MqttServer,now is nothing but MqttServer.pas
i want to:sqlite is serverd by saving persistent mqtts message， rest is serving manager api,log is loging...

Last edited by songshuang (2021-02-07 06:47:10)

AOG · 2021-02-07 06:43:05

Hi songshuang,
Thanks for sharing your code. Looks very interesting and welcome. A mORMot based MQTT server (and client) is very much appreciated (by me) !

songshuang · 2021-02-07 07:05:03

client is not inerested to me,it's be realized by wifi's chip normal.:)

ab · 2021-02-07 13:19:08

Currently, the default WebSockets server uses one thread per client.
The HTTP server has a thread pool. But not for WebSockets (yet: I will eventually add a thread-pool for the WS server too, but perhaps on mORMot 2).

But if you want to run on Windows, there is a WebSockets server based on http.sys - thanks to @mpv - which uses a thread pool.
For implementing a low-level protocol like MQTT, it could be very good, with no scaling issue.

mpv · 2021-02-07 15:17:32

Ups, I miss what songshuang need to implement a server (broker). When I wrote about curl I mean MQTT client.

IMHO create a valid broker implementation is very-very hard to do. Actual MQTT specification is on 137 pages, so protocol it not as simple as it might seem.
From other side there is a dozen of open source well tested MQTT brokers. We use a mosquitto in most case, or RabbitMQ with MQTT plugin if we need a complex routing.

songshuang · 2021-02-24 13:44:26

Please give me some suggestions!
pls see https://github.com/songshuang8/MqttServer, login,sub,unsub ,pub , ping, is ok. no retain!
based on rtsphttp.pas

ab · 2021-02-24 15:26:46

What is your exact problem?

songshuang · 2021-03-06 05:24:31

I'm building on mormot.net.async.pas's network architecture.
The main changed in mormot.net.async.pas is:The function of checking idle put in a single thread. because you put it to after ProcessWrite(30000) line,when a client is onlie ,Checking idle function will happen per 30 sec.
When a client accepted,will create a connction and fClients.Start(connection) at once.
I founded that error will be occurred when active close a connection
My code is:

procedure TAsyncConnections.IdleEverySecond;
...
  fConnectionLock.Lock;
  try
    for i:=0 to fConnectionCount-1 do
    begin
      aconn := fConnection[i];
      allowed := UnixTimeUtc - aconn.fCanIdleLen;
      if aconn.fLastOperation < allowed then
      begin
        IdleArr.Add(aconn.handle);
...
  for i:=0 to High(IntegerArr) do
    ConnectionRemove(IntegerArr[i]);
end;

Now the following error can occurred(under about 250 wifi's-thermostats connections,ubuntu)

EXCOS EAccessViolation (04) [MQTTSvrTAConnectionspseRead] at 6d8036 ../../SysDataManager/mORMot2/src/net/mormot.net.sock.posix.inc tpollsocketepoll.waitformodified (655) ../../SysDataManager/mORMot2/src/net/mormot.net.sock.pas pollandsearchwithinpending (1513) ../../SysDataManager/mORMot2/src/net/mormot.net.sock.pas tpollsockets.getone (1556) src/MqttServer/mormot.net.async_rw.pas tpollasyncsockets.processread (652) src/MqttServer/mormot.net.async_rw.pas tasyncconnectionsthread.execute (904) 
EXC   EInvalidCast {Message:"Invalid type cast"} [MQTTSvrTAConnectionspseRead] at 617b04 src/MqttServer/mormot.net.async_rw.pas tasyncconnectionssockets.onclose (816) src/MqttServer/mormot.net.async_rw.pas closeconnection (639) src/MqttServer/mormot.net.async_rw.pas tpollasyncsockets.processread (684) src/MqttServer/mormot.net.async_rw.pas tasyncconnectionsthread.execute (904)

No error founded on windows ,when one or two client connected.
I try to do in version 1.18 or 2,the above error will happen aslo.
How to active close a coneection?Please help me or some suggestions.

mORMot Open Source

#1 2021-02-06 12:14:39

Mqtt server based on mormot's network,give some suggestion pls

#2 2021-02-06 15:45:48

Re: Mqtt server based on mormot's network,give some suggestion pls

#3 2021-02-06 15:55:04

Re: Mqtt server based on mormot's network,give some suggestion pls

#4 2021-02-07 05:57:10

Re: Mqtt server based on mormot's network,give some suggestion pls

#5 2021-02-07 06:43:05

Re: Mqtt server based on mormot's network,give some suggestion pls

#6 2021-02-07 07:05:03

Re: Mqtt server based on mormot's network,give some suggestion pls

#7 2021-02-07 13:19:08

Re: Mqtt server based on mormot's network,give some suggestion pls

#8 2021-02-07 15:17:32

Re: Mqtt server based on mormot's network,give some suggestion pls

#9 2021-02-24 13:44:26

Re: Mqtt server based on mormot's network,give some suggestion pls

#10 2021-02-24 15:26:46

Re: Mqtt server based on mormot's network,give some suggestion pls

#11 2021-03-06 05:24:31

Re: Mqtt server based on mormot's network,give some suggestion pls

Board footer