#1 Re: mORMot 1 » Fast MM5 » 2022-10-29 19:57:05

Hi ab,

First let me thank you for your help and patience.  Second, let me apologize for the confusion that I’ve created on a number of fronts.  It’s because this project is really an old tool that I created years ago and update from time to time with “improvements.”  Hence, it isn’t really my focus.  I hope to correct the confusion with the following. 

1) Yes, it is really mormot.core.fpcx64mm that I’m using.  The confusion comes from the title of the topic, but most of the conversation seems to be related to fpcx64mm.

2) I had assumed that the problem that I was experiencing was associated with fpcx64mm and that it was related to the with statement as I previously mentioned.  The latter assumption was incorrect.  In all cases that I checked, the with construct works as expected.  However, the logical failure was so close to the code that I previously mentioned, that I thought it obvious that this was where the problem lies.

I still do have a problem with fpcx64mm however.  It has to do with with the following function:


I used this to dynamically determine the size of the file buffers I was using.  With fpcx64mm, it always seems to return 0.  With the fpc internal memory manager it returns a sensible number.  What was happening is that my code, running with 0 size file buffers would simply return identical CRC’s for each file.  The rest of the code worked as expected under the circumstances.

The work-around is to set my file buffer size with a fixed number, for now.  However, it would be nice to be able to use the above mentioned call to get an understanding of the available memory on the heap.  I assume that this is something not so difficult to do.

Many thanks for your important contributions to the Pascal world.

#2 Re: mORMot 1 » Fast MM5 » 2022-10-26 22:03:05

Thanks ab for getting back to me.  I think the problem is with FASTMM5.  I first discovered FASTMM5 here:

https://blog.synopse.info/?post/2020/05 … 4-assembly

while trying to track down memory management errors for my image processing application.  At that time I got a more recent copy from a link from this forum.  Most recently I picked up an updated version from here:

https://github.com/synopse/mORMot2/tree/master/src/core  (mormot.core.fpcx64mm)

I assume that they are all equal except for revisions to update or bug-fix.  As I said, I had good luck trying to fix the problems I was having with the FPC internal memory manager with this code for my image processing application.  It was much more stable and didn’t crash after rather heavy use.

I tried it on my an update to software that I’ve written for finding duplicate files whereupon I discovered this problem.  I’m using Lazarus v1.82 as my development environment and was interested in trying this memory manager to help speed things up when dealing with millions of files.

I haven’t done extensive testing to try to determine the exact location in the code where it fails, nor how it fails.  However, the logic for determining whether there are duplicate files is in the following code snippet:

      // mark the currently matching files
      no_match := true;
      for count_1 := 0 to number_of_files - 1 do
        with compare_files[count_1] do
            for count_2 := count_1 + 1 to number_of_files do
                if files[file_index - count_1].duplicated and
                   files[file_index - count_2].duplicated and
                   (CRC = compare_files[count_2].CRC) then
                    matched := true;
                    compare_files[count_2].matched := true;
                    no_match := false;
            // if not currently matched, short circuit future evaluations of this file
            files[file_index - count_1].duplicated := matched;

It is here that the logic fails with the FASTMM5 memory manager but works fine with the FPC memory manager.  matched and CRC are elements of the type comprising the array compare_files.

I’m happy to assist you in determining what the failure mode is.  However, I wanted to bring it to your attention ASAP so that others might avoid this problem.

#3 Re: mORMot 1 » Fast MM5 » 2022-10-25 21:59:23

I think I've discovered a bug.   It's probably associated with an interface to FastMM5.  When I use an array of a typed variable of multiple elements, and then reference an element of a member of that array using,

  with array[index] do

things don't work.  They work perfectly fine using FPC's built-in memory manager. 

I've used Fast MM5 with other projects (mostly image processing) and I love it, even though it is a bit more picky than FPC's built-in memory manager.  But, this seems to be a significant bug.

If you'd like a copy of the source code that is causing me a problem, then I can post it.  However, I think the description is adequate to start investigating.

Thanks for your great contributions.

Board footer

Powered by FluxBB