Page 1 of 4

Stack Commit Size, Stack Reserve Size

PostPosted: Thu Oct 11, 2018 6:24 pm
by Mobatec
Hello all,
I've been using BlackBox Component Builder for over 20 years to build a powerful modelling tool, but somehow never used this forum (yet).
Several years back we ran into a problem that very big models could not be loaded anymore into the tool we develloped with BlackBox. Back then I was advised by Marc Frei of Oberon to alter some parameters in the WriteHeader Procedure of the DevLinker module:

Write4(900000H); (* stack reserve size *) (* MRW: 900000H, ORIGINAL VALUE: 200000H *)
Write4(060000H); (* stack commit size *) (* MRW: 060000H, ORIGINAL VALUE: 10000H *)
IF isDll THEN
Write4(00100000H); (* heap reserve size *)
ELSE
Write4(04000000H); (* heap reserve size *) (* MRW: 04000000H, ORIGINAL VALUE: 00400000H *)
END;
Write4(080000H); (* heap commit size *) (* MRW: 80000H, ORIGINAL VALUE: 10000H *)
Write4(0);

This parameters I tried seemed to work fine, because the big models could now be loaded again. Since a few weeks, with Windows10, however, there seems to be a big problem when opening an Explorer Window to open or save a file (HostDialog.GetIntSpec or HostDialog.GetExtSpec). At first the Explorer Dialog loads, but not properly (black squares are shown, in stead of icons) and after a few tries I only get Trap 100 (meaning that the WinDlg.GetOpenFileNameW failed).

Does anybody know which parameters would work? Reverting back to the original values is not an option, since then the bigger models will result in a Stack Overflow again.

I'm working with BlackBox 1.7 (which I altered in several System and Host modules to better suit my needs).

Kind regards,
Mathieu.

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Fri Oct 12, 2018 6:50 am
by Josef Templ
Are you running under a 32-bit Win10 or under a 64 bit Win10?

I am asking this because it may be related with Windows/Explorer running out of memory.
If too much memory is committed to BlackBox then there is not enough memory left for Windows tools.
Under 64 bit Windows there should not be such a hard limitation.

- Josef

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Fri Oct 12, 2018 10:12 am
by Josef Templ
I found this posting on the internet. It may be related:

This sort of issue can be caused by a bad shell extension. Use Process Explorer to check what DLLs are loaded into your process when you use the File Open dialog. If you find anything suspicious, try uninstalling the associated software. (Printers are for some reason often a culprit here).
The reason is that the file open dialog loads Windows Explorer shell extensions, and unfortunately many of the third-party ones are buggy.

- Josef

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Fri Oct 12, 2018 10:43 am
by Mobatec
I'm running on 64bit Win10

- Mathieu.

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Fri Oct 12, 2018 10:55 am
by Ivan Denisov
I am working with SDL2 library and OpenGL graphics and observe the same problems with Window 10 64 bit using SDL2 library. I guess, that there are some troubles with hybrid graphics driver interaction with last windows updates. Now I updated both Intel and NVIDIA drivers and will research that problem further.
Does you laptop have hybrid graphics?

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Fri Oct 12, 2018 12:27 pm
by Josef Templ
I made some experiments with the Coroutines module of BB 1.7.1.
This allows me to set the stacksize of a newly created coroutine to any value
and then within the coroutine I try to open a file dialog. What I observed is this:

With small sizes (ca 1MB) everything is OK.
With huge sizes (ca 300MB) the coroutine cannot be created because there is not enough memory for the stack.
In between there is a point where I can create a coroutine but the file dialog then looks broken.
Some parts are not displayed as expected.
This indicates that the file dialog runs out of memory. Depending on when it runs out
of memory it produces a particular kind of misbehavior but it tries to display something
instead of reporting an out-of-memory error.
I don't know if there are any error log entries made in a Windows log file, at least I couldn't find one.

- Josef

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Sat Oct 13, 2018 9:24 am
by Mobatec
Thanks for your investigations, Josef (and Ivan),
That's indeed the behaviour I see, The question is, can BlackBox have higher stack sizes and also assign more memory to the Windows tools? Maybe it's some parameter in the DevLinker that I did not adapt. The WriteHeader of DevLinker is quite abacadabra for me :), so I have no clue how this works..
- Mathieu.

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Mon Oct 15, 2018 8:03 pm
by Mobatec
Hi Josef,
Do you (or anybody else) have some more insight in DevLinker? Can BlackBox have higher stack sizes and also assign more memory to the Windows tools? Maybe it's some parameter in the DevLinker that I did not adapt.
- Mathieu.

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Mon Oct 15, 2018 8:39 pm
by Josef Templ
I found the following limitations for Win32 applications in https://software.intel.com/en-us/articles/memory-limits-applications-windows:

Static data - 2GB
Dynamic data - 2GB
Stack data - 1GB (the stack size is set by the linker, the default is 1MB. This can be increased using the Linker property System > Stack Reserve Size)
Note that on 32-bit Windows, the sum of all types of data must be 2GB or less. The practical limit is about 1.75GB due to space used by Windows itself


It would also be interesting to see what happens if the heap is enlarged (by a big allocations with NEW, for example) instead of the stack.
Is it the overall limit of 2GB that is reached?

Ooops, this pops up an idea. There is a 1.5 GB heap space virtual memory allocation in Kernel.AllocHeapMem (see CONST M).
May be it helps to reduce this to something like 1.3 GB or even less. My own tests seem to indicate that it helps.

- Josef

Re: Stack Commit Size, Stack Reserve Size

PostPosted: Mon Oct 15, 2018 9:01 pm
by Mobatec
The Kernel.AllocHeapMem doesn't seem to make any difference (at least not with my current DevLinker values