Stack Commit Size, Stack Reserve Size

Kernel, Loader, code execution and working with memory
User avatar
Ivan Denisov
Posts: 362
Joined: Tue Sep 17, 2013 12:21 am
Location: Krasnoyarsk, Russia

Re: Stack Commit Size, Stack Reserve Size

Post by Ivan Denisov »

There is "natural" limit for the 32-bit application. The 64-bit BlackBox is not ready yet. And I think that it will not be ready soon without industry financial support.
You can think about optimization for big models in the way of distributed computations. BlackBox instances can exchange data with TCP.
Josef Templ
Posts: 262
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Post by Josef Templ »

Is it possible that you are using recursion for processing a linear list?
There is no tail-recursion optimization in BlackBox. If you have a list of length 1000 and if you process it recursively you get 1000 active stack frames. Recursion here means to process the first element and then apply your algorithm recursively to the remaining part of the list. This would be a catastrophy.

- Josef
User avatar
Mobatec
Posts: 18
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Post by Mobatec »

There is no run-time problem. The stack overflow problem only occurs when trying to load (internalize) a big model. Building the big model, running it and also saving (externalizing) it are no problem. Maybe it's indeed the 32-bit limit. There are quite some cyclic data structures in the defined objects, but I let the Stores implementation take care of that...
The quest for a solution continues in the meantime :)

= Mathieu.
Josef Templ
Posts: 262
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Post by Josef Templ »

I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.

When I start BB with your linker settings without any further changes I can open a file dialog after startup of BB without any problem.
As the next step I run the following test program and get a stack overflow trap as expected because there is an endless recursion.

Code: Select all

MODULE TestStack;

	PROCEDURE Recurse*;
	BEGIN Recurse
	END Recurse;

END TestStack.
Execute the command: TestStack.Recurse

When I then open a file dialog (e.g. with File->Open...) I get the anomalies.
Small black boxes for icons or sometimes even a trap with WinDlg.CommDlgExtendedError() = 2 indicating an out-of-memory error
from calling one of the Windows common dialog controls functions.
My interpretation is this: executing TestStack.Recurse allocates stack memory from the reserved stack address range of BB until the stack overflows.
The allocated stack memory is not deallocated after the stack overflow for efficiency reasons, i.e. the physical stack size grows dynamically.
Only the stack pointer is reset in the trap handler such that BlackBox can continue normally.
However, it is surprising that there is an observable side effect here because the reserved address ranges
are not changed at all by calling TestStack.Recurse. Only the Windows source code could explain that side effect, I guess.


Last but not least I changed the heap size in Kernel.AllocHeapMem as I described in a previous posting
from 1536 to something smaller, e.g. 1024:

Code: Select all

	PROCEDURE AllocHeapMem (size: INTEGER; VAR c: Cluster);
		(* allocate at least size bytes, typically at least 256 kbytes are allocated *)
		CONST M = 1024(*1536*) * 100000H;	(* 1.0 (1.5) GByte *)
and linked a new exe file and started that new exe file and the anomalies went away
even after calling TestStack.Recurse.
For linking a new exe file you can use the linker command in Dev/Docu/Build-Tool.odc.
I repeat it here:

Code: Select all

DevLinker.Link BlackBox2.exe := Kernel$+ Files HostFiles StdLoader
1 BlackBox.res 1 Applogo.ico 2 Doclogo.ico 3 SFLogo.ico 4 CFLogo.ico 5 DtyLogo.ico 6 folderimg.ico 7 openimg.ico
8 leafimg.ico 1 Move.cur 2 Copy.cur 3 Link.cur 4 Pick.cur 5 Stop.cur 6 Hand.cur 7 Table.cur
You have to start BlackBox2.exe in order to make the changes effective.

To me it seems that there has been an error in your experiment when changing the constant M.

- Josef
luowy
Posts: 87
Joined: Thu Dec 17, 2015 1:32 pm

Re: Stack Commit Size, Stack Reserve Size

Post by luowy »

Josef Templ wrote:I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.
surperised! your sloution worked well, the 1.5G is threshold:
CONST M = (1536-64)* 100000H; (* 1.4375 GByte *)
worked also; the reason is not known yet;

luowy
User avatar
Mobatec
Posts: 18
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Post by Mobatec »

Wow, Josef (and luowy), Thanks very much, this really seems to be the solution to the strange issue I have been encountering! Great.
Josef Templ
Posts: 262
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Post by Josef Templ »

Maybe it would be a good idea to add at least some comments in the related source files.
Any objections from other center embers?

- Josef
User avatar
Mobatec
Posts: 18
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Post by Mobatec »

Good point.
The more comments, the better, I would say. I tend to put a lot of comments in my source code, since typically a few years down the line, you have a hard time understanding your own "smart" code :geek:
Josef Templ
Posts: 262
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Post by Josef Templ »

I have created an issue for that.

- Josef
User avatar
Robert
Posts: 177
Joined: Sat Sep 28, 2013 11:04 am
Location: Edinburgh, Scotland

Re: Stack Commit Size, Stack Reserve Size

Post by Robert »

Ivan Denisov wrote:There is "natural" limit for the 32-bit application. ... BlackBox instances can exchange data with TCP.
Ivan, do you have any tutorial links or VERY simple examples of BlackBox instances exchanging data?
Post Reply