Stack Commit Size, Stack Reserve Size

Kernel, Loader, code execution and working with memory

Re: Stack Commit Size, Stack Reserve Size

Postby Ivan Denisov » Wed Oct 31, 2018 5:00 pm

There is "natural" limit for the 32-bit application. The 64-bit BlackBox is not ready yet. And I think that it will not be ready soon without industry financial support.
You can think about optimization for big models in the way of distributed computations. BlackBox instances can exchange data with TCP.
User avatar
Ivan Denisov
 
Posts: 231
Joined: Tue Sep 17, 2013 12:21 am
Location: Krasnoyarsk, Russia

Re: Stack Commit Size, Stack Reserve Size

Postby Josef Templ » Wed Oct 31, 2018 5:22 pm

Is it possible that you are using recursion for processing a linear list?
There is no tail-recursion optimization in BlackBox. If you have a list of length 1000 and if you process it recursively you get 1000 active stack frames. Recursion here means to process the first element and then apply your algorithm recursively to the remaining part of the list. This would be a catastrophy.

- Josef
Josef Templ
 
Posts: 211
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Postby Mobatec » Wed Oct 31, 2018 8:31 pm

There is no run-time problem. The stack overflow problem only occurs when trying to load (internalize) a big model. Building the big model, running it and also saving (externalizing) it are no problem. Maybe it's indeed the 32-bit limit. There are quite some cyclic data structures in the defined objects, but I let the Stores implementation take care of that...
The quest for a solution continues in the meantime :)

= Mathieu.
User avatar
Mobatec
 
Posts: 14
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Postby Josef Templ » Thu Nov 01, 2018 6:56 am

I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.

When I start BB with your linker settings without any further changes I can open a file dialog after startup of BB without any problem.
As the next step I run the following test program and get a stack overflow trap as expected because there is an endless recursion.

Code: Select all
MODULE TestStack;

   PROCEDURE Recurse*;
   BEGIN Recurse
   END Recurse;

END TestStack.

Execute the command: TestStack.Recurse

When I then open a file dialog (e.g. with File->Open...) I get the anomalies.
Small black boxes for icons or sometimes even a trap with WinDlg.CommDlgExtendedError() = 2 indicating an out-of-memory error
from calling one of the Windows common dialog controls functions.
My interpretation is this: executing TestStack.Recurse allocates stack memory from the reserved stack address range of BB until the stack overflows.
The allocated stack memory is not deallocated after the stack overflow for efficiency reasons, i.e. the physical stack size grows dynamically.
Only the stack pointer is reset in the trap handler such that BlackBox can continue normally.
However, it is surprising that there is an observable side effect here because the reserved address ranges
are not changed at all by calling TestStack.Recurse. Only the Windows source code could explain that side effect, I guess.


Last but not least I changed the heap size in Kernel.AllocHeapMem as I described in a previous posting
from 1536 to something smaller, e.g. 1024:

Code: Select all
   PROCEDURE AllocHeapMem (size: INTEGER; VAR c: Cluster);
      (* allocate at least size bytes, typically at least 256 kbytes are allocated *)
      CONST M = 1024(*1536*) * 100000H;   (* 1.0 (1.5) GByte *)


and linked a new exe file and started that new exe file and the anomalies went away
even after calling TestStack.Recurse.
For linking a new exe file you can use the linker command in Dev/Docu/Build-Tool.odc.
I repeat it here:

Code: Select all
DevLinker.Link BlackBox2.exe := Kernel$+ Files HostFiles StdLoader
1 BlackBox.res 1 Applogo.ico 2 Doclogo.ico 3 SFLogo.ico 4 CFLogo.ico 5 DtyLogo.ico 6 folderimg.ico 7 openimg.ico
8 leafimg.ico 1 Move.cur 2 Copy.cur 3 Link.cur 4 Pick.cur 5 Stop.cur 6 Hand.cur 7 Table.cur


You have to start BlackBox2.exe in order to make the changes effective.

To me it seems that there has been an error in your experiment when changing the constant M.

- Josef
Josef Templ
 
Posts: 211
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Postby luowy » Thu Nov 01, 2018 4:33 pm

Josef Templ wrote:I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.

surperised! your sloution worked well, the 1.5G is threshold:
CONST M = (1536-64)* 100000H; (* 1.4375 GByte *)
worked also; the reason is not known yet;

luowy
luowy
 
Posts: 27
Joined: Thu Dec 17, 2015 1:32 pm

Re: Stack Commit Size, Stack Reserve Size

Postby Mobatec » Thu Nov 01, 2018 10:33 pm

Wow, Josef (and luowy), Thanks very much, this really seems to be the solution to the strange issue I have been encountering! Great.
User avatar
Mobatec
 
Posts: 14
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Postby Josef Templ » Mon Nov 05, 2018 5:14 pm

Maybe it would be a good idea to add at least some comments in the related source files.
Any objections from other center embers?

- Josef
Josef Templ
 
Posts: 211
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Postby Mobatec » Tue Nov 06, 2018 8:10 pm

Good point.
The more comments, the better, I would say. I tend to put a lot of comments in my source code, since typically a few years down the line, you have a hard time understanding your own "smart" code :geek:
User avatar
Mobatec
 
Posts: 14
Joined: Thu Oct 11, 2018 3:20 pm

Re: Stack Commit Size, Stack Reserve Size

Postby Josef Templ » Tue Nov 06, 2018 9:20 pm

I have created an issue for that.

- Josef
Josef Templ
 
Posts: 211
Joined: Tue Sep 17, 2013 6:50 am

Re: Stack Commit Size, Stack Reserve Size

Postby Robert » Fri Nov 09, 2018 10:51 am

Ivan Denisov wrote:There is "natural" limit for the 32-bit application. ... BlackBox instances can exchange data with TCP.

Ivan, do you have any tutorial links or VERY simple examples of BlackBox instances exchanging data?
User avatar
Robert
 
Posts: 109
Joined: Sat Sep 28, 2013 11:04 am
Location: Edinburgh, Scotland

PreviousNext

Return to System

Who is online

Users browsing this forum: No registered users and 1 guest

cron