Stack Commit Size, Stack Reserve Size
- Ivan Denisov
- Posts: 362
- Joined: Tue Sep 17, 2013 12:21 am
- Location: Krasnoyarsk, Russia
Re: Stack Commit Size, Stack Reserve Size
There is "natural" limit for the 32-bit application. The 64-bit BlackBox is not ready yet. And I think that it will not be ready soon without industry financial support.
You can think about optimization for big models in the way of distributed computations. BlackBox instances can exchange data with TCP.
You can think about optimization for big models in the way of distributed computations. BlackBox instances can exchange data with TCP.
-
- Posts: 262
- Joined: Tue Sep 17, 2013 6:50 am
Re: Stack Commit Size, Stack Reserve Size
Is it possible that you are using recursion for processing a linear list?
There is no tail-recursion optimization in BlackBox. If you have a list of length 1000 and if you process it recursively you get 1000 active stack frames. Recursion here means to process the first element and then apply your algorithm recursively to the remaining part of the list. This would be a catastrophy.
- Josef
There is no tail-recursion optimization in BlackBox. If you have a list of length 1000 and if you process it recursively you get 1000 active stack frames. Recursion here means to process the first element and then apply your algorithm recursively to the remaining part of the list. This would be a catastrophy.
- Josef
Re: Stack Commit Size, Stack Reserve Size
There is no run-time problem. The stack overflow problem only occurs when trying to load (internalize) a big model. Building the big model, running it and also saving (externalizing) it are no problem. Maybe it's indeed the 32-bit limit. There are quite some cyclic data structures in the defined objects, but I let the Stores implementation take care of that...
The quest for a solution continues in the meantime
= Mathieu.
The quest for a solution continues in the meantime
= Mathieu.
-
- Posts: 262
- Joined: Tue Sep 17, 2013 6:50 am
Re: Stack Commit Size, Stack Reserve Size
I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.
When I start BB with your linker settings without any further changes I can open a file dialog after startup of BB without any problem.
As the next step I run the following test program and get a stack overflow trap as expected because there is an endless recursion.
Execute the command: TestStack.Recurse
When I then open a file dialog (e.g. with File->Open...) I get the anomalies.
Small black boxes for icons or sometimes even a trap with WinDlg.CommDlgExtendedError() = 2 indicating an out-of-memory error
from calling one of the Windows common dialog controls functions.
My interpretation is this: executing TestStack.Recurse allocates stack memory from the reserved stack address range of BB until the stack overflows.
The allocated stack memory is not deallocated after the stack overflow for efficiency reasons, i.e. the physical stack size grows dynamically.
Only the stack pointer is reset in the trap handler such that BlackBox can continue normally.
However, it is surprising that there is an observable side effect here because the reserved address ranges
are not changed at all by calling TestStack.Recurse. Only the Windows source code could explain that side effect, I guess.
Last but not least I changed the heap size in Kernel.AllocHeapMem as I described in a previous posting
from 1536 to something smaller, e.g. 1024:
and linked a new exe file and started that new exe file and the anomalies went away
even after calling TestStack.Recurse.
For linking a new exe file you can use the linker command in Dev/Docu/Build-Tool.odc.
I repeat it here:
You have to start BlackBox2.exe in order to make the changes effective.
To me it seems that there has been an error in your experiment when changing the constant M.
- Josef
and I can reproduce your observed anomaly pretty well.
When I start BB with your linker settings without any further changes I can open a file dialog after startup of BB without any problem.
As the next step I run the following test program and get a stack overflow trap as expected because there is an endless recursion.
Code: Select all
MODULE TestStack;
PROCEDURE Recurse*;
BEGIN Recurse
END Recurse;
END TestStack.
When I then open a file dialog (e.g. with File->Open...) I get the anomalies.
Small black boxes for icons or sometimes even a trap with WinDlg.CommDlgExtendedError() = 2 indicating an out-of-memory error
from calling one of the Windows common dialog controls functions.
My interpretation is this: executing TestStack.Recurse allocates stack memory from the reserved stack address range of BB until the stack overflows.
The allocated stack memory is not deallocated after the stack overflow for efficiency reasons, i.e. the physical stack size grows dynamically.
Only the stack pointer is reset in the trap handler such that BlackBox can continue normally.
However, it is surprising that there is an observable side effect here because the reserved address ranges
are not changed at all by calling TestStack.Recurse. Only the Windows source code could explain that side effect, I guess.
Last but not least I changed the heap size in Kernel.AllocHeapMem as I described in a previous posting
from 1536 to something smaller, e.g. 1024:
Code: Select all
PROCEDURE AllocHeapMem (size: INTEGER; VAR c: Cluster);
(* allocate at least size bytes, typically at least 256 kbytes are allocated *)
CONST M = 1024(*1536*) * 100000H; (* 1.0 (1.5) GByte *)
even after calling TestStack.Recurse.
For linking a new exe file you can use the linker command in Dev/Docu/Build-Tool.odc.
I repeat it here:
Code: Select all
DevLinker.Link BlackBox2.exe := Kernel$+ Files HostFiles StdLoader
1 BlackBox.res 1 Applogo.ico 2 Doclogo.ico 3 SFLogo.ico 4 CFLogo.ico 5 DtyLogo.ico 6 folderimg.ico 7 openimg.ico
8 leafimg.ico 1 Move.cur 2 Copy.cur 3 Link.cur 4 Pick.cur 5 Stop.cur 6 Hand.cur 7 Table.cur
To me it seems that there has been an error in your experiment when changing the constant M.
- Josef
Re: Stack Commit Size, Stack Reserve Size
surperised! your sloution worked well, the 1.5G is threshold:Josef Templ wrote:I have done some more tests now with your settings in DevLinker
and I can reproduce your observed anomaly pretty well.
CONST M = (1536-64)* 100000H; (* 1.4375 GByte *)
worked also; the reason is not known yet;
luowy
Re: Stack Commit Size, Stack Reserve Size
Wow, Josef (and luowy), Thanks very much, this really seems to be the solution to the strange issue I have been encountering! Great.
-
- Posts: 262
- Joined: Tue Sep 17, 2013 6:50 am
Re: Stack Commit Size, Stack Reserve Size
Maybe it would be a good idea to add at least some comments in the related source files.
Any objections from other center embers?
- Josef
Any objections from other center embers?
- Josef
Re: Stack Commit Size, Stack Reserve Size
Good point.
The more comments, the better, I would say. I tend to put a lot of comments in my source code, since typically a few years down the line, you have a hard time understanding your own "smart" code
The more comments, the better, I would say. I tend to put a lot of comments in my source code, since typically a few years down the line, you have a hard time understanding your own "smart" code
-
- Posts: 262
- Joined: Tue Sep 17, 2013 6:50 am
Re: Stack Commit Size, Stack Reserve Size
I have created an issue for that.
- Josef
- Josef
Re: Stack Commit Size, Stack Reserve Size
Ivan, do you have any tutorial links or VERY simple examples of BlackBox instances exchanging data?Ivan Denisov wrote:There is "natural" limit for the 32-bit application. ... BlackBox instances can exchange data with TCP.