Hershel 64-bit compiler

[Ivan Denisov]

Happy to announce, that we are starting a big project for making 64-bit compiler:
http://herschel.oberon.org
William Herschel discovered Oberon in 1787 using his unique telescope. BlackBox is unique IT-tool that we should not lost.

Anton Dmitriev (also known as adimetrius) took the role of dev leader. He is great BlackBox fan and experienced software engineer. He made major system level improvements for Cross-Platform version of BlackBox during the last year.

Anton published redesigned front-end utilizing Component Pascal language structures. Please see HrT module for the details.

We need support of the community for this project!

[Ivan Denisov]

The project is moving on: http://herschel.oberon.org/2020/09/09/post/

* The code generator is up and running

* Global variables can be addressed

* Module body (BEGIN section of a module) can be compiled

* The assignment operator works (for a limited number of basic types)

* The basic types that are implemented are: BYTE SHORTINT INTEGER LONGINT CHAR SHORTCHAR BOOLEAN SET

* Assignments of the form var := literal/var are implemented for the aforementioned types

* Safe type casts with CHR ORD and unsafe casts SYSTEM.VAL are implemented for the aforementioned types

* All of the above features coverd with tests

* Two kinds of standards (correct test responses) for the code generator are produced: the actual generated machine text standards and the field test standards. A field test standard is the values of global variables that the generated code ‘leaves’.

* A testing infrastructure for the code generator is implemented

* The CG tester consisting of an HTTP server receiving requests and a primitive loader actually executing generated 64-bit code is implemented in A2 - the Oberon-based active objects operating system

* The testing panel (front-end) in BB has been updated to accomodate now both front-end and back-end tests and standards.

* The test base now has about 100 test.

Thanks for people supporting Anton!
Mathieu Westerweele (https://www.mobatec.nl) making 250 € monthly donations.
anonymous supporter 200 €
and my self donated 100 €.

Please, support Anton with your donations.

Subscribe to the project Twitter: https://twitter.com/HerschelCP64
Watch & star repository: https://github.com/adimetrius/Herschel
Discuss in chat: https://chat.oberon.org/channel/blackbox64

[adimetrius]

Just actualized the repo. You can now compile Herschel in bare BB 1.8 in Linux. Try out the front-end (full language) of code gen (limited language), with shipped tests or your own. Running generated code requires A2 tester - not public yet, but coming soon.

[X512]

How it is related to existing Blackbox compiler? What kind of executables it produces? OCF adapted for 64 bits or something else?

[adimetrius]

Herschel = RECORD (CP2(OP2(Wirth's Oberon compiler)))

Herschel is based on CP2 - the BlackBox compiler, with quite a few changes. The most significant is the new front-end data structure which utilizes type extension; i call it semantic graph. This of course leads to changes all across the compiler backend as well. The thing I rely on strongly is the modular and procedural decomposition of CP2: I preserve it most of the time.

As far as the code file output that Herschel will produce - it is undecided yet: I haven't gotten that far yet. Any input on that issue?

We try to post progress updates at herschel.oberon.org, twit notifications https://twitter.com/HerschelCP64; and I post more verbose and more frequent articles about Herschel and the development process in Russian vk.com/herschelcompiler.

[Ivan Denisov]

Status update and lookahead
http://herschel.oberon.org/2020/10/16/post/

Since the last status update (that is, within the last month) the following language features have been implemented:

* All operations and relations on basic types (save reals)

* The relation IN for integers and sets

* Global RECORD variables

* Global ARRAY variables - thei required adjustment of the addressing mode within the backend

* Compilation of a global proper PROCEDURE body

* Local procedure variable addressing

* Value parameters of basic types (save reals)

* Variable parameters of basic types (save reals)

* Value and variable parameters of structured types, including open and fixed arrays

* Standard procedure LEN applied to global and local arrays

* Procedure call statement

* All of the above covered with tests

[X512]

Any input on that issue?

I created OCF decoder some time ago: viewtopic.php?f=21&t=268.

[Ivan Denisov]

Hershel can now compile simple modules and link well executable ELF64, bind modules with .so libraries.

You can try this from repository https://github.com/adimetrius/Herschel

Please support the project with your donations! https://herschel.oberon.org/Donate/

[X512]

Can someone explain 64 bit OCF use block format? It's format is incompatible with regular fixups. Also why first empty record is needed in proxy block?

It will be nice to have 64 bit OCF documentation.

[adimetrius]

Good question.

First, the OCF format is great - I like it much better than the ELF, had to study both in the last 5 weeks.
Second, While we can preserve the overall OCF file structure, some things in it will still have to change. The reason is that OCF holds the image for the metadata, and that hold pointers, and they grow from 4 to 8 bytes.

So my idea was that the processor code (a new one for amd64) could indicate that OCF holds 64-bit data. The new code would stop old tools from looking into the file and misinterpreting it.

Then, I've added two numbers to the header block: the importsCount and the externalsCount. Together these two values indicate the size of the proxy table used to link modules to each other and to external libraries. These numbers could be derived from analyzing the UseBlk at the end of OCF, however, having them in the header costs nothing in the compiler and saves the loader/binder from doing the analysis work. It allows the latter to allocate internal structures of sufficient size BEFORE processing the UseBlk, so I thought it's a nice optimization.

Now, to the UseBlk. Currently, Herschel generates PIC - it needs no relocations when it's base is changed (when it's loaded at varying base addresses). It also does not need any binding fixups (to bind/link to imported symbols). Addresses of imported CP procedures and external host library functions are collected in one place - the proxy table. References to the proxy table are hard-coded into the generated code. At load time, the proxy table is allocated at the end of the Meta segment, and should be filled out with appropriate addresses of imported symbols (by the BB or host os loader).
All this to say, the UseBlk's link list holds only one value.

This is how a UseBlk procedure entry was defined in OCF:
UProc = 4X name fprint link.
link = {fixupadr offset} 0X.

Now, in OCF/64, the fixupaddr is the address in the proxy table, and the offset is zero. (Also, fingerprinting is no yet implemented, so the fingerprints are written out as zeroes).

Does this answer your question?