A proposal to the Harbour maintainers

The compiler already knows. This branch makes it say so.

Harbour's compiler works out, on every build, exactly the facts a refactoring tool needs — and then throws them away. This branch keeps them. It changes nothing about the language, and nothing about your build.

don't take our word for it — compile the tree with both compilers and diff the bytes

# every .prg the two compilers both accept, built to portable pcode
# with the new switches OFF, then compared byte for byte:

$ harbour-stock   m.prg -gh -o/tmp/a/     # the compiler you have today
$ harbour-patched m.prg -gh -o/tmp/b/     # this branch, no -x, no -kt
$ cmp /tmp/a/m.hrb /tmp/b/m.hrb

divergent modules                             none

The patched compiler, with -x and -kt off, produces the same bytes as the compiler you have today. That result is the proposal; everything below only explains how it is possible. (The one corpus that did differ turned out to be the stock compiler segfaulting — see the bugs below.)

0 bytes of difference in the compiled output when the switches are off — the state every existing Harbour user is in.

What it adds, in two sentences

-x makes the compiler write down the facts it already computed — the token stream with exact positions, declarations with resolved scopes, calls, sends, the preprocessor's own rewrites — as JSON, while it compiles. Nothing is inferred; nothing new is analysed.

-kt makes the AS annotations that Harbour has parsed for twenty years actually mean something: the compiler emits runtime checks for them, so a declared type becomes an invariant that fails at the site instead of a comment with better syntax.

Why the compiler has to be the one to say it

Every attempt to build a safe Harbour refactoring tool has broken on the same rock: by the time code means something, the source text no longer describes it.

what a tool reading the .prg sees · what actually happens

#xcommand REGISTRO <n> => FUNCTION reg_<n>() ;; LOCAL <n> := <"n">

   REGISTRO Salva
   └─ one word written by the user. after the preprocessor:
      reg_Salva   a function name, pasted from it
      "Salva"     a string, dumped from it
      Salva       a local variable, cloned from it

Three different things, from one token, none of them visible in the file. A tool with a regex must guess which is which — and eventually it guesses wrong and corrupts someone's code. The preprocessor never guesses: it did the paste, the stringify and the clone, and it knows which was which. It simply had no way to tell anyone.

That is the entire thesis. Everything this branch adds is a channel for something Harbour already worked out and then discarded.

The consolidated shape

The dump grew one schema version at a time, but the versions are not the ideas: there are four channels. (The version-by-version record lives in NEWS.md; git has the rest.)

Channel one

Positions that survive the preprocessor

Every token with its real line and byte column — kept correct through #command/#translate rewrites, so an edit lands where the programmer actually typed.

Channel two

The preprocessor, made visible

The rules, every application of them, and what each directive did with each name — copied through, pasted into a symbol, or dumped into a string. Including directives that write other directives, which is how hbclass.ch works.

Channel three

The declared-type channel

AS, DECLARE, class and member declarations, inheritance — carried through exactly as written, including on codeblock parameters, where the parser used to discard them.

Channel four

Enforcement — `-kt`

The declared types stop being advisory. Checked on entry, after each store, on RETURN, and on every Eval of an annotated block. The is-a test runs on the live object, so runtime-assembled classes pass.

The shape of the diff

What a maintainer actually has to live with. The dump's logic is one new file; the hooks in existing files are flag-gated and almost entirely additive.

the shape of the diff — run it yourself, it is one command

$ git remote add ast https://github.com/diegopego/harbour-core
$ git fetch ast feature/compiler-ast-dump
$ git diff --stat master...ast/feature/compiler-ast-dump -- src include

# what you will see:
src/compiler/compast.c    a NEW file  — all of the dump logic. Delete it and the branch is gone.
src/pp/ppcore.c           the tracking tables  — positions, rules, derivation.
                          THE BIGGEST INTRUSION, and the one we most want reviewed.
src/compiler/hbmain.c     the -kt check generation, plus one-line dump hooks
src/compiler/harbour.y    block params keep their declared class; two productions
src/vm/classes.c          the runtime helper the -kt checks call
…and a handful of headers.

Regenerated parser output (.yyc/.yyh) excluded — it is bison's, not ours. Ninety lines removed in total: this branch adds, it does not rearrange.

Two bugs it fixes for everyone

These are independent of the proposal. They are in stock Harbour today, and the fixes stand on their own even if the rest is rejected.

A segfault in the compiler

hb_compClassFind() compared a class name against NULL — a strcmp on a null pointer, reachable from any module that knew about a class. Guarded. As a side effect, W0025 now names the real class instead of printing '(null)'.

A false `W0019`

The duplicate-declaration warning fired even when the second declaration merely completed a missing return type — a completion, not a conflict. Under -w3 -es2 that turned a legitimate build into a failure. A genuine conflict still warns.

This is not speculative

The channels exist because something consumes them. hbrefactor is a working refactoring tool built entirely on this dump — and it is what proves the facts are the right facts.

Its contract is the one that matters here: it refuses to guess. Every edit is applied, the project is recompiled, and the resulting symbols and pcode are compared. If anything moved that should not have, the edit is rolled back. A rename that "almost worked" does not exist. Where the compiler cannot prove something — a name built by &macro, a string that happens to match a symbol — the tool reports it and does not touch it.

That discipline is why the branch looks the way it does. Every time the tool was tempted to guess, the answer was to teach the compiler to keep the fact instead — which is how a twenty-year-old segfault surfaced, and how the preprocessor stopped discarding things it already knew.

make test the consumer's suite, executable, green against this branch — every channel here is exercised by real cases. Run it: that is the proof these facts are usable, and it does not go stale the way a number on a page does.

What we are asking of you

Review the preprocessor side first. ppcore.c is where we touched the most, and where a mistake would cost the most. If any of it should live elsewhere, say so.
Tell us how to split it. The dump (-x) and the enforced annotations (-kt) are independent and could land as separate PRs. The two bug fixes could go in first, alone.
Rule on the schema. It has moved fifteen times because consumers kept needing more. We do not know what stability guarantee, if any, Harbour wants to make here — and we would rather be told than guess.

What we do not have good answers for

Said plainly, because you would find them anyway:

The dump's cost is not free at compile time when -x is on — the tracking tables grow with the module. It is off by default and off in every normal build, but we have not profiled a very large project with it on.

-kt emits real checks that really run. It is a development and CI mode. We are not proposing it as a default, and it should probably never be one.

Where the detail lives

This page is the concept, kept consolidated on purpose — it is not a log, and it does not grow a new section every commit.

NEWS.md — what changed, for the Harbour programmer. (GNU convention: ChangeLog is the developer's log, NEWS is the user's.)
ChangeLog.txt — the project's own technical log, untouched.
git — the how. Every commit says which fact was missing, why the consumer could not work around it, and what it cost to keep it.
hbrefactor — the why. Each channel here exists because a real refactoring failed without it; that repo records which one, and how.