ingo/agenticCode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reap feature

Browse Source
This commit is contained in:
Ingo Schnabel
2026-08-10 13:36:44 +03:00
parent 97081718fd
commit b6007b2096
8 changed files with 711 additions and 479 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
ac-cli/src/main/resources/agenticcode.properties
View File

@@ -4,4 +4,4 @@
server.url=http://localhost:8787
# Stamped by manage-ac.sh (stamp_cli_version) from ac-code-server's agenticcode.version
# at build time. "dev" means this jar wasn't built via manage-ac.sh.
version=203
version=207

2
ac-code-server/src/main/resources/application.properties
View File

@@ -3,7 +3,7 @@ quarkus.http.port=8787
# AgenticCode's own release counter (not the Maven project version) — bump this by hand for each
# release. Single source of truth for the startup log line, GET /api/version, and the OpenAPI
# info version (referenced below via property expression, not duplicated).
agenticcode.version=203
agenticcode.version=207
# OpenAPI / Swagger UI (item 48) — the generated spec is the contract the web-UI TS client
# is generated against. Served at /q/openapi (yaml/json); Swagger UI at /q/swagger-ui in dev.
mp.openapi.extensions.smallrye.info.title=AgenticCode API

219
ac-code-server/src/test/java/com/agenticcode/codeserver/api/StaleCallEdgeReapIT.java Normal file
View File

@@ -0,0 +1,219 @@
package com.agenticcode.codeserver.api;
import io.quarkus.test.junit.QuarkusTest;
import io.restassured.RestAssured;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.*;
/**
* Item 124: a call edge must not outlive the call it was parsed from.
*
* <p>Item 58's sweep deletes stale <b>nodes</b>; an edge is only reaped when one of its endpoints goes
* with it. So an edge survives whenever both endpoints legitimately survive — which is exactly what a
* <b>parser fix</b> produces: the calling subroutine is untouched, and the old target is a placeholder
* ({@code sourceFile=""}) that is never file-swept. The corrected call is then merely <em>added</em>
* beside the wrong one, and both are served.
*
* <p>Found for real: after items 120/121/123 shipped, {@code DAGCHEN0/callees} in {@code upms} listed
* the correct {@code YAGCHBN0} <em>and</em> the pre-fix phantom {@code AGNT-CHG-CMP-SP} from the same
* call site, with 497 such edges surviving a full deep refresh.
*
* <p>The fixture edits only the CALLNAT target and keeps the enclosing subroutine, because that is the
* distinguishing case. {@code DerivedCallsModuleRefreshIT} removes the whole subroutine, so there the
* {@code FUNCTION} node disappears and {@code DETACH DELETE} takes the edge along — which is why the
* bug hid behind a green suite.
*/
@QuarkusTest
class StaleCallEdgeReapIT {
private static final String PROJECT = "nat-stale-call-reap";
private static final String TARGET_A = """
* First call target.
DEFINE DATA LOCAL
END-DEFINE
END
""";
private static final String TARGET_B = """
* Second call target.
DEFINE DATA LOCAL
END-DEFINE
END
""";
/** Calls RTARGETA from a subroutine that must survive the edit unchanged. */
private static final String CALLER_A = """
* Caller in its first state.
DEFINE DATA LOCAL
01 #TGT (A8)
END-DEFINE
*
PERFORM CALL-STEP
*
DEFINE SUBROUTINE CALL-STEP
CALLNAT 'RTARGETA'
END-SUBROUTINE
*
END
""";
/** Only the target changed — same subroutine, same line, same everything else. */
private static final String CALLER_B = """
* Caller in its first state.
DEFINE DATA LOCAL
01 #TGT (A8)
END-DEFINE
*
PERFORM CALL-STEP
*
DEFINE SUBROUTINE CALL-STEP
CALLNAT 'RTARGETB'
END-SUBROUTINE
*
END
""";
/** Calls a module that does not exist, so the target is an unresolved placeholder node. */
private static final String PHANTOM_CALLER = """
* Caller whose target does not exist — a placeholder, as a pre-fix parser artefact is.
DEFINE DATA LOCAL
END-DEFINE
*
PERFORM CALL-STEP
*
DEFINE SUBROUTINE CALL-STEP
CALLNAT 'RPHANTOM'
END-SUBROUTINE
*
END
""";
/** Same subroutine, now calling a real module: the phantom must be reaped, node and all. */
private static final String PHANTOM_CALLER_FIXED = """
* Caller whose target does not exist — a placeholder, as a pre-fix parser artefact is.
DEFINE DATA LOCAL
END-DEFINE
*
PERFORM CALL-STEP
*
DEFINE SUBROUTINE CALL-STEP
CALLNAT 'RTARGETA'
END-SUBROUTINE
*
END
""";
@TempDir
static Path root;
@BeforeAll
static void createProject() {
RestAssured.port = Integer.getInteger("quarkus.http.test-port", 8081);
write("RTARGETA.nat", TARGET_A);
write("RTARGETB.nat", TARGET_B);
write("RCALLER.nat", CALLER_A);
write("RPHANT.nat", PHANTOM_CALLER);
given().contentType("application/json")
.body(new ProjectResource.ProjectRequest(null, root.toString(), null, "natural", null, null))
.when().post("/api/projects/" + PROJECT)
.then().statusCode(201);
}
private static void write(String fileName, String content) {
try {
Files.writeString(root.resolve(fileName), content);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
private static void refresh() {
given().when().post("/api/projects/" + PROJECT + "/refresh?deep=true").then().statusCode(200);
}
private static io.restassured.path.json.JsonPath callees(String module) {
return given().pathParam("name", module)
.when().get("/api/projects/" + PROJECT + "/modules/{name}/callees")
.then().statusCode(200).extract().jsonPath();
}
/**
* One test per transition rather than per assertion: each asserts a before/after pair on shared
* mutable server state, so splitting the halves would make the outcome depend on JUnit method order.
*/
@Test
void aRetargetedCallDoesNotKeepItsOldTarget() {
write("RCALLER.nat", CALLER_A);
refresh();
// Guard: prove the first edge exists, so the "gone" assertion cannot pass vacuously.
org.junit.jupiter.api.Assertions.assertTrue(
callees("RCALLER").getList("items.name").contains("RTARGETA"),
"the first target must be there before we can prove it is removed");
write("RCALLER.nat", CALLER_B);
refresh();
var names = callees("RCALLER").getList("items.name");
org.junit.jupiter.api.Assertions.assertTrue(names.contains("RTARGETB"),
"the new target must be present: " + names);
org.junit.jupiter.api.Assertions.assertFalse(names.contains("RTARGETA"),
"the old target outlived the call it was parsed from: " + names);
}
/**
* The placeholder case — the shape a fixed parser bug actually leaves behind. Both the edge and the
* now-edgeless placeholder node must go; the node is never file-swept, so without the item-124 node
* sweep it would keep surfacing in identifier search as an unresolved call target nothing calls.
*/
@Test
void aReapedPlaceholderTargetIsAlsoRemovedAsANode() {
write("RPHANT.nat", PHANTOM_CALLER);
refresh();
org.junit.jupiter.api.Assertions.assertTrue(
callees("RPHANT").getList("items.name").contains("RPHANTOM"),
"the phantom target must exist before we can prove it is swept");
write("RPHANT.nat", PHANTOM_CALLER_FIXED);
refresh();
var names = callees("RPHANT").getList("items.name");
org.junit.jupiter.api.Assertions.assertFalse(names.contains("RPHANTOM"),
"the stale placeholder edge survived: " + names);
given().queryParam("name", "RPHANTOM")
.when().get("/api/projects/" + PROJECT + "/search/identifier")
.then().statusCode(200)
.body("findAll { it.name == 'RPHANTOM' }", is(empty()));
}
/**
* The reap deletes <em>all</em> of a re-parsed file's call edges, including the two kinds enrichment
* builds rather than the parser: {@code folded} and {@code resolvedBy: 'manual'}. Both are rebuilt by
* finalize steps that run at every enrichment level — this pins that, because if it were false the
* reap would silently discard a user's manual override on every refresh.
*/
@Test
void aManualOverrideSurvivesTheReap() {
write("RCALLER.nat", CALLER_B);
refresh();
given().contentType("application/json")
.body(Map.of("originFile", "RPHANT.nat", "lineNo", 8,
"targets", List.of("RTARGETB"), "variable", "MANUAL-PIN"))
.when().post("/api/projects/" + PROJECT + "/dynamic-calls/overrides")
.then().statusCode(200);
refresh();
given().when().get("/api/projects/" + PROJECT + "/dynamic-calls/overrides")
.then().statusCode(200)
.body("findAll { it.originFile == 'RPHANT.nat' }", not(empty()));
}
}

174
ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/CypherQueries.java
View File

@@ -152,6 +152,63 @@ public final class CypherQueries {
DELETE r
""";
/**
* Item 124: the {@code CALLS} counterpart of items 86 and 106 — reaps a re-parsed Natural file's
* call edges before the fresh ones are merged.
*
* <p>Item 58's node sweep only removes nodes the fresh parse no longer produces, so a call edge
* survives whenever <em>both</em> its endpoints legitimately survive. That is the normal case when a
* <b>parser fix</b> changes a call's target: the calling subroutine is unchanged (its {@code FUNCTION}
* node is re-merged and kept) and the old target is a placeholder ({@code sourceFile=""}, never
* file-swept), so nothing reaps the edge between them — the corrected call is merely <em>added</em>
* beside the wrong one. After items 120/121/123 shipped, {@code DAGCHEN0/callees} listed both the
* real {@code YAGCHBN0} and the pre-fix phantom {@code AGNT-CHG-CMP-SP} from the same call site;
* 497 such edges survived a full deep refresh of {@code upms}.
*
* <p>Reaches beyond item 86's placeholder-only scope on purpose: bug #63's `CALLNAT`-in-a-string
* matches resolved onto <b>real</b> modules, so a placeholder-only reap would leave that whole
* class of artefact behind. The one thing it must <em>not</em> touch is the dynamic-call
* resolvers' own output, hence the {@code WHERE}:
*
* <ul>
* <li><b>Reaped</b> — every edge to a placeholder target ({@code sourceFile=""}), plus every
* {@code PERFORM}/{@code CALLNAT}/{@code INCLUDE_MACRO} edge. All of these are emitted by the
* parser on every parse, at <em>both</em> tiers (the coarse scanner expands copycode exactly
* as the deep parser does), so the merge that follows re-creates the current ones and
* unchanged edges round-trip identically.</li>
* <li><b>Kept</b> — {@code CALLNAT_DYNAMIC} edges to a <em>real</em> module. The parser cannot
* know a dynamic target and always emits a placeholder, so such an edge is by construction
* enrichment-built: a fold, a manual override, or an intra/cross resolution. In {@code upms}
* 486 edges are of this kind and <b>392 of them carry no marker at all</b> — no
* {@code folded}, no {@code resolvedBy} — so the target's file is the only thing that
* distinguishes them from parser output.</li>
* </ul>
*
* <p>That exception is not theoretical: reaping them broke
* {@code AnalysisResourceIT#flowForwardPathWarmCrossesIntoDynamicallyDispatchedCallee}. The item-37a
* path-warm resolves a cross-module dispatch in one round and re-ingests in the next, and a
* <em>scoped</em> finalize does not reliably re-resolve an edge whose far side is outside its scope —
* so the reap deleted a resolution nothing rebuilt, and the dataflow trace stopped at the dispatch
* boundary.
*
* <p>Gated on {@code reconcile} (deep re-ingest), because {@code ..._INTRA_INDIRECT} and
* {@code ..._CROSS} are gated on {@code resolveFields}/{@code dataflow} and so do <em>not</em> run
* in a coarse finalize. Same rationale as item 74.
*
* <p><b>Scope limit.</b> Keyed on the source node's file, so it misses the 605 call edges whose
* source subroutine is defined <em>inside a copycode</em> (14 of them stale). Those nodes are
* MERGEd per {@code (type, name, sourceFile)} and therefore <b>shared</b> by every module that
* includes the copycode, so reaping them during a single module's refresh would delete edges other
* modules contributed and not re-create them. Fixing that needs a per-module identity for
* copycode-resident nodes — a separate item, not a side effect of this one.
*/
public static final String DELETE_STALE_NATURAL_CALL_EDGES = """
UNWIND $sourceFiles AS f
MATCH (src:AstNode {project: $project, sourceFile: f, language: 'natural'})-[r:CALLS]->(t)
WHERE t.sourceFile = "" OR r.callKind <> 'CALLNAT_DYNAMIC'
DELETE r
""";
/**
* Item 88: a finalize sweep that deletes an orphaned {@code DB_TABLE}/{@code WORKFILE} placeholder —
* one left with <em>no</em> relationships after edge reaping/resolution. Companion to item 86: that
@@ -231,6 +288,33 @@ public final class CypherQueries {
DELETE t
""";
/**
* Item 124, node half: the {@code MODULE} counterpart of item 88's table sweep. Once
* {@link #DELETE_STALE_NATURAL_CALL_EDGES} reaps the last call to a call-target placeholder, the
* placeholder <em>node</em> is left edgeless — it is never file-swept ({@code sourceFile=""}) — and
* still surfaces in {@code search/identifier} and the module inventory as an unresolved call target
* that nothing actually calls.
*
* <p>Guarded on {@code sourceFile = ""} so it can only ever remove a placeholder: a <b>real</b>
* parsed module that happens to call nothing and be called by nothing is a legitimate standalone
* program and must survive. Degree-0 only, so a placeholder still referenced by any call is kept.
* Runs beside item 88's table sweep, after all edge resolution.
*
* <p>{@code duplicatePaths IS NULL} excludes item 114's duplicate markers, which are a
* <em>second</em> reason for an edgeless placeholder to exist and are deliberately kept: an
* unreferenced duplicate identity is a degree-0 placeholder whose whole purpose is to record "this
* name exists in two files", so the module endpoints can answer {@code 409 DUPLICATE_IDENTITY}
* rather than {@code 404}. {@link #MARK_DUPLICATE_IDENTITIES} runs <em>before</em> finalize, so
* without this guard the sweep deleted the marker it had just written and the endpoints fell back
* to {@code 404} — caught by {@code DuplicateIdentityIT}. Markers are reaped on their own terms by
* {@link #CLEAR_DUPLICATE_MARKERS} when the source conflict goes away.
*/
public static final String DELETE_ORPHANED_PLACEHOLDER_MODULES = """
MATCH (t:AstNode {project: $project, type: 'MODULE', sourceFile: ""})
WHERE t.duplicatePaths IS NULL AND NOT (t)--()
DELETE t
""";
public static final String MODULE_SOURCE_FILE = """
MATCH (m:MODULE {project: $project})
// Item 117: this endpoint is not behind the resolving guard, so it accepts the identity or
@@ -1586,6 +1670,96 @@ public final class CypherQueries {
ON CREATE SET r2.folded = true
SET r2.dynamicVar = dyn.dynamicVar, r2.args = dyn.args
""";
/**
* Item 108: resolve the dispatch-<em>table</em> idiom — the one the {@code dispatch-table} endpoint
* is named after, and the one it did not understand. A router fills an array with literal program
* names and then calls through an indexed read of it:
* <pre>
* ASSIGN #WT-OBJ-PROG (1) = 'WXSPOD0S' ← one row per target
* ASSIGN #WT-OBJ-PROG (2) = 'WXSDAD0S'
* …
* #W-ACT-PROG := #WT-OBJ-PROG (#I-OBJ) ← scalar fed from an array element
* CALLNAT #W-ACT-PROG …
* </pre>
*
* <p>Item 83's fold does not reach this: it follows literals written to the dispatch variable
* <em>itself</em>, and here the variable is fed from an array element, so the literals sit one hop
* away on a different node. 65 modules in {@code upms} use the idiom, and every one of their targets
* is a literal in the source — nothing is runtime-dependent, yet the call sites sat in
* {@code dynamic-calls/unresolved} and the endpoint answered {@code []}.
*
* <p><b>Which index is live at runtime is not statically known, so this deliberately
* over-approximates:</b> it emits one candidate edge per <em>distinct literal ever assigned to any
* element</em> of the feeding array, exactly the multi-target model item 82 already allows a manual
* override. That is also what makes it robust against the 9 of 65 modules that build the table
* inside an {@code IF}/{@code ELSE} — there the same index carries different values per branch
* (`WGARCX0S` index 1 is `WXSFUD0S` in one branch and `WXSGAD0S` in the other), so any
* index-keyed pairing would be plainly wrong while the candidate <em>set</em> stays correct.
*
* <p>No line-ordering constraint between the array writes and the call site, unlike item 83's fold:
* a Natural subroutine is often <em>defined</em> after the statement that {@code PERFORM}s it, so
* requiring the init to precede the call would silently drop those. Ordering carries no meaning here
* anyway — the whole point is that the live index is unknown.
*
* <p>Module scope is expressed by {@code sourceFile} equality, never by {@code CONTAINS} — item 75's
* containment cycles make an unbounded traversal a correctness <em>and</em> latency hazard. Honours
* the manual-override precedence like the other resolvers, tags its edges {@code folded = true}
* (inferred, not observed) plus {@code viaTable = true} so this provenance stays distinguishable
* from item 83's string fold. Idempotent (MERGE); a literal that is not a real ingested module
* produces no edge.
*/
public static final String RESOLVE_DYNAMIC_CALLNAT_TABLE = """
MATCH (src:AstNode {project: $project})-[dyn:CALLS]->(ph:AstNode {project: $project, type: 'MODULE', sourceFile: ""})
WHERE dyn.callKind = 'CALLNAT_DYNAMIC' AND src.sourceFile <> ""
AND NOT EXISTS {
MATCH (o:DynamicCallOverride {project: $project})
WHERE o.originFile = coalesce(dyn.originFile, src.sourceFile) AND o.lineNo = dyn.lineNo
}
MATCH (caller:AstNode {type: 'MODULE', project: $project, sourceFile: src.sourceFile})
MATCH (fw:AstNode {project: $project, sourceFile: src.sourceFile})-[fwr:WRITES]->(:AstNode {project: $project, name: dyn.dynamicVar})
WHERE fwr.value IS NOT NULL AND NOT fwr.value STARTS WITH "'" AND fwr.value CONTAINS '('
WITH src, dyn, caller, coalesce(dyn.originFile, src.sourceFile) AS originFile,
trim(split(fwr.value, '(')[0]) AS arrName
WHERE arrName <> ""
MATCH (aw:AstNode {project: $project, sourceFile: src.sourceFile})-[awr:WRITES]->(:AstNode {project: $project, name: arrName})
WHERE awr.value IS NOT NULL AND awr.value STARTS WITH "'"
WITH DISTINCT caller, dyn, originFile, trim(replace(awr.value, "'", "")) AS lit
MATCH (target:AstNode {type: 'MODULE', project: $project, name: lit})
WHERE target.sourceFile <> ""
MERGE (caller)-[r2:CALLS {callKind: 'CALLNAT_DYNAMIC', lineNo: dyn.lineNo, originFile: originFile}]->(target)
ON CREATE SET r2.folded = true, r2.viaTable = true
SET r2.dynamicVar = dyn.dynamicVar, r2.args = dyn.args
""";
/**
* Scoped variant of {@link #RESOLVE_DYNAMIC_CALLNAT_TABLE}: only dynamic call sites in callers whose
* name is in {@code $names} (a deeply-ingested program tree).
*/
public static final String RESOLVE_DYNAMIC_CALLNAT_TABLE_SCOPED = """
UNWIND $names AS mn
MATCH (caller:AstNode {type: 'MODULE', project: $project, name: mn})
MATCH (src:AstNode {project: $project, sourceFile: caller.sourceFile})-[dyn:CALLS]->(ph:AstNode {project: $project, type: 'MODULE', sourceFile: ""})
WHERE dyn.callKind = 'CALLNAT_DYNAMIC'
AND NOT EXISTS {
MATCH (o:DynamicCallOverride {project: $project})
WHERE o.originFile = coalesce(dyn.originFile, src.sourceFile) AND o.lineNo = dyn.lineNo
}
MATCH (fw:AstNode {project: $project, sourceFile: src.sourceFile})-[fwr:WRITES]->(:AstNode {project: $project, name: dyn.dynamicVar})
WHERE fwr.value IS NOT NULL AND NOT fwr.value STARTS WITH "'" AND fwr.value CONTAINS '('
WITH src, dyn, caller, coalesce(dyn.originFile, src.sourceFile) AS originFile,
trim(split(fwr.value, '(')[0]) AS arrName
WHERE arrName <> ""
MATCH (aw:AstNode {project: $project, sourceFile: src.sourceFile})-[awr:WRITES]->(:AstNode {project: $project, name: arrName})
WHERE awr.value IS NOT NULL AND awr.value STARTS WITH "'"
WITH DISTINCT caller, dyn, originFile, trim(replace(awr.value, "'", "")) AS lit
MATCH (target:AstNode {type: 'MODULE', project: $project, name: lit})
WHERE target.sourceFile <> ""
MERGE (caller)-[r2:CALLS {callKind: 'CALLNAT_DYNAMIC', lineNo: dyn.lineNo, originFile: originFile}]->(target)
ON CREATE SET r2.folded = true, r2.viaTable = true
SET r2.dynamicVar = dyn.dynamicVar, r2.args = dyn.args
""";
/**
* Deletes the dynamic-call marker edges (the {@code CALLNAT_DYNAMIC} {@code CALLS} edges that
* point at a variable-named placeholder, {@code sourceFile = ""}) <em>only for call sites that

18
ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/GraphRepository.java
View File

@@ -172,6 +172,12 @@ public class GraphRepository {
// resolver and before apply-manual/placeholder cleanup.
statements.add(new EnrichmentStep("resolve-dynamic-callnat-fold" + sfx,
scoped ? CypherQueries.RESOLVE_DYNAMIC_CALLNAT_FOLD_SCOPED : CypherQueries.RESOLVE_DYNAMIC_CALLNAT_FOLD));
// Item 108: the dispatch-TABLE idiom — literals filled into an array, called through an indexed
// read of it. Item 83's fold cannot see these: the literals sit one hop away, on the array node
// rather than on the dispatch variable. Emits one candidate per distinct literal, since the live
// index is not statically known. Cheap (bounded by dynamic call sites), so it runs at every level.
statements.add(new EnrichmentStep("resolve-dynamic-callnat-table" + sfx,
scoped ? CypherQueries.RESOLVE_DYNAMIC_CALLNAT_TABLE_SCOPED : CypherQueries.RESOLVE_DYNAMIC_CALLNAT_TABLE));
if (resolveFields) {
statements.add(new EnrichmentStep("resolve-dynamic-callnat-intra-indirect" + sfx,
scoped ? CypherQueries.RESOLVE_DYNAMIC_CALLNAT_INTRA_INDIRECT_SCOPED : CypherQueries.RESOLVE_DYNAMIC_CALLNAT_INTRA_INDIRECT));
@@ -263,6 +269,9 @@ public class GraphRepository {
statements.add(new EnrichmentStep("resolve-view-alias-access-nodes",
CypherQueries.RESOLVE_VIEW_ALIAS_ACCESS_NODES));
statements.add(new EnrichmentStep("delete-orphaned-placeholder-tables", CypherQueries.DELETE_ORPHANED_PLACEHOLDER_TABLES));
// Item 124: the same for call-target placeholders left edgeless by the CALLS reap — otherwise a
// phantom target of a since-fixed parser bug keeps showing up as an unresolved module.
statements.add(new EnrichmentStep("delete-orphaned-placeholder-modules", CypherQueries.DELETE_ORPHANED_PLACEHOLDER_MODULES));
// Item 40: flag surviving placeholders as (un)resolved by whether a real definition now exists.
// Runs last so it sees the fully redirected/reaped graph. Project-wide, cheap, idempotent.
statements.add(new EnrichmentStep("stamp-unresolved-placeholders", CypherQueries.STAMP_UNRESOLVED_PLACEHOLDERS));
@@ -2239,6 +2248,15 @@ public class GraphRepository {
// covered by the placeholder-only reap above.
tx.run(CypherQueries.DELETE_STALE_NATURAL_USING_EDGES,
Map.of("project", project, "sourceFiles", List.copyOf(freshFiles)));
// Item 124: same for CALLS. A parser fix that changes a call's target leaves both endpoints
// alive (the calling subroutine is unchanged; the old target is a never-swept placeholder),
// so the corrected call was only ever added beside the wrong one. Deep re-ingest only —
// two dynamic-call resolvers are skipped in a coarse finalize and could not rebuild what
// this deletes.
if (reconcile) {
tx.run(CypherQueries.DELETE_STALE_NATURAL_CALL_EDGES,
Map.of("project", project, "sourceFiles", List.copyOf(freshFiles)));
}
}
for (Map.Entry<EdgeType, List<Map<String, @Nullable Object>>> entry : edgesByType.entrySet()) {
tx.run(CypherQueries.mergeEdgesBatch(entry.getKey()),

713
x-docs/agent-api-system-prompt.md
View File

@@ -1,541 +1,304 @@
# AgenticCode API — Agent System Prompt
You are an agent that analyzes source code (Software AG **Natural** and
**Java**) through the AgenticCode REST API. The server has already parsed the source into a unified AST
stored as a graph in Neo4j; you query that graph read-only. Use the API as
your source of truth about the code structure — do not guess at structure you
can look up. (You may still need to read actual source text — see "Reading
source" below.)
You analyze **Natural** (Software AG) and **Java** source through the AgenticCode REST API. The
server has parsed the source into a unified AST in Neo4j; you query that graph. It is your source of
truth for code structure — look structure up, don't guess.
**Read-only scope — with one exception.** You query already-ingested
projects; you do not create, modify, delete, or ingest. The **sole** write you
are expected to make is **resolving unresolvable dynamic `CALLNAT` targets**
(item 82): when the graph shows a dynamic call it could not resolve, you must
investigate the source and pin the correct target via the dynamic-call
override API. See "Resolving unresolved dynamic `CALLNAT` calls (required)"
below. Nothing else is in your write scope.
**Read-only, with one exception.** You never create, modify, delete, or ingest. The sole write you
must make is **pinning unresolvable dynamic `CALLNAT` targets** (§4) — required, not optional.
## Ground rules
## 1. Rules
- **Only use the `/source` endpoints (`/nodes/{id}/source`,
`/modules/{name}/source`) if you have no other access to the source
code.** If you can read the checkout directly (local clone, IDE, coding
agent), always read the file yourself instead — see "Reading source" below.
- **Base URL:** `http://localhost:8787`. Everything under `/api`, JSON
responses.
- **Project-scoped:** almost every endpoint is `/api/projects/{project}/...`.
List projects first (`GET /api/projects`), then scope to one.
- **`{name}` is a node name, never a file path** — the Natural
program/subprogram name, or a Java class's **fully-qualified** name
(`com.example.OrderService`, `com.example.Outer.Inner`). The Java simple name
is accepted as a short form and resolves when it identifies exactly one class;
otherwise you get `409 AMBIGUOUS_NAME`. **Case-sensitive.**
- **Errors are structured:** `{ "error", "code", "details" }`.
- `404 PROJECT_NOT_FOUND` — bad `{project}` (checked first on every endpoint).
- `400 INVALID_TYPE` — bad `type` on identifier/annotation search.
- `400 MISSING_VALUE` / `400 MISSING_NAME` — required query param missing/blank
(`value` on `search/value`, `name` on `search/annotation`).
- `400 MISSING_LINE_RANGE` — `startLine`/`endLine` missing on `modules/{name}/source`.
- `400 NO_SOURCE_FILE` — `/source` on an unresolved placeholder node.
- `404 NODE_NOT_FOUND` / `404 MODULE_NOT_FOUND` — unknown id / module name.
Every `/modules/{name}/…` endpoint checks this: an unknown module name is a
`404`, never an empty `200`.
- `409 AMBIGUOUS_NAME` — several real modules share this **short** name (ordinary
in Java: nested `@Nested` classes, `Builder`, `WorkingStorage`). A Java module's
real name is its fully-qualified name (item 117), and that always resolves
exactly; the short name is a convenience. `details.qualifiedNames` lists the
candidates — repeat the request with one of them (or `?sourceFile=`, listed in
`details.candidates`). Do **not** read this as "not found": the module exists
several times over. Natural names are unique, so this cannot occur there.
- `409 NOT_DEEPLY_INGESTED` / `409 NOT_INGESTED` — the module needs a
per-module deep ingest (`nextAction` names the endpoint). That's **out of
your read-only scope** — report it, don't call it. Not the same as "no
data": the data may exist once ingested. Two situations produce it:
- field-level dataflow endpoints (`flow-forward`/`flow-backward`/`field-flow`)
on a module that is only call-graph-ingested;
- **any** `/modules/{name}/…` endpoint on an *unresolved placeholder* — a
module something calls but whose source was never parsed. Its
`callers` and `graph` still answer `200` with real data (that comes
from the calling modules), so **fall back to `/callers`** to learn
what you can. Everything else about it is unknowable, not empty.
- **Never read an empty `200` from a module endpoint as "analysed, nothing
found".** Since item 107 the API distinguishes *unknown* (`404`), *not
analysable* (`409`) and *analysed, genuinely empty* (`200` with an empty
body). Only the last one licenses the conclusion "there is nothing here".
- **`null` means "not determined," not an error** — `dataType`, `value`,
`table`, `view`, `module`, `description` are nullable by design.
- **Don't fabricate endpoints.** Only what's listed below exists.
- **Reading source:** if you have direct filesystem access to the checkout
(e.g. you're a coding agent operating on a local clone), read files
directly — don't call `/nodes/{id}/source` or `/modules/{name}/source`.
Use `sourceFile`/`startLine`/`endLine` from graph responses to know where
to look. Only use the `/source` endpoints when you have no filesystem
access to the project root.
- **Base URL** `http://localhost:8787`, everything under `/api`, JSON. Almost every endpoint is
`/api/projects/{project}/...` — list projects first.
- **`{name}` is a node name, never a file path.** Natural program/subprogram name, or a Java
**fully-qualified** class name (`com.example.Outer.Inner`). A Java simple name works as a short
form when unique, else `409 AMBIGUOUS_NAME`. **Case-sensitive.** Encode `#` as `%23`.
- **Read source from disk if you can.** If you have filesystem access to the checkout, read files
yourself using `sourceFile`/`startLine`/`endLine` from responses. Use `/nodes/{id}/source` and
`/modules/{name}/source` **only** when you have no filesystem access.
- **`null` = "not determined", not an error** (`dataType`, `value`, `table`, `view`, `module`,
`description` are nullable by design).
- **Don't invent endpoints.** Only what is listed here exists.
## Language applicability
### Errors `{ error, code, details }`
Every endpoint runs for any module but returns data only where the concept
exists — an inapplicable query returns an empty list, not an error.
| Code | Meaning |
|--------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `404 PROJECT_NOT_FOUND` | bad `{project}` — checked first on every endpoint |
| `404 NODE_NOT_FOUND` / `MODULE_NOT_FOUND` | unknown id / module name. Every `/modules/{name}/…` checks this: unknown is `404`, never an empty `200` |
| `409 AMBIGUOUS_NAME` | several real modules share this **short** name (common in Java: `Builder`, `@Nested`). **Not** "not found" — it exists several times. `details.qualifiedNames` / `details.candidates` list them; retry with one, or `?sourceFile=`. Cannot occur for Natural |
| `409 NOT_DEEPLY_INGESTED` / `NOT_INGESTED` | needs a per-module deep ingest (`nextAction` names it). **Out of your scope — report, don't call.** Not "no data": data may exist once ingested |
| `400 INVALID_TYPE` | bad `type` on identifier/annotation search |
| `400 MISSING_VALUE` / `MISSING_NAME` | required param missing/blank |
| `400 MISSING_LINE_RANGE` | `startLine`/`endLine` missing on `modules/{name}/source` |
| `400 NO_SOURCE_FILE` | `/source` on an unresolved placeholder |
| `400 UNKNOWN_TARGET` | override target is not a real module |
| Endpoint | Natural | Java | Notes |
|---------------------------------------------------------------|---------|------|---------------------------------------------------------------------------------------------------------------------------------------------------|
| `/modules`, `/modules/{name}/context`, `/digest` | ✓ | ✓ | language-neutral |
| `/modules/{name}/callers` · `/callees` | ✓ | ✓ | `edgeKind`: `CALLNAT`/`PERFORM` (Natural) · `METHOD_CALL`/`CONSTRUCTOR` (Java) · `EXTENDS`/`IMPLEMENTS` (both) · `INJECTS`/`REFERENCES` (Java DI) |
| `/modules/{name}/call-tree` | ✓ | ✓ | `?resolveInterfaces=` / `?followWiring=` are Java-only |
| `/modules/{name}/db-accesses` · `/sql-statements` | ✓ | ✓ | Natural ADABAS/SQL; Java JPA/Panache/`@Query` |
| `/modules/{name}/workfile-accesses` | ✓ | — | Natural-only — `READ`/`WRITE WORK FILE` (sequential/flat-file I/O). Separate from `db-accesses`; a work file is not a DB table |
| `/db-tables/{name}/columns` | ✓ | ✓ | Natural `INTO VIEW`/PDA, or Java `@Entity` mapping |
| `/modules/{name}/columns` | — | ✓ | Java JPA/Hibernate entity columns |
| `/modules/{name}/functions` | ✓ | ✓ | `?includeInherited=true` is Java-only |
| `/modules/{name}/functions/{fn}/overrides`, `/overrides` | — | ✓ | Java-only — concrete subclass overrides of a base-class method (single or bulk) |
| `/modules/{name}/dispatch-table` | ✓ | — | Natural-only — `DECIDE ON VALUE OF` routing table |
| `/variables/{name}/reads` · `/writes` | ✓ | ✓ | Natural `VARIABLE`/`CONSTANT`; Java `FIELD` |
| `/variables/{name}/flow-forward` · `/flow-backward` | ✓ | ✓ | deep-ingest only, both languages |
| `/variables/{field}/field-flow` | ✓ | — | Natural-only — shared-PDA producer→consumer across the call graph |
| `/data-structures/{name}/fields` | ✓ | — | Natural-only — `DEFINE DATA`/LDA/PDA |
| `/search/identifier` · `/search/value` | ✓ | ✓ | language-neutral |
| `/search/annotation` | — | ✓ | Java-only |
| `/nodes/{id}`, `/nodes/{id}/source`, `/modules/{name}/source` | ✓ | ✓ | language-neutral |
| `/modules?extends=`, `?moduleKind=` | partial | ✓ | `extends` is Java-only; `moduleKind` also gives a best-effort Natural PROGRAM/SUBPROGRAM guess |
`409 NOT_INGESTED` arises two ways: field-level dataflow (`flow-forward`/`flow-backward`/
`field-flow`) on a call-graph-only module; or **any** `/modules/{name}/…` on an *unresolved
placeholder* (called by something, source never parsed). A placeholder's `callers` and `graph` still
answer `200` with real data — it comes from the calling modules — so **fall back to `/callers`**.
### Natural: `generatedDir` is canonical, `user_exit` is LoC-only
> **Never read an empty `200` as "analysed, nothing found".** The API separates *unknown* (`404`),
> *not analysable* (`409`) and *analysed, genuinely empty* (`200`). Only the last licenses that
> conclusion.
A Natural project may be configured with a **`generatedDir`**/**`userExitDir`** pair (e.g.
`generated_src`/`user_exit`). The generated module already contains its hand-written user-exit twin
inline, so **all structural analysis — modules, call graph, DB access, functions, data structures,
identifiers, dataflow, dispatch table — runs against the `generatedDir` source, which is the one and
only ingested module.** User-exit files are **not** ingested as standalone modules (they would collide
by name with the generated twin); they are scanned solely to compute the **generated-vs-manually-written
LoC split** surfaced by `/loc` (`userExitLoc`/`userExitSloc` vs `generatedExclusiveLoc`/…). Practical
consequence: when you read source to verify an API response for a Natural module, read the
`generatedDir` copy (e.g. `generated_src/subprogram/WGEAGB0S.nat`) — the `user_exit` copy is a partial
fragment and does not represent what was analysed. See the item-47 split in
`agent-api-usage-ac-implementation.md`.
### Natural: `generatedDir` is canonical
## 1. Pick a project
With a `generatedDir`/`userExitDir` pair (e.g. `generated_src`/`user_exit`), the generated module
already contains its hand-written user-exit twin inline. **All structural analysis runs against
`generatedDir`, the only ingested module.** User-exit files are scanned solely for the LoC split in
`/loc` (`userExitLoc` vs `generatedExclusiveLoc`). So when verifying a response against source, read
the `generatedDir` copy — the `user_exit` copy is a partial fragment.
### Language applicability
Every endpoint runs for any module but returns data only where the concept exists; inapplicable
queries return an empty list, not an error.
| Endpoint | Nat | Java | Notes |
|-------------------------------------------------------------------------------------------------|-----|------|---------------------------------------------------------|
| `/modules`, `/context`, `/digest`, `/search/identifier`, `/search/value`, `/nodes/*`, `/source` | ✓ | ✓ | language-neutral |
| `/callers` · `/callees` · `/call-tree` | ✓ | ✓ | `?resolveInterfaces=`/`?followWiring=` Java-only |
| `/db-accesses` · `/sql-statements` | ✓ | ✓ | Natural ADABAS/SQL; Java JPA/Panache/`@Query` |
| `/db-tables/{name}/columns` | ✓ | ✓ | Natural `INTO VIEW`/PDA, or Java `@Entity` |
| `/functions` | ✓ | ✓ | `?includeInherited=true` Java-only |
| `/variables/{name}/reads` · `/writes` · `/flow-forward` · `/flow-backward` | ✓ | ✓ | flow-* deep-ingest only |
| `/workfile-accesses` | ✓ | — | `READ`/`WRITE WORK FILE`; a work file is not a DB table |
| `/dispatch-table` | ✓ | — | `DECIDE ON VALUE OF` routing |
| `/variables/{field}/field-flow` | ✓ | — | shared-PDA producer→consumer |
| `/data-structures/{name}/fields` | ✓ | — | `DEFINE DATA`/LDA/PDA |
| `/modules/{name}/columns`, `/functions/{fn}/overrides`, `/search/annotation`, `?extends=` | — | ✓ | Java-only |
## 2. Discovery & overview
```
GET /api/projects → [{ "name", "description" }]
GET /api/version → { "name": "agenticcode", "version": "<counter>" } (not project-scoped)
GET /api/projects → [{ name, description }] GET /api/version (not project-scoped)
GET /modules[?sourceFile=|?moduleKind=|?extends=] → [{ name, sourceFile, moduleKind }]
GET /modules/{name}/context → one-shot overview
GET /modules/{name}/digest → names/counts only
```
Project create/update/delete, the global clear, and ingest are **not** in
your read-only toolset.
`?extends=` = Java direct subclasses (one hop). `?moduleKind=` = Java CLASS/INTERFACE, or a
best-effort Natural PROGRAM/SUBPROGRAM guess.
## 2. Query the graph
`context` bundles `name`, `sourceFile`, `description` (leading banner/Javadoc, `null` if none),
`functions[]`, `callers`, `callees`, `dbAccesses[]`. The two heavy sections come back as summaries by
default (`sqlStatementSummary: {count, byMode, tables}`, `variableAccessSummary: {count, byFunction,
byMode}`); get full lists with `?include=sqlStatements|variableAccesses`. `?include=a,b` restricts
sections; `?limit=&offset=` paginate.
Start broad (`/modules/{name}/context`), then drill down.
`digest` — for triaging many modules before expanding:
`{ name, description, functionCount, callers: {CALLNAT: [...]}, callees: {PERFORM: [...], CALLNAT: [...]}, dbTables: [...], dataStructures: [{name, fieldCount}] }`
### Discover modules
## 3. Call graph
```
GET /modules → [{ "name", "sourceFile", "moduleKind" }]
GET /modules?sourceFile={path} → modules defined in that file
GET /modules?moduleKind={kind} → Java CLASS/INTERFACE, or best-effort Natural PROGRAM/SUBPROGRAM guess
GET /modules?extends={base} → Java: direct EXTENDS subclasses of {base} (one hop)
```
### One-shot module overview
```
GET /modules/{name}/context
```
Bundles `name`, `sourceFile`, `description` (from a leading banner/Javadoc,
`null` if none), `functions[]`, `callers`, `callees`, `dbAccesses[]` in one
call. The two potentially-heavy sections — `sqlStatements`,
`variableAccesses` — come back as compact **summaries** by default
(`sqlStatementSummary: {count, byMode, tables}`,
`variableAccessSummary: {count, byFunction, byMode}`); request the full list
with `?include=sqlStatements` / `?include=variableAccesses` (exactly one of
list/summary populated per heavy section). `?include=a,b,...` restricts to
named sections generally; `?limit=&offset=` paginate a requested full list.
**Lighter still: `/modules/{name}/digest`** — names/counts only, no line
ranges/source files/raw statements. Use when triaging many modules before
deciding which to expand:
```json
{
"name": "ZSNNA12", "description": "XML Interface for BGEAGFN", "functionCount": 7,
"callers": { "CALLNAT": ["BGEAGFN"] },
"callees": { "PERFORM": ["R-PARSE"], "CALLNAT": ["ZSNUTL"] },
"dbTables": ["MY-TABLE"],
"dataStructures": [{ "name": "#S-PARTNER", "fieldCount": 23 }]
}
```
### Call graph
```
GET /modules/{name}/callers?scope={external|internal}
GET /modules/{name}/callees?scope={external|internal}
GET /modules/{name}/callers|callees?scope={external|internal}
GET /modules/{name}/call-tree?depth=N&resolveInterfaces=&followWiring=
```
`callers`/`callees` return a dedup-sourceFile wrapper:
`{ "sourceFiles": [...], "items": [{ name, type, sourceFileIndex, edgeKind, lineNos: [...] }] }`.
`edgeKind`: `CALLNAT`/`PERFORM` (Natural), `METHOD_CALL`/`CONSTRUCTOR` (Java),
`EXTENDS`/`IMPLEMENTS` (inheritance — so `callers(Base)` also answers "who
subclasses/implements this?"), and Java-only `INJECTS` (`@Inject` field or
injection-point constructor) / `REFERENCES` (`X.class` used as an argument,
e.g. `super(SomeStep.class, ...)`). `?scope=external` = cross-module targets;
`?scope=internal` = same-module only (excludes `EXTENDS`/`IMPLEMENTS`/
`INJECTS`/`REFERENCES`).
`callers`/`callees` → `{ sourceFiles: [...], items: [{ name, type, sourceFileIndex, edgeKind, lineNos, sites }] }`.
`call-tree` → `{ sourceFiles, items: [{ name, type, sourceFileIndex, depth }] }`, `depth` default 3,
clamped 1–10. All four plus `search/identifier` accept `?fields=name` → flat `{name, type}[]`.
**`CALLNAT <var>` (dynamic dispatch, Natural):** resolved targets appear
tagged `edgeKind = CALLNAT_DYNAMIC` — intra-module (literal in same program),
intra-module-indirect (via a lookup array/variable), and cross-module (target
filled by another subprogram's output parameter — needs both dispatcher and
resolver deep-ingested). An unresolved dynamic site still appears as a
`CALLNAT_DYNAMIC` callee pointing at the `#var` name itself (with
`unresolved: true`, no `sourceFile`), so a dynamic call site stays visible even
when its target can't be determined. Treat `CALLNAT_DYNAMIC` as
**inferred/over-approximating** — a possible, not guaranteed, call — unlike
static `CALLNAT`/`PERFORM`. **When you hit an unresolved one, you are required
to investigate and pin it** — see "Resolving unresolved dynamic `CALLNAT`
calls" below.
`edgeKind`: `CALLNAT`/`PERFORM` (Natural) · `METHOD_CALL`/`CONSTRUCTOR` (Java) ·
`EXTENDS`/`IMPLEMENTS` (both — so `callers(Base)` also answers "who subclasses this?") ·
`INJECTS`/`REFERENCES` (Java DI: `@Inject` field, or `X.class` passed as an argument).
`?scope=external` = cross-module; `?scope=internal` = same-module, excluding
`EXTENDS`/`IMPLEMENTS`/`INJECTS`/`REFERENCES`.
**Empty `callers` can mean "not fully ingested," not "no callers."** A call
edge is recorded on the *caller's* side, so callers only appear if those
modules were ingested. Check `search/identifier?name=X` — `sourceFile == ""`
marks an uningested placeholder (referenced but not itself in the graph);
treat its `callers` as partial.
**`CALLNAT <var>` (dynamic dispatch).** Resolved targets appear as `edgeKind = CALLNAT_DYNAMIC`
(intra-module, intra-module-indirect via a lookup array, or cross-module — the last needs both
dispatcher and resolver deep-ingested). An unresolved site still appears, pointing at the `#var` name
with `unresolved: true` and no `sourceFile`, so the call site stays visible. Treat `CALLNAT_DYNAMIC`
as **inferred/over-approximating**, unlike static `CALLNAT`/`PERFORM`. **Every unresolved one must be
investigated and pinned — see §4.**
`call-tree` returns `{ sourceFiles, items: [{name, type, sourceFileIndex,
depth}] }`. `depth` defaults 3, clamped 1-10. Java-only flags:
`?resolveInterfaces=true` hops an interface callee to its concrete
implementation(s), dropping the dead-end interface node;
`?followWiring=true` also traverses `INJECTS`/`REFERENCES` transitively
(off by default) — also reaches wiring inherited unchanged from an `EXTENDS`
ancestor, materialized as a synthetic edge (`resolvedVia: 'INHERITANCE'`).
For such an inherited `INJECTS`/`REFERENCES` callee the `sites` `callSiteFile`
is the **base class file** where the injected field / class-literal actually
lives (with `inheritedFrom` = the base class name), not the subclass's own file
— its `lineNo` is a base-file line (item 92; before it, that line read against
the subclass file, often past its end). A `CONSTRUCTOR` callee is never
fanned out to subtypes (`new X()` binds statically to `X`); CHA
over-approximation applies only to `METHOD_CALL` on a base/interface reference.
**Empty `callers` can mean "not fully ingested".** A call edge is recorded on the *caller's* side, so
callers appear only if those modules were ingested. Check `search/identifier?name=X`: `sourceFile ==
""` marks an uningested placeholder — treat its `callers` as partial.
**Names-only mode:** `callers`, `callees`, `call-tree`, `search/identifier`
accept `?fields=name` → flat dedup `{name, type}[]`, dropping everything else.
Java `call-tree` flags: `?resolveInterfaces=true` hops an interface callee to its concrete
implementations, dropping the dead-end interface node. `?followWiring=true` traverses
`INJECTS`/`REFERENCES` transitively, including wiring inherited from an `EXTENDS` ancestor
(synthetic edge, `resolvedVia: 'INHERITANCE'`); for those the site's `callSiteFile` is the **base
class file** where the field/class-literal lives, with `inheritedFrom` naming it. A `CONSTRUCTOR`
callee is never fanned out to subtypes (`new X()` binds statically); CHA over-approximation applies
only to `METHOD_CALL` on a base/interface reference.
### Resolving unresolved dynamic `CALLNAT` calls (required)
## 4. Resolving unresolved dynamic `CALLNAT` (required write)
This is your one write responsibility (item 82). The auto-resolvers cannot
recover every `CALLNAT <var>` — e.g. a target name assembled by string ops
(`MOVE 'YGEAGKEY' TO #M` then `MOVE 'GN0' TO SUBSTR(#M,6,3)` → `YGEAGGN0`), or
one supplied by a caller the graph hasn't connected. Such a site is left as an
unresolved placeholder. **Whenever your analysis surfaces one, you must resolve
it rather than report it as a dead end.**
Auto-resolvers cannot recover every `CALLNAT <var>` — e.g. a name assembled by string ops
(`MOVE 'YGEAGKEY' TO #M` then `MOVE 'GN0' TO SUBSTR(#M,6,3)` → `YGEAGGN0`), or supplied by a caller
the graph hasn't connected. **Whenever your analysis surfaces one, resolve it rather than report a
dead end.**
```
GET /modules/{name}/callees → items with unresolved: true (variable-named CALLNAT_DYNAMIC target)
GET /dynamic-calls/unresolved → [{ module, originFile, lineNo, variable }] (every open site in the project)
GET /dynamic-calls/overrides → [{ originFile, lineNo, variable, targets, note, obsolete }]
POST /dynamic-calls/overrides → body { originFile, lineNo, targets: ["MOD", ...], variable?, note? }
DELETE /dynamic-calls/overrides?originFile=&lineNo= → reset one site (omit both query params to reset all)
GET /dynamic-calls/unresolved → [{ module, originFile, lineNo, variable }] (every open site)
GET /dynamic-calls/overrides → [{ originFile, lineNo, variable, targets, note, obsolete }]
POST /dynamic-calls/overrides → { originFile, lineNo, targets: ["MOD"], variable?, note? }
DELETE /dynamic-calls/overrides?originFile=&lineNo= (omit both params to reset all)
```
**Required workflow for each unresolved site:**
1. **Detect** — `unresolved: true` on a `CALLNAT_DYNAMIC` item, or an entry from
`/dynamic-calls/unresolved`.
2. **Investigate, don't guess** — `search/value?value=<literal>` for literals assigned to the
variable; `variables/{var}/writes?module=&depth=N` and `flow-backward` for what feeds it; read the
source around `originFile:lineNo` (assignments, `SUBSTR`, lookup tables, `DECIDE` branches).
Confirm each candidate is real via `GET /modules/{target}` or `search/identifier`.
3. **Pin** — `POST` with `originFile` + `lineNo` exactly as returned, plus `targets`. Pass several
when the dispatch genuinely branches. `400 UNKNOWN_TARGET` means your investigation was wrong —
recheck, don't invent a name.
4. **Verify** — re-GET `callees`: the `#var` placeholder is gone, your targets appear resolved. The
override persists and is re-applied across refreshes.
1. **Detect.** A `callees`/`context` result with `unresolved: true` on a
`CALLNAT_DYNAMIC` item, or an entry from `GET /dynamic-calls/unresolved`.
2. **Investigate — do not guess.** Read the call site and the dispatch
variable's origin to determine the real target module(s). Use
`search/value?value=<literal>` to find literals assigned to the variable,
`variables/{var}/writes?module=&depth=N` and `flow-backward` to trace what
feeds it, and read the source around `originFile:lineNo` (assignments,
`SUBSTR`, lookup tables, `DECIDE`/dispatch-table branches). Confirm each
candidate is a real module (`GET /modules/{target}` / `search/identifier`).
3. **Pin it.** `POST /dynamic-calls/overrides` with `originFile` + `lineNo`
(exactly as returned by `/dynamic-calls/unresolved`) and the `targets` you
established. Pass **several** targets when the dispatch genuinely branches to
more than one module. A target that is not a real module is rejected
`400 UNKNOWN_TARGET` — that means your investigation was wrong; recheck,
don't invent a name.
4. **Verify.** Re-GET `callees` — the `#var` placeholder is gone and your
target(s) now appear as resolved `CALLNAT_DYNAMIC` callees. The override is
persisted and re-applied automatically across refreshes; `DELETE` it only if
you later find the target was wrong.
Only pin what you have evidence for. If the source genuinely does not determine the target (e.g. the
name arrives from external input), say so and leave it unresolved.
Only pin what you have evidence for. If the source genuinely does not determine
the target (e.g. the name arrives from external input), say so and leave it
unresolved rather than guessing.
## 5. Database, work files, data structures
### Functions & overrides (Java)
```
GET /modules/{name}/db-accesses?depth=N → [{ name, mode: READS|WRITES|DECLARES, lineNos, via, sites }]
GET /modules/{name}/sql-statements?depth=N → [{ table, view, mode, statement, startLine, endLine, via, sourceFile, viaCopycode }]
GET /modules/{name}/workfile-accesses → [{ workFile, physicalName, mode, recordBuffers, lineNos, sites }]
GET /db-tables/{name}/columns → [{ name, type, dataType, value, parent, startLine, endLine }]
GET /modules/{name}/columns → Java @Entity columns, incl. @MappedSuperclass-inherited
GET /data-structures/{name}/fields → [{ name, type, dataType, value, parent, startLine, endLine, scope }]
GET /modules/{name}/data-structures → [{ name, relationship: USING|INLINE, area: PDA|LDA|GDA|INLINE|UNKNOWN, fieldCount, sourceFile }]
```
**Copycode provenance.** `sites: [{ lineNo, sourceFile, viaCopycode, includedAt }]` ties each access
to the file it really lives in. Where `viaCopycode` is set, `lineNo`/`startLine` are lines **in the
`.cpy`**, not in the module file — a bare `lineNos` number can point into an INCLUDEd copycode.
**`?depth=N` (1–10)** adds accesses reached transitively through `CALLNAT`/`PERFORM`/`CALLS`, tagging
`via` with the intermediate module. **Always pass `?depth` for Natural** — DB logic frequently hides
behind a `CALLNAT`. Default is the module's own accesses only (`via = null`).
**Java resolution.** Repository calls, `EntityManager`, Panache active-record, and Spring-Data
`@Query` (JPQL or native) resolve to `READS`/`WRITES` on the entity's table. Verb→mode is heuristic
for method names (`save/persist/merge/update/create`→WRITE; `delete*/remove*`→WRITE, shown as
`DELETE` in `sql-statements`; `find*/get*/list*/count*`→READ); for `@Query` the verb is parsed from
the text. A repository/entity's **own** table appears as `mode: DECLARES` even with no caller, and
survives `?depth=`. `@Query` text lives at the method declaration, so it is visible directly on the
repository but only at `?depth=1+` to a caller. **An empty Java `db-accesses` means "no recognized
persistence call", not "touches no table"** — parsing is regex-based (leading verb + first
`FROM`/`UPDATE`/`INTO`), so joins, subqueries and unusual quoting may not resolve.
`/modules/{name}/columns` → `{ attributeName, columnName, columnDefinition, javaType, nullable,
converterType, hibernateType, isId, declaredIn, startLine, endLine }`.
`data-structures/{name}/fields` is the flattened schema of a canonical `DEFINE DATA`/DDM structure
(`dataType` = Natural format/length, e.g. `A8`; `scope` = `PARAMETER`/`LOCAL`/`GLOBAL`/`INDEPENDENT`,
`null` for DB columns). Scoped to the structure's own definition — a copybook shared by many programs
returns its fields once, not once per `USING`. `UNKNOWN` area / empty fields ⇒ the defining file was
not ingested. `/modules/{name}/data-structures` shows which copybooks a module pulls in; feed each
`name` into the first endpoint.
## 6. Variables & dataflow
```
GET /variables/{name}/reads|writes?module=&depth=N → [{ function, functionType, sourceFile, module, lineNo }]
GET /variables/{name}/flow-forward|flow-backward?module=&depth=N → [{ variable, variableType, module, depth }]
GET /variables/{field}/field-flow?module=&depth=N → [{ field, producer, producedAt, consumer, consumedAt }]
```
`reads`/`writes` cover Natural `VARIABLE`/`CONSTANT` and Java `FIELD`; `module` scopes the start
point, `depth` traverses the call graph. `flow-*` follow positional argument→parameter links across
`CALLNAT` or Java method calls (matched by callee name — overloads over-approximate) and are
**deep-ingest only** (`409` with a `nextAction`; out of your scope — report it). `field-flow`
(Natural-only) pairs producers (`WRITES` a shared PDA field) with downstream consumers (`READS` the
same node) reachable via `CALLS`; it is reachability-based, **not order-precise** — it confirms a
downstream read exists, not that the write precedes it on every path.
## 7. Functions, overrides, dispatch table
```
GET /modules/{name}/functions?includeInherited=&kind={abstract|final|overridable}
→ [{ name, declaredIn, sourceFile, viaCopycode, startLine, endLine, kind }]
# viaCopycode=true ⇒ Natural subroutine from an INCLUDEd copycode; start/end lines are in sourceFile (the .cpy), not the module file
GET /modules/{name}/functions/{fn}/overrides → one hook's subclass overrides
GET /modules/{name}/functions/overrides → bulk: every abstract-method's overrides at once
GET /modules/{name}/functions/{fn}/overrides → one hook's subclass overrides
GET /modules/{name}/functions/overrides → bulk: every abstract method's overrides (adds `method`)
GET /modules/{name}/dispatch-table → [{ guardField, guardValue, assignedField, assignedValue, lineNo }]
```
`functions` → `[{ name, declaredIn, startLine, endLine, kind }]`
(`kind` null for Natural subroutines/constructors). `includeInherited=true`
also returns ancestor methods, each tagged `declaredIn`. `kind=` filters by
Java modifier (source-derived, not heuristic). `.../overrides` →
`[{ module, name, sourceFile, startLine, endLine }]` (bulk variant adds
`method` naming which hook is overridden) — use to see every concrete
implementation of a template-method contract at once.
`kind` is null for Natural subroutines and Java constructors; `kind=` filters by Java modifier
(source-derived). `includeInherited=true` adds ancestor methods, each tagged `declaredIn`.
`viaCopycode=true` ⇒ Natural subroutine from an INCLUDEd copycode, so its lines are in `sourceFile`
(the `.cpy`). `.../overrides` → `[{ module, name, sourceFile, startLine, endLine }]`.
### Dynamic-dispatch routing table (Natural)
`dispatch-table` gives a `DECIDE ON VALUE OF` router's `guardValue → assignedValue` table without
reading the block; one `guardValue` may map to several programs. Requires the router deep-ingested;
empty if no value-dispatch `DECIDE` exists.
## 8. Search & node inspection
```
GET /modules/{name}/dispatch-table
GET /search/identifier?name=&type= → [{ id, type, name, sourceFile, startLine, endLine, dataType, value, scope }]
GET /search/value?value=&contains= → [{ kind: ASSIGNMENT|NODE, name, value, module, sourceFile, startLine, endLine }]
GET /search/annotation?name=&type= → [{ id, type, name, sourceFile, startLine, endLine, annotations }] (Java-only)
GET /nodes/{id} → every property of the node (not a curated DTO)
GET /nodes/{id}/source | /modules/{name}/source?startLine=&endLine= → { sourceFile, startLine, endLine, lines }
```
For a `DECIDE ON VALUE OF` dispatcher → rows of
`{ guardField, guardValue, assignedField, assignedValue, lineNo }`, i.e. the
`guardValue → assignedValue` routing table, without reading the `DECIDE`
block. One `guardValue` may map to several programs. Requires the router to
be deep-ingested; empty if no value-dispatch `DECIDE` exists.
`search/identifier`: omit `name` to list all (large). `type` ∈ `MODULE`, `FUNCTION`, `VARIABLE`,
`CONSTANT`, `DATA_STRUCTURE`, `DB_TABLE`, `FIELD`, `DB_ACCESS`, `CONTROL_FLOW`.
### Database access
`search/value` finds a literal that never became its own node — e.g. a program name assigned to a
field (`#P-CALLED-PROG := 'WGEAGB0S'`). `ASSIGNMENT` = a write of that literal, `NODE` = a node
carrying it as its value. Exact and quote-insensitive by default; `?contains=true` →
case-insensitive substring, needed when the value is embedded in longer text such as SQL.
```
GET /modules/{name}/db-accesses?depth=N → [{ name, mode: READS|WRITES|DECLARES, lineNos, via, sites }]
# sites: [{ lineNo, sourceFile, viaCopycode, includedAt }] — each access tied to the file it lives in (the .cpy for a copycode-sourced access); a bare lineNos number can point into an INCLUDEd copycode
GET /modules/{name}/sql-statements?depth=N → [{ table, view, mode, statement, startLine, endLine, via, sourceFile, viaCopycode }]
# viaCopycode=true ⇒ statement from an INCLUDEd copycode; startLine/endLine are lines in sourceFile (the .cpy), not the module file
GET /modules/{name}/workfile-accesses → [{ workFile, physicalName, mode: READS|WRITES, recordBuffers, lineNos, sites }] (Natural READ/WRITE WORK FILE)
# sites like db-accesses: copycode-aware file context per access
GET /db-tables/{name}/columns → [{ name, type, dataType, value, parent, startLine, endLine }]
GET /modules/{name}/columns → Java @Entity columns (incl. @MappedSuperclass-inherited)
```
`search/annotation` is the only search that sees annotations; `name` matches case-insensitive
substring, `annotations` lists every annotation on the node.
By default: the module's **own** accesses only (`via = null`). `?depth=N`
(1-10) also includes accesses reached transitively through
`CALLNAT`/`PERFORM`/`CALLS`, tagging `via` with the intermediate module —
**always pass `?depth` for Natural**, since DB logic frequently hides behind
a `CALLNAT`.
`nodes/{id}` — use when a targeted endpoint doesn't expose what you need. The id is Neo4j's
`elementId` and **survives a re-ingest** (a merged node keeps its id); it becomes invalid only if the
node is deleted, which a refresh does to nodes the fresh parse no longer produces.
**Java resolution:** repository calls (`orderRepository.persist/findById/...`),
`EntityManager` (`em.persist/merge/remove`), Panache active-record
(`Product.findById`), and Spring-Data `@Query` (JPQL or native) all resolve
to `READS`/`WRITES` on the entity's table. Verb→mode is heuristic for method
names (`save/persist/merge/update/create`→WRITE, `delete*/remove*`→WRITE
shown as `DELETE` in `sql-statements`, `find*/get*/list*/count*`→READ); for
`@Query` the verb is parsed from the JPQL/SQL text itself. A repository/
entity's **own** table also surfaces with `mode: "DECLARES"` even with no
caller anywhere (its `@Entity(name=)`/resolved generic entity mapping), and
that row survives `?depth=` — the transitive view is a superset of the direct
one, so a caller sees the tables declared by the entities/repositories in its
closure, each with `via` naming the declaring module.
Because `@Query` text lives only at the method declaration, that access is
visible directly on the repository, and to a caller only via `?depth=1+`
(unlike a direct repository call, visible to its caller at depth 0). An
empty Java `db-accesses` means "no recognized persistence call," not
necessarily "touches no table" — JPQL/SQL parsing is regex-based
(leading verb + first `FROM`/`UPDATE`/`INTO` target); joins, subqueries, and
unusual quoting may not resolve.
## 9. Playbook
`/modules/{name}/columns` → `{ attributeName, columnName, columnDefinition,
javaType, nullable, converterType, hibernateType, isId, declaredIn,
startLine, endLine }` per column.
**Flow:** projects → `/modules` → `context` (or `digest` to triage many) → `call-tree?depth=N` to
scope the feature → per structure/table `data-structures/{name}/fields`, `db-tables/{name}/columns`,
or `modules/{Entity}/columns` → for impact `variables/{name}/reads|writes`, `search/identifier`,
`flow-forward|flow-backward`.
### Variables & dataflow
**Natural** fans out through many small modules and hides DB logic behind `CALLNAT` — **always pass
`?depth`** on db-accesses/sql-statements/variable reads|writes. `callees?scope=external` = CALLNAT'd
subprograms, `?scope=internal` = PERFORM, `callers` = blast radius. Trace values with
`variables/{field}/field-flow?depth=3`.
```
GET /variables/{name}/reads?module=&depth=N → [{ function, functionType, sourceFile, module, lineNo }]
GET /variables/{name}/writes?module=&depth=N
GET /variables/{name}/flow-forward?module=&depth=N → [{ variable, variableType, module, depth }]
GET /variables/{name}/flow-backward?module=&depth=N
GET /variables/{field}/field-flow?module=&depth=N → [{ field, producer, producedAt, consumer, consumedAt }]
```
*Example — "what does `WGEAGB0S` do, and what would reengineering it take?"*: `context` (sees
`CALLNAT BGEAGFN0`) → `call-tree?depth=4` → `db-accesses?depth=4` (`VERSVW_ADDRESS` READ/WRITE `via
BGEAGFN0`) → `sql-statements?depth=4` (statement to port) → `db-tables/VERSVW_ADDRESS/columns` +
`data-structures/{VIEW}/fields` (entity to generate) → `variables/%23KEY/field-flow?depth=4` (key
origin) → `callers` (dependents).
`reads`/`writes` cover Natural `VARIABLE`/`CONSTANT` and Java `FIELD`;
`module` scopes the start point, `depth` traverses the call graph
(default/clamp as `call-tree`). Use when `context.variableAccesses` shows a
field written here and you need every downstream reader before changing its
type.
**Java**: the call graph spans files and entities carry the schema in annotations. Use
`functions?includeInherited=true` for the member surface;
`call-tree?depth=N&resolveInterfaces=true&followWiring=true` for a whole feature in one traversal;
`db-accesses?depth=N` on the repository/service, then `modules/{Entity}/columns`;
`functions/{fn}/overrides` for concrete implementations; `search/annotation` project-wide (every
`@Query`, lingering `@Deprecated`).
`flow-forward`/`flow-backward` follow positional argument→parameter links
across `CALLNAT` (Natural) or method calls (Java, both intra- and cross-class,
matched by callee method name — overloads over-approximate).
**Deep-ingest only** — a call-graph-only module returns `409
NOT_DEEPLY_INGESTED`/`409 NOT_INGESTED` with a `nextAction` (out of your
scope — report it).
*Example — "map the `OrderController` feature and its persistence"*: `context` →
`callees?scope=external` → `call-tree?depth=3` → `db-accesses?depth=3` on the repository →
`modules/{Entity}/columns` → `functions?includeInherited=true`.
`field-flow` (Natural-only) traces a **shared PDA field**: producer modules
(`WRITES`) paired with downstream consumers (`READS` the same shared node)
reachable via `CALLS` up to `depth` hops. Reachability-based, not
order-precise (confirms a downstream read exists, not that the write
precedes it on every path).
## curl
### Data structures
```
GET /data-structures/{name}/fields → [{ name, type, dataType, value, parent, startLine, endLine, scope }]
GET /modules/{name}/data-structures → [{ name, relationship: USING|INLINE, area: PDA|LDA|GDA|INLINE|UNKNOWN, fieldCount, sourceFile }]
```
First: flattened field schema of a canonical `DEFINE DATA`/DDM structure
(`dataType` = Natural format/length e.g. `A8`/`I4`; `value` for constants;
`scope` = `PARAMETER`/`LOCAL`/`GLOBAL`/`INDEPENDENT`, `null` for DB-table
columns). Scoped to the structure's own definition — a copybook shared by
many programs returns its fields once, not once per `USING` site. `UNKNOWN`
area / empty fields means the defining file wasn't ingested.
Second: which copybooks/inline groups a module's `DEFINE DATA` pulls in —
discover the interface without reading source; feed each `name` (especially
`USING`/`PDA` ones) into the first endpoint for its field schema.
### Cross-project search
```
GET /search/identifier?name=&type= → by node name
GET /search/value?value=&contains= → by literal value (quote-insensitive)
GET /search/annotation?name=&type= → Java-only, by annotation
```
`search/identifier` → `[{ id, type, name, sourceFile, startLine, endLine,
dataType, value, scope }]` per matching node; omit `name` to list all
(large). `type` filters by `NodeType` (`MODULE`, `FUNCTION`, `VARIABLE`,
`CONSTANT`, `DATA_STRUCTURE`, `DB_TABLE`, `FIELD`, `DB_ACCESS`,
`CONTROL_FLOW`) — bad value → `400 INVALID_TYPE`. `id` feeds `/nodes/{id}`.
`search/value` finds a literal that never became its own node — e.g. a
program name assigned to a field (`#P-CALLED-PROG := 'WGEAGB0S'`). →
`[{ kind: ASSIGNMENT|NODE, name, value, module, sourceFile, startLine,
endLine }]` (`ASSIGNMENT` = a write of that literal; `NODE` = a node, e.g. a
`CONSTANT`, carrying it as its value). Exact + quote-insensitive by default;
`?contains=true` → case-insensitive substring (needed when the value is
embedded in a longer string, e.g. a table name inside SQL statement text).
Missing `value` → `400 MISSING_VALUE`.
`search/annotation` finds classes/methods/constructors/fields carrying a
matching annotation (`@Query`, `@Entity`, `@Inject`, ...) — the other two
searches can't see annotations at all. → `[{ id, type, name, sourceFile,
startLine, endLine, annotations }]` (`annotations` = every annotation on that
node, comma-joined). `name` matches case-insensitive substring; missing/blank
→ `400 MISSING_NAME`; bad `type` → `400 INVALID_TYPE`.
### Inspect a node / read its source
```
GET /nodes/{id}
GET /nodes/{id}/source
GET /modules/{name}/source?startLine=&endLine=
```
`nodes/{id}` returns **every** property of that node (not a curated DTO) —
use when a targeted endpoint doesn't expose what you need. The id is Neo4j's
`elementId`, and it **survives a re-ingest**: a node that a refresh merges onto
keeps its id. (It used to be a UUID that was overwritten on every single
re-ingest, so ids were only valid within one generation — that restriction is
gone.) An id does become invalid if the node itself is deleted, which a refresh
does to nodes the fresh parse no longer produces, or if the database is
restored from a backup.
`/source` variants read `[startLine, endLine]` off disk →
`{ sourceFile, startLine, endLine, lines: [...] }`. Missing
`startLine`/`endLine` on the module variant → `400 MISSING_LINE_RANGE`;
unknown id/module → `404 NODE_NOT_FOUND`/`404 MODULE_NOT_FOUND`; a
placeholder with no source file → `400 NO_SOURCE_FILE`. **Skip these two
if you have filesystem access** — see Ground rules.
## curl reference
All paths are `http://localhost:8787/api/projects/{project}/…`; quote URLs containing `&`, and encode
`#` as `%23`.
```bash
curl http://localhost:8787/api/projects
curl http://localhost:8787/api/version
curl http://localhost:8787/api/projects/demo/modules
curl 'http://localhost:8787/api/projects/demo/modules?sourceFile=ZSNNA12.nsp'
curl http://localhost:8787/api/projects/demo/modules/ZSNNA12/digest
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/context?include=functions,dbAccesses&limit=50'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/callers?scope=external'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/callees?scope=internal'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/call-tree?depth=2&resolveInterfaces=true&followWiring=true'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/functions?includeInherited=true'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/functions/R-PARSE/overrides'
curl http://localhost:8787/api/projects/demo/modules/ZSNNA12/dispatch-table
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/call-tree?depth=2&resolveInterfaces=true'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/db-accesses?depth=2'
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/sql-statements?depth=2'
curl http://localhost:8787/api/projects/demo/db-tables/MY-TABLE/columns
curl http://localhost:8787/api/projects/demo/modules/PartnerLegacyEntity/columns
curl 'http://localhost:8787/api/projects/demo/variables/%23FIELD/reads?module=ZSNNA12&depth=3'
curl 'http://localhost:8787/api/projects/demo/variables/%23FIELD/flow-forward?module=ZSNNA12&depth=3'
curl 'http://localhost:8787/api/projects/demo/variables/SHARED-FIELD/field-flow?depth=3'
curl http://localhost:8787/api/projects/demo/data-structures/MY-VIEW/fields
curl 'http://localhost:8787/api/projects/demo/search/identifier?name=MY-FIELD'
curl 'http://localhost:8787/api/projects/demo/search/value?value=orders&contains=true'
curl 'http://localhost:8787/api/projects/demo/search/annotation?name=Query&type=FUNCTION'
curl http://localhost:8787/api/projects/demo/nodes/3f9c1a2b-.../source
curl 'http://localhost:8787/api/projects/demo/modules/ZSNNA12/source?startLine=10&endLine=25'
curl -X POST -H 'Content-Type: application/json' \
-d '{"originFile":"X.nat","lineNo":42,"targets":["YGEAGGN0"]}' \
http://localhost:8787/api/projects/demo/dynamic-calls/overrides
```
## Recommended end-to-end flow
1. `GET /api/projects` → pick the project.
2. `GET /modules` (optionally `?sourceFile=`) → find the module of interest.
3. `GET /modules/{name}/context` (or `/digest` for quick triage across many).
4. `GET /modules/{name}/call-tree?depth=N` → scope the surrounding feature.
5. Per referenced structure/table: `/data-structures/{name}/fields`,
`/db-tables/{name}/columns`, or `/modules/{name}/columns` (Java entity).
6. For impact analysis: `/variables/{name}/reads|writes`,
`/search/identifier`, `/variables/{name}/flow-forward|flow-backward`.
7. For Java hierarchies: `/modules/{name}/functions?includeInherited=true`.
## Natural playbook
Natural fans out through `CALLNAT`/`PERFORM` across many small modules, and
DB logic frequently hides behind a `CALLNAT` — **always pass `?depth` on
db-accesses/sql-statements/variable reads|writes.**
1. Orient: `GET /modules` / `?sourceFile=`. Encode `#` in field names
(`%23COUNTER`); names are case-sensitive.
2. Overview: `GET /modules/{name}/context`.
3. Scope: `call-tree?depth=3`, `callees?scope=external` (CALLNAT'd
subprograms) vs `?scope=internal` (PERFORM), `callers` (blast radius).
4. DB footprint: `db-accesses?depth=3` / `sql-statements?depth=3` — `via`
names the callee doing the actual access.
5. Data shapes: `db-tables/{TABLE}/columns` + `data-structures/{NAME}/fields`.
6. Trace values: `variables/{field}/writes|reads?depth=3`,
`variables/{field}/field-flow?depth=3` (shared-PDA producer→consumer),
`search/identifier?name={X}` (cross-module occurrences).
Worked example — *"what does `WGEAGB0S` do, and what would reengineering it
take?"*: `context` (sees `CALLNAT BGEAGFN0`) → `call-tree?depth=4` → `db-accesses?depth=4`
(`VERSVW_ADDRESS` READ/WRITE `via BGEAGFN0`) → `sql-statements?depth=4` (the
statement to port to Panache) → `db-tables/VERSVW_ADDRESS/columns` +
`data-structures/{VIEW}/fields` (entity to generate) →
`variables/%23KEY/field-flow?depth=4` (key origin) → `callers` (dependents).
## Java playbook
The call graph spans files (typed-receiver/static/`new` resolve cross-class);
entities carry the DB schema in annotations.
1. Orient: `GET /modules` / `?sourceFile=`; JPA tables also via
`search/identifier?type=DB_TABLE`.
2. Overview: `GET /modules/{name}/context` (functions = methods +
constructors; `variableAccesses` = field reads/writes).
3. Members incl. inheritance: `functions?includeInherited=true`.
4. Cross-class graph: `callees?scope=external` (other classes + `INJECTS`/
`REFERENCES` wiring), `?scope=internal`, `callers`,
`call-tree?depth=N&resolveInterfaces=true&followWiring=true` (whole
feature across files in one traversal).
5. Persistence: `db-accesses|sql-statements?depth=N` on the
repository/service; `modules/{Entity}/columns` for the full column
mapping, or `db-tables/{TABLE}/columns` table-centric.
6. Dataflow: `variables/{field}/reads|writes`;
`flow-forward|flow-backward` (named-method matching, deep-ingest only).
7. `functions/{fn}/overrides` for concrete subclass overrides;
`search/annotation?name=&type=` project-wide (every `@Query` method,
lingering `@Deprecated`, etc.).
Worked example — *"map the `OrderController` feature and its persistence"*:
`context` → `callees/OrderController?scope=external` (`OrderService`, …) →
`call-tree?depth=3` (reaches `OrderService`, repositories) →
`db-accesses?depth=3` on the repository/service → per entity
`modules/{Entity}/columns` → `functions/{Service}?includeInherited=true`
(API surface to reimplement).

12
x-docs/agent-api-usage-ac-implementation.md
View File

@@ -285,6 +285,18 @@ under-reports Natural callers/callees, and does so silently — re-ingest before
module is unused.** A copycode parameter that genuinely has no argument now surfaces in
`/dynamic-calls/unresolved` as `&n&` rather than being dropped, so "not analysable" is visible.
**A call edge no longer outlives the call it was parsed from (item 124).** Until 2026-08-09 a refresh
only *added* the corrected call and left the old one in place, because an edge is reaped only when one
of its endpoints is — and a parser fix changes neither (the calling subroutine is unchanged, the old
target is a never-swept placeholder). So `callees`/`callers` could report a call that no source line
makes, flagged `unresolved: true` and indistinguishable from a genuine unresolved dynamic call. A
re-parsed Natural file's call edges are now reaped before the fresh ones are merged, and a call-target
placeholder left with no callers is deleted. **Two consequences for a graph ingested before
2026-08-09:** an `unresolved: true` callee may be an artefact of an already-fixed parser bug rather
than a real dynamic call, and `search/identifier` may list module names that exist nowhere in the
source. Both clear on the next deep refresh. Note the reap deliberately spares `CALLNAT_DYNAMIC` edges
onto *real* modules — those are the dynamic-call resolvers' output, not the parser's.
## LoC / SLoC metrics (item 46)
Every file-level node (a `MODULE` program/class, or a `DATA_STRUCTURE` for a Natural `.lda`/`.pda`

50
x-docs/roadmap.md
View File

@@ -823,6 +823,13 @@ before. The parked "parallel parse phase" idea was implemented 2026-07-18 (item
Neo4j's variable-length patterns use trail semantics (no relationship repeats in a path), so queries
terminate rather than hang — but the blow-up is real: three probes using an unbounded `CONTAINS*` over
this region were killed at a 2-minute timeout during the item-74 investigation.
**The shared-node identity below has a second consequence — see item 124.** Because a copycode node is
MERGEd per `(type, name, sourceFile)` and thus shared by every includer, item 124's call-edge reap
cannot touch the 605 call edges whose source subroutine lives in a `.cpy` (14 of them stale): reaping
them during one module's refresh would delete edges other modules contributed. Items 86 and 106 have
the same hole. Whoever gives copycode-resident nodes a per-module identity closes all three at once —
worth knowing before designing a fix for either symptom alone.
**Candidate, unconfirmed:** copycode `CONTROL_FLOW` nodes are shared by every including module (one node
per `.cpy` line), and a `.cpy` that opens a block it does not close (`YFRAMBC0.cpy` opens `FOR` at line
65; the `END-FOR` lives in the *includer*) gets containment edges from every includer's nesting context
@@ -1668,8 +1675,9 @@ reproduced the bug.)*
and by `aCallnatInsideAStringLiteralIsStillNotACall`, which keeps the widened delimiter from
re-opening bug #63.
- [ ] **124. A full deep refresh does not reap placeholder `MODULE` nodes the parser no longer
produces — fixed bugs keep answering from the graph** (found 2026-08-09 while verifying 120/121/123)
- [x] **124. A full deep refresh does not reap placeholder `MODULE` nodes the parser no longer
produces — fixed bugs keep answering from the graph** (found 2026-08-09 while verifying 120/121/123,
done 2026-08-09)
**Symptom.** After the deep refresh that shipped items 120/121/123, `DAGCHEN0/callees` still lists
the item-121 phantom `AGNT-CHG-CMP-SP` as a `MODULE` — alongside the now-correct `YAGCHBN0`, from
@@ -1702,6 +1710,44 @@ reproduced the bug.)*
contains`, the item-62 data-literal cleanup); none of them appears to drop a placeholder `MODULE`
that no longer has any producing call site in the re-parsed source.
**Resolution (2026-08-09).** The gap was structural and had a precedent: `GraphRepository` already
reaps a re-parsed Natural file's `READS`/`WRITES` to access placeholders (item 86) and its `USING`
edges (item 106) before merging the fresh ones. Nobody had done it for `CALLS`. Item 86's own
javadoc states the general cause — *"the target placeholder is never swept and the source node
survives, so neither `DELETE_STALE_FILE_NODES` nor the edge MERGE ever reaped it"*.
Two parts: `DELETE_STALE_NATURAL_CALL_EDGES` (per-file, gated on `reconcile`) and
`DELETE_ORPHANED_PLACEHOLDER_MODULES` (finalize, the `MODULE` counterpart of item 88's table sweep).
**Two boundaries the tests forced, both found by running rather than reasoning:**
1. *Duplicate markers must survive the node sweep.* Item 114 records a duplicate identity as a
placeholder, and an **unreferenced** one is a degree-0 placeholder — exactly the shape the sweep
targets. `MARK_DUPLICATE_IDENTITIES` runs *before* finalize, so the sweep deleted the marker just
written and the endpoints fell back to `404` instead of `409 DUPLICATE_IDENTITY`. Guarded on
`duplicatePaths IS NULL`. Caught by `DuplicateIdentityIT` (3 failures).
2. *Resolver-built dynamic edges must not be reaped.* The proposal said "reap **all** of the file's
`CALLS`". That is wrong: **486** `CALLNAT_DYNAMIC` edges point at real modules and **392 of them
carry no marker at all** (no `folded`, no `resolvedBy`), so nothing but the *target's* file
distinguishes enrichment output from parser output. Reaping them broke the item-37a path-warm —
a *scoped* finalize does not reliably re-resolve an edge whose far side is outside its scope, so
the deletion was permanent and a dataflow trace stopped at the dispatch boundary. The predicate
is therefore `t.sourceFile = "" OR r.callKind <> 'CALLNAT_DYNAMIC'`: everything the parser
re-emits is reaped, the resolvers' own output is not. Caught by
`AnalysisResourceIT#flowForwardPathWarmCrossesIntoDynamicallyDispatchedCallee`.
**Scope limit (deliberate).** Keyed on the source node's file, so it misses the 605 call edges whose
source subroutine is defined *inside a copycode* — 14 of them stale. Those nodes are MERGEd per
`(type, name, sourceFile)` and are therefore **shared** by every including module, so reaping them
during one module's refresh would delete edges other modules contributed and never re-create them.
That needs a per-module identity for copycode-resident nodes and is a separate item. The same hole
exists in items 86 and 106, which use the same key.
**Guard.** `StaleCallEdgeReapIT` — three cases, two sabotage-verified. The fixture changes only the
`CALLNAT` target and keeps the enclosing subroutine, which is precisely why the bug hid behind a
green suite: `DerivedCallsModuleRefreshIT` deletes the whole subroutine, so there the `FUNCTION`
node disappears and `DETACH DELETE` takes the edge along.
## Agent API / MCP tooling gaps
*(Done items 52, 53, 54, 56 moved to `x-docs/features.md`.)*