diff --git a/.gitignore b/.gitignore index 75efa76..efcf1c7 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,5 @@ buildNumber.properties hs_err_pid* replay_pid* +/.mvn/ +/.idea/ diff --git a/.idea/.gitignore b/.idea/.gitignore deleted file mode 100644 index 30cf57e..0000000 --- a/.idea/.gitignore +++ /dev/null @@ -1,10 +0,0 @@ -# Default ignored files -/shelf/ -/workspace.xml -# Editor-based HTTP Client requests -/httpRequests/ -# Ignored default folder with query files -/queries/ -# Datasource local storage ignored files -/dataSources/ -/dataSources.local.xml diff --git a/.idea/GrepConsole.xml b/.idea/GrepConsole.xml deleted file mode 100644 index 8b6e8e7..0000000 --- a/.idea/GrepConsole.xml +++ /dev/null @@ -1,44 +0,0 @@ - - - - - - \ No newline at end of file diff --git a/.idea/QuarkusDeploymentProjectService.xml b/.idea/QuarkusDeploymentProjectService.xml deleted file mode 100644 index 753e3e6..0000000 --- a/.idea/QuarkusDeploymentProjectService.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - \ No newline at end of file diff --git a/.idea/checkstyle-idea.xml b/.idea/checkstyle-idea.xml deleted file mode 100644 index 48a0c7d..0000000 --- a/.idea/checkstyle-idea.xml +++ /dev/null @@ -1,16 +0,0 @@ - - - - 13.5.0 - JavaOnly - true - - - \ No newline at end of file diff --git a/.idea/compiler.xml b/.idea/compiler.xml deleted file mode 100644 index bd44938..0000000 --- a/.idea/compiler.xml +++ /dev/null @@ -1,69 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/.idea/encodings.xml b/.idea/encodings.xml deleted file mode 100644 index c2df5a8..0000000 --- a/.idea/encodings.xml +++ /dev/null @@ -1,23 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file diff --git a/.idea/inspectionProfiles/Project_Default.xml b/.idea/inspectionProfiles/Project_Default.xml deleted file mode 100644 index f684cf7..0000000 --- a/.idea/inspectionProfiles/Project_Default.xml +++ /dev/null @@ -1,31 +0,0 @@ - - - - \ No newline at end of file diff --git a/.idea/jarRepositories.xml b/.idea/jarRepositories.xml deleted file mode 100644 index f493510..0000000 --- a/.idea/jarRepositories.xml +++ /dev/null @@ -1,25 +0,0 @@ - - - - - - - - - - - - - \ No newline at end of file diff --git a/.idea/misc.xml b/.idea/misc.xml deleted file mode 100644 index 5ef94c0..0000000 --- a/.idea/misc.xml +++ /dev/null @@ -1,16 +0,0 @@ - - - - - - - - - - - - \ No newline at end of file diff --git a/.idea/sonarlint.xml b/.idea/sonarlint.xml deleted file mode 100644 index 574b89c..0000000 --- a/.idea/sonarlint.xml +++ /dev/null @@ -1,10 +0,0 @@ - - - - - - \ No newline at end of file diff --git a/.idea/vcs.xml b/.idea/vcs.xml deleted file mode 100644 index 35eb1dd..0000000 --- a/.idea/vcs.xml +++ /dev/null @@ -1,6 +0,0 @@ - - - - - - \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md index 3144dea..55968ee 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -25,6 +25,17 @@ `x-docs/agent-module-analysis.md` to reflect it (new/changed endpoints, response fields, semantics) — treat this doc update as part of the feature's Definition of Done, not an optional follow-up. - Never add a `Co-Authored-By:` line (or any AI-attribution trailer) to commit messages. +- **Dogfooding: use AgenticCode itself for static analysis of this repo whenever possible.** The + server runs at `http://localhost:8787` with this repo already ingested as project `ac` (CLI: `ac + `, e.g. `ac callers`, `ac callees`, `ac call-tree`, `ac context`, `ac db-accesses`; REST: + `GET /api/projects/ac/modules/{name}/...`; MCP tools are also available). Prefer it over grep/Explore + for call graphs, callers/callees, DB access, dataflow, and module overviews — it's exactly the tool + this project builds, so using it here is both faster and the best test of its own output. Re-ingest + after code changes (`ac ingest all` or `POST /api/projects/ac/ingest-all?deep=true`) before trusting + query results. **If the server is unavailable** (`http://localhost:8787` unreachable), try starting + it first with `./deploy.sh up` before falling back. **When it still can't answer the question** (info + the API doesn't expose, or the analysis needs exact source text/comments/formatting) fall back to + normal code analysis (Read/Grep/Explore agent) exactly as you would on any other codebase. ## 2. Mandatory 4-Phase Workflow @@ -99,6 +110,11 @@ agenticcode/ ## Build & Run +Maven settings: `.mvn/maven.config` (gitignored, machine-local) points `-s` at +`~/.m2/settings_my.xml` — the working repository configuration, since the default Maven settings +point at an internal artifactory not reachable from this environment. Plain `mvn ...` picks it up +automatically; no need to pass `-s` explicitly. + ```bash # Full build mvn clean install diff --git a/ac-code-server/src/main/docker/Dockerfile.jvm b/ac-code-server/src/main/docker/Dockerfile.jvm new file mode 100644 index 0000000..ac9a582 --- /dev/null +++ b/ac-code-server/src/main/docker/Dockerfile.jvm @@ -0,0 +1,16 @@ +# Quarkus JVM (fast-jar) runtime image. +# Expects `mvn package` to have already produced target/quarkus-app/ on the host +# (build context is the ac-code-server module directory). +FROM eclipse-temurin:21-jre + +WORKDIR /work/ + +COPY target/quarkus-app/lib/ /work/lib/ +COPY target/quarkus-app/*.jar /work/ +COPY target/quarkus-app/app/ /work/app/ +COPY target/quarkus-app/quarkus/ /work/quarkus/ + +EXPOSE 8787 + +ENV JAVA_OPTS="-Djava.util.logging.manager=org.jboss.logmanager.LogManager" +ENTRYPOINT ["java", "-jar", "/work/quarkus-run.jar"] diff --git a/ac-code-server/src/main/java/com/agenticcode/codeserver/api/AnalysisResource.java b/ac-code-server/src/main/java/com/agenticcode/codeserver/api/AnalysisResource.java index 7f5eb77..f18ab7a 100644 --- a/ac-code-server/src/main/java/com/agenticcode/codeserver/api/AnalysisResource.java +++ b/ac-code-server/src/main/java/com/agenticcode/codeserver/api/AnalysisResource.java @@ -264,9 +264,11 @@ public class AnalysisResource { public Uni callTree(@PathParam("project") String project, @PathParam("name") String name, @QueryParam("depth") @Nullable Integer depth, @QueryParam("fields") @Nullable String fields, - @QueryParam("resolveInterfaces") @Nullable Boolean resolveInterfaces) { + @QueryParam("resolveInterfaces") @Nullable Boolean resolveInterfaces, + @QueryParam("followWiring") @Nullable Boolean followWiring) { int effectiveDepth = Math.clamp(depth != null ? depth : defaultCallTreeDepth, 1, maxCallTreeDepth); - return withProject(project, () -> graphRepository.callTree(project, name, effectiveDepth, resolveInterfaces != null && resolveInterfaces) + return withProject(project, () -> graphRepository.callTree(project, name, effectiveDepth, + resolveInterfaces != null && resolveInterfaces, followWiring != null && followWiring) .map(resp -> namesOnly(fields) ? ok(callTreeNames(resp)) : Response.ok(resp).build())); diff --git a/ac-code-server/src/main/java/com/agenticcode/codeserver/mcp/McpQueryTools.java b/ac-code-server/src/main/java/com/agenticcode/codeserver/mcp/McpQueryTools.java index fa75e71..75fc759 100644 --- a/ac-code-server/src/main/java/com/agenticcode/codeserver/mcp/McpQueryTools.java +++ b/ac-code-server/src/main/java/com/agenticcode/codeserver/mcp/McpQueryTools.java @@ -173,16 +173,17 @@ public class McpQueryTools { resolveInterfaces != null && resolveInterfaces).map(support::ok)); } - @Tool(name = "call_tree", description = "Transitive call tree rooted at a module, up to 'depth' hops (clamped to the configured maximum). resolveInterfaces drops dead-end interface nodes whose implementations are already reached.") + @Tool(name = "call_tree", description = "Transitive call tree rooted at a module, up to 'depth' hops (clamped to the configured maximum). resolveInterfaces drops dead-end interface nodes whose implementations are already reached. followWiring also traverses Java INJECTS/REFERENCES edges (DI/class-literal wiring), not just CALLS.") @Blocking public Uni callTree( @ToolArg(description = "Project name") String project, @ToolArg(description = "Module name") String name, @ToolArg(description = "Traversal depth (default configured, clamped 1..max)", required = false) @Nullable Integer depth, - @ToolArg(description = "Drop interface nodes that have a known implementation (Java)", required = false) @Nullable Boolean resolveInterfaces) { + @ToolArg(description = "Drop interface nodes that have a known implementation (Java)", required = false) @Nullable Boolean resolveInterfaces, + @ToolArg(description = "Also traverse INJECTS/REFERENCES edges (Java DI/class-literal wiring), not just CALLS", required = false) @Nullable Boolean followWiring) { int effectiveDepth = Math.clamp(depth != null ? depth : defaultCallTreeDepth, 1, maxCallTreeDepth); return withProject(project, () -> graphRepository.callTree(project, name, effectiveDepth, - resolveInterfaces != null && resolveInterfaces).map(support::ok)); + resolveInterfaces != null && resolveInterfaces, followWiring != null && followWiring).map(support::ok)); } // ------------------------------------------------------------------------- diff --git a/ac-code-server/src/test/java/com/agenticcode/codeserver/api/JavaWiringIT.java b/ac-code-server/src/test/java/com/agenticcode/codeserver/api/JavaWiringIT.java index 2bdf03f..b03ea5d 100644 --- a/ac-code-server/src/test/java/com/agenticcode/codeserver/api/JavaWiringIT.java +++ b/ac-code-server/src/test/java/com/agenticcode/codeserver/api/JavaWiringIT.java @@ -14,6 +14,7 @@ import java.nio.file.Path; import static io.restassured.RestAssured.given; import static org.hamcrest.Matchers.hasItem; +import static org.hamcrest.Matchers.not; /** * End-to-end test for J2: CDI injection and class-literal wiring surface as {@code INJECTS}/ @@ -81,4 +82,21 @@ class JavaWiringIT { .then().statusCode(200) .body("items.findAll { it.edgeKind == 'INJECTS' }.name", hasItem("AccountKeyImportJob")); } + + @Test + void callTreeFollowsWiringOnlyWhenRequested() { + // Default (CALLS-only) call-tree does not reach INJECTS/REFERENCES targets. + given().pathParam("name", "AccountKeyImportJob") + .when().get("/api/projects/" + PROJECT + "/modules/{name}/call-tree?depth=2") + .then().statusCode(200) + .body("items.name", not(hasItem("AuditLogger"))) + .body("items.name", not(hasItem("AccountKeyInitStep"))); + + // followWiring=true (item J9) traverses INJECTS/REFERENCES too. + given().pathParam("name", "AccountKeyImportJob") + .when().get("/api/projects/" + PROJECT + "/modules/{name}/call-tree?depth=2&followWiring=true") + .then().statusCode(200) + .body("items.name", hasItem("AuditLogger")) + .body("items.name", hasItem("AccountKeyInitStep")); + } } diff --git a/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/CypherQueries.java b/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/CypherQueries.java index 930c968..5554ddb 100644 --- a/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/CypherQueries.java +++ b/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/CypherQueries.java @@ -674,15 +674,27 @@ public final class CypherQueries { * dead-end interface. {@code false} keeps the exact default query. */ public static String callTree(int maxDepth, boolean resolveInterfaces) { + return callTree(maxDepth, resolveInterfaces, false); + } + + /** + * @param followWiring when {@code true} (item J9), the traversal also follows {@code INJECTS}/ + * {@code REFERENCES} edges (Java CDI injection + class-literal wiring) in + * addition to {@code CALLS}, so a job reaches its steps/collaborators + * transitively instead of stopping at direct calls. {@code false} keeps the + * exact default {@code CALLS}-only query. + */ + public static String callTree(int maxDepth, boolean resolveInterfaces, boolean followWiring) { String interfaceFilter = resolveInterfaces ? "WHERE NOT (target)-[:IMPLEMENTED_BY]->()" : ""; + String relTypes = followWiring ? "CALLS|INJECTS|REFERENCES" : "CALLS"; return """ MATCH (m:AstNode {type: 'MODULE', name: $name, project: $project}) - MATCH p = (m)-[:CALLS*1..%d]->(target:AstNode) + MATCH p = (m)-[:%s*1..%d]->(target:AstNode) %s WITH target, min(length(p)) AS depth RETURN DISTINCT target.name AS name, target.type AS type, target.sourceFile AS sourceFile, depth AS depth ORDER BY depth, name - """.formatted(maxDepth, interfaceFilter); + """.formatted(relTypes, maxDepth, interfaceFilter); } /** diff --git a/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/GraphRepository.java b/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/GraphRepository.java index d65cc77..1dc62c4 100644 --- a/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/GraphRepository.java +++ b/ac-neo4j-store/src/main/java/com/agenticcode/neo4jstore/graph/GraphRepository.java @@ -201,8 +201,19 @@ public class GraphRepository { * ({@code IMPLEMENTED_BY}); their impls are already reached via CHA. */ public Uni callTree(String project, String moduleName, int maxDepth, boolean resolveInterfaces) { + return callTree(project, moduleName, maxDepth, resolveInterfaces, false); + } + + /** + * @param followWiring item J9 — also traverse {@code INJECTS}/{@code REFERENCES} edges (Java + * DI/class-literal wiring), not just {@code CALLS}, so a job reaches its + * steps/collaborators transitively. + */ + public Uni callTree(String project, String moduleName, int maxDepth, boolean resolveInterfaces, + boolean followWiring) { int clampedDepth = Math.min(Math.max(maxDepth, 1), CALL_TREE_MAX_DEPTH_LIMIT); - return read(CypherQueries.callTree(clampedDepth, resolveInterfaces), Map.of("project", project, "name", moduleName), + return read(CypherQueries.callTree(clampedDepth, resolveInterfaces, followWiring), + Map.of("project", project, "name", moduleName), GraphRepository::toCallTreeRow) .map(GraphRepository::buildCallTreeResponse); } diff --git a/deploy.sh b/deploy.sh new file mode 100755 index 0000000..e4fbe83 --- /dev/null +++ b/deploy.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +# Build and (re)deploy the ac-code-server + neo4j docker-compose stack, and +# install the ac-cli launcher to the user's local bin. +# +# Usage: +# ./deploy.sh [up|stop|cli|logs [-f]] +# up (default) - mvn build, rebuild+restart ac-code-server (neo4j left +# running untouched if already up), install ac-cli. +# stop - stop only the ac-code-server container (neo4j untouched). +# cli - build and (re)install ac-cli only, no Docker involved. +# logs [-f] - show the last 200 lines of ac-code-server logs (-f to follow). +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$SCRIPT_DIR" + +CLI_INSTALL_DIR="$HOME/.local/share/agenticcode" +CLI_JAR_DEST="$CLI_INSTALL_DIR/ac-cli.jar" +CLI_BIN="$HOME/.local/bin/ac" +CLI_MARKER="# agenticcode-ac-cli-launcher" + +log() { echo "==> $*"; } + +compose() { + if docker compose version >/dev/null 2>&1; then + docker compose "$@" + else + docker-compose "$@" + fi +} + +require_docker() { + if ! docker info >/dev/null 2>&1; then + echo "Docker does not appear to be running (or not reachable). Start Docker and re-run." >&2 + exit 1 + fi +} + +build_all() { + log "Building all modules (mvn clean install -DskipTests)" + mvn clean install -DskipTests +} + +build_cli_only() { + log "Building ac-cli (mvn -pl ac-cli -am package -DskipTests)" + mvn -pl ac-cli -am package -DskipTests +} + +install_cli() { + local jar + jar="$(find ac-cli/target -maxdepth 1 -name 'ac-cli-*.jar' ! -name 'original-*' | head -n1)" + if [[ -z "$jar" ]]; then + echo "No ac-cli jar found in ac-cli/target — build failed?" >&2 + exit 1 + fi + + mkdir -p "$CLI_INSTALL_DIR" "$HOME/.local/bin" + cp "$jar" "$CLI_JAR_DEST" + + if [[ -e "$CLI_BIN" ]] && ! grep -q "$CLI_MARKER" "$CLI_BIN" 2>/dev/null; then + echo "Refusing to overwrite $CLI_BIN — it already exists and wasn't installed by this script." >&2 + echo "Remove it manually first, or install elsewhere." >&2 + exit 1 + fi + + cat > "$CLI_BIN" < $CLI_BIN (jar: $CLI_JAR_DEST)" + + case ":$PATH:" in + *":$HOME/.local/bin:"*) ;; + *) echo "Note: $HOME/.local/bin is not on your PATH. Add it, e.g.: export PATH=\"\$HOME/.local/bin:\$PATH\"" ;; + esac +} + +cmd_up() { + build_all + require_docker + log "Stopping ac-code-server container (neo4j left untouched)" + compose stop ac-code-server || true + compose rm -f ac-code-server || true + log "Building ac-code-server image" + compose build ac-code-server + log "Starting neo4j + ac-code-server" + compose up -d neo4j ac-code-server + install_cli +} + +cmd_stop() { + require_docker + log "Stopping ac-code-server container only (neo4j left running)" + compose stop ac-code-server +} + +cmd_cli() { + build_cli_only + install_cli +} + +cmd_logs() { + require_docker + if [[ "${1:-}" == "-f" ]]; then + docker logs -f --tail 200 agenticcode-server + else + docker logs --tail 200 agenticcode-server + fi +} + +case "${1:-up}" in + up) cmd_up ;; + stop) cmd_stop ;; + cli) cmd_cli ;; + logs) shift; cmd_logs "${1:-}" ;; + *) + echo "Usage: $0 [up|stop|cli|logs [-f]]" >&2 + exit 1 + ;; +esac diff --git a/docker-compose.yml b/docker-compose.yml index 4993e95..23e28c6 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -9,6 +9,37 @@ services: NEO4J_AUTH: neo4j/agenticcode volumes: - neo4j-data:/data + healthcheck: + test: [ "CMD-SHELL", "cypher-shell -u neo4j -p agenticcode 'RETURN 1' || exit 1" ] + interval: 5s + timeout: 5s + retries: 20 + start_period: 30s + + ac-code-server: + build: + context: ./ac-code-server + dockerfile: src/main/docker/Dockerfile.jvm + container_name: agenticcode-server + ports: + - "8787:8787" + environment: + NEO4J_URI: bolt://neo4j:7687 + NEO4J_USER: neo4j + NEO4J_PASSWORD: agenticcode + volumes: + # Mounted at the same absolute host path so project roots registered via + # the API (which store absolute host paths) resolve inside the container too. + # One entry per registered project root (GET /api/projects) — add a line + # here whenever a new project root is registered outside this repo. + - /home/ingo/deve/agenticCode:/home/ingo/deve/agenticCode:ro + - /home/ingo/deve/uniqa/uniqa-upms-app:/home/ingo/deve/uniqa/uniqa-upms-app:ro + - /home/ingo/deve/uniqa/pur-sources/backend:/home/ingo/deve/uniqa/pur-sources/backend:ro + - /home/ingo/deve/tools/conqat/system/src/250401_UMPS/src/pur-analysis/pur-legacy:/home/ingo/deve/tools/conqat/system/src/250401_UMPS/src/pur-analysis/pur-legacy:ro + depends_on: + neo4j: + condition: service_healthy + restart: unless-stopped volumes: neo4j-data: diff --git a/scripts/restore-test-projects.sh b/scripts/restore-test-projects.sh index 14f2dd3..119df7e 100755 --- a/scripts/restore-test-projects.sh +++ b/scripts/restore-test-projects.sh @@ -24,7 +24,7 @@ create() { create app "/home/ingo/deve/uniqa/uniqa-upms-app" '["test","target"]' create pur "/home/ingo/deve/uniqa/pur-sources/backend" '["test","target"]' create upms "/home/ingo/deve/tools/conqat/system/src/250401_UMPS/src/pur-analysis/pur-legacy" '["user_exit"]' - +create ac "/home/ingo/deve/agenticCode" '["test","target"]' echo "Done. Current projects:" curl -s "$BASE/api/projects" echo diff --git a/x-docs/agent-module-analysis.md b/x-docs/agent-module-analysis.md index dfca034..2014ece 100644 --- a/x-docs/agent-module-analysis.md +++ b/x-docs/agent-module-analysis.md @@ -451,25 +451,25 @@ A missing/blank `value` returns `400 MISSING_VALUE`. ## Endpoint summary -| Endpoint | Use for | -|------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------| -| `GET /modules?sourceFile=...` | Step 0: list modules in the project, or look up the module name for a source file | -| `GET /modules/{name}/context` | Step 1: one-shot overview (functions, callers, callees, DB accesses, SQL/variable **summaries** by default; full lists via `?include=`) | -| `GET /modules/{name}/digest` | Tiny triage view (names/counts only): description, functionCount, callers/callees by edgeKind, dbTables, dataStructures (name+fieldCount) | -| `GET /modules/{name}/callers` | Who calls/extends/implements/**injects/references** this module (direct): `CALLS`/`PERFORM` + incoming `EXTENDS`/`IMPLEMENTS`/`INJECTS`/`REFERENCES` | -| `GET /modules/{name}/callees` | What this module calls/extends/implements + Java `INJECTS`/`REFERENCES` wiring (direct only) | -| `GET /variables/{name}/reads?module=&depth=N` | Where a variable is read, optionally scoped to a module and traversed across the call graph | -| `GET /variables/{name}/writes?module=&depth=N` | Where a variable is written, optionally scoped to a module and traversed across the call graph | -| `GET /modules/{name}/call-tree?depth=N` | Transitive call graph, to scope a feature. `?resolveInterfaces=true` drops dead-end interface nodes (Java) | -| `GET /modules/{name}/db-accesses` | DB tables + READ/WRITE mode. Natural: `READ/FIND/STORE/…`. Java: JPA/Panache repository, `EntityManager` and Panache active-record calls | -| `GET /modules/{name}/sql-statements` | Raw SQL/ADABAS (Natural) or the resolved persistence call text (Java) + table/view/mode | -| `GET /modules/{name}/data-structures` | The `DEFINE DATA` areas a module uses (USING copybooks + inline groups) with `area`/`fieldCount` | -| `GET /modules/{name}/dispatch-table` | Routing table of a `DECIDE ON VALUE OF` dispatcher: `guardValue → assignedField := assignedValue` | -| `GET /modules/{name}/functions/{fn}/overrides` | Java: concrete subclass overrides of a base-class method (virtual/template-method dispatch) — `{module, name, sourceFile, lines}` | -| `GET /data-structures/{name}/fields` | Flattened field schema of a `DEFINE DATA`/DDM structure (canonical definition only) | -| `GET /db-tables/{name}/columns` | Column schema derived from a table's `INTO VIEW` structure | -| `GET /search/identifier?name=&type=` | Cross-module occurrences of an identifier (by **name**), optionally filtered by `NodeType` | -| `GET /search/value?value=` | Occurrences of a literal **value** (constant nodes + variable assignments) — find dynamically-used names | +| Endpoint | Use for | +|------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `GET /modules?sourceFile=...` | Step 0: list modules in the project, or look up the module name for a source file | +| `GET /modules/{name}/context` | Step 1: one-shot overview (functions, callers, callees, DB accesses, SQL/variable **summaries** by default; full lists via `?include=`) | +| `GET /modules/{name}/digest` | Tiny triage view (names/counts only): description, functionCount, callers/callees by edgeKind, dbTables, dataStructures (name+fieldCount) | +| `GET /modules/{name}/callers` | Who calls/extends/implements/**injects/references** this module (direct): `CALLS`/`PERFORM` + incoming `EXTENDS`/`IMPLEMENTS`/`INJECTS`/`REFERENCES` | +| `GET /modules/{name}/callees` | What this module calls/extends/implements + Java `INJECTS`/`REFERENCES` wiring (direct only) | +| `GET /variables/{name}/reads?module=&depth=N` | Where a variable is read, optionally scoped to a module and traversed across the call graph | +| `GET /variables/{name}/writes?module=&depth=N` | Where a variable is written, optionally scoped to a module and traversed across the call graph | +| `GET /modules/{name}/call-tree?depth=N` | Transitive call graph, to scope a feature. `?resolveInterfaces=true` drops dead-end interface nodes (Java); `?followWiring=true` also traverses `INJECTS`/`REFERENCES` (Java) | +| `GET /modules/{name}/db-accesses` | DB tables + READ/WRITE mode. Natural: `READ/FIND/STORE/…`. Java: JPA/Panache repository, `EntityManager` and Panache active-record calls | +| `GET /modules/{name}/sql-statements` | Raw SQL/ADABAS (Natural) or the resolved persistence call text (Java) + table/view/mode | +| `GET /modules/{name}/data-structures` | The `DEFINE DATA` areas a module uses (USING copybooks + inline groups) with `area`/`fieldCount` | +| `GET /modules/{name}/dispatch-table` | Routing table of a `DECIDE ON VALUE OF` dispatcher: `guardValue → assignedField := assignedValue` | +| `GET /modules/{name}/functions/{fn}/overrides` | Java: concrete subclass overrides of a base-class method (virtual/template-method dispatch) — `{module, name, sourceFile, lines}` | +| `GET /data-structures/{name}/fields` | Flattened field schema of a `DEFINE DATA`/DDM structure (canonical definition only) | +| `GET /db-tables/{name}/columns` | Column schema derived from a table's `INTO VIEW` structure | +| `GET /search/identifier?name=&type=` | Cross-module occurrences of an identifier (by **name**), optionally filtered by `NodeType` | +| `GET /search/value?value=` | Occurrences of a literal **value** (constant nodes + variable assignments) — find dynamically-used names | **Names-only mode (P1-w).** `callers`, `callees`, `call-tree`, and `search/identifier` accept `?fields=name`: the response becomes a flat array of @@ -502,7 +502,10 @@ enumerate names. Any other `fields` value (or none) returns the full shape. argument position, e.g. `super(SomeStep.class, …)`). So `callees(Job)` shows the beans it injects and the steps it wires by class literal — reconnecting a job to collaborators that aren't reached by a method call. Like `EXTENDS`/`IMPLEMENTS`, these are **not** - followed by `call-tree` (which stays CALLS-only). + followed by `call-tree` by default (which stays CALLS-only) — pass `?followWiring=true` + (roadmap J9) to also traverse `INJECTS`/`REFERENCES` transitively, e.g. to reach + `call-tree(Job)` → step → step's own repository in one call instead of chaining + `digest`/`callees` by hand. Off by default; composes with `?resolveInterfaces=true`. - **`?resolveInterfaces=true` (Java)** on `callees`/`call-tree` hops an interface to its concrete implementation(s): `callees` replaces the interface callee with its impl(s) (a single implementation is a clean deterministic hop; several expand to all), and diff --git a/x-docs/features.md b/x-docs/features.md index 25a721c..1e3dfca 100644 --- a/x-docs/features.md +++ b/x-docs/features.md @@ -452,10 +452,32 @@ items). Each entry records what was built; IDs are preserved from the roadmap them cross-file. Surfaced in `callers`/`callees` (query match widened to `CALLS|EXTENDS|IMPLEMENTS|INJECTS|REFERENCES`) as `edgeKind = INJECTS`/`REFERENCES`, mirroring how `EXTENDS`/`IMPLEMENTS` already appear; **deliberately excluded from - `call-tree`** (kept CALLS-only). New fixtures `fixtures/java/wiring/*` (a job with a + `call-tree`** by default (kept CALLS-only; see J9 below for the opt-in traversal). + New fixtures `fixtures/java/wiring/*` (a job with a class-literal step + `@Inject` collaborator, a constructor-injected bean) and `JavaWiringIT` (3 tests); all 68 prior ITs green (shared callers/callees query change verified non-regressive), 11 parser tests green. + **Re-test 2026-07-07 confirms this works:** all 3 probed PUR job digests now list + their steps via `callees.REFERENCES` (`RiskImportJob` → `RiskInitStep`/`RiskProcessingStep`/ + `RiskEndStep`; same for `MultiTableImportJob`, `KeyTableExportJob`), and the step's + `callers.REFERENCES` names the job. The former job-root dead-end is fixed at digest level. + Follow-up usability gap fixed by **J9** (below). + +- [x] **J9. Opt-in traversal that follows `REFERENCES`/`INJECTS` (Java)** (2026-07-07) — + J2 wired the `INJECTS`/`REFERENCES` edges into `callees`/`callers` but kept them out of + `call-tree` (CALLS-only), so there was no automatic transitive tree from a job to its + steps to their repositories — an agent had to chain `digest`/`callees` calls by hand. + New `followWiring` boolean, mirroring the J3 `resolveInterfaces` pattern end-to-end + (REST `call-tree` query param, MCP `call_tree` tool arg, `GraphRepository.callTree`, + `CypherQueries.callTree`): when `true`, the transitive-traversal relationship pattern + becomes `CALLS|INJECTS|REFERENCES` instead of `CALLS` (both `INJECTS`/`REFERENCES` are + `MODULE`→`MODULE`, same as the `CALLS` edges `call-tree` already traverses, so mixing + them into one variable-length path is schema-compatible). Composes with + `resolveInterfaces`. `callees`/`callers` unchanged (already surfaced these edges at one + hop since J2) — only `call-tree`'s transitive traversal needed the flag. Off by default. + New test `callTreeFollowsWiringOnlyWhenRequested` in `JavaWiringIT` (default call-tree + excludes `INJECTS`/`REFERENCES` targets; `followWiring=true` includes them), reusing the + existing `fixtures/java/wiring/*` fixtures. - [x] **J1a. JPA/Panache repository calls as Java DB accesses** (2026-07-07) — Java `db-accesses`/`sql-statements` were always empty; now repository/EntityManager/ @@ -481,6 +503,11 @@ items). Each entry records what was built; IDs are preserved from the roadmap entity) and `JavaDbAccessIT` (3 tests); all 68 server ITs + 11 parser tests green. **J1b deferred** (roadmap): `@Query`/JPQL/native-SQL string parsing, derived-name filters, and no-generic custom repositories (need symbol resolution). + **Re-test 2026-07-07 fixed by J7 + J8** (below): `JavaDbAccessInheritedRepoIT` + (repo → project base → Panache base, plus a constant-valued `@Entity(name=…)`) now + passes end-to-end — `db-accesses`/`sql-statements` resolve to `RISK`. A regression + check against the actual 9 PUR jobs is still worth doing as a follow-up acceptance + pass, but the two traced root causes are fixed. - [x] **J7. Panache-ness inherited through a project base class** (2026-07-07) — J1a only recovered `repositoryEntity` when a repository *directly* extended a @@ -783,6 +810,37 @@ items). Each entry records what was built; IDs are preserved from the roadmap `callers`/`callees`/`db-accesses`/`search-identifier` (and the transitive `db-accesses` variant), default `limit=50`, `offset=0`. CLI gains `--limit`/`--offset`. New IT `calleesPaginationLimitsAndOffsets`. +- [x] **P1-t. `/context` is heavy by default — make sub-arrays opt-in, return counts** + (2026-06-22) (found 2026-06-21) — the live `WGEAGB0S` `/context` was **~50 KB**, ~70% + of it the `variableAccesses` array (**347 entries**) which the analysis never used. + `?include=` projection existed but the *default* still returned every section fully + expanded. Flipped the default to **lean**: heavy sub-arrays (`variableAccesses`, large + `sqlStatements`) return a **summary** by default — e.g. + `variableAccesses: { count, byFunction: {...}, byMode: { READS, WRITES } }` — and the + full list is emitted only via `?include=`. A summary line replaces 347 objects; the + biggest single token lever found in the `WGEAGB0S` analysis. +- [x] **P1-v. Tiny `/modules/{name}/digest` triage endpoint** (2026-06-22) + (found 2026-06-21) — `/context` is the "expand" call (tens of KB); there was no + "should I dig deeper" call. Added `GET /modules/{name}/digest` with a deliberately + small contract: `description` (P1-u), function **count**, callers/callees + **names only** grouped by `edgeKind`, DB table **names**, referenced data-structure + **names + field counts** (P1-r) — designed to a token budget, not a full dump. +- [x] **P1-w. Names-only / field-projection mode on list endpoints** (2026-06-22) + (found 2026-06-21) — `call-tree`, `callers`, `callees`, `search/identifier` are often + used only to **enumerate names**, yet each row still carried `sourceFile(Index)`, line + ranges, `dataType`, etc. Added a `?fields=name` variant that drops per-row detail to + just the name (+ type) when the agent is enumerating, on top of the existing + `sourceFiles`-index dedup (P1-l). Complements P1-t/P1-v. +- [x] **P1-x. Dynamic `CALLNAT` via lookup array + keep unresolved dynamic calls visible** + (2026-06-22) — a `CALLNAT ` whose dispatch variable is loaded from a lookup array + (e.g. `ASSIGN #TBL(1) = 'WPARTD2S'`, `#W-ACT-PROG := #TBL(#I)`, `CALLNAT #W-ACT-PROG` — + WPARTX2S L994) was dropped entirely: (1) the parser missed the subscripted literal + write (`ASSIGN #TBL (1) = …`, space before `(`), and (2) the unresolved dynamic marker + was deleted, erasing the call site. Fix: parser now captures subscripted assignment + targets; a new intra-module **indirect** resolver follows the dispatch var's same-line + `READS` to the source array and resolves its literals (tagged `indirect`); unresolved + dynamic markers are now **kept** (only resolved-site markers are reaped) so an agent + can still see/investigate the dynamic call in `callees`/`context`. ## Integration & ops diff --git a/x-docs/roadmap.md b/x-docs/roadmap.md index 88c9141..9a05541 100644 --- a/x-docs/roadmap.md +++ b/x-docs/roadmap.md @@ -23,17 +23,7 @@ Motivated by a batch-job analysis pass over the PUR `pur-batch` module findings at the *step-class* level, but three structural gaps forced the work to be **source-driven rather than graph-driven**. Root cause: the Java graph models only direct method/constructor calls, not the framework/DI/JPA indirection PUR -actually runs on. Ordered by analytical impact. - -- [x] **J1a. JPA/Panache repository calls as DB accesses (Java)** — done 2026-07-07, - see `x-docs/features.md`. Repository/EntityManager/Panache active-record calls now - resolve to `READS`/`WRITES` on the entity's `DB_TABLE`, so Java `db-accesses`/ - `sql-statements` are populated. **J1b remains open** (below). - - ✅ **Re-test 2026-07-07 fixed by J7 + J8** (both below): `JavaDbAccessInheritedRepoIT` - (repo → project base → Panache base, plus a constant-valued `@Entity(name=…)`) now - passes end-to-end — `db-accesses`/`sql-statements` resolve to `RISK`. A regression - check against the actual 9 PUR jobs is still worth doing as a follow-up acceptance - pass, but the two traced root causes are fixed. +actually runs on. J1a-J9 done (see `x-docs/features.md`); J1b remains open. - [ ] **J1b. Parse `@Query`/JPQL/native-SQL strings + derived-name filters (Java)** (found 2026-07-06) — J1a resolves calls whose entity is syntactically recoverable @@ -44,119 +34,6 @@ actually runs on. Ordered by analytical impact. interface) — needs symbol resolution / an entity-hint, since pure syntactic parsing can't recover the entity. -- [x] **J2. DI + class-literal edges (Java)** — done 2026-07-07, see - `x-docs/features.md`. `INJECTS` (from `@Inject`/constructor-injected fields) and - `REFERENCES` (from `X.class` in argument position) edges now reconnect a job to its - injected collaborators and referenced steps; surfaced in `callees`/`callers` as - `edgeKind = INJECTS`/`REFERENCES`. Kept out of `call-tree` (CALLS-only) by design. - - ✅ **Re-test 2026-07-07 confirms this works:** all 3 probed PUR job digests now list - their steps via `callees.REFERENCES` (`RiskImportJob` → `RiskInitStep`/`RiskProcessingStep`/ - `RiskEndStep`; same for `MultiTableImportJob`, `KeyTableExportJob`), and the step's - `callers.REFERENCES` names the job. The former job-root dead-end is fixed at digest level. - Follow-up usability gap captured as **J9** below. - -- [x] **J7. Panache-ness inherited through a project base class (Java)** — done 2026-07-07. - J1a resolved DB accesses only when the repository *directly* extends a Panache type; PUR-shaped - repositories don't (`RiskRepository extends AbstractPurRepository`, with the Panache - marker one hop up on `AbstractPurRepository implements PanacheRepositoryBase`). - Fixed as a parser + graph-enrichment pair: the parser tags a project base class with - `panacheEntityTypeParam` (which of its own type parameters is the entity slot) + its ordered - `typeParams`, and tags an `EXTENDS` edge with the concrete `typeArgs` a subclass supplies; a new - enrichment step (`resolve-panache-inherited-entity`, `CypherQueries.RESOLVE_PANACHE_INHERITED_ENTITY`) - binds the two to set `repositoryEntity` on the concrete repository, which `RESOLVE_JAVA_DB_ACCESS` - then uses unchanged. Covers one level of indirection (the observed shape); regression test - `JavaDbAccessInheritedRepoIT`. Distinct from J1b's "no generic entity type at all" (the - `IRiskRepository` interface) case. - -- [x] **J8. `@Entity(name=CONST)` / `@Table` with a constant table name (Java)** — verified - 2026-07-07 already working (no change needed): the physical table name isn't always a string - literal — PUR-shaped entities use a constant reference (`@Entity(name = Risk.TABLE_NAME)` with - `public static final String TABLE_NAME = "RISK";`). `JavaParser.resolveTableName` already resolves - same-class constant references (via `collectConstants`/`resolveAnnotationString`, from earlier - constant-value-resolution work), so this was already handled; confirmed by direct parse of the - `JavaDbAccessInheritedRepoIT` fixture (`Risk.java` → `DB_TABLE` node named `RISK`) before any J7 fix - was applied. - -- [x] **J3. Interface → implementation resolution (Java)** — done 2026-07-07, see - `x-docs/features.md`. Explicit `IMPLEMENTED_BY` edge (interface → concrete impl) built - in enrichment, plus `?resolveInterfaces=true` on `callees`/`call-tree` (REST + MCP) that - hops interface callees to their implementation(s) and drops the dead-end interface. Java - MODULE nodes now carry `isInterface`. (The traversal dead-end itself was already fixed - earlier by the polymorphic CHA step `link-calls-to-implementations`.) - -- [x] **J4. Virtual/override (template-method) dispatch (Java)** — done 2026-07-07, - see `x-docs/features.md`. `OVERRIDDEN_BY` edge (base method → same-named subclass - override, transitive over `EXTENDS`) built in enrichment, exposed via a new - `GET /modules/{name}/functions/{fn}/overrides` endpoint + `function_overrides` MCP - tool — "whose code runs when this template method calls `clearTable()`?". - -- [x] **J5. Cross-class dataflow (Java)** — done 2026-07-07, see `x-docs/features.md`. - Cross-class Java `flow-forward`/`flow-backward` now map arguments to the **specific** - callee method's parameters (method-aware `ARG_TO_PARAM`), replacing the old imprecise - behavior where the language-agnostic linker matched an argument to the param-at-position - of *every* method in the callee class. Parser stamps `callerFn`/`calleeMethod` on - cross-class call edges; a new dataflow step builds the precise edges; the generic linker - is now guarded off Java cross-class edges. - -- [x] **J6. Ingest noise: JDK/framework allowlist** — done 2026-07-07, see - `x-docs/features.md`. A by-name ingest now skips JDK/stdlib/framework types - (`List`, `String`, `Optional`, `EntityManager`, …) instead of chasing them and - reporting them in `unresolved`; `target/` build output is excluded from the scan by - default (no generated-source duplicates). So `unresolved` surfaces only genuine gaps. - -- [ ] **J9. Opt-in traversal that follows `REFERENCES`/`INJECTS` (Java)** (found 2026-07-07) — - J2 wired the `REFERENCES`/`INJECTS` edges but keeps them out of `call-tree` (CALLS-only). For - framework/DI-heavy code (JBeret jobs, CDI) this means there is **no automatic transitive tree** - from a job to its steps to their repositories — the agent must chain `digest` calls by hand - (job-digest → step-digest → repo/entity). Mirror the J3 solution: an opt-in flag on - `call-tree`/`callees` (e.g. `?includeEdges=REFERENCES,INJECTS` or `?followWiring=true`, REST + - MCP) that traverses these runtime-wiring edges so `call-tree(RiskImportJob)` can reach - `RiskProcessingStep` → `RiskRepository` in one call. Keep the CALLS-only default. Combined with - J7/J8 this is what would finally make batch-job analysis **graph-driven rather than - source-driven**. - -## Token efficiency (payload shape) - -- [x] **P1-t. `/context` is heavy by default — make sub-arrays opt-in, return counts** (done 2026-06-22) - (found 2026-06-21) — the live `WGEAGB0S` `/context` was **~50 KB**, ~70% of it the - `variableAccesses` array (**347 entries**) which the analysis never used. `?include=` - projection exists but the *default* still returns every section fully expanded, - so the common case pays for data it discards. → Flip the default to **lean**: heavy - sub-arrays (`variableAccesses`, and large `sqlStatements`) return a **summary** by - default — e.g. `variableAccesses: { count, byFunction: {...}, byMode: { READS, WRITES } }` - — and the full list is emitted only when explicitly requested via `?include=`. A - summary line replaces 347 objects. Biggest single token lever found in the `WGEAGB0S` - analysis. Must update `x-docs/agent-module-analysis.md`. - -- [x] **P1-v. Tiny `/modules/{name}/digest` triage endpoint** (done 2026-06-22) - (found 2026-06-21) — `/context` is the "expand" call (tens of KB); there is no "should I - dig deeper" call. An agent scanning many modules (e.g. the ~80 `Wxxxx0S` browse - subprograms) wants a ~1–2 KB triage, not N × 50 KB. → Add `GET /modules/{name}/digest` - with a deliberately small contract: `description` (P1-u), function **count**, - callers/callees **names only** grouped by `edgeKind`, DB table **names**, referenced - data-structure **names + field counts** (P1-r). Designed to a token budget, not a full - dump. Must update `x-docs/agent-module-analysis.md`. - -- [x] **P1-w. Names-only / field-projection mode on list endpoints** (done 2026-06-22) - (found 2026-06-21) — `call-tree`, `callers`, `callees`, `search/identifier` are often - used only to **enumerate names**, yet each row still carries `sourceFile(Index)`, line - ranges, `dataType`, etc. → Add a `?fields=name` (or `names-only=true`) variant that - drops per-row detail to just the name (+ type) when the agent is enumerating, on top of - the existing `sourceFiles`-index dedup (P1-l). Complements P1-t/P1-v. Must update - `x-docs/agent-module-analysis.md`. - -- [x] **P1-x. Dynamic `CALLNAT` via lookup array + keep unresolved dynamic calls visible** - (done 2026-06-22) (found 2026-06-22) — A `CALLNAT ` whose dispatch variable is - loaded from a lookup array (e.g. `ASSIGN #TBL(1) = 'WPARTD2S'`, `#W-ACT-PROG := #TBL(#I)`, - `CALLNAT #W-ACT-PROG` — WPARTX2S L994) was dropped entirely: (1) the parser missed the - subscripted literal write (`ASSIGN #TBL (1) = …`, space before `(`), and (2) the - unresolved dynamic marker was deleted, erasing the call site. Fix: parser now captures - subscripted assignment targets; a new intra-module **indirect** resolver follows the - dispatch var's same-line `READS` to the source array and resolves its literals (tagged - `indirect`); and unresolved dynamic markers are now **kept** (only resolved-site markers - are reaped) so an agent can still see/investigate the dynamic call in `callees`/`context`. - Updated `x-docs/agent-module-analysis.md`. - ## Ingest performance - [ ] **24. `ingest-all` performance: parallel parse phase** — the parse phase of