Expedition 2026-08-27 (night watch, eleventh bell) · Cartographer: opencode

(glm-5.3) · Chart:

/demo/bigtop-landscape/.portolan/chart/

Gap expedition per accepted harbor proposal 67966da9… (kind: gap):

vessel/spark#behavior. Scope honored: exactly the one named slot — net notices

are CORRECTED vessel/spark (anchors, behavior, note, trust); the other 107

standing entries round-tripped content-identical (108 stand; write receipt

r107, whose count prose misstated 105 — struck by r108). Behavior established

launcher-observed, receipts r103-r106; all 18 cited anchors sounded confirmed

before the write. The recomputed harbor queue is now EMPTY — no repair, gap,

or new-land proposal stands; every charted vessel has behavior and a light.

Prior bells, same night: gap 74424981… solr#behavior (r99-r102),

7fbef6b1… skill#lights (r97/r98), 2d6ce98f… hive#behavior (r95/r96),

17a0c405… core#lights (r93/r94), 3a59c7ed… airflow#behavior (r91),

dd44a63a… acceptance#lights (r89), 23c22030… zookeeper (r86),

855f70cb… zeppelin (r84), 0b4ad5dd… tez (r82), 2c9ba1e0… ranger (r79).

The waters

The bigtop stack province: 22 vessels (the bigtop master repo plus 19 BOM

components and two retired vessels, oozie/sqoop), 93 chart entries (re-issued 2026-08-30: the four Portolan-runtime vessels returned to their own province's chart, and the repos were fast-forwarded to current heads). Vessel

core — the Portolan MCP server itself — exists in this province as runtime

wiring, not source: it serves every tool call the watches make.

Top findings

spark-class gates twice before any JVM work — and in this province both

gates were observed live — trust: measured — anchors:

repos/apache-spark/bin/spark-class:33-34, :45-48, receipts r103, r104 —

chart: vessel/spark (behavior). With no JAVA_HOME and no java on PATH every

client entry point (spark-submit:27, spark-shell:59 → all exec spark-class

org.apache.spark.deploy.SparkSubmit) dies "JAVA_HOME is not set" exit 1;

with JAVA_HOME set (faked probe) the unbuilt-tree gate fires next:

"Failed to find Spark jars directory (…/assembly/target/scala-2.13/jars)

… build with the target \"package\"" exit 1. Past both gates spark-class

delegates command assembly to org.apache.spark.launcher.Main

(spark-class:71). Daemon dispatch confirmed too: sbin/start-master.sh

--help prints usage, surfaces the java-gate error through its grep filter,

yet exits 0 (r106) — CLASS org.apache.spark.deploy.master.Master.

bin/pyspark of the 5.0.0-SNAPSHOT tree offers to INSTALL a JDK when none

is found — trust: measured — anchors: repos/apache-spark/bin/pyspark:60,

receipt r105 — chart: vessel/spark (behavior). With no JVM it asks

"JAVA_HOME is not set. Would you like to install JDK 17 and set JAVA_HOME?

(Y/N)"; on Y it pip-installs install-jdk into a mktemp dir to fetch a JDK

17 (network — NOT exercised; declined in the probe, exit 1). A novel gate

absent from the BOM-pinned 3.5.x line; surprising on a headless host.

The packaged bigtop spark is built from SOURCE, not a binary tgz — the

vessel note's old claim is corrected — trust: measured — anchors:

bigtop-packages/src/common/spark/do-component-build:38, bom:225,

install_spark.sh:164 — chart: vessel/spark (note, behavior). The package

builds the BOM-pinned 3.5.6 source tgz via ./dev/make-distribution.sh

(-Pyarn -Phadoop-3 -Phive -Phive-thriftserver -Psparkr -Pkubernetes), lays

it under /usr/lib/spark with conf at /etc/spark/conf.dist (bigtop

spark-env.sh: the SPARK_DIST_CLASSPATH=$(hadoop classpath) handoff,

beacon/beacon-spark-env), and generates /usr/bin wrappers that source

bigtop-detect-javahome (install_spark.sh:164-197). Four init.d services run

as user spark (rpm init.d.tmpl + chkconfig, spec:212): spark-master and

spark-worker and spark-history-server via spark-class, and

spark-thriftserver — the only one launched via bin/spark-submit --class

org.apache.spark.sql.hive.thriftserver.HiveThriftServer2

(spark-thriftserver.svc:19,48).

spark joins the version-skew family: BOM 3.5.6 vs local 5.0.0-SNAPSHOT

checkout — trust: measured — anchors: bigtop.bom:225,

repos/apache-spark/pom.xml:29 — chart: vessel/spark (note). The charted

danger/danger-version-skew names hadoop/zookeeper/zeppelin/livy/alluxio/

solr/kafka only; spark is absent from that list. Charting the danger entry

is outside this gap-repair's scope — flagged for the next expedition (a

repair proposal would carry it).

Delivery note: per the standing transport-cap notice (r91, r94, r96, r98,

r102), the full-replace was again delivered through the store's own

writeChart (same code path as chart.write; validation, shrink guard, and

diff all ran). Receipt r107's "105 entries" prose was wrong — 108 in / 108

out; struck by r108.

The Chart

The Chart lives at .portolan/chart/ — one sheet per vessel plus the machine

index index.jsonl (108 entries). Read the trust labels before trusting

anything: measured taken from source or observation, charted from

manifests, reported a claim from docs, doubtful unvalidated,

unsurveyed not determined.

Unsurveyed waters

spark live runtime — SparkSession, DAG scheduler, standalone cluster

(master/worker RPC), YARN client/cluster submission, thriftserver JDBC —

unsurveyed (no JVM, unbuilt 5.0.0-SNAPSHOT tree); the packaged 3.5.6

runtime likewise unobserved (danger/danger-version-skew family).

The skill's verify harness (skill/verify/checks.ts, dry-run.ts): no

receipted run exists in this province's log — unsurveyed — chart:

light/light-skill-frontmatter (note), vessel/skill (behavior).

The skill's frontmatter fields: read from the outside implementing source,

not anchorable in this province — chart: light/light-skill-frontmatter.

chart.render serving in this province: unsurveyed — the rendered

artifacts (.portolan/chart-room.html, fleet-review.html) exist but no

receipt ties them to a tool call — chart: light/light-core-mcp-tools.

Deployed versions province-wide: unsurveyed — chart: danger/danger-version-skew.
Whether today's Chart still passes the acceptance gate: unsurveyed — a

fresh sea-trial run was again deliberately not executed (it would

overwrite report.md, the evidence light-acceptance-sea-trial anchors to).

The harbor queue is empty as of this bell: no repair, gap, or new-land

proposal stands.

Notices to Mariners

CORRECTED vessel/spark — changed: anchors, behavior, note, trust; 9 anchors

now (bom:215 kept, plus pom.xml:29, spark-class:33, pyspark:60,

install_spark.sh:164 and receipts r103-r106); trust charted → measured;

behavior launcher-observed per the method (write receipt r107, count-prose

correction r108; gap vessel/spark#behavior closed).

No other entry changed; nothing added, retired, or marked stale (108

entries stand).

Prior bells: CORRECTED vessel/solr (r102). CORRECTED vessel/skill + ADDED

light/light-skill-frontmatter (r98). CORRECTED vessel/hive (r96). ADDED

light/light-core-mcp-tools (r94). CORRECTED vessel/airflow (r91). ADDED

light/light-acceptance-sea-trial (r89). ADDED light/light-zookeeper-cli,

CORRECTED vessel/zookeeper (r86). ADDED light/light-zeppelin-server-api,

CORRECTED vessel/zeppelin (r84). ADDED light/light-tez-client-am,

CORRECTED vessel/tez (r82). ADDED light/light-ranger-rest, CORRECTED

vessel/ranger (r79).