Gold Trace Forge
Authors domain packs, scenarios, and evaluation protocols with the underlying sources and intended outcome attached.
Evidence-bound agent data
Agent training data with a record of how it was made.
Gold Trace Dataworks turns agent runs and contained simulations into curated datasets. Every row keeps its source, transformations, evaluation, rights status, and limits attached.
- Traceable inputs
- Witnessed runs
- Curated outputs
claim / 0042source-backed
The scenario pack passes its structural validator.
- Posture
- Observed
- Scope
- Files, policy, configuration
- Runtime outcome
- Unknown
- Evidence gaps
- Container execution not performed
manifest sha256
5df2309c…6a43803Failed runs, unknown outcomes, and inconvenient evidence stay in the record instead of being smoothed into a success score.
The system
The system keeps four jobs separate.
A service that writes the task, runs it, grades itself, and sells the result is difficult to audit. Dataworks separates those responsibilities and connects them with versioned handoffs.
Gold Trace Arena
Runs admitted scenarios inside bounded environments and records proposals, policy decisions, tool results, effects, gaps, and cleanup.
Gold Trace Studio
Gives people a place to request work and inspect receipts without changing the evidence or upgrading a claim.
Gold Trace Dataworks
Takes admitted traces through provenance, privacy, rights, deduplication, grading, packaging, delivery, and recall.
Lineage
From request to dataset.
The links are part of the product. Removing them turns a dataset back into an unexplained pile of examples.
- 01Intent
The requested outcome, limits, and stop conditions.
- 02Pack
Domain knowledge, scenario, and evaluation source material.
- 03Run
Model proposals joined to policy and runtime evidence.
- 04Curation
Lineage, privacy, rights, deduplication, and rubrics.
- 05Dataset bar
A reproducible product with claims and limitations.
A source-backed example
One scenario pack supports two very different claims.
An owned-network investigation pack passes its structural validator. Its files, policy, and configuration agree. The containerized range was not run in the authoring environment, so the pack does not establish a successful investigation.
Keeping those claims separate prevents a configuration check from being recycled as evidence that an agent or sandbox performed correctly.
Read the complete exampleObservedStructural scope
The pack passes validation.
Required files, identifiers, policy, and configuration agree.
UnknownRuntime scope
The investigation succeeded.
No container run, model action, runtime effect, or resulting finding was observed.
Open schemas
Read the data contracts.
The public examples use synthetic identifiers and contain no raw trace content. They show how claims, provenance, and rubric bindings stay connected.
Evidence-bound claim
statement · posture · scope · evidence_refs · known_gapsProduct lineage
artifact · upstream · transforms · rights_ref · privacy_refEvaluation binding
core · domain · eval · context · release_profileOpen schemas
The fields behind the claims.
These working examples show the public shape of a Gold Trace record. They use synthetic identifiers, omit raw trace content, and do not expose private source, customer, or security data.
Working contractsVersions and compatibility guarantees are reported directly. A draft is labeled as a draft.
goldtrace.claim.v1
Evidence-bound claim
A claim carries its posture, scope, evidence, gaps, and falsifiers. Missing evidence is not converted into a zero or a pass.
- posture
- UNKNOWN, OBSERVED, INFERRED, ATTESTED, or VERIFIED
- scope
- The boundary inside which the statement applies
- evidence_refs
- Records that support the statement
- known_gaps
- What the available evidence does not establish
{
"schema_version": "goldtrace.claim.v1",
"claim_id": "claim:pack-structure",
"statement": "The scenario pack passes structural validation.",
"posture": "OBSERVED",
"scope": "Files, policy, and configuration only.",
"evidence_refs": [
"scenario.json@sha256:01c909…",
"policy.json@sha256:8486a0…"
],
"known_gaps": [
"Runtime execution was not performed."
]
}goldtrace.provenance.v1
Product provenance
A dataset artifact points back to its inputs and records each transform. Quality, privacy, rights, and release status remain separate.
- artifact
- The exact output identity and digest
- upstream
- Source packs, bundles, and admitted records
- transforms
- Parse, redact, merge, grade, split, and package steps
- disposition
- Custody and release state without implied quality
{
"schema_version": "goldtrace.provenance.v1",
"artifact": {
"id": "dataset:example-bar",
"kind": "dataset",
"digest": "sha256:bbbb…"
},
"upstream": [
"forge-pack:example",
"arena-bundle:example"
],
"transforms": [
"privacy-review",
"deduplication",
"rubric-evaluation"
],
"disposition": "CURATED_CANDIDATE"
}goldtrace.rubric-bundle.v1
Rubric composition
Every evaluation binds a common core, a domain-specific overlay, and an evaluation protocol where one applies. The intended outcome stays attached.
- heart
- The cross-domain core and its hard gates
- domain_overlay
- Criteria tuned to the intended outcome
- eval_protocol
- The evaluator, oracle, or comparison source
- context
- What, why, who, when, where, and method
{
"schema_version": "goldtrace.rubric-bundle.v1",
"heart": "goldtrace-heart@1.0.0",
"domain_overlay": "cyber-red-blue@0.0.1",
"eval_protocol": "authorized-range@0.0.1",
"evaluation_context": {
"what": "An admitted simulation trace",
"why": "Defensive-agent training data"
},
"status": "DRAFT"
}Field note · Evidence
What did the agent actually do?
A configuration check, a permitted action, and an observed runtime result are different facts. The record should not blur them together.
If an agent says, “I scanned the network and found an exposed service,” the sentence can describe several different events. The model may have proposed a scan that policy rejected. The command may have run and failed. It may have completed without producing useful evidence. A transcript can make those outcomes look the same.
The owned-network investigation fixture provides a concrete example. It describes a disposable range, restricts the actor to that range, keeps verifier-only material away from the agent, and requires action and observation evidence. Its structural validator checks that the scenario, policy, event, capture, and tool contracts agree.
The validator passed. The containerized range did not run in the authoring environment. Both facts belong in the record.
The validation result supports a narrow claim about the pack’s structure. It does not support a claim about containment, agent behavior, discovered services, or task completion. Those require an admitted Arena run and the evidence produced by that run.
{
"schema_version": "goldtrace.claim.v1",
"claim_id": "claim:owned-network-pack-structure",
"statement": "The owned-network investigation pack passes its structural validator.",
"posture": "OBSERVED",
"scope": "Scenario-pack files, identifiers, policy, and configuration.",
"method": "python3 scripts/validate_pack.py .",
"known_gaps": [
"The containerized range was not started.",
"Runtime isolation was not exercised.",
"No agent investigation or findings were observed."
]
}This separation matters once traces become training data. “The model planned to call a tool” and “the tool ran with these arguments and returned this result” teach different things. Treating them as equivalent creates training records that sound confident but cannot be checked.
Gold Trace keeps the proposal, policy decision, dispatch, result, observation, and evidence gaps as separate events. Dataworks can evaluate and package the trace after those records exist. Until then, the fixture is a structurally valid scenario pack, not a successful investigation.
Gold Trace Dataworks