> ## Documentation Index
> Fetch the complete documentation index at: https://engramviz.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Portable regression tests

> Turn a verified memory incident into executable retrieval, context, and answer assertions.

A verified incident exports an `engram.memory-regression` artifact. Version 2 declares semantic lifecycle and answer assertions plus a controlled perturbation matrix, without claiming access to hidden model reasoning.

## Run with the Studio executor

Your project owns the executor boundary. The recommended LangGraph setup puts
that module in `engram.config.json`, so Studio and CI import the same code:

```json theme={"dark"}
{
  "version": 1,
  "framework": "langgraph",
  "executor": "engram.executor.mjs",
  "regressions": ["regressions"]
}
```

For custom integrations, an executor can also be a legacy function:

```js theme={"dark"}
export default async function run({ source, variant }) {
  const result = await runAgentVersionUnderTest({
    memories: source.memoryState.before,
    userMessage: source.input
  });

  return {
    variantId: variant.id,
    memories: result.memories,
    answer: result.answer,
    selectedMemoryIds: result.selectedMemoryIds,
    loadedMemoryIds: result.loadedMemoryIds,
    forcedMemoryIds: result.forcedMemoryIds ?? []
  };
}
```

Execute the artifact:

```bash theme={"dark"}
npx --yes @engramviz/cli test incident.engram-test.json \
  --executor ./engram-regression.mjs \
  --format pretty
```

With project configuration, omit both paths and run the entire suite:

```bash theme={"dark"}
npx engram test
```

The command invokes the executor once per matrix variant and exits nonzero when
any required selection, exclusion, context load, or answer assertion fails.
Failed findings identify the variant and show expected and observed values.

## Check a recorded observation

When another system already produced the matrix results, compare the observation
array directly:

```bash theme={"dark"}
npx --yes @engramviz/cli test incident.engram-test.json \
  --observation ./latest-observation.json
```

## Run in CI

```yaml theme={"dark"}
- run: npm ci
- run: >-
    npx engram test
    --format github
    --output engram-regression-report.json
- if: always()
  uses: actions/upload-artifact@v7
  with:
    name: engram-memory-regression
    path: engram-regression-report.json
```

The repository includes a complete copyable workflow at
[`examples/github-actions/memory-regression.yml`](https://github.com/Meyk0/engram-viz/blob/main/examples/github-actions/memory-regression.yml).

<Info>
  Version 1 artifacts and their ID-based executor contract remain supported. A
  version 2 artifact proves that every supplied matrix variant passed observable
  semantic assertions. Production equivalence exists only when the executor calls
  the production-equivalent retrieval and generation stack.
</Info>