Test parts in CI

Write a test that loads a part and states what it must be, read a pass, a failure and a refusal in the run, and run it on every push.

Turn a part into a test suite: load the exported STEP or GLB, state what it must be, and read the outcome from the run. A failed or refused claim fails its test, so the run reports it.

npm install --save-dev geospec@beta

pip install geospec-engine pytest

1. Write the test

geospec run discovers *.geospec.ts files under the directory it runs in and supplies the engine, so a spec imports no engine and has no hooks; loadModel reads source relative to that directory. A test that returns has passed.

plate.geospec.ts

import { describe, expectGeo, it } from 'geospec';
import { loadModel } from 'geospec/model';

describe('plate', () => {
  it('is 40 x 40 x 20 with a 10 mm bore', async () => {
    const plate = await loadModel({ source: 'plate-with-through-bore.step', format: 'step' });
    expectGeo(plate).toHaveBoundingBox({ size: { x: 40, y: 40, z: 20 } });
    expectGeo(plate).toHaveCircularHole({ diameter: 10 });
  });

  it('has no 12 mm bore', async () => {
    const plate = await loadModel({ source: 'plate-with-through-bore.step', format: 'step' });
    expectGeo(plate).toHaveCircularHole({ diameter: 12 });
  });

  it('refuses a hole claim on the gear mesh', async () => {
    const gear = await loadModel({ source: 'involute-gear.glb', format: 'glb', sourceUnit: 'mm' });
    expectGeo(gear).toHaveCircularHole({ diameter: 4 });
  });
});

The pytest plugin gives each test an engine rooted at pytest's rootpath. expect_geo returns a report on a pass and raises GeoSpecAssertionError otherwise, so a test that returns has passed; a millimetre GLB declares source_unit="mm".

test_plate.py

from geospec import expect_geo, load_model


def test_plate_size_and_bore():
    plate = load_model(file="plate-with-through-bore.step", format="step")
    expect_geo(plate).to_have_bounding_box(size={"x": 40, "y": 40, "z": 20})
    expect_geo(plate).to_have_circular_hole(diameter=10)


def test_plate_has_no_12_mm_bore():
    plate = load_model(file="plate-with-through-bore.step", format="step")
    expect_geo(plate).to_have_circular_hole(diameter=12)


def test_gear_mesh_refuses_a_hole_claim():
    gear = load_model(file="involute-gear.glb", format="glb", source_unit="mm")
    expect_geo(gear).to_have_circular_hole(diameter=4)

Expected output

The plate has a 10 mm bore, so the second test fails; the gear is a triangle mesh with no hole evidence, so the third is refused. The runner prints each message under its test.

$ npx geospec run .
FAIL plate.geospec.ts
  fail plate > has no 12 mm bore
    GEOSPEC_FEATURE_MISMATCH: No circular hole matches diameter 12 within 0.02 mm.
  fail plate > refuses a hole claim on the gear mesh
    GEOSPEC_UNSUPPORTED_EVIDENCE: expectGeo(...).toHaveCircularHole() needs exact BRep circular-hole evidence, which this subject does not carry.
1 passed, 1 failed, 3 selected

The exit code is 1. The refused test is inconclusive: it is in neither count on the last line, and a run with one never succeeds.

$ pytest -q --tb=short
.FF                                                                      [100%]
=================================== FAILURES ===================================
_________________________ test_plate_has_no_12_mm_bore _________________________
test_plate.py:12: in test_plate_has_no_12_mm_bore
    expect_geo(plate).to_have_circular_hole(diameter=12)
.venv/lib/python3.13/site-packages/geospec/_api.py:742: in assertion
    raise GeoSpecAssertionError.from_report(report)
E   geospec._api.GeoSpecAssertionError: No circular hole matches diameter 12 within 0.02 mm.
_____________________ test_gear_mesh_refuses_a_hole_claim ______________________
test_plate.py:17: in test_gear_mesh_refuses_a_hole_claim
    expect_geo(gear).to_have_circular_hole(diameter=4)
.venv/lib/python3.13/site-packages/geospec/_api.py:742: in assertion
    raise GeoSpecAssertionError.from_report(report)
E   geospec._api.GeoSpecAssertionError: expectGeo(...).toHaveCircularHole() needs exact BRep circular-hole evidence, which this subject does not carry.
=========================== short test summary info ============================
FAILED test_plate.py::test_plate_has_no_12_mm_bore - geospec._api.GeoSpecAsse...
FAILED test_plate.py::test_gear_mesh_refuses_a_hole_claim - geospec._api.GeoS...
2 failed, 1 passed in 0.08s

A failure names what was measured; a refusal names the evidence the subject lacks. Fix the model for the first and the subject for the second: admit the gear's STEP export, or claim what a mesh can answer.

2. Read the status, then the code

A refusal never counts as a pass. The report's status tells a refusal from a failure, and the first diagnostic's code names the reason.

--json prints the report in place of the lines above; pipe it to jq to read it here, and keep the CI step unpiped so its exit code stands. Its accounting counts the refused test as inconclusive, and each test entry carries its status, its diagnostics and its claim reports, canonical bytes included. No rewrite turns the third test green under geospec run: give the claim the STEP export, or claim what the mesh answers and report the requirement as unsupported.

$ npx geospec run . --json | jq .accounting
{
  "requestedFiles": [
    "plate.geospec.ts"
  ],
  "completedFiles": [
    "plate.geospec.ts"
  ],
  "notRunFiles": [],
  "discoveryComplete": true,
  "discovered": 3,
  "selected": 3,
  "completed": 3,
  "passed": 1,
  "failed": 1,
  "unsupported": 0,
  "inconclusive": 1,
  "skipped": 0,
  "notRun": 0,
  "cancelled": false,
  "bailed": false
}

Rewritten this way, the third test passes and the run reports one failure, the 12 mm bore. With pytest and GeoSpecAssertionError imported:

def test_gear_mesh_refuses_a_hole_claim():
    gear = load_model(file="involute-gear.glb", format="glb", source_unit="mm")
    with pytest.raises(GeoSpecAssertionError) as raised:
        expect_geo(gear).to_have_circular_hole(diameter=4)
    assert raised.value.result["status"] == "refused"
    assert raised.value.code == "GEOSPEC_UNSUPPORTED_EVIDENCE"

3. Run it in CI

The step's exit code is the verdict: 0 only when every selected test passed.

.github/workflows/geospec.yml

on: [push, pull_request]
jobs:
  geospec:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - run: npm ci
      - run: npx geospec run .

Wheels cover CPython 3.13 and 3.14 on manylinux_2_28 x86_64 and aarch64 and on macOS arm64.

.github/workflows/geospec.yml

on: [push, pull_request]
jobs:
  geospec:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-python@v7
        with:
          python-version: '3.13'
      - run: pip install geospec-engine pytest
      - run: pytest -q --tb=short

Variations

Vitest instead of geospec run. geospec/vitest registers the matchers on Vitest's expect, over an engine the file creates: npm install --save-dev vitest@4 geospec@beta geospec-engine. A matcher returns a promise, so await it. A failed or refused claim fails the test with the claim's result as its message, and a refusal fails under .not too.

plate.test.ts

import { readFile } from 'node:fs/promises';
import { afterAll, describe, expect, it } from 'vitest';
import { createModelLoader } from 'geospec/model';
import { setupGeoSpecVitest } from 'geospec/vitest';
import { Engine } from 'geospec-engine/node';

const engine = new Engine();
const loadModel = createModelLoader({
  engine,
  readSource: async (source) => new Uint8Array(await readFile(source as string)),
});
setupGeoSpecVitest({ engine });
afterAll(async () => {
  await loadModel.dispose();
  engine.close();
});

describe('plate', () => {
  it('is 40 x 40 x 20 with a 10 mm bore', async () => {
    const plate = await loadModel({ source: 'plate-with-through-bore.step', format: 'step' });
    await expect(plate).toHaveBoundingBox({ size: { x: 40, y: 40, z: 20 } });
    await expect(plate).toHaveCircularHole({ diameter: 10 });
  });

  it('has no 12 mm bore', async () => {
    const plate = await loadModel({ source: 'plate-with-through-bore.step', format: 'step' });
    await expect(plate).toHaveCircularHole({ diameter: 12 });
  });

  it('refuses a hole claim on the gear mesh', async () => {
    const gear = await loadModel({ source: 'involute-gear.glb', format: 'glb', sourceUnit: 'mm' });
    await expect(gear).toHaveCircularHole({ diameter: 4 });
  });
});

The result JSON after each Error: and the diff Vitest prints under it are cut below.

$ npx vitest run
 ❯ plate.test.ts (3 tests | 2 failed) 58ms
     × has no 12 mm bore 23ms
     × refuses a hole claim on the gear mesh 7ms

 FAIL  plate.test.ts > plate > has no 12 mm bore
Error: {"claimId":"geospec-claim-3","diagnostics":[{"code":"GEOSPEC_FEATURE_MISMATCH",…

 FAIL  plate.test.ts > plate > refuses a hole claim on the gear mesh
Error: {"claimId":"geospec-claim-4","diagnostics":[{"code":"GEOSPEC_UNSUPPORTED_EVIDENCE",…

 Test Files  1 failed (1)
      Tests  2 failed | 1 passed (3)

pytest is the runner; the wheel's plugin supplies the engine and needs no adapter.

A budget for the suite. TypeScript sets workUnitLimit per chain or per loaded model, and the host limits where it constructs the Engine. Python overrides the plugin's geospec_engine_factory fixture in conftest.py to build GeoSpecEngine(model_root=..., work_unit_budget=..., limits=...). Set tolerances and budgets explains budgets and limits.