workspace-ci
Installation
Entrypoint
Types
WorkspaceCi

workspace-ci

Anyone maintaining a repository of modules ends up writing the same CI by
hand: enumerate the checks, work out which ones a change could affect, route
each to the module that owns it, and avoid re-running what a previous run
already proved good. This module is that engine. It reads the workspace it is
invoked from, diffs a commit range, and returns the checks to run — already
routed, with timeouts and memoization hashes applied — so a CI system needs one
call and at most a format shim.

Nothing here loads a module the plan does not need. Checks are enumerated per
module (Module.checks), never through a root aggregator that installs every
suite as a toolchain, and the run-everything path emits one leg per module so
it loads none at all. See README.md for what counts as a change, what is never
memoized, and how base-image drift is bounded.

Installation

dagger install github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb

Entrypoint

Return Type

WorkspaceCi !

Arguments

Name	Type	Default Value	Description
globalPaths	[String ! ]	-	Repo-relative path prefixes that govern how CI runs rather than what any check computes; a change to one runs everything. They belong to no module's source context, so nothing else would attribute them. Defaults to .github/workflows/, which costs nothing in a workspace that has none.
splitModules	[String ! ]	-	Repo-relative directories of modules whose checks must each get their own leg even when everything runs. The run-everything path otherwise emits one leg per module, which is right when a module's checks share their containers and wrong when each one boots a stack of its own: those land in a single engine on a single runner. Splitting a module costs loading it — the one thing that path exists to avoid — so name only the modules that need it.
timeouts	String	"{}"	Per-leg check-step budgets in minutes, as a JSON object keyed by a leg's display name, by a module directory (which covers every leg of that module), or by "<module-dir>:" (which covers that module's coarse run-everything leg and none of its per-check legs, since a coarse leg's display name is* its module directory). It is JSON because Dagger function parameters cannot be Go maps.
defaultTimeout	Integer	6	The check-step budget in minutes for a leg with no override.
memoStore	Enum	"ACTIONS_CACHE"	Where recorded passes live. ACTIONS_CACHE is read-only from this module and leaves recording to an actions/cache/save step; GIT_REFS is a store the module owns both sides of, so RecordPass can write it from anywhere a token reaches. See README.md — the two have different trust arguments.
memoToken	Secret	-	A credential for the memoization store: a GitHub token on memoRepo with actions:read for ACTIONS_CACHE, or contents:read for GIT_REFS — plus contents:write if this run is to record anything.
memoRepo	String	-	The owner/name whose Actions cache, or whose git refs, hold the memoization store.
memoApi	String	-	The GitHub API root the store is reached through. Defaults to https://api.github.com; set it for GitHub Enterprise Server.
memoRefs	[String ! ]	-	The git refs whose scopes may be trusted to hold recorded passes, spelled in full (refs/heads/main, refs/pull/12/merge). They are the refs a plan reads, and the only refs RecordPass will write from. Defaults to none, which reads nothing and records nothing: a scope a run can write is a scope that must be chosen deliberately.
memoTtl	Integer	86400	How long, in seconds, a recorded pass may be honoured. This is the answer to base-image drift, which a source-derived hash cannot see.

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \

func (m *MyModule) Example() *dagger.WorkspaceCi  {
	return dag.
			Workspaceci()
}

@function
def example() -> dagger.WorkspaceCi:
	return (
		dag.workspace_ci()
	)

@func()
example(): WorkspaceCi {
	return dag
		.workspaceCi()
}

Types

WorkspaceCi 🔗

WorkspaceCi plans CI for the workspace it is invoked from.

affectedModules() 🔗

AffectedModules returns, as a JSON array of repo-relative directories, the modules whose checks a change could affect. It is the same attribution Plan applies, stopping before any module is loaded, and answers “what did this change reach” without paying for check enumeration.

The arguments mean what they mean on Plan.

Return Type

String !

Arguments

Name	Type	Default Value	Description
base	String !	-	The revision the change is measured from: a commit SHA, a branch or tag name, HEAD, or any of those with git’s ~ and ^ suffixes.
head	String !	-	The revision the change is measured to, in the same forms as base.
repo	Directory	-	The repository to plan for. Defaults to the calling workspace.
workspace	Workspace	-	The workspace to read repo from when repo is omitted. Defaults to the caller’s.

Example

echo 'Custom types are not supported in shell examples'

func (m *MyModule) Example(ctx context.Context, base string, head string) string  {
	return dag.
			Workspaceci().
			Affectedmodules(ctx, base, head)
}

@function
async def example(base: str, head: str) -> str:
	return await (
		dag.workspace_ci()
		.affectedmodules(base, head)
	)

@func()
async example(base: string, head: string): Promise<string> {
	return dag
		.workspaceCi()
		.affectedModules(base, head)
}

generated() 🔗

Generated verifies that every committed dagger.gen.go and internal/dagger/*.gen.go in the calling workspace matches what dagger develop would produce at each module’s pinned engineVersion.

Every module in the workspace is checked, including the root one and every tests or examples module.

This check is why generated files need not be global inputs to the memoization hash: it proves they are derived from inputs that are, it belongs to the root module so a plan always runs it, and it is never memoized. The result is deliberately never cached either — the workspace is read at call time rather than passed as an argument, so a cached pass would be a pass for a tree the check never looked at.

Return Type

Void !

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \
 generated

func (m *MyModule) Example(ctx context.Context)   {
	return dag.
			Workspaceci().
			Generated(ctx)
}

@function
async def example() -> None:
	return await (
		dag.workspace_ci()
		.generated()
	)

@func()
async example(): Promise<void> {
	return dag
		.workspaceCi()
		.generated()
}

generatedSelfTest() 🔗

GeneratedSelfTest pins that Generated can actually fail.

The check this repo extracted it from silently verified nothing for months (it routed through Workspace.Generators, which is empty unless a module declares a +generator function), so a green Generated is only worth as much as the proof that a stale module turns it red (#184).

It runs the same codegen comparison against a single module, first pristine (expecting no drift) and then with that module’s committed bindings deliberately made stale (expecting drift naming the file).

Return Type

Void !

Arguments

Name	Type	Default Value	Description
probeModule	String	-	The module to make stale, repo-relative. Defaults to the first dependency-free module in the workspace, which is the cheapest one to regenerate.

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \
 generated-self-test

func (m *MyModule) Example(ctx context.Context)   {
	return dag.
			Workspaceci().
			Generatedselftest(ctx)
}

@function
async def example() -> None:
	return await (
		dag.workspace_ci()
		.generatedselftest()
	)

@func()
async example(): Promise<void> {
	return dag
		.workspaceCi()
		.generatedSelfTest()
}

memoStoreSelfTest() 🔗

MemoStoreSelfTest verifies the store this module owns both sides of — that a pass is recorded under its own ref’s scope, that recording the same hash twice writes nothing and refreshes no TTL, that entries read back, that one older than the TTL does not, and that one ref’s scope stays out of another’s — against an in-process stub of GitHub’s API.

It is a check rather than only a Go test because this is the half of memoization that fails silently: a store that quietly takes nothing costs every later run its full time and looks exactly like a workspace nobody has recorded against yet, and a scope that leaks costs correctness. Like SelectionSelfTest it runs in-process and needs no network, no credential and no services.

Return Type

Void !

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \
 memo-store-self-test

func (m *MyModule) Example(ctx context.Context)   {
	return dag.
			Workspaceci().
			Memostoreselftest(ctx)
}

@function
async def example() -> None:
	return await (
		dag.workspace_ci()
		.memostoreselftest()
	)

@func()
async example(): Promise<void> {
	return dag
		.workspaceCi()
		.memoStoreSelfTest()
}

plan() 🔗

Plan returns the legs of CI to run for a change, each already routed to the module that owns it and bounded by a timeout.

Each leg is a {name, module, filter, hash, timeout, jobTimeout} object: the display name, the repo-relative module to invoke with -m, the check pattern to pass to dagger check (empty to run every check the module has), the input hash a pass may be recorded under (empty means never memoize), and the step and job budgets in minutes.

base and head are the revisions to diff, three-dot (merge-base) like a PR’s change set. Either may be written in any form git’s rev-parse takes — a full or abbreviated commit SHA, a branch or tag name, HEAD, or those with ~ and ^ suffixes — so CI can pass the SHAs its event payload carries and a person can pass --base=main --head=HEAD. Either side empty or all-zeros — a new branch, a missing base — means “run everything”, and so does a revision this repository cannot resolve.

A plan that cannot read the workspace is an error, never an empty plan: an empty matrix skips the run job and passes the gate having run nothing. Everything else fails safe towards running too much — an unusable diff range, an unreadable source context, a module whose checks cannot be enumerated.

repo defaults to the calling workspace and is where everything is read from: module discovery is a dagger.json walk, source contexts and check enumeration work off the exported tree, and the change set comes from its .git. Passing it explicitly is also the escape hatch for a caller whose .git is a file rather than a directory (a git worktree), which would otherwise degrade to running everything.

Return Type

String !

Arguments

Name	Type	Default Value	Description
base	String !	-	The revision the change is measured from: a commit SHA, a branch or tag name, HEAD, or any of those with git’s ~ and ^ suffixes.
head	String !	-	The revision the change is measured to, in the same forms as base.
format	Enum	"JSON"	No description provided
repo	Directory	-	The repository to plan for. Defaults to the calling workspace.
workspace	Workspace	-	The workspace to read repo from when repo is omitted. Defaults to the caller’s.
knownGood	String	"[]"	Input hashes a previous run already proved good, as a JSON array. They are honoured on the same terms as the ones read from the memoization store, and are how a CI system that reads its own store — or a test — supplies them without one. Anything unparseable is treated as empty: a store that cannot be read must cost speed, never correctness.
recordCommand	String	-	The command a JENKINS branch runs to record its own pass, with `--hash=<that leg's hash>` appended — conventionally a `record-pass` call complete but for the hash: `dagger -m <this module> --memo-store=GIT_REFS --memo-repo=<repo> --memo-token=env:GH_TOKEN --memo-refs=refs/heads/main call record-pass --ref="$GIT_REF" --commit="$GIT_COMMIT"` It is a whole command rather than a set of fields because the credential is in it: rendering a token into a plan a pipeline writes to disk and `load`s is not something this module should ever do, and the pipeline already knows how to name its own. Nothing is rendered for a leg with no hash, and nothing runs for a branch whose check failed — see README.md. Only the JENKINS form takes one; every other format carries each leg’s hash as data for the surrounding job to record, so passing it with those is an error rather than a silent no-op.
diagnostics	Boolean	-	Emit a diagnostics object — the plan plus which modules had to be loaded to produce it, whether everything was selected, which legs a recorded pass retired, and whether recorded passes were honoured at all — instead of the bare plan. Intended for tests and for explaining a plan, not for CI.

Example

echo 'Custom types are not supported in shell examples'

func (m *MyModule) Example(ctx context.Context, base string, head string) string  {
	return dag.
			Workspaceci().
			Plan(ctx, base, head)
}

@function
async def example(base: str, head: str) -> str:
	return await (
		dag.workspace_ci()
		.plan(base, head)
	)

@func()
async example(base: string, head: string): Promise<string> {
	return dag
		.workspaceCi()
		.plan(base, head)
}

recordPass() 🔗

RecordPass records that the leg whose inputs hashed to hash passed, so a later run that computes the same hash can skip it. It is the write half of what Plan reads, for the stores this module owns both sides of.

It reports what it did, as one word, and never fails a check that passed. Recording happens after the work is already green, so a store that will not take the entry has to cost a later run its time and nothing else. A caller who wants a store problem to be loud should compare the returned word:

RECORDED         a new entry now names this hash
ALREADY_RECORDED an earlier run got there; nothing was written or refreshed
REFUSED          ref is not one of memoRefs, so this scope is not writable
SKIPPED          nothing to record: an empty hash, or no store configured
UNSUPPORTED      the configured store cannot be written from this module
FAILED           the store would not take the entry; stderr says why

The error return is not how a store problem is reported. It carries exactly one thing: a call that named no ref, because with no ref there is no scope to judge and refusing silently would be indistinguishable from a scope that was judged and rejected.

Return Type

String !

Arguments

Name	Type	Default Value	Description
hash	String !	-	The leg’s input hash, exactly as Plan emitted it. An empty hash is a leg that may never be memoized, and recording one is a no-op.
ref	String !	-	The git ref the run that passed is on, spelled the way memoRefs spells it — refs/heads/main, refs/pull/12/merge. Nothing is written unless it is one of them.
commit	String !	-	The commit whose checks passed. The entry points at it, so `git log` on a recorded ref shows what proved the hash.

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \
 record-pass --hash string --ref string --commit string

func (m *MyModule) Example(ctx context.Context, hash string, ref string, commit string) string  {
	return dag.
			Workspaceci().
			Recordpass(ctx, hash, ref, commit)
}

@function
async def example(hash: str, ref: str, commit: str) -> str:
	return await (
		dag.workspace_ci()
		.recordpass(hash, ref, commit)
	)

@func()
async example(hash: string, ref: string, commit: string): Promise<string> {
	return dag
		.workspaceCi()
		.recordPass(hash, ref, commit)
}

selectionSelfTest() 🔗

SelectionSelfTest verifies the change -> modules -> legs mapping, the properties a recorded pass depends on, and the shape each format renders a plan in, against fixed fixtures — so a regression in any of them fails CI rather than silently under-running a consumer’s checks or handing their CI system something it cannot parse. It runs in-process and needs no services, so it is cheap enough to run on every leg set.

Return Type

Void !

Example

dagger -m github.com/z5labs/devex/daggerverse/workspace-ci@c10a12007999eb807f68cd9af9000fbaeab159cb call \
 selection-self-test

func (m *MyModule) Example(ctx context.Context)   {
	return dag.
			Workspaceci().
			Selectionselftest(ctx)
}

@function
async def example() -> None:
	return await (
		dag.workspace_ci()
		.selectionselftest()
	)

@func()
async example(): Promise<void> {
	return dag
		.workspaceCi()
		.selectionSelfTest()
}