Reliable API automation

This page defines the safe operating contract for a developer or AI agent that changes a Moltaro workspace through its APIs. It starts after the workspace owner has prepared access in the Moltaro Portal or administrative Web Application interface.

Workspace setup is an administrator task

Installing or registering the Web Application, creating or rotating credentials, and preparing Agent integration are not part of the agent workflow. If the handoff below is incomplete, the agent must stop and ask the user to complete Agent integration in the Portal or administrative interface. It must not try to provision its own access through workspace APIs.

For a lost Entity mutation response with a previously saved handle, use Read a mutation confirmation. Bind the handle to the installation and actual user before the first send. The read-only check reports confirmation or no confirmed receipt; it supplies no automatic retry and cannot reconstruct a lost handle or original body. Failures of the check do not establish the outcome of the earlier write.

Required handoff

The workspace owner gives the developer or agent:

1. the downloaded workspace-specific AGENTS.md or CLAUDE.md;
2. the exact WORKSPACE_API_BASE_URL;
3. an API key through an appropriate secret channel;
4. a task and an account with the narrowest roles needed for that task.

The guide must not contain the key. A missing URL, guide, key, or permission is an administrative prerequisite, not an API-discovery problem.

This prerequisite boundary does not remove the configured Application’s administrative APIs from the agent workflow. Once authenticated with the provided roles, the agent uses documented /api/workspace/admin/... Configuration API routes for entity definitions, Net Operation Project, Boards and Entitlement configuration, Agent integration, and Workspace UI. Those are workspace Application endpoints. Portal administration endpoints are a different surface and are not part of the developer contract.

Keep the public and workspace origins visibly separate in scripts and prompts:

PUBLIC_DOCS_BASE=https://moltaro.com
WORKSPACE_API_BASE_URL=https://customer-workspace-api.example.com

PUBLIC_DOCS_BASE is for documentation only. Every /api/workspace/... and installation-local /openapi/... path resolves against WORKSPACE_API_BASE_URL. The Web Application used by people may have another origin; never derive the API hostname from it.

Preflight every workspace

Before changing anything:

GET ${WORKSPACE_API_BASE_URL}/api/workspace/context
GET ${WORKSPACE_API_BASE_URL}/api/workspace/auth/me
GET ${WORKSPACE_API_BASE_URL}/openapi/moltaro-public-v1.json
GET ${WORKSPACE_API_BASE_URL}/openapi/moltaro-config-v1.json
GET ${WORKSPACE_API_BASE_URL}/api/workspace/error-catalog
GET ${WORKSPACE_API_BASE_URL}/api/workspace/admin/net-operation-project/developer-surface

The error catalog needs no administrative role; any active user or Service user may read it. See Installed error catalog.

After this handoff, the authenticated Application endpoints below may also be used to refresh the installation manifest or generated guide:

GET ${WORKSPACE_API_BASE_URL}/api/workspace/admin/agent-integration
GET ${WORKSPACE_API_BASE_URL}/api/workspace/admin/agent-integration/agents-md

They do not discover an unknown workspace; they require the API base URL and authorized key that the user already supplied.

Read the workspace locale and time zone, the caller’s roles and module states, the installed API contract, the installed error catalog, and the supported C# surface. Treat the installation-local OpenAPI documents, error catalog and developer-surface response as authoritative for that workspace. Do not infer an endpoint, enum value, injectable service, or module availability from a different release or from a public CLR type.

Safe operating loop

Use the same sequence for schema, records, C# logic, Boards, Entitlement Operations, and custom pages:

1. Discover the installed contract, entity schema, module state, current revision, and permissions.
2. Read before writing. Capture ids, stable keys, row versions, source checksums, and file hashes from current responses.
3. Preview where supported. Use schema-change plans, source-template preview, page-generator preview, validation, and language diagnostics before creating a revision or changing data.
4. Make one bounded change. Keep a stable external correlation or idempotency key when the operation supports one.
5. Verify the saved state. Read the entity, revision, source index, board item, entitlement history, or generated artifact back from the API.
6. Check before publishing. Net Operation Project Build activates on success; Workspace UI Build does not. Use their exact lifecycles below.
7. Verify the runtime result and history. Check the Function Catalog or active artifact, invoke only when requested, and read the relevant audit or ledger surface.

Do not delete schema, purge artifacts, deactivate pages, rotate credentials, or activate code merely because an endpoint exists. Those actions require an explicit task. For a destructive schema change, inspect the server-provided change plan and surface its data-loss warnings before proceeding.

Concurrency and HTTP 409

Moltaro rejects stale writes instead of silently overwriting newer work:

• record updates use the last observed RowVersion; an update that writes a field issues a new RowVersion even when the sent values equal the stored ones (an exact retry returns the RowVersion issued by the first attempt), so send only intended field changes;
• Net Operation Project manual edits use BaseRevisionId, BaseSourceChecksum, and ExpectedContentHash for every changed existing file;
• Workspace UI source edits use the same base revision/checksum pattern and file hashes where the request schema requires them;
• artifact activation, deactivation, rollback, and purge use expected project and artifact row versions.

On a stale version, do not resend the old body. Re-read the object or source tree, compare the intervening change with the intended change, merge when safe, and submit a new request using the fresh concurrency values. If the correct merge is ambiguous, stop and ask the user.

HTTP 409 is not always a stale version: branch on the error code. For Entity records, moltaro.security.concurrency.staleRowVersion is the stale version; moltaro.instances.conflict means that another write held the record at the same moment and wrote nothing, so the identical request may be repeated after a delay; moltaro.security.idempotency.keyReuse and moltaro.security.idempotency.replayInvalid are replay answers described below. The installed error catalog gives the reaction for each code of the installation you call.

Retries, idempotency, and stable keys

The shared mutation receipt journal retains successful results for seven days. It has no operation-count limit: the number of retained receipts never refuses a new key, for a workspace or for one user or integration account. Expired receipts are cleaned automatically in the background, independently of incoming requests. After the retention window, first reconcile the target state before retrying an old key; the original result may no longer be available for replay.

In .NET command results, error details are available in SecurityMutationError.Metadata; Arguments remains available for message arguments. For import and bulk commands, inspect each item result as well as the outer command result.

• Retry reads after a transient transport failure.
• Do not blindly retry a mutation after an unknown outcome. First read the target state or use the operation’s documented idempotency key. A single-record Entity create, update, archive, restore or delete retried with the same OperationKey and body returns the original result with IsIdempotentReplay: true, over HTTP as in the .NET SDK. If the caller can no longer read the record, the retry answers a receipt without record data; a Create retry answers HTTP 403 moltaro.instances.replay.resultNotViewable instead, because the record exists: never repeat it with a new key. The same key with a different body or optional X-Moltaro-Markdown-* header fails with HTTP 409 moltaro.security.idempotency.keyReuse; a stored result that can no longer be read fails with HTTP 409 moltaro.security.idempotency.replayInvalid and is never executed again. A check that runs before the replay, such as access, the record type’s state or a disabled operation, can reject the retry with another error; that error does not settle whether the first attempt committed.
• A composite Entity mutation has one OperationKey for all its steps. The identical request returns the stored result with IsIdempotentReplay: true, a RecordReceipt step for an updated record the caller can no longer read, and HTTP 403 moltaro.instances.replay.resultNotViewable for the whole answer when a created record is no longer readable. The key is shared by the same user’s REST request and C# IEntityCompositeMutationService call.
• Boards automation commands mirror Runtime API semantics and do not have a separate durable receipt. After an unknown add outcome, query the open item; after an unknown mutation outcome, read the item and compare its current state and RowVersion before deciding whether another command is needed.
• Reuse the same durable idempotency key when retrying the same logical Entitlement grant, renewal, lifecycle, consume, reverse, adjustment, or renewal-operation command. Do not generate a new key for each attempt. The Entitlement facade reports a native durable replay as Replayed = true.
• A source template’s StableKey identifies the published function or data source across revisions. Read the template catalog to learn whether it is required; do not derive it from a display name.
• A command CorrelationId connects one external operation with its run and downstream history. It is observability context, not a substitute for an operation-specific idempotency key.
• Validation, permission, not-found, and stale-version responses require a corrected request or fresh state, not automatic retry. Repeat an Entity mutation unchanged only when its catalog reaction has AutomaticRetry SameRequest and every value of the reaction is one your client recognizes, such as moltaro.instances.conflict, HTTP 503 moltaro.referenceEligibility.metadataState.unavailable, or, for a composite mutation, 503 moltaro.instances.composite.budgetExceeded (nothing was saved) and moltaro.instances.composite.outcomeUnknown (the repeat settles whether it saved), only a few times with a delay (these answers carry no retry delay), and only while the first attempt of the operation is less than seven days old. Keep that time with the OperationKey; when it is unknown, do not repeat automatically.

Polling checks, builds, and jobs

Queue a build once, retain its returned id, and poll that exact resource with a bounded delay and overall client deadline. Do not queue duplicate builds because a build remains Queued or Running longer than expected.

First establish that the operation is actually queued. A C# Task, an Async method name, or HTTP 200 is not an execution-mode signal: Moltaro enqueue endpoints return a normal success envelope containing a queued resource. Use Function Catalog contract, binding source, API publication kind, schedule metadata, and the response schema. The complete discovery table, CRON flow, job/run distinction, and polling algorithm are in Asynchronous operations and polling.

Both project build status enums use Queued = 0, Running = 1, Succeeded = 2, Failed = 3, and Cancelled = 4. 2, 3, and 4 are terminal. On failure, read the stored diagnostics rather than immediately submitting an identical build.

The build kinds and activation behavior are different:

Project	Check	Build	Activation
Net Operation Project	1	0	A successful Build automatically replaces the active C# artifact. A failed Build leaves the previous artifact active.
Workspace UI Project	0	1	Build creates an inactive artifact. Activation is a separate, row-version-protected operation; rollback can restore a compatible prior artifact.

Function jobs use Queued = 0, Leased = 1, Completed = 2, Failed = 3, and Cancelled = 4. Poll the job id returned by enqueue; do not enqueue the same business operation again simply because it is still queued or leased.

For recurring work, create one Function Schedule through the Configuration API instead of running a client-side timer that repeatedly calls a manual enqueue endpoint. The schedule processor leases each due occurrence and the function queue suppresses a new scheduled job while the target function has queued or running work. That due occurrence remains visible as a diagnostic Skipped run. Do not add another singleton or deduplication layer and do not submit a compensating manual job for Skipped; observe the active job and let the next CRON occurrence proceed normally.

Generated entity contract after schema changes

Moltaro regenerates GeneratedWorkspaceContract on the server. Each C# language request and Check/Build receives the current generated entity source; there is no contract download or refresh endpoint in the API-first workflow.

After creating, changing, or deleting an entity field:

1. read the entity definition again and use its current keys and ids;
2. call completions, hover, signature help, or diagnostics again for the current source buffer;
3. fix any renamed or removed generated members;
4. if the source changed, create a new immutable source revision; run Check against the exact revision before Build.

Do not continue from completion results captured before the schema change.

Net Operation Project source limits

The current source contract accepts at most 500 files, 512 KiB of UTF-8 text per file, 5 MiB of UTF-8 source in total, and 500 changes in one manual-edit request. Paths are relative, use /, and are at most 512 characters. Supported stored paths are:

• C# files below src/;
• Markdown at the project root or below docs/;
• root package-references.json, with at most 50 package references.

Generated, build, dependency, and repository directories such as GeneratedWorkspaceContract, bin, obj, lib, node_modules, .git, and .moltaro cannot be edited through the source API. Language requests use the same 512 KiB active-buffer and 5 MiB snapshot limits.

Successful diagnostics and manual-edit responses include Limits, which is the machine-readable contract for source size, manual-edit count, diagnostics timeout, manual-edit timeout, and diagnostics concurrency. A size rejection includes safe MaxBytes and ActualBytes metadata without source text. File-count and manual-edit-count rejections likewise include exact maximum and actual counts. Diagnostics that exceed the server deadline return HTTP 503 with netOperationProject.language.analysis.timeout. A manual edit that exceeds its deadline or cannot acquire the project write lock in time returns HTTP 503 with netOperationProject.source.manualEdit.timeout; it creates no revision. After either response, read current status and retry with current concurrency values. Client cancellation may leave analysis completing safely in the background, but it does not retain the project write lock or block status reads. An ExpectedContentHash mismatch is a distinct HTTP 409 response.

Data and error conventions

• JSON properties are PascalCase and public enums are numeric. Read x-enum-varnames from the installed OpenAPI document instead of guessing a number.
• Check the response envelope’s Success before using Data. Branch on the stable error Code and optional Field, never on localized Message.
• Read locale and time zone from /api/workspace/context. Send dates, times, decimals, and currency values in the exact JSON shape declared by OpenAPI; never apply locale-specific display formatting to an API value.
• A 403 can mean the account lacks a role or resource-context permission. The agent asks the user or workspace administrator to review access; it does not attempt to grant itself permissions.

Recovery guide

Symptom	Safe response
401	Confirm the request used WORKSPACE_API_BASE_URL and the provided bearer key. If the key is missing, expired, or revoked, ask the user to update it in the Portal or administrative interface.
403	Read the operation’s permission requirements and ask the administrator for the narrowest missing permission. Do not switch to a personal login or change roles yourself. For moltaro.instances.replay.resultNotViewable, the record was already created: never recreate it with a new key.
404	First verify the API origin. Then re-discover the entity, function, board, module resource, or artifact; ids and visibility are workspace-specific.
409	Branch on the code. For a stale version (for records moltaro.security.concurrency.staleRowVersion), re-read, compare, merge, and send a new request with fresh row versions, revision ids, checksums, and hashes; never overwrite blindly. For moltaro.instances.conflict, repeat the identical request after a delay, a few times at most. For moltaro.security.idempotency.keyReuse, use a new key only for a genuinely new operation; for moltaro.security.idempotency.replayInvalid, do not retry and reconcile the target state by hand.
An Entity mutation answers a code the installed error catalog does not list for it	Show the code, Field, and message to a person. Keep the OperationKey and body; do not send the operation with a new key until its outcome is clear, and do not infer the reaction from the suffix or the HTTP status.
An Entity mutation returns 503 moltaro.referenceEligibility.metadataState.unavailable	Repeat the identical request a few times with a growing delay. If it persists, ask an administrator to check the workspace upgrade.
A composite Entity mutation returns 503 moltaro.instances.composite.budgetExceeded or moltaro.instances.composite.outcomeUnknown	Repeat the identical request, same OperationKey and body, a few times with a delay and only within seven days of the first attempt. budgetExceeded saved nothing; after outcomeUnknown never send the steps with a new key until a repeat answers.
Net Operation Project source processing returns 503	For netOperationProject.language.analysis.timeout or netOperationProject.source.manualEdit.timeout, read current status, respect the returned retry metadata and Limits, then retry from fresh revision, checksum, and file-hash values. A timed-out manual edit creates no revision.
Diagnostics pass but Check fails	Diagnostics replace one unsaved buffer inside a revision snapshot. Save every intended file, then inspect Check diagnostics for the exact immutable revision and installed dependencies.
Check passes but Build fails	Read the stored Build stage and nullable structured Failure. Branch on ReasonCode, not the localized summary. The previous Net Operation Project artifact remains active. For storage_capacity_exhausted, ask the operator to free or expand storage before queueing a new Build. For publication_commit_unconfirmed, re-read the Build and active artifact before retrying; do not repair catalog rows manually. Workspace UI never activates a failed build.
Workspace UI Build succeeds but users see no change	Find the artifact by BuildId and RevisionId, activate it with current row versions, then perform the required full WebApp refresh.
Module is disabled or an injectable service is absent	Read workspace module state and developer-surface. Ask the administrator to enable/configure the supported module, or remove the dependency. Do not inject an unlisted internal service.
moltaro.automation.executionPhase.unsupported	Move the module operation out of Validation/BeforeSaveMutation and into an Action, TriggerHandler, Command, Job, or HttpEndpoint. Prefer an after-commit trigger when it depends on a saved record.
moltaro.automation.entity.currentUserInvocationRequired or moltaro.automation.entity.ambientTransactionUnsupported	IEntityCompositeMutationService runs only in a Command or HTTP endpoint run by a user, outside an open transaction of the function’s database context. Move the call there; it never falls back to the system automation account.
Polling reaches the client deadline	Keep the build or job id, report its latest state, and let the user decide whether to continue polling or request cancellation. Do not enqueue a duplicate.

Audit responsibility

Change path	History behavior
Runtime record create/update/archive through Moltaro APIs and product surfaces	The platform writes field, table, assignment, and system-field history when audit is enabled for the entity definition.
C# IEntityAutomationCommandService create/update/import	The Entity application flow writes field history with the function run as the change source. Import is one definition per call; inspect every item, including failures. See the import and history guide.
Composite Entity mutation over REST or C# IEntityCompositeMutationService	Every step writes the history of the single-record save, with the caller or the invoking user as the author, in one transaction with the other steps; a rollback or a replay writes none. See Composite Entity mutations.
Net Operation Project direct-DB save	Automatic field-level audit is not added. Emit a display-safe AddBusinessEventAsync event for each meaningful custom state change.
Explicit historical audit import	IDynamicEntityAuditTrailImportService records supplied trustworthy history. It does not reconstruct missing old/new values.
Boards application-automation operation	Stored actor is moltaro-system-automation; join Function operations to board history/resource events by correlation and origin metadata.
Entitlement application-automation operation	Stored actor is moltaro-system-automation; treat the append-only ledger and entitlement history as truth and join them to the function run by correlation/origin metadata.
Net Operation Project or Workspace UI publication	Project build and publication governance is separate from one record’s change feed. Verify the active artifact through the project APIs.

See Record history and audit for the record change feed and business-event recipe, and Errors and responses for the common response shape.