Compare commits

..
Author SHA1 Message Date
AIOS Governed Continuation 4642b74af6 fix(admission): make V2R2C2 concurrent idempotency contract gate order-independent
autonomy-smoke-tests V2R2C2 PASS evidence_sha256=8ed5285cd9e236a2d674877b183325c3bcb8d7dbf699eed9f516b97ef1bd95ab
autonomy-unit-tests V2R2C2 PASS evidence_sha256=c82cbd9c6151b6e5f7dd87cc4588ab8018adbdedae2ae51705c43212f33ee6d3
campaign-approval-gate V2R2C2 PASS evidence_sha256=fc3707693b9b28a683f6cce43994fa272000cd6d52e31ac2800d329466559c54
git-diff-check V2R2C2 PASS evidence_sha256=6facd39176e65a80ffa3591db4e59c8a0a7dc04a3301dcd7b457d456e380305e
py-compile V2R2C2 PASS evidence_sha256=32381d24d691a64a82acbe2faecec51b8d3f43ac56f1884cc1ff83c2202deb34
reconciliation-contract V2R2C2 PASS evidence_sha256=b2177f09c2ec27acbb593eac7ff0039513eaa2d2202415d414a61c6ef1a0421c
release-state-machine V2R2C2 PASS evidence_sha256=0eed2af4f75ee18b68a013c9f02cb966181decd2eaa8d582f7456b3ac5f5ef7e
secret-scan V2R2C2 PASS evidence_sha256=1c584be64c3d82a4716a39b754458dfea7e3300cdd0b4dc3e9a8c86c263f95d1
signer-validator-contract V2R2C2 PASS evidence_sha256=74b3d59f0fa186c46ba282715c0490d50ea0580b4e63352f57caa0de729535da
aios.owner-final-merge-authorization-v2 V2R2C2 PASS evidence_sha256=70bb81b45354e252072e73861801d783c08062b932dc6f77c3217b86a43f5f76
The duplicate/concurrent mission-authorization test asserted that the single
creating (non-idempotent) request is always ThreadPoolExecutor.map result
index 0. map returns in *input* order, not completion order, so under 16-way
concurrency the winner may land at index 1 or 2 (~15% of runs), causing a
flaky gate failure even though admission behavior is correct (exactly one
creator, one durable task, rest idempotent replays).

Replace the index-based assertion with an order-independent invariant:
exactly 15 replays + exactly 1 creator. Behavior unchanged; stronger and
deterministic.
2026-09-12 06:49:03 +08:00
AIOS Governed Continuation 34395c7100 fix(v2r2c2): gate recovery — normalize LF, EOF blank lines, deterministic test path-mode, secret-scan allowlist for release-freeze OIDs 2026-09-12 02:44:54 +08:00
AIOS Governed Continuation 623fd61042 feat: shard-b governed provider-auth integration (V2R2C2 Round3) 2026-09-10 12:09:47 +08:00
AIOS Governed Continuation 691bf76aec feat: shard-a governed provider-auth integration (V2R2C2 Round3) 2026-09-10 11:54:49 +08:00
AIOS Governed Continuation 0136a53426 feat(governed): make Owner merge authority final
autonomy-smoke-tests phase2-pr76 PASS evidence_sha256=c16c68f86ee0c4e8cfd0af3f76078be45441ee151395fbde5800145c08e189b5
autonomy-unit-tests phase2-pr76 PASS evidence_sha256=15bc76da47bdf8218f18fc8a7da65fafd841677d5f754b71a0c9f56e07d84c1b
campaign-approval-gate phase2-pr76 PASS evidence_sha256=ea7ed0516efe6736a8eaee426685e40a2393fbf3ae84104a743f394204c2ec33
git-diff-check phase2-pr76 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
py-compile phase2-pr76 PASS evidence_sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
reconciliation-contract phase2-pr76 PASS evidence_sha256=211444cb0c146f5d4a57e13b780cfe7ffb406e8a8bc2e11c02a3a4728f66b0b2
release-state-machine phase2-pr76 PASS evidence_sha256=6cea88752b3e58ae5cee56ff3b1b13eab34117918d6bad9661194c031e2dc157
secret-scan phase2-pr76 PASS evidence_sha256=7cb68dd604dc6cd4904246e16333072d512dba14c096301e08c5a2b78a45bd55
signer-validator-contract phase2-pr76 PASS evidence_sha256=775b7aa27d0c17f48b3feff20c836f14b6ed8ebcb90ec7e0b76d2a13c579268a
aios.owner-final-merge-authorization-v2 real v2 contract tests passed
2026-09-07 23:14:29 +08:00
owner 1f9f65d8c7 Merge pull request 'feat: formal lineage supersession P0' (#75) from agent/singlehost-lineage-supersession-p0-20260905 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/75
2026-09-07 21:54:16 +08:00
AIOS Governed Continuation 591884bf5f fix(governed): provide collision-safe task storage identity
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=d3ed00952a599a4d17d6cf5e8f9714c233ada76347b3567acab8455fb07c7717
autonomy-unit-tests phase1-p0 PASS evidence_sha256=77c39918bb5883e62fbbae39f1812a1b5b6e19c98ba422d988ab509f84575b35
campaign-approval-gate phase1-p0 PASS evidence_sha256=8f562805faf8d739233301596d3a95d6a128eff7324fd30d663d94a196cf5c13
git-diff-check phase1-p0 PASS evidence_sha256=dc18ae7a546ada66b0bfdd9045464f8413312234baf8d3004eac78387c614134
py-compile phase1-p0 PASS evidence_sha256=ed151cbcc033e00f78a86e86840883d5996b2c7a33ed4520421cc3a6f4423dd3
reconciliation-contract phase1-p0 PASS evidence_sha256=3a4243f272ab2458dc631eb751f267f93c48713d96a59b311ae885e4d0f70728
release-state-machine phase1-p0 PASS evidence_sha256=d24c81a9969c35fc4c599195e3078c10fb06a0b6addfed9b44f4972b228464b9
secret-scan phase1-p0 PASS evidence_sha256=649005c3fef5220b20c9f616658ad283a28d506fe231d8bb6f795de183ad51c5
signer-validator-contract phase1-p0 PASS evidence_sha256=b5461be6646b4347c1ad85cf13889eb03de515ce5e89c041a8f24626fae2ec17
2026-09-07 21:32:46 +08:00
AIOS Governed Continuation 5cf599e1d1 feat: integrate formal lineage supersession P0 2026-09-07 21:28:09 +08:00
owner b3cee6ae48 Merge pull request 'feat(discovery): governed immutable repository discovery P0' (#73) from agent/repository-discovery-p0-20260902 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/73
2026-09-07 20:43:02 +08:00
AIOS Governed Continuation c8c25fa672 fix(discovery): make P0 contract hermetic
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=13aff2dfe664be536470ca26a6ac3f20ba15fdca37cd4f8e799c0c8c36974be9
autonomy-unit-tests phase1-p0 PASS evidence_sha256=118708029df05d861585179a31e4f4eaf0d7953563f5347b76732f2055785ba0
campaign-approval-gate phase1-p0 PASS evidence_sha256=4e39d586b32ad2d17da8fc38f5d359ace8a239af8687b4f894a338fe9f0bf83c
git-diff-check phase1-p0 PASS evidence_sha256=2638b909fa281012b6a43aaf34e1d611b0dba1fc78bf47f54941739973ee6281
py-compile phase1-p0 PASS evidence_sha256=440cf54c4225f809a575eff9b24238acd361497fd18bffbb359087e88310dcd6
reconciliation-contract phase1-p0 PASS evidence_sha256=131fb1042ff1ceb565dbf1ec7aed1cd3879d728f912745c0589a02b96381baaf
release-state-machine phase1-p0 PASS evidence_sha256=a74b5192b248b6e342afeea01c85ecbb8a02d71891bcf334df2e89d96cc1577d
secret-scan phase1-p0 PASS evidence_sha256=d4789e5c45ba681c651c613427d70d911a4cdd70fbead0772c4e1e376bef41cd
signer-validator-contract phase1-p0 PASS evidence_sha256=a29c7664de7aa9e30e966f8c4a14c0eaf8191fbb92ba726a72ba3c7ec2659eec
2026-09-07 20:33:11 +08:00
AIOS Governed Continuation 1735371e06 feat(discovery): integrate P0 discovery with task admission bootstrap
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=a96e50527a39e969b53df42ce799526faa8e3bcd618b6f5e2ec2e2e13c574f02
autonomy-unit-tests phase1-p0 FAIL evidence_sha256=d61da9d108d6ace7d845f78b1010773b4b1ce315a538209d7e60eb870c8eb953
campaign-approval-gate phase1-p0 PASS evidence_sha256=29d249ebfb389c3137e411c1d87fb88c7f755cdba4fea5118be187259c707295
git-diff-check phase1-p0 FAIL evidence_sha256=a0adbb54c3b220b9ffc82656470f1d67904823e461fb8ffedd713846814acc78
py-compile phase1-p0 PASS evidence_sha256=f504e34aa0a28ad009ba3b3fd3a75448297be43c452f4dc5ba445d3a4d0f7f07
reconciliation-contract phase1-p0 PASS evidence_sha256=8965ee765da46e32e79d046eeb57db023a2e9bd5105f057a27546b42eed470e3
release-state-machine phase1-p0 PASS evidence_sha256=e5a4c127d4b5b6eb27918fd9c54c06ee23d36d613bb816be475006c81b8ff479
secret-scan phase1-p0 PASS evidence_sha256=fc82389ca65008f045aa9357e5f276eeb0ce040ac5511197d65f1027975bc238
signer-validator-contract phase1-p0 PASS evidence_sha256=79a1d0246c73f0485816f77345fbc769c23f659cd177fefba90bb27e842214f3
2026-09-07 20:14:22 +08:00
AIOS Governed Continuation 447163fe82 test(security): allow exact historical idempotency fixture
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=fdb08fc46698d7d38b3981df0e27bc5b5bb1da1c57f63fc756a4f08b649a132c
autonomy-unit-tests phase1-p0 PASS evidence_sha256=a7d74280ec71babbb7ff2824c0a8f1ec29db647478f395457979e02164360f9d
campaign-approval-gate phase1-p0 PASS evidence_sha256=d0d77e65dee6f2cf1303a1030473331754ab91982b830684284a562fe5320f0d
git-diff-check phase1-p0 PASS evidence_sha256=5e9628231a64e6faa04570e4817ff4109d791cea2838541994698847987983fb
py-compile phase1-p0 PASS evidence_sha256=47df225f475589b8d9c843ba6d49908c39ce7920890e8b2f355d24818d0558e0
reconciliation-contract phase1-p0 PASS evidence_sha256=24840ca69508034e002e4b12a5525822042bb01b5a6f73a697ca5fdbea30f8ec
release-state-machine phase1-p0 PASS evidence_sha256=805e5a17e6cfb8a2070a34c36e52c89808b89fd4055800dc1fa9b3d58c6c656b
secret-scan phase1-p0 PASS evidence_sha256=eb3dc75806b3af6c1963f1d284cb8b90971694356fdac63d2a09cd08972d8c74
signer-validator-contract phase1-p0 PASS evidence_sha256=da317f9fc7c8ce90b99417c4ec129ed5565c058ac96bb89e56a1a72ce5ab97b3
2026-09-07 18:00:05 +08:00
AIOS Governed Continuation 35f8d189e6 test(governed): normalize bootstrap contract line endings
autonomy-smoke-tests phase1-p0 PASS evidence_sha256=e174976180ecd717a6d24749c91cc53e35a573273de650004fb4a5b7983ccc23
autonomy-unit-tests phase1-p0 PASS evidence_sha256=45c6e4bd0d53ee3f0c64186f16cadbe13adf07b8d4fa24d4624ed90fc08468b2
campaign-approval-gate phase1-p0 PASS evidence_sha256=b182a5af7b5605da5bda1856d6f69774ec19305666fb563b6b4254b85eea167f
git-diff-check phase1-p0 PASS evidence_sha256=6b50f3b275db390e111297b3780a66a97269aa102317dae7584f9987fbd72981
py-compile phase1-p0 PASS evidence_sha256=c82d4aedfb4c752b67467a25ea9f8b9f75f0bea008553abcbcaa78f03751f164
reconciliation-contract phase1-p0 PASS evidence_sha256=cd0071eb74bc0606654eea7159ab835a6ea087a24ce594657a0d7b286206cb59
release-state-machine phase1-p0 PASS evidence_sha256=d4b8b0d8838fc523eaab64dbb4618971b341148f068c1da2395c3955d6fd6e23
secret-scan phase1-p0 FAIL evidence_sha256=47d13c1b7d3c967277c56ce2c62c12bd2b9f221e1e42cbb38461c44574f0919f
signer-validator-contract phase1-p0 PASS evidence_sha256=cdb8009765558b5acd2c3de15bdf5681814e64b2c1587676a5528be5bbca1e95
2026-09-07 17:40:07 +08:00
AIOS Governed Continuation 6d4f79a8c2 fix(governed): make P0 task admission self-contained 2026-09-07 15:22:20 +08:00
AIOS Governed Continuation 1aee083564 feat: formal lineage supersession P0 2026-09-07 08:27:33 +08:00
AIOS Governed Continuation 1efa7724c1 feat(discovery): governed immutable repository discovery P0 2026-09-04 00:16:46 +08:00
AIOS Governed Continuation a7e835b9d3 feat(governed): close P0 task admission bootstrap 2026-09-03 21:40:32 +08:00
AIOS Governed Continuation 91f104ef62 fix(gitops): repair trusted-builder receipt replay framing
autonomy-smoke-tests focused receipt/replay and Task lifecycle smoke: PASS
autonomy-unit-tests full Linux regression: 834 passed, 3 skipped, 154 subtests passed
campaign-approval-gate standing Owner V15 exact Repair Task authorization and formal Task verification: PASS
git-diff-check exact two-file governed diff and whitespace check: PASS
py-compile active-runtime and tests compileall: PASS
reconciliation-contract WriterLease completed replay and governed branch convergence: PASS
release-state-machine full Linux regression includes release state-machine contract: PASS; no release mutation
secret-scan exact changed-file high-signal credential/private-key scan: PASS
signer-validator-contract receipt framing/replay/adoption contract: 18 focused tests PASS; private key non-exportable
2026-08-30 00:05:00 +08:00
owner f8ad8889a4 Merge pull request 'fix(gitops): bind trusted-builder provisioning to final common main' (#71) from agent/v15-post-pr70-builder-release-readiness-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/71
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 21:39:20 +08:00
AIOS Governed Continuation 36ffb00ae9 fix(gitops): bind trusted-builder provisioning to final common main
autonomy-smoke-tests authorized task-writer smoke: PASS
autonomy-unit-tests isolated unit suite: 28 passed; 2 opt-in skipped
campaign-approval-gate standing Owner V15 authorization bound and verified
git-diff-check exact four-file diff and whitespace check: PASS
py-compile trusted-builder provisioner compile: PASS
reconciliation-contract authority/Gitea Git/API durable receipt convergence: PASS
release-state-machine release workflow state unit contract: PASS
secret-scan exact changeset credential and private-key scan: PASS
signer-validator-contract canonical signer compatibility/self-test contract: PASS
2026-08-29 20:53:18 +08:00
owner c2db4ade3f Merge pull request 'feat(release): add purpose-bound trusted-builder signer provisioner' (#70) from agent/v15-trusted-builder-signer-provisioning-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/70
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 17:23:29 +08:00
AIOS Governed Continuation d58227f637 feat(release): add purpose-bound trusted-builder signer provisioner
py-compile Python compileall PASS on exact PR head
git-diff-check git diff-tree --check PASS on exact PR head
secret-scan Exact changed-file secret scan PASS; no credential material
signer-validator-contract Trusted-builder signer provisioner contract suite PASS; live signer remains intentionally unprovisioned
autonomy-unit-tests 821 passed, 3 skipped, 154 subtests passed
autonomy-smoke-tests Full repository source regression and smoke contract suite PASS
reconciliation-contract Full repository regression includes reconciliation/replay contract PASS
release-state-machine Full repository regression includes release controller/state-machine contract PASS; no production mutation
campaign-approval-gate Source governance and campaign gate contract PASS; candidate creation remains post-merge
2026-08-29 17:02:00 +08:00
owner c391d5baf7 Merge pull request 'fix(release): bind trusted builder to canonical purpose' (#69) from agent/v15-trusted-builder-purpose-repair-r3-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/69
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 15:54:53 +08:00
AIOS Governed Continuation 98bdeae6b4 fix(release): bind trusted builder to canonical purpose
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
2026-08-29 15:25:31 +08:00
owner 8e90f317b3 Merge pull request 'feat(v2r2c2): activate durable trusted release controller' (#68) from agent/v12-trusted-release-controller-activation-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/68
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 13:11:38 +08:00
AIOS Governed Continuation 98fc05d761 style(v2r2c2): normalize source eof
autonomy-smoke-tests Source snapshot smoke/contract suite passed; production mutations disabled
autonomy-unit-tests 816 passed, 3 skipped, 154 subtests passed
git-diff-check git diff --check PASS on exact PR head
py-compile Python compileall PASS on active-runtime/app
reconciliation-contract Full source regression includes completed-replay/reconciliation contract PASS
secret-scan Diff-added lines secret scan PASS; no credential material
signer-validator-contract Signer contract suite PASS; live signer remains opt-in and no key was provisioned
campaign-approval-gate V2R2C2 trusted-controller source binding gate PASS; durable candidate creation remains post-merge
release-state-machine V2R2C2 release-controller source contract PASS; no production mutation enabled
2026-08-29 13:06:09 +08:00
AIOS Governed Continuation c82fa9dad8 test(v2r2c2): update canonical release route assertions 2026-08-29 13:03:33 +08:00
AIOS Governed Continuation 6499b61a86 test(v2r2c2): align release compatibility contracts 2026-08-29 13:01:43 +08:00
AIOS Governed Continuation 54c8e47b2e fix(v2r2c2): preserve release query compatibility 2026-08-29 12:58:43 +08:00
AIOS Governed Continuation 9718109977 feat(v2r2c2): activate trusted candidate builder 2026-08-29 12:52:24 +08:00
owner d2b5a8339a Merge pull request 'fix(v2r2c2): bind trusted production runtime payload' (#67) from agent/v2r2c2-release-runtime-binding-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/67
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 10:16:33 +08:00
AIOS Governed Continuation 0d9d0479bf feat(v2r2c2): bind trusted release runtime and deterministic artifact
autonomy-smoke-tests V2R2C2 release-runtime binding smoke and offline package preflight PASS
autonomy-unit-tests 782 passed, 3 skipped, 154 subtests passed on source branch
git-diff-check governed source diff-check PASS against merged main f1695ce
py-compile all tracked Python source compile PASS
reconciliation-contract completed-replay reconciliation contract preserved; 11 focused tests PASS
secret-scan narrow source and deployable payload secret scan PASS; no credentials included
signer-validator-contract trusted release plan/adapter contract 48 tests PASS; formal Owner signature not asserted here
campaign-approval-gate V2R2C2 candidate binding contract PASS; production approval remains formal-envelope gated
release-state-machine V2R2C2 plan-driven release state contract PASS; no production mutation performed
2026-08-29 09:37:46 +08:00
owner f1695cebf6 Merge pull request 'feat(v2r2c2): integrate completed-replay reconciliation' (#66) from agent/v2r2c2-release-source-integration-20260829 into main
Reviewed-on: https://g.cnnetmkt.top/git/aios/aios/pulls/66
Reviewed-by: owner <owner@cnnetmkt.top>
2026-08-29 09:13:44 +08:00
AIOS Governed Continuation 8427c7bf22 test(release): cover trusted plan dispatch and adapter binding
autonomy-smoke-tests V2R2C2 trusted-plan/runtime contract smoke passed; 67 focused tests
autonomy-unit-tests governed source full Linux suite: 781 passed, 3 skipped, 154 subtests
git-diff-check final PR head diff-check passed against protected main
py-compile all tracked Python sources compile in memory without errors
reconciliation-contract completed-replay reconciliation contract: 11 passed
secret-scan final PR diff high-signal secret scan found no credentials
signer-validator-contract signer/validator contract: 5 passed, 2 explicit environment skips
campaign-approval-gate trusted V2R2C2 release-plan dispatch and exact binding contract passed
release-state-machine plan-driven release handler and generic adapter contract passed; legacy path isolated
2026-08-29 09:10:05 +08:00
AIOS Governed Continuation cdacdfc1b7 feat(release): bind new release execution to trusted plan 2026-08-29 09:09:02 +08:00
AIOS Governed Continuation e5b466ed2e test(signer): skip root-only live self-test when unprivileged 2026-08-29 08:54:44 +08:00
AIOS Governed Continuation 4a1433c306 fix(release): enforce candidate identity in trusted plan binding 2026-08-29 08:51:50 +08:00
AIOS Governed Continuation 5cd992e211 test(release): bind candidate identity in trusted plan fixture 2026-08-29 08:50:37 +08:00
AIOS Governed Continuation 036c72b553 test(release): cover trusted plan tamper and binding denial 2026-08-29 08:49:23 +08:00
AIOS Governed Continuation c8796a47dc feat(release): bind campaign acceptance and executor to trusted plan 2026-08-29 08:48:08 +08:00
AIOS Governed Continuation e13a3e4a3e feat(release): add canonical trusted release execution plan 2026-08-29 08:45:08 +08:00
AIOS Governed Continuation 44d5f7a3c1 fix(v2r2c2): normalize staging adapter file ending
autonomy-smoke-tests V2R2C2 candidate staging smoke and health projection passed on frozen candidate
autonomy-unit-tests governed source full Linux suite: 775 passed, 2 skipped, 154 subtests
git-diff-check PR head diff-check passed after governed whitespace normalization
py-compile full source tree compileall passed with repository PYTHONPATH
reconciliation-contract completed-replay reconciliation focused tests passed: 18 passed
secret-scan immutable candidate/package secret preflight passed for governed source payload
signer-validator-contract signer/validator contract included in full suite and passed
campaign-approval-gate blocked: required V2R2C2 release binding and final release evidence are not present
release-state-machine blocked: current release handler/executor is static V2R2C1; no legal V2R2C2 binding
2026-08-29 05:59:23 +08:00
AIOS Governed Continuation 7bd493d449 fix(v2r2c2): restore release version validation block indentation 2026-08-29 05:58:34 +08:00
AIOS Governed Continuation 1fae628eab feat(v2r2c2): publish completed-replay reconciliation source
aios/v2r2c2-source-regression V2R2C2 governed source integration exact-head checks passed: 1fae628eab849cf7cccf9711ca5a014560efccf2
2026-08-29 05:17:51 +08:00
AIOS Governed Continuation 1d35a7c0cd fix: restore signed deterministic recovery and bridge sandbox contract 2026-08-28 13:44:37 +08:00
AIOS governed merge broker ece82ad789 Merge commit '348c6db9a9c73464d5d39b65e4c4291ec4e99ba6' into HEAD 2026-08-26 17:53:33 +08:00
aios-autonomy-agent 348c6db9a9 feat(openbao): add autonomy runtime v1 candidate 2026-08-26 16:51:07 +08:00
AIOS Governed Continuation 2afc2cf41f docs: add unattended V2R2C2 convergence execution contract 2026-08-25 19:04:51 +08:00
AIOS Governed Continuation 7526fef619 test: cover convergence acceptance ownership and continuation 2026-08-25 18:57:05 +08:00
AIOS Governed Continuation c121527194 feat: add minimal AIOS convergence contracts 2026-08-25 18:54:47 +08:00
AIOS Owner Reconciliation 174e64dd02 fix(action-surface): bind governed reads to canonical release bridge 2026-08-25 18:17:42 +08:00
AIOS Owner Reconciliation d86e900332 docs: record restart persistence evidence 2026-08-25 16:30:54 +08:00
AIOS Owner Reconciliation 353f524a53 docs: record action contract staging repair evidence 2026-08-25 16:29:11 +08:00
AIOS Owner Reconciliation 2ec49005bd fix: wire staging release projections to runtime inputs 2026-08-25 16:17:23 +08:00
AIOS Owner Reconciliation 304bf174ae fix: bind release checks to staging baseline authority 2026-08-25 16:13:06 +08:00
AIOS Owner Reconciliation 92706e387e fix: reconcile authority-only isolated branch 2026-08-25 16:01:46 +08:00
AIOS Owner Reconciliation dfc37560e6 fix: reconcile absent remote branch creation 2026-08-25 15:59:30 +08:00
AIOS Owner Reconciliation 108aca1264 fix: preserve remote-create mode during durable reconcile 2026-08-25 15:57:14 +08:00
AIOS Owner Reconciliation 20462e58bb fix: recover initial remote branch push during reconcile 2026-08-25 15:56:36 +08:00
AIOS Owner Reconciliation 73e02c2324 fix: permit governed staging branch push 2026-08-25 15:54:27 +08:00
AIOS Owner Reconciliation 27f4be40e4 fix: make governed create-new lease replay idempotent 2026-08-25 15:45:46 +08:00
AIOS Owner Reconciliation c6d9ed4873 fix: normalize task branch refs in authority CAS 2026-08-25 15:42:10 +08:00
AIOS Owner Reconciliation c9aaa19873 fix: initialize governed create-new branch through authority CAS 2026-08-25 15:38:18 +08:00
AIOS Owner Reconciliation 1d073f8400 fix: include canonical V2R2C1 staging registry projection 2026-08-25 15:22:21 +08:00
AIOS Owner Reconciliation 79a1ad1f59 fix: close staging baseline and empty-state contracts 2026-08-25 15:18:28 +08:00
AIOS Owner Reconciliation 1d9b2ffef1 fix: repair staging action contracts and release overlay 2026-08-25 15:02:10 +08:00
AIOS Governed Continuation 35eec9afe1 runbook: add unattended agent execution contract for Action repair 2026-08-25 14:24:41 +08:00
AIOS Governed Continuation efcae0a58c docs: capture Action regression remediation and acceptance gates 2026-08-25 14:19:00 +08:00
342 changed files with 46936 additions and 30211 deletions
+17
View File
@@ -0,0 +1,17 @@
title = "AIOS repository secret scan policy"
[extend]
useDefault = true
[allowlist]
description = "Exact non-secret fixtures: Task Admission idempotency test fixture and release-freeze git commit OIDs (generic-api-key false positive on 40-hex git hashes)"
condition = "AND"
paths = [
'''tests/test_task_admission_bootstrap_http\.py$''',
'''docs/V2R2C2-P0-convergence-20260909\.md$''',
'''artifacts/round3-provider-auth/ledger\.json$''',
]
regexes = [
'''idempotency-1''',
'''f7f53d0568767d6deec2c47008a89f4d8413f93d''',
]
+7 -14
View File
@@ -13,7 +13,6 @@ from hashlib import sha256
from pathlib import Path
import json
from typing import Any
from message_handoff import stable_event_id
def utc_now() -> str:
@@ -56,19 +55,13 @@ class AgentEventBus:
payload: dict[str, Any],
correlation_id: str | None = None,
) -> dict[str, Any]:
event_id = stable_event_id(
source=source, target=target, message_type=kind, payload=payload,
tenant_id=(payload.get('tenant_id') if isinstance(payload, dict) else None),
user_id=(payload.get('user_id') if isinstance(payload, dict) else None),
project_id=(payload.get('project_id') if isinstance(payload, dict) else None),
conversation_id=(payload.get('conversation_id') if isinstance(payload, dict) else None),
task_id=(payload.get('task_id') if isinstance(payload, dict) else None),
run_id=(payload.get('run_id') if isinstance(payload, dict) else None),
correlation_id=(payload.get('correlation_id') if isinstance(payload, dict) else None),
causation_id=(payload.get('causation_id') if isinstance(payload, dict) else None),
sequence=(payload.get('sequence') if isinstance(payload, dict) else None),
idempotency_key=(payload.get('idempotency_key') if isinstance(payload, dict) else None),
)
event_id = "evt_" + sha256(
json.dumps(
{"kind": kind, "source": source, "summary": summary, "payload": payload},
sort_keys=True,
ensure_ascii=False,
).encode("utf-8")
).hexdigest()[:16]
event = AgentEvent(event_id, kind, source, target, summary, payload, correlation_id=correlation_id)
self._write("pending", event.to_dict())
self._audit("created", event.to_dict())
@@ -7,7 +7,7 @@ class AgentEventObservabilityBridge:
def __init__(self, hooks: AgentObservabilityHooks | None = None):
self.hooks = hooks or AgentObservabilityHooks()
def consume(self, event: dict, *, actor=None, trace=None):
def consume(self, event: dict):
kind = event.get("kind")
payload = event.get("payload", {})
if kind == "agent_started":
@@ -16,8 +16,6 @@ class AgentEventObservabilityBridge:
visible_scope=payload.get("scope", "self"),
entry_agent=payload.get("agent", "unknown"),
execution_id=event.get("id"),
actor=actor,
trace=trace,
)
if kind == "agent_delegated":
return self.hooks.delegate(event["execution_id"], payload.get("agent", "unknown"))
-12
View File
@@ -31,18 +31,6 @@ class AgentExecutionRecord:
status: str = "completed"
started_at: str = field(default_factory=utc_now)
ended_at: str = field(default_factory=utc_now)
trace_id: str | None = None
span_id: str | None = None
parent_span_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
tenant_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
claim_id: str | None = None
run_id: str | None = None
message_id: str | None = None
def to_dict(self):
return asdict(self)
+6 -30
View File
@@ -9,49 +9,25 @@ try:
except Exception:
APIRouter = None
from .agent_observability import AgentExecutionRecord, AgentObservability
from .agent_observability import AgentExecutionRecord
from .agent_observability_dashboard import AgentObservabilityDashboard
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import GovernanceActor, TraceContext, authorize_scope
store = AgentObservabilityStore()
dashboard = AgentObservabilityDashboard(store.load())
def record_agent_execution(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None):
scoped = any(payload.get(name) is not None for name in (
"trace_id", "span_id", "tenant_id", "project_id", "conversation_id", "task_id", "run_id"
))
values = dict(payload)
if scoped:
if actor is None or trace is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted = GovernanceActor.from_value(actor)
authorize_scope(trusted, trace.scope)
bound = {
"trace_id": trace.trace_id, "span_id": trace.span_id,
"parent_span_id": trace.parent_span_id, "correlation_id": trace.correlation_id,
"causation_id": trace.causation_id, "tenant_id": trace.scope.tenant_id,
"user_id": trace.scope.user_id, "project_id": trace.scope.project_id,
"conversation_id": trace.scope.conversation_id, "task_id": trace.task_id,
"claim_id": trace.claim_id, "run_id": trace.scope.run_id, "message_id": trace.message_id,
}
for name, authoritative in bound.items():
if values.get(name) not in (None, authoritative):
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:" + name)
values[name] = authoritative
record = AgentExecutionRecord(**values)
def record_agent_execution(payload: dict[str, Any]):
record = AgentExecutionRecord(**payload)
return {"status": "ok", "record": store.append(record)}
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None):
records = store.query(user_id, group_id, role, actor=actor)
visible = AgentObservability()
visible.records = [AgentExecutionRecord(**record) for record in records]
def get_agent_observability(user_id: str | None = None, group_id: str | None = None, role: str = "user"):
records = store.query(user_id, group_id, role)
return {
"status": "ok",
"records": records,
"summary": visible.summary(None),
"summary": store.load().summary(user_id if role != "owner" else None),
}
@@ -14,11 +14,6 @@ from uuid import uuid4
from .agent_observability import AgentExecutionRecord
from .agent_observability_store import AgentObservabilityStore
from .trace_tool_credential_governance import (
GovernanceActor,
TraceContext,
authorize_scope,
)
def utc_now() -> str:
@@ -41,7 +36,6 @@ class AgentRunContext:
input_tokens: int = 0
output_tokens: int = 0
status: str = "running"
trace: TraceContext | None = None
def delegate(self, agent_name: str) -> None:
if agent_name not in self.path:
@@ -65,19 +59,7 @@ class AgentObservabilityHooks:
visible_scope: str,
entry_agent: str,
execution_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
if execution_id is not None and execution_id in self.active:
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
if trace is not None:
if actor is None:
raise RuntimeError("TRACE_ACTOR_CONTEXT_REQUIRED")
trusted_actor = GovernanceActor.from_value(actor)
authorize_scope(trusted_actor, trace.scope)
if trace.scope.user_id is not None and trace.scope.user_id != user_id:
raise RuntimeError("TRACE_SCOPE_MISMATCH_DENY:user_id")
run = AgentRunContext(
execution_id=execution_id or f"exec_{uuid4().hex[:12]}",
user_id=user_id,
@@ -85,7 +67,6 @@ class AgentObservabilityHooks:
entry_agent=entry_agent,
path=[entry_agent],
used_hermes=entry_agent.lower() == "hermes",
trace=trace,
)
self.active[run.execution_id] = run
return {"status": "started", "execution_id": run.execution_id, "run": run.__dict__}
@@ -125,18 +106,6 @@ class AgentObservabilityHooks:
status=status,
started_at=run.started_at,
ended_at=ended_at,
trace_id=run.trace.trace_id if run.trace else None,
span_id=run.trace.span_id if run.trace else None,
parent_span_id=run.trace.parent_span_id if run.trace else None,
correlation_id=run.trace.correlation_id if run.trace else None,
causation_id=run.trace.causation_id if run.trace else None,
tenant_id=run.trace.scope.tenant_id if run.trace else None,
project_id=run.trace.scope.project_id if run.trace else None,
conversation_id=run.trace.scope.conversation_id if run.trace else None,
task_id=run.trace.task_id if run.trace else None,
claim_id=run.trace.claim_id if run.trace else None,
run_id=run.trace.scope.run_id if run.trace else None,
message_id=run.trace.message_id if run.trace else None,
)
return {"status": "stored", "record": self.store.append(record)}
@@ -12,12 +12,6 @@ from pathlib import Path
from typing import Any
from .agent_observability import AgentExecutionRecord, AgentObservability
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceScope,
authorize_scope,
)
class AgentObservabilityStore:
@@ -29,11 +23,6 @@ class AgentObservabilityStore:
def append(self, record: AgentExecutionRecord) -> dict[str, Any]:
path = self.records_dir / f"{record.execution_id}.json"
data = record.to_dict()
if path.exists():
existing = json.loads(path.read_text(encoding="utf-8"))
if existing == data:
return existing
raise RuntimeError("OBSERVABILITY_EXECUTION_CONFLICT")
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
return data
@@ -44,36 +33,8 @@ class AgentObservabilityStore:
observability.records.append(AgentExecutionRecord(**raw))
return observability
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user", *, actor: GovernanceActor | dict[str, Any] | None = None) -> list[dict[str, Any]]:
def query(self, user_id: str | None = None, group_id: str | None = None, role: str = "user") -> list[dict[str, Any]]:
records = [record.to_dict() for record in self.load().records]
scoped = [record for record in records if record.get("tenant_id") is not None]
legacy = [record for record in records if record.get("tenant_id") is None]
if actor is not None:
trusted = GovernanceActor.from_value(actor)
visible: list[dict[str, Any]] = []
for record in scoped:
try:
authorize_scope(
trusted,
GovernanceScope(
"tenant",
tenant_id=record.get("tenant_id"),
user_id=record.get("user_id"),
project_id=record.get("project_id"),
conversation_id=record.get("conversation_id"),
run_id=record.get("run_id"),
),
)
except GovernanceError:
continue
visible.append(record)
if trusted.privileged:
visible.extend(legacy)
else:
visible.extend(record for record in legacy if record.get("user_id") == trusted.scope.user_id)
return visible
# Caller-supplied role remains a legacy compatibility path only.
records = legacy
if role == "owner":
return records
if role == "group_admin" and group_id:
@@ -0,0 +1,122 @@
"""Staging HTTP overlay for the exact Task lineage supersession capability."""
from __future__ import annotations
from http.server import ThreadingHTTPServer
from typing import Any, Callable, Mapping
from urllib.parse import urlparse
from aios_task_lineage_supersession import (
DECISION_SCHEMA,
TaskLineageSupersessionController,
TaskLineageSupersessionError,
)
SUPERSESSION_ROUTE = "/v1/control/task-lineage-supersession"
def _status(error: TaskLineageSupersessionError) -> int:
return error.http_status
class LineageSupersessionEndpoint:
"""Framework-neutral endpoint; authentication must precede all parsing/mutation."""
def __init__(
self,
db_path,
verifier,
authenticate_control: Callable[[Mapping[str, str]], Any],
*,
clock=None,
identity_digest=None,
) -> None:
self.authenticate_control = authenticate_control
self.controller = TaskLineageSupersessionController(
db_path, verifier, clock=clock, identity_digest=identity_digest
)
def post(self, headers: Mapping[str, str], body: Mapping[str, Any]) -> tuple[int, dict[str, Any]]:
self.authenticate_control(headers)
try:
result = self.controller.supersede(body)
except TaskLineageSupersessionError as exc:
return _status(exc), {"error": exc.code}
return (200 if result.get("idempotent") else 201), result
def build_runtime():
"""Bind to the already-active recovery API and its formal Owner verifier lazily."""
try:
from aios_agent_task_api import APIConfig
from aios_agent_task_api_p0_failed_lease_recovery import (
P0FailedLeaseRecoveryHandler,
P0FailedLeaseRecoveryTaskApplication,
)
from aios_agent_task_protocol import TaskProtocolError
from aios_owner_approval_verifier import (
GrantRootOwnerApprovalVerifier,
OwnerApprovalVerifierError,
)
except ImportError as exc:
raise RuntimeError("LINEAGE_SUPERSESSION_RUNTIME_OVERLAY_UNAVAILABLE") from exc
# The active entrypoint layers failed- and active-lease recovery on top of
# expired recovery. Inherit that current layer so adding lineage does not
# silently remove either already-deployed recovery route.
class Application(P0FailedLeaseRecoveryTaskApplication):
def __init__(self, config: Any) -> None:
super().__init__(config)
try:
verifier = GrantRootOwnerApprovalVerifier()
except OwnerApprovalVerifierError as exc:
raise TaskProtocolError(str(exc), 503) from exc
self.lineage_supersession = LineageSupersessionEndpoint(
config.db_path, verifier, self.authenticate_control
)
def health(self) -> dict[str, Any]:
value = dict(super().health())
value.update({
"task_lineage_supersession": True,
"task_lineage_supersession_schema": DECISION_SCHEMA,
"task_lineage_supersession_exact_scope": True,
"task_lineage_supersession_exact_cas": True,
"task_lineage_supersession_append_only": True,
"task_lineage_supersession_caller_payload": False,
})
return value
class Handler(P0FailedLeaseRecoveryHandler):
app: Application | None = None
def do_POST(self) -> None:
path = urlparse(self.path).path.rstrip("/") or "/"
if path != SUPERSESSION_ROUTE:
super().do_POST()
return
def action() -> None:
if self.app is None:
raise TaskProtocolError("SERVICE_UNAVAILABLE", 503)
status, result = self.app.lineage_supersession.post(
self._headers(), self._body()
)
if "error" in result:
raise TaskProtocolError(str(result["error"]), status)
self._json(result, status)
self._run(action)
return APIConfig, Application, Handler
def serve(config=None) -> None:
APIConfig, Application, Handler = build_runtime()
config = config or APIConfig.from_env()
application = Application(config)
Handler.app = application
ThreadingHTTPServer((config.host, config.port), Handler).serve_forever()
if __name__ == "__main__":
serve()
@@ -1,722 +0,0 @@
"""Owner-controlled CAS-bound execution-envelope rebind for Online Agent Tasks.
This service exists for a narrow recovery case: a Task was projected/claimed with a read-only
source_capture envelope before its already-registered deterministic plan became authoritative.
It never creates Tasks/Jobs/branches and never accepts arbitrary metadata patches. It only replaces
metadata.execution + the derived execution_projection for the same governed binding, after an
official NEEDS_REWORK transition has returned the Task to QUEUED with no active lease.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
from typing import Any, Callable, Mapping, Sequence
import urllib.error
import urllib.request
from aios_online_agent_execution import (
GovernedExecutionBinding,
validate_execution_envelope,
)
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
MAX_BODY = 64 * 1024
AUTHORIZATION_ID_ENV = "AIOS_AGENT_TASK_EXECUTION_REBIND_AUTHORIZATION_ID"
class ExecutionRebindError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
def _canonical(value: object) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False
).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _require_string(
value: object,
code: str,
pattern: re.Pattern[str] | None = None,
) -> str:
if not isinstance(value, str) or not value.strip():
raise ExecutionRebindError(code, 400)
value = value.strip()
if pattern is not None and pattern.fullmatch(value) is None:
raise ExecutionRebindError(code, 400)
return value
def _required_authorization_id() -> str:
value = os.environ.get(AUTHORIZATION_ID_ENV, "").strip()
if not value:
raise RuntimeError("TASK_EXECUTION_REBIND_AUTHORIZATION_ID_MISSING")
if KEY_RE.fullmatch(value) is None:
raise RuntimeError("TASK_EXECUTION_REBIND_AUTHORIZATION_ID_INVALID")
return value
@dataclass(frozen=True)
class ExecutionRebindRequest:
expected_status: str
expected_revision: int
expected_attempt: int
previous_verification_sha256: str
expected_old_execution_digest: str
expected_new_execution_digest: str
authorization_id: str
idempotency_key: str
reason: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ExecutionRebindRequest":
allowed = {
"expected_status",
"expected_revision",
"expected_attempt",
"previous_verification_sha256",
"expected_old_execution_digest",
"expected_new_execution_digest",
"authorization_id",
"idempotency_key",
"reason",
}
if set(value) != allowed:
raise ExecutionRebindError("EXECUTION_REBIND_FIELDS_INVALID", 400)
status = _require_string(value["expected_status"], "EXPECTED_STATUS_INVALID")
if status != "QUEUED":
raise ExecutionRebindError("EXPECTED_STATUS_INVALID", 400)
revision = value["expected_revision"]
attempt = value["expected_attempt"]
if (
not isinstance(revision, int)
or isinstance(revision, bool)
or revision < 0
or not isinstance(attempt, int)
or isinstance(attempt, bool)
or attempt < 1
):
raise ExecutionRebindError("EXECUTION_REBIND_INTEGER_INVALID", 400)
verification = _require_string(
value["previous_verification_sha256"], "VERIFICATION_SHA256_INVALID"
)
old_digest = _require_string(
value["expected_old_execution_digest"], "OLD_EXECUTION_DIGEST_INVALID"
)
new_digest = _require_string(
value["expected_new_execution_digest"], "NEW_EXECUTION_DIGEST_INVALID"
)
for digest, code in (
(verification, "VERIFICATION_SHA256_INVALID"),
(old_digest, "OLD_EXECUTION_DIGEST_INVALID"),
(new_digest, "NEW_EXECUTION_DIGEST_INVALID"),
):
if SHA_RE.fullmatch(digest) is None:
raise ExecutionRebindError(code, 400)
if old_digest == new_digest:
raise ExecutionRebindError("EXECUTION_REBIND_DIGEST_UNCHANGED", 400)
authorization = _require_string(
value["authorization_id"], "AUTHORIZATION_ID_INVALID", KEY_RE
)
key = _require_string(value["idempotency_key"], "IDEMPOTENCY_KEY_INVALID", KEY_RE)
reason = _require_string(value["reason"], "EXECUTION_REBIND_REASON_INVALID")
if len(reason) > 512:
raise ExecutionRebindError("EXECUTION_REBIND_REASON_INVALID", 400)
return cls(
status,
revision,
attempt,
verification,
old_digest,
new_digest,
authorization,
key,
reason,
)
def public(self) -> dict[str, Any]:
return {
"expected_status": self.expected_status,
"expected_revision": self.expected_revision,
"expected_attempt": self.expected_attempt,
"previous_verification_sha256": self.previous_verification_sha256,
"expected_old_execution_digest": self.expected_old_execution_digest,
"expected_new_execution_digest": self.expected_new_execution_digest,
"authorization_id": self.authorization_id,
"idempotency_key": self.idempotency_key,
"reason": self.reason,
}
class CanonicalBridgeExecutionResolver:
"""Resolve the already-registered plan from the canonical governed Bridge."""
def __init__(self, base_url: str, control_token: str) -> None:
self.base_url = base_url.rstrip("/")
self.control_token = control_token
def __call__(
self,
task_id: str,
required_capabilities: Sequence[str],
) -> dict[str, Any]:
body = json.dumps(
{
"task_id": task_id,
"required_capabilities": list(required_capabilities),
},
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
request = urllib.request.Request(
self.base_url + "/v2/execution-envelope",
method="POST",
data=body,
headers={
"Authorization": "Bearer " + self.control_token,
"Content-Type": "application/json",
"Accept": "application/json",
},
)
try:
with urllib.request.urlopen(request, timeout=15) as response:
raw = response.read()
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read()
status = exc.code
try:
value = json.loads(raw.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ExecutionRebindError("CANONICAL_EXECUTION_RESPONSE_INVALID", 502) from exc
if not isinstance(value, Mapping):
raise ExecutionRebindError("CANONICAL_EXECUTION_RESPONSE_INVALID", 502)
if status != 200:
raise ExecutionRebindError(
str(value.get("error") or "CANONICAL_EXECUTION_RESOLUTION_FAILED"),
502 if status >= 500 else 409,
)
return dict(value)
class ExecutionRebindStore:
"""Official owner-controlled mutation with CAS, plan resolution and durable audit."""
def __init__(
self,
db_path: Path,
resolver: Callable[[str, Sequence[str]], Mapping[str, Any]],
) -> None:
self.db_path = Path(db_path)
self.resolver = resolver
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(
connection: sqlite3.Connection,
table: str,
) -> dict[str, sqlite3.Row]:
return {
str(row["name"]): row
for row in connection.execute(f'PRAGMA table_info("{table}")')
}
@staticmethod
def _json_value(raw: object, code: str) -> dict[str, Any]:
if isinstance(raw, Mapping):
return dict(raw)
if not isinstance(raw, str):
raise ExecutionRebindError(code, 409)
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise ExecutionRebindError(code, 409) from exc
if not isinstance(value, Mapping):
raise ExecutionRebindError(code, 409)
return dict(value)
@staticmethod
def _verification_hash(raw: object) -> str:
if raw in (None, ""):
value: object = {}
elif isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise ExecutionRebindError("TASK_VERIFICATION_INVALID", 409) from exc
else:
value = raw
return _sha(value)
@staticmethod
def _metadata_column(columns: Mapping[str, sqlite3.Row]) -> str:
for name in ("metadata_json", "metadata"):
if name in columns:
return name
raise ExecutionRebindError("TASK_METADATA_SCHEMA_UNSUPPORTED", 503)
@staticmethod
def _required_capabilities(
task: sqlite3.Row,
columns: Mapping[str, sqlite3.Row],
old_execution: Mapping[str, Any],
) -> list[str]:
for field in ("required_capabilities_json", "required_capabilities"):
if field not in columns:
continue
raw = task[field]
if raw in (None, ""):
continue
if isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError:
value = [item for item in raw.split(",") if item.strip()]
else:
value = raw
if isinstance(value, list) and all(isinstance(item, str) for item in value):
return [item.strip() for item in value if item.strip()]
value = old_execution.get("capability_scope")
if isinstance(value, list) and all(isinstance(item, str) for item in value):
return [item for item in value if item]
raise ExecutionRebindError("TASK_CAPABILITIES_UNAVAILABLE", 503)
@staticmethod
def _event_table(
connection: sqlite3.Connection,
) -> tuple[str, dict[str, sqlite3.Row]]:
tables = {
row["name"]
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
for name in ("task_events", "events"):
if name not in tables:
continue
columns = ExecutionRebindStore._columns(connection, name)
if "task_id" in columns and ("event_type" in columns or "type" in columns):
return name, columns
raise ExecutionRebindError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _insert_event(
connection: sqlite3.Connection,
task_id: str,
payload: Mapping[str, Any],
) -> None:
table, columns = ExecutionRebindStore._event_table(connection)
values: dict[str, Any] = {"task_id": task_id}
if "event_type" in columns:
values["event_type"] = "TASK_EXECUTION_REBIND_AUTHORIZED"
else:
values["type"] = "TASK_EXECUTION_REBIND_AUTHORIZED"
payload_text = json.dumps(payload, sort_keys=True, separators=(",", ":"))
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise ExecutionRebindError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "owner-control"
if "actor_id" in columns:
values["actor_id"] = "owner-control"
names = list(values)
quoted = ",".join('"' + name + '"' for name in names)
placeholders = ",".join("?" for _ in names)
connection.execute(
f'INSERT INTO "{table}" ({quoted}) VALUES ({placeholders})',
tuple(values[name] for name in names),
)
@staticmethod
def _ensure_rebind_table(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_execution_rebinds (
task_id TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
authorization_id TEXT NOT NULL,
previous_verification_sha256 TEXT NOT NULL,
old_execution_digest TEXT NOT NULL,
new_execution_digest TEXT NOT NULL,
expected_revision INTEGER NOT NULL,
applied_revision INTEGER NOT NULL,
attempt INTEGER NOT NULL,
reason TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id,idempotency_key)
)
"""
)
def _load_and_validate(
self,
connection: sqlite3.Connection,
task_id: str,
request: ExecutionRebindRequest,
) -> tuple[sqlite3.Row, dict[str, sqlite3.Row], str, dict[str, Any], dict[str, Any]]:
columns = self._columns(connection, "tasks")
required = {
"task_id",
"status",
"attempt",
"revision",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"verification_json",
}
if not required.issubset(columns):
raise ExecutionRebindError("TASK_SCHEMA_EXECUTION_REBIND_UNSUPPORTED", 503)
metadata_column = self._metadata_column(columns)
task = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(task_id,),
).fetchone()
if task is None:
raise ExecutionRebindError("TASK_NOT_FOUND", 404)
if (
str(task["status"]) != request.expected_status
or int(task["revision"]) != request.expected_revision
or int(task["attempt"]) != request.expected_attempt
):
raise ExecutionRebindError("EXECUTION_REBIND_CAS_MISMATCH", 409)
if task["lease_token_hash"] not in (None, "") or task["lease_expires_at"] not in (None, ""):
raise ExecutionRebindError("EXECUTION_REBIND_ACTIVE_LEASE_DENY", 409)
verification_sha = self._verification_hash(task["verification_json"])
if verification_sha != request.previous_verification_sha256:
raise ExecutionRebindError("EXECUTION_REBIND_VERIFICATION_DRIFT", 409)
metadata = self._json_value(
task[metadata_column],
"TASK_METADATA_INVALID",
)
execution = metadata.get("execution")
if not isinstance(execution, Mapping):
raise ExecutionRebindError("TASK_EXECUTION_METADATA_MISSING", 409)
old_execution = validate_execution_envelope(dict(execution))
if old_execution["execution_digest"] != request.expected_old_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_OLD_DIGEST_DRIFT", 409)
if old_execution.get("action") != "source_capture":
raise ExecutionRebindError("EXECUTION_REBIND_SOURCE_CAPTURE_REQUIRED", 409)
projection = metadata.get("execution_projection")
if isinstance(projection, Mapping):
projected_digest = str(projection.get("execution_digest") or "")
if projected_digest and projected_digest != request.expected_old_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_PROJECTION_DRIFT", 409)
return task, columns, metadata_column, metadata, old_execution
def rebind(
self,
task_id: str,
request: ExecutionRebindRequest,
) -> dict[str, Any]:
task_id = _require_string(task_id, "TASK_ID_INVALID", TASK_RE)
required_authorization = _required_authorization_id()
if request.authorization_id != required_authorization:
raise ExecutionRebindError("EXECUTION_REBIND_AUTHORIZATION_ID_MISMATCH", 403)
request_sha = _sha(request.public())
# Idempotent replay must win before CAS validation because a successful
# rebind necessarily increments the Task revision and changes the persisted envelope.
with self._connect() as connection:
self._ensure_rebind_table(connection)
replay = connection.execute(
"SELECT * FROM task_execution_rebinds WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise ExecutionRebindError("EXECUTION_REBIND_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": int(replay["attempt"]),
"revision": int(replay["applied_revision"]),
"old_execution_digest": replay["old_execution_digest"],
"new_execution_digest": replay["new_execution_digest"],
"authorization_id": replay["authorization_id"],
"idempotent": True,
}
# Read/CAS snapshot first. Canonical Bridge resolution happens outside the DB write lock.
with self._connect() as connection:
task, columns, _, _, old_execution = self._load_and_validate(
connection, task_id, request
)
capabilities = self._required_capabilities(task, columns, old_execution)
binding = GovernedExecutionBinding.from_mapping(old_execution)
resolved = dict(self.resolver(task_id, capabilities))
new_execution = validate_execution_envelope(resolved, binding=binding)
if new_execution.get("action") != "deterministic":
raise ExecutionRebindError("EXECUTION_REBIND_DETERMINISTIC_PLAN_REQUIRED", 409)
if new_execution["execution_digest"] != request.expected_new_execution_digest:
raise ExecutionRebindError("EXECUTION_REBIND_NEW_DIGEST_DRIFT", 409)
if new_execution["execution_digest"] == old_execution["execution_digest"]:
raise ExecutionRebindError("EXECUTION_REBIND_DIGEST_UNCHANGED", 409)
# Re-read under BEGIN IMMEDIATE and apply only if the exact snapshot is unchanged.
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_rebind_table(connection)
replay = connection.execute(
"SELECT * FROM task_execution_rebinds WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise ExecutionRebindError("EXECUTION_REBIND_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": int(replay["attempt"]),
"revision": int(replay["applied_revision"]),
"old_execution_digest": replay["old_execution_digest"],
"new_execution_digest": replay["new_execution_digest"],
"authorization_id": replay["authorization_id"],
"idempotent": True,
}
task, columns, metadata_column, metadata, current_old = self._load_and_validate(
connection, task_id, request
)
if current_old["execution_digest"] != old_execution["execution_digest"]:
raise ExecutionRebindError("EXECUTION_REBIND_OLD_DIGEST_DRIFT", 409)
new_metadata = dict(metadata)
new_metadata["execution"] = new_execution
new_metadata["execution_projection"] = {
"schema": "aios.online-agent.task-projection.v1",
"task_id": binding.task_id,
"job_id": binding.job_id,
"execution_digest": new_execution["execution_digest"],
}
applied_revision = request.expected_revision + 1
previous_claimed_by = task["claimed_by"]
cursor = connection.execute(
f"""
UPDATE tasks
SET "{metadata_column}"=?, revision=?, claimed_by=NULL,
lease_token_hash=NULL, lease_expires_at=NULL
WHERE task_id=? AND status=? AND revision=? AND attempt=?
AND claimed_by IS ? AND lease_token_hash IS NULL AND lease_expires_at IS NULL
""",
(
json.dumps(new_metadata, sort_keys=True, separators=(",", ":")),
applied_revision,
task_id,
request.expected_status,
request.expected_revision,
request.expected_attempt,
previous_claimed_by,
),
)
if cursor.rowcount != 1:
raise ExecutionRebindError("EXECUTION_REBIND_CAS_MISMATCH", 409)
event_payload = {
"schema": "aios.task-execution-rebind.event.v1",
"task_id": task_id,
"job_id": binding.job_id,
"authorization_id": request.authorization_id,
"idempotency_key": request.idempotency_key,
"reason": request.reason,
"previous_verification_sha256": request.previous_verification_sha256,
"old_execution_digest": old_execution["execution_digest"],
"new_execution_digest": new_execution["execution_digest"],
"old_action": old_execution.get("action"),
"new_action": new_execution.get("action"),
"previous_revision": request.expected_revision,
"applied_revision": applied_revision,
"attempt": request.expected_attempt,
"request_sha256": request_sha,
"previous_claimed_by": previous_claimed_by,
"stale_claim_normalized": previous_claimed_by not in (None, ""),
}
self._insert_event(connection, task_id, event_payload)
connection.execute(
"""
INSERT INTO task_execution_rebinds(
task_id,idempotency_key,request_sha256,authorization_id,
previous_verification_sha256,old_execution_digest,new_execution_digest,
expected_revision,applied_revision,attempt,reason,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
task_id,
request.idempotency_key,
request_sha,
request.authorization_id,
request.previous_verification_sha256,
old_execution["execution_digest"],
new_execution["execution_digest"],
request.expected_revision,
applied_revision,
request.expected_attempt,
request.reason,
_now(),
),
)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"revision": applied_revision,
"old_execution_digest": old_execution["execution_digest"],
"new_execution_digest": new_execution["execution_digest"],
"authorization_id": request.authorization_id,
"idempotent": False,
}
class ExecutionRebindHandler(BaseHTTPRequestHandler):
store: ExecutionRebindStore | None = None
control_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def _json(self, value: Mapping[str, Any], status: int) -> None:
data = json.dumps(
dict(value), sort_keys=True, separators=(",", ":")
).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def _authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.control_token)
and hmac.compare_digest(header[7:], self.control_token)
)
def do_GET(self) -> None:
if self.path == "/health":
self._json(
{"status": "ok", "service": "aios-agent-task-execution-rebind"},
200,
)
else:
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
match = re.fullmatch(r"/v1/tasks/([^/]+)/execution-rebind", self.path)
if match is None:
self._json({"error": "NOT_FOUND"}, 404)
return
if not self._authorized():
self._json({"error": "UNAUTHORIZED"}, 401)
return
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
self._json({"error": "CONTENT_LENGTH_INVALID"}, 400)
return
if length < 1 or length > MAX_BODY:
self._json({"error": "REQUEST_SIZE_INVALID"}, 413)
return
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, Mapping):
raise ValueError
request = ExecutionRebindRequest.from_mapping(value)
if self.store is None:
raise ExecutionRebindError("RUNTIME_UNAVAILABLE", 503)
result = self.store.rebind(match.group(1), request)
self._json(result, 200 if result["idempotent"] else 201)
except ExecutionRebindError as exc:
self._json({"error": exc.code}, exc.status)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
self._json({"error": "REQUEST_INVALID"}, 400)
def load_control_token(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError("TASK_EXECUTION_REBIND_CONTROL_CREDENTIAL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
raise RuntimeError("TASK_EXECUTION_REBIND_CONTROL_CREDENTIAL_INVALID")
return value
def serve() -> None:
db = Path(os.environ.get("AIOS_AGENT_TASK_DB", "/var/lib/aios-agent-task/state.sqlite3"))
_required_authorization_id()
token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(
Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
/ "agent_task_control_token"
),
)
)
token = load_control_token(token_path)
bridge = os.environ.get(
"AIOS_GOVERNED_WRITE_API_URL",
"http://127.0.0.1:18892",
)
host = os.environ.get("AIOS_TASK_EXECUTION_REBIND_HOST", "127.0.0.1")
port = int(os.environ.get("AIOS_TASK_EXECUTION_REBIND_PORT", "18942"))
ExecutionRebindHandler.store = ExecutionRebindStore(
db,
CanonicalBridgeExecutionResolver(bridge, token),
)
ExecutionRebindHandler.control_token = token
ThreadingHTTPServer((host, port), ExecutionRebindHandler).serve_forever()
if __name__ == "__main__":
serve()
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,811 @@
"""Canonical Task projection intent and same-identity recovery primitives.
This module is intentionally Task-Plane-owned. It does not open a network
listener and it does not authenticate callers; existing control-plane services
must enforce their normal service-side ``agent-task:control`` boundary before
calling it.
The recovery primitive has one purpose: converge a durable, complete
ProjectionIntent to exactly one canonical Task row. It never fabricates task
semantics from a Bridge binding and it never generates a replacement task id.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping, Sequence
TASK_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
JOB_ID_RE = TASK_ID_RE
SHA40_RE = re.compile(r"^[0-9a-f]{40}$")
SHA64_RE = re.compile(r"^[0-9a-f]{64}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9._/-]{1,220}$")
SCHEMA = "aios.canonical-task-projection-intent.v1"
RECEIPT_SCHEMA = "aios.canonical-task-projection-receipt.v1"
class ProjectionRecoveryError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
class ProjectionState(str, Enum):
PENDING = "PENDING"
CONVERGED = "CONVERGED"
DRIFT = "DRIFT"
FAILED = "FAILED"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
BACKFILL_REQUIRED = "BACKFILL_REQUIRED"
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise ProjectionRecoveryError(code, 400)
result = value.strip()
if pattern is not None and pattern.fullmatch(result) is None:
raise ProjectionRecoveryError(code, 400)
return result
def _integer(value: object, code: str, *, minimum: int = 0) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < minimum:
raise ProjectionRecoveryError(code, 400)
return value
def _plain_mapping(value: object, code: str) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise ProjectionRecoveryError(code, 400)
return {str(k): v for k, v in value.items()}
def _plain_list(value: object, code: str) -> list[Any]:
if not isinstance(value, list):
raise ProjectionRecoveryError(code, 400)
return list(value)
@dataclass(frozen=True, slots=True)
class CanonicalTaskPayload:
task_id: str
goal: str
instructions: str
success_criteria: tuple[Mapping[str, Any], ...]
constraints: Mapping[str, Any]
required_capabilities: tuple[str, ...]
metadata: Mapping[str, Any]
priority: int = 100
max_attempts: int = 20
deadline: str | None = None
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskPayload":
allowed = {
"task_id",
"goal",
"instructions",
"success_criteria",
"constraints",
"required_capabilities",
"metadata",
"priority",
"max_attempts",
"deadline",
}
if set(value) - allowed:
raise ProjectionRecoveryError("TASK_PAYLOAD_UNKNOWN_FIELDS", 400)
task_id = _string(value.get("task_id"), "TASK_ID_INVALID", TASK_ID_RE)
goal = _string(value.get("goal"), "TASK_GOAL_INVALID")
instructions = _string(value.get("instructions"), "TASK_INSTRUCTIONS_INVALID")
if len(goal) > 8192 or len(instructions) > 65536:
raise ProjectionRecoveryError("TASK_TEXT_TOO_LARGE", 413)
criteria_raw = _plain_list(value.get("success_criteria"), "TASK_SUCCESS_CRITERIA_INVALID")
criteria: list[Mapping[str, Any]] = []
for item in criteria_raw:
if not isinstance(item, Mapping):
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_INVALID", 400)
criteria.append(dict(item))
if not criteria:
raise ProjectionRecoveryError("TASK_SUCCESS_CRITERIA_REQUIRED", 400)
constraints = _plain_mapping(value.get("constraints"), "TASK_CONSTRAINTS_INVALID")
capabilities_raw = _plain_list(
value.get("required_capabilities", []),
"TASK_CAPABILITIES_INVALID",
)
capabilities: list[str] = []
for item in capabilities_raw:
capabilities.append(_string(item, "TASK_CAPABILITY_INVALID"))
# Recovery recreates the pristine canonical row. Initial execution
# projection remains the sole owner of runtime capabilities.
if capabilities:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_CAPABILITIES",
409,
)
metadata = _plain_mapping(value.get("metadata", {}), "TASK_METADATA_INVALID")
if "execution" in metadata or "execution_projection" in metadata:
raise ProjectionRecoveryError(
"TASK_RECOVERY_REQUIRES_PRISTINE_METADATA",
409,
)
priority = _integer(value.get("priority", 100), "TASK_PRIORITY_INVALID", minimum=0)
max_attempts = _integer(
value.get("max_attempts", 20),
"TASK_MAX_ATTEMPTS_INVALID",
minimum=1,
)
deadline = value.get("deadline")
if deadline is not None:
deadline = _string(deadline, "TASK_DEADLINE_INVALID")
return cls(
task_id=task_id,
goal=goal,
instructions=instructions,
success_criteria=tuple(criteria),
constraints=constraints,
required_capabilities=tuple(capabilities),
metadata=metadata,
priority=priority,
max_attempts=max_attempts,
deadline=deadline,
)
def public(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"goal": self.goal,
"instructions": self.instructions,
"success_criteria": [dict(x) for x in self.success_criteria],
"constraints": dict(self.constraints),
"required_capabilities": list(self.required_capabilities),
"metadata": dict(self.metadata),
"priority": self.priority,
"max_attempts": self.max_attempts,
"deadline": self.deadline,
}
@property
def payload_sha256(self) -> str:
return sha256_json(self.public())
@dataclass(frozen=True, slots=True)
class GovernedProjectionBinding:
governed_task_id: str
bridge_job_id: str
repository: str
branch: str
generation: int
base_oid: str
expected_branch_oid: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "GovernedProjectionBinding":
allowed = {
"governed_task_id",
"bridge_job_id",
"repository",
"branch",
"generation",
"base_oid",
"expected_branch_oid",
}
if set(value) != allowed:
raise ProjectionRecoveryError("GOVERNED_BINDING_FIELDS_INVALID", 400)
repository = _string(value["repository"], "REPOSITORY_INVALID")
if repository != "aios/aios":
raise ProjectionRecoveryError("REPOSITORY_MISMATCH", 409)
return cls(
governed_task_id=_string(
value["governed_task_id"],
"GOVERNED_TASK_ID_INVALID",
TASK_ID_RE,
),
bridge_job_id=_string(
value["bridge_job_id"],
"BRIDGE_JOB_ID_INVALID",
JOB_ID_RE,
),
repository=repository,
branch=_string(value["branch"], "BRANCH_INVALID", BRANCH_RE),
generation=_integer(value["generation"], "GENERATION_INVALID", minimum=1),
base_oid=_string(value["base_oid"], "BASE_OID_INVALID", SHA40_RE),
expected_branch_oid=_string(
value["expected_branch_oid"],
"EXPECTED_OID_INVALID",
SHA40_RE,
),
)
def public(self) -> dict[str, Any]:
return asdict(self)
@dataclass(frozen=True, slots=True)
class ProjectionSourceArtifact:
ref: str
sha256: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "ProjectionSourceArtifact":
if set(value) != {"ref", "sha256"}:
raise ProjectionRecoveryError("SOURCE_ARTIFACT_FIELDS_INVALID", 400)
return cls(
ref=_string(value["ref"], "SOURCE_ARTIFACT_REF_INVALID"),
sha256=_string(value["sha256"], "SOURCE_ARTIFACT_SHA_INVALID", SHA64_RE),
)
@dataclass(frozen=True, slots=True)
class CanonicalTaskProjectionIntent:
projection_id: str
projection_generation: int
idempotency_key: str
task: CanonicalTaskPayload
binding: GovernedProjectionBinding
source_artifact: ProjectionSourceArtifact
schema: str = SCHEMA
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "CanonicalTaskProjectionIntent":
allowed = {
"schema",
"projection_id",
"projection_generation",
"idempotency_key",
"task",
"governed_binding",
"source_artifact",
"task_payload_sha256",
}
if set(value) != allowed:
raise ProjectionRecoveryError("PROJECTION_INTENT_FIELDS_INVALID", 400)
if value["schema"] != SCHEMA:
raise ProjectionRecoveryError("PROJECTION_INTENT_SCHEMA_INVALID", 400)
task = CanonicalTaskPayload.from_mapping(
_plain_mapping(value["task"], "TASK_PAYLOAD_INVALID")
)
binding = GovernedProjectionBinding.from_mapping(
_plain_mapping(value["governed_binding"], "GOVERNED_BINDING_INVALID")
)
source = ProjectionSourceArtifact.from_mapping(
_plain_mapping(value["source_artifact"], "SOURCE_ARTIFACT_INVALID")
)
if binding.governed_task_id != task.task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
expected_hash = _string(
value["task_payload_sha256"],
"TASK_PAYLOAD_SHA_INVALID",
SHA64_RE,
)
if task.payload_sha256 != expected_hash:
raise ProjectionRecoveryError("TASK_PAYLOAD_SHA_MISMATCH", 409)
projection_id = _string(
value["projection_id"],
"PROJECTION_ID_INVALID",
TASK_ID_RE,
)
idempotency_key = _string(
value["idempotency_key"],
"IDEMPOTENCY_KEY_INVALID",
TASK_ID_RE,
)
generation = _integer(
value["projection_generation"],
"PROJECTION_GENERATION_INVALID",
minimum=1,
)
return cls(
projection_id=projection_id,
projection_generation=generation,
idempotency_key=idempotency_key,
task=task,
binding=binding,
source_artifact=source,
)
def public(self) -> dict[str, Any]:
return {
"schema": self.schema,
"projection_id": self.projection_id,
"projection_generation": self.projection_generation,
"idempotency_key": self.idempotency_key,
"task": self.task.public(),
"governed_binding": self.binding.public(),
"source_artifact": asdict(self.source_artifact),
"task_payload_sha256": self.task.payload_sha256,
}
@property
def request_sha256(self) -> str:
return sha256_json(self.public())
class CanonicalTaskProjectionStore:
"""Task-Plane-owned intent/recovery store.
The caller must already be inside the Task Plane authorization boundary.
"""
def __init__(self, db_path: Path | str) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {
str(row["name"]): row
for row in connection.execute(f'PRAGMA table_info("{table}")')
}
@staticmethod
def _table_names(connection: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
@classmethod
def _event_table(
cls,
connection: sqlite3.Connection,
) -> tuple[str, dict[str, sqlite3.Row]]:
tables = cls._table_names(connection)
for name in ("task_events", "events"):
if name in tables:
columns = cls._columns(connection, name)
if "task_id" in columns and "event_type" in columns:
return name, columns
raise ProjectionRecoveryError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _ensure_control_tables(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_intents (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
task_payload_sha256 TEXT NOT NULL,
intent_json TEXT NOT NULL,
state TEXT NOT NULL,
last_error TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(task_id, projection_generation),
UNIQUE(task_id, idempotency_key)
)
"""
)
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_projection_receipts (
projection_id TEXT PRIMARY KEY,
task_id TEXT NOT NULL,
bridge_job_id TEXT NOT NULL,
projection_generation INTEGER NOT NULL,
task_payload_sha256 TEXT NOT NULL,
canonical_revision INTEGER NOT NULL,
state TEXT NOT NULL,
created_new_row INTEGER NOT NULL,
observed_at TEXT NOT NULL,
receipt_json TEXT NOT NULL
)
"""
)
def persist_intent(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
now = _now()
request_sha = intent.request_sha256
intent_text = _canonical(intent.public()).decode("utf-8")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=? AND projection_generation=?
""",
(intent.task.task_id, intent.projection_generation),
).fetchone()
if existing is not None:
if (
str(existing["projection_id"]) != intent.projection_id
or str(existing["idempotency_key"]) != intent.idempotency_key
or str(existing["request_sha256"]) != request_sha
):
raise ProjectionRecoveryError(
"PROJECTION_INTENT_IDEMPOTENCY_CONFLICT",
409,
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": str(existing["state"]),
"idempotent": True,
}
connection.execute(
"""
INSERT INTO task_projection_intents(
projection_id,task_id,bridge_job_id,projection_generation,
idempotency_key,request_sha256,task_payload_sha256,intent_json,
state,last_error,created_at,updated_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.idempotency_key,
request_sha,
intent.task.payload_sha256,
intent_text,
ProjectionState.PENDING.value,
None,
now,
now,
),
)
connection.commit()
return {
"projection_id": intent.projection_id,
"task_id": intent.task.task_id,
"state": ProjectionState.PENDING.value,
"idempotent": False,
}
def _task_row_payload(
self,
row: sqlite3.Row,
) -> dict[str, Any]:
try:
success = json.loads(row["success_criteria_json"])
constraints = json.loads(row["constraints_json"])
capabilities = json.loads(row["required_capabilities_json"])
metadata = json.loads(row["metadata_json"])
except (TypeError, json.JSONDecodeError) as exc:
raise ProjectionRecoveryError("CANONICAL_TASK_JSON_INVALID", 409) from exc
if (
not isinstance(success, list)
or not isinstance(constraints, Mapping)
or not isinstance(capabilities, list)
or not isinstance(metadata, Mapping)
):
raise ProjectionRecoveryError("CANONICAL_TASK_SHAPE_INVALID", 409)
return {
"task_id": str(row["task_id"]),
"goal": str(row["goal"]),
"instructions": str(row["instructions"]),
"success_criteria": success,
"constraints": dict(constraints),
"required_capabilities": capabilities,
"metadata": dict(metadata),
"priority": int(row["priority"]),
"max_attempts": int(row["max_attempts"]),
"deadline": row["deadline"],
}
def _verify_existing_task(
self,
row: sqlite3.Row,
intent: CanonicalTaskProjectionIntent,
) -> int:
observed = self._task_row_payload(row)
observed_hash = sha256_json(observed)
if observed_hash != intent.task.payload_sha256:
raise ProjectionRecoveryError(
"CANONICAL_TASK_PROJECTION_IDENTITY_CONFLICT",
409,
)
return int(row["revision"])
@staticmethod
def _required_task_columns() -> set[str]:
return {
"task_id",
"goal",
"instructions",
"success_criteria_json",
"required_capabilities_json",
"constraints_json",
"metadata_json",
"priority",
"max_attempts",
"attempt",
"status",
"claimed_by",
"lease_token_hash",
"lease_expires_at",
"result_json",
"verification_json",
"revision",
"created_at",
"updated_at",
}
def _insert_task(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> int:
columns = self._columns(connection, "tasks")
if not self._required_task_columns().issubset(columns):
raise ProjectionRecoveryError("TASK_SCHEMA_RECOVERY_UNSUPPORTED", 503)
now = _now()
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"goal": intent.task.goal,
"instructions": intent.task.instructions,
"success_criteria_json": _canonical(
[dict(x) for x in intent.task.success_criteria]
).decode("utf-8"),
"required_capabilities_json": "[]",
"constraints_json": _canonical(dict(intent.task.constraints)).decode("utf-8"),
"metadata_json": _canonical(dict(intent.task.metadata)).decode("utf-8"),
"deadline": intent.task.deadline,
"priority": intent.task.priority,
"max_attempts": intent.task.max_attempts,
"attempt": 0,
"status": "QUEUED",
"claimed_by": None,
"lease_token_hash": None,
"lease_expires_at": None,
"result_json": None,
"verification_json": None,
"revision": 1,
"created_at": now,
"updated_at": now,
}
for optional in (
"result_digest",
"result_idempotency_key",
):
if optional in columns:
values[optional] = None
names = [name for name in values if name in columns]
placeholders = ",".join("?" for _ in names)
connection.execute(
f'INSERT INTO tasks ({",".join(chr(34)+n+chr(34) for n in names)}) '
f"VALUES ({placeholders})",
tuple(values[name] for name in names),
)
self._insert_task_created_event(connection, intent)
return 1
def _insert_task_created_event(
self,
connection: sqlite3.Connection,
intent: CanonicalTaskProjectionIntent,
) -> None:
table, columns = self._event_table(connection)
event = {
"schema": "aios.task-projection-recovery.event.v1",
"task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_id": intent.projection_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"source_artifact_ref": intent.source_artifact.ref,
"source_artifact_sha256": intent.source_artifact.sha256,
"recovery": True,
}
values: dict[str, Any] = {
"task_id": intent.task.task_id,
"event_type": "TASK_CREATED",
}
payload_text = _canonical(event).decode("utf-8")
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise ProjectionRecoveryError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "control-plane-projection-recovery"
if "actor_id" in columns:
values["actor_id"] = "control-plane-projection-recovery"
names = list(values)
connection.execute(
f'INSERT INTO "{table}" ({",".join(chr(34)+n+chr(34) for n in names)}) '
f'VALUES ({",".join("?" for _ in names)})',
tuple(values[name] for name in names),
)
def reconcile(
self,
intent: CanonicalTaskProjectionIntent,
) -> dict[str, Any]:
# Persist first so a crash before Task creation leaves a durable PENDING
# intent for startup/supervisor reconciliation.
self.persist_intent(intent)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_control_tables(connection)
existing_receipt = connection.execute(
"SELECT * FROM task_projection_receipts WHERE projection_id=?",
(intent.projection_id,),
).fetchone()
if existing_receipt is not None:
if (
str(existing_receipt["task_id"]) != intent.task.task_id
or str(existing_receipt["bridge_job_id"]) != intent.binding.bridge_job_id
or str(existing_receipt["task_payload_sha256"])
!= intent.task.payload_sha256
):
raise ProjectionRecoveryError(
"PROJECTION_RECEIPT_IDENTITY_CONFLICT",
409,
)
connection.commit()
receipt = json.loads(existing_receipt["receipt_json"])
receipt["idempotent"] = True
return receipt
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
created_new = False
if row is None:
revision = self._insert_task(connection, intent)
created_new = True
row = connection.execute(
"SELECT * FROM tasks WHERE task_id=?",
(intent.task.task_id,),
).fetchone()
if row is None:
raise ProjectionRecoveryError(
"TASK_RECOVERY_OBSERVATION_FAILED",
503,
)
else:
revision = self._verify_existing_task(row, intent)
# Observation after mutation is mandatory.
revision = self._verify_existing_task(row, intent)
observed_at = _now()
receipt = {
"schema": RECEIPT_SCHEMA,
"projection_id": intent.projection_id,
"governed_task_id": intent.binding.governed_task_id,
"canonical_task_id": intent.task.task_id,
"bridge_job_id": intent.binding.bridge_job_id,
"projection_generation": intent.projection_generation,
"task_payload_sha256": intent.task.payload_sha256,
"canonical_revision": revision,
"state": ProjectionState.CONVERGED.value,
"created_new_row": created_new,
"source_artifact": asdict(intent.source_artifact),
"observed_at": observed_at,
"idempotent": False,
}
connection.execute(
"""
INSERT INTO task_projection_receipts(
projection_id,task_id,bridge_job_id,projection_generation,
task_payload_sha256,canonical_revision,state,created_new_row,
observed_at,receipt_json
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
intent.projection_id,
intent.task.task_id,
intent.binding.bridge_job_id,
intent.projection_generation,
intent.task.payload_sha256,
revision,
ProjectionState.CONVERGED.value,
1 if created_new else 0,
observed_at,
_canonical(receipt).decode("utf-8"),
),
)
connection.execute(
"""
UPDATE task_projection_intents
SET state=?, last_error=NULL, updated_at=?
WHERE projection_id=?
""",
(
ProjectionState.CONVERGED.value,
observed_at,
intent.projection_id,
),
)
connection.commit()
return receipt
def get_projection(
self,
task_id: str,
) -> dict[str, Any] | None:
task_id = _string(task_id, "TASK_ID_INVALID", TASK_ID_RE)
with self._connect() as connection:
self._ensure_control_tables(connection)
intent = connection.execute(
"""
SELECT * FROM task_projection_intents
WHERE task_id=?
ORDER BY projection_generation DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if intent is None:
return None
receipt = connection.execute(
"SELECT receipt_json FROM task_projection_receipts WHERE projection_id=?",
(intent["projection_id"],),
).fetchone()
return {
"projection_id": intent["projection_id"],
"task_id": task_id,
"bridge_job_id": intent["bridge_job_id"],
"projection_generation": int(intent["projection_generation"]),
"state": intent["state"],
"last_error": intent["last_error"],
"task_payload_sha256": intent["task_payload_sha256"],
"receipt": json.loads(receipt[0]) if receipt else None,
}
@@ -0,0 +1,44 @@
"""HTTP integration helpers for canonical Task projection recovery.
This module intentionally contains no HTTP server. Existing Task Plane /
projection-owner handlers should call ``recover_from_json`` only after their
normal service-side ``agent-task:control`` authorization succeeds.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any, Mapping
from aios_agent_task_projection_recovery import (
CanonicalTaskProjectionIntent,
CanonicalTaskProjectionStore,
ProjectionRecoveryError,
)
def recover_from_json(
db_path: Path | str,
task_id: str,
body: Mapping[str, Any],
) -> tuple[int, dict[str, Any]]:
try:
intent = CanonicalTaskProjectionIntent.from_mapping(body)
if intent.task.task_id != task_id:
raise ProjectionRecoveryError("PROJECTION_TASK_IDENTITY_MISMATCH", 409)
result = CanonicalTaskProjectionStore(db_path).reconcile(intent)
return (200 if result.get("idempotent") else 201), result
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
def get_projection(
db_path: Path | str,
task_id: str,
) -> tuple[int, dict[str, Any]]:
try:
result = CanonicalTaskProjectionStore(db_path).get_projection(task_id)
except ProjectionRecoveryError as exc:
return exc.status, {"error": exc.code}
if result is None:
return 404, {"error": "TASK_PROJECTION_NOT_FOUND"}
return 200, result
@@ -1,427 +0,0 @@
"""Owner-controlled, CAS-bound retry extension for exhausted Online Agent Tasks."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
from typing import Any, Mapping
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
KEY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,159}$")
MAX_BODY = 64 * 1024
AUTHORIZE_EXTENSION_ID_ENV = "AIOS_AGENT_TASK_AUTHORIZATION_ID"
class RetryExtensionError(RuntimeError):
def __init__(self, code: str, status: int = 409) -> None:
super().__init__(code)
self.code = code
self.status = status
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _required_authorization_id() -> str:
value = os.environ.get(AUTHORIZE_EXTENSION_ID_ENV, "").strip()
if not value:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_MISSING")
if KEY_RE.fullmatch(value) is None:
raise RuntimeError("TASK_RETRY_AUTHORIZATION_ID_INVALID")
return value
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _require_string(value: object, code: str, pattern: re.Pattern[str] | None = None) -> str:
if not isinstance(value, str) or not value.strip():
raise RetryExtensionError(code, 400)
value = value.strip()
if pattern is not None and pattern.fullmatch(value) is None:
raise RetryExtensionError(code, 400)
return value
@dataclass(frozen=True)
class RetryExtensionRequest:
expected_status: str
expected_revision: int
expected_attempt: int
expected_max_attempts: int
next_max_attempts: int
previous_verification_sha256: str
authorization_id: str
idempotency_key: str
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "RetryExtensionRequest":
allowed = {
"expected_status", "expected_revision", "expected_attempt", "expected_max_attempts",
"next_max_attempts", "previous_verification_sha256", "authorization_id", "idempotency_key",
}
if set(value) != allowed:
raise RetryExtensionError("RETRY_EXTENSION_FIELDS_INVALID", 400)
status = _require_string(value["expected_status"], "EXPECTED_STATUS_INVALID")
if status != "FAILED":
raise RetryExtensionError("EXPECTED_STATUS_INVALID", 400)
ints: dict[str, int] = {}
for field in ("expected_revision", "expected_attempt", "expected_max_attempts", "next_max_attempts"):
item = value[field]
if not isinstance(item, int) or isinstance(item, bool) or item < 0:
raise RetryExtensionError("RETRY_EXTENSION_INTEGER_INVALID", 400)
ints[field] = item
if ints["expected_attempt"] != ints["expected_max_attempts"]:
raise RetryExtensionError("RETRY_EXTENSION_NOT_EXHAUSTED", 409)
if ints["next_max_attempts"] != ints["expected_max_attempts"] + 1:
raise RetryExtensionError("RETRY_EXTENSION_MUST_BE_BOUNDED_ONE_STEP", 400)
if ints["next_max_attempts"] > 50:
raise RetryExtensionError("RETRY_EXTENSION_LIMIT_EXCEEDED", 400)
verification = _require_string(value["previous_verification_sha256"], "VERIFICATION_SHA256_INVALID")
if re.fullmatch(r"[0-9a-f]{64}", verification) is None:
raise RetryExtensionError("VERIFICATION_SHA256_INVALID", 400)
authorization = _require_string(value["authorization_id"], "AUTHORIZATION_ID_INVALID", KEY_RE)
key = _require_string(value["idempotency_key"], "IDEMPOTENCY_KEY_INVALID", KEY_RE)
return cls(
status,
ints["expected_revision"],
ints["expected_attempt"],
ints["expected_max_attempts"],
ints["next_max_attempts"],
verification,
authorization,
key,
)
def public(self) -> dict[str, Any]:
return {
"expected_status": self.expected_status,
"expected_revision": self.expected_revision,
"expected_attempt": self.expected_attempt,
"expected_max_attempts": self.expected_max_attempts,
"next_max_attempts": self.next_max_attempts,
"previous_verification_sha256": self.previous_verification_sha256,
"authorization_id": self.authorization_id,
"idempotency_key": self.idempotency_key,
}
class RetryExtensionStore:
"""Owns the official mutation. Direct/manual DB edits remain forbidden."""
def __init__(self, db_path: Path) -> None:
self.db_path = Path(db_path)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _columns(connection: sqlite3.Connection, table: str) -> dict[str, sqlite3.Row]:
return {str(row["name"]): row for row in connection.execute(f'PRAGMA table_info("{table}")')}
@staticmethod
def _event_table(connection: sqlite3.Connection) -> tuple[str, dict[str, sqlite3.Row]]:
tables = {
row["name"]
for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")
}
for name in ("task_events", "events"):
if name not in tables:
continue
columns = RetryExtensionStore._columns(connection, name)
if "task_id" in columns and ("event_type" in columns or "type" in columns):
return name, columns
raise RetryExtensionError("TASK_EVENT_TABLE_UNSUPPORTED", 503)
@staticmethod
def _verification_hash(raw: object) -> str:
if raw in (None, ""):
value: object = {}
elif isinstance(raw, str):
try:
value = json.loads(raw)
except json.JSONDecodeError as exc:
raise RetryExtensionError("TASK_VERIFICATION_INVALID", 409) from exc
else:
value = raw
return _sha(value)
@staticmethod
def _ensure_extension_table(connection: sqlite3.Connection) -> None:
connection.execute(
"""
CREATE TABLE IF NOT EXISTS task_retry_extensions (
task_id TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
authorization_id TEXT NOT NULL,
previous_verification_sha256 TEXT NOT NULL,
previous_max_attempts INTEGER NOT NULL,
next_max_attempts INTEGER NOT NULL,
expected_revision INTEGER NOT NULL,
applied_revision INTEGER NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, idempotency_key)
)
"""
)
@staticmethod
def _insert_event(
connection: sqlite3.Connection,
task_id: str,
payload: Mapping[str, Any],
) -> None:
table, columns = RetryExtensionStore._event_table(connection)
values: dict[str, Any] = {"task_id": task_id}
if "event_type" in columns:
values["event_type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
else:
values["type"] = "TASK_RETRY_EXTENSION_AUTHORIZED"
payload_text = json.dumps(payload, sort_keys=True, separators=(",", ":"))
if "payload_json" in columns:
values["payload_json"] = payload_text
elif "payload" in columns:
values["payload"] = payload_text
else:
raise RetryExtensionError("TASK_EVENT_PAYLOAD_UNSUPPORTED", 503)
if "created_at" in columns:
values["created_at"] = _now()
if "agent_id" in columns:
values["agent_id"] = "owner-control"
if "actor_id" in columns:
values["actor_id"] = "owner-control"
names = list(values)
sql = f'INSERT INTO "{table}" ({",".join(f"""\"{name}\"""" for name in names)}) VALUES ({",".join("?" for _ in names)})'
connection.execute(sql, tuple(values[name] for name in names))
def extend(self, task_id: str, request: RetryExtensionRequest) -> dict[str, Any]:
task_id = _require_string(task_id, "TASK_ID_INVALID", TASK_RE)
required_authorization_id = _required_authorization_id()
if request.authorization_id != required_authorization_id:
raise RetryExtensionError("RETRY_EXTENSION_AUTHORIZATION_ID_MISMATCH", 403)
request_payload = request.public()
request_sha = _sha(request_payload)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
self._ensure_extension_table(connection)
replay = connection.execute(
"SELECT * FROM task_retry_extensions WHERE task_id=? AND idempotency_key=?",
(task_id, request.idempotency_key),
).fetchone()
if replay is not None:
if replay["request_sha256"] != request_sha:
raise RetryExtensionError("RETRY_EXTENSION_IDEMPOTENCY_CONFLICT", 409)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": int(replay["next_max_attempts"]),
"revision": int(replay["applied_revision"]),
"idempotent": True,
"authorization_id": replay["authorization_id"],
"previous_verification_sha256": replay["previous_verification_sha256"],
}
columns = self._columns(connection, "tasks")
required = {
"task_id", "status", "attempt", "max_attempts", "revision", "claimed_by",
"lease_token_hash", "lease_expires_at", "verification_json",
}
if not required.issubset(columns):
raise RetryExtensionError("TASK_SCHEMA_RETRY_EXTENSION_UNSUPPORTED", 503)
task = connection.execute("SELECT * FROM tasks WHERE task_id=?", (task_id,)).fetchone()
if task is None:
raise RetryExtensionError("TASK_NOT_FOUND", 404)
actual_verification_sha = self._verification_hash(task["verification_json"])
if actual_verification_sha != request.previous_verification_sha256:
raise RetryExtensionError("RETRY_EXTENSION_VERIFICATION_DRIFT", 409)
if (
task["status"] != request.expected_status
or int(task["revision"]) != request.expected_revision
or int(task["attempt"]) != request.expected_attempt
or int(task["max_attempts"]) != request.expected_max_attempts
):
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
for lease_field in ("lease_token_hash", "lease_expires_at"):
if lease_field in columns and task[lease_field] not in (None, ""):
raise RetryExtensionError("RETRY_EXTENSION_ACTIVE_LEASE_DENY", 409)
previous_claimed_by = task["claimed_by"]
applied_revision = request.expected_revision + 1
cursor = connection.execute(
"""
UPDATE tasks
SET status='QUEUED', max_attempts=?, revision=?, claimed_by=NULL,
lease_token_hash=NULL, lease_expires_at=NULL
WHERE task_id=? AND status=? AND revision=? AND attempt=? AND max_attempts=?
AND claimed_by IS ?
AND lease_token_hash IS NULL AND lease_expires_at IS NULL
""",
(
request.next_max_attempts,
applied_revision,
task_id,
request.expected_status,
request.expected_revision,
request.expected_attempt,
request.expected_max_attempts,
previous_claimed_by,
),
)
if cursor.rowcount != 1:
raise RetryExtensionError("RETRY_EXTENSION_CAS_MISMATCH", 409)
event_payload = {
"schema": "aios.task-retry-extension.event.v1",
"task_id": task_id,
"authorization_id": request.authorization_id,
"idempotency_key": request.idempotency_key,
"previous_verification_sha256": request.previous_verification_sha256,
"previous_max_attempts": request.expected_max_attempts,
"next_max_attempts": request.next_max_attempts,
"previous_revision": request.expected_revision,
"applied_revision": applied_revision,
"request_sha256": request_sha,
"previous_claimed_by": previous_claimed_by,
"stale_terminal_claim_normalized": previous_claimed_by not in (None, ""),
}
self._insert_event(connection, task_id, event_payload)
connection.execute(
"""
INSERT INTO task_retry_extensions(
task_id,idempotency_key,request_sha256,authorization_id,previous_verification_sha256,
previous_max_attempts,next_max_attempts,expected_revision,applied_revision,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?)
""",
(
task_id,
request.idempotency_key,
request_sha,
request.authorization_id,
request.previous_verification_sha256,
request.expected_max_attempts,
request.next_max_attempts,
request.expected_revision,
applied_revision,
_now(),
),
)
connection.commit()
return {
"task_id": task_id,
"status": "QUEUED",
"attempt": request.expected_attempt,
"max_attempts": request.next_max_attempts,
"revision": applied_revision,
"idempotent": False,
"authorization_id": request.authorization_id,
"previous_verification_sha256": request.previous_verification_sha256,
}
class RetryExtensionHandler(BaseHTTPRequestHandler):
store: RetryExtensionStore | None = None
control_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def _json(self, value: Mapping[str, Any], status: int) -> None:
data = json.dumps(dict(value), sort_keys=True, separators=(",", ":")).encode("utf-8")
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def _authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.control_token)
and hmac.compare_digest(header[7:], self.control_token)
)
def do_GET(self) -> None:
if self.path == "/health":
self._json({"status": "ok", "service": "aios-agent-task-retry-extension"}, 200)
else:
self._json({"error": "NOT_FOUND"}, 404)
def do_POST(self) -> None:
match = re.fullmatch(r"/v1/tasks/([^/]+)/retry-extension", self.path)
if match is None:
self._json({"error": "NOT_FOUND"}, 404)
return
if not self._authorized():
self._json({"error": "UNAUTHORIZED"}, 401)
return
try:
length = int(self.headers.get("Content-Length", "0"))
except ValueError:
self._json({"error": "CONTENT_LENGTH_INVALID"}, 400)
return
if length < 1 or length > MAX_BODY:
self._json({"error": "REQUEST_SIZE_INVALID"}, 413)
return
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
if not isinstance(value, Mapping):
raise ValueError
request = RetryExtensionRequest.from_mapping(value)
if self.store is None:
raise RetryExtensionError("RUNTIME_UNAVAILABLE", 503)
result = self.store.extend(match.group(1), request)
self._json(result, 200 if result["idempotent"] else 201)
except RetryExtensionError as exc:
self._json({"error": exc.code}, exc.status)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError):
self._json({"error": "REQUEST_INVALID"}, 400)
def load_control_token(path: Path) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32:
raise RuntimeError("TASK_RETRY_CONTROL_CREDENTIAL_INVALID")
return value
def serve() -> None:
db = Path(os.environ.get("AIOS_AGENT_TASK_DB", "/var/lib/aios-agent-task/state.sqlite3"))
_required_authorization_id()
token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials")) / "agent_task_control_token"),
)
)
host = os.environ.get("AIOS_TASK_RETRY_HOST", "127.0.0.1")
port = int(os.environ.get("AIOS_TASK_RETRY_PORT", "18941"))
RetryExtensionHandler.store = RetryExtensionStore(db)
RetryExtensionHandler.control_token = load_control_token(token_path)
ThreadingHTTPServer((host, port), RetryExtensionHandler).serve_forever()
if __name__ == "__main__":
serve()
@@ -1,470 +0,0 @@
#!/usr/bin/env python3
"""Canonical AIOS all-modified-files evidence packager.
The bundle contract is intentionally compatible with the historical
aios_package_all_modified_files_v1.sh output while keeping secret scanning in
version-controlled, tested code.
"""
from __future__ import annotations
from datetime import datetime, timezone
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import tempfile
import zipfile
from typing import Iterable
OID_RE = re.compile(r"^[0-9a-f]{40}$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
# A PEM private-key finding requires the complete PEM header. Source code,
# tests, docs, or error text containing only "BEGIN PRIVATE KEY" are not
# credentials and must not be rejected.
PEM_PRIVATE_KEY_RE = re.compile(
rb"-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----",
re.IGNORECASE,
)
BEARER_RE = re.compile(
rb"authorization\s*:\s*bearer\s+[A-Za-z0-9._~+/=-]{20,}",
re.IGNORECASE,
)
KEY_VALUE_SECRET_RE = re.compile(
rb"(?:password|passwd|secret|api[_-]?key)\s*[:=]\s*(?!//)[^\s,;]{12,}",
re.IGNORECASE,
)
DOCUMENT_SUFFIXES = {".md", ".rst", ".adoc", ".txt"}
DOCUMENT_PREFIXES = (
"README",
"CHANGELOG",
"RUNBOOK",
"ADR",
"ARCHITECTURE",
"DEPLOYMENT",
"TROUBLESHOOTING",
)
class PackagerError(RuntimeError):
pass
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def secret_finding(data: bytes) -> str | None:
if PEM_PRIVATE_KEY_RE.search(data):
return "PEM_PRIVATE_KEY"
if BEARER_RE.search(data):
return "AUTHORIZATION_BEARER"
if KEY_VALUE_SECRET_RE.search(data):
return "KEY_VALUE_SECRET"
return None
def scan_secret(data: bytes, label: str) -> None:
finding = secret_finding(data)
if finding is not None:
raise PackagerError(
f"MODIFIED_FILES_SECRET_SCAN_MATCH:{finding}:{label}"
)
def git_bytes(git_dir: Path, oid: str, path: str) -> bytes:
try:
return subprocess.check_output(
[
"/usr/bin/git",
f"--git-dir={git_dir}",
"show",
f"{oid}:{path}",
],
timeout=30,
)
except subprocess.CalledProcessError as exc:
raise PackagerError(
"MODIFIED_FILES_GIT_SHOW_FAILED:" + path
) from exc
def git_text(git_dir: Path, *args: str) -> str:
try:
return subprocess.check_output(
["/usr/bin/git", f"--git-dir={git_dir}", *args],
text=True,
timeout=30,
)
except subprocess.CalledProcessError as exc:
raise PackagerError(
"MODIFIED_FILES_GIT_COMMAND_FAILED:" + " ".join(args)
) from exc
def validate_lineage(git_dir: Path, base: str, candidate: str) -> None:
if OID_RE.fullmatch(base) is None or OID_RE.fullmatch(candidate) is None:
raise PackagerError("MODIFIED_FILES_OID_INVALID")
for oid in (base, candidate):
try:
subprocess.run(
[
"/usr/bin/git",
f"--git-dir={git_dir}",
"cat-file",
"-e",
f"{oid}^{{commit}}",
],
check=True,
timeout=30,
)
except subprocess.CalledProcessError as exc:
raise PackagerError("MODIFIED_FILES_COMMIT_MISSING:" + oid) from exc
if subprocess.run(
[
"/usr/bin/git",
f"--git-dir={git_dir}",
"merge-base",
"--is-ancestor",
base,
candidate,
],
check=False,
timeout=30,
).returncode != 0:
raise PackagerError("MODIFIED_FILES_LINEAGE_INVALID")
def safe_destination(root: Path, relative: str) -> Path:
path = Path(relative)
if path.is_absolute() or ".." in path.parts:
raise PackagerError("MODIFIED_FILE_PATH_INVALID:" + relative)
destination = root / path
destination.parent.mkdir(parents=True, exist_ok=True)
return destination
def is_document_path(path: str) -> bool:
value = Path(path)
if path.startswith("docs/"):
return True
if value.suffix.lower() in DOCUMENT_SUFFIXES:
return True
upper = value.name.upper()
return any(upper.startswith(prefix) for prefix in DOCUMENT_PREFIXES)
def diff_entries(
git_dir: Path,
base: str,
candidate: str,
before_root: Path,
after_root: Path,
) -> list[dict[str, object]]:
raw = git_text(
git_dir,
"diff",
"--name-status",
"-M",
"-C",
base,
candidate,
)
entries: list[dict[str, object]] = []
for line in raw.splitlines():
if not line.strip():
continue
parts = line.split("\t")
status = parts[0]
code = status[0]
old_path: str | None = None
new_path: str | None = None
if code in {"R", "C"}:
if len(parts) != 3:
raise PackagerError(
"MODIFIED_FILES_DIFF_PARSE_INVALID:" + line
)
old_path, new_path = parts[1], parts[2]
else:
if len(parts) != 2:
raise PackagerError(
"MODIFIED_FILES_DIFF_PARSE_INVALID:" + line
)
new_path = parts[1] if code != "D" else None
old_path = parts[1] if code != "A" else None
entry: dict[str, object] = {"status": status}
if old_path is not None:
entry["previous_path"] = old_path
before = git_bytes(git_dir, base, old_path)
scan_secret(before, "before:" + old_path)
safe_destination(before_root, old_path).write_bytes(before)
entry["before_sha256"] = sha256(before)
entry["before_bytes"] = len(before)
if new_path is not None:
entry["path"] = new_path
after = git_bytes(git_dir, candidate, new_path)
scan_secret(after, "after:" + new_path)
safe_destination(after_root, new_path).write_bytes(after)
entry["after_sha256"] = sha256(after)
entry["after_bytes"] = len(after)
entry["deleted"] = code == "D"
entry["added"] = code == "A"
entry["renamed"] = code == "R"
entry["copied"] = code == "C"
entries.append(entry)
return entries
def execution_entries(
extra_root: Path | None,
execution_root: Path,
) -> list[dict[str, object]]:
if extra_root is None or not extra_root.exists():
return []
if extra_root.is_symlink() or not extra_root.is_dir():
raise PackagerError("EXTRA_FILES_DIR_INVALID")
entries: list[dict[str, object]] = []
for path in sorted(extra_root.rglob("*")):
if path.is_symlink():
raise PackagerError(
"EXTRA_FILE_SYMLINK_REJECTED:" + str(path)
)
if not path.is_file():
continue
relative = path.relative_to(extra_root).as_posix()
data = path.read_bytes()
scan_secret(data, "execution:" + relative)
safe_destination(execution_root, relative).write_bytes(data)
entries.append(
{
"path": relative,
"sha256": sha256(data),
"bytes": len(data),
}
)
return entries
def write_checksums(bundle: Path) -> None:
checks: list[tuple[str, str]] = []
for path in sorted(bundle.rglob("*")):
if path.is_file() and path.name != "checksums.sha256":
checks.append(
(
sha256(path.read_bytes()),
path.relative_to(bundle).as_posix(),
)
)
(bundle / "checksums.sha256").write_text(
"".join(f"{digest} {name}\n" for digest, name in checks),
encoding="utf-8",
)
def verify_checksums(bundle: Path) -> None:
checksum_path = bundle / "checksums.sha256"
for line in checksum_path.read_text(encoding="utf-8").splitlines():
digest, relative = line.split(" ", 1)
path = bundle / relative
if not path.is_file() or sha256(path.read_bytes()) != digest:
raise PackagerError(
"MODIFIED_FILES_CHECKSUM_CLOSURE_INVALID:" + relative
)
def build_bundle(
*,
git_dir: Path,
base: str,
candidate: str,
task_id: str,
extra_dir: Path | None = None,
output_root: Path | None = None,
now: datetime | None = None,
) -> dict[str, object]:
if TASK_RE.fullmatch(task_id) is None:
raise PackagerError("TASK_ID_INVALID")
validate_lineage(git_dir, base, candidate)
timestamp = (now or datetime.now(timezone.utc)).strftime(
"%Y%m%dT%H%M%SZ"
)
safe_task = re.sub(r"[^A-Za-z0-9._-]+", "-", task_id).strip("-")
if output_root is None:
output_root = Path(
tempfile.mkdtemp(
prefix=f"aios-modified-files-bundle-{safe_task}-{timestamp}-",
dir="/var/tmp",
)
)
else:
output_root.mkdir(parents=True, exist_ok=True)
bundle = output_root / "bundle"
before_root = bundle / "repo" / "before"
after_root = bundle / "repo" / "after"
execution_root = bundle / "execution"
before_root.mkdir(parents=True)
after_root.mkdir(parents=True)
execution_root.mkdir(parents=True)
changes = diff_entries(
git_dir,
base,
candidate,
before_root,
after_root,
)
execution_files = execution_entries(extra_dir, execution_root)
document_changes = [
entry
for entry in changes
if is_document_path(
str(entry.get("path") or entry.get("previous_path") or "")
)
]
execution_documents = [
entry
for entry in execution_files
if is_document_path(str(entry["path"]))
]
documents_index = {
"schema": "aios.modified-documents.index.v1",
"task_id": task_id,
"base_oid": base,
"candidate_oid": candidate,
"repository_documents": document_changes,
"execution_documents": execution_documents,
"repository_document_count": len(document_changes),
"execution_document_count": len(execution_documents),
}
(bundle / "documents-index.json").write_text(
json.dumps(documents_index, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
manifest = {
"schema": "aios.all-modified-files.bundle.v1",
"task_id": task_id,
"base_oid": base,
"candidate_oid": candidate,
"generated_at": (
now or datetime.now(timezone.utc)
).isoformat(),
"changed_file_count": len(changes),
"execution_file_count": len(execution_files),
"repository_document_count": len(document_changes),
"execution_document_count": len(execution_documents),
"changes": changes,
"execution_files": execution_files,
}
(bundle / "all-modified-files-manifest.json").write_text(
json.dumps(manifest, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
write_checksums(bundle)
verify_checksums(bundle)
zip_name = (
f"AIOS_{task_id}_ALL_MODIFIED_FILES_{candidate[:12]}.zip"
.replace("/", "-")
)
zip_path = output_root / zip_name
with zipfile.ZipFile(
zip_path,
"w",
compression=zipfile.ZIP_DEFLATED,
) as archive:
for path in sorted(bundle.rglob("*")):
if path.is_symlink():
raise PackagerError(
"BUNDLE_SYMLINK_REJECTED:" + str(path)
)
if path.is_file():
archive.write(
path,
arcname=path.relative_to(bundle).as_posix(),
)
zip_sha = sha256(zip_path.read_bytes())
summary = {
"schema": "aios.all-modified-files.bundle-summary.v1",
"task_id": task_id,
"base_oid": base,
"candidate_oid": candidate,
"bundle_path": str(zip_path),
"bundle_sha256": zip_sha,
"changed_file_count": len(changes),
"execution_file_count": len(execution_files),
"repository_document_count": len(document_changes),
"execution_document_count": len(execution_documents),
}
(output_root / "ALL_MODIFIED_FILES_BUNDLE.json").write_text(
json.dumps(summary, indent=2, sort_keys=True) + "\n",
encoding="utf-8",
)
return summary
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("base")
parser.add_argument("candidate")
parser.add_argument("task_id")
parser.add_argument("extra_dir", nargs="?")
parser.add_argument(
"--authority-git-dir",
default=os.environ.get(
"AIOS_AUTHORITY_GIT_DIR",
"/data/aios-gitops/repo.git",
),
)
parser.add_argument("--output-root")
args = parser.parse_args()
try:
summary = build_bundle(
git_dir=Path(args.authority_git_dir),
base=args.base,
candidate=args.candidate,
task_id=args.task_id,
extra_dir=Path(args.extra_dir) if args.extra_dir else None,
output_root=(
Path(args.output_root) if args.output_root else None
),
)
except PackagerError as exc:
print("FATAL:" + str(exc))
return 20
print("ALL_MODIFIED_FILES_BUNDLE=PASS")
print("ALL_MODIFIED_FILES_BUNDLE_PATH=" + str(summary["bundle_path"]))
print("ALL_MODIFIED_FILES_BUNDLE_SHA256=" + str(summary["bundle_sha256"]))
print("CHANGED_FILE_COUNT=" + str(summary["changed_file_count"]))
print("EXECUTION_FILE_COUNT=" + str(summary["execution_file_count"]))
print(
"REPOSITORY_DOCUMENT_COUNT="
+ str(summary["repository_document_count"])
)
print(
"EXECUTION_DOCUMENT_COUNT="
+ str(summary["execution_document_count"])
)
print(
"ALL_MODIFIED_FILES_EVIDENCE="
+ str(Path(str(summary["bundle_path"])).parent)
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -94,89 +94,23 @@ class AuthorityGitExecutor:
self._git(["worktree", "add", "--detach", str(path), base_oid])
return path
def recover_failed_detached_worktree_exact(
self,
*,
path: Path,
expected_oid: str,
stale_oid: str,
target_ref: str,
) -> str:
"""Recreate one exact clean, detached, single-attempt worktree at its durable head."""
_oid(expected_oid, "expected_oid")
_oid(stale_oid, "stale_oid")
if not TASK_BRANCH_RE.fullmatch(target_ref):
def create_task_branch(self, branch: str, base_oid: str) -> str:
"""Create an isolated agent branch once, using an empty-ref CAS."""
ref = branch if branch.startswith("refs/heads/") else "refs/heads/" + branch
if not TASK_BRANCH_RE.fullmatch(ref):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
if self.read_ref(target_ref) is not None:
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if (
not path.is_absolute()
or path.parent != self.worktree_root
or path.name in {"", ".", ".."}
or path.is_symlink()
or not path.is_dir()
):
raise AuthorityGitError("WORKTREE_RECOVERY_PATH_INVALID")
head = subprocess.run(
["git", "-C", str(path), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
if head.returncode != 0 or head.stdout.strip() != stale_oid or stale_oid == expected_oid:
raise AuthorityGitError("WORKTREE_RECOVERY_HEAD_INVALID")
symbolic = subprocess.run(
["git", "-C", str(path), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if symbolic.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_DETACHED")
status = subprocess.run(
["git", "-C", str(path), "status", "--porcelain=v1", "--untracked-files=all"],
text=True, capture_output=True, check=False,
)
if status.returncode != 0 or status.stdout:
raise AuthorityGitError("WORKTREE_RECOVERY_NOT_CLEAN")
parents = self._git(["rev-list", "--parents", "-n", "1", stale_oid]).split()
if parents != [stale_oid, expected_oid]:
raise AuthorityGitError("WORKTREE_RECOVERY_LINEAGE_INVALID")
registered = [
line.removeprefix("worktree ")
for line in self._git(["worktree", "list", "--porcelain"]).splitlines()
if line.startswith("worktree ")
]
if registered.count(str(path)) != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_REGISTRATION_INVALID")
self._git(["worktree", "remove", "--force", str(path)])
if path.exists() or path.is_symlink():
raise AuthorityGitError("WORKTREE_RECOVERY_REMOVE_FAILED")
recreated = self.create_detached_worktree(expected_oid, path.name, path.parent)
readback = subprocess.run(
["git", "-C", str(recreated), "rev-parse", "HEAD"],
text=True, capture_output=True, check=False,
)
detached = subprocess.run(
["git", "-C", str(recreated), "symbolic-ref", "-q", "HEAD"],
text=True, capture_output=True, check=False,
)
if readback.returncode != 0 or readback.stdout.strip() != expected_oid or detached.returncode != 1:
raise AuthorityGitError("WORKTREE_RECOVERY_READBACK_MISMATCH")
return stale_oid
def create_task_branch_CAS(self, branch: str, expected_base_oid: str, new_oid: str) -> str:
if not TASK_BRANCH_RE.fullmatch(branch):
raise AuthorityGitError("TASK_BRANCH_SCOPE_DENIED")
_oid(expected_base_oid, "expected_base_oid")
_oid(new_oid, "new_oid")
current = self.read_ref(branch)
_oid(base_oid, "base_oid")
if not self.commit_exists(base_oid):
raise AuthorityGitError("BASE_COMMIT_NOT_FOUND")
current = self.read_ref(ref)
if current is not None:
if current == new_oid:
return new_oid
raise AuthorityGitError("AUTHORITY_TARGET_BRANCH_ALREADY_EXISTS")
if not self.ancestry_check(expected_base_oid, new_oid):
raise AuthorityGitError("NEW_OID_NOT_DESCENDANT")
self._git(["update-ref", branch, new_oid, "0" * 40])
readback = self.read_ref(branch)
if readback != new_oid:
raise AuthorityGitError("AUTHORITY_CREATE_READBACK_MISMATCH")
if current == base_oid:
return current
raise AuthorityGitError("AUTHORITY_BRANCH_EXPECTATION_MISMATCH")
self._git(["update-ref", ref, base_oid, "0" * 40])
readback = self.read_ref(ref)
if readback != base_oid:
raise AuthorityGitError("AUTHORITY_CAS_READBACK_MISMATCH")
return readback
def advance_task_branch_CAS(self, branch: str, expected_oid: str, new_oid: str) -> str:
@@ -1,288 +0,0 @@
"""Unified autonomous control-plane primitives.
This is deliberately a library, not another daemon. Task API, workers and host
recovery code consume the same Admission -> Invariant -> Repair semantics.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
from enum import Enum
import re
from typing import Callable, Iterable, Mapping, Sequence
from aios_capability_contract import (
capability_contract,
has_capabilities,
normalize_required_capabilities,
normalize_worker_capabilities,
registered_capability,
validate_capability_contract_registry,
)
class FailureClass(str, Enum):
CAPABILITY_CONTRACT_ERROR = "CAPABILITY_CONTRACT_ERROR"
CREDENTIAL_PROJECTION_ERROR = "CREDENTIAL_PROJECTION_ERROR"
DEPENDENCY_UNAVAILABLE = "DEPENDENCY_UNAVAILABLE"
POLICY_DENIED = "POLICY_DENIED"
RESOURCE_UNAVAILABLE = "RESOURCE_UNAVAILABLE"
WORKER_CRASH = "WORKER_CRASH"
LEASE_LOST = "LEASE_LOST"
BUSINESS_PERMANENT = "BUSINESS_PERMANENT"
UNKNOWN = "UNKNOWN"
class RepairLevel(int, Enum):
AUTO_LOCAL = 0
AUTO_REPAIR_TASK = 1
OWNER_APPROVAL = 2
MANUAL = 3
@dataclass(frozen=True)
class InvariantResult:
name: str
passed: bool
detail: str = ""
def as_dict(self) -> dict[str, object]:
return asdict(self)
@dataclass(frozen=True)
class AdmissionDecision:
admitted: bool
state: str
invariants: tuple[InvariantResult, ...]
failure_class: FailureClass | None = None
repair_level: RepairLevel | None = None
repair_action: str | None = None
def as_dict(self) -> dict[str, object]:
return {
"schema": "aios.runtime-admission.decision.v1",
"admitted": self.admitted,
"state": self.state,
"invariants": [item.as_dict() for item in self.invariants],
"failure_class": None if self.failure_class is None else self.failure_class.value,
"repair_level": None if self.repair_level is None else int(self.repair_level),
"repair_action": self.repair_action,
}
@dataclass(frozen=True)
class RepairDecision:
failure_class: FailureClass
level: RepairLevel
action: str
consumes_business_attempt: bool
def as_dict(self) -> dict[str, object]:
return {
"failure_class": self.failure_class.value,
"level": int(self.level),
"action": self.action,
"consumes_business_attempt": self.consumes_business_attempt,
}
def classify_failure(error: object, *, phase: str = "") -> FailureClass:
text = (str(error) + " " + str(phase)).lower()
if any(token in text for token in (
"credential", "permission denied", "token registry", "secret", "unauthorized"
)):
return FailureClass.CREDENTIAL_PROJECTION_ERROR
if any(token in text for token in (
"transport", "unavailable", "timeout", "connection", "bad gateway", "http_5"
)):
return FailureClass.DEPENDENCY_UNAVAILABLE
if any(token in text for token in (
"capability", "eligible", "matcher", "contract"
)):
return FailureClass.CAPABILITY_CONTRACT_ERROR
if any(token in text for token in (
"policy", "forbidden", "approval required", "not authorized"
)):
return FailureClass.POLICY_DENIED
if any(token in text for token in ("lease lost", "lease_expired", "heartbeat_failed")):
return FailureClass.LEASE_LOST
if any(token in text for token in ("out of memory", "disk full", "resource")):
return FailureClass.RESOURCE_UNAVAILABLE
if any(token in text for token in ("worker crash", "process exited", "signal")):
return FailureClass.WORKER_CRASH
if any(token in text for token in ("invalid release", "business permanent")):
return FailureClass.BUSINESS_PERMANENT
return FailureClass.UNKNOWN
def repair_for_failure(failure: FailureClass) -> RepairDecision:
table = {
FailureClass.CAPABILITY_CONTRACT_ERROR: RepairDecision(
failure, RepairLevel.AUTO_REPAIR_TASK,
"create_same_lineage_capability_contract_repair", False,
),
FailureClass.CREDENTIAL_PROJECTION_ERROR: RepairDecision(
failure, RepairLevel.AUTO_REPAIR_TASK,
"restore_service_private_credential_projection", False,
),
FailureClass.DEPENDENCY_UNAVAILABLE: RepairDecision(
failure, RepairLevel.AUTO_LOCAL,
"retry_dependency_health_and_resume", False,
),
FailureClass.RESOURCE_UNAVAILABLE: RepairDecision(
failure, RepairLevel.AUTO_LOCAL,
"heal_runtime_resource_and_resume", False,
),
FailureClass.WORKER_CRASH: RepairDecision(
failure, RepairLevel.AUTO_LOCAL,
"restart_worker_and_resume_same_lineage", False,
),
FailureClass.LEASE_LOST: RepairDecision(
failure, RepairLevel.AUTO_LOCAL,
"recover_lease_and_resume_same_lineage", False,
),
FailureClass.POLICY_DENIED: RepairDecision(
failure, RepairLevel.OWNER_APPROVAL,
"request_bound_owner_authorization", False,
),
FailureClass.BUSINESS_PERMANENT: RepairDecision(
failure, RepairLevel.MANUAL,
"terminal_business_failure", True,
),
FailureClass.UNKNOWN: RepairDecision(
failure, RepairLevel.AUTO_REPAIR_TASK,
"create_diagnostic_repair_task", False,
),
}
return table[failure]
def _probe(
name: str,
probes: Mapping[str, Callable[[], bool]],
) -> InvariantResult:
fn = probes.get(name)
if fn is None:
return InvariantResult(name, False, "PROBE_MISSING")
try:
passed = bool(fn())
except Exception as exc:
return InvariantResult(name, False, "PROBE_ERROR:" + type(exc).__name__)
return InvariantResult(name, passed, "PASS" if passed else "NOT_READY")
def runtime_admission(
*,
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
requested_operations: Sequence[str] = (),
probes: Mapping[str, Callable[[], bool]] | None = None,
) -> AdmissionDecision:
"""Admission is evaluated before claim; failures do not consume attempts."""
worker_values = tuple(worker_capabilities)
required_values = tuple(required_capabilities)
probe_map = probes or {}
invariants: list[InvariantResult] = []
registry = validate_capability_contract_registry()
invariants.append(InvariantResult(
"capability_contract_registry",
bool(registry.get("valid")),
"PASS" if registry.get("valid") else "REGISTRY_INVALID",
))
task_known = all(registered_capability(item) for item in required_values)
worker_known = all(registered_capability(item, worker=True) for item in worker_values)
invariants.extend((
InvariantResult("task_capabilities_registered", task_known, "PASS" if task_known else "UNKNOWN_TASK_CAPABILITY"),
InvariantResult("worker_capabilities_registered", worker_known, "PASS" if worker_known else "UNKNOWN_WORKER_CAPABILITY"),
))
capability_match = (
task_known
and worker_known
and has_capabilities(worker_values, required_values)
)
invariants.append(InvariantResult(
"capability_match",
capability_match,
"PASS" if capability_match else "WORKER_NOT_ELIGIBLE",
))
normalized = normalize_required_capabilities(required_values)
contracts = [
capability_contract(item)
for item in normalized
if not str(item).startswith("exact:")
]
contracts = [item for item in contracts if item is not None]
forbidden: set[str] = set()
dependencies: set[str] = set()
health: set[str] = set()
for contract in contracts:
forbidden.update(contract.forbidden)
dependencies.update(contract.dependencies)
health.update(contract.health_requirements)
requested = {str(item).strip().lower() for item in requested_operations if str(item).strip()}
forbidden_hit = sorted(requested & {item.lower() for item in forbidden})
invariants.append(InvariantResult(
"forbidden_operations_absent",
not forbidden_hit,
"PASS" if not forbidden_hit else "FORBIDDEN:" + ",".join(forbidden_hit),
))
for name in sorted(dependencies | health):
invariants.append(_probe(name, probe_map))
failed = [item for item in invariants if not item.passed]
if not failed:
return AdmissionDecision(
True,
"EXECUTION_READY",
tuple(invariants),
)
details = " ".join(item.name + ":" + item.detail for item in failed)
failure = classify_failure(details, phase="admission")
if any(item.name == "forbidden_operations_absent" and not item.passed for item in failed):
failure = FailureClass.POLICY_DENIED
elif any("credential" in item.name for item in failed):
failure = FailureClass.CREDENTIAL_PROJECTION_ERROR
elif any(item.name in {"capability_contract_registry", "task_capabilities_registered", "worker_capabilities_registered", "capability_match"} for item in failed):
failure = FailureClass.CAPABILITY_CONTRACT_ERROR
elif any(item.name in dependencies or item.name in health for item in failed):
failure = FailureClass.DEPENDENCY_UNAVAILABLE
repair = repair_for_failure(failure)
return AdmissionDecision(
False,
"EXECUTION_READY_FAILED",
tuple(invariants),
failure,
repair.level,
repair.action,
)
def system_invariant_snapshot(
*,
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
requested_operations: Sequence[str] = (),
probes: Mapping[str, Callable[[], bool]] | None = None,
) -> dict[str, object]:
decision = runtime_admission(
worker_capabilities=worker_capabilities,
required_capabilities=required_capabilities,
requested_operations=requested_operations,
probes=probes,
)
return {
"schema": "aios.system-invariants.v1",
"status": "HEALTHY" if decision.admitted else "DEGRADED",
"execution_ready": decision.admitted,
"admission": decision.as_dict(),
}
@@ -0,0 +1,2 @@
"""Implementation Baseline 1.1 integrations for existing AIOS authorities."""
BASELINE_VERSION = '1.1'
File diff suppressed because it is too large Load Diff
+281
View File
@@ -0,0 +1,281 @@
"""Executable validation profile and thin Action client.
This supplied ZIP does not establish the online Task authority. Consequently
the runnable profile creates an explicitly isolated validation authority. Online
activation requires the authority mapping gates documented in this package.
"""
from __future__ import annotations
import argparse
import json
import os
from pathlib import Path
import secrets
import re
import signal
import stat
import threading
import time
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey,Ed25519PublicKey
from task_plane.store import TaskPlaneStore
from aios_gitops_job_controller import JobStore
from .authorization import AuthorizationService,sign_owner_delegation_grant
from .host import SystemdHost
from .lineage import LineageLedger
from .operations import OperationRegistry
from .runtime_contract import validate_contract
from .service import BaselineService,Identity
from .worker_policy import WorkerProfile
def _unique_object(pairs):
result={}
for key,value in pairs:
if key in result: raise ValueError('DUPLICATE_JSON_KEY')
result[key]=value
return result
def read_json(path):
return json.loads(_safe_read(Path(path).absolute()).decode('utf-8'),object_pairs_hook=_unique_object,
parse_constant=lambda _value: (_ for _ in ()).throw(ValueError('NON_FINITE_JSON')))
def _path(value):
if not isinstance(value,str): raise ValueError('PATH_INVALID')
prefix='${CREDENTIALS_DIRECTORY}/'
if value.startswith(prefix):
directory=os.environ.get('CREDENTIALS_DIRECTORY')
name=value[len(prefix):]
if not directory or '/' in name or name in ('','.','..'): raise ValueError('CREDENTIAL_PATH_INVALID')
value=str(Path(directory)/name)
if '$' in value or not Path(value).is_absolute() or '..' in Path(value).parts:
raise ValueError('PATH_MUST_BE_EXPLICIT_ABSOLUTE')
return Path(value)
def _safe_read(path,*,private=False):
path=_path(str(path))
fd=os.open(path,os.O_RDONLY|os.O_NOFOLLOW|os.O_CLOEXEC)
try:
info=os.fstat(fd)
forbidden=0o077 if private else 0o022
if not stat.S_ISREG(info.st_mode) or info.st_uid not in {0,os.geteuid()} or info.st_mode & forbidden or info.st_size>65536:
raise ValueError('CONFIG_OR_SECRET_FILE_PERMISSIONS_INVALID')
with os.fdopen(fd,'rb',closefd=False) as stream:
data=stream.read(65537)
if len(data)>65536: raise ValueError('CONFIG_OR_SECRET_FILE_TOO_LARGE')
return data
finally: os.close(fd)
def secret(path):
return _safe_read(path,private=True)
def load_configuration(path):
config=read_json(path)
required={'schema','profile','state_dir','listen','machine_key_id','machine_key_file',
'owner_public_key_files','owner_key_scopes','identities','workers','contracts','operations','host_bindings'}
if type(config) is not dict or set(config)-{'receipt_public_key_files'}!=required or config['schema']!='aios.baseline-server-config.v1':
raise ValueError('SERVER_CONFIG_FIELDS_INVALID')
if config['profile']!='isolated-validation': raise ValueError('PROFILE_REQUIRES_VERIFIED_AUTHORITY_ADAPTER')
listen=config['listen']
if type(listen) is not dict or set(listen)!={'host','port'} or listen['host']!='127.0.0.1' or type(listen['port']) is not int or not 1024<=listen['port']<=65535:
raise ValueError('LOOPBACK_LISTENER_REQUIRED')
state=_path(config['state_dir'])
if state.is_symlink() or state==Path('/') or state.name in {'aios','active-runtime','app'}:
raise ValueError('ISOLATED_STATE_PATH_REQUIRED')
if not isinstance(config['machine_key_id'],str) or not config['machine_key_id']: raise ValueError('MACHINE_ID_REQUIRED')
machine=serialization.load_pem_private_key(secret(config['machine_key_file']),password=None)
if not isinstance(machine,Ed25519PrivateKey): raise ValueError('ED25519_MACHINE_KEY_REQUIRED')
keys={}
for key_id,filename in config['owner_public_key_files'].items():
public=serialization.load_pem_public_key(_safe_read(filename))
if not isinstance(public,Ed25519PublicKey): raise ValueError('ED25519_OWNER_KEY_REQUIRED')
keys[key_id]=public
if not keys: raise ValueError('OWNER_TRUST_ROOT_REQUIRED')
receipt_keys={}
for key_id,filename in config.get('receipt_public_key_files',{}).items():
public=serialization.load_pem_public_key(_safe_read(filename))
if not isinstance(public,Ed25519PublicKey): raise ValueError('ED25519_RECEIPT_KEY_REQUIRED')
receipt_keys[key_id]=public
tokens={}
for entry in config['identities']:
if type(entry) is not dict or set(entry)-{'token_file','tenant_id','subject_id','role','worker_id'}:
raise ValueError('IDENTITY_FIELDS_INVALID')
token=secret(entry['token_file']).decode().strip()
if len(token)<32 or len(token)>512 or any(x.isspace() for x in token) or token in tokens: raise ValueError('TOKEN_INVALID_OR_DUPLICATE')
identity=Identity(**{k:v for k,v in entry.items() if k!='token_file'})
if identity.role not in {'user','owner','worker'} or any(not isinstance(v,str) or not v for v in (identity.tenant_id,identity.subject_id)):
raise ValueError('IDENTITY_INVALID')
if (identity.role=='worker') != bool(identity.worker_id): raise ValueError('WORKER_IDENTITY_INVALID')
tokens[token]=identity
workers={}
for entry in config['workers']:
worker=WorkerProfile.from_dict(entry)
if worker.worker_id in workers: raise ValueError('WORKER_DUPLICATE')
workers[worker.worker_id]=worker
for identity in tokens.values():
if identity.role=='worker':
worker=workers.get(identity.worker_id)
if not worker or worker.identity!=identity.subject_id or identity.tenant_id not in worker.tenant_scope:
raise ValueError('WORKER_IDENTITY_SCOPE_MISMATCH')
contracts=[validate_contract(c) for c in config['contracts']]
operations=OperationRegistry(config['operations'])
host=SystemdHost(config['host_bindings'])
config['_resolved']={'state':state,'machine':machine,'owner_keys':keys,'workers':workers,
'contracts':contracts,'operations':operations,'host':host,'tokens':tokens,'receipt_keys':receipt_keys}
return config,tokens
def build_service(config):
resolved=config['_resolved']
state=resolved['state']
marker=state/'ISOLATED_VALIDATION_AUTHORITY'
if state.exists() and not marker.is_file() and any(state.iterdir()):
raise ValueError('EXISTING_STATE_AUTHORITY_NOT_IDENTIFIED')
if marker.exists() and _safe_read(marker)!=b'aios.baseline.isolated-validation.v1\n':
raise ValueError('STATE_AUTHORITY_MARKER_INVALID')
state.mkdir(parents=True,exist_ok=True,mode=0o700)
if state.stat().st_mode & 0o077: raise ValueError('STATE_DIRECTORY_PERMISSIONS_INVALID')
if not marker.exists(): marker.write_text('aios.baseline.isolated-validation.v1\n')
auth=AuthorizationService(state/'authorization.sqlite3',resolved['owner_keys'],resolved['machine'],config['machine_key_id'],
owner_key_scopes=config['owner_key_scopes'],
control_context_verifier=lambda ctx:isinstance(ctx,Identity) and ctx.role=='owner' and any(ctx is known for known in resolved['tokens'].values()))
store=TaskPlaneStore(str(state/'tasks.sqlite3'),high_assurance=True)
return BaselineService(store,state/'jobs',auth,resolved['contracts'],resolved['operations'],resolved['workers'],
resolved['host'],LineageLedger(state/'lineage.sqlite3',store,JobStore(state/'jobs').read),resolved['machine'],
machine_key_id=config['machine_key_id'],trusted_receipt_keys=resolved['receipt_keys'])
def _write_json(path,value,mode=0o600):
data=(json.dumps(value,ensure_ascii=False,sort_keys=True,indent=2,allow_nan=False)+'\n').encode()
fd=os.open(path,os.O_WRONLY|os.O_CREAT|os.O_EXCL,mode)
with os.fdopen(fd,'wb') as out: out.write(data)
def initialize_validation(directory,*,unit):
"""Create explicit local-only example identities, not production trust roots."""
directory=Path(directory).absolute()
directory.mkdir(mode=0o700) # Never overwrite an existing installation.
# Validate unit syntax before writing any identity material.
SystemdHost({'validation:staging:sample':unit})
owner=Ed25519PrivateKey.generate()
machine=Ed25519PrivateKey.generate()
for name,key in [('owner-private.pem',owner),('machine-private.pem',machine)]:
path=directory/name
fd=os.open(path,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
with os.fdopen(fd,'wb') as out:
out.write(key.private_bytes(serialization.Encoding.PEM,serialization.PrivateFormat.PKCS8,serialization.NoEncryption()))
(directory/'owner-public.pem').write_bytes(owner.public_key().public_bytes(serialization.Encoding.PEM,serialization.PublicFormat.SubjectPublicKeyInfo))
for name in ('user.token','worker.token'):
fd=os.open(directory/name,os.O_WRONLY|os.O_CREAT|os.O_EXCL,0o600)
with os.fdopen(fd,'w') as out: out.write(secrets.token_urlsafe(48)+'\n')
operation={'operation_id':'service-status','version':1,'effect_class':'READ_ONLY','handler_ref':'host.service_status',
'input_schema':'aios.operation.service-status.input.v1','required_permissions':['service-status'],'timeout_seconds':10,
'readback':{'mode':'NONE','required':False},'retry':{'mode':'SAFE','max_attempts':2},'reconcile':{'mode':'NONE'},
'rollback':{'mode':'NONE'},'owner_boundary':{'mode':'WITHIN_GRANT'}}
contract={'contract_schema':'aios.runtime-contract.v1','contract_revision':1,'component_id':'sample','environment':'staging',
'tenant_scope':'validation','lifecycle':'active','entrypoints':{'status':{'component_id':'sample','environment':'staging','resolver_ref':'service://sample/status'}},
'bindings':{},'resource_refs':{},'credential_refs':{},'operations':['service-status']}
config={'schema':'aios.baseline-server-config.v1','profile':'isolated-validation','state_dir':str(directory/'state'),
'listen':{'host':'127.0.0.1','port':8769},'machine_key_id':'validation-machine','machine_key_file':str(directory/'machine-private.pem'),
'owner_public_key_files':{'validation-owner':str(directory/'owner-public.pem')},
'owner_key_scopes':{'validation-owner':{'tenant_ids':['validation'],'subject_ids':['owner']}},
'identities':[{'token_file':str(directory/'user.token'),'tenant_id':'validation','subject_id':'owner','role':'owner'},
{'token_file':str(directory/'worker.token'),'tenant_id':'validation','subject_id':'machine-worker','role':'worker','worker_id':'diagnostic-worker'}],
'workers':[{'worker_id':'diagnostic-worker','identity':'machine-worker','tenant_scope':['validation'],'subject_scope':['owner'],
'capabilities':['service-status'],'priority':10,'fallback_executor':False,'heartbeat_seconds':15,'max_concurrency':1}],
'contracts':[contract],'operations':[operation],'host_bindings':{'validation:staging:sample':unit}}
_write_json(directory/'server.json',config)
_write_json(directory/'worker.json',{'schema':'aios.baseline-worker-config.v1','base_url':'http://127.0.0.1:8769',
'token_file':str(directory/'worker.token'),'poll_seconds':2})
grant=sign_owner_delegation_grant(owner,'validation-owner',grant_id='validation-grant',generation=1,
tenant_id='validation',subject_id='owner',mission_ids=('validation-mission',),task_ids=('*',),operations=('service-status',),
component_ids=('sample',),environments=('staging',),issued_at=int(time.time()),validity_mode='UNTIL_REVOKED',max_effects=20)
_write_json(directory/'grant.json',grant)
_write_json(directory/'goal.json',{'mission_id':'validation-mission','grant_id':'validation-grant','operation':'service-status',
'component_id':'sample','environment':'staging','inputs':{},'intent':'检查已绑定 staging 服务并生成诊断结果'})
return directory
def main(argv=None):
parser=argparse.ArgumentParser(description=__doc__)
sub=parser.add_subparsers(dest='command',required=True)
init=sub.add_parser('init-validation');init.add_argument('--directory',required=True);init.add_argument('--service-unit',required=True)
for name in ('preflight','server','grant-import','grant-revoke'):
item=sub.add_parser(name);item.add_argument('--config',required=True)
if name=='grant-import': item.add_argument('--grant',required=True)
if name=='grant-revoke':
item.add_argument('--grant',required=True);item.add_argument('--owner-token-file',required=True)
worker=sub.add_parser('worker');worker.add_argument('--config',required=True);worker.add_argument('--once',action='store_true')
for name in ('goal-submit','goal-explain','task-get','task-cancel'):
item=sub.add_parser(name);item.add_argument('--base-url',default='http://127.0.0.1:8769');item.add_argument('--token-file',required=True)
if name=='goal-submit': item.add_argument('--request',required=True);item.add_argument('--idempotency-key',required=True)
elif name=='goal-explain': item.add_argument('--mission-id',required=True)
else: item.add_argument('--task-id',required=True)
args=parser.parse_args(argv)
try:
if args.command=='init-validation':
print(json.dumps({'profile':'isolated-validation','directory':str(initialize_validation(args.directory,unit=args.service_unit))}));return 0
if args.command in {'preflight','server','grant-import','grant-revoke'}:
config,tokens=load_configuration(args.config)
if args.command=='preflight':
print(json.dumps({'status':'CONFIG_VALID','profile':config['profile'],'online_authority':'NOT_VERIFIED',
'production_activation':'BLOCKED','registered_operations':len(config['operations'])}));return 0
service=build_service(config)
if args.command=='grant-import':
grant=service.authorization.register_grant(read_json(args.grant))
print(json.dumps({'grant_id':grant['payload']['grant_id'],'registered':True}));return 0
if args.command=='grant-revoke':
context=tokens.get(secret(args.owner_token_file).decode().strip())
grant=read_json(args.grant)
payload=grant['payload']
if context is None or context.role!='owner' or payload['tenant_id']!=context.tenant_id or payload['subject_id']!=context.subject_id:
raise ValueError('OWNER_SCOPE_DENIED')
# Register verifies signature and ID before a target can be revoked.
service.authorization.register_grant(grant)
generation=service.authorization.revoke_grant(payload['grant_id'],control_context=context)
print(json.dumps({'grant_id':payload['grant_id'],'revocation_generation':generation}));return 0
from .transport import serve
server=serve(service,tokens,**config['listen'])
stop=threading.Event()
def recover_loop():
while not stop.wait(5):
try: service.recover()
except Exception: service.storage_blocked=True
recovery=threading.Thread(target=recover_loop,daemon=True);recovery.start()
def shutdown(_signal,_frame):
stop.set();threading.Thread(target=server.shutdown,daemon=True).start()
signal.signal(signal.SIGTERM,shutdown);signal.signal(signal.SIGINT,shutdown)
print(json.dumps({'status':'LISTENING','host':config['listen']['host'],'port':config['listen']['port'],'profile':config['profile']}),flush=True)
try: server.serve_forever(poll_interval=0.5)
finally: stop.set();server.server_close()
return 0
from .transport import HttpTransport
if args.command=='worker':
from .worker import OperationWorker
config=read_json(args.config)
if type(config) is not dict or set(config)!={'schema','base_url','token_file','poll_seconds'} or config['schema']!='aios.baseline-worker-config.v1': raise ValueError('WORKER_CONFIG_INVALID')
transport=HttpTransport(config['base_url'],secret(config['token_file']).decode().strip(),timeout_seconds=20)
worker=OperationWorker(transport,poll_seconds=config['poll_seconds'])
if args.once: print(json.dumps(worker.run_once(),ensure_ascii=False));return 0
worker.run_forever();return 0
transport=HttpTransport(args.base_url,secret(args.token_file).decode().strip(),timeout_seconds=20)
if args.command=='goal-submit': result=transport.submit_goal(read_json(args.request),idempotency_key=args.idempotency_key)
elif args.command=='goal-explain': result=transport.explain(args.mission_id)
elif args.command=='task-get': result=transport.get_task(args.task_id)
else: result=transport.cancel_task(args.task_id)
print(json.dumps(result,ensure_ascii=False,indent=2,allow_nan=False));return 0
except (Exception,KeyboardInterrupt) as exc:
# Avoid dumping request bodies, credential material or Python tracebacks.
text=str(exc)
code=text if re.fullmatch(r'[A-Z][A-Z0-9_]{2,100}',text) else getattr(exc,'code',type(exc).__name__)
print(json.dumps({'status':'ERROR','error_type':type(exc).__name__,'code':code,'message':'See local configuration and acceptance runbook; no action was reported successful.'}))
return 1
if __name__=='__main__':
raise SystemExit(main())
@@ -0,0 +1,39 @@
"""Canonical, domain-separated machine receipts, distinct from Owner grants."""
import base64
import hashlib
import json
from cryptography.exceptions import InvalidSignature
def canonical(value):
return json.dumps(value,sort_keys=True,separators=(',',':'),ensure_ascii=False,allow_nan=False).encode()
def digest(value):
return hashlib.sha256(canonical(value)).hexdigest()
class ReceiptSigner:
def __init__(self, private_key, *, key_id='baseline-machine', trusted_public_keys=None):
self.private_key=private_key
self.public_key=private_key.public_key()
self.key_id=key_id
self.trusted_public_keys=dict(trusted_public_keys or {})
self.trusted_public_keys[key_id]=self.public_key
def sign(self,payload):
raw=canonical({'domain':'aios.operation-receipt.v1','key_id':self.key_id,'payload':payload})
return {'schema':'aios.operation-receipt.v1','key_id':self.key_id,'payload':payload,
'signature':base64.b64encode(self.private_key.sign(raw)).decode()}
def verify(self,envelope):
try:
if type(envelope) is not dict or set(envelope)!={'schema','key_id','payload','signature'} or envelope['schema']!='aios.operation-receipt.v1' or type(envelope['payload']) is not dict:
return False
public=self.trusted_public_keys.get(envelope['key_id'])
if public is None: return False
raw=canonical({'domain':envelope['schema'],'key_id':envelope['key_id'],'payload':envelope['payload']})
public.verify(base64.b64decode(envelope['signature'],validate=True),raw)
return True
except (ValueError,TypeError,InvalidSignature,KeyError):
return False
+103
View File
@@ -0,0 +1,103 @@
"""Small registered host diagnostics; unit bindings come from trusted configuration."""
from __future__ import annotations
import os
from pathlib import Path
import re
import signal
import selectors
import stat
import subprocess
import time
class HostError(RuntimeError):
pass
class SystemdHost:
PROPERTIES = ('Id', 'ActiveState', 'SubState', 'MainPID', 'InvocationID')
def __init__(self, bindings: dict[str, str], *, executable: str = '/usr/bin/systemctl',
timeout_seconds: float = 10):
if type(timeout_seconds) not in (int,float) or not 0 < timeout_seconds <= 60:
raise HostError('HOST_TIMEOUT_INVALID')
self.bindings = dict(bindings)
for binding,unit in self.bindings.items():
if not isinstance(binding,str) or len(binding.split(':'))!=3 or any(not re.fullmatch(r'[A-Za-z0-9_.-]+',part) for part in binding.split(':')):
raise HostError('HOST_TENANT_BINDING_INVALID')
if not isinstance(unit, str) or not re.fullmatch(r'[A-Za-z0-9_.@:-]+\.service', unit):
raise HostError('HOST_UNIT_INVALID')
if not Path(executable).is_absolute() or not Path(executable).is_file() or Path(executable).is_symlink():
raise HostError('HOST_EXECUTABLE_UNAVAILABLE')
info=Path(executable).stat()
if info.st_uid not in {0,os.geteuid()} or info.st_mode & 0o022:
raise HostError('HOST_EXECUTABLE_UNTRUSTED')
self._executable_identity=(info.st_dev,info.st_ino,info.st_size,info.st_mtime_ns)
self.executable = executable
self.timeout_seconds = timeout_seconds
def status(self, component_id: str, environment: str, *, tenant_id: str, deadline=None) -> dict:
if any(not isinstance(part,str) or not re.fullmatch(r'[A-Za-z0-9_.-]+',part) for part in (tenant_id,environment,component_id)):
raise HostError('HOST_SCOPE_INVALID')
unit = self.bindings.get(f'{tenant_id}:{environment}:{component_id}')
if not unit:
raise HostError('HOST_BINDING_NOT_FOUND')
remaining=self.timeout_seconds if deadline is None else min(self.timeout_seconds,deadline-time.time())
if remaining<=0: raise HostError('HOST_OPERATION_TIMEOUT')
stop_at=time.monotonic()+remaining
executable_fd=os.open(self.executable,os.O_RDONLY|os.O_NOFOLLOW|os.O_CLOEXEC)
try:
info=os.fstat(executable_fd)
if not stat.S_ISREG(info.st_mode) or (info.st_dev,info.st_ino,info.st_size,info.st_mtime_ns)!=self._executable_identity:
raise HostError('HOST_EXECUTABLE_IDENTITY_CHANGED')
executable_path=f'/proc/self/fd/{executable_fd}'
if not Path(executable_path).exists(): raise HostError('HOST_EXECUTOR_PLATFORM_UNSUPPORTED')
argv=[executable_path,'show','--no-pager','--property='+','.join(self.PROPERTIES),'--',unit]
process=subprocess.Popen(argv,stdin=subprocess.DEVNULL,stdout=subprocess.PIPE,stderr=subprocess.PIPE,
env={'PATH':'/usr/bin:/bin','LANG':'C'},start_new_session=True,pass_fds=(executable_fd,))
finally: os.close(executable_fd)
output=bytearray()
byte_count=0
try:
with selectors.DefaultSelector() as selector:
selector.register(process.stdout,selectors.EVENT_READ,'stdout')
selector.register(process.stderr,selectors.EVENT_READ,'stderr')
while selector.get_map():
remaining=stop_at-time.monotonic()
if remaining<=0: raise HostError('HOST_OPERATION_TIMEOUT')
for event,_mask in selector.select(remaining):
chunk=os.read(event.fileobj.fileno(),4096)
if not chunk:
selector.unregister(event.fileobj)
continue
byte_count+=len(chunk)
if byte_count>65536: raise HostError('HOST_RESPONSE_TOO_LARGE')
if event.data=='stdout': output.extend(chunk)
process.wait(timeout=max(0.001,stop_at-time.monotonic()))
except (HostError,subprocess.TimeoutExpired) as exc:
try: os.killpg(process.pid,signal.SIGKILL)
except ProcessLookupError: pass
process.wait()
if isinstance(exc,HostError): raise
raise HostError('HOST_OPERATION_TIMEOUT') from exc
finally:
process.stdout.close()
process.stderr.close()
if process.returncode != 0:
raise HostError('HOST_STATUS_UNAVAILABLE')
try: out=output.decode('utf-8')
except UnicodeDecodeError as exc: raise HostError('HOST_OBSERVATION_INVALID') from exc
properties = {}
for line in out.splitlines():
key, sep, value = line.partition('=')
if sep and key in self.PROPERTIES:
if key in properties:
raise HostError('HOST_DUPLICATE_PROPERTY')
properties[key] = value
if set(properties) != set(self.PROPERTIES) or properties['Id'] != unit:
raise HostError('HOST_OBSERVATION_INCOMPLETE')
if not properties['MainPID'].isdigit():
raise HostError('HOST_OBSERVATION_INVALID')
return {'operation':'service-status', 'tenant_id':tenant_id, 'component_id':component_id, 'environment':environment,
'unit':unit, 'healthy':properties['ActiveState'] == 'active',
'properties':properties, 'observed_at':time.time(), 'source':'systemd.show'}
+650
View File
@@ -0,0 +1,650 @@
"""Persistent correlation relationships for the AIOS baseline.
This is deliberately a relationship authority. It does not create or advance
Tasks, Jobs, Releases, or Deployments. In particular, a Task is admitted only
after the configured Task authority returns the real Task and its authenticated
tenant/subject binding has been checked.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, is_dataclass
from datetime import datetime, timezone
import json
from pathlib import Path
import sqlite3
from typing import Any, Callable, Mapping
class LineageError(RuntimeError):
"""Base class for persistent lineage failures."""
class LineageConflict(LineageError):
"""An opaque identity was already bound to different lineage."""
class LineageVerificationError(LineageError):
"""An external authority did not verify the requested relationship."""
@dataclass(frozen=True)
class VerifiedAcceptanceEvidence:
"""Typed output of a configured acceptance evidence verifier."""
acceptance_id: str
tenant_id: str
subject_id: str
component_id: str
deployment_id: str
release_id: str
artifact: Any
accepted: bool
evidence_id: str
observed: Mapping[str, Any] | None = None
observed_at: str | None = None
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _required(value: Any, name: str) -> str:
text = str(value or "")
if not text or len(text) > 512 or any(ord(char) < 32 for char in text):
raise ValueError(f"invalid {name}")
return text
def _json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
_SELF_HASH_FIELDS = {
"manifest_sha256",
"manifest_hash",
"outer_manifest_hash",
"outer_manifest_sha256",
"self_hash",
"self_sha256",
}
def canonical_artifact(artifact: Any) -> str:
"""Return an exact artifact identity without permitting outer self-hashes."""
if not isinstance(artifact, (str, Mapping)):
raise ValueError("artifact must be a string or object")
if isinstance(artifact, str):
_required(artifact, "artifact")
else:
if not artifact:
raise ValueError("artifact must not be empty")
if any(not isinstance(key,str) for key in artifact):
raise ValueError("artifact object keys must be strings")
if _SELF_HASH_FIELDS.intersection(key.lower() for key in artifact):
raise ValueError("outer artifact manifest must not contain a self hash")
try:
return _json(artifact)
except (TypeError, ValueError) as exc:
raise ValueError("artifact must be canonical JSON") from exc
def decode_artifact(value: str) -> Any:
return json.loads(value)
def _mapping(value: Any) -> Mapping[str, Any]:
if isinstance(value, Mapping):
return value
if is_dataclass(value):
return asdict(value)
converter = getattr(value, "to_dict", None)
if not callable(converter):
converter = getattr(value, "as_dict", None)
if callable(converter):
try:
converted = converter(redact_payload=False)
except TypeError:
converted = converter()
if isinstance(converted, Mapping):
return converted
raise LineageVerificationError("task_authority returned an invalid Task")
def _task_plan(task: Any) -> tuple[Mapping[str, Any], Mapping[str, Any]]:
raw = _mapping(task)
plan: Any = raw.get("plan") or raw.get("task_plan")
if plan is None and hasattr(task, "task_plan"):
plan = getattr(task, "task_plan")
if plan is None:
plan = raw
plan_map = _mapping(plan)
return raw, plan_map
class LineageLedger:
"""SQLite-backed authority for cross-domain identity relationships."""
def __init__(self, path: str | Path, task_authority: Any, job_authority: Any):
if task_authority is None:
raise ValueError("task_authority is required")
if job_authority is None:
raise ValueError("job_authority is required")
self.path = str(path)
self.task_authority = task_authority
self.job_authority = job_authority
Path(self.path).parent.mkdir(parents=True, exist_ok=True)
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
tables={row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")}
if "lineage_tasks" in tables and "lineage_schema_meta" not in tables:
raise LineageConflict("unversioned candidate lineage schema requires explicit migration")
if "lineage_schema_meta" in tables:
version=connection.execute("SELECT version FROM lineage_schema_meta WHERE domain='lineage'").fetchone()
if version is None or version[0] != 2:
raise LineageConflict("unsupported lineage schema version")
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS lineage_schema_meta (
domain TEXT PRIMARY KEY,
version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS lineage_tasks (
tenant_id TEXT NOT NULL,
task_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
source_task_id TEXT,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, task_id),
UNIQUE (tenant_id, task_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_task_id)
REFERENCES lineage_tasks(tenant_id, task_id)
);
CREATE TABLE IF NOT EXISTS lineage_effects (
tenant_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
task_id TEXT NOT NULL,
job_id TEXT NOT NULL,
source_effect_id TEXT,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, effect_id),
UNIQUE (tenant_id, job_id),
UNIQUE (tenant_id, effect_id, subject_id, mission_id, component_id),
UNIQUE (tenant_id, effect_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, task_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_tasks(tenant_id, task_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_effect_id, subject_id, mission_id, component_id)
REFERENCES lineage_effects(tenant_id, effect_id, subject_id, mission_id, component_id)
);
CREATE TABLE IF NOT EXISTS lineage_releases (
tenant_id TEXT NOT NULL,
release_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
source_task_id TEXT NOT NULL,
source_effect_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, release_id),
UNIQUE (tenant_id, effect_id),
UNIQUE (tenant_id, release_id, subject_id, mission_id, component_id, artifact_json),
FOREIGN KEY (tenant_id, effect_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_effects(tenant_id, effect_id, subject_id, mission_id, component_id, grant_id),
FOREIGN KEY (tenant_id, source_task_id, subject_id, mission_id, component_id, grant_id)
REFERENCES lineage_tasks(tenant_id, task_id, subject_id, mission_id, component_id, grant_id)
);
CREATE TABLE IF NOT EXISTS lineage_deployments (
tenant_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
release_id TEXT NOT NULL,
environment TEXT NOT NULL,
target_revision INTEGER NOT NULL,
artifact_json TEXT NOT NULL,
metadata_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, deployment_id),
UNIQUE (tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json),
FOREIGN KEY (tenant_id, release_id, subject_id, mission_id, component_id, artifact_json)
REFERENCES lineage_releases(tenant_id, release_id, subject_id, mission_id, component_id, artifact_json)
);
CREATE TABLE IF NOT EXISTS lineage_acceptances (
tenant_id TEXT NOT NULL,
acceptance_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
accepted INTEGER NOT NULL CHECK (accepted IN (0,1)),
evidence_id TEXT NOT NULL,
evidence_json TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, acceptance_id),
FOREIGN KEY (tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json)
REFERENCES lineage_deployments(
tenant_id, deployment_id, subject_id, mission_id, component_id, release_id, artifact_json
)
);
"""
)
connection.execute("INSERT OR IGNORE INTO lineage_schema_meta(domain,version) VALUES('lineage',2)")
def close(self) -> None:
"""Compatibility hook; connections are transaction-scoped."""
def _authorized_task(self, tenant_id: str, subject_id: str, task_id: str) -> Any:
authority = self.task_authority
try:
if hasattr(authority, "get_task"):
task = authority.get_task(task_id)
elif callable(authority):
try:
task = authority(tenant_id, subject_id, task_id)
except TypeError:
task = authority(task_id)
else:
raise TypeError("authority has no task lookup")
except Exception as exc:
raise LineageVerificationError("task_authority did not confirm task") from exc
raw, plan = _task_plan(task)
returned_id = raw.get("task_id") or plan.get("task_id") or getattr(task, "task_id", None)
if str(returned_id or "") != task_id:
raise LineageVerificationError("task_authority returned wrong task_id")
issuer = plan.get("issuer") if isinstance(plan.get("issuer"), Mapping) else {}
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
tenant_values = [value for value in (issuer.get("tenant_id"), metadata.get("tenant_id"), baseline.get("tenant_id")) if value is not None]
subject_values = [
value for value in (issuer.get("subject_id"), metadata.get("subject_id"), baseline.get("subject_id")) if value is not None
]
# issuer.id is accepted as subject identity only when no explicit subject_id exists.
if not subject_values and issuer.get("id") is not None:
subject_values.append(issuer["id"])
if not tenant_values or any(str(value) != tenant_id for value in tenant_values):
raise LineageVerificationError("task_authority tenant mismatch")
if not subject_values or any(str(value) != subject_id for value in subject_values):
raise LineageVerificationError("task_authority subject mismatch")
return task
def _authorized_job(
self, tenant_id: str, subject_id: str, mission_id: str, grant_id: str,
task_id: str, effect_id: str, job_id: str,
) -> None:
authority=self.job_authority
try:
if hasattr(authority,"get_job"):
job=authority.get_job(job_id)
elif hasattr(authority,"read"):
job=authority.read(job_id)
elif callable(authority):
job=authority(job_id)
else:
raise TypeError("authority has no Job lookup")
except Exception as exc:
raise LineageVerificationError("job_authority did not confirm Job") from exc
raw=_mapping(job)
request=raw.get("request") or raw.get("binding")
if request is None and isinstance(raw.get("request_json"),str):
try:
request=json.loads(raw["request_json"])
except json.JSONDecodeError as exc:
raise LineageVerificationError("job_authority returned invalid request") from exc
if not isinstance(request,Mapping):
request=raw
if isinstance(request.get("binding"),Mapping):
request=request["binding"]
bindings={
"job_id":raw.get("job_id") or request.get("job_id"),
"tenant_id":request.get("tenant_id"),"subject_id":request.get("subject_id"),
"mission_id":request.get("mission_id"),"grant_id":request.get("grant_id"),
"task_id":request.get("task_id"),"effect_id":request.get("effect_id"),
}
expected={"job_id":job_id,"tenant_id":tenant_id,"subject_id":subject_id,
"mission_id":mission_id,"grant_id":grant_id,"task_id":task_id,"effect_id":effect_id}
if any(str(bindings[key] or "") != value for key,value in expected.items()):
raise LineageVerificationError("job_authority binding mismatch")
@staticmethod
def _check_task_context(task: Any, mission_id: str, grant_id: str) -> None:
_, plan = _task_plan(task)
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
mission_values = [value for value in (metadata.get("mission_id"), baseline.get("mission_id")) if value is not None]
grant_values = [value for value in (metadata.get("grant_id"), baseline.get("grant_id")) if value is not None]
if any(str(value) != mission_id for value in mission_values):
raise LineageVerificationError("task_authority mission mismatch")
if any(str(value) != grant_id for value in grant_values):
raise LineageVerificationError("task_authority grant mismatch")
@staticmethod
def _task_component(task: Any, requested: str | None) -> str:
_, plan = _task_plan(task)
metadata = plan.get("metadata") if isinstance(plan.get("metadata"), Mapping) else {}
baseline = metadata.get("baseline") if isinstance(metadata.get("baseline"), Mapping) else {}
values=[value for value in (metadata.get("component_id"),baseline.get("component_id")) if value is not None]
if requested is not None:
requested=_required(requested,"component_id")
if any(str(value) != requested for value in values):
raise LineageVerificationError("task_authority component mismatch")
return requested
if not values or any(str(value) != str(values[0]) for value in values):
raise LineageVerificationError("task_authority component missing or inconsistent")
return _required(values[0],"component_id")
def link_task(
self,
tenant_id: str,
subject_id: str,
mission_id: str,
grant_id: str,
task_id: str,
*,
component_id: str | None = None,
source_task_id: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, grant_id, task_id = (
_required(tenant_id, "tenant_id"), _required(subject_id, "subject_id"),
_required(mission_id, "mission_id"), _required(grant_id, "grant_id"),
_required(task_id, "task_id"),
)
task = self._authorized_task(tenant_id, subject_id, task_id)
self._check_task_context(task, mission_id, grant_id)
component_id=self._task_component(task,component_id)
source_task_id = _required(source_task_id, "source_task_id") if source_task_id else None
raw_task, task_plan = _task_plan(task)
authoritative_source = task_plan.get("parent_task_id") or raw_task.get("parent_task_id")
if authoritative_source is not None:
authoritative_source = _required(authoritative_source, "authoritative source_task_id")
if source_task_id is not None and source_task_id != authoritative_source:
raise LineageVerificationError("source_task_id does not match Task authority")
source_task_id = authoritative_source
metadata_json = _json(dict(metadata or {}))
values = (subject_id, mission_id, component_id, grant_id, source_task_id)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT * FROM lineage_tasks WHERE tenant_id=? AND task_id=?", (tenant_id, task_id)
).fetchone()
if existing:
if tuple(existing[key] for key in ("subject_id", "mission_id", "component_id", "grant_id", "source_task_id")) != values:
raise LineageConflict("task_id already has different lineage")
return self._task_row(existing)
if source_task_id:
self._require_row(connection, "lineage_tasks", "task_id", tenant_id, source_task_id, subject_id, mission_id, component_id)
connection.execute(
"INSERT INTO lineage_tasks VALUES(?,?,?,?,?,?,?,?,?)",
(tenant_id, task_id, subject_id, mission_id, component_id, grant_id, source_task_id, metadata_json, _now()),
)
row = connection.execute(
"SELECT * FROM lineage_tasks WHERE tenant_id=? AND task_id=?", (tenant_id, task_id)
).fetchone()
return self._task_row(row)
@staticmethod
def _require_row(connection, table, id_column, tenant_id, object_id, subject_id, mission_id, component_id=None):
row = connection.execute(
f"SELECT * FROM {table} WHERE tenant_id=? AND {id_column}=?", (tenant_id, object_id)
).fetchone()
if row is None:
raise LineageConflict(f"{id_column} has no linked authority record")
if row["subject_id"] != subject_id or row["mission_id"] != mission_id:
raise LineageConflict(f"{id_column} belongs to different subject or mission")
if component_id is not None and row["component_id"] != component_id:
raise LineageConflict(f"{id_column} belongs to different component")
return row
def link_effect(
self, tenant_id: str, subject_id: str, mission_id: str, task_id: str,
effect_id: str, job_id: str, *, source_effect_id: str | None = None,
metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, task_id, effect_id, job_id = map(
lambda pair: _required(pair[0], pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(task_id,"task_id"),(effect_id,"effect_id"),(job_id,"job_id")),
)
source_effect_id = _required(source_effect_id, "source_effect_id") if source_effect_id else None
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
task=self._require_row(connection, "lineage_tasks", "task_id", tenant_id, task_id, subject_id, mission_id)
component_id=task["component_id"]
grant_id=task["grant_id"]
self._authorized_job(tenant_id,subject_id,mission_id,grant_id,task_id,effect_id,job_id)
existing = connection.execute(
"SELECT * FROM lineage_effects WHERE tenant_id=? AND effect_id=?", (tenant_id, effect_id)
).fetchone()
expected = (subject_id, mission_id, component_id, grant_id, task_id, job_id, source_effect_id)
if existing:
actual = tuple(existing[key] for key in ("subject_id","mission_id","component_id","grant_id","task_id","job_id","source_effect_id"))
if actual != expected:
raise LineageConflict("effect_id already has different task/job identity")
return self._effect_row(existing)
job = connection.execute(
"SELECT effect_id,task_id,subject_id FROM lineage_effects WHERE tenant_id=? AND job_id=?",
(tenant_id, job_id),
).fetchone()
if job:
raise LineageConflict("job_id already belongs to exactly one Task/effect")
if source_effect_id:
self._require_row(connection, "lineage_effects", "effect_id", tenant_id, source_effect_id, subject_id, mission_id, component_id)
connection.execute(
"INSERT INTO lineage_effects VALUES(?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,effect_id,subject_id,mission_id,component_id,grant_id,task_id,job_id,source_effect_id,_json(dict(metadata or {})),_now()),
)
row = connection.execute("SELECT * FROM lineage_effects WHERE tenant_id=? AND effect_id=?",(tenant_id,effect_id)).fetchone()
return self._effect_row(row)
def link_release(
self, tenant_id: str, subject_id: str, mission_id: str, effect_id: str,
release_id: str, artifact: Any, *, metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id, subject_id, mission_id, effect_id, release_id = map(
lambda pair: _required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(effect_id,"effect_id"),(release_id,"release_id")),
)
artifact_json = canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
effect = self._require_row(connection,"lineage_effects","effect_id",tenant_id,effect_id,subject_id,mission_id)
existing = connection.execute("SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=?",(tenant_id,release_id)).fetchone()
expected = (subject_id,mission_id,effect["component_id"],effect["grant_id"],effect_id,effect["task_id"],effect_id,artifact_json)
if existing:
actual = tuple(existing[key] for key in ("subject_id","mission_id","component_id","grant_id","effect_id","source_task_id","source_effect_id","artifact_json"))
if actual != expected:
raise LineageConflict("release_id already has different source lineage")
return self._release_row(existing)
producer = connection.execute("SELECT release_id FROM lineage_releases WHERE tenant_id=? AND effect_id=?",(tenant_id,effect_id)).fetchone()
if producer:
raise LineageConflict("release-producing effect already points to a unique release")
connection.execute(
"INSERT INTO lineage_releases VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,release_id,subject_id,mission_id,effect["component_id"],effect["grant_id"],effect_id,effect["task_id"],effect_id,artifact_json,_json(dict(metadata or {})),_now()),
)
row=connection.execute("SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=?",(tenant_id,release_id)).fetchone()
return self._release_row(row)
def link_deployment(
self, tenant_id: str, subject_id: str, mission_id: str, release_id: str,
deployment_id: str, component_id: str, environment: str, target_revision: int, artifact: Any,
*, metadata: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
tenant_id,subject_id,mission_id,release_id,deployment_id,component_id,environment=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),
(release_id,"release_id"),(deployment_id,"deployment_id"),(component_id,"component_id"),(environment,"environment")),
)
if not isinstance(target_revision,int) or isinstance(target_revision,bool) or target_revision < 1:
raise ValueError("invalid target_revision")
artifact_json=canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
release=self._require_row(connection,"lineage_releases","release_id",tenant_id,release_id,subject_id,mission_id,component_id)
if release["artifact_json"] != artifact_json:
raise LineageConflict("deployment artifact does not match release")
existing=connection.execute("SELECT * FROM lineage_deployments WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
expected=(subject_id,mission_id,component_id,release_id,environment,target_revision,artifact_json)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","release_id","environment","target_revision","artifact_json"))
if actual != expected:
raise LineageConflict("deployment_id already has different release identity")
return self._deployment_row(existing)
connection.execute(
"INSERT INTO lineage_deployments VALUES(?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,deployment_id,subject_id,mission_id,component_id,release_id,environment,target_revision,artifact_json,_json(dict(metadata or {})),_now()),
)
row=connection.execute("SELECT * FROM lineage_deployments WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
return self._deployment_row(row)
def record_acceptance(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, acceptance_id: str,
deployment_id: str, release_id: str, artifact: Any, accepted: bool, *,
evidence: Any, verifier: Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence],
) -> dict[str, Any]:
if not callable(verifier):
raise LineageVerificationError("acceptance verifier is required")
tenant_id,subject_id,mission_id,component_id,acceptance_id,deployment_id,release_id=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),(component_id,"component_id"),
(acceptance_id,"acceptance_id"),(deployment_id,"deployment_id"),(release_id,"release_id")),
)
if type(accepted) is not bool:
raise ValueError("accepted must be boolean")
artifact_json=canonical_artifact(artifact)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
deployment=self._require_row(connection,"lineage_deployments","deployment_id",tenant_id,deployment_id,subject_id,mission_id,component_id)
if deployment["release_id"] != release_id:
raise LineageConflict("acceptance release_id does not match deployment")
if deployment["artifact_json"] != artifact_json:
raise LineageConflict("acceptance artifact does not match deployment")
expected={"acceptance_id":acceptance_id,"tenant_id":tenant_id,"subject_id":subject_id,"component_id":component_id,
"deployment_id":deployment_id,"release_id":release_id,"artifact":decode_artifact(artifact_json),
"accepted":accepted}
try:
verified=verifier(evidence,expected)
except Exception as exc:
raise LineageVerificationError("acceptance evidence was not verified") from exc
if not isinstance(verified,VerifiedAcceptanceEvidence):
raise LineageVerificationError("verifier must return VerifiedAcceptanceEvidence")
bindings=(verified.acceptance_id,verified.tenant_id,verified.subject_id,verified.component_id,verified.deployment_id,verified.release_id,canonical_artifact(verified.artifact),verified.accepted)
if bindings != (acceptance_id,tenant_id,subject_id,component_id,deployment_id,release_id,artifact_json,accepted):
raise LineageVerificationError("verified acceptance evidence binding mismatch")
evidence_id=_required(verified.evidence_id,"evidence_id")
evidence_json=_json({
"evidence_id":evidence_id,
"observed":dict(verified.observed or {}),
"observed_at":verified.observed_at,
"signed_envelope":dict(evidence) if isinstance(evidence,Mapping) else None,
})
existing=connection.execute("SELECT * FROM lineage_acceptances WHERE tenant_id=? AND acceptance_id=?",(tenant_id,acceptance_id)).fetchone()
expected_row=(subject_id,mission_id,component_id,deployment_id,release_id,artifact_json,int(accepted),evidence_id,evidence_json)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","deployment_id","release_id","artifact_json","accepted","evidence_id","evidence_json"))
if actual != expected_row:
raise LineageConflict("acceptance_id already has different binding")
return self._acceptance_row(existing)
connection.execute(
"INSERT INTO lineage_acceptances VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,acceptance_id,subject_id,mission_id,component_id,deployment_id,release_id,artifact_json,int(accepted),evidence_id,evidence_json,_now()),
)
row=connection.execute("SELECT * FROM lineage_acceptances WHERE tenant_id=? AND acceptance_id=?",(tenant_id,acceptance_id)).fetchone()
return self._acceptance_row(row)
def explain(self, tenant_id: str, subject_id: str, mission_id: str) -> dict[str, Any]:
tenant_id,subject_id,mission_id=(_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),_required(mission_id,"mission_id"))
specs=(("tasks","lineage_tasks",self._task_row),("effects","lineage_effects",self._effect_row),
("releases","lineage_releases",self._release_row),("deployments","lineage_deployments",self._deployment_row),
("acceptances","lineage_acceptances",self._acceptance_row))
result={"tenant_id":tenant_id,"subject_id":subject_id,"mission_id":mission_id}
with self._connect() as connection:
for key,table,converter in specs:
rows=connection.execute(f"SELECT * FROM {table} WHERE tenant_id=? AND subject_id=? AND mission_id=? ORDER BY created_at",(tenant_id,subject_id,mission_id)).fetchall()
result[key]=[converter(row) for row in rows]
return result
def get_release(self, tenant_id: str, subject_id: str, release_id: str) -> dict[str, Any]:
"""Resolve an exact tenant/subject release without disclosing other scopes."""
tenant_id, subject_id, release_id = (
_required(tenant_id, "tenant_id"), _required(subject_id, "subject_id"),
_required(release_id, "release_id"),
)
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=? AND subject_id=?",
(tenant_id, release_id, subject_id),
).fetchone()
if row is None:
raise KeyError("release lineage not found")
return self._release_row(row)
def get_release_for_component(self, tenant_id: str, component_id: str, release_id: str) -> dict[str, Any]:
tenant_id,component_id,release_id=(_required(tenant_id,"tenant_id"),_required(component_id,"component_id"),_required(release_id,"release_id"))
with self._connect() as connection:
row=connection.execute(
"SELECT * FROM lineage_releases WHERE tenant_id=? AND release_id=? AND component_id=?",
(tenant_id,release_id,component_id),
).fetchone()
if row is None:
raise KeyError("release component lineage not found")
return self._release_row(row)
@staticmethod
def _common(row):
return {key:row[key] for key in row.keys() if not key.endswith("_json")}
@classmethod
def _task_row(cls,row):
value=cls._common(row); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _effect_row(cls,row):
value=cls._common(row); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _release_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _deployment_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["metadata"]=json.loads(row["metadata_json"]); return value
@classmethod
def _acceptance_row(cls,row):
value=cls._common(row); value["artifact"]=decode_artifact(row["artifact_json"]); value["accepted"]=bool(row["accepted"]); value["evidence"]=json.loads(row["evidence_json"]); return value
__all__ = [
"LineageConflict", "LineageError", "LineageLedger", "LineageVerificationError",
"VerifiedAcceptanceEvidence", "canonical_artifact", "decode_artifact",
]
@@ -0,0 +1,258 @@
"""Declarative, non-executing operation registry and timeout safety rules."""
from __future__ import annotations
from copy import deepcopy
from dataclasses import dataclass
from datetime import datetime, timezone
import math
import re
from typing import Any, Iterable, Mapping, Sequence
EFFECT_CLASSES = frozenset(
{"READ_ONLY", "IDEMPOTENT", "CONDITIONAL", "RECONCILABLE", "IRREVERSIBLE"}
)
_FORBIDDEN_EXECUTION_FIELDS = frozenset({"command", "shell", "argv", "executable", "script"})
_HANDLER_REF = re.compile(r"^[A-Za-z][A-Za-z0-9_.:-]{0,126}$")
def _invalid(code: str, field: str = "") -> ValueError:
return ValueError(f"{code}:{field}" if field else code)
def _text(value: object, field: str) -> str:
if not isinstance(value, str) or not value.strip():
raise _invalid("field_invalid", field)
return value.strip()
def _policy(value: object, field: str, allowed_modes: set[str]) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise _invalid("policy_invalid", field)
result = deepcopy(dict(value))
mode = _text(result.get("mode"), f"{field}.mode").upper()
if mode not in allowed_modes:
raise _invalid("policy_mode_invalid", f"{field}.mode")
result["mode"] = mode
return result
@dataclass(frozen=True, slots=True)
class OperationDefinition:
operation_id: str
version: int | str
effect_class: str
handler_ref: str
input_schema: str | Mapping[str, Any]
required_permissions: tuple[str, ...]
timeout_seconds: float
readback: Mapping[str, Any]
retry: Mapping[str, Any]
reconcile: Mapping[str, Any]
rollback: Mapping[str, Any]
owner_boundary: Mapping[str, Any]
@classmethod
def from_dict(cls, data: Mapping[str, Any]) -> "OperationDefinition":
if not isinstance(data, Mapping):
raise _invalid("operation_definition_invalid")
forbidden = sorted(_FORBIDDEN_EXECUTION_FIELDS.intersection(data))
if forbidden:
raise _invalid("arbitrary_execution_forbidden", forbidden[0])
operation_id = _text(data.get("operation_id"), "operation_id")
version = data.get("version")
if (
isinstance(version, bool)
or not isinstance(version, (int, str))
or (isinstance(version, int) and version < 1)
or (isinstance(version, str) and not version.strip())
):
raise _invalid("field_invalid", "version")
effect_class = _text(data.get("effect_class"), "effect_class").upper()
if effect_class not in EFFECT_CLASSES:
raise _invalid("effect_class_invalid", "effect_class")
handler_ref = _text(data.get("handler_ref", data.get("handler")), "handler_ref")
if not _HANDLER_REF.fullmatch(handler_ref):
raise _invalid("handler_ref_invalid", "handler_ref")
input_schema = data.get("input_schema")
if not isinstance(input_schema, (str, Mapping)) or not input_schema:
raise _invalid("field_invalid", "input_schema")
input_schema = deepcopy(input_schema)
permissions = data.get("required_permissions", data.get("permissions"))
if not isinstance(permissions, Sequence) or isinstance(permissions, (str, bytes, bytearray)):
raise _invalid("field_invalid", "required_permissions")
required_permissions = tuple(
_text(permission, f"required_permissions[{index}]")
for index, permission in enumerate(permissions)
)
if len(set(required_permissions)) != len(required_permissions):
raise _invalid("duplicate_permission", "required_permissions")
timeout_seconds = data.get("timeout_seconds")
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(timeout_seconds)
or timeout_seconds <= 0
):
raise _invalid("field_invalid", "timeout_seconds")
readback = _policy(data.get("readback"), "readback", {"NONE", "STATE", "EXACT"})
if "required" in readback and not isinstance(readback["required"], bool):
raise _invalid("field_invalid", "readback.required")
retry = _policy(
data.get("retry"),
"retry",
{"NEVER", "SAFE", "AFTER_READBACK", "RECONCILE_FIRST"},
)
attempts = retry.get("max_attempts", 1)
if isinstance(attempts, bool) or not isinstance(attempts, int) or attempts < 1:
raise _invalid("field_invalid", "retry.max_attempts")
retry["max_attempts"] = attempts
reconcile = _policy(
data.get("reconcile"), "reconcile", {"NONE", "EXACT_READBACK", "CUSTOM", "REQUIRED"}
)
rollback = _policy(
data.get("rollback"), "rollback", {"NONE", "OPERATION", "COMPENSATE"}
)
if rollback["mode"] == "OPERATION":
_text(rollback.get("operation_id"), "rollback.operation_id")
owner_boundary = _policy(
data.get("owner_boundary"),
"owner_boundary",
{"NONE", "WITHIN_GRANT", "EXPLICIT_AUTHORIZATION", "CROSS_BOUNDARY_DENIED"},
)
restart_operation = (
"service.restart" in operation_id
or "service-restart" in operation_id
or handler_ref.endswith("service_restart")
)
if restart_operation and (
readback["mode"] != "EXACT"
or not readback.get("required", False)
or reconcile["mode"] == "NONE"
):
raise _invalid("service_restart_exact_readback_required")
return cls(
operation_id=operation_id,
version=version,
effect_class=effect_class,
handler_ref=handler_ref,
input_schema=input_schema,
required_permissions=required_permissions,
timeout_seconds=float(timeout_seconds),
readback=readback,
retry=retry,
reconcile=reconcile,
rollback=rollback,
owner_boundary=owner_boundary,
)
def timeout_action(self, outcome: str, attempts: int = 1) -> str:
return operation_timeout_action(self, outcome=outcome, attempts=attempts)
@property
def permissions(self) -> tuple[str, ...]:
return self.required_permissions
class OperationRegistry:
"""Lookup-only registry; it deliberately has no execute or shell interface."""
def __init__(self, definitions: Iterable[OperationDefinition | Mapping[str, Any]]) -> None:
self._definitions: dict[str, OperationDefinition] = {}
for raw in definitions:
definition = raw if isinstance(raw, OperationDefinition) else OperationDefinition.from_dict(raw)
if definition.operation_id in self._definitions:
raise _invalid("duplicate_operation_id", definition.operation_id)
self._definitions[definition.operation_id] = definition
def get(self, operation_id: str) -> OperationDefinition:
return self._definitions[_text(operation_id, "operation_id")]
@property
def definitions(self) -> tuple[OperationDefinition, ...]:
return tuple(self._definitions.values())
def operation_timeout_action(
definition: OperationDefinition,
outcome: str = "AMBIGUOUS",
attempts: int = 1,
) -> str:
"""Return ``RETRY``, ``RECONCILE`` or ``STOP`` after a timeout.
An unknown response from a mutating sink never authorizes a blind repeat.
Exact readback/reconciliation must first establish whether the effect happened.
"""
if not isinstance(definition, OperationDefinition):
raise _invalid("operation_definition_invalid")
normalized_outcome = _text(outcome, "outcome").upper()
if isinstance(attempts, bool) or not isinstance(attempts, int) or attempts < 1:
raise _invalid("field_invalid", "attempts")
if definition.effect_class == "IRREVERSIBLE":
return "STOP"
if definition.effect_class != "READ_ONLY" and normalized_outcome in {
"AMBIGUOUS",
"UNKNOWN",
"MAY_HAVE_SUCCEEDED",
}:
if definition.reconcile["mode"] != "NONE" or definition.readback.get("required", False):
return "RECONCILE"
return "STOP"
retry_mode = definition.retry["mode"]
if attempts >= definition.retry["max_attempts"] or retry_mode == "NEVER":
return "STOP"
if retry_mode in {"AFTER_READBACK", "RECONCILE_FIRST"} and normalized_outcome not in {
"NOT_OBSERVED",
"PROVEN_ABSENT",
}:
return "RECONCILE" if definition.reconcile["mode"] != "NONE" else "STOP"
return "RETRY"
def _timestamp(value: object) -> datetime | None:
if not isinstance(value, str):
return None
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
return None
if parsed.tzinfo is None or parsed.utcoffset() is None:
return None
return parsed.astimezone(timezone.utc)
def verify_service_restart(
before: Mapping[str, Any], after: Mapping[str, Any]
) -> dict[str, Any]:
"""Verify a restart with instance identity and start-time evidence.
Merely observing an active unit proves current activity, not that the requested
restart occurred.
"""
if not isinstance(before, Mapping) or not isinstance(after, Mapping):
return {"status": "UNKNOWN", "reasons": ["restart_observation_invalid"]}
if not isinstance(after.get("active"), bool):
return {"status": "UNKNOWN", "reasons": ["service_active_invalid"]}
if after["active"] is False:
return {"status": "DRIFT", "reasons": ["service_not_active"]}
before_id = before.get("instance_id")
after_id = after.get("instance_id")
before_started = _timestamp(before.get("started_at"))
after_started = _timestamp(after.get("started_at"))
if (
not isinstance(before_id, str)
or not before_id
or not isinstance(after_id, str)
or not after_id
or before_started is None
or after_started is None
):
return {"status": "UNKNOWN", "reasons": ["restart_identity_missing"]}
if before_id == after_id or after_started <= before_started:
return {"status": "DRIFT", "reasons": ["restart_not_observed"]}
return {"status": "VERIFIED", "reasons": []}
@@ -0,0 +1,866 @@
"""Bounded, task-only optimization proposals for Baseline 1.1.
This module deliberately has no configuration or policy write adapter. Its only
effect boundary is ``submit_task``, supplied by the authenticated root service.
"""
from __future__ import annotations
import hashlib
import json
import math
from dataclasses import dataclass
from pathlib import Path
import re
import sqlite3
import time
from typing import Any, Callable, Mapping
ENVELOPE_SCHEMA = "aios.optimization-envelope.v1"
PROPOSAL_SCHEMA = "aios.optimization-proposal.v1"
TASK_REQUEST_SCHEMA = "aios.optimization-task-request.v1"
_IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}")
_DIGEST = re.compile(r"sha256:[0-9a-f]{64}")
# An envelope may narrow these limits but cannot introduce another path or
# expand a range. Every entry describes a low-risk operational tuning knob.
_HARD_PARAMETER_LIMITS: dict[str, tuple[float, float, bool]] = {
"retry.max_attempts": (1, 10, True),
"retry.backoff_seconds": (0, 60, False),
"backoff.initial_seconds": (0, 60, False),
"backoff.maximum_seconds": (0.1, 600, False),
"routing.candidate_weight_percent": (0, 25, False),
"routing.weight_percent": (0, 25, False),
"evidence_reuse.max_age_seconds": (0, 3600, True),
"evidence_reuse.ttl_seconds": (0, 3600, True),
"timeout.operation_seconds": (0.1, 300, False),
"timeout.seconds": (0.1, 300, False),
"poll.interval_seconds": (0.1, 300, False),
"notification_dedup.window_seconds": (0, 3600, True),
"test_selection.optional_sample_percent": (1, 100, False),
"test_selection.optional_percentage": (1, 100, False),
}
_NON_PRODUCTION_ENVIRONMENTS = frozenset({"development", "test", "staging", "canary"})
class OptimizationError(RuntimeError):
"""A fail-closed optimization boundary error with a stable code."""
def __init__(self, code: str):
self.code = code
super().__init__(code)
def canonical_json_bytes(value: Any) -> bytes:
try:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise OptimizationError("OPTIMIZATION_DOCUMENT_NOT_CANONICAL") from exc
def _digest(value: Any) -> str:
return "sha256:" + hashlib.sha256(canonical_json_bytes(value)).hexdigest()
def _number(value: Any, code: str, *, minimum: float | None = None,
maximum: float | None = None) -> int | float:
if isinstance(value, bool) or not isinstance(value, (int, float)):
raise OptimizationError(code)
if not math.isfinite(float(value)):
raise OptimizationError(code)
if minimum is not None and value < minimum:
raise OptimizationError(code)
if maximum is not None and value > maximum:
raise OptimizationError(code)
return value
def _identifier(value: Any, code: str) -> str:
if not isinstance(value, str) or _IDENTIFIER.fullmatch(value) is None:
raise OptimizationError(code)
return value
def _unique_identifiers(value: Any, code: str, *, maximum: int = 256) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)) or len(value) > maximum:
raise OptimizationError(code)
items = tuple(_identifier(item, code) for item in value)
if len(set(items)) != len(items):
raise OptimizationError(code)
return items
@dataclass(frozen=True)
class ParameterRange:
minimum: int | float
maximum: int | float
integer: bool
@dataclass(frozen=True)
class QualityFloor:
direction: str
threshold: int | float
@dataclass(frozen=True)
class PromotionRules:
objective_metric: str
minimum_improvement: int | float
mandatory_tests: tuple[str, ...]
mandatory_quality_gates: tuple[str, ...]
required_receipt_schema: str
@dataclass(frozen=True)
class OptimizationEnvelope:
schema: str
envelope_id: str
tenant_id: str
allowed_parameters: Mapping[str, ParameterRange]
allowed_components: tuple[str, ...]
allowed_environments: tuple[str, ...]
max_change_size: int
max_compute_budget: int | float
quality_floor: Mapping[str, QualityFloor]
rollback_requirement: bool
promotion_rules: PromotionRules
source_config_digest: str
key_id: str
signature: str
@classmethod
def from_dict(
cls,
data: Mapping[str, Any],
*,
verify_envelope: Callable[[str, bytes, str], bool] | None,
) -> "OptimizationEnvelope":
"""Parse a trusted-config envelope after authenticating its exact bytes.
``verify_envelope`` is intentionally mandatory. The caller cannot mark
a document trusted with a boolean or construct an unrestricted envelope.
"""
if verify_envelope is None or not callable(verify_envelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_VERIFIER_REQUIRED")
if not isinstance(data, Mapping):
raise OptimizationError("OPTIMIZATION_ENVELOPE_INVALID")
required = {
"schema", "envelope_id", "tenant_id", "allowed_parameters",
"allowed_components", "allowed_environments", "max_change_size",
"max_compute_budget", "quality_floor", "rollback_requirement",
"promotion_rules", "source_config_digest", "key_id", "signature",
}
if set(data) != required:
raise OptimizationError("OPTIMIZATION_ENVELOPE_FIELDS_INVALID")
unsigned = dict(data)
signature = unsigned.pop("signature")
key_id = data.get("key_id")
if not isinstance(signature, str) or not signature or not isinstance(key_id, str):
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID")
try:
verified = verify_envelope(key_id, canonical_json_bytes(unsigned), signature)
except Exception as exc:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID") from exc
if verified is not True:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SIGNATURE_INVALID")
if data["schema"] != ENVELOPE_SCHEMA:
raise OptimizationError("OPTIMIZATION_ENVELOPE_SCHEMA_INVALID")
envelope_id = _identifier(data["envelope_id"], "OPTIMIZATION_ENVELOPE_ID_INVALID")
tenant_id = _identifier(data["tenant_id"], "OPTIMIZATION_TENANT_INVALID")
if not _DIGEST.fullmatch(str(data["source_config_digest"])):
raise OptimizationError("OPTIMIZATION_SOURCE_CONFIG_DIGEST_INVALID")
key_id = _identifier(key_id, "OPTIMIZATION_ENVELOPE_KEY_ID_INVALID")
raw_parameters = data["allowed_parameters"]
if not isinstance(raw_parameters, Mapping) or not raw_parameters or len(raw_parameters) > 16:
raise OptimizationError("OPTIMIZATION_ALLOWED_PARAMETERS_INVALID")
parameters: dict[str, ParameterRange] = {}
for path, raw_rule in raw_parameters.items():
if path not in _HARD_PARAMETER_LIMITS:
raise OptimizationError("OPTIMIZATION_PARAMETER_PATH_FORBIDDEN")
if not isinstance(raw_rule, Mapping) or not {"minimum", "maximum"} <= set(raw_rule):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
if set(raw_rule) - {"minimum", "maximum", "integer"}:
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
hard_min, hard_max, hard_integer = _HARD_PARAMETER_LIMITS[path]
minimum = _number(raw_rule["minimum"], "OPTIMIZATION_PARAMETER_RANGE_INVALID")
maximum = _number(raw_rule["maximum"], "OPTIMIZATION_PARAMETER_RANGE_INVALID")
integer = raw_rule.get("integer", hard_integer)
if type(integer) is not bool or minimum > maximum:
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
if minimum < hard_min or maximum > hard_max or (hard_integer and not integer):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_TOO_BROAD")
if integer and (type(minimum) is not int or type(maximum) is not int):
raise OptimizationError("OPTIMIZATION_PARAMETER_RANGE_INVALID")
parameters[path] = ParameterRange(minimum, maximum, integer)
components = _unique_identifiers(
data["allowed_components"], "OPTIMIZATION_COMPONENTS_INVALID", maximum=32
)
environments = _unique_identifiers(
data["allowed_environments"], "OPTIMIZATION_ENVIRONMENTS_INVALID", maximum=4
)
if not components or not environments or not set(environments) <= _NON_PRODUCTION_ENVIRONMENTS:
raise OptimizationError("OPTIMIZATION_ENVIRONMENTS_INVALID")
max_change_size = _number(
data["max_change_size"], "OPTIMIZATION_CHANGE_SIZE_INVALID", minimum=1, maximum=16
)
if type(max_change_size) is not int:
raise OptimizationError("OPTIMIZATION_CHANGE_SIZE_INVALID")
max_compute_budget = _number(
data["max_compute_budget"],
"OPTIMIZATION_COMPUTE_BUDGET_INVALID",
minimum=0.001,
maximum=10_000,
)
raw_quality = data["quality_floor"]
if not isinstance(raw_quality, Mapping) or not raw_quality or len(raw_quality) > 16:
raise OptimizationError("OPTIMIZATION_QUALITY_FLOOR_INVALID")
quality: dict[str, QualityFloor] = {}
for metric, raw_rule in raw_quality.items():
metric = _identifier(metric, "OPTIMIZATION_QUALITY_METRIC_INVALID")
if not isinstance(raw_rule, Mapping) or set(raw_rule) != {"direction", "threshold"}:
raise OptimizationError("OPTIMIZATION_QUALITY_FLOOR_INVALID")
if raw_rule["direction"] not in {"higher", "lower"}:
raise OptimizationError("OPTIMIZATION_QUALITY_DIRECTION_INVALID")
threshold = _number(
raw_rule["threshold"], "OPTIMIZATION_QUALITY_FLOOR_INVALID",
minimum=-1e12, maximum=1e12,
)
quality[metric] = QualityFloor(raw_rule["direction"], threshold)
if type(data["rollback_requirement"]) is not bool:
raise OptimizationError("OPTIMIZATION_ROLLBACK_REQUIREMENT_INVALID")
raw_promotion = data["promotion_rules"]
promotion_fields = {
"objective_metric", "minimum_improvement", "mandatory_tests",
"mandatory_quality_gates", "required_receipt_schema",
}
if not isinstance(raw_promotion, Mapping) or set(raw_promotion) != promotion_fields:
raise OptimizationError("OPTIMIZATION_PROMOTION_RULES_INVALID")
objective = _identifier(
raw_promotion["objective_metric"], "OPTIMIZATION_OBJECTIVE_METRIC_INVALID"
)
if objective not in quality:
raise OptimizationError("OPTIMIZATION_OBJECTIVE_METRIC_INVALID")
minimum_improvement = _number(
raw_promotion["minimum_improvement"],
"OPTIMIZATION_MINIMUM_IMPROVEMENT_INVALID",
minimum=0,
maximum=1e12,
)
mandatory_tests = _unique_identifiers(
raw_promotion["mandatory_tests"], "OPTIMIZATION_MANDATORY_TESTS_INVALID"
)
mandatory_gates = _unique_identifiers(
raw_promotion["mandatory_quality_gates"], "OPTIMIZATION_MANDATORY_GATES_INVALID"
)
if not mandatory_tests or not mandatory_gates:
raise OptimizationError("OPTIMIZATION_PROMOTION_RULES_INVALID")
receipt_schema = _identifier(
raw_promotion["required_receipt_schema"], "OPTIMIZATION_RECEIPT_SCHEMA_INVALID"
)
if not receipt_schema.startswith("aios."):
raise OptimizationError("OPTIMIZATION_RECEIPT_SCHEMA_INVALID")
promotion = PromotionRules(
objective, minimum_improvement, mandatory_tests, mandatory_gates, receipt_schema
)
return cls(
ENVELOPE_SCHEMA, envelope_id, tenant_id, parameters, components,
environments, max_change_size, max_compute_budget, quality,
data["rollback_requirement"], promotion, data["source_config_digest"],
key_id, signature,
)
_PROPOSAL_REQUIRED = {
"schema", "proposal_id", "envelope_id", "tenant_id", "component",
"environment", "changes", "compute_budget", "tests", "quality_gates",
"baseline_metrics", "rollback_plan",
}
def validate_proposal(envelope: OptimizationEnvelope, proposal: Mapping[str, Any]) -> dict:
"""Return a canonical candidate constrained by ``envelope``."""
if not isinstance(envelope, OptimizationEnvelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_REQUIRED")
if not isinstance(proposal, Mapping):
raise OptimizationError("OPTIMIZATION_PROPOSAL_INVALID")
allowed_fields = _PROPOSAL_REQUIRED | {"evidence_reuse"}
if set(proposal) - allowed_fields or not _PROPOSAL_REQUIRED <= set(proposal):
raise OptimizationError("OPTIMIZATION_PROPOSAL_FIELDS_INVALID")
if proposal["schema"] != PROPOSAL_SCHEMA:
raise OptimizationError("OPTIMIZATION_PROPOSAL_SCHEMA_INVALID")
proposal_id = _identifier(proposal["proposal_id"], "OPTIMIZATION_PROPOSAL_ID_INVALID")
if proposal["envelope_id"] != envelope.envelope_id:
raise OptimizationError("OPTIMIZATION_ENVELOPE_BINDING_MISMATCH")
if proposal["tenant_id"] != envelope.tenant_id:
raise OptimizationError("OPTIMIZATION_PROPOSAL_TENANT_MISMATCH")
component = _identifier(proposal["component"], "OPTIMIZATION_COMPONENT_INVALID")
if component not in envelope.allowed_components:
raise OptimizationError("OPTIMIZATION_COMPONENT_FORBIDDEN")
environment = _identifier(proposal["environment"], "OPTIMIZATION_ENVIRONMENT_INVALID")
if environment not in envelope.allowed_environments:
raise OptimizationError("OPTIMIZATION_ENVIRONMENT_FORBIDDEN")
raw_changes = proposal["changes"]
if not isinstance(raw_changes, list) or not raw_changes or len(raw_changes) > envelope.max_change_size:
raise OptimizationError("OPTIMIZATION_CHANGE_SIZE_EXCEEDED")
changes: list[dict[str, Any]] = []
paths: set[str] = set()
for change in raw_changes:
if not isinstance(change, Mapping) or set(change) != {"path", "value"}:
raise OptimizationError("OPTIMIZATION_CHANGE_INVALID")
path = change["path"]
if not isinstance(path, str) or path not in envelope.allowed_parameters:
raise OptimizationError("OPTIMIZATION_PARAMETER_PATH_FORBIDDEN")
if path in paths:
raise OptimizationError("OPTIMIZATION_PARAMETER_DUPLICATE")
paths.add(path)
rule = envelope.allowed_parameters[path]
value = _number(change["value"], "OPTIMIZATION_PARAMETER_VALUE_INVALID")
if rule.integer and type(value) is not int:
raise OptimizationError("OPTIMIZATION_PARAMETER_VALUE_INVALID")
if value < rule.minimum or value > rule.maximum:
raise OptimizationError("OPTIMIZATION_PARAMETER_OUT_OF_RANGE")
changes.append({"path": path, "value": value})
compute_budget = _number(
proposal["compute_budget"],
"OPTIMIZATION_COMPUTE_BUDGET_EXCEEDED",
minimum=0.001,
maximum=float(envelope.max_compute_budget),
)
tests = _unique_identifiers(proposal["tests"], "OPTIMIZATION_TESTS_INVALID")
gates = _unique_identifiers(proposal["quality_gates"], "OPTIMIZATION_QUALITY_GATES_INVALID")
for mandatory in envelope.promotion_rules.mandatory_tests:
if mandatory not in tests:
raise OptimizationError("OPTIMIZATION_MANDATORY_TEST_MISSING")
for mandatory in envelope.promotion_rules.mandatory_quality_gates:
if mandatory not in gates:
raise OptimizationError("OPTIMIZATION_MANDATORY_GATE_MISSING")
raw_baseline = proposal["baseline_metrics"]
if not isinstance(raw_baseline, Mapping) or set(raw_baseline) != set(envelope.quality_floor):
raise OptimizationError("OPTIMIZATION_BASELINE_METRICS_INVALID")
baseline = {
metric: _number(raw_baseline[metric], "OPTIMIZATION_BASELINE_METRICS_INVALID")
for metric in sorted(raw_baseline)
}
rollback_plan = proposal["rollback_plan"]
if envelope.rollback_requirement:
if (
not isinstance(rollback_plan, Mapping)
or set(rollback_plan) != {"strategy", "reference"}
or rollback_plan.get("strategy") != "restore_previous_values"
):
raise OptimizationError("OPTIMIZATION_ROLLBACK_PLAN_REQUIRED")
_identifier(rollback_plan.get("reference"), "OPTIMIZATION_ROLLBACK_PLAN_INVALID")
normalized_rollback: dict[str, str] | None = {
"strategy": "restore_previous_values",
"reference": rollback_plan["reference"],
}
else:
if rollback_plan is not None:
raise OptimizationError("OPTIMIZATION_ROLLBACK_PLAN_INVALID")
normalized_rollback = None
normalized_evidence = None
if "evidence_reuse" in proposal:
evidence = proposal["evidence_reuse"]
if not isinstance(evidence, Mapping) or set(evidence) != {"scope", "evidence_ids"}:
raise OptimizationError("OPTIMIZATION_EVIDENCE_REUSE_INVALID")
expected_scope = {
"tenant_id": envelope.tenant_id,
"component": component,
"environment": environment,
}
if evidence["scope"] != expected_scope:
raise OptimizationError("OPTIMIZATION_EVIDENCE_SCOPE_MISMATCH")
evidence_ids = _unique_identifiers(
evidence["evidence_ids"], "OPTIMIZATION_EVIDENCE_REUSE_INVALID", maximum=128
)
normalized_evidence = {"scope": expected_scope, "evidence_ids": list(evidence_ids)}
normalized = {
"schema": PROPOSAL_SCHEMA,
"proposal_id": proposal_id,
"envelope_id": envelope.envelope_id,
"tenant_id": envelope.tenant_id,
"component": component,
"environment": environment,
"changes": changes,
"compute_budget": float(compute_budget),
"tests": list(tests),
"quality_gates": list(gates),
"baseline_metrics": baseline,
"rollback_plan": normalized_rollback,
}
if normalized_evidence is not None:
normalized["evidence_reuse"] = normalized_evidence
if len(canonical_json_bytes(normalized)) > 65_536:
raise OptimizationError("OPTIMIZATION_PROPOSAL_TOO_LARGE")
return normalized
class OptimizationController:
"""Durable proposal state machine whose only external effect is task admission."""
def __init__(
self,
path: str | Path,
envelope: OptimizationEnvelope,
submit_task: Callable[[Mapping[str, Any]], Mapping[str, Any]],
*,
verify_receipt: Callable[[Mapping[str, Any]], bool],
):
if not isinstance(envelope, OptimizationEnvelope):
raise OptimizationError("OPTIMIZATION_ENVELOPE_REQUIRED")
if not callable(submit_task):
raise OptimizationError("OPTIMIZATION_TASK_SUBMITTER_REQUIRED")
if not callable(verify_receipt):
raise OptimizationError("OPTIMIZATION_RECEIPT_VERIFIER_REQUIRED")
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
self.envelope = envelope
self.submit_task = submit_task
self.verify_receipt = verify_receipt
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
connection.execute(
"""
CREATE TABLE IF NOT EXISTS optimization_proposals (
proposal_id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
envelope_id TEXT NOT NULL,
proposal_digest TEXT NOT NULL,
proposal_json TEXT NOT NULL,
status TEXT NOT NULL,
evaluation_task_id TEXT,
promotion_task_id TEXT,
baseline_json TEXT NOT NULL,
candidate_json TEXT,
effect_json TEXT,
evaluation_json TEXT,
receipt_json TEXT,
error_code TEXT,
created_at REAL NOT NULL,
updated_at REAL NOT NULL
)
"""
)
def _context(self, value: Mapping[str, Any]) -> dict[str, str]:
if not isinstance(value, Mapping):
raise OptimizationError("OPTIMIZATION_AUTHENTICATED_CONTEXT_REQUIRED")
fields = ("tenant_id", "subject_id", "grant_id", "mission_id", "service_identity")
try:
context = {
field: _identifier(value[field], "OPTIMIZATION_AUTHENTICATED_CONTEXT_INVALID")
for field in fields
}
except KeyError as exc:
raise OptimizationError("OPTIMIZATION_AUTHENTICATED_CONTEXT_REQUIRED") from exc
if context["service_identity"] != "aios-root-service":
raise OptimizationError("OPTIMIZATION_ROOT_SERVICE_REQUIRED")
if context["tenant_id"] != self.envelope.tenant_id:
raise OptimizationError("OPTIMIZATION_CONTEXT_TENANT_MISMATCH")
return context
def _row(self, proposal_id: str) -> sqlite3.Row:
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM optimization_proposals WHERE proposal_id=?", (proposal_id,)
).fetchone()
if row is None:
raise OptimizationError("OPTIMIZATION_PROPOSAL_NOT_FOUND")
return row
def _authorized_row(self, proposal_id: str, context: Mapping[str, Any]) -> sqlite3.Row:
checked = self._context(context)
row = self._row(_identifier(proposal_id, "OPTIMIZATION_PROPOSAL_ID_INVALID"))
if row["tenant_id"] != checked["tenant_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_TENANT_MISMATCH")
if row["subject_id"] != checked["subject_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_SUBJECT_MISMATCH")
if row["grant_id"] != checked["grant_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_GRANT_MISMATCH")
if row["mission_id"] != checked["mission_id"]:
raise OptimizationError("OPTIMIZATION_CONTEXT_MISSION_MISMATCH")
return row
@staticmethod
def _decode(value: str | None) -> Any:
return None if value is None else json.loads(value)
def _record(self, row: sqlite3.Row) -> dict:
proposal = self._decode(row["proposal_json"])
result = dict(proposal)
result.update(
{
"proposal_id": row["proposal_id"],
"proposal_digest": row["proposal_digest"],
"status": row["status"],
"evaluation_task_id": row["evaluation_task_id"],
"promotion_task_id": row["promotion_task_id"],
"baseline_metrics": self._decode(row["baseline_json"]),
"candidate_metrics": self._decode(row["candidate_json"]),
"effect_comparison": self._decode(row["effect_json"]),
"error_code": row["error_code"],
}
)
return result
def get_proposal(self, proposal_id: str, *, authenticated_context: Mapping[str, Any]) -> dict:
return self._record(self._authorized_row(proposal_id, authenticated_context))
def _persist_invalid(self, proposal: Mapping[str, Any], context: Mapping[str, str],
error: OptimizationError) -> None:
proposal_id = proposal.get("proposal_id")
tenant_id = proposal.get("tenant_id")
if (
not isinstance(proposal_id, str)
or _IDENTIFIER.fullmatch(proposal_id) is None
or tenant_id != self.envelope.tenant_id
):
return
try:
encoded = canonical_json_bytes(proposal)
except OptimizationError:
return
if len(encoded) > 65_536:
return
digest = "sha256:" + hashlib.sha256(encoded).hexdigest()
baseline = proposal.get("baseline_metrics")
if not isinstance(baseline, Mapping):
baseline = {}
now = time.time()
with self._connect() as connection:
connection.execute(
"""
INSERT OR IGNORE INTO optimization_proposals
(proposal_id,tenant_id,subject_id,grant_id,mission_id,envelope_id,
proposal_digest,proposal_json,status,baseline_json,error_code,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
proposal_id, self.envelope.tenant_id, context["subject_id"],
context["grant_id"], context["mission_id"], self.envelope.envelope_id,
digest, encoded.decode("utf-8"), "STOPPED_INVALID",
canonical_json_bytes(baseline).decode("utf-8"), error.code, now, now,
),
)
def propose(self, proposal: Mapping[str, Any], *, authenticated_context: Mapping[str, Any]) -> dict:
context = self._context(authenticated_context)
try:
normalized = validate_proposal(self.envelope, proposal)
except OptimizationError as exc:
if isinstance(proposal, Mapping):
self._persist_invalid(proposal, context, exc)
raise
proposal_digest = _digest(normalized)
proposal_id = normalized["proposal_id"]
now = time.time()
encoded = canonical_json_bytes(normalized).decode("utf-8")
with self._connect() as connection:
existing = connection.execute(
"SELECT * FROM optimization_proposals WHERE proposal_id=?", (proposal_id,)
).fetchone()
if existing is not None:
if existing["proposal_digest"] != proposal_digest:
raise OptimizationError("OPTIMIZATION_PROPOSAL_ID_CONFLICT")
if (
existing["tenant_id"] != context["tenant_id"]
or existing["subject_id"] != context["subject_id"]
or existing["grant_id"] != context["grant_id"]
or existing["mission_id"] != context["mission_id"]
):
raise OptimizationError("OPTIMIZATION_CONTEXT_BINDING_MISMATCH")
if existing["status"] != "PROPOSAL_RECORDED":
return self._record(existing)
else:
connection.execute(
"""
INSERT INTO optimization_proposals
(proposal_id,tenant_id,subject_id,grant_id,mission_id,envelope_id,
proposal_digest,proposal_json,status,baseline_json,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
proposal_id, context["tenant_id"], context["subject_id"],
context["grant_id"], context["mission_id"], self.envelope.envelope_id,
proposal_digest, encoded, "PROPOSAL_RECORDED",
canonical_json_bytes(normalized["baseline_metrics"]).decode("utf-8"),
now, now,
),
)
requested_task_id = f"optimization-evaluate-{proposal_id}"
task_request = {
"schema": TASK_REQUEST_SCHEMA,
"task_id": requested_task_id,
"action": "evaluate",
"tenant_id": context["tenant_id"],
"subject_id": context["subject_id"],
"grant_id": context["grant_id"],
"mission_id": context["mission_id"],
"proposal_id": proposal_id,
"proposal_digest": proposal_digest,
"component": normalized["component"],
"environment": normalized["environment"],
"compute_budget": normalized["compute_budget"],
}
try:
receipt = self.submit_task(task_request)
if not isinstance(receipt, Mapping):
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_INVALID")
task_id = _identifier(
receipt.get("task_id"), "OPTIMIZATION_TASK_SUBMISSION_INVALID"
)
except Exception as exc:
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,error_code=?,updated_at=? "
"WHERE proposal_id=? AND status='PROPOSAL_RECORDED'",
("STOPPED_SUBMISSION_FAILED", "OPTIMIZATION_TASK_SUBMISSION_FAILED", time.time(), proposal_id),
)
if isinstance(exc, OptimizationError):
raise
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_FAILED") from exc
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,evaluation_task_id=?,updated_at=? "
"WHERE proposal_id=? AND status='PROPOSAL_RECORDED'",
("EVALUATION_SUBMITTED", task_id, time.time(), proposal_id),
)
return self.get_proposal(proposal_id, authenticated_context=context)
def _trusted_receipt(self, receipt: Any) -> Mapping[str, Any]:
if not isinstance(receipt, Mapping):
raise OptimizationError("OPTIMIZATION_RECEIPT_INVALID")
try:
trusted = self.verify_receipt(receipt)
except Exception as exc:
raise OptimizationError("OPTIMIZATION_RECEIPT_UNTRUSTED") from exc
if trusted is not True:
raise OptimizationError("OPTIMIZATION_RECEIPT_UNTRUSTED")
return receipt
def record_evaluation(
self,
proposal_id: str,
evaluation: Mapping[str, Any],
*,
authenticated_context: Mapping[str, Any],
) -> dict:
row = self._authorized_row(proposal_id, authenticated_context)
if row["status"] != "EVALUATION_SUBMITTED":
raise OptimizationError("OPTIMIZATION_EVALUATION_STATE_INVALID")
if not isinstance(evaluation, Mapping) or set(evaluation) != {
"candidate_metrics", "compute_used", "tests_passed",
"quality_gates_passed", "receipt",
}:
raise OptimizationError("OPTIMIZATION_EVALUATION_INVALID")
result_body = {key: evaluation[key] for key in evaluation if key != "receipt"}
receipt = self._trusted_receipt(evaluation["receipt"])
required_receipt = {
"schema", "task_id", "proposal_id", "proposal_digest", "action",
"success", "result_digest", "key_id", "signature",
}
if set(receipt) != required_receipt:
raise OptimizationError("OPTIMIZATION_RECEIPT_INVALID")
if receipt["schema"] != self.envelope.promotion_rules.required_receipt_schema:
raise OptimizationError("OPTIMIZATION_RECEIPT_SCHEMA_MISMATCH")
if receipt["task_id"] != row["evaluation_task_id"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_TASK_MISMATCH")
if receipt["proposal_id"] != row["proposal_id"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_PROPOSAL_MISMATCH")
if receipt["proposal_digest"] != row["proposal_digest"]:
raise OptimizationError("OPTIMIZATION_RECEIPT_PROPOSAL_DIGEST_MISMATCH")
if receipt["action"] != "evaluate" or receipt["success"] is not True:
raise OptimizationError("OPTIMIZATION_RECEIPT_RESULT_INVALID")
if receipt["result_digest"] != _digest(result_body):
raise OptimizationError("OPTIMIZATION_RECEIPT_RESULT_DIGEST_MISMATCH")
proposal = self._decode(row["proposal_json"])
compute_used = _number(
evaluation["compute_used"], "OPTIMIZATION_EVALUATION_BUDGET_EXCEEDED", minimum=0
)
if compute_used > proposal["compute_budget"] or compute_used > self.envelope.max_compute_budget:
status = "STOPPED_BUDGET_EXCEEDED"
error_code = "OPTIMIZATION_EVALUATION_BUDGET_EXCEEDED"
return self._store_evaluation(row, evaluation, receipt, {}, status, error_code)
tests_passed = _unique_identifiers(
evaluation["tests_passed"], "OPTIMIZATION_EVALUATION_TESTS_INVALID"
)
gates_passed = _unique_identifiers(
evaluation["quality_gates_passed"], "OPTIMIZATION_EVALUATION_GATES_INVALID"
)
if not set(self.envelope.promotion_rules.mandatory_tests) <= set(tests_passed):
return self._store_evaluation(
row, evaluation, receipt, {}, "STOPPED_MANDATORY_TEST_FAILED",
"OPTIMIZATION_MANDATORY_TEST_FAILED",
)
if not set(self.envelope.promotion_rules.mandatory_quality_gates) <= set(gates_passed):
return self._store_evaluation(
row, evaluation, receipt, {}, "STOPPED_MANDATORY_GATE_FAILED",
"OPTIMIZATION_MANDATORY_GATE_FAILED",
)
raw_candidate = evaluation["candidate_metrics"]
if not isinstance(raw_candidate, Mapping) or set(raw_candidate) != set(self.envelope.quality_floor):
raise OptimizationError("OPTIMIZATION_CANDIDATE_METRICS_INVALID")
candidate = {
metric: _number(raw_candidate[metric], "OPTIMIZATION_CANDIDATE_METRICS_INVALID")
for metric in sorted(raw_candidate)
}
baseline = self._decode(row["baseline_json"])
effect: dict[str, float] = {}
quality_regression = False
for metric in sorted(self.envelope.quality_floor):
rule = self.envelope.quality_floor[metric]
if rule.direction == "higher":
delta = float(candidate[metric]) - float(baseline[metric])
below_floor = candidate[metric] < rule.threshold
else:
delta = float(baseline[metric]) - float(candidate[metric])
below_floor = candidate[metric] > rule.threshold
effect[metric] = round(delta, 12)
quality_regression = quality_regression or below_floor or delta < -1e-12
objective = self.envelope.promotion_rules.objective_metric
if quality_regression:
status, error_code = "STOPPED_QUALITY_REGRESSION", "OPTIMIZATION_QUALITY_REGRESSION"
elif effect[objective] + 1e-12 < self.envelope.promotion_rules.minimum_improvement:
status, error_code = "STOPPED_NO_IMPROVEMENT", "OPTIMIZATION_NO_IMPROVEMENT"
else:
status, error_code = "ELIGIBLE", None
return self._store_evaluation(row, evaluation, receipt, effect, status, error_code, candidate)
def _store_evaluation(
self,
row: sqlite3.Row,
evaluation: Mapping[str, Any],
receipt: Mapping[str, Any],
effect: Mapping[str, Any],
status: str,
error_code: str | None,
candidate: Mapping[str, Any] | None = None,
) -> dict:
candidate = dict(candidate or {})
with self._connect() as connection:
connection.execute(
"""
UPDATE optimization_proposals
SET status=?,candidate_json=?,effect_json=?,evaluation_json=?,receipt_json=?,
error_code=?,updated_at=?
WHERE proposal_id=?
""",
(
status, canonical_json_bytes(candidate).decode("utf-8"),
canonical_json_bytes(effect).decode("utf-8"),
canonical_json_bytes({k: evaluation[k] for k in evaluation if k != "receipt"}).decode("utf-8"),
canonical_json_bytes(receipt).decode("utf-8"), error_code, time.time(),
row["proposal_id"],
),
)
return self._record(self._row(row["proposal_id"]))
def promote(
self,
proposal_id: str,
promotion: Mapping[str, Any],
*,
authenticated_context: Mapping[str, Any],
) -> dict:
context = self._context(authenticated_context)
row = self._authorized_row(proposal_id, context)
if not isinstance(promotion, Mapping) or set(promotion) != {
"evaluation_task_id", "proposal_digest"
}:
raise OptimizationError("OPTIMIZATION_PROMOTION_REQUEST_INVALID")
if promotion["evaluation_task_id"] != row["evaluation_task_id"]:
raise OptimizationError("OPTIMIZATION_PROMOTION_TASK_MISMATCH")
if promotion["proposal_digest"] != row["proposal_digest"]:
raise OptimizationError("OPTIMIZATION_PROMOTION_PROPOSAL_MISMATCH")
if row["status"] == "PROMOTION_SUBMITTED":
return self._record(row)
if row["status"] != "ELIGIBLE":
raise OptimizationError("OPTIMIZATION_NOT_PROMOTABLE")
stored_receipt = self._trusted_receipt(self._decode(row["receipt_json"]))
if (
stored_receipt.get("task_id") != row["evaluation_task_id"]
or stored_receipt.get("proposal_digest") != row["proposal_digest"]
or stored_receipt.get("action") != "evaluate"
or stored_receipt.get("success") is not True
):
raise OptimizationError("OPTIMIZATION_PROMOTION_RECEIPT_BINDING_INVALID")
evaluation = self._decode(row["evaluation_json"])
if stored_receipt.get("result_digest") != _digest(evaluation):
raise OptimizationError("OPTIMIZATION_PROMOTION_QUALITY_RESULT_INVALID")
task_request = {
"schema": TASK_REQUEST_SCHEMA,
"task_id": f"optimization-promote-{proposal_id}",
"action": "promote",
"tenant_id": context["tenant_id"],
"subject_id": context["subject_id"],
"grant_id": context["grant_id"],
"mission_id": context["mission_id"],
"proposal_id": row["proposal_id"],
"proposal_digest": row["proposal_digest"],
"evaluation_task_id": row["evaluation_task_id"],
"evaluation_result_digest": stored_receipt["result_digest"],
}
try:
submission = self.submit_task(task_request)
if not isinstance(submission, Mapping):
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_INVALID")
task_id = _identifier(
submission.get("task_id"), "OPTIMIZATION_TASK_SUBMISSION_INVALID"
)
except Exception as exc:
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,error_code=?,updated_at=? WHERE proposal_id=?",
("STOPPED_PROMOTION_SUBMISSION_FAILED", "OPTIMIZATION_TASK_SUBMISSION_FAILED", time.time(), proposal_id),
)
if isinstance(exc, OptimizationError):
raise
raise OptimizationError("OPTIMIZATION_TASK_SUBMISSION_FAILED") from exc
with self._connect() as connection:
connection.execute(
"UPDATE optimization_proposals SET status=?,promotion_task_id=?,error_code=NULL,updated_at=? "
"WHERE proposal_id=? AND status='ELIGIBLE'",
("PROMOTION_SUBMITTED", task_id, time.time(), proposal_id),
)
return self.get_proposal(proposal_id, authenticated_context=context)
__all__ = [
"OptimizationController", "OptimizationEnvelope", "OptimizationError",
"canonical_json_bytes", "validate_proposal",
]
+742
View File
@@ -0,0 +1,742 @@
"""Release-owned deployment targets and a receipt-gated persistent saga.
The module persists intent before invoking a governed operation adapter. It
does not contain a shell runner, production signer, or success simulator. A
configured verifier must translate an adapter/sink receipt into a typed,
operation-bound :class:`VerifiedReceipt` before a stage can advance.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import sqlite3
from typing import Any, Callable, Mapping, Sequence
from .lineage import (
LineageLedger,
VerifiedAcceptanceEvidence,
canonical_artifact,
decode_artifact,
)
class ReleaseError(RuntimeError):
pass
class ReleaseConflict(ReleaseError):
pass
class ReleaseVerificationError(ReleaseError):
pass
class ReleaseAuthorizationError(ReleaseError):
pass
class ReleaseReconciliationRequired(ReleaseError):
pass
@dataclass(frozen=True)
class VerifiedReceipt:
"""Trusted verifier output, fully bound to one persisted stage effect."""
receipt_id: str
tenant_id: str
subject_id: str
component_id: str
task_id: str
source_task_id: str
source_effect_id: str
grant_id: str
job_id: str
effect_id: str
operation_id: str
deployment_id: str
release_id: str
artifact: Any
observed: Mapping[str, Any]
outcome: str = "SUCCEEDED"
observed_at: str | None = None
@dataclass(frozen=True)
class EnvironmentDeploymentTarget:
tenant_id: str
component_id: str
environment: str
release_id: str
artifact: Any
revision: int
stage: str
source_revision: str | None
observed_main_revision: str | None
previous_release_id: str | None
previous_artifact: Any | None
rollback_release_id: str | None
rollback_artifact: Any | None
rollback_compatible: bool
updated_by_subject_id: str
updated_at: str
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _required(value: Any, name: str) -> str:
text = str(value or "")
if not text or len(text) > 512 or any(ord(char) < 32 for char in text):
raise ValueError(f"invalid {name}")
return text
def _json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
def _recipe_digest(stages: Sequence[Mapping[str, Any]]) -> str:
return hashlib.sha256(_json(list(stages)).encode()).hexdigest()
class ReleaseLedger:
"""The release domain authority for dynamic targets and deployment sagas."""
TARGET_STAGES = {"STAGED", "ACTIVE", "ROLLBACK_STAGED"}
def __init__(
self,
path: str | Path,
*,
lineage: LineageLedger,
receipt_verifier: Callable[[Any, Mapping[str, Any]], VerifiedReceipt],
target_authorizer: Callable[[str,str,str,str,str], bool],
):
if not isinstance(lineage, LineageLedger):
raise ValueError("lineage ledger is required")
if not callable(receipt_verifier):
raise ValueError("receipt_verifier is required")
if not callable(target_authorizer):
raise ValueError("target_authorizer is required")
self.path = str(path)
self.lineage = lineage
self.receipt_verifier = receipt_verifier
self.target_authorizer = target_authorizer
Path(self.path).parent.mkdir(parents=True, exist_ok=True)
self._initialize()
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _initialize(self) -> None:
with self._connect() as connection:
connection.execute("PRAGMA journal_mode=WAL")
tables={row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")}
if "release_targets" in tables and "release_schema_meta" not in tables:
raise ReleaseConflict("unversioned candidate release schema requires explicit migration")
if "release_schema_meta" in tables:
version=connection.execute("SELECT version FROM release_schema_meta WHERE domain='release'").fetchone()
if version is None or version[0] != 2:
raise ReleaseConflict("unsupported release schema version")
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS release_schema_meta (
domain TEXT PRIMARY KEY,
version INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS release_recipes (
recipe_id TEXT PRIMARY KEY,
recipe_digest TEXT NOT NULL,
stages_json TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS release_targets (
tenant_id TEXT NOT NULL,
component_id TEXT NOT NULL,
environment TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
revision INTEGER NOT NULL,
stage TEXT NOT NULL,
source_revision TEXT,
observed_main_revision TEXT,
previous_release_id TEXT,
previous_artifact_json TEXT,
rollback_release_id TEXT,
rollback_artifact_json TEXT,
rollback_compatible INTEGER NOT NULL CHECK (rollback_compatible IN (0,1)),
updated_by_subject_id TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, component_id, environment)
);
CREATE TABLE IF NOT EXISTS release_sagas (
tenant_id TEXT NOT NULL,
saga_id TEXT NOT NULL,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
source_task_id TEXT NOT NULL,
source_effect_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
release_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
environment TEXT NOT NULL,
artifact_json TEXT NOT NULL,
recipe_id TEXT NOT NULL,
recipe_digest TEXT NOT NULL,
target_revision INTEGER NOT NULL,
stage_index INTEGER NOT NULL,
state TEXT NOT NULL,
revision INTEGER NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, saga_id),
UNIQUE (tenant_id, deployment_id),
FOREIGN KEY (recipe_id) REFERENCES release_recipes(recipe_id)
);
CREATE TABLE IF NOT EXISTS release_effect_intents (
tenant_id TEXT NOT NULL,
saga_id TEXT NOT NULL,
stage_index INTEGER NOT NULL,
stage_id TEXT NOT NULL,
operation_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
job_id TEXT,
state TEXT NOT NULL,
receipt_id TEXT,
receipt_json TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
PRIMARY KEY (tenant_id, saga_id, stage_index),
UNIQUE (tenant_id, effect_id),
FOREIGN KEY (tenant_id, saga_id)
REFERENCES release_sagas(tenant_id, saga_id)
);
CREATE TABLE IF NOT EXISTS release_deployment_facts (
tenant_id TEXT NOT NULL,
fact_id INTEGER PRIMARY KEY AUTOINCREMENT,
subject_id TEXT NOT NULL,
mission_id TEXT NOT NULL,
component_id TEXT NOT NULL,
task_id TEXT NOT NULL,
grant_id TEXT NOT NULL,
job_id TEXT NOT NULL,
deployment_id TEXT NOT NULL,
release_id TEXT NOT NULL,
artifact_json TEXT NOT NULL,
saga_id TEXT NOT NULL,
stage_id TEXT NOT NULL,
operation_id TEXT NOT NULL,
effect_id TEXT NOT NULL,
receipt_id TEXT NOT NULL,
observed_json TEXT NOT NULL,
created_at TEXT NOT NULL,
UNIQUE (tenant_id, effect_id, receipt_id),
FOREIGN KEY (tenant_id, saga_id)
REFERENCES release_sagas(tenant_id, saga_id)
);
"""
)
connection.execute("INSERT OR IGNORE INTO release_schema_meta(domain,version) VALUES('release',2)")
def close(self) -> None:
"""Compatibility hook; connections are transaction-scoped."""
def _authorize(self, tenant_id: str, subject_id: str, component_id: str, environment: str, action: str) -> None:
try:
authorized=self.target_authorizer(tenant_id,subject_id,component_id,environment,action)
except Exception as exc:
raise ReleaseAuthorizationError("deployment target authorization unavailable") from exc
if authorized is not True:
raise ReleaseAuthorizationError("deployment target authorization denied")
def register_recipe(self, recipe_id: str, stages: Sequence[Mapping[str, Any]]) -> dict[str, Any]:
recipe_id = _required(recipe_id, "recipe_id")
if not isinstance(stages, Sequence) or isinstance(stages, (str, bytes)) or not stages:
raise ValueError("recipe stages must be a non-empty sequence")
normalized = []
stage_ids = set()
for item in stages:
if not isinstance(item, Mapping) or set(item) - {"stage_id", "operation_id", "parameters"}:
raise ValueError("invalid recipe stage")
stage_id = _required(item.get("stage_id"), "stage_id")
operation_id = _required(item.get("operation_id"), "operation_id")
if stage_id in stage_ids:
raise ValueError("duplicate recipe stage_id")
stage_ids.add(stage_id)
parameters = item.get("parameters", {})
if not isinstance(parameters, Mapping):
raise ValueError("stage parameters must be an object")
normalized.append({"stage_id": stage_id, "operation_id": operation_id, "parameters": dict(parameters)})
stages_json = _json(normalized)
digest = _recipe_digest(normalized)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute("SELECT * FROM release_recipes WHERE recipe_id=?", (recipe_id,)).fetchone()
if existing:
if existing["recipe_digest"] != digest:
raise ReleaseConflict("recipe_id already has different immutable stages")
else:
connection.execute("INSERT INTO release_recipes VALUES(?,?,?,?)", (recipe_id, digest, stages_json, _now()))
return {"recipe_id": recipe_id, "recipe_digest": digest, "stages": normalized}
def set_target(
self,
tenant_id: str,
subject_id: str,
component_id: str,
environment: str,
release_id: str,
artifact: Any,
*,
expected_revision: int | None,
stage: str = "STAGED",
source_revision: str | None = None,
observed_main_revision: str | None = None,
rollback_release_id: str | None = None,
rollback_artifact: Any | None = None,
rollback_compatible: bool = False,
) -> EnvironmentDeploymentTarget:
tenant_id, subject_id, component_id, environment, release_id = map(
lambda pair: _required(pair[0], pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(component_id,"component_id"),(environment,"environment"),(release_id,"release_id")),
)
self._authorize(tenant_id,subject_id,component_id,environment,"write")
if stage not in self.TARGET_STAGES:
raise ValueError("invalid target stage")
if expected_revision is not None and (not isinstance(expected_revision, int) or isinstance(expected_revision, bool) or expected_revision < 0):
raise ValueError("invalid expected_revision")
if type(rollback_compatible) is not bool:
raise ValueError("rollback_compatible must be boolean")
artifact_json = canonical_artifact(artifact)
try:
linked_release = self.lineage.get_release_for_component(tenant_id, component_id, release_id)
except KeyError as exc:
raise ReleaseConflict("deployment target release has no lineage") from exc
if canonical_artifact(linked_release["artifact"]) != artifact_json:
raise ReleaseConflict("deployment target artifact does not match release")
rollback_artifact_json = canonical_artifact(rollback_artifact) if rollback_artifact is not None else None
source_revision = _required(source_revision, "source_revision") if source_revision else None
observed_main_revision = _required(observed_main_revision, "observed_main_revision") if observed_main_revision else None
rollback_release_id = _required(rollback_release_id, "rollback_release_id") if rollback_release_id else None
if (rollback_release_id is None) != (rollback_artifact_json is None):
raise ValueError("rollback release and artifact must be supplied together")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id, component_id, environment),
).fetchone()
if current is None:
if expected_revision is not None:
raise ReleaseConflict("expected revision does not match absent target")
if stage == "ROLLBACK_STAGED":
raise ReleaseConflict("rollback requires a current target")
revision = 1
previous_release_id = previous_artifact_json = None
else:
if expected_revision != current["revision"]:
raise ReleaseConflict("expected revision does not match target")
if stage == "ROLLBACK_STAGED":
if not current["rollback_compatible"]:
raise ReleaseConflict("current target has no compatible rollback")
if release_id != current["rollback_release_id"] or artifact_json != current["rollback_artifact_json"]:
raise ReleaseConflict("rollback target does not match compatible rollback binding")
revision = current["revision"] + 1
previous_release_id = current["release_id"]
previous_artifact_json = current["artifact_json"]
if rollback_compatible and rollback_release_id is None:
rollback_release_id = previous_release_id
rollback_artifact_json = previous_artifact_json
if rollback_compatible and current is not None and rollback_release_id is None:
raise ReleaseConflict("compatible rollback requires a previous target")
now = _now()
connection.execute(
"""INSERT INTO release_targets VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(tenant_id,component_id,environment) DO UPDATE SET
release_id=excluded.release_id, artifact_json=excluded.artifact_json,
revision=excluded.revision, stage=excluded.stage,
source_revision=excluded.source_revision,
observed_main_revision=excluded.observed_main_revision,
previous_release_id=excluded.previous_release_id,
previous_artifact_json=excluded.previous_artifact_json,
rollback_release_id=excluded.rollback_release_id,
rollback_artifact_json=excluded.rollback_artifact_json,
rollback_compatible=excluded.rollback_compatible,
updated_by_subject_id=excluded.updated_by_subject_id,
updated_at=excluded.updated_at""",
(tenant_id,component_id,environment,release_id,artifact_json,revision,stage,
source_revision,observed_main_revision,previous_release_id,previous_artifact_json,
rollback_release_id,rollback_artifact_json,int(rollback_compatible),subject_id,now),
)
row = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id,component_id,environment),
).fetchone()
return self._target(row)
def get_target(self, tenant_id: str, subject_id: str, component_id: str, environment: str) -> EnvironmentDeploymentTarget:
tenant_id,subject_id,component_id,environment=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),
_required(component_id,"component_id"),_required(environment,"environment"),
)
self._authorize(tenant_id,subject_id,component_id,environment,"read")
with self._connect() as connection:
row = connection.execute(
"SELECT * FROM release_targets WHERE tenant_id=? AND component_id=? AND environment=?",
(tenant_id,component_id,environment),
).fetchone()
if row is None:
raise KeyError("deployment target not found")
return self._target(row)
@staticmethod
def _target(row: sqlite3.Row) -> EnvironmentDeploymentTarget:
return EnvironmentDeploymentTarget(
tenant_id=row["tenant_id"], component_id=row["component_id"], environment=row["environment"],
release_id=row["release_id"], artifact=decode_artifact(row["artifact_json"]), revision=row["revision"],
stage=row["stage"], source_revision=row["source_revision"], observed_main_revision=row["observed_main_revision"],
previous_release_id=row["previous_release_id"],
previous_artifact=decode_artifact(row["previous_artifact_json"]) if row["previous_artifact_json"] else None,
rollback_release_id=row["rollback_release_id"],
rollback_artifact=decode_artifact(row["rollback_artifact_json"]) if row["rollback_artifact_json"] else None,
rollback_compatible=bool(row["rollback_compatible"]),
updated_by_subject_id=row["updated_by_subject_id"], updated_at=row["updated_at"],
)
def start_saga(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, saga_id: str,
release_id: str, deployment_id: str, environment: str, artifact: Any,
recipe_id: str, *, expected_target_revision: int,
) -> dict[str, Any]:
tenant_id,subject_id,mission_id,component_id,saga_id,release_id,deployment_id,environment,recipe_id=map(
lambda pair:_required(pair[0],pair[1]),
((tenant_id,"tenant_id"),(subject_id,"subject_id"),(mission_id,"mission_id"),(component_id,"component_id"),(saga_id,"saga_id"),
(release_id,"release_id"),(deployment_id,"deployment_id"),(environment,"environment"),(recipe_id,"recipe_id")),
)
if not isinstance(expected_target_revision,int) or isinstance(expected_target_revision,bool) or expected_target_revision < 1:
raise ValueError("invalid expected_target_revision")
self._authorize(tenant_id,subject_id,component_id,environment,"deploy")
artifact_json=canonical_artifact(artifact)
target=self.get_target(tenant_id,subject_id,component_id,environment)
if target.revision != expected_target_revision:
raise ReleaseConflict("expected target revision does not match")
if target.stage not in {"STAGED","ROLLBACK_STAGED"}:
raise ReleaseConflict("deployment target is not staged")
if target.release_id != release_id or canonical_artifact(target.artifact) != artifact_json:
raise ReleaseConflict("saga does not match deployment target")
graph=self.lineage.explain(tenant_id,subject_id,mission_id)
release=next((item for item in graph["releases"] if item["release_id"] == release_id),None)
if release is None or release["component_id"] != component_id or canonical_artifact(release["artifact"]) != artifact_json:
raise ReleaseConflict("release has no exact source lineage")
source_task=next((item for item in graph["tasks"] if item["task_id"] == release["source_task_id"]),None)
if source_task is None or source_task["component_id"] != component_id:
raise ReleaseConflict("release source Task component lineage missing")
with self._connect() as connection:
recipe=connection.execute("SELECT * FROM release_recipes WHERE recipe_id=?",(recipe_id,)).fetchone()
if recipe is None:
raise ReleaseConflict("recipe not found")
# The relationship is durable before saga admission and is idempotent on retry.
self.lineage.link_deployment(
tenant_id,subject_id,mission_id,release_id,deployment_id,component_id,environment,
expected_target_revision,decode_artifact(artifact_json),
metadata={"saga_id":saga_id,"recipe_id":recipe_id,"recipe_digest":recipe["recipe_digest"]},
)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
existing=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
expected=(subject_id,mission_id,component_id,release["source_task_id"],release["source_effect_id"],source_task["grant_id"],release_id,deployment_id,environment,artifact_json,recipe_id,recipe["recipe_digest"],expected_target_revision)
if existing:
actual=tuple(existing[key] for key in ("subject_id","mission_id","component_id","source_task_id","source_effect_id","grant_id","release_id","deployment_id","environment","artifact_json","recipe_id","recipe_digest","target_revision"))
if actual != expected:
raise ReleaseConflict("saga_id already has different immutable binding")
return self._saga(connection,existing)
collision=connection.execute("SELECT saga_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",(tenant_id,deployment_id)).fetchone()
if collision:
raise ReleaseConflict("deployment_id already has a different saga")
now=_now()
connection.execute(
"INSERT INTO release_sagas VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,saga_id,subject_id,mission_id,component_id,release["source_task_id"],release["source_effect_id"],source_task["grant_id"],release_id,deployment_id,environment,artifact_json,
recipe_id,recipe["recipe_digest"],expected_target_revision,0,"RUNNING",0,now,now),
)
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
return self._saga(connection,row)
def get_saga(self, tenant_id: str, subject_id: str, saga_id: str) -> dict[str, Any]:
with self._connect() as connection:
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(_required(tenant_id,"tenant_id"),_required(saga_id,"saga_id"))).fetchone()
if row is None:
raise KeyError("release saga not found")
if row["subject_id"] != _required(subject_id,"subject_id"):
raise ReleaseAuthorizationError("release saga subject denied")
return self._saga(connection,row)
@staticmethod
def _stages(connection: sqlite3.Connection, recipe_id: str) -> list[dict[str, Any]]:
row=connection.execute("SELECT stages_json FROM release_recipes WHERE recipe_id=?",(recipe_id,)).fetchone()
if row is None:
raise ReleaseConflict("persisted recipe missing")
return json.loads(row["stages_json"])
def _saga(self, connection: sqlite3.Connection, row: sqlite3.Row) -> dict[str, Any]:
stages=self._stages(connection,row["recipe_id"])
index=row["stage_index"]
return {
"tenant_id":row["tenant_id"],"subject_id":row["subject_id"],"mission_id":row["mission_id"],
"component_id":row["component_id"],"source_task_id":row["source_task_id"],
"source_effect_id":row["source_effect_id"],"grant_id":row["grant_id"],
"saga_id":row["saga_id"],"release_id":row["release_id"],"deployment_id":row["deployment_id"],
"environment":row["environment"],"artifact":decode_artifact(row["artifact_json"]),
"recipe_id":row["recipe_id"],"recipe_digest":row["recipe_digest"],
"target_revision":row["target_revision"],"stage_index":index,
"stage_id":stages[index]["stage_id"] if index < len(stages) else None,
"state":row["state"],"revision":row["revision"],"created_at":row["created_at"],"updated_at":row["updated_at"],
}
def advance_saga(
self, tenant_id: str, subject_id: str, saga_id: str, expected_revision: int,
operation_adapter: Any,
) -> dict[str, Any]:
tenant_id,subject_id,saga_id=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),_required(saga_id,"saga_id")
)
if not isinstance(expected_revision,int) or isinstance(expected_revision,bool) or expected_revision < 0:
raise ValueError("invalid expected_revision")
prepare=getattr(operation_adapter,"prepare",None)
execute=getattr(operation_adapter,"execute",None)
if not callable(prepare) or not callable(execute):
raise ValueError("governed operation adapter requires prepare and execute")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
saga=connection.execute(
"SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)
).fetchone()
if saga is None:
raise KeyError("release saga not found")
if saga["subject_id"] != subject_id:
raise ReleaseAuthorizationError("saga belongs to different subject")
self._authorize(tenant_id,subject_id,saga["component_id"],saga["environment"],"deploy")
if saga["revision"] != expected_revision:
raise ReleaseConflict("expected revision does not match saga")
if saga["state"] == "COMPLETED":
return self._saga(connection,saga)
stages=self._stages(connection,saga["recipe_id"])
stage=stages[saga["stage_index"]]
effect_id=f"release:{saga_id}:{saga['stage_index']}:{stage['stage_id']}"
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if intent is None:
now=_now()
connection.execute(
"INSERT INTO release_effect_intents VALUES(?,?,?,?,?,?,?,?,?,?,?,?)",
(tenant_id,saga_id,saga["stage_index"],stage["stage_id"],stage["operation_id"],effect_id,
None,"PENDING_JOB",None,None,now,now),
)
elif (intent["effect_id"],intent["stage_id"],intent["operation_id"]) != (
effect_id,stage["stage_id"],stage["operation_id"]
):
raise ReleaseConflict("persisted stage effect identity mismatch")
operation={
"tenant_id":tenant_id,"subject_id":subject_id,"mission_id":saga["mission_id"],
"component_id":saga["component_id"],"task_id":saga["source_task_id"],
"source_task_id":saga["source_task_id"],"source_effect_id":saga["source_effect_id"],
"grant_id":saga["grant_id"],"saga_id":saga_id,
"deployment_id":saga["deployment_id"],"release_id":saga["release_id"],
"environment":saga["environment"],"artifact":decode_artifact(saga["artifact_json"]),
"target_revision":saga["target_revision"],"stage_index":saga["stage_index"],
"stage_id":stage["stage_id"],"operation_id":stage["operation_id"],"effect_id":effect_id,
"parameters":stage.get("parameters",{}),
}
with self._connect() as connection:
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
job_id=intent["job_id"]
if job_id is None:
prepared=prepare(dict(operation))
job_id=prepared.get("job_id") if isinstance(prepared,Mapping) else prepared
job_id=_required(job_id,"prepared job_id")
# This is the first point where the relationship graph may name a Job:
# link_effect independently resolves the configured real Job authority.
self.lineage.link_effect(
tenant_id,subject_id,saga["mission_id"],saga["source_task_id"],effect_id,job_id,
source_effect_id=saga["source_effect_id"],
metadata={"deployment_id":saga["deployment_id"],"release_id":saga["release_id"],
"component_id":saga["component_id"],"stage_id":stage["stage_id"]},
)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute(
"SELECT revision,stage_index FROM release_sagas WHERE tenant_id=? AND saga_id=?",
(tenant_id,saga_id),
).fetchone()
if current is None or current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before Job binding")
bound=connection.execute(
"SELECT job_id FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if bound["job_id"] not in (None,job_id):
raise ReleaseConflict("stage effect already bound to different Job")
connection.execute(
"UPDATE release_effect_intents SET job_id=?,state='JOB_LINKED',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(job_id,_now(),tenant_id,saga_id,saga["stage_index"]),
)
operation["job_id"]=job_id
with self._connect() as connection:
intent=connection.execute(
"SELECT * FROM release_effect_intents WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(tenant_id,saga_id,saga["stage_index"]),
).fetchone()
if intent["receipt_json"]:
raw_receipt=json.loads(intent["receipt_json"])
elif intent["state"] == "EFFECT_UNKNOWN":
reconcile=getattr(operation_adapter,"reconcile",None)
if not callable(reconcile):
raise ReleaseReconciliationRequired("RECONCILIATION_REQUIRED: adapter readback unavailable")
raw_receipt=reconcile(dict(operation))
else:
try:
raw_receipt=execute(dict(operation))
except Exception:
with self._connect() as connection:
connection.execute(
"UPDATE release_effect_intents SET state='EFFECT_UNKNOWN',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(_now(),tenant_id,saga_id,saga["stage_index"]),
)
raise
if not isinstance(raw_receipt,Mapping):
with self._connect() as connection:
connection.execute(
"UPDATE release_effect_intents SET state='EFFECT_UNKNOWN',updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(_now(),tenant_id,saga_id,saga["stage_index"]),
)
raise ReleaseVerificationError("operation adapter receipt must be an object")
try:
unverified_receipt_json=_json(dict(raw_receipt))
except (TypeError,ValueError) as exc:
raise ReleaseVerificationError("operation adapter receipt must be canonical JSON") from exc
if len(unverified_receipt_json.encode("utf-8")) > 262_144:
raise ReleaseVerificationError("operation adapter receipt is too large")
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute(
"SELECT revision,stage_index FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)
).fetchone()
if current is None or current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before receipt persistence")
connection.execute(
"UPDATE release_effect_intents SET state='RECEIPT_UNVERIFIED',receipt_json=?,updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(unverified_receipt_json,_now(),tenant_id,saga_id,saga["stage_index"]),
)
expected={key:operation[key] for key in (
"tenant_id","subject_id","component_id","task_id","source_task_id","source_effect_id","grant_id",
"job_id","effect_id","operation_id","deployment_id","release_id","artifact"
)}
try:
verified=self.receipt_verifier(raw_receipt,expected)
except Exception as exc:
raise ReleaseVerificationError("configured verifier rejected operation receipt") from exc
if not isinstance(verified,VerifiedReceipt):
raise ReleaseVerificationError("receipt_verifier must return VerifiedReceipt")
if verified.outcome != "SUCCEEDED":
raise ReleaseVerificationError("verified receipt outcome is not SUCCEEDED")
fields=("tenant_id","subject_id","component_id","task_id","source_task_id","source_effect_id","grant_id",
"job_id","effect_id","operation_id","deployment_id","release_id","artifact")
bindings=tuple(canonical_artifact(getattr(verified,name)) if name == "artifact" else getattr(verified,name) for name in fields)
expected_bindings=tuple(canonical_artifact(operation[name]) if name == "artifact" else operation[name] for name in fields)
if bindings != expected_bindings:
wrong=next(name for name,actual,wanted in zip(fields,bindings,expected_bindings) if actual != wanted)
raise ReleaseVerificationError(f"verified receipt {wrong} binding mismatch")
receipt_id=_required(verified.receipt_id,"receipt_id")
if not isinstance(verified.observed,Mapping):
raise ReleaseVerificationError("verified receipt observation must be an object")
observed_json=_json(dict(verified.observed))
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
current=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
if current["revision"] != expected_revision or current["stage_index"] != saga["stage_index"]:
raise ReleaseConflict("expected revision changed before receipt commit")
connection.execute(
"UPDATE release_effect_intents SET state='VERIFIED',receipt_id=?,updated_at=? WHERE tenant_id=? AND saga_id=? AND stage_index=?",
(receipt_id,_now(),tenant_id,saga_id,saga["stage_index"]),
)
connection.execute(
"""INSERT OR IGNORE INTO release_deployment_facts
(tenant_id,subject_id,mission_id,component_id,task_id,grant_id,job_id,deployment_id,release_id,artifact_json,saga_id,stage_id,operation_id,effect_id,receipt_id,observed_json,created_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
(tenant_id,subject_id,saga["mission_id"],saga["component_id"],saga["source_task_id"],saga["grant_id"],job_id,
saga["deployment_id"],saga["release_id"],saga["artifact_json"],saga_id,stage["stage_id"],stage["operation_id"],
effect_id,receipt_id,observed_json,_now()),
)
next_index=saga["stage_index"]+1
state="COMPLETED" if next_index == len(stages) else "RUNNING"
updated=connection.execute(
"UPDATE release_sagas SET stage_index=?,state=?,revision=revision+1,updated_at=? WHERE tenant_id=? AND saga_id=? AND revision=?",
(next_index,state,_now(),tenant_id,saga_id,expected_revision),
)
if updated.rowcount != 1:
raise ReleaseConflict("expected revision changed")
row=connection.execute("SELECT * FROM release_sagas WHERE tenant_id=? AND saga_id=?",(tenant_id,saga_id)).fetchone()
return self._saga(connection,row)
def deployment_facts(self, tenant_id: str, subject_id: str, component_id: str, deployment_id: str) -> list[dict[str, Any]]:
tenant_id,subject_id,component_id,deployment_id=(
_required(tenant_id,"tenant_id"),_required(subject_id,"subject_id"),
_required(component_id,"component_id"),_required(deployment_id,"deployment_id"),
)
with self._connect() as connection:
saga=connection.execute(
"SELECT environment,subject_id,component_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",
(tenant_id,deployment_id),
).fetchone()
if saga is None or saga["subject_id"] != subject_id or saga["component_id"] != component_id:
raise ReleaseAuthorizationError("deployment facts scope denied")
self._authorize(tenant_id,subject_id,component_id,saga["environment"],"read")
rows=connection.execute(
"SELECT * FROM release_deployment_facts WHERE tenant_id=? AND subject_id=? AND component_id=? AND deployment_id=? ORDER BY fact_id",
(tenant_id,subject_id,component_id,deployment_id),
).fetchall()
return [{**{key:row[key] for key in row.keys() if not key.endswith("_json")},
"artifact":decode_artifact(row["artifact_json"]),"observed":json.loads(row["observed_json"])} for row in rows]
def record_acceptance(
self, tenant_id: str, subject_id: str, mission_id: str, component_id: str, acceptance_id: str,
deployment_id: str, release_id: str, artifact: Any, accepted: bool, *,
evidence: Any,
verifier: Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence],
) -> dict[str, Any]:
with self._connect() as connection:
saga=connection.execute(
"SELECT environment,subject_id,component_id FROM release_sagas WHERE tenant_id=? AND deployment_id=?",
(tenant_id,deployment_id),
).fetchone()
if saga is None or saga["subject_id"] != subject_id or saga["component_id"] != component_id:
raise ReleaseAuthorizationError("acceptance deployment scope denied")
self._authorize(tenant_id,subject_id,component_id,saga["environment"],"accept")
return self.lineage.record_acceptance(
tenant_id,subject_id,mission_id,component_id,acceptance_id,deployment_id,release_id,
artifact,accepted,evidence=evidence,verifier=verifier,
)
__all__ = [
"EnvironmentDeploymentTarget", "ReleaseAuthorizationError", "ReleaseConflict", "ReleaseError", "ReleaseLedger",
"ReleaseReconciliationRequired", "ReleaseVerificationError", "VerifiedReceipt",
]
@@ -0,0 +1,375 @@
"""Pure validation and deployment-truth evaluation for runtime contracts.
Runtime contracts describe stable structure. A release-owned deployment target
is supplied separately so that changing a release does not rewrite structural
configuration or turn a source branch into runtime evidence.
"""
from __future__ import annotations
from copy import deepcopy
from datetime import datetime, timezone
import re
from pathlib import PurePosixPath
from typing import Any, Mapping, Sequence
CONTRACT_SCHEMA = "aios.runtime-contract.v1"
LIFECYCLES = frozenset({"active", "candidate", "legacy", "example"})
_DYNAMIC_FIELDS = frozenset(
{"desired", "desired_commit", "desired_release", "desired_target", "desired_targets"}
)
_SECRET_FIELDS = frozenset(
{"secret", "secret_bytes", "token", "password", "private_key", "credential_value", "value"}
)
_DIGEST = re.compile(r"^(?:sha256:)?[0-9a-fA-F]{64}$")
_SAFE_SYMBOLIC_REF = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*:[/A-Za-z0-9_.:@+-]+$")
def _error(code: str, path: str = "") -> ValueError:
return ValueError(f"{code}:{path}" if path else code)
def _nonempty_string(value: object, path: str) -> str:
if not isinstance(value, str) or not value.strip():
raise _error("field_invalid", path)
return value.strip()
def _positive_int(value: object, path: str) -> int:
if isinstance(value, bool) or not isinstance(value, int) or value < 1:
raise _error("field_invalid", path)
return value
def _reject_dynamic_fields(value: object, path: str = "contract") -> None:
if isinstance(value, Mapping):
for key, child in value.items():
if not isinstance(key, str):
raise _error("field_name_invalid", path)
child_path = f"{path}.{key}"
if key.lower() in _DYNAMIC_FIELDS:
raise _error("dynamic_field_forbidden", child_path)
_reject_dynamic_fields(child, child_path)
elif isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
for index, child in enumerate(value):
_reject_dynamic_fields(child, f"{path}[{index}]")
def _safe_path_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
if "\x00" in text or "\\" in text:
raise _error("unsafe_path_ref", path)
candidate = PurePosixPath(text)
if candidate.is_absolute() or any(part in {"", ".", ".."} for part in candidate.parts):
raise _error("unsafe_path_ref", path)
return text
def _symbolic_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
if "\x00" in text or ".." in text or not _SAFE_SYMBOLIC_REF.fullmatch(text):
raise _error("resolver_ref_invalid", path)
return text
def _credential_ref(value: object, path: str) -> str:
text = _nonempty_string(value, path)
allowed = (
text.startswith("credential://")
or text.startswith("secret://")
or text.startswith("systemd-credential:")
)
if not allowed or "\x00" in text or ".." in text or text.startswith("/run/credentials/"):
raise _error("credential_ref_invalid", path)
return text
def _validate_binding_group(
group: object,
group_name: str,
component_id: str,
environment: str,
tenant_scope: str,
) -> dict[str, Any]:
if not isinstance(group, Mapping):
raise _error("binding_group_invalid", group_name)
normalized: dict[str, Any] = {}
for name, raw in group.items():
item_path = f"{group_name}.{name}"
_nonempty_string(name, f"{group_name}.name")
if isinstance(raw, str):
reference_field = (
"credential_ref"
if group_name == "credential_refs"
else "path_ref"
if group_name == "resource_refs" and ":" not in raw
else "resolver_ref"
)
item = {
"component_id": component_id,
"environment": environment,
reference_field: raw,
}
elif isinstance(raw, Mapping):
item = deepcopy(dict(raw))
else:
raise _error("binding_invalid", item_path)
if _nonempty_string(item.get("component_id"), f"{item_path}.component_id") != component_id:
raise _error("binding_component_mismatch", item_path)
if _nonempty_string(item.get("environment"), f"{item_path}.environment") != environment:
raise _error("binding_environment_mismatch", item_path)
if "tenant_scope" in item and (
_nonempty_string(item["tenant_scope"], f"{item_path}.tenant_scope") != tenant_scope
):
raise _error("binding_tenant_mismatch", item_path)
reference_fields = [key for key in ("resolver_ref", "path_ref", "credential_ref") if key in item]
if len(reference_fields) != 1:
raise _error("binding_reference_invalid", item_path)
reference_field = reference_fields[0]
reference_path = f"{item_path}.{reference_field}"
if group_name == "credential_refs" or reference_field == "credential_ref":
for key in item:
if key.lower() in _SECRET_FIELDS:
raise _error("secret_material_forbidden", f"{item_path}.{key}")
item[reference_field] = _credential_ref(item[reference_field], reference_path)
elif reference_field == "path_ref":
item[reference_field] = _safe_path_ref(item[reference_field], reference_path)
else:
item[reference_field] = _symbolic_ref(item[reference_field], reference_path)
normalized[str(name)] = item
return normalized
def validate_contract(document: Mapping[str, Any]) -> dict[str, Any]:
"""Validate and copy a stable ``aios.runtime-contract.v1`` document.
The returned mapping owns its nested containers, so callers cannot mutate the
source after validation. Invalid contracts raise ``ValueError`` with a stable
machine-readable code prefix.
"""
if not isinstance(document, Mapping):
raise _error("contract_invalid")
_reject_dynamic_fields(document)
result = deepcopy(dict(document))
if result.get("contract_schema") != CONTRACT_SCHEMA:
raise _error("contract_schema_invalid", "contract_schema")
result["contract_revision"] = _positive_int(result.get("contract_revision"), "contract_revision")
component_id = _nonempty_string(result.get("component_id"), "component_id")
environment = _nonempty_string(result.get("environment"), "environment")
result["component_id"] = component_id
result["environment"] = environment
tenant_scope = _nonempty_string(result.get("tenant_scope"), "tenant_scope")
result["tenant_scope"] = tenant_scope
lifecycle = _nonempty_string(result.get("lifecycle"), "lifecycle").lower()
if lifecycle not in LIFECYCLES:
raise _error("lifecycle_invalid", "lifecycle")
result["lifecycle"] = lifecycle
if "resources" in result and "resource_refs" in result:
raise _error("duplicate_resource_group")
if "resources" in result:
result["resource_refs"] = result.pop("resources")
for group_name in ("entrypoints", "bindings", "resource_refs", "credential_refs"):
result[group_name] = _validate_binding_group(
result.get(group_name), group_name, component_id, environment, tenant_scope
)
operations = result.get("operations")
if (
not isinstance(operations, Sequence)
or isinstance(operations, (str, bytes, bytearray))
or not operations
):
raise _error("operations_invalid", "operations")
normalized_operations = [
_nonempty_string(operation, f"operations[{index}]")
for index, operation in enumerate(operations)
]
if len(set(normalized_operations)) != len(normalized_operations):
raise _error("duplicate_operation", "operations")
result["operations"] = normalized_operations
return result
def _unknown(*reasons: str) -> dict[str, Any]:
return {
"status": "UNKNOWN",
"state": "UNKNOWN",
"verification": "UNKNOWN",
"converged": False,
"reasons": list(dict.fromkeys(reasons)),
}
def _drift(reasons: list[str]) -> dict[str, Any]:
return {
"status": "DRIFT",
"state": "DRIFT",
"verification": "FAILED",
"converged": False,
"reasons": list(dict.fromkeys(reasons)),
}
def _parse_time(value: object) -> datetime | None:
if isinstance(value, datetime):
parsed = value
elif isinstance(value, str):
try:
parsed = datetime.fromisoformat(value.strip().replace("Z", "+00:00"))
except ValueError:
return None
else:
return None
if parsed.tzinfo is None or parsed.utcoffset() is None:
return None
return parsed.astimezone(timezone.utc)
def _digest_value(document: Mapping[str, Any], *names: str) -> object:
for name in names:
if name in document:
return document[name]
return None
def _valid_digest(value: object) -> bool:
return isinstance(value, str) and bool(_DIGEST.fullmatch(value))
def evaluate_deployment(
contract: Mapping[str, Any],
target: Mapping[str, Any] | None,
observation: Mapping[str, Any] | None,
provenance: Mapping[str, Any] | None,
now: datetime | str,
) -> dict[str, Any]:
"""Evaluate one target using fresh runtime and provenance evidence.
Missing or malformed evidence is ``UNKNOWN``. ``DRIFT`` is reserved for a
complete evidence set that proves an incompatible target or runtime fact.
Repository HEAD is intentionally not an input to this decision.
"""
stable = validate_contract(contract)
missing: list[str] = []
if target is None:
missing.append("target_missing")
elif not isinstance(target, Mapping):
missing.append("target_invalid")
if observation is None:
missing.append("observation_missing")
elif not isinstance(observation, Mapping):
missing.append("observation_invalid")
if provenance is None:
missing.append("provenance_missing")
elif not isinstance(provenance, Mapping):
missing.append("provenance_invalid")
current_time = _parse_time(now)
if current_time is None:
missing.append("now_invalid")
if missing:
return _unknown(*missing)
assert target is not None and observation is not None and provenance is not None
assert current_time is not None
target_errors: list[str] = []
for field in ("component_id", "environment", "release_id", "deployment_id"):
if not isinstance(target.get(field), str) or not str(target[field]).strip():
target_errors.append(f"target_{field}_invalid")
if "tenant_scope" in target and (
not isinstance(target["tenant_scope"], str) or not target["tenant_scope"].strip()
):
target_errors.append("target_tenant_scope_invalid")
target_artifact = target.get("artifact_digest")
if not _valid_digest(target_artifact):
target_errors.append("target_artifact_digest_invalid")
if isinstance(target.get("revision"), bool) or not isinstance(target.get("revision"), int) or target.get("revision", 0) < 1:
target_errors.append("target_revision_invalid")
observed_at = _parse_time(observation.get("observed_at"))
max_age = observation.get("max_age_seconds", observation.get("max_age"))
observed_artifact = _digest_value(
observation, "actual_artifact_digest", "actual_artifact", "artifact_digest"
)
observation_errors: list[str] = []
if observed_at is None:
observation_errors.append("observation_observed_at_invalid")
if isinstance(max_age, bool) or not isinstance(max_age, (int, float)) or max_age <= 0:
observation_errors.append("observation_max_age_invalid")
if not isinstance(observation.get("source"), str) or not observation.get("source", "").strip():
observation_errors.append("observation_source_invalid")
if not _valid_digest(observed_artifact):
observation_errors.append("observation_actual_artifact_invalid")
if not isinstance(observation.get("healthy"), bool):
observation_errors.append("observation_healthy_invalid")
for field in ("component_id", "environment", "tenant_scope", "deployment_id"):
if field in observation and (
not isinstance(observation[field], str) or not observation[field].strip()
):
observation_errors.append(f"observation_{field}_invalid")
if "revision" in observation and (
isinstance(observation["revision"], bool)
or not isinstance(observation["revision"], int)
or observation["revision"] < 1
):
observation_errors.append("observation_revision_invalid")
artifact_provenance = _digest_value(provenance, "artifact_digest", "artifact_hash")
source_provenance = _digest_value(provenance, "source_digest", "source_hash")
builder_provenance = _digest_value(provenance, "builder_digest", "builder_hash")
provenance_errors: list[str] = []
if not _valid_digest(artifact_provenance):
provenance_errors.append("provenance_artifact_digest_invalid")
if not _valid_digest(source_provenance):
provenance_errors.append("provenance_source_digest_invalid")
if not _valid_digest(builder_provenance):
provenance_errors.append("provenance_builder_digest_invalid")
evidence_errors = target_errors + observation_errors + provenance_errors
if evidence_errors:
return _unknown(*evidence_errors)
assert observed_at is not None and isinstance(max_age, (int, float))
age = (current_time - observed_at).total_seconds()
if age < 0:
return _unknown("observation_from_future")
if age > max_age:
return _unknown("observation_stale")
drift: list[str] = []
if target["component_id"] != stable["component_id"]:
drift.append("target_component_id_mismatch")
if target["environment"] != stable["environment"]:
drift.append("target_environment_mismatch")
if "tenant_scope" in target and target["tenant_scope"] != stable["tenant_scope"]:
drift.append("target_tenant_scope_mismatch")
if "tenant_scope" in observation and observation["tenant_scope"] != stable["tenant_scope"]:
drift.append("observation_tenant_scope_mismatch")
for field in ("component_id", "environment", "deployment_id", "revision"):
if field in observation and observation[field] != target[field]:
drift.append(f"observation_{field}_mismatch")
if observed_artifact != target_artifact:
drift.append("artifact_mismatch")
if artifact_provenance != target_artifact:
drift.append("provenance_artifact_mismatch")
if observation["healthy"] is False:
drift.append("observation_unhealthy")
if drift:
return _drift(drift)
return {
"status": "VERIFIED",
"state": "CONVERGED",
"verification": "VERIFIED",
"converged": True,
"reasons": [],
"component_id": target["component_id"],
"environment": target["environment"],
"release_id": target["release_id"],
"deployment_id": target["deployment_id"],
"artifact_digest": target_artifact,
"revision": target["revision"],
"observed_at": observation["observed_at"],
"observation_source": observation["source"],
}
+450
View File
@@ -0,0 +1,450 @@
"""Thin authenticated application service over existing TaskPlaneStore and JobStore.
This module owns no second Task lifecycle store. Added task runtime columns live
in the supplied Task authority database; the lineage ledger owns only relations.
The built-in host adapter is READ_ONLY. Mutating operations require separately
registered governed adapters with receiver fencing/readback, never arbitrary argv.
"""
from __future__ import annotations
from dataclasses import dataclass
from contextlib import contextmanager
from datetime import datetime,timezone
import base64
import json
import re
import sqlite3
import threading
import time
from pathlib import Path
from typing import Mapping
from task_plane.models import TaskPlan,WorkerRegistration,ProbeEvidence,ProbeOutcome,Budget
from task_plane.errors import TaskPlaneHTTPError
from aios_gitops_job_controller import GitOpsJobController,JobStore,JobState,JobError
from .evidence import ReceiptSigner,canonical,digest
from .runtime_contract import validate_contract
from .worker_policy import eligible
from .authorization import AuthorizationError
from .host import HostError
class ServiceError(RuntimeError):
def __init__(self,code,status=409):
super().__init__(code)
self.code=code
self.status=status
@dataclass(frozen=True)
class Identity:
tenant_id: str
subject_id: str
role: str
worker_id: str | None=None
class BaselineService:
def __init__(self,store,job_root,authorization,contracts,operations,worker_profiles,host,lineage,machine_key,
*,machine_key_id='baseline-machine',trusted_receipt_keys=None):
self.store=store
self.signer=ReceiptSigner(machine_key,key_id=machine_key_id,trusted_public_keys=trusted_receipt_keys)
self.jobs=GitOpsJobController(JobStore(Path(job_root)),generic_receipt_verifier=self.signer.verify)
self.authorization=authorization
self.contracts={}
for entry in contracts:
value=validate_contract(entry)
key=(value['tenant_scope'],value['environment'],value['component_id'])
if key in self.contracts: raise ServiceError('CONTRACT_DUPLICATE')
self.contracts[key]=value
self.operations=operations
self.worker_profiles=dict(worker_profiles)
self.host=host
self.lineage=lineage
self.storage_blocked=False
self._execute_lock=threading.Lock()
with self.store._transaction():
self.store.conn.execute('''CREATE TABLE IF NOT EXISTS baseline_task_runtime(
task_id TEXT PRIMARY KEY REFERENCES tasks(task_id),ready_at REAL,
blocker_code TEXT,job_id TEXT,effect_id TEXT NOT NULL UNIQUE)''')
self.store.conn.execute('''CREATE TABLE IF NOT EXISTS baseline_worker_policy(
worker_id TEXT PRIMARY KEY,policy_json TEXT NOT NULL)''')
self.store.probe_attestation_verifier=self._verify_probe
def _user(self,identity):
if not isinstance(identity,Identity) or identity.role not in {'user','owner'}:
raise ServiceError('USER_IDENTITY_REQUIRED',403)
def _worker(self,identity):
if not isinstance(identity,Identity) or identity.role!='worker':
raise ServiceError('WORKER_IDENTITY_REQUIRED',403)
profile=self.worker_profiles.get(identity.worker_id)
if not profile or profile.identity!=identity.subject_id or identity.tenant_id not in profile.tenant_scope:
raise ServiceError('WORKER_SCOPE_DENIED',403)
return profile
def _task(self,identity,task_id):
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
if meta.get('tenant_id')!=identity.tenant_id or meta.get('subject_id')!=identity.subject_id:
raise ServiceError('TASK_SCOPE_DENIED',403)
return task
def _scope(self,task):
meta=task.task_plan.metadata['baseline']
return {**{key:meta[key] for key in ('tenant_id','subject_id','mission_id','operation','component_id','environment')},'task_id':task.task_id}
def _principal(self,task):
meta=task.task_plan.metadata['baseline']
return self.authorization.issue_principal(meta['grant_id'],ttl_seconds=60,**self._scope(task))
def _fresh_write(self):
if self.storage_blocked: raise ServiceError('STORAGE_RECOVERY_REQUIRED',503)
try:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=blocker_code WHERE 0')
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('TASK_STORAGE_UNAVAILABLE',503) from exc
def submit_goal(self,identity,request,*,idempotency_key):
self._user(identity)
allowed={'mission_id','grant_id','operation','component_id','environment','inputs','intent'}
if type(request) is not dict or set(request)!=allowed or not isinstance(idempotency_key,str) or not 1<=len(idempotency_key)<=128:
raise ServiceError('GOAL_SCHEMA_INVALID',400)
if any(not isinstance(request[key],str) or not request[key] or len(request[key])>2048 for key in allowed-{'inputs'}):
raise ServiceError('GOAL_SCHEMA_INVALID',400)
if request['inputs']!={}: raise ServiceError('OPERATION_INPUT_SCHEMA_INVALID',400)
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}',request['mission_id']): raise ServiceError('MISSION_ID_INVALID',400)
operation=self.operations.get(request['operation'])
if operation.handler_ref!='host.service_status' or operation.effect_class!='READ_ONLY' or operation.required_permissions!=(operation.operation_id,):
raise ServiceError('REGISTERED_HANDLER_UNAVAILABLE',503)
contract=self.contracts.get((identity.tenant_id,request['environment'],request['component_id']))
if not contract or contract['lifecycle']!='active' or request['operation'] not in contract['operations']:
raise ServiceError('CONTRACT_OPERATION_UNAVAILABLE',403)
host_key=identity.tenant_id+':'+request['environment']+':'+request['component_id']
if host_key not in self.host.bindings: raise ServiceError('HOST_BINDING_UNAVAILABLE',503)
key=digest({'tenant':identity.tenant_id,'subject':identity.subject_id,'key':idempotency_key})
task_id='task-'+key[:40]
effect_id='effect-'+digest({'tenant':identity.tenant_id,'task':task_id,'request':request})[:40]
meta={**request,'tenant_id':identity.tenant_id,'subject_id':identity.subject_id,'effect_id':effect_id,
'operation_version':operation.version,'contract_revision':contract['contract_revision']}
plan=TaskPlan.from_dict({'task_id':task_id,'task_type':'host.diagnostic','intent':request['intent'],
'issuer':{'id':identity.subject_id,'tenant_id':identity.tenant_id,'subject_id':identity.subject_id},
'required_capabilities':[request['operation']],'risk':'low','approval_policy':'governed-grant',
'correlation_id':request['mission_id'],'metadata':{'baseline':meta},
'budgets':{'attempts':operation.retry['max_attempts'],'time_seconds':30*operation.retry['max_attempts']},'evidence_requirements':['diagnostic']})
# Validate business authorization before creating an executable task.
self.authorization.issue_principal(request['grant_id'],ttl_seconds=60,tenant_id=identity.tenant_id,
subject_id=identity.subject_id,task_id=task_id,**{key:request[key] for key in ('mission_id','operation','component_id','environment')})
self._fresh_write()
try:
with self.store._transaction():
task=self.store.create_task_with_event(plan,idempotency_key=key)
self.store.conn.execute('INSERT OR IGNORE INTO baseline_task_runtime(task_id,ready_at,effect_id) VALUES(?,?,?)',
(task_id,time.time(),effect_id))
self.lineage.link_task(identity.tenant_id,identity.subject_id,request['mission_id'],request['grant_id'],task_id)
return self.get_task(identity,task_id)
except TaskPlaneHTTPError as exc:
raise ServiceError('TASK_CONFLICT') from exc
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('TASK_STORAGE_UNAVAILABLE',503) from exc
def get_task(self,identity,task_id):
self._user(identity)
task=self._task(identity,task_id)
with self.store._transaction():
row=self.store.conn.execute('SELECT * FROM baseline_task_runtime WHERE task_id=?',(task_id,)).fetchone()
result=task.to_dict()
if row:
result.update({'ready_at':row['ready_at'],'blocker_code':row['blocker_code'],'effect_id':row['effect_id'],'job_id':row['job_id']})
if row['job_id']:
job=self.jobs.store.read(row['job_id'])
result['job_state']=job['state']
result['side_effect_state']=job['side_effect_state']
envelope=json.loads(job['receipt_json']) if job.get('receipt_json') else None
if self._receipt_valid(task,envelope): result['receipt']=envelope['payload']
result['remaining_budget']=self.store.get_task_budget(task_id).to_dict()
return result
def explain(self,identity,mission_id):
self._user(identity)
result=self.lineage.explain(identity.tenant_id,identity.subject_id,mission_id)
result['tasks_runtime']=[self.get_task(identity,t.task_id) for t in self.store.list_tasks()
if t.task_plan.metadata.get('baseline',{}).get('mission_id')==mission_id and
t.task_plan.issuer.get('tenant_id')==identity.tenant_id and t.task_plan.issuer.get('subject_id')==identity.subject_id]
return result
def register_worker(self,identity):
profile=self._worker(identity)
try:
existing=self.store.get_worker(profile.worker_id)
except TaskPlaneHTTPError:
existing=None
now=time.time()
worker=WorkerRegistration.from_dict({'worker_id':profile.worker_id,'worker_type':'baseline-operation-worker',
'identity':profile.identity,'capabilities':list(profile.capabilities),'trust_zone':'governed-host',
'max_concurrency':profile.max_concurrency,'heartbeat_interval_seconds':profile.heartbeat_seconds,
'supported_protocols':['aios-baseline-worker-v1'],'last_heartbeat':now})
self.store.register_worker(worker,actor_identity=profile.identity,is_admin=False,
expected_revision=existing.registration_revision if existing else None)
with self.store._transaction():
self.store.conn.execute('INSERT INTO baseline_worker_policy VALUES(?,?) ON CONFLICT(worker_id) DO UPDATE SET policy_json=excluded.policy_json',
(profile.worker_id,canonical(profile.__dict__).decode()))
return {'worker_id':profile.worker_id,'registered':True,'fallback_executor':profile.fallback_executor}
def heartbeat(self,identity):
profile=self._worker(identity)
worker=self.store.get_worker(profile.worker_id)
self.store.heartbeat_worker(profile.worker_id,actor_identity=profile.identity,expected_revision=worker.registration_revision)
return {'worker_id':profile.worker_id,'alive':True}
def claim_next(self,identity):
profile=self._worker(identity)
worker=self.store.get_worker(profile.worker_id)
if worker.disabled: raise ServiceError('WORKER_DISABLED',403)
self._fresh_write()
for task in sorted(self.store.list_tasks(),key=lambda t:(t.created_at,t.task_id)):
meta=task.task_plan.metadata.get('baseline',{})
if not meta or task.status not in {'CREATED','QUEUED','FAILED_RECOVERABLE'}: continue
with self.store._transaction():
row=self.store.conn.execute('SELECT ready_at FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
if not row: continue
now=time.time()
task_scope={**meta,'ready_at':row['ready_at']}
if not eligible(profile,task_scope,now=now,last_heartbeat=worker.last_heartbeat): continue
# Priority is configuration on ordinary eligible workers; no special main actor.
candidates=[]
for registered in self.store.list_workers():
peer=self.worker_profiles.get(registered.worker_id)
if not peer or registered.disabled or not eligible(peer,task_scope,now=now,last_heartbeat=registered.last_heartbeat): continue
with self.store._transaction():
active=self.store.conn.execute("SELECT count(*) FROM claims WHERE worker_id=? AND status='ACTIVE' AND lease_until>?",(peer.worker_id,now)).fetchone()[0]
if active<peer.max_concurrency: candidates.append(peer)
if not candidates or sorted(candidates,key=lambda p:(-p.priority,p.worker_id))[0].worker_id!=profile.worker_id: continue
try:
self._principal(task)
with self.store._transaction():
self.store.consume_budget(task.task_id,Budget(attempts=1,time_seconds=30))
claim=self.store.claim_task(task.task_id,profile.worker_id,profile.identity,lease_seconds=30,
expected_revision=task.revision,operation_digest=task.task_plan.operation_digest())
except TaskPlaneHTTPError:
budget=self.store.get_task_budget(task.task_id)
if budget.attempts==0 or budget.time_seconds==0:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',('TASK_BUDGET_EXHAUSTED',task.task_id))
continue
except AuthorizationError:
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',('AUTHORIZATION_REQUIRED',task.task_id))
continue
return {'task_id':task.task_id,'claim_id':claim.claim_id,'generation':claim.attempt,
'lease_until':claim.lease_until,'operation':meta['operation']}
return None
def renew(self,identity,task_id,claim_id):
profile=self._worker(identity)
claim=self.store.get_claim(claim_id)
if claim.task_id!=task_id or claim.worker_id!=profile.worker_id or claim.worker_identity!=profile.identity:
raise ServiceError('CLAIM_IDENTITY_MISMATCH',403)
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
if meta.get('tenant_id') not in profile.tenant_scope or meta.get('subject_id') not in profile.subject_scope or task.status=='CANCELLED':
raise ServiceError('CLAIM_SCOPE_OR_STATE_DENIED',403)
self._principal(task)
self.store.heartbeat_claim(task_id,claim_id,30)
return {'task_id':task_id,'claim_id':claim_id,'renewed':True}
def _advance(self,job_id,state,**kwargs):
row=self.jobs.store.read(job_id)
return self.jobs.advance(job_id,state,expected_revision=int(row['revision']),**kwargs)
@contextmanager
def _execution_slot(self,deadline):
remaining=deadline-time.time()
if remaining<=0 or not self._execute_lock.acquire(timeout=remaining):
raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
try:
if time.time()>=deadline: raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
yield
finally: self._execute_lock.release()
def execute_claim(self,identity,task_id,claim_id,*,deadline=None):
profile=self._worker(identity)
deadline=time.time()+30 if deadline is None else deadline
with self._execution_slot(deadline):
self._fresh_write()
task=self.store.get_task(task_id)
meta=task.task_plan.metadata.get('baseline',{})
claim=self.store.get_claim(claim_id)
if claim.task_id!=task_id or claim.worker_id!=profile.worker_id or claim.worker_identity!=profile.identity:
raise ServiceError('CLAIM_IDENTITY_MISMATCH',403)
if meta.get('tenant_id') not in profile.tenant_scope or meta.get('subject_id') not in profile.subject_scope:
raise ServiceError('WORKER_SCOPE_DENIED',403)
user=Identity(meta['tenant_id'],meta['subject_id'],'user')
if task.status=='COMPLETED': return self.get_task(user,task_id)
if claim.status!='ACTIVE' or claim.lease_until<=time.time() or task.status=='CANCELLED':
raise ServiceError('CLAIM_EXPIRED_OR_CANCELLED')
principal=self._principal(task)
self.authorization.authorize(principal,**self._scope(task))
operation=self.operations.get(meta['operation'])
if operation.handler_ref!='host.service_status' or operation.effect_class!='READ_ONLY':
raise ServiceError('HANDLER_NOT_REGISTERED',503)
self.authorization.assert_effect_bounds(principal,operation,**self._scope(task))
request={'schema':'aios.operation-job.v1','tenant_id':meta['tenant_id'],'subject_id':meta['subject_id'],
'mission_id':meta['mission_id'],'grant_id':meta['grant_id'],'task_id':task_id,'effect_id':meta['effect_id'],
'operation':meta['operation'],'operation_version':str(operation.version),'resource_kind':'systemd-service',
'resource_id':digest({'tenant':meta['tenant_id'],'environment':meta['environment'],'component':meta['component_id']}),'input_digest':digest(meta['inputs'])}
try:
job=self.jobs.submit(request)
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET job_id=? WHERE task_id=?',(job.job_id,task_id))
self.lineage.link_task(meta['tenant_id'],meta['subject_id'],meta['mission_id'],meta['grant_id'],task_id)
self.lineage.link_effect(meta['tenant_id'],meta['subject_id'],meta['mission_id'],task_id,meta['effect_id'],job.job_id)
row=self.jobs.store.read(job.job_id)
if row['state']=='COMPLETED':
self._close_task(task,claim,json.loads(row['receipt_json']))
return self.get_task(user,task_id)
if row['state'] not in {'REQUESTED','AUTHORIZED'}:
raise ServiceError('JOB_RECONCILIATION_REQUIRED')
self.jobs.assert_effect_admission(exclude_job_id=job.job_id)
self.authorization.reserve_effect(meta['grant_id'],meta['effect_id'],principal,**self._scope(task))
if row['state']=='REQUESTED': self._advance(job.job_id,JobState.AUTHORIZED,side_effect_state='AUTHORIZED')
row=self.jobs.store.read(job.job_id)
self.jobs.lease_resource_job(job.job_id,lease_id=claim_id,resource_kind=request['resource_kind'],
resource_id=request['resource_id'],generation=None,
lease_expires_at=datetime.fromtimestamp(min(claim.lease_until,time.time()+30,deadline),timezone.utc).isoformat(),
expected_revision=int(row['revision']))
effect_lease=self.jobs.get_execution_lease(job.job_id)
fence_generation=int(effect_lease['generation'])
self._advance(job.job_id,JobState.EXECUTING,side_effect_state='IN_PROGRESS')
# Last authorization/claim check directly before the registered host boundary.
self.authorization.authorize(principal,**self._scope(task))
if self.store.get_task(task_id).status=='CANCELLED' or self.store.get_claim(claim_id).lease_until<=time.time():
raise ServiceError('CLAIM_EXPIRED_OR_CANCELLED')
self.jobs.assert_execution_lease(job.job_id,generation=fence_generation,lease_id=claim_id)
if time.time()>=deadline: raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
observation=self.host.status(meta['component_id'],meta['environment'],tenant_id=meta['tenant_id'],deadline=deadline)
if time.time()>=deadline or self.store.get_claim(claim_id).lease_until<=time.time():
raise ServiceError('EXECUTION_DEADLINE_EXPIRED',408)
envelope=self.signer.sign({'task_id':task_id,'job_id':job.job_id,'effect_id':meta['effect_id'],
'tenant_id':meta['tenant_id'],'subject_id':meta['subject_id'],'mission_id':meta['mission_id'],
'grant_id':meta['grant_id'],'claim_id':claim_id,'generation':fence_generation,
'operation':meta['operation'],'operation_version':str(operation.version),
'input_digest':request['input_digest'],'resource_kind':request['resource_kind'],
'resource_id':request['resource_id'],'observation':observation,'verified':True})
self._advance(job.job_id,JobState.LOCAL_VERIFIED,side_effect_state='READBACK_VERIFIED',receipt=envelope)
self._advance(job.job_id,JobState.COMPLETED,side_effect_state='READBACK_VERIFIED',receipt=envelope)
self._close_task(task,claim,envelope)
return self.get_task(user,task_id)
except (sqlite3.Error,OSError) as exc:
self.storage_blocked=True
raise ServiceError('STORAGE_RECOVERY_REQUIRED',503) from exc
except HostError as exc:
try:
self._advance(job.job_id,JobState.RECONCILIATION_REQUIRED,side_effect_state='UNKNOWN')
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=? WHERE task_id=?',(str(exc),task_id))
except (sqlite3.Error,OSError,JobError): self.storage_blocked=True
raise ServiceError(str(exc),503) from exc
except JobError as exc:
if 'UNAVAILABLE' in str(exc) or 'STORAGE' in str(exc): self.storage_blocked=True
raise ServiceError(str(exc),503 if self.storage_blocked else 409) from exc
def _receipt_valid(self,task,envelope):
meta=task.task_plan.metadata['baseline']
if not self.signer.verify(envelope): return False
payload=envelope['payload']
with self.store._transaction():
runtime=self.store.conn.execute('SELECT job_id FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
if not runtime or not runtime['job_id']: return False
expected={**{k:meta[k] for k in ('effect_id','tenant_id','subject_id','mission_id','grant_id','operation')},
'task_id':task.task_id,'job_id':runtime['job_id'],'operation_version':str(meta['operation_version']),
'input_digest':digest(meta['inputs']),'resource_kind':'systemd-service',
'resource_id':digest({'tenant':meta['tenant_id'],'environment':meta['environment'],'component':meta['component_id']})}
if any(payload.get(k)!=v for k,v in expected.items()) or payload.get('verified') is not True: return False
try:
lease=self.jobs.get_execution_lease(runtime['job_id'])
source_claim=self.store.get_claim(payload.get('claim_id'))
if payload.get('claim_id')!=lease['lease_id'] or payload.get('generation')!=lease['generation']: return False
if source_claim.task_id!=task.task_id or source_claim.operation_digest!=task.task_plan.operation_digest(): return False
except (JobError,TaskPlaneHTTPError,TypeError): return False
observation=payload.get('observation')
if type(observation) is not dict: return False
for k in ('operation','tenant_id','component_id','environment'):
if observation.get(k)!=meta[k]: return False
binding=meta['tenant_id']+':'+meta['environment']+':'+meta['component_id']
if observation.get('unit')!=self.host.bindings.get(binding) or observation.get('source')!='systemd.show': return False
return True
def _close_task(self,task,claim,envelope):
if not self._receipt_valid(task,envelope): raise ServiceError('RESULT_ATTESTATION_INVALID',403)
current=self.store.get_task(task.task_id)
if current.status=='CANCELLED': raise ServiceError('TASK_CANCELLED')
if current.status=='CLAIMED': self.store.update_task_status(task.task_id,current.revision,'RUNNING')
self.store.complete_claim(task.task_id,claim.claim_id,'COMPLETED',envelope,
evidence_entries=[('diagnostic',envelope,claim.worker_identity)])
def cancel_task(self,identity,task_id):
self._user(identity)
task=self._task(identity,task_id)
self.store.cancel_task_with_event(task_id,task.revision,'cancelled by authenticated subject')
return self.get_task(identity,task_id)
def _verify_probe(self,probe):
try:
self.signer.public_key.verify(base64.b64decode(probe.signature,validate=True),canonical(probe.digest_payload()))
return True
except Exception:
return False
def recover(self):
"""Reconcile expired baseline claims from durable signed receipts; never fake a result."""
with self._execute_lock:
return self._recover_locked()
def _recover_locked(self):
recovered=[]
with self.store._transaction():
rows=self.store.conn.execute("SELECT claim_id FROM claims WHERE status='ACTIVE' AND lease_until<?",(time.time(),)).fetchall()
for row in rows:
claim=self.store.get_claim(row['claim_id'])
task=self.store.get_task(claim.task_id)
meta=task.task_plan.metadata.get('baseline')
if not meta: continue
with self.store._transaction():
rt=self.store.conn.execute('SELECT job_id FROM baseline_task_runtime WHERE task_id=?',(task.task_id,)).fetchone()
outcome=ProbeOutcome.NOT_APPLIED
ref='read-only-no-persistent-effect'
if rt and rt['job_id']:
job=self.jobs.store.read(rt['job_id'])
env=json.loads(job['receipt_json']) if job.get('receipt_json') else None
if job['state']=='COMPLETED':
if self._receipt_valid(task,env):
outcome=ProbeOutcome.APPLIED_VERIFIED
ref=digest(env)
else: outcome=ProbeOutcome.INDETERMINATE
elif self.operations.get(meta['operation']).effect_class!='READ_ONLY':
outcome=ProbeOutcome.INDETERMINATE
now=time.time()
probe=ProbeEvidence.create(task_id=task.task_id,claim_id=claim.claim_id,worker_id=claim.worker_id,
worker_identity=claim.worker_identity,operation_digest=claim.operation_digest,outcome=outcome,
observed_at=now,expires_at=now+30,probe_executor_identity='baseline-reconciler',attestation_ref=ref)
from dataclasses import replace
probe=replace(probe,signature=base64.b64encode(self.signer.private_key.sign(canonical(probe.digest_payload()))).decode())
self.store.recover_claim_with_probe_and_event(task.task_id,claim.claim_id,probe)
if outcome==ProbeOutcome.NOT_APPLIED and rt and rt['job_id']:
job=self.jobs.store.read(rt['job_id'])
if job['state']!='COMPLETED':
self._advance(rt['job_id'],JobState.RECONCILIATION_REQUIRED,side_effect_state='UNKNOWN')
self.jobs.reconcile(rt['job_id'],lambda _:{'state':'REQUESTED','side_effect_state':'READ_ONLY_SAFE_TO_RETRY'})
recovered.append({'task_id':task.task_id,'outcome':outcome.value})
# A successful writable transaction permits recovery processing, not blind effect replay.
with self.store._transaction():
self.store.conn.execute('UPDATE baseline_task_runtime SET blocker_code=blocker_code WHERE 0')
try:
self.jobs.assert_effect_admission()
self.storage_blocked=False
except JobError:
self.storage_blocked=True
return recovered
@@ -0,0 +1,513 @@
"""Authenticated loopback HTTP transport for the Baseline 1.1 service."""
from __future__ import annotations
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import ipaddress
import json
import math
import re
import time
import threading
from typing import Any, Mapping
from urllib.error import HTTPError, URLError
from urllib.parse import quote, unquote, urlsplit
from urllib.request import HTTPRedirectHandler, ProxyHandler, Request, build_opener
from .service import Identity, ServiceError
MAX_REQUEST_BYTES = 65_536
MAX_RESPONSE_BYTES = 1_048_576
MAX_DEADLINE_SECONDS = 30
_IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}")
_IDENTITY_FIELDS = frozenset({"tenant_id", "subject_id", "role", "worker_id", "identity"})
class TransportError(RuntimeError):
def __init__(self, code: str, status: int | None = None):
self.code = code
self.status = status
super().__init__(code)
class _RequestError(RuntimeError):
def __init__(self, code: str, status: int):
self.code = code
self.status = status
super().__init__(code)
def _safe_identifier(value: Any, code: str = "HTTP_PATH_INVALID") -> str:
if not isinstance(value, str) or _IDENTIFIER.fullmatch(value) is None:
raise _RequestError(code, 400)
return value
def _has_identity_field(value: Any) -> bool:
if isinstance(value, Mapping):
if any(key in _IDENTITY_FIELDS for key in value):
return True
return any(_has_identity_field(item) for item in value.values())
if isinstance(value, list):
return any(_has_identity_field(item) for item in value)
return False
def _loopback_host(host: str, code: str) -> str:
if not isinstance(host, str):
raise TransportError(code)
try:
address = ipaddress.ip_address(host)
except ValueError as exc:
raise TransportError(code) from exc
if not address.is_loopback or address.version != 4:
raise TransportError(code)
return str(address)
class _BaselineHTTPServer(ThreadingHTTPServer):
daemon_threads = True
allow_reuse_address = True
def __init__(self, address, handler, service, identities):
self.baseline_service = service
self.trusted_identities = identities
self.request_slots=threading.BoundedSemaphore(16)
super().__init__(address, handler)
def get_request(self):
connection, address = super().get_request()
connection.settimeout(MAX_DEADLINE_SECONDS)
return connection, address
def process_request(self,request,client_address):
if not self.request_slots.acquire(blocking=False):
try:
body=b'{"error":{"code":"HTTP_CAPACITY_EXCEEDED"}}'
request.sendall(b'HTTP/1.1 503 Service Unavailable\r\nContent-Type: application/json\r\nConnection: close\r\nContent-Length: '+str(len(body)).encode()+b'\r\n\r\n'+body)
except OSError: pass
self.shutdown_request(request)
return
try: super().process_request(request,client_address)
except Exception:
self.request_slots.release()
raise
def process_request_thread(self,request,client_address):
try: super().process_request_thread(request,client_address)
finally: self.request_slots.release()
class _Handler(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.1"
server_version = "AIOSBaselineHTTP/1"
sys_version = ""
def log_message(self, _format, *_args):
# Request paths can contain tenant-owned identifiers; do not emit them or
# bearer-related failure details through the default stderr logger.
return
def send_error(self, code, message=None, explain=None):
# BaseHTTPRequestHandler otherwise emits an HTML page which may include
# parser details. Keep pre-dispatch protocol failures in the same closed
# JSON error format.
status = code if isinstance(code, int) and 400 <= code <= 599 else 400
self._send(status, {"error": {"code": "HTTP_PROTOCOL_ERROR"}})
def _send(self, status: int, value: Any) -> None:
try:
body = json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError):
status = 500
body = b'{"error":{"code":"HTTP_RESPONSE_INVALID"}}'
if len(body) > MAX_RESPONSE_BYTES:
status = 500
body = b'{"error":{"code":"HTTP_RESPONSE_TOO_LARGE"}}'
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("Connection", "close")
self.end_headers()
try:
self.wfile.write(body)
except OSError:
# A peer can disconnect or let its deadline expire while a response
# is being written. Do not let the server print a handler traceback.
pass
self.close_connection = True
def _error(self, error: _RequestError) -> None:
self._send(error.status, {"error": {"code": error.code}})
def _deadline(self) -> float:
values = self.headers.get_all("X-AIOS-Deadline", [])
if len(values) != 1:
raise _RequestError("HTTP_DEADLINE_REQUIRED", 400)
try:
deadline = float(values[0])
except (TypeError, ValueError) as exc:
raise _RequestError("HTTP_DEADLINE_INVALID", 400) from exc
now = time.time()
if not math.isfinite(deadline) or deadline > now + MAX_DEADLINE_SECONDS:
raise _RequestError("HTTP_DEADLINE_INVALID", 400)
if deadline <= now:
raise _RequestError("HTTP_DEADLINE_EXPIRED", 408)
self.connection.settimeout(deadline - now)
return deadline
def _identity(self) -> Identity:
values = self.headers.get_all("Authorization", [])
if len(values) != 1 or not values[0].startswith("Bearer "):
raise _RequestError("HTTP_UNAUTHORIZED", 401)
token = values[0][7:]
if not token or any(ord(char) < 33 or ord(char) > 126 for char in token):
raise _RequestError("HTTP_UNAUTHORIZED", 401)
identity = self.server.trusted_identities.get(token)
if identity is None:
raise _RequestError("HTTP_UNAUTHORIZED", 401)
return identity
def _json_body(self) -> dict:
if self.headers.get("Transfer-Encoding") is not None:
raise _RequestError("HTTP_TRANSFER_ENCODING_FORBIDDEN", 400)
content_types = self.headers.get_all("Content-Type", [])
if content_types != ["application/json"]:
raise _RequestError("HTTP_CONTENT_TYPE_INVALID", 415)
lengths = self.headers.get_all("Content-Length", [])
if len(lengths) != 1 or not lengths[0].isdigit():
raise _RequestError("HTTP_CONTENT_LENGTH_INVALID", 411)
length = int(lengths[0])
if length > MAX_REQUEST_BYTES:
raise _RequestError("HTTP_BODY_TOO_LARGE", 413)
if length <= 0:
raise _RequestError("HTTP_BODY_REQUIRED", 400)
raw = self.rfile.read(length)
if len(raw) != length:
raise _RequestError("HTTP_BODY_TRUNCATED", 400)
try:
def unique_object(pairs):
result = {}
for key, item in pairs:
if key in result:
raise _RequestError("HTTP_JSON_DUPLICATE_KEY", 400)
result[key] = item
return result
def reject_constant(_value):
raise _RequestError("HTTP_JSON_NON_FINITE", 400)
value = json.loads(
raw.decode("utf-8"), object_pairs_hook=unique_object,
parse_constant=reject_constant,
)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise _RequestError("HTTP_JSON_INVALID", 400) from exc
if type(value) is not dict:
raise _RequestError("HTTP_JSON_OBJECT_REQUIRED", 400)
if _has_identity_field(value):
raise _RequestError("HTTP_IDENTITY_FIELD_FORBIDDEN", 400)
return value
def _path(self) -> list[str]:
parsed = urlsplit(self.path)
if parsed.query or parsed.fragment or not parsed.path.startswith("/"):
raise _RequestError("HTTP_PATH_INVALID", 400)
raw_segments = parsed.path.split("/")[1:]
if any(not segment for segment in raw_segments):
raise _RequestError("HTTP_NOT_FOUND", 404)
segments = [unquote(segment) for segment in raw_segments]
if any("/" in segment or "\\" in segment for segment in segments):
raise _RequestError("HTTP_PATH_INVALID", 400)
return segments
@staticmethod
def _exact(body: Mapping[str, Any], fields: set[str]) -> None:
if set(body) != fields:
raise _RequestError("HTTP_BODY_SCHEMA_INVALID", 400)
def _dispatch(self, method: str, identity: Identity, segments: list[str], body: dict | None):
service = self.server.baseline_service
if method == "POST" and segments == ["v1", "goals", "submit"]:
self._exact(body, {"request", "idempotency_key"})
if type(body["request"]) is not dict:
raise _RequestError("HTTP_BODY_SCHEMA_INVALID", 400)
return 201, service.submit_goal(
identity, body["request"], idempotency_key=body["idempotency_key"]
)
if method == "GET" and len(segments) == 3 and segments[:2] == ["v1", "tasks"]:
return 200, service.get_task(identity, _safe_identifier(segments[2]))
if (
method == "POST" and len(segments) == 4
and segments[:2] == ["v1", "tasks"] and segments[3] == "cancel"
):
self._exact(body, set())
return 200, service.cancel_task(identity, _safe_identifier(segments[2]))
if (
method == "GET" and len(segments) == 4
and segments[:2] == ["v1", "missions"] and segments[3] == "explain"
):
return 200, service.explain(identity, _safe_identifier(segments[2]))
if method == "POST" and segments == ["v1", "workers", "register"]:
self._exact(body, set())
return 200, service.register_worker(identity)
if method == "POST" and segments == ["v1", "workers", "heartbeat"]:
self._exact(body, set())
return 200, service.heartbeat(identity)
if method == "POST" and segments == ["v1", "workers", "claim"]:
self._exact(body, set())
return 200, service.claim_next(identity)
if method == "POST" and segments == ["v1", "workers", "renew"]:
self._exact(body, {"task_id", "claim_id"})
return 200, service.renew(
identity, _safe_identifier(body["task_id"]), _safe_identifier(body["claim_id"])
)
if method == "POST" and segments == ["v1", "workers", "execute"]:
self._exact(body, {"task_id", "claim_id"})
return 200, service.execute_claim(
identity, _safe_identifier(body["task_id"]), _safe_identifier(body["claim_id"]),deadline=self.request_deadline
)
raise _RequestError("HTTP_NOT_FOUND", 404)
def _handle(self, method: str) -> None:
try:
self.request_deadline=self._deadline()
identity = self._identity()
segments = self._path()
body = self._json_body() if method == "POST" else None
if method == "GET":
lengths = self.headers.get_all("Content-Length", [])
if self.headers.get("Transfer-Encoding") is not None:
raise _RequestError("HTTP_GET_BODY_FORBIDDEN", 400)
if len(lengths) > 1 or (lengths and (not lengths[0].isdigit() or int(lengths[0]) != 0)):
raise _RequestError("HTTP_GET_BODY_FORBIDDEN", 400)
status, result = self._dispatch(method, identity, segments, body)
self._send(status, result)
except _RequestError as exc:
self._error(exc)
except ServiceError as exc:
status = exc.status if isinstance(exc.status, int) and 400 <= exc.status <= 599 else 500
self._send(status, {"error": {"code": exc.code}})
except TimeoutError:
self._send(408, {"error": {"code": "HTTP_DEADLINE_EXPIRED"}})
except Exception:
# No exception string, traceback, request headers, or token crosses
# this boundary.
self._send(500, {"error": {"code": "HTTP_INTERNAL_ERROR"}})
def do_GET(self):
self._handle("GET")
def do_POST(self):
self._handle("POST")
def _method_not_allowed(self):
try:
self._deadline()
self._identity()
self._send(405, {"error": {"code": "HTTP_METHOD_NOT_ALLOWED"}})
except _RequestError as exc:
self._error(exc)
do_PUT = _method_not_allowed
do_PATCH = _method_not_allowed
do_DELETE = _method_not_allowed
do_OPTIONS = _method_not_allowed
do_HEAD = _method_not_allowed
do_TRACE = _method_not_allowed
do_CONNECT = _method_not_allowed
def serve(
service: Any,
tokens: Mapping[str, Identity],
host: str = "127.0.0.1",
port: int = 0,
) -> ThreadingHTTPServer:
"""Build an unstarted real HTTP server bound only to an IPv4 loopback."""
host = _loopback_host(host, "HTTP_LOOPBACK_REQUIRED")
if type(port) is not int or not 0 <= port <= 65_535:
raise TransportError("HTTP_PORT_INVALID")
if not isinstance(tokens, Mapping) or not tokens:
raise TransportError("HTTP_TRUSTED_IDENTITIES_REQUIRED")
identities: dict[str, Identity] = {}
for token, identity in tokens.items():
if (
not isinstance(token, str) or not 32<=len(token)<=512
or any(ord(char) < 33 or ord(char) > 126 for char in token)
or not isinstance(identity, Identity)
):
raise TransportError("HTTP_TRUSTED_IDENTITY_INVALID")
identities[token] = identity
return _BaselineHTTPServer((host, port), _Handler, service, identities)
class _NoRedirect(HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
class HttpTransport:
"""Small urllib client for the loopback API; redirects and proxies are off."""
def __init__(
self,
base_url: str,
token: str,
*,
timeout_seconds: float = 15,
max_response_bytes: int = MAX_RESPONSE_BYTES,
):
try:
parsed = urlsplit(base_url)
parsed_port = parsed.port
except ValueError as exc:
raise TransportError("TRANSPORT_BASE_URL_INVALID") from exc
if (
parsed.scheme != "http" or parsed.username is not None or parsed.password is not None
or parsed.query or parsed.fragment or parsed.path not in {"", "/"}
or parsed.hostname is None or parsed_port is None
):
raise TransportError("TRANSPORT_BASE_URL_INVALID")
_loopback_host(parsed.hostname, "TRANSPORT_LOOPBACK_REQUIRED")
if not isinstance(token, str) or not 32<=len(token)<=512 or any(
ord(char) < 33 or ord(char) > 126 for char in token
):
raise TransportError("TRANSPORT_TOKEN_INVALID")
if (
isinstance(timeout_seconds, bool)
or not isinstance(timeout_seconds, (int, float))
or not math.isfinite(float(timeout_seconds))
or not 0.1 <= timeout_seconds <= MAX_DEADLINE_SECONDS
):
raise TransportError("TRANSPORT_TIMEOUT_INVALID")
if type(max_response_bytes) is not int or not 1 <= max_response_bytes <= 4 * 1024 * 1024:
raise TransportError("TRANSPORT_RESPONSE_LIMIT_INVALID")
self.base_url = f"http://{parsed.hostname}:{parsed_port}"
self.token = token
self.timeout_seconds = float(timeout_seconds)
self.max_response_bytes = max_response_bytes
self._opener = build_opener(ProxyHandler({}), _NoRedirect())
def _decode(self, response) -> Any:
content_type = response.headers.get("Content-Type", "").split(";", 1)[0]
if content_type != "application/json":
raise TransportError("TRANSPORT_RESPONSE_TYPE_INVALID")
declared = response.headers.get("Content-Length")
if declared is None or not declared.isdigit() or int(declared) > self.max_response_bytes:
raise TransportError("TRANSPORT_RESPONSE_TOO_LARGE")
raw = response.read(self.max_response_bytes + 1)
if len(raw) > self.max_response_bytes or len(raw) != int(declared):
raise TransportError("TRANSPORT_RESPONSE_SIZE_INVALID")
try:
def unique_object(pairs):
result = {}
for key, item in pairs:
if key in result:
raise ValueError("duplicate")
result[key] = item
return result
return json.loads(
raw.decode("utf-8"), object_pairs_hook=unique_object,
parse_constant=lambda _value: (_ for _ in ()).throw(ValueError("non-finite")),
)
except (UnicodeDecodeError, json.JSONDecodeError, ValueError) as exc:
raise TransportError("TRANSPORT_RESPONSE_JSON_INVALID") from exc
def _request(self, method: str, path: str, body: Mapping[str, Any] | None = None) -> Any:
if not isinstance(path, str) or not path.startswith("/") or "?" in path or "#" in path:
raise TransportError("TRANSPORT_PATH_INVALID")
data = None
headers = {
"Authorization": "Bearer " + self.token,
"Accept": "application/json",
"X-AIOS-Deadline": str(time.time() + self.timeout_seconds),
}
if method == "POST":
if not isinstance(body, Mapping):
raise TransportError("TRANSPORT_BODY_INVALID")
try:
data = json.dumps(
body, sort_keys=True, separators=(",", ":"), ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise TransportError("TRANSPORT_BODY_INVALID") from exc
if not data or len(data) > MAX_REQUEST_BYTES:
raise TransportError("TRANSPORT_BODY_TOO_LARGE")
headers["Content-Type"] = "application/json"
request = Request(self.base_url + path, data=data, headers=headers, method=method)
try:
with self._opener.open(request, timeout=self.timeout_seconds) as response:
return self._decode(response)
except HTTPError as exc:
if 300 <= exc.code < 400:
exc.close()
raise TransportError("TRANSPORT_REDIRECT_REJECTED", exc.code) from None
try:
payload = self._decode(exc)
except TransportError:
payload = None
except (TimeoutError, OSError):
raise TransportError("TRANSPORT_UNAVAILABLE") from None
finally:
exc.close()
code = None
if isinstance(payload, Mapping) and isinstance(payload.get("error"), Mapping):
code = payload["error"].get("code")
if not isinstance(code, str) or _IDENTIFIER.fullmatch(code) is None:
code = "TRANSPORT_HTTP_ERROR"
raise TransportError(code, exc.code) from None
except (URLError, TimeoutError, OSError):
raise TransportError("TRANSPORT_UNAVAILABLE") from None
def submit_goal(self, request: Mapping[str, Any], *, idempotency_key: str) -> dict:
return self._request(
"POST", "/v1/goals/submit",
{"request": request, "idempotency_key": idempotency_key},
)
def get_task(self, task_id: str) -> dict:
return self._request("GET", "/v1/tasks/" + quote(task_id, safe=""))
def cancel_task(self, task_id: str) -> dict:
return self._request("POST", "/v1/tasks/" + quote(task_id, safe="") + "/cancel", {})
def explain(self, mission_id: str) -> dict:
return self._request("GET", "/v1/missions/" + quote(mission_id, safe="") + "/explain")
def register_worker(self) -> dict:
return self._request("POST", "/v1/workers/register", {})
def heartbeat(self) -> dict:
return self._request("POST", "/v1/workers/heartbeat", {})
def claim_next(self) -> dict | None:
return self._request("POST", "/v1/workers/claim", {})
def renew(self, task_id: str, claim_id: str) -> dict:
return self._request(
"POST", "/v1/workers/renew", {"task_id": task_id, "claim_id": claim_id}
)
def execute_claim(self, task_id: str, claim_id: str) -> dict:
return self._request(
"POST", "/v1/workers/execute", {"task_id": task_id, "claim_id": claim_id}
)
# Goal-oriented names used by thin clients.
submit = submit_goal
query = get_task
cancel = cancel_task
Transport = HttpTransport
__all__ = ["HttpTransport", "Transport", "TransportError", "serve"]
@@ -0,0 +1,204 @@
"""Verification-only Ed25519 boundaries for release and acceptance evidence."""
from __future__ import annotations
import base64
import json
import re
from types import MappingProxyType
from typing import Any, Callable, Mapping
from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from .lineage import VerifiedAcceptanceEvidence, canonical_artifact
from .release import VerifiedReceipt
RELEASE_RECEIPT_DOMAIN = b"aios.release-operation-receipt.v1"
ACCEPTANCE_EVIDENCE_DOMAIN = b"aios.release-acceptance-evidence.v1"
_ENVELOPE_FIELDS = {"schema", "key_id", "payload", "signature"}
_RECEIPT_FIELDS = {
"receipt_id", "tenant_id", "subject_id", "component_id", "task_id",
"source_task_id", "source_effect_id", "grant_id", "job_id", "effect_id", "operation_id",
"deployment_id", "release_id", "artifact", "outcome", "observed", "observed_at",
}
_ACCEPTANCE_FIELDS = {
"acceptance_id", "tenant_id", "subject_id", "component_id", "deployment_id", "release_id",
"artifact", "accepted", "evidence_id", "observed", "observed_at",
}
_B64URL = re.compile(r"[A-Za-z0-9_-]+")
class SignatureVerificationError(ValueError):
"""A configured trust root did not authenticate an exact evidence payload."""
def _canonical(value: Any) -> bytes:
try:
return json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":"),
allow_nan=False,
).encode("utf-8")
except (TypeError, ValueError) as exc:
raise SignatureVerificationError("signed payload is not canonical JSON") from exc
def _decode_urlsafe(value: Any, name: str, expected_size: int) -> bytes:
if not isinstance(value, str) or not value or not _B64URL.fullmatch(value) or "=" in value:
raise SignatureVerificationError(f"invalid {name} encoding")
try:
raw = base64.urlsafe_b64decode(value + "=" * (-len(value) % 4))
except (ValueError, TypeError) as exc:
raise SignatureVerificationError(f"invalid {name} encoding") from exc
if len(raw) != expected_size:
raise SignatureVerificationError(f"invalid {name} size")
return raw
def _public_key(value: Any) -> Ed25519PublicKey:
if isinstance(value, Ed25519PublicKey):
return value
if isinstance(value, bytes):
raw = value
elif isinstance(value, str):
raw = _decode_urlsafe(value, "trusted public key", 32)
else:
raise ValueError("trusted public key must be Ed25519PublicKey, raw bytes, or base64url")
if len(raw) != 32:
raise ValueError("trusted Ed25519 public key must be 32 bytes")
return Ed25519PublicKey.from_public_bytes(raw)
def _trusted_keys(values: Mapping[str, Any]) -> Mapping[str, Ed25519PublicKey]:
if not isinstance(values, Mapping) or not values:
raise ValueError("trusted_public_keys must be a non-empty mapping")
keys: dict[str, Ed25519PublicKey] = {}
for key_id, value in values.items():
if not isinstance(key_id, str) or not key_id or len(key_id) > 128:
raise ValueError("invalid trusted key_id")
keys[key_id] = _public_key(value)
return MappingProxyType(keys)
def _verified_payload(
envelope: Any,
keys: Mapping[str, Ed25519PublicKey],
*,
schema: str,
domain: bytes,
payload_fields: set[str],
) -> Mapping[str, Any]:
if not isinstance(envelope, Mapping) or set(envelope) != _ENVELOPE_FIELDS:
raise SignatureVerificationError("signed envelope fields invalid")
if envelope.get("schema") != schema:
raise SignatureVerificationError("signed envelope schema mismatch")
key_id = envelope.get("key_id")
if not isinstance(key_id, str) or key_id not in keys:
raise SignatureVerificationError("untrusted signing key_id")
payload = envelope.get("payload")
if not isinstance(payload, Mapping) or set(payload) != payload_fields:
raise SignatureVerificationError("signed payload fields invalid")
signature = _decode_urlsafe(envelope.get("signature"), "signature", 64)
try:
keys[key_id].verify(signature, domain + b"\x00" + _canonical(dict(payload)))
except InvalidSignature as exc:
raise SignatureVerificationError("Ed25519 signature invalid") from exc
return payload
def _text(payload: Mapping[str, Any], name: str) -> str:
value = payload.get(name)
if not isinstance(value, str) or not value or len(value) > 512:
raise SignatureVerificationError(f"signed {name} invalid")
return value
def _expected(payload: Mapping[str, Any], expected: Mapping[str, Any], fields: tuple[str, ...]) -> None:
for name in fields:
if name not in expected:
raise SignatureVerificationError(f"verifier expected {name} missing")
actual = canonical_artifact(payload[name]) if name == "artifact" else payload[name]
wanted = canonical_artifact(expected[name]) if name == "artifact" else expected[name]
if actual != wanted:
raise SignatureVerificationError(f"signed {name} binding mismatch")
def ed25519_release_receipt_verifier(
trusted_public_keys: Mapping[str, Any],
) -> Callable[[Any, Mapping[str, Any]], VerifiedReceipt]:
"""Build a receipt verifier from local/configured public keys only."""
keys = _trusted_keys(trusted_public_keys)
def verify(envelope: Any, expected: Mapping[str, Any]) -> VerifiedReceipt:
payload = _verified_payload(
envelope, keys,
schema="aios.release-operation-receipt-envelope.v1",
domain=RELEASE_RECEIPT_DOMAIN,
payload_fields=_RECEIPT_FIELDS,
)
_expected(
payload, expected,
("tenant_id", "subject_id", "component_id", "task_id", "source_task_id", "source_effect_id",
"grant_id", "job_id", "effect_id", "operation_id", "deployment_id", "release_id", "artifact"),
)
if payload.get("outcome") != "SUCCEEDED":
raise SignatureVerificationError("signed receipt outcome is not SUCCEEDED")
if not isinstance(payload.get("observed"), Mapping):
raise SignatureVerificationError("signed receipt observation invalid")
return VerifiedReceipt(
receipt_id=_text(payload, "receipt_id"), tenant_id=_text(payload, "tenant_id"),
subject_id=_text(payload, "subject_id"), effect_id=_text(payload, "effect_id"),
component_id=_text(payload,"component_id"), task_id=_text(payload,"task_id"),
source_task_id=_text(payload,"source_task_id"), source_effect_id=_text(payload,"source_effect_id"),
grant_id=_text(payload,"grant_id"), job_id=_text(payload,"job_id"),
operation_id=_text(payload, "operation_id"), deployment_id=_text(payload, "deployment_id"),
release_id=_text(payload, "release_id"), artifact=payload["artifact"],
observed=dict(payload["observed"]), outcome="SUCCEEDED",
observed_at=_text(payload, "observed_at"),
)
return verify
def ed25519_acceptance_evidence_verifier(
trusted_public_keys: Mapping[str, Any],
) -> Callable[[Any, Mapping[str, Any]], VerifiedAcceptanceEvidence]:
"""Build an acceptance verifier in a domain separate from stage receipts."""
keys = _trusted_keys(trusted_public_keys)
def verify(envelope: Any, expected: Mapping[str, Any]) -> VerifiedAcceptanceEvidence:
payload = _verified_payload(
envelope, keys,
schema="aios.release-acceptance-evidence-envelope.v1",
domain=ACCEPTANCE_EVIDENCE_DOMAIN,
payload_fields=_ACCEPTANCE_FIELDS,
)
_expected(
payload, expected,
("acceptance_id", "tenant_id", "subject_id", "component_id", "deployment_id", "release_id", "artifact", "accepted"),
)
if type(payload.get("accepted")) is not bool:
raise SignatureVerificationError("signed accepted decision invalid")
if not isinstance(payload.get("observed"), Mapping):
raise SignatureVerificationError("signed acceptance observation invalid")
return VerifiedAcceptanceEvidence(
acceptance_id=_text(payload, "acceptance_id"), tenant_id=_text(payload, "tenant_id"),
subject_id=_text(payload, "subject_id"), deployment_id=_text(payload, "deployment_id"),
component_id=_text(payload,"component_id"),
release_id=_text(payload, "release_id"), artifact=payload["artifact"],
accepted=payload["accepted"], evidence_id=_text(payload, "evidence_id"),
observed=dict(payload["observed"]), observed_at=_text(payload, "observed_at"),
)
return verify
__all__ = [
"ACCEPTANCE_EVIDENCE_DOMAIN", "RELEASE_RECEIPT_DOMAIN",
"SignatureVerificationError", "ed25519_acceptance_evidence_verifier",
"ed25519_release_receipt_verifier",
]
+121
View File
@@ -0,0 +1,121 @@
"""Thin resident worker loop over the authenticated HTTP transport.
Worker policy, operation handlers, effects, receipts, and completion remain in
the trusted server. This process only maintains its registered transport
identity and asks the service to run an offered claim.
"""
from __future__ import annotations
import math
import threading
import time
from typing import Any
from .transport import TransportError
class WorkerError(RuntimeError):
pass
class OperationWorker:
HEARTBEAT_INTERVAL_SECONDS = 15.0
def __init__(self, transport: Any, poll_seconds: float = 1.0, stop_event=None):
required = (
"register_worker", "heartbeat", "claim_next", "renew", "execute_claim"
)
if transport is None or any(not callable(getattr(transport, name, None)) for name in required):
raise WorkerError("WORKER_TRANSPORT_INVALID")
if (
isinstance(poll_seconds, bool)
or not isinstance(poll_seconds, (int, float))
or not math.isfinite(float(poll_seconds))
or not 0.001 <= poll_seconds <= 60
):
raise WorkerError("WORKER_POLL_INTERVAL_INVALID")
if stop_event is not None and (
not callable(getattr(stop_event, "is_set", None))
or not callable(getattr(stop_event, "wait", None))
):
raise WorkerError("WORKER_STOP_EVENT_INVALID")
self.transport = transport
self.poll_seconds = float(poll_seconds)
self.stop_event = stop_event if stop_event is not None else threading.Event()
self.registered = False
self.worker_id: str | None = None
self._last_heartbeat = 0.0
self.last_error: str | None = None
def _register(self) -> None:
result = self.transport.register_worker()
if (
not isinstance(result, dict) or result.get("registered") is not True
or not isinstance(result.get("worker_id"), str) or not result["worker_id"]
):
raise WorkerError("WORKER_REGISTRATION_RESULT_INVALID")
self.worker_id = result["worker_id"]
self.registered = True
def _heartbeat(self, *, force: bool = False) -> None:
now = time.monotonic()
if not force and now < self._last_heartbeat + self.HEARTBEAT_INTERVAL_SECONDS:
return
result = self.transport.heartbeat()
if (
not isinstance(result, dict) or result.get("alive") is not True
or result.get("worker_id") != self.worker_id
):
raise WorkerError("WORKER_HEARTBEAT_RESULT_INVALID")
self._last_heartbeat = time.monotonic()
def run_once(self):
"""Perform one register/heartbeat/claim/renew/execute iteration."""
if self.stop_event.is_set():
return None
just_registered = False
if not self.registered:
self._register()
just_registered = True
self._heartbeat(force=just_registered)
claim = self.transport.claim_next()
if claim is None:
self.last_error = None
return None
required = {"task_id", "claim_id", "generation", "lease_until", "operation"}
if not isinstance(claim, dict) or set(claim) != required:
raise WorkerError("WORKER_CLAIM_RESULT_INVALID")
task_id = claim["task_id"]
claim_id = claim["claim_id"]
if not isinstance(task_id, str) or not task_id or not isinstance(claim_id, str) or not claim_id:
raise WorkerError("WORKER_CLAIM_RESULT_INVALID")
renewed = self.transport.renew(task_id, claim_id)
if (
not isinstance(renewed, dict) or renewed.get("renewed") is not True
or renewed.get("task_id") != task_id or renewed.get("claim_id") != claim_id
):
raise WorkerError("WORKER_RENEW_RESULT_INVALID")
result = self.transport.execute_claim(task_id, claim_id)
if not isinstance(result, dict) or result.get("task_id") != task_id:
raise WorkerError("WORKER_EXECUTION_RESULT_INVALID")
self.last_error = None
return result
def run_forever(self) -> None:
"""Run until the supplied stop event is set, using interruptible waits."""
while not self.stop_event.is_set():
try:
self.run_once()
except TransportError as exc:
# Transport failures are observable and retried on the next
# bounded poll. Re-register after a server restart; programming
# and schema errors still terminate.
self.last_error = exc.code
self.registered = False
self.stop_event.wait(self.poll_seconds)
Worker = OperationWorker
__all__ = ["OperationWorker", "Worker", "WorkerError"]
@@ -0,0 +1,51 @@
"""Registered worker eligibility; fallback is deployment policy, never a special actor."""
from dataclasses import dataclass
import math
from typing import Mapping
@dataclass(frozen=True)
class WorkerProfile:
worker_id: str
identity: str
tenant_scope: tuple[str, ...]
subject_scope: tuple[str, ...]
capabilities: tuple[str, ...]
priority: int
fallback_executor: bool
heartbeat_seconds: int
max_concurrency: int
@classmethod
def from_dict(cls, value: Mapping):
if set(value) != set(cls.__dataclass_fields__):
raise ValueError('WORKER_PROFILE_FIELDS_INVALID')
data = dict(value)
for name in ('worker_id', 'identity'):
if not isinstance(data[name], str) or not data[name] or len(data[name]) > 128:
raise ValueError('WORKER_ID_INVALID')
for name in ('tenant_scope', 'subject_scope', 'capabilities'):
items = data[name]
if not isinstance(items, (list, tuple)) or not items or any(not isinstance(x, str) or not x or '*' in x for x in items):
raise ValueError('WORKER_SCOPE_INVALID')
data[name] = tuple(sorted(set(items)))
if type(data['fallback_executor']) is not bool:
raise ValueError('WORKER_FALLBACK_INVALID')
for name, lower, upper in (('priority', 0, 1000), ('heartbeat_seconds', 1, 300), ('max_concurrency', 1, 32)):
if type(data[name]) is not int or not lower <= data[name] <= upper:
raise ValueError('WORKER_LIMIT_INVALID')
return cls(**data)
def eligible(worker: WorkerProfile, task: Mapping, *, now: float, last_heartbeat: float,
fallback_seconds: int = 600) -> bool:
ready_at = task.get('ready_at')
if any(type(value) not in (int,float) or not math.isfinite(value) for value in (ready_at,now,last_heartbeat)) or ready_at > now:
return False
if last_heartbeat > now or now > last_heartbeat + worker.heartbeat_seconds * 3:
return False
if task.get('tenant_id') not in worker.tenant_scope or task.get('subject_id') not in worker.subject_scope:
return False
if task.get('operation') not in worker.capabilities:
return False
return not worker.fallback_executor or now >= ready_at + fallback_seconds
@@ -1,334 +0,0 @@
"""Canonical fail-closed capability semantics for task eligibility."""
from __future__ import annotations
from collections.abc import Iterable
from dataclasses import asdict, dataclass
from typing import Any
_TASK_ALIASES: dict[str, frozenset[str]] = {
"repository_read": frozenset({"repo.read"}),
"git.read": frozenset({"repo.read"}),
"repo.read": frozenset({"repo.read"}),
"test_execution": frozenset({"test.execute"}),
"git.test": frozenset({"test.execute"}),
"test.execute": frozenset({"test.execute"}),
"governed_git_write": frozenset({"governed.git.write.branch", "governed.git.write.file"}),
"governed.git.write": frozenset({"governed.git.write.branch", "governed.git.write.file"}),
"governed.write.branch": frozenset({"governed.git.write.branch"}),
"governed.write.file": frozenset({"governed.git.write.file"}),
"shell": frozenset({"runtime.shell"}),
"git": frozenset({"runtime.git"}),
"python": frozenset({"runtime.python"}),
"code.read": frozenset({"legacy.code.read"}),
"code.review": frozenset({"legacy.code.review"}),
"code.write": frozenset({"legacy.code.write"}),
"code.write.patch": frozenset({"legacy.code.write.patch"}),
"test.python": frozenset({"legacy.test.python"}),
"test.shell.allowlisted": frozenset({"legacy.test.shell.allowlisted"}),
"git.commit": frozenset({"legacy.git.commit"}),
"git.diff.read": frozenset({"legacy.git.diff.read"}),
"git.force_push": frozenset({"legacy.git.force_push"}),
"deploy.production": frozenset({"legacy.deploy.production"}),
"secret.transform": frozenset({"legacy.secret.transform"}),
"secure_file_transformer": frozenset({"legacy.secure_file_transformer"}),
"runtime.shadow": frozenset({"legacy.runtime.shadow"}),
"review.security": frozenset({"legacy.review.security"}),
"review.api-contract": frozenset({"legacy.review.api-contract"}),
"gpu.train": frozenset({"legacy.gpu.train"}),
"agent.runtime.hermes": frozenset({"agent.runtime.hermes"}),
"agent.runtime.openclaw": frozenset({"agent.runtime.openclaw"}),
"release:candidate": frozenset({"release:candidate"}),
}
_WORKER_ALIASES: dict[str, frozenset[str]] = {
"git.read": frozenset({"repo.read"}),
"repo.read": frozenset({"repo.read"}),
"repository_read": frozenset({"repo.read"}),
"git.test": frozenset({"test.execute"}),
"test.execute": frozenset({"test.execute"}),
"test_execution": frozenset({"test.execute"}),
"governed.write.branch": frozenset({"governed.git.write.branch"}),
"governed.git.write.branch": frozenset({"governed.git.write.branch"}),
"governed.write.file": frozenset({"governed.git.write.file"}),
"governed.git.write.file": frozenset({"governed.git.write.file"}),
"shell": frozenset({"runtime.shell"}),
"git": frozenset({"runtime.git"}),
"python": frozenset({"runtime.python"}),
"code.read": frozenset({"legacy.code.read"}),
"code.review": frozenset({"legacy.code.review"}),
"code.write": frozenset({"legacy.code.write"}),
"code.write.patch": frozenset({"legacy.code.write.patch"}),
"test.python": frozenset({"legacy.test.python"}),
"test.shell.allowlisted": frozenset({"legacy.test.shell.allowlisted"}),
"git.commit": frozenset({"legacy.git.commit"}),
"git.diff.read": frozenset({"legacy.git.diff.read"}),
"git.force_push": frozenset({"legacy.git.force_push"}),
"deploy.production": frozenset({"legacy.deploy.production"}),
"secret.transform": frozenset({"legacy.secret.transform"}),
"secure_file_transformer": frozenset({"legacy.secure_file_transformer"}),
"runtime.shadow": frozenset({"legacy.runtime.shadow"}),
"review.security": frozenset({"legacy.review.security"}),
"review.api-contract": frozenset({"legacy.review.api-contract"}),
"gpu.train": frozenset({"legacy.gpu.train"}),
"agent.runtime.hermes": frozenset({"agent.runtime.hermes"}),
"agent.runtime.openclaw": frozenset({"agent.runtime.openclaw"}),
"release:candidate": frozenset({"release:candidate"}),
}
@dataclass(frozen=True)
class CapabilityContract:
"""Versioned runtime contract attached to one canonical semantic capability."""
name: str
min_version: int = 1
max_version: int = 1
permissions: tuple[str, ...] = ()
forbidden: tuple[str, ...] = ()
dependencies: tuple[str, ...] = ()
health_requirements: tuple[str, ...] = ()
def as_dict(self) -> dict[str, Any]:
return asdict(self)
# This registry is the semantic source of truth. Alias tables below are retained
# as compatibility adapters; startup/tests validate every alias target against
# this registry so a name cannot be accepted in one layer and be unknown in
# another.
_CAPABILITY_CONTRACTS: dict[str, CapabilityContract] = {
"repo.read": CapabilityContract("repo.read", permissions=("repository_read",)),
"test.execute": CapabilityContract("test.execute", permissions=("test_execution",)),
"governed.git.write.branch": CapabilityContract(
"governed.git.write.branch",
permissions=("governed_branch_write",),
dependencies=("writer_lease", "governed_gitops"),
),
"governed.git.write.file": CapabilityContract(
"governed.git.write.file",
permissions=("governed_file_write",),
dependencies=("writer_lease", "governed_gitops"),
),
"governed.git.write": CapabilityContract(
"governed.git.write",
permissions=("governed_git_write",),
dependencies=("writer_lease", "governed_gitops"),
),
"runtime.shell": CapabilityContract("runtime.shell", permissions=("runtime_shell",)),
"runtime.git": CapabilityContract("runtime.git", permissions=("runtime_git",)),
"runtime.python": CapabilityContract("runtime.python", permissions=("runtime_python",)),
"legacy.code.read": CapabilityContract("legacy.code.read"),
"legacy.code.review": CapabilityContract("legacy.code.review"),
"legacy.code.write": CapabilityContract("legacy.code.write"),
"legacy.code.write.patch": CapabilityContract("legacy.code.write.patch"),
"legacy.test.python": CapabilityContract("legacy.test.python"),
"legacy.test.shell.allowlisted": CapabilityContract("legacy.test.shell.allowlisted"),
"legacy.git.commit": CapabilityContract("legacy.git.commit"),
"legacy.git.diff.read": CapabilityContract("legacy.git.diff.read"),
"legacy.git.force_push": CapabilityContract(
"legacy.git.force_push",
forbidden=("protected_ref_bypass",),
),
"legacy.deploy.production": CapabilityContract(
"legacy.deploy.production",
permissions=("production_deploy",),
dependencies=("owner_approval", "deployment_controller"),
health_requirements=("approval_envelope_verified", "runtime_acceptance_available"),
),
"legacy.secret.transform": CapabilityContract("legacy.secret.transform"),
"legacy.secure_file_transformer": CapabilityContract("legacy.secure_file_transformer"),
"legacy.runtime.shadow": CapabilityContract("legacy.runtime.shadow"),
"legacy.review.security": CapabilityContract("legacy.review.security"),
"legacy.review.api-contract": CapabilityContract("legacy.review.api-contract"),
"legacy.gpu.train": CapabilityContract("legacy.gpu.train"),
"agent.runtime.hermes": CapabilityContract(
"agent.runtime.hermes",
dependencies=("hermes_runtime",),
health_requirements=("worker_liveness",),
),
"agent.runtime.openclaw": CapabilityContract(
"agent.runtime.openclaw",
dependencies=("openclaw_runtime",),
health_requirements=("worker_liveness",),
),
"release:candidate": CapabilityContract(
"release:candidate",
min_version=1,
max_version=1,
permissions=("prepare_release",),
forbidden=(
"production_deploy",
"submit_owner_approval",
"tag_publish",
"baseline_registry_finalize",
"runtime_switch",
),
dependencies=(
"credential_projection",
"trusted_builder",
),
health_requirements=(
"authenticated_probe",
"release_api_health",
"worker_liveness",
),
),
}
def capability_contract(name: object) -> CapabilityContract | None:
"""Return the contract for one canonical or registered capability name."""
text = _clean(name)
direct = _CAPABILITY_CONTRACTS.get(text)
if direct is not None:
return direct
mapped = _TASK_ALIASES.get(text) or _WORKER_ALIASES.get(text)
if mapped is None or len(mapped) != 1:
return None
return _CAPABILITY_CONTRACTS.get(next(iter(mapped)))
def registered_capability(name: object, *, worker: bool = False) -> bool:
"""Fail-closed validation used by registration/task admission."""
text = _clean(name)
if "*" in text:
return False
aliases = _WORKER_ALIASES if worker else _TASK_ALIASES
mapped = aliases.get(text)
if mapped is None:
return text in _CAPABILITY_CONTRACTS
return all(item in _CAPABILITY_CONTRACTS for item in mapped)
def validate_capability_contract_registry() -> dict[str, object]:
"""Prove alias/contract closure and contract safety invariants."""
alias_targets: set[str] = set()
for mapping in (_TASK_ALIASES, _WORKER_ALIASES):
for values in mapping.values():
alias_targets.update(values)
missing = sorted(alias_targets - set(_CAPABILITY_CONTRACTS))
invalid_versions = sorted(
name
for name, contract in _CAPABILITY_CONTRACTS.items()
if contract.min_version < 1 or contract.max_version < contract.min_version
)
permission_forbidden_overlap = sorted(
name
for name, contract in _CAPABILITY_CONTRACTS.items()
if set(contract.permissions) & set(contract.forbidden)
)
return {
"schema": "aios.capability-contract-registry.v1",
"contract_count": len(_CAPABILITY_CONTRACTS),
"missing_alias_targets": missing,
"invalid_versions": invalid_versions,
"permission_forbidden_overlap": permission_forbidden_overlap,
"valid": not missing and not invalid_versions and not permission_forbidden_overlap,
}
def capability_contract_snapshot() -> dict[str, dict[str, Any]]:
return {
name: contract.as_dict()
for name, contract in sorted(_CAPABILITY_CONTRACTS.items())
}
_UNKNOWN_PREFIX = "exact:"
def _clean(value: object) -> str:
if not isinstance(value, str):
raise ValueError("capability must be a string")
text = value.strip().lower()
if not text:
raise ValueError("capability must not be empty")
return text
def normalize_required_capabilities(values: Iterable[object]) -> frozenset[str]:
"""Map task-facing names to one semantic vocabulary.
Unknown names remain explicit exact requirements. They are never treated
as wildcards or granted by a failed normalization.
"""
result: set[str] = set()
for value in values:
text = _clean(value)
if "*" in text:
result.add(_UNKNOWN_PREFIX + text)
else:
mapped = _TASK_ALIASES.get(text)
if mapped is None or any(item not in _CAPABILITY_CONTRACTS for item in mapped):
result.add(_UNKNOWN_PREFIX + text)
else:
result.update(mapped)
return frozenset(result)
def normalize_worker_capabilities(values: Iterable[object]) -> frozenset[str]:
"""Map registered worker names to effective semantic capabilities."""
result: set[str] = set()
for value in values:
text = _clean(value)
if "*" in text:
continue
mapped = _WORKER_ALIASES.get(text)
if mapped is not None and all(item in _CAPABILITY_CONTRACTS for item in mapped):
result.update(mapped)
if {
"governed.git.write.branch",
"governed.git.write.file",
}.issubset(result):
result.add("governed.git.write")
return frozenset(result)
def has_capabilities(
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
) -> bool:
"""Return the single eligibility decision used by poll and claim."""
required = normalize_required_capabilities(required_capabilities)
effective = normalize_worker_capabilities(worker_capabilities)
if any(value.startswith(_UNKNOWN_PREFIX) for value in required):
return False
return required.issubset(effective)
def required_capabilities_subset(
granted_requirements: Iterable[object],
required_requirements: Iterable[object],
) -> bool:
"""Check semantic capability containment for delegation constraints."""
granted = normalize_required_capabilities(granted_requirements)
required = normalize_required_capabilities(required_requirements)
if any(value.startswith(_UNKNOWN_PREFIX) for value in granted | required):
return False
return required.issubset(granted)
def normalized_capability_matrix(
worker_capabilities: Iterable[object],
required_capabilities: Iterable[object],
) -> dict[str, object]:
"""Expose non-secret decision evidence for tests and diagnostics."""
worker_values = tuple(worker_capabilities)
required_values = tuple(required_capabilities)
worker = normalize_worker_capabilities(worker_values)
required = normalize_required_capabilities(required_values)
return {
"worker_effective": sorted(worker),
"task_required": sorted(required),
"eligible": has_capabilities(worker_values, required_values),
}
+125 -9
View File
@@ -22,8 +22,15 @@ from gitea_tag_publication import (
TagPublisher,
redact,
validate_version,
validate_gitea_api_url,
)
from orchestrator.action_monitor import ActionMonitor
try:
from orchestrator.action_monitor import ActionMonitor
except ModuleNotFoundError:
# The deployed service normally supplies runtime/agent-os on sys.path.
# Keep direct, isolated contract execution bound to the repository copy.
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "runtime" / "agent-os"))
from orchestrator.action_monitor import ActionMonitor
from silent_release_hardening import (
ApprovalKeyRing,
ApprovalRevocationStore,
@@ -63,6 +70,11 @@ except ModuleNotFoundError:
repository_snapshot,
run_authoritative_git,
)
try:
from aios_repository_discovery import RepositoryDiscovery, RepositoryDiscoveryError
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parent))
from aios_repository_discovery import RepositoryDiscovery, RepositoryDiscoveryError
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
WS = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
@@ -108,6 +120,8 @@ READ_ONLY_GET_PATHS = frozenset({
'/admin/ai-evolution/api/data',
'/api/release-consistency', '/api/tag-publication-status',
'/api/release-acceptance',
'/api/repository/tree', '/api/repository/search',
'/api/repository/diff-index', '/api/repository/file',
'/getReleaseCandidate', '/getReleasePlan', '/getReleaseAcceptance',
'/getApprovalEnvelope', '/getReleaseSaga', '/getCapabilityStatus',
'/getRecoveryStatus',
@@ -308,8 +322,7 @@ def resolve_commit_oid(value: str | None, field: str = 'target_oid') -> str:
raise ValueError(f'{field} does not resolve to the exact approved commit') from exc
def existing_tag_target(tag: str) -> str | None:
if not re.fullmatch(r'V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$', str(tag or '')):
raise ValueError('invalid release version')
validate_version(tag)
try:
result = run_authoritative_git(
['rev-parse', '--verify', f'refs/tags/{tag}^{{commit}}'],
@@ -342,9 +355,80 @@ def required_authoritative_git_dir() -> Path:
return Path(str(snapshot["authoritative_git_dir"]))
_DISCOVERY_ACTIONS = {
'/api/repository/tree': (
'list_repository_tree',
{'repository', 'ref', 'limit'},
{'prefix', 'depth', 'cursor'},
),
'/api/repository/search': (
'search_repository_source',
{'repository', 'ref', 'query', 'max_matches', 'max_bytes'},
{'path_globs', 'cursor'},
),
'/api/repository/diff-index': (
'get_diff_index',
{'repository', 'from', 'to', 'limit'},
{'path_prefix', 'cursor'},
),
'/api/repository/file': (
'read_repository_file',
{'repository', 'ref', 'path'},
{'max_bytes'},
),
}
def repository_discovery_request(
path: str, query: dict[str, list[str]]
) -> dict[str, Any]:
operation, required, optional = _DISCOVERY_ACTIONS[path]
if set(query) - required - optional:
raise ValueError('unknown repository discovery query field')
request: dict[str, Any] = {}
for key, values in query.items():
if key == 'path_globs':
if not values:
raise ValueError('empty repository discovery path glob')
request[key] = list(values)
continue
if len(values) != 1:
raise ValueError('duplicate repository discovery query field')
value: Any = values[0]
if key in {'depth', 'limit', 'max_matches', 'max_bytes'}:
try:
value = int(value, 10)
except (TypeError, ValueError) as exc:
raise ValueError('invalid repository discovery integer') from exc
request[key] = value
if required - set(request):
raise ValueError('missing repository discovery query field')
if not REPOSITORY_IDENTITY:
raise RepositoryDiscoveryError('REPOSITORY_DISCOVERY_UNAVAILABLE')
try:
discovery = RepositoryDiscovery(
allowed_repositories=(REPOSITORY_IDENTITY,),
git_dir=required_authoritative_git_dir(),
cursor_secret=os.getenv('AIOS_REPOSITORY_DISCOVERY_CURSOR_SECRET'),
)
except RuntimeError as exc:
if isinstance(exc, RepositoryDiscoveryError):
raise
raise RepositoryDiscoveryError('REPOSITORY_DISCOVERY_UNAVAILABLE') from exc
public_to_internal = {
'ref': 'revision',
'path': 'file_path',
'path_prefix': 'prefix',
}
internal_request = {
public_to_internal.get(key, key): value for key, value in request.items()
}
return dict(getattr(discovery, operation)(internal_request))
def publication_publisher() -> TagPublisher:
authoritative_repo = required_authoritative_git_dir()
return TagPublisher(
publisher = TagPublisher(
WS,
GitTagBackend(WS, authoritative_repo=authoritative_repo),
GiteaTagClient(
@@ -355,6 +439,10 @@ def publication_publisher() -> TagPublisher:
),
os.getenv('AIOS_GITOPS_GITEA_REMOTE','gitea'),
)
publisher.baseline_path = Path(os.getenv(
'AIOS_GITOPS_BASELINE_AUTHORITY', str(WS / 'baseline' / 'current.json')
))
return publisher
def commit(paths, msg):
ensure_git(); paths = [relpath(p) for p in paths] if paths else ['.']; sh(['add'] + paths, True)
@@ -437,6 +525,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error': 'GOVERNED_BRIDGE_UNAVAILABLE', 'detail': type(exc).__name__}, 503)
def do_POST(self):
path = urllib.parse.urlparse(self.path).path
if path in _DISCOVERY_ACTIONS:
return self.sendj({'error': 'method_not_allowed'},405)
if path in {
'/createTagApprovalRequest', '/requestOrCreateTag', '/getTagPublicationStatus',
'/buildProductionArtifact', '/executeApprovedProductionRelease',
@@ -452,12 +542,21 @@ class H(BaseHTTPRequestHandler):
def do_GET(self):
u=urllib.parse.urlparse(self.path); path=u.path.rstrip('/') if u.path != '/' else u.path
if path not in READ_ONLY_GET_PATHS: return self.sendj({'error':'not_found','path':path},404)
q=urllib.parse.parse_qs(u.query)
if len(self.path) > 8192:
return self.sendj({'error':'query_too_large'},413)
try:
q=urllib.parse.parse_qs(
u.query, keep_blank_values=True, max_num_fields=64
)
except ValueError:
return self.sendj({'error':'query_invalid'},400)
try:
if path in ('/health','/api/health'): return self.sendj({'status':'ok','service':'chatgpt-gitops-adapter','workspace':str(WS)})
if path in ('/api/openapi.yaml','/api/openapi-chatgpt-actions.yaml','/api/schema'):
return self.sendt(OPENAPI.read_text(encoding='utf-8'), 'application/yaml; charset=utf-8') if OPENAPI.exists() else self.sendj({'error':'schema_missing'},404)
if path in ('/api','/api/status'): return self.sendj(api_spec())
if path in _DISCOVERY_ACTIONS:
return self.sendj(repository_discovery_request(path, q))
if path in ('/api/base','/api/baseline','/base'):
data=jload(WS/'baseline'/'current.json', {})
data.setdefault('authoritative_repository', repository_snapshot(
@@ -510,7 +609,15 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':redact(exc)},413)
except (AuthoritativeGitError, RuntimeError) as exc:
return self.sendj({'error':redact(exc)},503)
return self.sendj({'from':frm,'to':to,'diff':diff})
payload = {'from': frm, 'to': to, 'diff': diff}
payload.update({
key: diff[key]
for key in (
'from_oid', 'to_oid', 'changed_paths', 'stats',
'total_size_bytes', 'truncated', 'patch_limit_bytes',
)
})
return self.sendj(payload)
if path in ('/api/patch','/api/patches','/api/branches','/patch'):
snapshot = repository_snapshot(
git_dir=AUTHORITATIVE_GIT_DIR,
@@ -622,7 +729,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj(
service.get_capability_status(candidate_id)
)
return self.sendj(service.get_release_saga(candidate_id))
result = service.get_release_saga(candidate_id)
return self.sendj(result, 404 if result.get('status') == 'NOT_FOUND' else 200)
if path == '/getRecoveryStatus':
return self.sendj(
silent_release_service().get_recovery_status()
@@ -635,6 +743,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':'not_found','path':path},404)
except FileNotFoundError:
return self.sendj({'error':'release candidate not found'},404)
except RepositoryDiscoveryError as e:
return self.sendj({'error': redact(e)}, getattr(e, 'status_code', 400))
except ValueError as e:
error=redact(e)
monitor.record_failure('do_GET', error, {'path':path})
@@ -651,6 +761,8 @@ class H(BaseHTTPRequestHandler):
return self.sendj({'error':'not_found','path':path},404)
def do_POST(self):
path=urllib.parse.urlparse(self.path).path.rstrip('/')
if path in _DISCOVERY_ACTIONS:
return self.sendj({'error':'method_not_allowed'},405)
if reject_non_read_http(self, 'POST', path): return
data=self.body()
try:
@@ -773,7 +885,10 @@ class H(BaseHTTPRequestHandler):
a=self.need('tag:create');
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[A-Za-z0-9_.-]+)?$', ver or ''): return self.sendj({'error':'version must match VxRxCx'},400)
try:
validate_version(ver)
except ValueError:
return self.sendj({'error':'version must match VxRxCx'},400)
if not data.get('target_oid') or not data.get('evidence_closure_oid') or not data.get('approval_plan_id'):
return self.sendj({
'error':'production_tag_evidence_required',
@@ -944,9 +1059,10 @@ return{vs,cmds,ar,cmd,cdet,tot,sok,sfail,runs,baseline,refreshMsg,loadCmd,selVer
def api_spec():
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
return {'version':'V1R1C0-chatgpt-actions','openapi':'/api/openapi.yaml','read':['GET /api/base','GET /api/ver','GET /api/diff','GET /api/repository/tree','GET /api/repository/search','GET /api/repository/diff-index','GET /api/repository/file','GET /api/patch','GET /api/model','GET /api/run','GET /getReleaseCandidate','GET /getApprovalEnvelope','GET /getReleaseSaga','GET /admin/ai-evolution'],'write':['POST /api/branch','POST /api/file','POST /api/commit','POST /api/patch','POST /api/tag','POST /api/rollback','POST /api/model','PUT /api/model-priority','POST /api/nl','POST /createReleaseCandidate','POST /submitOwnerApproval','POST /startDeployment'],'policy':{'bearer_required_for_write':True,'direct_main_push':False,'delete_tag':False,'owner_approval_required_for_v_level':True,'api_keys_in_git':False,'git_secret_field':'secret_ref','online_git_sync':True,'silent_release_execution_enabled':SILENT_RELEASE_EXECUTION_ENABLED,'silent_release_shadow_default':True}}
def main():
validate_gitea_api_url(os.getenv('AIOS_GITOPS_GITEA_API_URL', ''))
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
+192 -67
View File
@@ -6,15 +6,20 @@ import hashlib
from pathlib import Path
from typing import Any, Callable, Mapping
from aios_task_admission_bridge_contract import (
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS,
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS,
TASK_ADMISSION_ROUTES,
)
BRIDGE_SCHEMA_VERSION = "aios.chatgpt-gitops-bridge.v2"
BRIDGE_OPERATIONS = (
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"createGovernedTask",
"projectOnlineAgentExecution",
"acquireWriterLease",
"bootstrapWriterLease",
"renewWriterLease",
"commitFileGoverned",
"commitFilesGoverned",
"getGitOpsJob",
@@ -28,18 +33,23 @@ BRIDGE_OPERATIONS = (
"getReleaseConsistency",
"runReleaseAcceptance",
"getControlPlaneSnapshot",
"listRepositoryTree",
"searchRepositorySource",
"getDiffIndex",
"readRepositoryFile",
)
TASK_ADMISSION_OPERATIONS = tuple(TASK_ADMISSION_OPERATION_REQUIRED_FIELDS)
OPERATION_REQUIRED_FIELDS = {
"createGovernedTask": ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key"),
"acquireWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"bootstrapWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"renewWriterLease": ("schema", "task_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "lease_seconds"),
"commitFileGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("job_id",),
"resumeGitOpsJob": ("job_id", "expected_revision"),
"reconcileGitOpsJob": ("job_id", "expected_revision"),
"projectOnlineAgentExecution": ("task_id", "required_capabilities"),
"acquireWriterLease": ("schema", "task_id", "job_id", "repository", "branch_mode", "base_oid", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "lease_seconds"),
"commitFileGoverned": ("task_id", "job_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "path", "content_base64", "message"),
"commitFilesGoverned": ("task_id", "job_id", "repository", "expected_branch_oid", "target_branch", "generation", "idempotency_key", "files", "message"),
"getGitOpsJob": ("task_id", "job_id"),
"resumeGitOpsJob": ("task_id", "job_id", "expected_revision"),
"reconcileGitOpsJob": ("task_id", "job_id", "expected_revision"),
"createTagApprovalRequest": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "owner_approval", "execute"),
"requestOrCreateTag": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id", "execute"),
"getTagPublicationStatus": ("version",),
@@ -48,51 +58,145 @@ OPERATION_REQUIRED_FIELDS = {
"getReleaseConsistency": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"runReleaseAcceptance": ("version", "target_oid", "evidence_closure_oid", "approval_plan_id"),
"getControlPlaneSnapshot": (),
**TASK_ADMISSION_OPERATION_REQUIRED_FIELDS,
"listRepositoryTree": ("repository", "ref", "limit"),
"searchRepositorySource": ("repository", "ref", "query", "max_matches", "max_bytes"),
"getDiffIndex": ("repository", "from", "to", "limit"),
"readRepositoryFile": ("repository", "ref", "path"),
}
BRIDGE_OPERATION_CONTRACTS = {
"acquireWriterLease": {
"method": "POST",
"path": "/v2/writer-lease",
"required_fields": OPERATION_REQUIRED_FIELDS["acquireWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
),
},
"bootstrapWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/bootstrap",
"required_fields": OPERATION_REQUIRED_FIELDS["bootstrapWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"create_target_branch", "worktree_mode", "cas_ref_update_required",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"renewWriterLease": {
"method": "POST",
"path": "/v2/writer-lease/renew",
"required_fields": OPERATION_REQUIRED_FIELDS["renewWriterLease"],
"response_fields": (
"task_id", "job_id", "generation", "lease_seconds", "lease_expires_at",
"recovered", "bootstrapped", "runtime_rehydrated",
"durable_branch_lease_present", "idempotent",
),
},
"commitFileGoverned": {
"method": "POST", "path": "/v2/commit-file",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFileGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
"commitFilesGoverned": {
"method": "POST", "path": "/v2/commit-files",
"required_fields": OPERATION_REQUIRED_FIELDS["commitFilesGoverned"],
"response_fields": ("job_id", "commit_oid", "commit_visible", "branch_mode", "target_branch", "generation"),
},
OPERATION_OPTIONAL_FIELDS = {
"listRepositoryTree": ("prefix", "depth", "cursor"),
"searchRepositorySource": ("path_globs", "cursor"),
"getDiffIndex": ("path_prefix", "cursor"),
"readRepositoryFile": ("max_bytes",),
}
READ_ONLY_OPERATIONS = frozenset(
{
"getRuntimeStatus",
"getGovernedWriteCapabilities",
"getControlPlaneSnapshot",
"listRepositoryTree",
"searchRepositorySource",
"getDiffIndex",
"readRepositoryFile",
}
)
_OPERATION_PATHS = {
"createGovernedTask": "/v2/governed-task",
"projectOnlineAgentExecution": "/v2/execution-envelope",
"acquireWriterLease": "/v2/writer-lease",
"commitFileGoverned": "/v2/commit-file",
"commitFilesGoverned": "/v2/commit-files",
"getGitOpsJob": "/v2/job",
"resumeGitOpsJob": "/v2/job/resume",
"reconcileGitOpsJob": "/v2/job/reconcile",
"getTagPublicationStatus": "/v2/tag/status",
"getReleaseConsistency": "/v2/release/consistency",
"runReleaseAcceptance": "/v2/release/acceptance",
**{operation: path for path, operation in TASK_ADMISSION_ROUTES.items()},
"listRepositoryTree": "/v2/repository/tree",
"searchRepositorySource": "/v2/repository/search",
"getDiffIndex": "/v2/repository/diff-index",
"readRepositoryFile": "/v2/repository/file",
}
def _has_complete_task_admission_handlers(
handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None,
) -> bool:
return handlers is not None and all(
callable(handlers.get(operation)) for operation in TASK_ADMISSION_OPERATIONS
)
def operation_contracts(
handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None,
) -> dict[str, dict[str, object]]:
"""Return the runtime contract consumed by adapters and schema tooling."""
contracts = {
operation: {
"method": "GET" if operation in OPERATION_OPTIONAL_FIELDS else "POST",
"path": _OPERATION_PATHS.get(operation),
"required_fields": list(fields),
"optional_fields": list(OPERATION_OPTIONAL_FIELDS.get(operation, ())),
"read_only": operation in READ_ONLY_OPERATIONS,
}
for operation, fields in OPERATION_REQUIRED_FIELDS.items()
if operation not in TASK_ADMISSION_OPERATIONS
}
if _has_complete_task_admission_handlers(handlers):
contracts.update(
{
operation: {
"method": "GET" if operation == "getTaskAdmissionStatus" else "POST",
"path": _OPERATION_PATHS[operation],
"required_fields": list(
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS[operation]
),
"optional_fields": list(
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS[operation]
),
}
for operation in TASK_ADMISSION_OPERATIONS
}
)
return contracts
def render_openapi_contract_fragment() -> str:
required = "\n".join(
f" - {field}"
for field in OPERATION_REQUIRED_FIELDS["commitFilesGoverned"]
)
return (
" /v2/commit-files:\n"
" post:\n"
" operationId: commitFilesGoverned\n"
" summary: Submit one atomic governed commit containing multiple files\n"
" security: [{taskPrincipalAuth: []}]\n"
" x-aios-task-principal: Bridge resolves a fresh server-side TaskPrincipal from the authoritative admitted+claimed Task and re-verifies its Owner epoch, then checks exact task/job/repository/branch/generation/capability binding; caller JSON cannot provide a principal and missing verification is denied. An explicitly injected legacy bearer verifier may still be used by a trusted host.\n"
" requestBody:\n"
" required: true\n"
" content:\n"
" application/json:\n"
" schema:\n"
" type: object\n"
" additionalProperties: false\n"
" required:\n"
f"{required}\n"
" properties:\n"
" task_id: {type: string}\n"
" job_id: {type: string, description: External durable Job identity for exact completed-replay reconciliation}\n"
" source_lease_epoch: {type: integer, minimum: 1}\n"
" repository: {type: string, example: aios/aios}\n"
" expected_branch_oid: {type: string, pattern: '^[0-9a-f]{40}$'}\n"
" target_branch: {type: string, pattern: '^agent/[A-Za-z0-9._/-]+$'}\n"
" generation: {type: integer, minimum: 1}\n"
" idempotency_key: {type: string}\n"
" files:\n"
" type: array\n"
" minItems: 1\n"
" items:\n"
" type: object\n"
" additionalProperties: false\n"
" required: [path, content_base64]\n"
" properties:\n"
" path: {type: string, minLength: 1, maxLength: 512}\n"
" content_base64: {type: string}\n"
" message: {type: string, minLength: 1, maxLength: 200}\n"
" responses:\n"
" '200': {description: Governed commit job and verified branch result}\n"
" '202': {description: Existing external Job alias requires authoritative reconciliation; no replacement Job is created}\n"
" '401': {description: No verified Task principal is available}\n"
" '403': {description: Verified Task principal lacks exact task, external Job, or capability scope}\n"
" '400': {description: Contract or policy rejection}\n"
" '409': {description: Branch CAS or idempotency conflict}\n"
" '423': {description: Narrow P0 Task-principal mutation gate is frozen; assertion is not consumed}\n"
)
class RuntimeDriftError(RuntimeError):
pass
@@ -156,27 +260,38 @@ class ControlPlaneReleaseManifest:
class BridgeV2Adapter:
"""Thin operation adapter; all mutating handlers are injected seams."""
def __init__(self, *, runtime_status: Mapping[str, Any], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self.runtime_status = dict(runtime_status)
def __init__(self, *, runtime_status: Mapping[str, Any] | Callable[[], Mapping[str, Any]], handlers: Mapping[str, Callable[[Mapping[str, Any]], Any]] | None = None) -> None:
self._runtime_status_source = runtime_status if callable(runtime_status) else dict(runtime_status)
self.handlers = dict(handlers or {})
def _runtime_status(self) -> dict[str, Any]:
value = self._runtime_status_source() if callable(self._runtime_status_source) else self._runtime_status_source
if not isinstance(value, Mapping):
raise BridgeV2Error("RUNTIME_STATUS_INVALID")
return dict(value)
def getRuntimeStatus(self) -> dict[str, Any]:
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self.runtime_status}
return {"bridge_schema_version": BRIDGE_SCHEMA_VERSION, **self._runtime_status()}
def getGovernedWriteCapabilities(self) -> dict[str, Any]:
runtime_status = self._runtime_status()
governed_writes_enabled = runtime_status.get("governed_writes_enabled") is True
return {
"bridge_schema_version": BRIDGE_SCHEMA_VERSION,
"legacy_writes_enabled": False,
"governed_writes_enabled": True,
"operations": list(BRIDGE_OPERATIONS),
"operation_contracts": {
operation: {
**contract,
"required_fields": list(contract["required_fields"]),
"response_fields": list(contract["response_fields"]),
}
for operation, contract in BRIDGE_OPERATION_CONTRACTS.items()
},
"governed_writes_enabled": governed_writes_enabled,
"governed_write_blockers": list(
runtime_status.get("governed_write_blockers") or []
),
"operations": [
*BRIDGE_OPERATIONS,
*(
TASK_ADMISSION_OPERATIONS
if _has_complete_task_admission_handlers(self.handlers)
else ()
),
],
"operation_contracts": operation_contracts(self.handlers),
"force_push": False,
"main_write": False,
"tag_write": False,
@@ -193,21 +308,24 @@ class BridgeV2Adapter:
@staticmethod
def legacy_write_disabled() -> dict[str, object]:
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": True}
return {"error": "LEGACY_WRITE_DISABLED", "replacement": "createGovernedTask", "governed_writes_available": False}
def _call(self, operation: str, request: Mapping[str, Any]) -> Any:
required = OPERATION_REQUIRED_FIELDS[operation]
if any(key not in request for key in required):
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_MISSING")
if operation in OPERATION_OPTIONAL_FIELDS:
allowed = set(required) | set(OPERATION_OPTIONAL_FIELDS[operation])
if set(request) - allowed:
raise BridgeV2Error("BRIDGE_REQUEST_FIELDS_UNKNOWN")
handler = self.handlers.get(operation)
if handler is None:
raise BridgeV2Error("GOVERNED_HANDLER_UNAVAILABLE")
return handler(dict(request))
def createGovernedTask(self, request: Mapping[str, Any]) -> Any: return self._call("createGovernedTask", request)
def projectOnlineAgentExecution(self, request: Mapping[str, Any]) -> Any: return self._call("projectOnlineAgentExecution", request)
def acquireWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("acquireWriterLease", request)
def bootstrapWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("bootstrapWriterLease", request)
def renewWriterLease(self, request: Mapping[str, Any]) -> Any: return self._call("renewWriterLease", request)
def commitFileGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFileGoverned", request)
def commitFilesGoverned(self, request: Mapping[str, Any]) -> Any: return self._call("commitFilesGoverned", request)
def getGitOpsJob(self, request: Mapping[str, Any]) -> Any: return self._call("getGitOpsJob", request)
@@ -220,5 +338,12 @@ class BridgeV2Adapter:
def executeApprovedProductionRelease(self, request: Mapping[str, Any]) -> Any: return self._call("executeApprovedProductionRelease", request)
def getReleaseConsistency(self, request: Mapping[str, Any]) -> Any: return self._call("getReleaseConsistency", request)
def runReleaseAcceptance(self, request: Mapping[str, Any]) -> Any: return self._call("runReleaseAcceptance", request)
def getTaskAdmissionEnvelope(self, request: Mapping[str, Any]) -> Any: return self._call("getTaskAdmissionEnvelope", request)
def submitTaskAdmissionAuthorization(self, request: Mapping[str, Any]) -> Any: return self._call("submitTaskAdmissionAuthorization", request)
def getTaskAdmissionStatus(self, request: Mapping[str, Any]) -> Any: return self._call("getTaskAdmissionStatus", request)
def listRepositoryTree(self, request: Mapping[str, Any]) -> Any: return self._call("listRepositoryTree", request)
def searchRepositorySource(self, request: Mapping[str, Any]) -> Any: return self._call("searchRepositorySource", request)
def getDiffIndex(self, request: Mapping[str, Any]) -> Any: return self._call("getDiffIndex", request)
def readRepositoryFile(self, request: Mapping[str, Any]) -> Any: return self._call("readRepositoryFile", request)
def getControlPlaneSnapshot(self, request: Mapping[str, Any] | None = None) -> Any: return self._call("getControlPlaneSnapshot", request or {})
File diff suppressed because it is too large Load Diff
+60 -137
View File
@@ -51,14 +51,12 @@ class AuthorityPort(Protocol):
def branch_exists(self, branch: str) -> bool: ...
def branch_head(self, branch: str) -> str: ...
def is_ancestor(self, older: str, newer: str) -> bool: ...
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str: ...
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str: ...
class RemotePort(Protocol):
def repository_identity(self) -> str: ...
def push_new_exact(self, branch: str, new: str) -> None: ...
def push_exact(self, branch: str, expected: str, new: str) -> None: ...
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None: ...
def read_git_ref(self, branch: str) -> str | None: ...
def read_api_branch(self, branch: str) -> str | None: ...
def commit_visible(self, oid: str) -> bool: ...
@@ -105,9 +103,6 @@ class AuthorityGitAdapter:
def is_ancestor(self, older: str, newer: str) -> bool:
return self.executor.ancestry_check(older, newer)
def create_task_branch_cas(self, branch: str, expected_base: str, new: str) -> str:
return self.executor.create_task_branch_CAS(self._ref(branch), expected_base, new)
def compare_and_swap_branch(self, branch: str, expected: str, new: str) -> str:
return self.executor.advance_task_branch_CAS(self._ref(branch), expected, new)
@@ -263,19 +258,11 @@ class GiteaContinuationRemote:
raise ContinuationExecutionError("GITEA_GIT_READBACK_INVALID")
return oid
def push_new_exact(self, branch: str, new: str) -> None:
def push_exact(self, branch: str, expected: str, new: str, *, allow_absent: bool = False) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) is not None:
raise ContinuationExecutionError("GITEA_TARGET_BRANCH_ALREADY_EXISTS")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
if self.read_git_ref(branch) != new:
raise ContinuationExecutionError("GITEA_CREATE_READBACK_MISMATCH")
def push_exact(self, branch: str, expected: str, new: str) -> None:
self._require_projected()
self.repository_identity()
if self.read_git_ref(branch) != expected:
current = self.read_git_ref(branch)
if current != expected and not (allow_absent and current is None):
raise ContinuationExecutionError("GITEA_BRANCH_EXPECTATION_MISMATCH")
_run(["git", "push", self.remote_name, f"{new}:refs/heads/{branch}"], cwd=self.workspace)
@@ -343,13 +330,16 @@ class ContinuationPermissions:
@dataclass(frozen=True, slots=True)
class ContinuationRuntimeBinding:
request: WriterLeaseRequestV2
job_id: str
workspace_path: Path
lease_not_before: datetime
lease_expires_at: datetime
original_receipt_valid: bool
original_grant_consumed: bool
permissions: ContinuationPermissions
remote_branch_create_allowed: bool = False
external_job_id: str | None = None
worker_principal: str = "aios-autonomy-agent"
worker_lineage: str = "governed-continuation"
@dataclass(frozen=True, slots=True)
@@ -404,20 +394,13 @@ class ContinuationExecutor:
def observe_generation(self, branch: str, generation: int) -> None:
self._observed_generations[branch] = max(generation, self._observed_generations.get(branch, 0))
def _max_generation(self, branch: str, *, exclude_job_id: str | None = None) -> int:
def _max_generation(self, branch: str) -> int:
observed = self._observed_generations.get(branch, 0)
with self.controller.store._connect() as connection:
if exclude_job_id is None:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
else:
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases "
"WHERE branch=? AND job_id!=?",
(branch, exclude_job_id),
).fetchone()
row = connection.execute(
"SELECT MAX(generation) AS generation FROM branch_leases WHERE branch=?",
(branch,),
).fetchone()
durable = int(row["generation"] or 0)
return max(observed, durable)
@@ -426,7 +409,6 @@ class ContinuationExecutor:
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
@@ -436,21 +418,10 @@ class ContinuationExecutor:
"lease_seconds": request.lease_seconds,
}
def _validate_preflight(self, binding: ContinuationRuntimeBinding, *, job_id: str | None = None) -> None:
def _validate_preflight(self, binding: ContinuationRuntimeBinding) -> None:
request = binding.request
request_mapping = self._request_mapping(request)
normalized = validate_writer_lease_request_v2(request_mapping, self.authority)
expected_normalized = {
**request_mapping,
"create_target_branch": request.branch_mode == "CREATE_NEW",
"worktree_mode": (
"detached_at_base_oid"
if request.branch_mode == "CREATE_NEW"
else "detached_at_expected_head"
),
"cas_ref_update_required": True,
}
if normalized != expected_normalized:
normalized = validate_writer_lease_request_v2(self._request_mapping(request), self.authority)
if normalized != {**self._request_mapping(request), "create_target_branch": False, "worktree_mode": "detached_at_expected_head", "cas_ref_update_required": True}:
raise ContinuationExecutionError("CONTINUATION_NORMALIZATION_INVALID")
if not binding.original_receipt_valid or not binding.original_grant_consumed:
raise ContinuationExecutionError("ORIGINAL_GRANT_LINEAGE_INVALID")
@@ -460,7 +431,7 @@ class ContinuationExecutor:
raise ContinuationExecutionError("LEASE_NOT_ACTIVE")
if now >= binding.lease_expires_at:
raise ContinuationExecutionError("LEASE_EXPIRED")
if request.generation <= self._max_generation(request.target_branch, exclude_job_id=job_id):
if request.generation <= self._max_generation(request.target_branch):
raise ContinuationExecutionError("GENERATION_NOT_MONOTONIC")
if not self.workspace.is_detached():
raise ContinuationExecutionError("WORKTREE_NOT_DETACHED")
@@ -484,104 +455,70 @@ class ContinuationExecutor:
"payload_digest": hashlib.sha256(payload).hexdigest(),
"lineage_anchor_oid": request.base_oid,
"generation": request.generation,
"remote_branch_create_allowed": binding.remote_branch_create_allowed,
"external_job_id": binding.external_job_id,
"files_digest": hashlib.sha256(
json.dumps(
{path: hashlib.sha256(content).hexdigest() for path, content in sorted(commit.files.items())},
sort_keys=True, separators=(",", ":"),
).encode("utf-8")
).hexdigest(),
"message_digest": hashlib.sha256(commit.message.encode("utf-8")).hexdigest(),
"worker_principal": binding.worker_principal,
"worker_lineage": binding.worker_lineage,
}
def execute(
self,
binding: ContinuationRuntimeBinding,
commit: ContinuationCommit,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
commit.payload()
if not binding.job_id or binding.request.job_id != binding.job_id:
raise ContinuationExecutionError("RUNTIME_LINEAGE_MISSING_JOB_ID")
current = self.controller.store.read(binding.job_id)
current_state = JobState(current["state"])
if current_state == JobState.COMPLETED:
def execute(self, binding: ContinuationRuntimeBinding, commit: ContinuationCommit) -> dict[str, object]:
payload = commit.payload()
submitted = self.controller.submit(self._job_request(binding, commit, payload), payload=payload)
current = self.controller.store.read(submitted.job_id)
if JobState(current["state"]) == JobState.COMPLETED:
return self.controller.store._receipt(current, duplicate=True).as_dict()
if current_state == JobState.REMOTE_VERIFIED:
receipt = json.loads(str(current["receipt_json"] or "{}"))
commit_oid = str(receipt.get("authority_oid") or "")
if not OID_RE.fullmatch(commit_oid):
raise ContinuationExecutionError("GOVERNED_COMMIT_OID_INVALID")
if before_complete is not None:
before_complete(commit_oid)
return self.controller.advance(
binding.job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
self._validate_preflight(binding, job_id=binding.job_id)
self._validate_preflight(binding)
request = binding.request
projected = False
try:
self.credential.project()
projected = True
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.REQUESTED:
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.REQUESTED:
self.controller.advance(
binding.job_id,
submitted.job_id,
JobState.AUTHORIZED,
side_effect_state="AUTHORIZED",
receipt={"task_id": request.task_id, "original_grant_consumed": True},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.AUTHORIZED:
row = self.controller.store.read(submitted.job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(
binding.job_id,
lease_id=f"lease-{binding.job_id}",
submitted.job_id,
lease_id=f"lease-{submitted.job_id}",
lineage_anchor_oid=request.base_oid,
current_head_oid=request.expected_branch_oid,
generation=request.generation,
lease_expires_at=binding.lease_expires_at.astimezone(timezone.utc).isoformat(),
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.LEASED:
self.controller.advance(binding.job_id, JobState.WORKTREE_READY,
side_effect_state="WORKTREE_READY", expected_revision=int(row["revision"]))
row = self.controller.store.read(binding.job_id)
state = JobState(row["state"])
if state == JobState.WORKTREE_READY:
self.controller.advance(binding.job_id, JobState.EXECUTING,
side_effect_state="IN_PROGRESS", expected_revision=int(row["revision"]))
elif state != JobState.EXECUTING:
raise ContinuationExecutionError("JOB_EXECUTION_STATE_INVALID")
self.controller.advance(submitted.job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY")
self.controller.advance(submitted.job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
new_oid = self.workspace.commit_files(commit.files, commit.message)
if not OID_RE.fullmatch(new_oid) or not self.authority.is_ancestor(request.expected_branch_oid, new_oid):
raise ContinuationExecutionError("NON_FAST_FORWARD_COMMIT")
if request.branch_mode == "CREATE_NEW":
authority_oid = self.authority.create_task_branch_cas(request.target_branch, request.base_oid, new_oid)
else:
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
authority_oid = self.authority.compare_and_swap_branch(request.target_branch, request.expected_branch_oid, new_oid)
if authority_oid != new_oid:
raise ContinuationExecutionError("AUTHORITY_CAS_READBACK_MISMATCH")
self.controller.advance(
binding.job_id,
submitted.job_id,
JobState.LOCAL_VERIFIED,
side_effect_state="LOCAL_VERIFIED",
receipt={"authority_oid": authority_oid, "force_push": False},
)
try:
self.controller.advance(binding.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
if request.branch_mode == "CREATE_NEW":
self.remote.push_new_exact(request.target_branch, new_oid)
else:
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid)
self.controller.advance(submitted.job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS")
self.remote.push_exact(request.target_branch, request.expected_branch_oid, new_oid, allow_absent=binding.remote_branch_create_allowed)
except Exception:
return self._remote_pending(binding.job_id, authority_oid, new_oid)
return self._finish_remote_verification(
binding.job_id,
authority_oid,
new_oid,
before_complete=before_complete,
)
return self._remote_pending(submitted.job_id, authority_oid, new_oid)
return self._finish_remote_verification(submitted.job_id, authority_oid, new_oid)
finally:
if projected:
self.credential.revoke()
@@ -592,6 +529,7 @@ class ContinuationExecutor:
def _remote_pending(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"desired_remote_oid": new_oid,
"remote_verified": False,
"force_push": False,
@@ -604,14 +542,7 @@ class ContinuationExecutor:
)
return result.as_dict()
def _finish_remote_verification(
self,
job_id: str,
authority_oid: str,
new_oid: str,
*,
before_complete: Callable[[str], None] | None = None,
) -> dict[str, object]:
def _finish_remote_verification(self, job_id: str, authority_oid: str, new_oid: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
branch = row["branch"]
request = json.loads(row["request_json"])["binding"]
@@ -621,6 +552,7 @@ class ContinuationExecutor:
return self._remote_pending(job_id, authority_oid, new_oid)
receipt = {
"authority_oid": authority_oid,
"commit_oid": new_oid,
"git_remote_oid": git_oid,
"api_branch_oid": api_oid,
"commit_visible": True,
@@ -628,20 +560,8 @@ class ContinuationExecutor:
"reconciliation_decision": decision.action.value,
"force_push": False,
}
self.controller.advance(
job_id,
JobState.REMOTE_VERIFIED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
)
if before_complete is not None:
before_complete(new_oid)
return self.controller.advance(
job_id,
JobState.COMPLETED,
side_effect_state="REMOTE_VERIFIED",
receipt=receipt,
).as_dict()
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt=receipt)
return self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt=receipt).as_dict()
def reconcile(self, job_id: str) -> dict[str, object]:
row = self.controller.store.read(job_id)
@@ -681,11 +601,14 @@ class ContinuationExecutor:
and self.authority.is_ancestor(git_oid, new_oid)
):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_CHAIN")
self.remote.push_exact(row["branch"], git_oid, new_oid)
self.remote.push_exact(row["branch"], git_oid, new_oid, allow_absent=bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW")
return self._finish_remote_verification(job_id, new_oid, new_oid)
if authority_oid == new_oid and git_oid == api_oid == expected_oid:
allow_remote_create = bool(request.get("remote_branch_create_allowed")) or request.get("branch_mode") == "CREATE_NEW" or (git_oid is None and api_oid is None)
remote_matches_expected = git_oid == api_oid == expected_oid
remote_absent_for_create = git_oid is None and api_oid is None
if authority_oid == new_oid and (remote_matches_expected or remote_absent_for_create):
self.controller.advance(job_id, JobState.RECONCILING, side_effect_state="OBSERVATION_FIRST_RETRY")
self.remote.push_exact(row["branch"], expected_oid, new_oid)
self.remote.push_exact(row["branch"], expected_oid, new_oid, allow_absent=allow_remote_create)
return self._finish_remote_verification(job_id, new_oid, new_oid)
return self.controller.store._receipt(row, duplicate=True).as_dict()
finally:
@@ -49,7 +49,6 @@ class WriterLeaseRequestV2:
target_branch: str
generation: int
lease_seconds: int = 900
job_id: str = ""
schema = "aios.task-writer-lease.v2"
accepted_schemas = frozenset({"aios.task-writer-lease.v2", "aios.task-writer-lease-request.v2"})
@@ -57,8 +56,6 @@ class WriterLeaseRequestV2:
def validate(self, probe: BranchProbe) -> None:
if not self.task_id or self.repository != "aios/aios":
raise ContinuationError("request_identity_invalid")
if not self.job_id or not self.job_id.startswith("job-"):
raise ContinuationError("job_id_invalid")
if self.branch_mode not in {"CREATE_NEW", "CONTINUE_EXISTING"}:
raise ContinuationError("branch_mode_invalid")
_oid(self.base_oid, "base_oid")
@@ -10,12 +10,16 @@ def _copy(value: Mapping[str, Any] | None, keys: tuple[str, ...]) -> dict[str, A
def build_control_plane_snapshot(*, target_version: str, runtime: Mapping[str, Any] | None = None, repository: Mapping[str, Any] | None = None, production: Mapping[str, Any] | None = None, task: Mapping[str, Any] | None = None, capabilities: Mapping[str, Any] | None = None, credential_broker: Mapping[str, Any] | None = None) -> dict[str, Any]:
task_snapshot = _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state"))
for key in ("external_job_id", "commit_oid", "current_head_oid", "completed_replay_reconciliation_id", "receipt_authority_oid"):
if task is not None and key in task:
task_snapshot[key] = task[key]
return {
"version": {"target_release": target_version},
"runtime": _copy(runtime, ("bridge_schema_version", "control_plane_release_version", "runtime_release_commit", "runtime_path", "artifact_sha256", "drift_status")),
"repository": _copy(repository, ("identity", "authority_git_dir", "authority_main_oid")),
"production": _copy(production, ("baseline_version", "target_oid", "deployment_state")),
"task": _copy(task, ("task_id", "branch", "authority_branch_oid", "gitea_git_oid", "gitea_api_oid", "commit_visible", "generation", "reconciliation_state")),
"task": task_snapshot,
"capabilities": _copy(capabilities, ("legacy_writes_enabled", "governed_writes_enabled", "force_push", "main_write", "tag_write", "baseline_write", "production_write")),
"credential_broker": _copy(credential_broker, ("status", "provider")),
"secret_material_exposed": False,
@@ -0,0 +1,478 @@
"""Minimal convergence contracts for AIOS V2R2C2.
This module deliberately contains no external side effects. It provides the shared
data contracts and pure evaluation rules used by domain controllers. Domain
controllers remain authoritative for their own final states.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from enum import Enum
import hashlib
import json
from typing import Any, Iterable, Mapping, Sequence
class ConvergenceContractError(ValueError):
"""Raised when an invalid convergence contract is constructed."""
class EvidenceState(str, Enum):
PASS = "PASS"
FAIL = "FAIL"
UNKNOWN = "UNKNOWN"
BLOCKED = "BLOCKED"
NOT_APPLICABLE = "NOT_APPLICABLE"
class AcceptanceDecision(str, Enum):
ACCEPT = "ACCEPT"
REJECT = "REJECT"
BLOCKED = "BLOCKED"
UNKNOWN = "UNKNOWN"
class ObservedHealth(str, Enum):
HEALTHY = "HEALTHY"
DEGRADED = "DEGRADED"
FAILED = "FAILED"
UNKNOWN = "UNKNOWN"
class ReconciliationState(str, Enum):
PENDING = "PENDING"
OBSERVING = "OBSERVING"
DIFF_FOUND = "DIFF_FOUND"
APPLYING = "APPLYING"
VERIFYING = "VERIFYING"
CONVERGED = "CONVERGED"
DEGRADED = "DEGRADED"
BLOCKED = "BLOCKED"
class ContinuationStopReason(str, Enum):
CONTEXT_BOUNDARY = "context_boundary"
WORKER_RESTART = "worker_restart"
HANDOFF = "handoff"
AUTHORITY_GATE = "authority_gate"
WAIT_EXTERNAL = "wait_external"
def _utc_now() -> str:
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _sha256(value: Mapping[str, Any]) -> str:
return "sha256:" + hashlib.sha256(_canonical_json(value)).hexdigest()
def _nonempty(value: object, field_name: str) -> str:
text = str(value or "").strip()
if not text:
raise ConvergenceContractError(f"{field_name}_required")
return text
def _generation(value: object, field_name: str = "generation") -> int:
try:
parsed = int(value)
except (TypeError, ValueError) as exc:
raise ConvergenceContractError(f"{field_name}_invalid") from exc
if parsed < 1:
raise ConvergenceContractError(f"{field_name}_invalid")
return parsed
@dataclass(frozen=True, slots=True)
class StateOwnership:
object_type: str
authoritative_service: str
authoritative_store: str
final_writer: str | None = None
projections: tuple[str, ...] = ()
schema_version: str = "aios.state-ownership.v1"
def __post_init__(self) -> None:
_nonempty(self.object_type, "object_type")
_nonempty(self.authoritative_service, "authoritative_service")
_nonempty(self.authoritative_store, "authoritative_store")
if self.final_writer is not None:
_nonempty(self.final_writer, "final_writer")
def as_dict(self) -> dict[str, Any]:
return asdict(self)
DEFAULT_STATE_OWNERSHIP: tuple[StateOwnership, ...] = (
StateOwnership(
object_type="goal",
authoritative_service="goal-kernel",
authoritative_store="goal-store",
final_writer="goal-kernel",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="task_run",
authoritative_service="task-plane-18940",
authoritative_store="durable-task-store",
final_writer="task-plane-18940",
projections=("admin-web", "context-pack"),
),
StateOwnership(
object_type="release",
authoritative_service="release-controller",
authoritative_store="release-registry",
final_writer="release-controller",
projections=("baseline-authority", "admin-web"),
),
StateOwnership(
object_type="git_branch",
authoritative_service="authority-git-executor",
authoritative_store="authoritative-git",
final_writer="authority-git-executor",
projections=("gitea",),
),
StateOwnership(
object_type="external_projection",
authoritative_service="projection-controller",
authoritative_store="projection-store",
final_writer="projection-controller",
projections=("openapi", "gpt-action", "nginx"),
),
)
def ownership_index(
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
) -> dict[str, StateOwnership]:
result: dict[str, StateOwnership] = {}
for row in ownership:
if row.object_type in result:
raise ConvergenceContractError(f"duplicate_state_owner:{row.object_type}")
result[row.object_type] = row
return result
@dataclass(frozen=True, slots=True)
class AcceptanceCriterion:
criterion_id: str
required: bool
state: EvidenceState
verifier_id: str
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.criterion_id, "criterion_id")
_nonempty(self.verifier_id, "verifier_id")
@dataclass(frozen=True, slots=True)
class AcceptanceInvariant:
invariant_id: str
state: EvidenceState
evidence_refs: tuple[str, ...] = ()
def __post_init__(self) -> None:
_nonempty(self.invariant_id, "invariant_id")
@dataclass(frozen=True, slots=True)
class AcceptanceVector:
subject_type: str
subject_id: str
subject_revision: int
criteria: tuple[AcceptanceCriterion, ...]
invariants: tuple[AcceptanceInvariant, ...] = ()
unresolved_side_effects: tuple[str, ...] = ()
unresolved_blockers: tuple[str, ...] = ()
runtime_proofs: tuple[str, ...] = ()
schema: str = "aios.acceptance-vector.v1"
def __post_init__(self) -> None:
_nonempty(self.subject_type, "subject_type")
_nonempty(self.subject_id, "subject_id")
_generation(self.subject_revision, "subject_revision")
ids = [row.criterion_id for row in self.criteria]
if len(ids) != len(set(ids)):
raise ConvergenceContractError("duplicate_acceptance_criterion")
invariant_ids = [row.invariant_id for row in self.invariants]
if len(invariant_ids) != len(set(invariant_ids)):
raise ConvergenceContractError("duplicate_acceptance_invariant")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
for row in value["criteria"]:
row["state"] = row["state"].value
for row in value["invariants"]:
row["state"] = row["state"].value
return value
@dataclass(frozen=True, slots=True)
class AcceptanceResult:
decision: AcceptanceDecision
reasons: tuple[str, ...]
vector_hash: str
def as_dict(self) -> dict[str, Any]:
return {
"decision": self.decision.value,
"reasons": list(self.reasons),
"vector_hash": self.vector_hash,
}
def evaluate_acceptance(vector: AcceptanceVector) -> AcceptanceResult:
"""Pure fail-closed acceptance evaluator.
The evaluator never writes a domain final state. The domain owner consumes this
result and remains the only final writer for Goal/Release/Product state.
"""
reasons: list[str] = []
required = [row for row in vector.criteria if row.required]
for row in required:
if row.state == EvidenceState.FAIL:
reasons.append(f"criterion_failed:{row.criterion_id}")
elif row.state == EvidenceState.BLOCKED:
reasons.append(f"criterion_blocked:{row.criterion_id}")
elif row.state == EvidenceState.UNKNOWN:
reasons.append(f"criterion_unknown:{row.criterion_id}")
for row in vector.invariants:
if row.state == EvidenceState.FAIL:
reasons.append(f"invariant_failed:{row.invariant_id}")
elif row.state in {EvidenceState.UNKNOWN, EvidenceState.BLOCKED}:
reasons.append(f"invariant_unknown:{row.invariant_id}")
if vector.unresolved_side_effects:
reasons.append("unresolved_side_effects")
if vector.unresolved_blockers:
reasons.append("unresolved_blockers")
digest = _sha256(vector.as_dict())
if any(reason.startswith(("criterion_failed:", "invariant_failed:")) for reason in reasons):
decision = AcceptanceDecision.REJECT
elif any(reason.startswith("criterion_blocked:") for reason in reasons) or vector.unresolved_blockers:
decision = AcceptanceDecision.BLOCKED
elif reasons:
decision = AcceptanceDecision.UNKNOWN
else:
decision = AcceptanceDecision.ACCEPT
return AcceptanceResult(decision=decision, reasons=tuple(reasons), vector_hash=digest)
@dataclass(frozen=True, slots=True)
class DesiredState:
resource_id: str
resource_type: str
desired_generation: int
desired_spec: Mapping[str, Any]
owner_goal_id: str | None = None
lifecycle: str = "active"
schema: str = "aios.desired-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.resource_type, "resource_type")
_generation(self.desired_generation, "desired_generation")
if self.lifecycle not in {"active", "suspended", "deleted"}:
raise ConvergenceContractError("desired_lifecycle_invalid")
def identity(self) -> tuple[str, int]:
return self.resource_id, self.desired_generation
@dataclass(frozen=True, slots=True)
class ObservedState:
resource_id: str
observed_generation: int | None
observed_spec: Mapping[str, Any]
health: ObservedHealth
source_identity: str
observed_at: str
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.observed-state.v1"
def __post_init__(self) -> None:
_nonempty(self.resource_id, "resource_id")
_nonempty(self.source_identity, "source_identity")
_nonempty(self.observed_at, "observed_at")
if self.observed_generation is not None:
_generation(self.observed_generation, "observed_generation")
@dataclass(frozen=True, slots=True)
class ReconciliationPlan:
resource_id: str
desired_generation: int
state: ReconciliationState
diff: Mapping[str, Any]
observation_required: bool
idempotency_key: str
reason: str
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["state"] = self.state.value
return value
def plan_reconciliation(
desired: DesiredState,
observed: ObservedState | None,
) -> ReconciliationPlan:
"""Return a side-effect-free reconciliation decision.
UNKNOWN/missing observed state always requires observation and never converges.
"""
identity = {
"resource_id": desired.resource_id,
"desired_generation": desired.desired_generation,
"desired_spec": desired.desired_spec,
}
key = _sha256(identity)
if observed is None:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"observed": None},
True,
key,
"observed_state_missing",
)
if observed.resource_id != desired.resource_id:
raise ConvergenceContractError("resource_identity_mismatch")
if observed.health == ObservedHealth.UNKNOWN:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.OBSERVING,
{"health": observed.health.value},
True,
key,
"observed_state_unknown",
)
same_generation = observed.observed_generation == desired.desired_generation
same_spec = dict(observed.observed_spec) == dict(desired.desired_spec)
if same_generation and same_spec and observed.health == ObservedHealth.HEALTHY:
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.CONVERGED,
{},
False,
key,
"desired_equals_observed",
)
return ReconciliationPlan(
desired.resource_id,
desired.desired_generation,
ReconciliationState.DIFF_FOUND,
{
"desired_generation": desired.desired_generation,
"observed_generation": observed.observed_generation,
"desired_spec": dict(desired.desired_spec),
"observed_spec": dict(observed.observed_spec),
"observed_health": observed.health.value,
},
False,
key,
"desired_observed_drift",
)
@dataclass(frozen=True, slots=True)
class SideEffectReceipt:
mutation_id: str
external_system: str
state: str
observed_at: str
external_request_id: str | None = None
external_object_id: str | None = None
observed_result_hash: str | None = None
evidence_refs: tuple[str, ...] = ()
schema: str = "aios.side-effect-receipt.v1"
def __post_init__(self) -> None:
_nonempty(self.mutation_id, "mutation_id")
_nonempty(self.external_system, "external_system")
_nonempty(self.observed_at, "observed_at")
if self.state not in {
"confirmed_applied",
"confirmed_not_applied",
"unknown",
}:
raise ConvergenceContractError("side_effect_receipt_state_invalid")
@dataclass(frozen=True, slots=True)
class ContinuationManifest:
trace_id: str
goal_id: str
goal_revision: int
task_id: str
run_id: str
current_node_id: str
acceptance_vector_ref: str
next_action: str
stop_reason: ContinuationStopReason
checkpoint_ref: str | None = None
unresolved_blocker_refs: tuple[str, ...] = ()
authoritative_decision_refs: tuple[str, ...] = ()
resource_refs: tuple[str, ...] = ()
artifact_refs: tuple[str, ...] = ()
evidence_refs: tuple[str, ...] = ()
created_at: str = field(default_factory=_utc_now)
schema: str = "aios.continuation-manifest.v1"
def __post_init__(self) -> None:
for name in (
"trace_id",
"goal_id",
"task_id",
"run_id",
"current_node_id",
"acceptance_vector_ref",
"next_action",
"created_at",
):
_nonempty(getattr(self, name), name)
_generation(self.goal_revision, "goal_revision")
def as_dict(self) -> dict[str, Any]:
value = asdict(self)
value["stop_reason"] = self.stop_reason.value
return value
@property
def content_hash(self) -> str:
return _sha256(self.as_dict())
def validate_state_ownership(
object_type: str,
writer: str,
ownership: Iterable[StateOwnership] = DEFAULT_STATE_OWNERSHIP,
*,
final_state: bool = False,
) -> None:
row = ownership_index(ownership).get(object_type)
if row is None:
raise ConvergenceContractError(f"state_owner_missing:{object_type}")
expected = row.final_writer if final_state and row.final_writer else row.authoritative_service
if writer != expected:
raise ConvergenceContractError(
f"STATE_OWNERSHIP_VIOLATION:{object_type}:{writer}:{expected}"
)
+4 -50
View File
@@ -6,7 +6,6 @@ from datetime import datetime, timedelta, timezone
import os
from pathlib import Path
import secrets
import re
from typing import Any, Callable, Mapping
@@ -22,14 +21,9 @@ class CredentialLease:
scopes: tuple[str, ...]
provider: str
state: str = "ACTIVE"
scope_type: str = "system"
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
run_id: str | None = None
def as_dict(self) -> dict[str, Any]:
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state, "scope_type": self.scope_type, "tenant_id": self.tenant_id, "user_id": self.user_id, "project_id": self.project_id, "run_id": self.run_id}
return {"lease_id": self.lease_id, "task_id": self.task_id, "repository": self.repository, "target_branch": self.target_branch, "generation": self.generation, "issued_at": self.issued_at, "expires_at": self.expires_at, "scopes": list(self.scopes), "provider": self.provider, "state": self.state}
class FileCredentialProvider:
@@ -49,19 +43,11 @@ class CredentialBroker:
self.now = now or (lambda: datetime.now(timezone.utc))
self._leases: dict[str, CredentialLease] = {}
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> dict[str, Any]:
def issue(self, *, task_id: str, repository: str, target_branch: str, generation: int, scopes: tuple[str, ...] = ("git:push", "gitea:read"), lease_seconds: int = 900) -> dict[str, Any]:
if repository != "aios/aios" or not target_branch.startswith("agent/") or generation < 1 or not 1 <= lease_seconds <= 86400:
raise ValueError("CREDENTIAL_LEASE_BINDING_INVALID")
if scope_type == "tenant":
if not tenant_id:
raise ValueError("CREDENTIAL_SCOPE_DENY")
elif scope_type in {"system", "global"}:
if any((tenant_id, user_id, project_id, run_id)):
raise ValueError("CREDENTIAL_SCOPE_DENY")
else:
raise ValueError("CREDENTIAL_SCOPE_DENY")
now = self.now().astimezone(timezone.utc)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__, "ACTIVE", scope_type, tenant_id, user_id, project_id, run_id)
lease = CredentialLease("lease-" + secrets.token_hex(16), task_id, repository, target_branch, generation, now.isoformat(), (now + timedelta(seconds=lease_seconds)).isoformat(), tuple(scopes), type(self.provider).__name__)
self._leases[lease.lease_id] = lease
return lease.as_dict()
@@ -78,39 +64,7 @@ class CredentialBroker:
lease = self._get(lease_id)
if lease.state != "ACTIVE":
raise RuntimeError("CREDENTIAL_LEASE_NOT_ACTIVE")
environment = self.provider.environment(lease)
for key, value in environment.items():
upper = str(key).upper()
if not isinstance(value, str):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
sensitive = any(marker in upper for marker in ("CREDENTIAL", "TOKEN", "SECRET", "PASSWORD", "PRIVATE_KEY", "API_KEY"))
if re.search(r"(?i)bearer\s+\S+|(?:token|secret|password|api[_-]?key)\s*[:=]", value):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if sensitive and upper.endswith("_FILE"):
candidate = Path(value)
credential_root = Path("/run/credentials")
try:
resolved = candidate.resolve(strict=False)
except OSError as exc:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY") from exc
if not candidate.is_absolute() or not (resolved == credential_root or credential_root in resolved.parents):
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive and upper.endswith("_REFERENCE"):
if re.fullmatch(r"(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}", value) is None:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
elif sensitive:
raise RuntimeError("CREDENTIAL_SECRET_EXPOSURE_DENY")
return environment
def authorize(self, lease_id: str, *, task_id: str, operation: str, scope_type: str = "system", tenant_id: str | None = None, user_id: str | None = None, project_id: str | None = None, run_id: str | None = None) -> Mapping[str, str]:
lease = self._get(lease_id)
if lease.state != "ACTIVE" or lease.task_id != task_id or operation not in lease.scopes:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
requested = (scope_type, tenant_id, user_id, project_id, run_id)
actual = (lease.scope_type, lease.tenant_id, lease.user_id, lease.project_id, lease.run_id)
if requested != actual:
raise RuntimeError("CREDENTIAL_SCOPE_DENY")
return self.environment(lease_id)
return self.provider.environment(lease)
def status(self, lease_id: str) -> dict[str, Any]:
return self._get(lease_id).as_dict()
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -24,6 +24,25 @@ def _safe_oid(value: str, field: str) -> str:
return value
def _safe_branch(value: str) -> str:
branch = str(value or "")
if (
not branch
or len(branch) > 240
or branch != branch.strip()
or branch.startswith(("/", "-", "."))
or branch.endswith(("/", ".", ".lock"))
or ".." in branch
or "//" in branch
or "@{" in branch
or "\\" in branch
or any(part in {"", ".", ".."} for part in branch.split("/"))
or any(ch not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._/-" for ch in branch)
):
raise GiteaTransportError("BRANCH_INVALID")
return branch
@dataclass(slots=True)
class GiteaMachineTransport:
api_base: str
@@ -91,6 +110,7 @@ class GiteaMachineTransport:
return self.repository_info()
def read_branch(self, branch: str) -> str | None:
branch = _safe_branch(branch)
owner, name = self.repository_id.split("/", 1)
try:
value = self._request("GET", f"/repos/{quote(owner)}/{quote(name)}/branches/{quote(branch, safe='/')}")[1]
@@ -136,6 +156,99 @@ class GiteaMachineTransport:
finally:
askpass.unlink(missing_ok=True)
def fetch_branch_exact(self, branch: str, expected_oid: str) -> str:
"""Fetch one exact remote branch tip into a quarantine ref.
This never updates ``main`` or any caller-visible branch. The remote
ref is read before and after the fetch so a concurrent branch move
cannot be silently admitted.
"""
branch = _safe_branch(branch)
expected_oid = _safe_oid(expected_oid, "expected_oid")
remote_before = self.read_branch(branch)
if remote_before != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MISMATCH")
target_ref = f"refs/aios/release-sources/{expected_oid}"
current = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"rev-parse",
"--verify",
f"{target_ref}^{{commit}}",
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
if current.returncode == 0 and current.stdout.strip().lower() == expected_oid:
remote_after = self.read_branch(branch)
if remote_after != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MOVED")
return expected_oid
askpass, additions = self._askpass()
environment = os.environ.copy()
environment.update(additions)
try:
result = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"fetch",
"--no-tags",
"--no-write-fetch-head",
self.remote_url,
f"refs/heads/{branch}:{target_ref}",
],
text=True,
capture_output=True,
check=False,
env=environment,
timeout=self.timeout,
)
if result.returncode != 0:
raise GiteaTransportError("GITEA_SOURCE_FETCH_FAILED")
finally:
askpass.unlink(missing_ok=True)
local = subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"rev-parse",
"--verify",
f"{target_ref}^{{commit}}",
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
local_oid = local.stdout.strip().lower()
if local.returncode != 0 or local_oid != expected_oid:
subprocess.run(
[
"git",
f"--git-dir={self.authority_git_dir}",
"update-ref",
"-d",
target_ref,
],
text=True,
capture_output=True,
check=False,
timeout=self.timeout,
)
raise GiteaTransportError("GITEA_SOURCE_FETCH_MISMATCH")
remote_after = self.read_branch(branch)
if remote_after != expected_oid:
raise GiteaTransportError("GITEA_SOURCE_REF_MOVED")
return expected_oid
def compare_and_swap_branch(self, branch: str, expected_oid: str, new_oid: str) -> str:
_safe_oid(expected_oid, "expected_oid")
_safe_oid(new_oid, "new_oid")
File diff suppressed because it is too large Load Diff
+363 -124
View File
@@ -4,15 +4,16 @@ from __future__ import annotations
import argparse
import base64
from dataclasses import dataclass, replace
from dataclasses import dataclass, field, replace
from datetime import datetime, timedelta, timezone
import json
from pathlib import Path
from typing import Any, Mapping
from typing import Any, Callable, Mapping
from aios_authority_git_executor import AuthorityGitExecutor, AuthorityGitError
from aios_campaign_recovery import RecoveryEvidenceError, load_recovered_authorization
from aios_gitea_machine_broker import CampaignCapabilityReceipt, GiteaMachineBroker, MachineBrokerError
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore
from aios_gitops_job_controller import GitOpsJobController, JobError, JobState, JobStore, ServiceUnavailable
TASK_ID = "AIOS_AUTONOMOUS_RELEASE_CAMPAIGN_V1"
@@ -34,6 +35,17 @@ class DurableJobExecutor:
authority: AuthorityGitExecutor
broker: GiteaMachineBroker
recovery_path: Path
clock: Callable[[], datetime] = field(default=lambda: datetime.now(timezone.utc))
lease_seconds: int = 60
_persistence_halted: bool = field(default=False, init=False, repr=False)
def __post_init__(self) -> None:
try:
self.controller.assert_effect_admission()
except JobError as exc:
if str(exc) != "EFFECT_ADMISSION_LATCHED":
raise
self._persistence_halted = True
def _request(self, job_id: str) -> tuple[dict[str, Any], dict[str, Any]]:
row = self.controller.store.read(job_id)
@@ -41,7 +53,14 @@ class DurableJobExecutor:
request = json.loads(row["request_json"])
except (KeyError, TypeError, json.JSONDecodeError) as exc:
raise JobExecutionError("JOB_REQUEST_INVALID") from exc
return row, dict(request.get("binding") or {})
binding = dict(request.get("binding") or {})
# Payload location is derived from and protected by payload_digest, but
# is intentionally outside the idempotency binding. Execution still
# needs it after a durable round trip.
for field_name in ("payload_ref", "payload_size"):
if field_name in request:
binding[field_name] = request[field_name]
return row, binding
def _validate_campaign(self, request: Mapping[str, Any]) -> str:
operation = str(request.get("operation", ""))
@@ -89,151 +108,372 @@ class DurableJobExecutor:
raise JobExecutionError("NEW_OID_NOT_DESCENDANT")
if not self.authority.ancestry_check(EXPECTED_BRANCH_OID, new_oid):
raise JobExecutionError("NEW_OID_OUTSIDE_AUTHORIZED_LINEAGE")
if operation == "RECONCILE_AUTHORITY_MAIN":
old_main = request.get("expected_main_oid")
if (
not isinstance(old_main, str)
or len(old_main) != 40
or any(ch not in "0123456789abcdef" for ch in old_main)
):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
if not self.authority.ancestry_check(old_main, new_oid):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
identity = self.broker.verify_identity()
if identity.get("repository") != REPOSITORY or identity.get("login") == "aiman":
raise JobExecutionError("MACHINE_IDENTITY_SCOPE_INVALID")
return new_oid
def dry_run(self, job_id: str) -> dict[str, Any]:
_row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
remote = self.broker.readback()
if remote is None:
action = "CREATE_REQUIRED"
elif remote == new_oid:
action = "ALREADY_IN_SYNC"
elif remote == request["expected_branch_oid"]:
action = "FAST_FORWARD_REQUIRED"
else:
action = "MANUAL_OWNER_REVIEW_REQUIRED"
if action == "MANUAL_OWNER_REVIEW_REQUIRED":
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
return {
row, request = self._request(job_id)
operation = str(request.get("operation") or "")
if operation not in {
"ADVANCE_EXISTING_CAMPAIGN_BRANCH", "COMMIT_FILE",
"SYNC_TASK_BRANCH_TO_GITEA", "CREATE_PR", "PUBLISH_CHECK",
"PROTECTED_MERGE", "RECONCILE_AUTHORITY_MAIN",
}:
raise JobExecutionError("OPERATION_NOT_ALLOWED")
self._validate_campaign(request)
if operation == "COMMIT_FILE":
self._commit_file_spec(row, request)
observation = self._observe_operation(request)
result = {
"mode": "DRY_RUN",
"operation": operation,
"decision": "SAFE_TO_ADVANCE",
"authority_action": "CAS_REQUIRED",
"gitea_action": action,
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": new_oid,
"force_push": False,
"effect_performed": False,
"readback": observation,
}
def _recover_requested(self, job_id: str) -> dict[str, Any]:
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.resume(job_id)
row = self.controller.store.read(job_id)
return row
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
result.update({
"authority_action": "CAS_REQUIRED",
"gitea_action": "ALREADY_IN_SYNC" if observation["verification"] == "VERIFIED" else "FAST_FORWARD_REQUIRED",
"previous_branch_oid": request["expected_branch_oid"],
"final_branch_oid": request["new_branch_oid"],
"force_push": False,
})
return result
def execute(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
if dry_run:
return self.dry_run(job_id)
if self._persistence_halted:
raise JobExecutionError("PERSISTENCE_FAILURE_LATCHED")
try:
return self._execute_unprotected(job_id, dry_run=dry_run)
return self._execute_unprotected(job_id)
except ServiceUnavailable:
self._persistence_halted = True
raise
except Exception as exc:
try:
row = self.controller.store.read(job_id)
state = JobState(row["state"])
if state not in {JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED, JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED}:
self.controller.advance(job_id, JobState.RECOVERABLE_FAILURE, side_effect_state="UNKNOWN", receipt={"error_code": type(exc).__name__})
except Exception:
pass
if state in {
JobState.EXECUTING, JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING, JobState.REMOTE_VERIFIED,
}:
self.controller.advance(
job_id,
JobState.RECONCILIATION_REQUIRED,
side_effect_state="UNKNOWN",
receipt={"error_code": type(exc).__name__, "verification": "UNKNOWN"},
)
elif state not in {
JobState.COMPLETED, JobState.MANUAL_OWNER_REVIEW_REQUIRED,
JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED,
}:
self.controller.advance(
job_id, JobState.RECOVERABLE_FAILURE,
side_effect_state="NONE",
receipt={"error_code": type(exc).__name__, "effect_performed": False},
)
except ServiceUnavailable:
self._persistence_halted = True
raise
def _execute_unprotected(self, job_id: str, *, dry_run: bool = False) -> dict[str, Any]:
def _execute_unprotected(self, job_id: str) -> dict[str, Any]:
row, request = self._request(job_id)
operation = request.get("operation")
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
if dry_run:
return self.dry_run(job_id)
return self._advance_campaign(job_id)
if operation == "COMMIT_FILE":
return self._commit_file(job_id, request)
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
new_oid = self._validate_campaign(request)
final = self.broker.push_branch(new_oid)
return {"operation": operation, "remote_verified": final == new_oid, "final_network_oid": final}
if operation == "CREATE_PR":
self._validate_campaign(request)
return dict(self.broker.create_pr("AIOS autonomous release control plane", "Durable campaign continuation; exact head and recovery evidence are bound in the Job."))
if operation == "PUBLISH_CHECK":
self._validate_campaign(request)
context = request.get("check_context", "aios/durable-release")
state = request.get("check_state", "success")
description = request.get("check_description", "Durable campaign control-plane check passed")
return dict(self.broker.status(str(state), str(description), str(context)))
if operation == "PROTECTED_MERGE":
self._validate_campaign(request)
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
expected_main = request.get("expected_main_oid")
if expected_main is not None:
self.broker.transport.expected_merge_base = expected_main
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
self._validate_campaign(request)
old = request.get("expected_main_oid")
new = request.get("new_branch_oid")
if not isinstance(old, str) or not isinstance(new, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
gitea_main = self.broker.transport.read_branch("main")
if gitea_main != new:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
if not self.authority.ancestry_check(old, new):
raise JobExecutionError("MAIN_OID_NOT_DESCENDANT")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new)}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
state = JobState(row["state"])
if state == JobState.COMPLETED:
return self.controller.store._receipt(row, duplicate=True).as_dict()
if state in {
JobState.RECOVERABLE_FAILURE, JobState.RECONCILIATION_REQUIRED,
JobState.RECONCILING, JobState.EXECUTING, JobState.LOCAL_VERIFIED,
JobState.REMOTE_SYNCING, JobState.REMOTE_VERIFIED,
}:
return self.reconcile(job_id)
def _advance_campaign(self, job_id: str) -> dict[str, Any]:
self._recover_requested(job_id)
row, request = self._request(job_id)
new_oid = self._validate_campaign(request)
expected_oid = request["expected_branch_oid"]
operation = str(request.get("operation") or "")
if operation not in {
"ADVANCE_EXISTING_CAMPAIGN_BRANCH", "COMMIT_FILE",
"SYNC_TASK_BRANCH_TO_GITEA", "CREATE_PR", "PUBLISH_CHECK",
"PROTECTED_MERGE", "RECONCILE_AUTHORITY_MAIN",
}:
raise JobExecutionError("OPERATION_NOT_ALLOWED")
try:
self.controller.assert_effect_admission(exclude_job_id=job_id)
except JobError as exc:
if str(exc) == "EFFECT_ADMISSION_LATCHED":
self._persistence_halted = True
raise JobExecutionError("PERSISTENCE_FAILURE_LATCHED") from exc
raise
self._validate_campaign(request)
if operation == "COMMIT_FILE":
self._commit_file_spec(row, request)
row = self._prepare_intent(job_id, row, request)
generation = int(request.get("generation", "1"))
self.controller.assert_execution_lease(job_id, generation=generation)
before = self._observe_operation(request)
if before["verification"] == "VERIFIED":
return self._complete(job_id, request, {"reconciled_before_effect": True}, before)
if before["verification"] != "ABSENT":
return self._record_unknown(job_id, request, before)
effect = self._perform_effect(job_id, row, request, generation)
after = self._observe_operation(request)
if after["verification"] != "VERIFIED":
return self._record_unknown(job_id, request, after, effect=effect)
return self._complete(job_id, request, effect, after)
def _prepare_intent(
self, job_id: str, row: Mapping[str, Any], request: Mapping[str, Any]
) -> dict[str, Any]:
state = JobState(row["state"])
if state == JobState.REQUESTED:
receipt = self.controller.advance(job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED", receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID})
self.controller.advance(
job_id, JobState.AUTHORIZED, side_effect_state="AUTHORIZED",
receipt={"authorization_digest": AUTHORIZATION_DIGEST, "task_id": TASK_ID},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.AUTHORIZED:
self.controller.lease_job(job_id, lease_id=f"lease-{job_id}", lineage_anchor_oid=LINEAGE_ANCHOR, current_head_oid=expected_oid, generation=1, lease_expires_at="2099-01-01T00:00:00Z", expected_revision=int(row["revision"]))
state = JobState(row["state"])
if state == JobState.AUTHORIZED:
deadline = self.clock().astimezone(timezone.utc) + timedelta(seconds=self.lease_seconds)
self.controller.lease_job(
job_id,
lease_id=f"lease-{job_id}",
lineage_anchor_oid=LINEAGE_ANCHOR,
current_head_oid=str(request["expected_branch_oid"]),
generation=int(request.get("generation", "1")),
lease_expires_at=deadline.isoformat(),
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.LEASED:
self.controller.advance(job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY", receipt={"branch": BRANCH, "expected_oid": expected_oid})
state = JobState(row["state"])
if state == JobState.LEASED:
self.controller.advance(
job_id, JobState.WORKTREE_READY, side_effect_state="WORKTREE_READY",
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.WORKTREE_READY:
self.controller.advance(job_id, JobState.EXECUTING, side_effect_state="IN_PROGRESS")
state = JobState(row["state"])
if state == JobState.WORKTREE_READY:
self.controller.advance(
job_id, JobState.EXECUTING, side_effect_state="INTENT_COMMITTED",
receipt={
"effect_id": job_id,
"operation": request["operation"],
"input_digest": row["payload_digest"],
"intent_committed": True,
},
expected_revision=int(row["revision"]),
)
row = self.controller.store.read(job_id)
authority_before = self.authority.read_ref(f"refs/heads/{BRANCH}")
if authority_before == expected_oid:
authority_oid = self.authority.advance_task_branch_CAS(f"refs/heads/{BRANCH}", expected_oid, new_oid)
elif authority_before == new_oid:
authority_oid = new_oid
else:
raise JobExecutionError("AUTHORITY_BRANCH_CONFLICT")
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.EXECUTING:
self.controller.advance(job_id, JobState.LOCAL_VERIFIED, side_effect_state="LOCAL_VERIFIED", receipt={"authority": {"branch": BRANCH, "previous_oid": expected_oid, "new_oid": new_oid, "CAS_success": True, "readback_oid": authority_oid}})
row = self.controller.store.read(job_id)
remote_before = self.broker.readback()
if remote_before != new_oid:
if remote_before not in {None, expected_oid}:
raise JobExecutionError("GITEA_BRANCH_CONFLICT")
if JobState(row["state"]) == JobState.LOCAL_VERIFIED:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected_oid)
self.controller.advance(job_id, JobState.REMOTE_SYNCING, side_effect_state="IN_PROGRESS", receipt={"gitea": {"previous_oid": remote_before, "desired_oid": new_oid, "force_push": False}})
final_network = self.broker.push_branch(new_oid)
else:
final_network = new_oid
final_api = self.broker.readback()
if final_api != new_oid:
raise JobExecutionError("GITEA_BRANCH_READBACK_MISMATCH")
row = self.controller.store.read(job_id)
if JobState(row["state"]) in {JobState.LOCAL_VERIFIED, JobState.REMOTE_SYNCING}:
self.controller.advance(job_id, JobState.REMOTE_VERIFIED, side_effect_state="REMOTE_VERIFIED", receipt={"gitea": {"machine_actor": "aios-autonomy-agent", "previous_oid": remote_before, "desired_oid": new_oid, "final_network_oid": final_network, "final_API_oid": final_api, "force_push": False}})
row = self.controller.store.read(job_id)
if JobState(row["state"]) == JobState.REMOTE_VERIFIED:
self.controller.advance(job_id, JobState.COMPLETED, side_effect_state="REMOTE_VERIFIED", receipt={"authority_oid": authority_oid, "network_oid": final_network, "gitea_api_oid": final_api, "force_push": False})
return self.controller.store._receipt(self.controller.store.read(job_id), duplicate=True).as_dict()
if JobState(row["state"]) != JobState.EXECUTING:
raise JobExecutionError("JOB_EXECUTION_STATE_INVALID")
return dict(row)
def _commit_file(self, job_id: str, request: Mapping[str, Any]) -> dict[str, Any]:
def _perform_effect(
self,
job_id: str,
row: Mapping[str, Any],
request: Mapping[str, Any],
generation: int,
) -> dict[str, Any]:
operation = str(request["operation"])
new_oid = str(request["new_branch_oid"])
self.controller.assert_execution_lease(job_id, generation=generation)
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
expected = str(request["expected_branch_oid"])
current = self.authority.read_ref(f"refs/heads/{BRANCH}")
authority_oid = new_oid if current == new_oid else self.authority.advance_task_branch_CAS(
f"refs/heads/{BRANCH}", expected, new_oid
)
self.controller.assert_execution_lease(job_id, generation=generation)
remote = self.broker.readback()
if remote != new_oid:
if hasattr(self.broker, "receipt"):
self.broker.receipt = replace(self.broker.receipt, expected_oid=expected)
self.broker.push_branch(new_oid)
return {"authority_oid": authority_oid, "network_oid": new_oid, "force_push": False}
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
return {"final_network_oid": self.broker.push_branch(new_oid), "force_push": False}
if operation == "CREATE_PR":
return dict(self.broker.create_pr(
"AIOS autonomous release control plane",
"Durable campaign continuation; exact head and recovery evidence are bound in the Job.",
))
if operation == "PUBLISH_CHECK":
return dict(self.broker.status(
str(request.get("check_state", "success")),
str(request.get("check_description", "Durable campaign control-plane check passed")),
str(request.get("check_context", "aios/durable-release")),
))
if operation == "PROTECTED_MERGE":
number = int(request.get("pr_number", "0"))
self.broker.transport.expected_merge_head = request["expected_branch_oid"]
if request.get("expected_main_oid") is not None:
self.broker.transport.expected_merge_base = request["expected_main_oid"]
return dict(self.broker.merge(number))
if operation == "RECONCILE_AUTHORITY_MAIN":
old = request.get("expected_main_oid")
if not isinstance(old, str):
raise JobExecutionError("MAIN_RECONCILIATION_BINDING_INVALID")
if self.broker.transport.read_branch("main") != new_oid:
raise JobExecutionError("GITEA_MAIN_EXPECTATION_MISMATCH")
return {"authority_main": self.authority.ff_CAS_main_after_verified_protected_merge(old, new_oid)}
if operation == "COMMIT_FILE":
path, content = self._commit_file_spec(row, request)
return {
"new_oid": self.authority.commit_file_in_isolated_worktree(
base_oid=row["expected_branch_oid"], files={path: content},
message="chore(gitops): durable commit-file", worktree_name=job_id,
)
}
raise JobExecutionError("OPERATION_NOT_ALLOWED")
def _observe_operation(self, request: Mapping[str, Any]) -> dict[str, Any]:
operation = str(request["operation"])
new_oid = str(request["new_branch_oid"])
if operation == "ADVANCE_EXISTING_CAMPAIGN_BRANCH":
authority = self.authority.read_ref(f"refs/heads/{BRANCH}")
remote = self.broker.readback()
if authority == new_oid and remote == new_oid:
return {"verification": "VERIFIED", "authority_oid": authority, "remote_oid": remote}
expected = request["expected_branch_oid"]
if authority == expected and remote in {None, expected}:
return {"verification": "ABSENT", "authority_oid": authority, "remote_oid": remote}
return {"verification": "UNKNOWN", "blocker": "PARTIAL_OR_CONFLICTING_BRANCH_EFFECT", "authority_oid": authority, "remote_oid": remote}
if operation == "SYNC_TASK_BRANCH_TO_GITEA":
remote = self.broker.readback()
if remote == new_oid:
return {"verification": "VERIFIED", "remote_oid": remote}
if remote in {None, EXPECTED_BRANCH_OID}:
return {"verification": "ABSENT", "remote_oid": remote}
return {"verification": "UNKNOWN", "blocker": "REMOTE_BRANCH_CONFLICT", "remote_oid": remote}
if operation == "RECONCILE_AUTHORITY_MAIN":
current = self.authority.read_ref("refs/heads/main")
if current == new_oid:
return {"verification": "VERIFIED", "authority_main": current}
if current == request.get("expected_main_oid"):
return {"verification": "ABSENT", "authority_main": current}
return {"verification": "UNKNOWN", "blocker": "AUTHORITY_MAIN_CONFLICT", "authority_main": current}
if operation == "CREATE_PR" and hasattr(self.broker, "read_pr"):
value = self.broker.read_pr(BRANCH, "main", new_oid)
return {"verification": "VERIFIED" if value else "ABSENT", "pr": value}
if operation == "PUBLISH_CHECK" and hasattr(self.broker, "read_status"):
value = self.broker.read_status(
new_oid, str(request.get("check_context", "aios/durable-release"))
)
if value is None:
return {"verification": "ABSENT"}
expected_state = str(request.get("check_state", "success"))
return {"verification": "VERIFIED" if value.get("status") == expected_state else "UNKNOWN", "status": dict(value), **({} if value.get("status") == expected_state else {"blocker": "STATUS_READBACK_MISMATCH"})}
if operation == "PROTECTED_MERGE" and hasattr(self.broker, "read_merge"):
value = self.broker.read_merge(int(request.get("pr_number", "0")))
if value is None:
return {"verification": "ABSENT"}
if value.get("merged") is True and value.get("head_oid") == request["expected_branch_oid"]:
return {"verification": "VERIFIED", "merge": dict(value)}
if value.get("merged") is False:
return {"verification": "ABSENT", "merge": dict(value)}
return {"verification": "UNKNOWN", "blocker": "MERGE_READBACK_MISMATCH", "merge": dict(value)}
return {"verification": "UNKNOWN", "blocker": "EXACT_READBACK_UNSUPPORTED"}
def _complete(
self, job_id: str, request: Mapping[str, Any], effect: Mapping[str, Any], observation: Mapping[str, Any]
) -> dict[str, Any]:
row = self.controller.store.read(job_id)
receipt = {
"effect_id": job_id,
"operation": request["operation"],
"effect": dict(effect),
"verification": "VERIFIED",
"readback": dict(observation),
}
completed = self.controller.advance(
job_id, JobState.COMPLETED, side_effect_state="VERIFIED",
receipt=receipt, expected_revision=int(row["revision"]),
)
return completed.as_dict()
def _record_unknown(
self,
job_id: str,
request: Mapping[str, Any],
observation: Mapping[str, Any],
*,
effect: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
row = self.controller.store.read(job_id)
receipt = {
"effect_id": job_id,
"operation": request["operation"],
"effect": dict(effect) if effect is not None else None,
"verification": "UNKNOWN",
"blocker": observation.get("blocker", "READBACK_INCONCLUSIVE"),
"readback": dict(observation),
}
pending = self.controller.advance(
job_id, JobState.RECONCILIATION_REQUIRED, side_effect_state="UNKNOWN",
receipt=receipt, expected_revision=int(row["revision"]),
)
return pending.as_dict()
def reconcile(self, job_id: str) -> dict[str, Any]:
def observer(_current: Mapping[str, Any]) -> Mapping[str, Any]:
_row, request = self._request(job_id)
self._validate_campaign(request)
observation = self._observe_operation(request)
if observation["verification"] == "VERIFIED":
return {
"state": JobState.COMPLETED.value,
"side_effect_state": "VERIFIED",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"effect": None, "verification": "VERIFIED",
"readback": observation, "reconciled": True,
},
}
if observation["verification"] == "ABSENT":
return {
"state": JobState.REQUESTED.value,
"side_effect_state": "RECONCILED_ABSENT",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"verification": "ABSENT", "readback": observation,
"reconciled": True,
},
}
return {
"state": JobState.RECONCILIATION_REQUIRED.value,
"side_effect_state": "UNKNOWN",
"receipt": {
"effect_id": job_id, "operation": request["operation"],
"verification": "UNKNOWN",
"blocker": observation.get("blocker", "READBACK_INCONCLUSIVE"),
"readback": observation, "reconciled": True,
},
}
result = self.controller.reconcile(job_id, observer).as_dict()
if result["state"] in {JobState.COMPLETED.value, JobState.REQUESTED.value}:
self._persistence_halted = False
return result
def _commit_file_spec(
self, row: Mapping[str, Any], request: Mapping[str, Any]
) -> tuple[str, bytes]:
payload_ref = request.get("payload_ref")
if not payload_ref:
raise JobExecutionError("PAYLOAD_REQUIRED")
@@ -244,8 +484,7 @@ class DurableJobExecutor:
content = base64.b64decode(spec["content_base64"], validate=True)
except Exception as exc:
raise JobExecutionError("COMMIT_FILE_PAYLOAD_INVALID") from exc
new_oid = self.authority.commit_file_in_isolated_worktree(base_oid=row["expected_branch_oid"], files={path: content}, message="chore(gitops): durable commit-file", worktree_name=job_id)
return {"operation": "COMMIT_FILE", "new_oid": new_oid}
return path, content
def build_from_environment(job_root: Path) -> DurableJobExecutor:
@@ -0,0 +1,32 @@
"""Collision-resistant storage keys for durable governed Task state."""
from __future__ import annotations
import hashlib
import re
class JobIdentityError(RuntimeError):
pass
def legacy_task_storage_key(task_id: str) -> str:
"""Return the historical lossy key solely to detect migration conflicts."""
if not isinstance(task_id, str) or not task_id:
raise JobIdentityError("TASK_ID_INVALID")
return re.sub(r"[^A-Za-z0-9_.-]+", "-", task_id).strip("-")[:120] or "task"
def task_storage_key(task_id: str) -> str:
"""Return a canonical collision-resistant key for every Task state sink."""
if (
not isinstance(task_id, str)
or not task_id
or task_id != task_id.strip()
or len(task_id) > 512
):
raise JobIdentityError("TASK_ID_INVALID")
readable = re.sub(r"[^A-Za-z0-9_.-]+", "-", task_id).strip("-")[:48] or "task"
digest = hashlib.sha256(
("aios-task-storage-v1\0" + task_id).encode("utf-8")
).hexdigest()
return f"{readable}-{digest}"
+7 -1
View File
@@ -5,6 +5,8 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from typing import Any
from release_version import validate_release_version
ROOT = Path(os.getenv('AIOS_GITOPS_ROOT', '/data/aios-gitops'))
WORKSPACE = Path(os.getenv('AIOS_GITOPS_WORKSPACE', str(ROOT / 'workspace')))
DB_PATH = Path(os.getenv('AIOS_GITOPS_DB', str(ROOT / 'runtime' / 'gitops_model.sqlite3')))
@@ -218,7 +220,11 @@ class Handler(BaseHTTPRequestHandler):
a = self.require_write(['owner']);
if not a: return
ver=data.get('version') or data.get('tag')
if not re.match(r'^V\d+R\d+C\d+(-[a-zA-Z0-9_.-]+)?$', ver or ''): self.send_json({'error':'version must match VxRxCx'},400); return
try:
validate_release_version(ver)
except ValueError:
self.send_json({'error':'version must match VxRxCx'},400)
return
msg=data.get('message') or f'release {ver}'; out=git_cmd(['tag','-a',ver,'-m',msg]);
if data.get('push',False): git_cmd(['push','origin',ver])
level='V' if re.match(r'^V[1-9]\d*R0C0', ver) else ('R' if re.match(r'^V\d+R[1-9]\d*C0', ver) else 'C'); exec_sql('INSERT OR REPLACE INTO gitops_versions(version,tag,level,summary,status,created_by,created_at) VALUES(?,?,?,?,?,?,?)',(ver,ver,level,msg,'ready',a['username'],now())); exec_sql('UPDATE gitops_baseline SET active_version=?,active_tag=?,updated_at=? WHERE id=?',(ver,ver,now(),'current')); audit(a['username'],'git_tag',ver,{'level':level}); self.send_json({'status':'tagged','version':ver,'git':out}); return
@@ -4,11 +4,21 @@ from __future__ import annotations
from dataclasses import dataclass, asdict
import hashlib
import json
import os
from pathlib import Path
import re
import stat
import tempfile
from typing import Any, Mapping
from aios_gitops_job_controller import GitOpsJobController, JobStore
from aios_gitops_job_identity import legacy_task_storage_key, task_storage_key
from aios_online_agent_execution import GovernedExecutionBinding, project_execution_envelope
try: # pragma: no cover - exercised on the Linux activation host
import fcntl
except ImportError: # pragma: no cover - non-POSIX execution stays fail closed
fcntl = None # type: ignore[assignment]
OID_RE = re.compile(r"^[0-9a-f]{40}$")
BRANCH_RE = re.compile(r"^agent/[A-Za-z0-9][A-Za-z0-9._/-]{2,179}$")
@@ -18,6 +28,29 @@ class BootstrapError(RuntimeError):
pass
def _lexists(path: Path) -> bool:
try:
path.lstat()
return True
except FileNotFoundError:
return False
def _require_runtime_directory(path: Path) -> None:
try:
metadata = path.lstat()
except OSError as exc:
raise BootstrapError("TASK_STORAGE_PARENT_INVALID") from exc
expected_uid = getattr(os, "geteuid", lambda: metadata.st_uid)()
if (
stat.S_ISLNK(metadata.st_mode)
or not stat.S_ISDIR(metadata.st_mode)
or metadata.st_mode & 0o022
or metadata.st_uid != expected_uid
):
raise BootstrapError("TASK_STORAGE_PARENT_INVALID")
@dataclass(frozen=True, slots=True)
class GovernedTaskBinding:
task_id: str
@@ -36,49 +69,235 @@ class GovernedTaskBinding:
return {"task_id": self.task_id, "repository": self.repository, "base_oid": self.base_oid, "target_branch": self.target_branch, "expected_branch_oid": self.expected_branch_oid, "generation": self.generation, "branch_mode": self.branch_mode, "create_target_branch": self.branch_mode == "CREATE_NEW", "worktree_mode": "detached_at_base_oid" if self.branch_mode == "CREATE_NEW" else "detached_at_expected_head", "cas_ref_update_required": True, "idempotent": idempotent, "state": "BOUND", "job_id": self.job_id}
def _safe(value: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "-", value).strip("-")[:120] or "task"
# ===== AIOS Formal Lineage producer-native reservation v12 =====
import re as _fl_re
_FL_PRODUCER_KEY = _fl_re.compile(r"^[0-9a-f]{64}$")
def _formal_lineage_reservation_receipt(
value,
*,
request_idempotency_key: str,
expected_side_effect_state: str,
):
if callable(getattr(value, "as_dict", None)):
raw=value.as_dict()
elif isinstance(value, dict):
raw=dict(value)
else:
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_TYPE_INVALID")
producer_key=raw.get("idempotency_key")
job_id=raw.get("job_id")
if not isinstance(job_id,str) or not job_id:
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_JOB_ID_INVALID")
if raw.get("state")!="REQUESTED":
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_STATE_INVALID")
if raw.get("side_effect_state")!=expected_side_effect_state:
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_SIDE_EFFECT_STATE_INVALID")
if not isinstance(producer_key,str) or not _FL_PRODUCER_KEY.fullmatch(producer_key):
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_PRODUCER_IDEMPOTENCY_INVALID")
if job_id!="job-"+producer_key[:32]:
raise RuntimeError("GOVERNED_TASK_SUBMIT_RECEIPT_JOB_PRODUCER_KEY_CONFLICT")
if not isinstance(request_idempotency_key,str) or not request_idempotency_key:
raise RuntimeError("GOVERNED_TASK_REQUEST_IDEMPOTENCY_INVALID")
return {
"job_id":job_id,
"state":"REQUESTED",
"side_effect_state":raw["side_effect_state"],
"producer_idempotency_key":producer_key,
"request_idempotency_key":request_idempotency_key,
}
def _formal_lineage_attach_receipt(response, receipt):
if not isinstance(response, dict):
raise RuntimeError("GOVERNED_TASK_BINDING_RESPONSE_INVALID")
if response.get("state")!="BOUND":
raise RuntimeError("GOVERNED_TASK_BINDING_STATE_INVALID")
if response.get("job_id")!=receipt.get("job_id"):
raise RuntimeError("GOVERNED_TASK_BINDING_RECEIPT_JOB_CONFLICT")
result=dict(response)
result["reservation_receipt"]=dict(receipt)
return result
class GovernedTaskBootstrap:
def __init__(self, authority: Any, root: Path, *, remote_name: str, token_path: Path, ssh_key_path: Path, api_base: str = "https://g.cnnetmkt.top/git/api/v1") -> None:
self.authority = authority; self.root = Path(root); self.remote_name = remote_name; self.token_path = Path(token_path); self.ssh_key_path = Path(ssh_key_path); self.api_base = api_base; self.task_meta_root = self.root / "task-bindings"; self.job_root = self.root / "jobs"; self.task_meta_root.mkdir(parents=True, exist_ok=True); self.job_root.mkdir(parents=True, exist_ok=True)
self.authority = authority; self.root = Path(root); self.remote_name = remote_name; self.token_path = Path(token_path); self.ssh_key_path = Path(ssh_key_path); self.api_base = api_base; self.task_meta_root = self.root / "task-bindings"; self.job_root = self.root / "jobs"; self.lock_root = self.root / "task-binding-locks"; self.task_meta_root.mkdir(parents=True, exist_ok=True, mode=0o700); self.job_root.mkdir(parents=True, exist_ok=True, mode=0o700); self.lock_root.mkdir(parents=True, exist_ok=True, mode=0o700); _require_runtime_directory(self.task_meta_root); _require_runtime_directory(self.job_root); _require_runtime_directory(self.lock_root)
def _store(self, task_id: str) -> tuple[JobStore, GitOpsJobController]:
root = self.job_root / _safe(task_id); store = JobStore(root, payload_root=root / "payloads"); return store, GitOpsJobController(store)
root = self.job_root / task_storage_key(task_id)
legacy = self.job_root / legacy_task_storage_key(task_id)
if legacy != root and _lexists(legacy):
if _lexists(root):
raise BootstrapError("TASK_STORAGE_LEGACY_CONFLICT")
raise BootstrapError("TASK_STORAGE_LEGACY_MIGRATION_REQUIRED")
store = JobStore(root, payload_root=root / "payloads")
return store, GitOpsJobController(store)
def _metadata(self, task_id: str) -> Path:
return self.task_meta_root / (_safe(task_id) + ".json")
return self.task_meta_root / (task_storage_key(task_id) + ".json")
def _legacy_metadata(self, task_id: str) -> Path:
return self.task_meta_root / (legacy_task_storage_key(task_id) + ".json")
def _read_existing(self, task_id: str) -> GovernedTaskBinding | None:
path = self._metadata(task_id)
if not path.is_file() or path.is_symlink(): return None
return GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
legacy = self._legacy_metadata(task_id)
if legacy != path and _lexists(legacy):
if _lexists(path):
raise BootstrapError("TASK_BINDING_LEGACY_CONFLICT")
raise BootstrapError("TASK_BINDING_LEGACY_MIGRATION_REQUIRED")
if not _lexists(path):
return None
if not path.is_file() or path.is_symlink():
raise BootstrapError("TASK_BINDING_PATH_INVALID")
binding = GovernedTaskBinding(**json.loads(path.read_text(encoding="utf-8")))
if binding.task_id != task_id:
raise BootstrapError("TASK_BINDING_IDENTITY_CONFLICT")
return binding
def get_binding(self, task_id: str) -> GovernedTaskBinding:
"""Read the exact durable binding; never synthesize missing lineage."""
binding = self._read_existing(str(task_id))
if binding is None or not binding.job_id:
raise BootstrapError("TASK_EXECUTION_PROJECTION_MISSING")
return binding
def execution_binding(self, binding: GovernedTaskBinding) -> GovernedExecutionBinding:
return GovernedExecutionBinding.from_mapping(
{
"task_id": binding.task_id,
"job_id": binding.job_id,
"repository": binding.repository,
"base_oid": binding.base_oid,
"expected_branch_oid": binding.expected_branch_oid,
"target_branch": binding.target_branch,
"branch_mode": binding.branch_mode,
"generation": binding.generation,
"idempotency_key": binding.idempotency_key,
}
)
def execution_envelope(
self,
task_id: str,
execution_request: Mapping[str, Any] | None = None,
*,
required_capabilities: tuple[str, ...] = (),
expected_job_id: str | None = None,
) -> dict[str, Any]:
binding = self.get_binding(task_id)
if expected_job_id and binding.job_id != expected_job_id:
raise BootstrapError("JOB_BINDING_CONFLICT")
return project_execution_envelope(
self.execution_binding(binding),
execution_request,
required_capabilities=required_capabilities,
)
def _write(self, binding: GovernedTaskBinding) -> None:
path = self._metadata(binding.task_id); temporary = path.with_suffix(".tmp"); temporary.write_text(json.dumps(asdict(binding), sort_keys=True), encoding="utf-8"); temporary.replace(path)
path = self._metadata(binding.task_id)
descriptor, temporary_name = tempfile.mkstemp(
prefix=".task-binding-", dir=self.task_meta_root,
)
temporary = Path(temporary_name)
try:
os.fchmod(descriptor, 0o600)
with os.fdopen(descriptor, "w", encoding="utf-8", newline="\n") as stream:
stream.write(json.dumps(asdict(binding), sort_keys=True) + "\n")
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, path)
os.chmod(path, 0o600)
parent = os.open(self.task_meta_root, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(parent)
finally:
os.close(parent)
except BaseException:
temporary.unlink(missing_ok=True)
raise
def _task_lock(self, task_id: str) -> int:
if fcntl is None:
raise BootstrapError("TASK_BINDING_LOCK_UNAVAILABLE")
path = self.lock_root / (task_storage_key(task_id) + ".lock")
descriptor = os.open(
path,
os.O_RDWR | os.O_CREAT | getattr(os, "O_NOFOLLOW", 0),
0o600,
)
metadata = os.fstat(descriptor)
expected_uid = getattr(os, "geteuid", lambda: metadata.st_uid)()
if (
not stat.S_ISREG(metadata.st_mode)
or metadata.st_nlink != 1
or metadata.st_uid != expected_uid
):
os.close(descriptor)
raise BootstrapError("TASK_BINDING_LOCK_INVALID")
os.fchmod(descriptor, 0o600)
try:
fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError as exc:
os.close(descriptor)
raise BootstrapError("TASK_BINDING_BUSY") from exc
return descriptor
@staticmethod
def _unlock_task(descriptor: int) -> None:
if fcntl is not None:
fcntl.flock(descriptor, fcntl.LOCK_UN)
os.close(descriptor)
def create(self, request: Mapping[str, Any]) -> dict[str, Any]:
values = {key: str(request[key]) for key in ("task_id", "repository", "base_oid", "target_branch", "risk", "description", "idempotency_key")}
if values["repository"] != "aios/aios" or not OID_RE.fullmatch(values["base_oid"]): raise BootstrapError("TASK_BINDING_INVALID")
if not BRANCH_RE.fullmatch(values["target_branch"]): raise BootstrapError("TARGET_BRANCH_INVALID")
existing = self._read_existing(values["task_id"])
if existing is not None:
if any(getattr(existing, key) != values[key] for key in ("repository", "base_oid", "target_branch", "risk", "description")): raise BootstrapError("TASK_BINDING_CONFLICT")
return existing.response(idempotent=True)
if not self.authority.commit_exists(values["base_oid"]): raise BootstrapError("BASE_COMMIT_NOT_FOUND")
current = self.authority.read_ref("refs/heads/" + values["target_branch"])
expected = str(request.get("expected_branch_oid") or values["base_oid"])
if current is None:
mode, generation = "CREATE_NEW", 1
if expected != values["base_oid"]: raise BootstrapError("EXPECTED_BASE_CONFLICT")
else:
if expected != current: raise BootstrapError("EXISTING_BRANCH_OID_CONFLICT")
mode, generation = "CONTINUE_EXISTING", int(request.get("generation", 2))
if generation < 1: raise BootstrapError("GENERATION_INVALID")
payload = json.dumps(values, sort_keys=True, separators=(",", ":")).encode()
_, controller = self._store(values["task_id"])
receipt = controller.submit({"idempotency_key": values["idempotency_key"], "campaign_id": values["task_id"], "task_id": values["task_id"], "repository": values["repository"], "operation": "GOVERNED_TASK_CREATE", "branch": values["target_branch"], "expected_branch_oid": expected, "payload_digest": hashlib.sha256(payload).hexdigest(), "lineage_anchor_oid": values["base_oid"]}, payload=payload)
binding = GovernedTaskBinding(**values, expected_branch_oid=expected, branch_mode=mode, generation=generation, job_id=receipt.job_id)
self._write(binding)
return binding.response()
descriptor = self._task_lock(values["task_id"])
try:
existing = self._read_existing(values["task_id"])
if existing is not None:
if any(getattr(existing, key) != values[key] for key in ("repository", "base_oid", "target_branch", "risk", "description", "idempotency_key")): raise BootstrapError("TASK_BINDING_CONFLICT")
expected = str(request.get('expected_branch_oid') or values['base_oid'])
payload = json.dumps(values, sort_keys=True, separators=(',', ':')).encode()
_, controller = self._store(values['task_id'])
_fl_submit_receipt = controller.submit({'idempotency_key': values['idempotency_key'], 'campaign_id': values['task_id'], 'task_id': values['task_id'], 'repository': values['repository'], 'operation': 'GOVERNED_TASK_CREATE', 'branch': values['target_branch'], 'expected_branch_oid': expected, 'payload_digest': hashlib.sha256(payload).hexdigest(), 'lineage_anchor_oid': values['base_oid']}, payload=payload)
_fl_receipt = _formal_lineage_reservation_receipt(
_fl_submit_receipt, request_idempotency_key=request["idempotency_key"],
expected_side_effect_state='SAME_JOB'
)
_fl_response = existing.response(idempotent=True)
return _formal_lineage_attach_receipt(_fl_response,_fl_receipt)
if not self.authority.commit_exists(values["base_oid"]): raise BootstrapError("BASE_COMMIT_NOT_FOUND")
current = self.authority.read_ref("refs/heads/" + values["target_branch"])
expected = str(request.get("expected_branch_oid") or values["base_oid"])
if current is None:
mode, generation = "CREATE_NEW", 1
if expected != values["base_oid"]: raise BootstrapError("EXPECTED_BASE_CONFLICT")
else:
if expected != current: raise BootstrapError("EXISTING_BRANCH_OID_CONFLICT")
mode, generation = "CONTINUE_EXISTING", int(request.get("generation", 2))
if generation < 1: raise BootstrapError("GENERATION_INVALID")
payload = json.dumps(values, sort_keys=True, separators=(",", ":")).encode()
_, controller = self._store(values["task_id"])
expected_job_id = controller.internal_job_id_for_idempotency_key(values["idempotency_key"])
with controller.store._connect() as connection:
bootstrap_jobs = connection.execute(
"SELECT job_id FROM jobs WHERE operation='GOVERNED_TASK_CREATE' ORDER BY job_id"
).fetchall()
if bootstrap_jobs and [str(row[0]) for row in bootstrap_jobs] != [expected_job_id]:
raise BootstrapError("TASK_BINDING_RECOVERY_CONFLICT")
receipt = controller.submit({"idempotency_key": values["idempotency_key"], "campaign_id": values["task_id"], "task_id": values["task_id"], "repository": values["repository"], "operation": "GOVERNED_TASK_CREATE", "branch": values["target_branch"], "expected_branch_oid": expected, "payload_digest": hashlib.sha256(payload).hexdigest(), "lineage_anchor_oid": values["base_oid"]}, payload=payload)
binding = GovernedTaskBinding(**values, expected_branch_oid=expected, branch_mode=mode, generation=generation, job_id=receipt.job_id)
self._write(binding)
_fl_receipt = _formal_lineage_reservation_receipt(
receipt, request_idempotency_key=request["idempotency_key"],
expected_side_effect_state="QUEUED"
)
_fl_response = binding.response()
return _formal_lineage_attach_receipt(_fl_response,_fl_receipt)
finally:
self._unlock_task(descriptor)
-456
View File
@@ -1,456 +0,0 @@
#!/usr/bin/env python3
"""Disabled-by-default TLS edge for the read-only GPT Online Task ingress.
The internal AIOS ingress remains loopback-only. This service is a separate edge
that may bind non-loopback only when the owner explicitly approves it. It
forwards only the tiny read-only ingress API and never proxies the canonical
Bridge or Task API.
"""
from __future__ import annotations
from collections import defaultdict, deque
from dataclasses import dataclass
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import secrets
import ssl
import sys
import threading
import time
import urllib.error
import urllib.request
from typing import Any, Mapping
from urllib.parse import urlsplit
UPSTREAM = "http://127.0.0.1:18945"
MAX_BODY = 64 * 1024
MAX_RESPONSE = 256 * 1024
MAX_CONCURRENT_REQUESTS = 32
REQUEST_TIMEOUT_SECONDS = 20
RATE_WINDOW_SECONDS = 60
RATE_LIMIT_REQUESTS = 60
RATE_BURST_REQUESTS = 15
RATE_MAX_CLIENTS = 4096
_AUDIT_SALT = secrets.token_bytes(32)
PUBLIC_APPROVAL_ENV = "AIOS_GPT_EDGE_PUBLIC_BIND_APPROVED"
PUBLIC_DOMAIN_ENV = "AIOS_GPT_EDGE_PUBLIC_DOMAIN"
TASK_PATH_RE = re.compile(r"^/v1/read-only-tasks(?:/[A-Za-z0-9][A-Za-z0-9._:-]{7,127})?$")
def _client_pseudonym(value: str) -> str:
return hashlib.sha256(_AUDIT_SALT + value.encode("utf-8", "replace")).hexdigest()[:16]
def audit(event: str, **fields: object) -> None:
safe = {
"event": event,
"ts_unix_ms": int(time.time() * 1000),
}
for key, value in fields.items():
if key in {"authorization", "body", "token", "cookie", "client_ip"}:
continue
safe[key] = value
print(
json.dumps(safe, sort_keys=True, separators=(",", ":")),
file=sys.stderr,
flush=True,
)
class EdgeError(RuntimeError):
def __init__(self, code: str, status: int = 400) -> None:
super().__init__(code)
self.code = code
self.status = status
def canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def public_bind_allowed(host: str, approval: str) -> str:
value = host.strip()
if value in {"127.0.0.1", "::1", "localhost"}:
return value
if approval != "1":
raise RuntimeError("GPT_EDGE_PUBLIC_BIND_OWNER_APPROVAL_REQUIRED")
if value not in {"0.0.0.0", "::"}:
raise RuntimeError("GPT_EDGE_BIND_ADDRESS_INVALID")
return value
def validated_public_domain(value: str) -> str:
domain = value.strip().lower()
if not domain:
return ""
if len(domain) > 253 or re.fullmatch(
r"(?=.{1,253}\Z)(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?",
domain,
) is None:
raise RuntimeError("GPT_EDGE_PUBLIC_DOMAIN_INVALID")
return domain
def validated_edge_identity(host: str, approval: str, domain: str) -> tuple[str, str]:
bind_host = public_bind_allowed(host, approval)
public_domain = validated_public_domain(domain)
if bind_host not in {"127.0.0.1", "::1", "localhost"} and not public_domain:
raise RuntimeError("GPT_EDGE_PUBLIC_DOMAIN_REQUIRED")
return bind_host, public_domain
def validate_tls_file(path: Path, *, key: bool) -> Path:
if path.is_symlink() or not path.is_file():
raise RuntimeError("GPT_EDGE_TLS_FILE_INVALID")
size = path.stat().st_size
if size < 16 or size > 256 * 1024:
raise RuntimeError("GPT_EDGE_TLS_FILE_INVALID")
if key and (path.stat().st_mode & 0o077):
raise RuntimeError("GPT_EDGE_TLS_KEY_MODE_INVALID")
return path
def build_tls_context(cert_path: Path, key_path: Path) -> ssl.SSLContext:
cert = validate_tls_file(cert_path, key=False)
key = validate_tls_file(key_path, key=True)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.minimum_version = ssl.TLSVersion.TLSv1_2
context.options |= ssl.OP_NO_COMPRESSION
context.load_cert_chain(certfile=str(cert), keyfile=str(key))
return context
def allowed_route(method: str, raw_path: str) -> str:
parsed = urlsplit(raw_path)
if parsed.query or parsed.fragment:
raise EdgeError("QUERY_OR_FRAGMENT_DENY", 400)
path = parsed.path
if method == "GET" and path in {"/health", "/openapi.json"}:
return path
if method == "POST" and path == "/v1/read-only-tasks":
return path
if method == "GET" and TASK_PATH_RE.fullmatch(path):
return path
raise EdgeError("ROUTE_DENY", 404)
def route_label(method: str, path: str) -> str:
if method == "GET" and path == "/health":
return "health"
if method == "GET" and path == "/openapi.json":
return "openapi"
if method == "POST" and path == "/v1/read-only-tasks":
return "task_create"
if method == "GET" and path.startswith("/v1/read-only-tasks/"):
return "task_status"
return "unknown"
def forwarded_headers(headers: Mapping[str, str], *, body: bool) -> dict[str, str]:
result = {"Accept": "application/json"}
authorization = headers.get("Authorization")
if not isinstance(authorization, str) or not authorization.startswith("Bearer "):
raise EdgeError("AUTHORIZATION_REQUIRED", 401)
token = authorization[7:]
if len(token) < 32 or len(token) > 4096 or any(ch.isspace() for ch in token):
raise EdgeError("AUTHORIZATION_INVALID", 401)
result["Authorization"] = authorization
if body:
content_type = headers.get("Content-Type", "")
if content_type.split(";", 1)[0].strip().lower() != "application/json":
raise EdgeError("CONTENT_TYPE_INVALID", 415)
result["Content-Type"] = "application/json"
return result
class RateLimiter:
def __init__(
self,
*,
window_seconds: int = RATE_WINDOW_SECONDS,
limit: int = RATE_LIMIT_REQUESTS,
burst: int = RATE_BURST_REQUESTS,
max_clients: int = RATE_MAX_CLIENTS,
now_fn=time.monotonic,
) -> None:
self.window_seconds = window_seconds
self.limit = limit
self.burst = burst
self.max_clients = max_clients
self.now_fn = now_fn
self._lock = threading.Lock()
self._events: dict[str, deque[float]] = defaultdict(deque)
def _prune_stale(self, cutoff: float) -> None:
stale = [
key
for key, events in self._events.items()
if not events or events[-1] <= cutoff
]
for key in stale:
self._events.pop(key, None)
def allowed(self, key: str) -> bool:
now = self.now_fn()
cutoff = now - self.window_seconds
with self._lock:
if key not in self._events and len(self._events) >= self.max_clients:
self._prune_stale(cutoff)
if len(self._events) >= self.max_clients:
return False
events = self._events[key]
while events and events[0] <= cutoff:
events.popleft()
if len(events) >= self.limit + self.burst:
return False
events.append(now)
return True
@dataclass(frozen=True)
class Proxy:
upstream: str = UPSTREAM
timeout: int = REQUEST_TIMEOUT_SECONDS
def request(
self,
method: str,
path: str,
headers: Mapping[str, str],
body: bytes | None,
) -> tuple[int, bytes, str]:
if self.upstream != UPSTREAM:
raise EdgeError("UPSTREAM_DRIFT_DENY", 503)
req = urllib.request.Request(
self.upstream + path,
method=method,
data=body,
headers=dict(headers),
)
try:
with urllib.request.urlopen(req, timeout=self.timeout) as response:
raw = response.read(MAX_RESPONSE + 1)
status = response.status
content_type = response.headers.get("Content-Type", "application/json")
except urllib.error.HTTPError as exc:
raw = exc.read(MAX_RESPONSE + 1)
status = exc.code
content_type = exc.headers.get("Content-Type", "application/json")
except Exception as exc:
raise EdgeError("UPSTREAM_UNAVAILABLE", 502) from exc
if len(raw) > MAX_RESPONSE:
raise EdgeError("UPSTREAM_RESPONSE_TOO_LARGE", 502)
if content_type.split(";", 1)[0].strip().lower() != "application/json":
raise EdgeError("UPSTREAM_CONTENT_TYPE_INVALID", 502)
return status, raw, "application/json"
class EdgeServer(ThreadingHTTPServer):
daemon_threads = True
request_queue_size = 64
allow_reuse_address = True
class Handler(BaseHTTPRequestHandler):
proxy = Proxy()
limiter = RateLimiter()
concurrency = threading.BoundedSemaphore(MAX_CONCURRENT_REQUESTS)
expected_domain = ""
server_version = "AIOSGPTIngressEdge/1.0"
sys_version = ""
def log_message(self, *_args: Any) -> None:
return
def respond_json(self, value: Mapping[str, Any], status: int) -> None:
raw = canonical(value)
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("Referrer-Policy", "no-referrer")
self.send_header("Content-Security-Policy", "default-src 'none'")
self.send_header("Strict-Transport-Security", "max-age=31536000")
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def send_proxy_response(self, status: int, raw: bytes, content_type: str) -> None:
self.send_response(status)
self.send_header("Content-Type", content_type)
self.send_header("Cache-Control", "no-store")
self.send_header("X-Content-Type-Options", "nosniff")
self.send_header("Referrer-Policy", "no-referrer")
self.send_header("Content-Security-Policy", "default-src 'none'")
self.send_header("Strict-Transport-Security", "max-age=31536000")
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def client_key(self) -> str:
return str(self.client_address[0])
def validate_host(self) -> None:
if not self.expected_domain:
return
host = self.headers.get("Host", "").split(":", 1)[0].lower()
if not hmac.compare_digest(host, self.expected_domain):
raise EdgeError("HOST_HEADER_DENY", 421)
def body(self) -> bytes:
try:
length = int(self.headers.get("Content-Length", "-1"))
except ValueError as exc:
raise EdgeError("CONTENT_LENGTH_INVALID") from exc
if length < 1 or length > MAX_BODY:
raise EdgeError("REQUEST_SIZE_INVALID", 413)
raw = self.rfile.read(length)
if len(raw) != length:
raise EdgeError("REQUEST_BODY_TRUNCATED", 400)
# Validate JSON at the edge so malformed bodies do not consume upstream work.
try:
value = json.loads(raw.decode("utf-8"))
except Exception as exc:
raise EdgeError("REQUEST_JSON_INVALID", 400) from exc
if not isinstance(value, Mapping):
raise EdgeError("REQUEST_JSON_OBJECT_REQUIRED", 400)
return canonical(value)
def handle_proxy(self, method: str) -> None:
client_id = _client_pseudonym(self.client_key())
started = time.monotonic()
acquired = self.concurrency.acquire(blocking=False)
if not acquired:
audit("EDGE_BUSY", client_id=client_id, method=method)
self.respond_json({"error": "EDGE_BUSY"}, 503)
return
label = "unknown"
try:
self.validate_host()
path = allowed_route(method, self.path)
label = route_label(method, path)
if path == "/health" and method == "GET":
self.respond_json(
{
"status": "ok",
"service": "aios-gpt-https-edge",
"upstream": "loopback-readonly-ingress",
},
200,
)
audit(
"EDGE_REQUEST",
client_id=client_id,
method=method,
route=label,
status=200,
duration_ms=int((time.monotonic() - started) * 1000),
)
return
if not self.limiter.allowed(self.client_key()):
audit(
"EDGE_RATE_LIMIT",
client_id=client_id,
method=method,
route=label,
)
self.respond_json({"error": "RATE_LIMITED"}, 429)
return
has_body = method == "POST"
headers = forwarded_headers(self.headers, body=has_body)
body = self.body() if has_body else None
status, raw, content_type = self.proxy.request(
method,
path,
headers,
body,
)
self.send_proxy_response(status, raw, content_type)
audit(
"EDGE_REQUEST",
client_id=client_id,
method=method,
route=label,
status=status,
duration_ms=int((time.monotonic() - started) * 1000),
)
except EdgeError as exc:
audit(
"EDGE_REJECT",
client_id=client_id,
method=method,
route=label,
status=exc.status,
error=exc.code,
)
self.respond_json({"error": exc.code}, exc.status)
finally:
self.concurrency.release()
def do_GET(self) -> None:
self.handle_proxy("GET")
def do_POST(self) -> None:
self.handle_proxy("POST")
def do_PUT(self) -> None:
self.respond_json({"error": "METHOD_DENY"}, 405)
def do_PATCH(self) -> None:
self.respond_json({"error": "METHOD_DENY"}, 405)
def do_DELETE(self) -> None:
self.respond_json({"error": "METHOD_DENY"}, 405)
def do_CONNECT(self) -> None:
self.respond_json({"error": "METHOD_DENY"}, 405)
def do_OPTIONS(self) -> None:
self.respond_json({"error": "METHOD_DENY"}, 405)
def serve() -> None:
host, public_domain = validated_edge_identity(
os.environ.get("AIOS_GPT_EDGE_HOST", "127.0.0.1"),
os.environ.get(PUBLIC_APPROVAL_ENV, "0"),
os.environ.get(PUBLIC_DOMAIN_ENV, ""),
)
port = int(os.environ.get("AIOS_GPT_EDGE_PORT", "18443"))
if not 1024 <= port <= 65535:
raise RuntimeError("GPT_EDGE_PORT_INVALID")
Handler.expected_domain = public_domain
credentials = Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
cert_path = Path(
os.environ.get(
"AIOS_GPT_EDGE_TLS_CERT_FILE",
str(credentials / "tls_cert"),
)
)
key_path = Path(
os.environ.get(
"AIOS_GPT_EDGE_TLS_KEY_FILE",
str(credentials / "tls_key"),
)
)
context = build_tls_context(cert_path, key_path)
server = EdgeServer((host, port), Handler)
server.socket = context.wrap_socket(server.socket, server_side=True)
server.serve_forever()
if __name__ == "__main__":
serve()
@@ -1,561 +0,0 @@
#!/usr/bin/env python3
"""Owner-approved activation/rollback controller for the AIOS GPT HTTPS edge.
This controller never creates approval, TLS, or ingress credentials. It consumes
owner-managed inputs, applies one exact public-bind overlay, verifies the live
edge, and provides an idempotent rollback path.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import socket
import ssl
import stat
import subprocess
import tempfile
import time
import urllib.request
from typing import Any, Mapping
APPROVAL_SCHEMA = "aios.gpt-https-edge.public-approval.v1"
STATE_SCHEMA = "aios.gpt-https-edge.activation-state.v1"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
APPROVAL_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$")
DOMAIN_RE = re.compile(
r"(?=.{1,253}\Z)"
r"(?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+"
r"[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?"
)
EDGE_UNIT = "aios-gpt-https-edge.service"
INGRESS_UNIT = "aios-gpt-online-task-ingress.service"
PUBLIC_HOST = "0.0.0.0"
PUBLIC_PORT = 18443
INGRESS_HEALTH = "http://127.0.0.1:18945/health"
DROPIN_NAME = "zzzzzzzzzz-owner-approved-public.conf"
class ActivationError(RuntimeError):
pass
def canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def sha256_file(path: Path) -> str:
return sha256_bytes(path.read_bytes())
def parse_time(value: object, *, field: str) -> datetime:
if not isinstance(value, str) or not value.strip():
raise ActivationError("APPROVAL_" + field.upper() + "_INVALID")
text = value.strip()
if text.endswith("Z"):
text = text[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(text)
except ValueError as exc:
raise ActivationError("APPROVAL_" + field.upper() + "_INVALID") from exc
if parsed.tzinfo is None:
raise ActivationError("APPROVAL_" + field.upper() + "_INVALID")
return parsed.astimezone(timezone.utc)
def validate_domain(value: object) -> str:
if not isinstance(value, str):
raise ActivationError("APPROVAL_DOMAIN_INVALID")
domain = value.strip().lower()
if DOMAIN_RE.fullmatch(domain) is None:
raise ActivationError("APPROVAL_DOMAIN_INVALID")
return domain
def validate_root_file(
path: Path,
*,
exact_mode: int | None = None,
max_size: int = 256 * 1024,
) -> Path:
if path.is_symlink() or not path.is_file():
raise ActivationError("OWNER_FILE_INVALID")
info = path.stat()
if info.st_uid != 0 or info.st_gid != 0:
raise ActivationError("OWNER_FILE_OWNERSHIP_INVALID")
mode = stat.S_IMODE(info.st_mode)
if exact_mode is not None and mode != exact_mode:
raise ActivationError("OWNER_FILE_MODE_INVALID")
if info.st_size < 16 or info.st_size > max_size:
raise ActivationError("OWNER_FILE_SIZE_INVALID")
return path
def certificate_dns_names(cert_path: Path) -> tuple[str, ...]:
try:
decoded = ssl._ssl._test_decode_cert(str(cert_path))
except Exception as exc:
raise ActivationError("TLS_CERT_DECODE_INVALID") from exc
names = [
value
for kind, value in decoded.get("subjectAltName", ())
if kind == "DNS"
]
if not names:
for rdn in decoded.get("subject", ()):
for key, value in rdn:
if key == "commonName":
names.append(value)
return tuple(names)
def cert_matches_domain(cert_path: Path, domain: str) -> bool:
names = certificate_dns_names(cert_path)
try:
return bool(names) and any(ssl._dnsname_match(name, domain) for name in names)
except ssl.CertificateError:
return False
def validate_tls_pair(
cert_path: Path,
key_path: Path,
domain: str,
*,
now_fn=time.time,
) -> dict[str, Any]:
validate_root_file(cert_path)
validate_root_file(key_path, exact_mode=0o600)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.minimum_version = ssl.TLSVersion.TLSv1_2
context.options |= ssl.OP_NO_COMPRESSION
try:
context.load_cert_chain(str(cert_path), str(key_path))
except Exception as exc:
raise ActivationError("TLS_CERT_KEY_PAIR_INVALID") from exc
if not cert_matches_domain(cert_path, domain):
raise ActivationError("TLS_CERT_DOMAIN_MISMATCH")
decoded = ssl._ssl._test_decode_cert(str(cert_path))
not_after = decoded.get("notAfter")
if not not_after:
raise ActivationError("TLS_CERT_EXPIRY_MISSING")
remaining = ssl.cert_time_to_seconds(not_after) - now_fn()
if remaining < 7 * 86400:
raise ActivationError("TLS_CERT_EXPIRY_TOO_CLOSE")
return {
"cert_sha256": sha256_file(cert_path),
"remaining_days": int(remaining // 86400),
}
@dataclass(frozen=True)
class Approval:
approval_id: str
candidate_oid: str
domain: str
bind_host: str
port: int
tls_cert_sha256: str
owner: str
issued_at: datetime
not_before: datetime
expires_at: datetime
raw_sha256: str
@classmethod
def load(
cls,
path: Path,
*,
candidate_oid: str,
cert_path: Path,
now: datetime | None = None,
) -> "Approval":
validate_root_file(path, exact_mode=0o600, max_size=64 * 1024)
try:
value = json.loads(path.read_text(encoding="utf-8"))
except Exception as exc:
raise ActivationError("APPROVAL_JSON_INVALID") from exc
if not isinstance(value, Mapping):
raise ActivationError("APPROVAL_JSON_INVALID")
allowed = {
"schema",
"approval_id",
"candidate_oid",
"domain",
"public_bind",
"bind_host",
"port",
"tls_cert_sha256",
"issued_at",
"not_before",
"expires_at",
"owner",
}
if set(value) != allowed or value.get("schema") != APPROVAL_SCHEMA:
raise ActivationError("APPROVAL_FIELDS_INVALID")
approval_id = value.get("approval_id")
if not isinstance(approval_id, str) or APPROVAL_ID_RE.fullmatch(approval_id) is None:
raise ActivationError("APPROVAL_ID_INVALID")
if not OID_RE.fullmatch(str(value.get("candidate_oid") or "")):
raise ActivationError("APPROVAL_CANDIDATE_INVALID")
if value["candidate_oid"] != candidate_oid:
raise ActivationError("APPROVAL_CANDIDATE_MISMATCH")
if value.get("public_bind") is not True:
raise ActivationError("APPROVAL_PUBLIC_BIND_REQUIRED")
if value.get("bind_host") != PUBLIC_HOST or value.get("port") != PUBLIC_PORT:
raise ActivationError("APPROVAL_BINDING_INVALID")
cert_sha = str(value.get("tls_cert_sha256") or "")
if SHA_RE.fullmatch(cert_sha) is None:
raise ActivationError("APPROVAL_CERT_SHA_INVALID")
if not cert_path.is_file() or sha256_file(cert_path) != cert_sha:
raise ActivationError("APPROVAL_CERT_SHA_MISMATCH")
domain = validate_domain(value.get("domain"))
owner = value.get("owner")
if not isinstance(owner, str) or not owner.strip() or len(owner) > 255:
raise ActivationError("APPROVAL_OWNER_INVALID")
issued_at = parse_time(value.get("issued_at"), field="issued_at")
not_before = parse_time(value.get("not_before"), field="not_before")
expires_at = parse_time(value.get("expires_at"), field="expires_at")
current = now or datetime.now(timezone.utc)
if not (issued_at <= not_before <= expires_at):
raise ActivationError("APPROVAL_TIME_ORDER_INVALID")
if current < not_before:
raise ActivationError("APPROVAL_NOT_YET_VALID")
if current >= expires_at:
raise ActivationError("APPROVAL_EXPIRED")
if expires_at - issued_at > __import__("datetime").timedelta(days=7):
raise ActivationError("APPROVAL_VALIDITY_TOO_LONG")
return cls(
approval_id=approval_id,
candidate_oid=candidate_oid,
domain=domain,
bind_host=PUBLIC_HOST,
port=PUBLIC_PORT,
tls_cert_sha256=cert_sha,
owner=owner.strip(),
issued_at=issued_at,
not_before=not_before,
expires_at=expires_at,
raw_sha256=sha256_file(path),
)
def validate_release(release_root: Path) -> dict[str, Path]:
if release_root.is_symlink() or not release_root.is_dir():
raise ActivationError("IMMUTABLE_RELEASE_INVALID")
edge = release_root / "active-runtime/app/aios_gpt_https_edge.py"
controller = release_root / "active-runtime/app/aios_gpt_https_edge_activation.py"
for path in (edge, controller):
if path.is_symlink() or not path.is_file():
raise ActivationError("IMMUTABLE_RELEASE_INVALID")
return {"edge": edge, "controller": controller}
def render_dropin(approval: Approval, release_root: Path) -> str:
app = release_root / "active-runtime/app/aios_gpt_https_edge.py"
return (
"[Service]\n"
f"Environment=AIOS_GPT_EDGE_HOST={approval.bind_host}\n"
f"Environment=AIOS_GPT_EDGE_PORT={approval.port}\n"
"Environment=AIOS_GPT_EDGE_PUBLIC_BIND_APPROVED=1\n"
f"Environment=AIOS_GPT_EDGE_PUBLIC_DOMAIN={approval.domain}\n"
"ExecStart=\n"
f"ExecStart=/usr/bin/python3 -B {app}\n"
f"ReadOnlyPaths={release_root}\n"
)
def atomic_write(path: Path, data: str, mode: int = 0o600) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
fd, name = tempfile.mkstemp(prefix="." + path.name + ".", dir=path.parent)
try:
os.fchmod(fd, mode)
with os.fdopen(fd, "w", encoding="utf-8") as handle:
handle.write(data)
handle.flush()
os.fsync(handle.fileno())
os.replace(name, path)
finally:
if os.path.exists(name):
os.unlink(name)
class Systemd:
def run(self, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
["/usr/bin/systemctl", *args],
check=check,
text=True,
capture_output=True,
timeout=30,
)
def active(self, unit: str) -> bool:
return self.run("is-active", "--quiet", unit, check=False).returncode == 0
def enabled(self, unit: str) -> bool:
return self.run("is-enabled", "--quiet", unit, check=False).returncode == 0
def show(self, unit: str, prop: str) -> str:
return self.run("show", unit, f"--property={prop}", "--value").stdout.strip()
def require_ingress_ready(systemd: Systemd) -> None:
if not systemd.active(INGRESS_UNIT):
raise ActivationError("INGRESS_SERVICE_NOT_ACTIVE")
try:
with urllib.request.urlopen(INGRESS_HEALTH, timeout=5) as response:
if response.status != 200:
raise ActivationError("INGRESS_HEALTH_FAILED")
value = json.loads(response.read().decode())
except ActivationError:
raise
except Exception as exc:
raise ActivationError("INGRESS_HEALTH_FAILED") from exc
if value.get("status") != "ok":
raise ActivationError("INGRESS_HEALTH_FAILED")
def tls_probe(domain: str, cert_path: Path, *, port: int = PUBLIC_PORT) -> None:
context = ssl.create_default_context(cafile=str(cert_path))
context.check_hostname = True
context.verify_mode = ssl.CERT_REQUIRED
try:
with socket.create_connection(("127.0.0.1", port), timeout=5) as sock:
with context.wrap_socket(sock, server_hostname=domain) as tls_sock:
if tls_sock.version() not in {"TLSv1.2", "TLSv1.3"}:
raise ActivationError("EDGE_TLS_VERSION_INVALID")
except ActivationError:
raise
except Exception as exc:
raise ActivationError("EDGE_TLS_PROBE_FAILED") from exc
def receipt_for(
approval: Approval,
release_root: Path,
*,
state: str,
) -> dict[str, Any]:
return {
"schema": STATE_SCHEMA,
"state": state,
"approval_id": approval.approval_id,
"approval_sha256": approval.raw_sha256,
"candidate_oid": approval.candidate_oid,
"domain": approval.domain,
"bind_host": approval.bind_host,
"port": approval.port,
"tls_cert_sha256": approval.tls_cert_sha256,
"release_root": str(release_root),
"updated_at": datetime.now(timezone.utc).isoformat(),
}
def activate(
*,
approval: Approval,
release_root: Path,
cert_path: Path,
key_path: Path,
dropin_path: Path,
state_path: Path,
systemd: Systemd,
) -> dict[str, Any]:
if os.geteuid() != 0:
raise ActivationError("ROOT_REQUIRED")
release = validate_release(release_root)
app = release["edge"]
require_ingress_ready(systemd)
validate_tls_pair(cert_path, key_path, approval.domain)
if state_path.is_file():
try:
existing = json.loads(state_path.read_text(encoding="utf-8"))
except Exception:
existing = {}
if (
existing.get("schema") == STATE_SCHEMA
and existing.get("state") == "ACTIVE"
and existing.get("approval_sha256") == approval.raw_sha256
and existing.get("candidate_oid") == approval.candidate_oid
and systemd.active(EDGE_UNIT)
):
return existing
dropin = render_dropin(approval, release_root)
modified = False
try:
atomic_write(dropin_path, dropin, mode=0o644)
modified = True
systemd.run("daemon-reload")
systemd.run("enable", EDGE_UNIT)
systemd.run("restart", EDGE_UNIT)
if not systemd.active(EDGE_UNIT):
raise ActivationError("EDGE_SERVICE_NOT_ACTIVE")
exec_start = systemd.show(EDGE_UNIT, "ExecStart")
environment = systemd.show(EDGE_UNIT, "Environment")
if str(app) not in exec_start:
raise ActivationError("EDGE_EFFECTIVE_EXECSTART_INVALID")
for expected in (
f"AIOS_GPT_EDGE_HOST={approval.bind_host}",
f"AIOS_GPT_EDGE_PORT={approval.port}",
"AIOS_GPT_EDGE_PUBLIC_BIND_APPROVED=1",
f"AIOS_GPT_EDGE_PUBLIC_DOMAIN={approval.domain}",
):
if expected not in environment:
raise ActivationError("EDGE_EFFECTIVE_ENVIRONMENT_INVALID")
tls_probe(approval.domain, cert_path, port=approval.port)
receipt = receipt_for(approval, release_root, state="ACTIVE")
atomic_write(state_path, json.dumps(receipt, indent=2, sort_keys=True) + "\n")
return receipt
except Exception:
if modified:
rollback(
dropin_path=dropin_path,
state_path=state_path,
systemd=systemd,
reason="activation_failed",
)
raise
def rollback(
*,
dropin_path: Path,
state_path: Path,
systemd: Systemd,
reason: str,
) -> dict[str, Any]:
if os.geteuid() != 0:
raise ActivationError("ROOT_REQUIRED")
systemd.run("stop", EDGE_UNIT, check=False)
systemd.run("disable", EDGE_UNIT, check=False)
dropin_path.unlink(missing_ok=True)
systemd.run("daemon-reload")
if systemd.active(EDGE_UNIT):
raise ActivationError("EDGE_ROLLBACK_SERVICE_STILL_ACTIVE")
previous: dict[str, Any] = {}
if state_path.is_file():
try:
previous = json.loads(state_path.read_text(encoding="utf-8"))
except Exception:
previous = {}
receipt = {
"schema": STATE_SCHEMA,
"state": "ROLLED_BACK",
"reason": reason[:255],
"candidate_oid": previous.get("candidate_oid"),
"approval_id": previous.get("approval_id"),
"domain": previous.get("domain"),
"updated_at": datetime.now(timezone.utc).isoformat(),
}
atomic_write(state_path, json.dumps(receipt, indent=2, sort_keys=True) + "\n")
return receipt
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("command", choices=("check", "activate", "rollback"))
parser.add_argument("--candidate")
parser.add_argument("--approval", default="/etc/aios-gitops/gpt-https-edge.approved")
parser.add_argument("--tls-cert", default="/etc/aios-gitops/gpt-https-edge-tls.crt")
parser.add_argument("--tls-key", default="/etc/aios-gitops/gpt-https-edge-tls.key")
parser.add_argument("--release-root")
parser.add_argument(
"--dropin",
default=f"/etc/systemd/system/{EDGE_UNIT}.d/{DROPIN_NAME}",
)
parser.add_argument(
"--state",
default="/var/lib/aios-gpt-https-edge-activation/state.json",
)
parser.add_argument("--reason", default="owner_requested")
args = parser.parse_args()
systemd = Systemd()
if args.command == "rollback":
value = rollback(
dropin_path=Path(args.dropin),
state_path=Path(args.state),
systemd=systemd,
reason=args.reason,
)
print(json.dumps(value, indent=2, sort_keys=True))
return 0
candidate = str(args.candidate or "")
if OID_RE.fullmatch(candidate) is None:
raise SystemExit("CANDIDATE_REQUIRED")
release_root = (
Path(args.release_root)
if args.release_root
else Path(f"/opt/aios/releases/{candidate}-gpt-https-edge-public")
)
cert_path = Path(args.tls_cert)
key_path = Path(args.tls_key)
approval = Approval.load(
Path(args.approval),
candidate_oid=candidate,
cert_path=cert_path,
)
tls = validate_tls_pair(cert_path, key_path, approval.domain)
release = validate_release(release_root)
require_ingress_ready(systemd)
if args.command == "check":
value = {
"status": "READY",
"candidate_oid": candidate,
"approval_id": approval.approval_id,
"approval_sha256": approval.raw_sha256,
"domain": approval.domain,
"bind_host": approval.bind_host,
"port": approval.port,
"edge_sha256": sha256_file(release["edge"]),
"controller_sha256": sha256_file(release["controller"]),
**tls,
"public_network_mutation": False,
}
print(json.dumps(value, indent=2, sort_keys=True))
return 0
value = activate(
approval=approval,
release_root=release_root,
cert_path=cert_path,
key_path=key_path,
dropin_path=Path(args.dropin),
state_path=Path(args.state),
systemd=systemd,
)
print(json.dumps(value, indent=2, sort_keys=True))
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -1,816 +0,0 @@
#!/usr/bin/env python3
"""Credential-isolated host-side ingress for externally scheduled read-only Online Tasks.
This service is intentionally not a public AIOS control plane. It accepts a tiny,
authenticated external intent and translates it to the canonical loopback Bridge.
Callers cannot select repository coordinates, capabilities, execution action,
worker, branch policy, credentials, verification decisions, or write permissions.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import hmac
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import json
import os
from pathlib import Path
import re
import sqlite3
import threading
import urllib.error
import urllib.request
from typing import Any, Mapping
SCHEMA = "aios.gpt-online-task-ingress.request.v1"
RESPONSE_SCHEMA = "aios.gpt-online-task-ingress.response.v1"
OPENAPI_VERSION = "3.1.0"
REQUEST_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,127}$")
TASK_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
JOB_ID_RE = re.compile(r"^job-[0-9a-f]{32}$")
OID_RE = re.compile(r"^[0-9a-f]{40}$")
MAX_BODY = 64 * 1024
MAX_GOAL = 12_000
MAX_INSTRUCTIONS = 24_000
MAX_CRITERIA = 16
MAX_CRITERION_TEXT = 1_000
READONLY_CAPABILITIES = ("git.read", "git.test")
REPOSITORY = "aios/aios"
class IngressError(RuntimeError):
def __init__(self, code: str, status: int = 400) -> None:
super().__init__(code)
self.code = code
self.status = status
def canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def digest(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def now() -> str:
return datetime.now(timezone.utc).isoformat()
def load_secret(path: Path, *, code: str) -> str:
if path.is_symlink() or not path.is_file():
raise RuntimeError(code + "_UNAVAILABLE")
value = path.read_text(encoding="utf-8").strip()
if len(value) < 32 or "\n" in value or "\r" in value:
raise RuntimeError(code + "_INVALID")
return value
def normalized_request(value: Mapping[str, Any]) -> dict[str, Any]:
allowed = {
"schema",
"request_id",
"goal",
"instructions",
"success_criteria",
"constraints",
}
if set(value) - allowed:
raise IngressError("UNKNOWN_FIELDS")
if value.get("schema") != SCHEMA:
raise IngressError("SCHEMA_INVALID")
request_id = value.get("request_id")
if not isinstance(request_id, str) or REQUEST_ID_RE.fullmatch(request_id) is None:
raise IngressError("REQUEST_ID_INVALID")
goal = value.get("goal")
if not isinstance(goal, str) or not goal.strip() or len(goal) > MAX_GOAL:
raise IngressError("GOAL_INVALID")
instructions = value.get("instructions", "")
if not isinstance(instructions, str) or len(instructions) > MAX_INSTRUCTIONS:
raise IngressError("INSTRUCTIONS_INVALID")
criteria = value.get("success_criteria") or []
if not isinstance(criteria, list) or len(criteria) > MAX_CRITERIA:
raise IngressError("SUCCESS_CRITERIA_INVALID")
normalized_criteria = []
for item in criteria:
if not isinstance(item, Mapping) or set(item) != {"id", "description"}:
raise IngressError("SUCCESS_CRITERIA_INVALID")
criterion_id = item.get("id")
description = item.get("description")
if (
not isinstance(criterion_id, str)
or not criterion_id.strip()
or len(criterion_id) > 120
or not isinstance(description, str)
or not description.strip()
or len(description) > MAX_CRITERION_TEXT
):
raise IngressError("SUCCESS_CRITERIA_INVALID")
normalized_criteria.append(
{"id": criterion_id.strip(), "description": description.strip()}
)
constraints = value.get("constraints") or {}
if not isinstance(constraints, Mapping):
raise IngressError("CONSTRAINTS_INVALID")
allowed_constraints = {
"max_files",
"focus_paths",
"notes",
}
if set(constraints) - allowed_constraints:
raise IngressError("CONSTRAINTS_FIELDS_INVALID")
normalized_constraints: dict[str, Any] = {}
if "max_files" in constraints:
max_files = constraints["max_files"]
if (
not isinstance(max_files, int)
or isinstance(max_files, bool)
or not 1 <= max_files <= 500
):
raise IngressError("MAX_FILES_INVALID")
normalized_constraints["max_files"] = max_files
if "focus_paths" in constraints:
paths = constraints["focus_paths"]
if (
not isinstance(paths, list)
or len(paths) > 32
or not all(
isinstance(path, str)
and path
and len(path) <= 512
and not path.startswith("/")
and ".." not in Path(path).parts
for path in paths
)
):
raise IngressError("FOCUS_PATHS_INVALID")
normalized_constraints["focus_paths"] = paths
if "notes" in constraints:
notes = constraints["notes"]
if not isinstance(notes, str) or len(notes) > 4_000:
raise IngressError("CONSTRAINT_NOTES_INVALID")
normalized_constraints["notes"] = notes
return {
"schema": SCHEMA,
"request_id": request_id,
"goal": goal.strip(),
"instructions": instructions,
"success_criteria": normalized_criteria,
"constraints": normalized_constraints,
}
def validated_bind_host(value: str) -> str:
host = value.strip()
if host not in {"127.0.0.1", "::1", "localhost"}:
raise RuntimeError("GPT_INGRESS_NON_LOOPBACK_BIND_DENY")
return host
def task_id_for(request_id: str) -> str:
suffix = hashlib.sha256(request_id.encode("utf-8")).hexdigest()[:24]
return "gpt-readonly-" + suffix
def branch_for(task_id: str) -> str:
return "agent/external-gpt-readonly/" + task_id[-24:]
@dataclass(frozen=True)
class Upstream:
bridge_base: str
task_base: str
control_token: str
timeout: int = 20
def _request(
self,
base: str,
method: str,
path: str,
body: Mapping[str, Any] | None = None,
) -> tuple[int, dict[str, Any]]:
raw_body = None if body is None else canonical(body)
headers = {
"Authorization": "Bearer " + self.control_token,
"Accept": "application/json",
}
if raw_body is not None:
headers["Content-Type"] = "application/json"
req = urllib.request.Request(
base.rstrip("/") + path,
method=method,
data=raw_body,
headers=headers,
)
try:
with urllib.request.urlopen(req, timeout=self.timeout) as response:
raw = response.read()
status = response.status
except urllib.error.HTTPError as exc:
raw = exc.read()
status = exc.code
except Exception as exc:
raise IngressError("UPSTREAM_TRANSPORT_FAILED", 503) from exc
try:
value = json.loads(raw.decode("utf-8"))
except Exception as exc:
raise IngressError("UPSTREAM_RESPONSE_INVALID", 502) from exc
if not isinstance(value, Mapping):
raise IngressError("UPSTREAM_RESPONSE_INVALID", 502)
return status, dict(value)
def control_snapshot(self) -> dict[str, Any]:
status, value = self._request(
self.bridge_base,
"GET",
"/v2/control-plane-snapshot",
)
if status != 200:
raise IngressError("CONTROL_SNAPSHOT_UNAVAILABLE", 503)
return value
def create_governed_task(
self,
body: Mapping[str, Any],
) -> tuple[int, dict[str, Any]]:
return self._request(
self.bridge_base,
"POST",
"/v2/governed-task",
body,
)
def execution_envelope(
self,
body: Mapping[str, Any],
) -> tuple[int, dict[str, Any]]:
return self._request(
self.bridge_base,
"POST",
"/v2/execution-envelope",
body,
)
def get_task(self, task_id: str) -> tuple[int, dict[str, Any]]:
return self._request(
self.task_base,
"GET",
"/v1/tasks/" + task_id,
)
class Store:
def __init__(self, path: Path) -> None:
self.path = Path(path)
self._lock = threading.RLock()
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as con:
con.execute(
"""
CREATE TABLE IF NOT EXISTS external_task_requests(
request_id TEXT PRIMARY KEY,
request_sha256 TEXT NOT NULL,
task_id TEXT NOT NULL UNIQUE,
job_id TEXT,
base_oid TEXT,
state TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
response_json TEXT
)
"""
)
con.commit()
def _connect(self) -> sqlite3.Connection:
con = sqlite3.connect(self.path, timeout=30)
con.row_factory = sqlite3.Row
con.execute("PRAGMA busy_timeout=30000")
return con
def get(self, request_id: str) -> sqlite3.Row | None:
with self._connect() as con:
return con.execute(
"SELECT * FROM external_task_requests WHERE request_id=?",
(request_id,),
).fetchone()
def reserve(self, request: Mapping[str, Any], task_id: str) -> sqlite3.Row:
request_sha = digest(request)
timestamp = now()
with self._lock, self._connect() as con:
con.execute("BEGIN IMMEDIATE")
row = con.execute(
"SELECT * FROM external_task_requests WHERE request_id=?",
(request["request_id"],),
).fetchone()
if row is not None:
if row["request_sha256"] != request_sha:
raise IngressError("IDEMPOTENCY_CONFLICT", 409)
con.commit()
return row
con.execute(
"""
INSERT INTO external_task_requests(
request_id,request_sha256,task_id,state,created_at,updated_at
) VALUES(?,?,?,?,?,?)
""",
(
request["request_id"],
request_sha,
task_id,
"PENDING",
timestamp,
timestamp,
),
)
con.commit()
row = self.get(request["request_id"])
assert row is not None
return row
def bound(
self,
request_id: str,
*,
job_id: str,
base_oid: str,
response: Mapping[str, Any],
) -> None:
with self._lock, self._connect() as con:
con.execute(
"""
UPDATE external_task_requests
SET job_id=?,base_oid=?,state='BOUND',response_json=?,updated_at=?
WHERE request_id=?
""",
(
job_id,
base_oid,
json.dumps(response, sort_keys=True, separators=(",", ":")),
now(),
request_id,
),
)
con.commit()
class IngressService:
def __init__(self, upstream: Upstream, store: Store) -> None:
self.upstream = upstream
self.store = store
@staticmethod
def _main_oid(snapshot: Mapping[str, Any]) -> str:
candidates = [
snapshot.get("authoritative_head_oid"),
snapshot.get("head"),
(snapshot.get("aggregate") or {}).get("authoritative_head_oid")
if isinstance(snapshot.get("aggregate"), Mapping)
else None,
]
for candidate in candidates:
if isinstance(candidate, str) and OID_RE.fullmatch(candidate):
return candidate
raise IngressError("AUTHORITATIVE_HEAD_UNAVAILABLE", 503)
@staticmethod
def _public_response(
request_id: str,
task_id: str,
job_id: str,
state: str,
) -> dict[str, Any]:
return {
"schema": RESPONSE_SCHEMA,
"request_id": request_id,
"task_id": task_id,
"job_id": job_id,
"state": state,
"execution_mode": "source_capture",
"capabilities": list(READONLY_CAPABILITIES),
"write_access": False,
"official_verification_required": True,
}
def _recover_existing(
self,
request: Mapping[str, Any],
row: sqlite3.Row,
) -> dict[str, Any] | None:
task_id = str(row["task_id"])
status, task = self.upstream.get_task(task_id)
if status == 404:
return None
if status != 200:
raise IngressError("TASK_RECOVERY_FAILED", 503)
metadata = task.get("metadata")
execution = (
metadata.get("execution")
if isinstance(metadata, Mapping)
and isinstance(metadata.get("execution"), Mapping)
else {}
)
job_id = str(execution.get("job_id") or row["job_id"] or "")
base_oid = str(execution.get("base_oid") or row["base_oid"] or "")
if (
task.get("task_id") != task_id
or not JOB_ID_RE.fullmatch(job_id)
or not OID_RE.fullmatch(base_oid)
or execution.get("action") != "source_capture"
or sorted(execution.get("capability_scope") or [])
!= sorted(READONLY_CAPABILITIES)
or (execution.get("governance") or {}).get("writer_lease") is not False
):
raise IngressError("TASK_RECOVERY_BINDING_INVALID", 409)
response = self._public_response(
request["request_id"],
task_id,
job_id,
str(task.get("status") or "BOUND"),
)
self.store.bound(
request["request_id"],
job_id=job_id,
base_oid=base_oid,
response=response,
)
return response
def create(self, raw: Mapping[str, Any]) -> dict[str, Any]:
request = normalized_request(raw)
task_id = task_id_for(request["request_id"])
row = self.store.reserve(request, task_id)
if row["state"] == "BOUND" and row["response_json"]:
return json.loads(row["response_json"])
recovered = self._recover_existing(request, row)
if recovered is not None:
return recovered
snapshot = self.upstream.control_snapshot()
base_oid = self._main_oid(snapshot)
target_branch = branch_for(task_id)
binding_key = "gpt-readonly-binding-" + digest(
{"request_id": request["request_id"], "base_oid": base_oid}
)[:24]
constraints = {
**request["constraints"],
"repository_mutation": "forbidden",
"control_plane_side_effects": "forbidden",
"credential_access": "forbidden",
}
body = {
"task_id": task_id,
"repository": REPOSITORY,
"base_oid": base_oid,
"expected_branch_oid": base_oid,
"target_branch": target_branch,
"branch_mode": "CREATE_NEW",
"generation": 1,
"risk": "low",
"description": "Externally requested read-only governed Online Task.",
"idempotency_key": binding_key,
"priority": 0,
"max_attempts": 3,
"require_online_task_projection": True,
"online_task": {
"goal": request["goal"],
"instructions": request["instructions"],
"success_criteria": request["success_criteria"],
"constraints": constraints,
"required_capabilities": list(READONLY_CAPABILITIES),
"priority": 0,
"max_attempts": 3,
},
"execution": {
"action": "source_capture",
"adapter": "source_capture",
"capability_scope": list(READONLY_CAPABILITIES),
"source_signals": [
"external read-only source analysis request",
"no repository mutation no credential access no control-plane side effects",
],
},
}
status, value = self.upstream.create_governed_task(body)
if status not in {200, 201, 202}:
if status == 409:
recovered = self._recover_existing(request, row)
if recovered is not None:
return recovered
raise IngressError(
"GOVERNED_TASK_CREATE_FAILED",
409 if status == 409 else 503,
)
job_id = str(value.get("job_id") or "")
if value.get("task_id") != task_id or JOB_ID_RE.fullmatch(job_id) is None:
raise IngressError("GOVERNED_TASK_RESPONSE_INVALID", 502)
envelope_status, envelope = self.upstream.execution_envelope(
{
"task_id": task_id,
"job_id": job_id,
"required_capabilities": list(READONLY_CAPABILITIES),
"execution": body["execution"],
}
)
if envelope_status != 200:
raise IngressError("EXECUTION_PROJECTION_FAILED", 503)
if (
envelope.get("task_id") != task_id
or envelope.get("job_id") != job_id
or envelope.get("base_oid") != base_oid
or envelope.get("action") != "source_capture"
or sorted(envelope.get("capability_scope") or [])
!= sorted(READONLY_CAPABILITIES)
or (envelope.get("governance") or {}).get("writer_lease") is not False
):
raise IngressError("EXECUTION_PROJECTION_INVALID", 502)
response = self._public_response(
request["request_id"],
task_id,
job_id,
"BOUND",
)
self.store.bound(
request["request_id"],
job_id=job_id,
base_oid=base_oid,
response=response,
)
return response
def status(self, request_id: str) -> dict[str, Any]:
if REQUEST_ID_RE.fullmatch(request_id) is None:
raise IngressError("REQUEST_ID_INVALID")
row = self.store.get(request_id)
if row is None:
raise IngressError("REQUEST_NOT_FOUND", 404)
task_id = str(row["task_id"])
status, task = self.upstream.get_task(task_id)
if status == 200 and not row["job_id"]:
metadata = task.get("metadata")
execution = (
metadata.get("execution")
if isinstance(metadata, Mapping)
and isinstance(metadata.get("execution"), Mapping)
else {}
)
job_id = str(execution.get("job_id") or "")
base_oid = str(execution.get("base_oid") or "")
if (
JOB_ID_RE.fullmatch(job_id)
and OID_RE.fullmatch(base_oid)
and execution.get("action") == "source_capture"
and sorted(execution.get("capability_scope") or [])
== sorted(READONLY_CAPABILITIES)
and (execution.get("governance") or {}).get("writer_lease") is False
):
recovered = self._public_response(
request_id,
task_id,
job_id,
str(task.get("status") or "BOUND"),
)
self.store.bound(
request_id,
job_id=job_id,
base_oid=base_oid,
response=recovered,
)
row = self.store.get(request_id) or row
if status == 404:
return {
"schema": RESPONSE_SCHEMA,
"request_id": request_id,
"task_id": task_id,
"state": "PENDING",
"official_verification_required": True,
}
if status != 200:
raise IngressError("TASK_STATUS_UNAVAILABLE", 503)
result = task.get("result") if isinstance(task.get("result"), Mapping) else {}
verification = (
task.get("verification")
if isinstance(task.get("verification"), Mapping)
else {}
)
safe = {
"schema": RESPONSE_SCHEMA,
"request_id": request_id,
"task_id": task_id,
"job_id": str(row["job_id"] or ""),
"state": str(task.get("status") or "UNKNOWN"),
"attempt": int(task.get("attempt") or 0),
"claimed": bool(task.get("claimed_by")),
"result_status": result.get("status"),
"result_classification": result.get("classification"),
"official_verification": verification.get("decision"),
"official_classification": verification.get("classification"),
"write_access": False,
}
if result.get("execution_mode") == "source_capture":
snapshot = result.get("source_snapshot")
if isinstance(snapshot, Mapping):
safe["source_snapshot_sha256"] = snapshot.get("payload_sha256")
safe["source_snapshot_file_count"] = snapshot.get("file_count")
return safe
def openapi_document() -> dict[str, Any]:
return {
"openapi": OPENAPI_VERSION,
"info": {
"title": "AIOS Read-Only Online Task Ingress",
"version": "1.0.0",
},
"paths": {
"/v1/read-only-tasks": {
"post": {
"operationId": "createReadOnlyOnlineTask",
"security": [{"bearerAuth": []}],
"requestBody": {"required": True},
"responses": {"200": {"description": "Task accepted"}},
}
},
"/v1/read-only-tasks/{request_id}": {
"get": {
"operationId": "getReadOnlyOnlineTaskStatus",
"security": [{"bearerAuth": []}],
"responses": {"200": {"description": "Sanitized task status"}},
}
},
},
"components": {
"securitySchemes": {
"bearerAuth": {
"type": "http",
"scheme": "bearer",
}
}
},
}
class Handler(BaseHTTPRequestHandler):
service: IngressService | None = None
ingress_token: str = ""
def log_message(self, *_args: Any) -> None:
return
def respond(self, value: Mapping[str, Any], status: int) -> None:
raw = canonical(value)
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(raw)))
self.end_headers()
self.wfile.write(raw)
def authorized(self) -> bool:
header = self.headers.get("Authorization", "")
return (
header.startswith("Bearer ")
and bool(self.ingress_token)
and hmac.compare_digest(header[7:], self.ingress_token)
)
def body(self) -> dict[str, Any]:
try:
length = int(self.headers.get("Content-Length", "-1"))
except ValueError as exc:
raise IngressError("CONTENT_LENGTH_INVALID") from exc
if length < 1 or length > MAX_BODY:
raise IngressError("REQUEST_SIZE_INVALID", 413)
try:
value = json.loads(self.rfile.read(length).decode("utf-8"))
except Exception as exc:
raise IngressError("REQUEST_JSON_INVALID") from exc
if not isinstance(value, Mapping):
raise IngressError("REQUEST_OBJECT_REQUIRED")
return dict(value)
def do_GET(self) -> None:
if self.path == "/health":
self.respond(
{
"status": "ok",
"service": "aios-gpt-online-task-ingress",
"mode": "read-only",
},
200,
)
return
if self.path == "/openapi.json":
self.respond(openapi_document(), 200)
return
if not self.authorized():
self.respond({"error": "UNAUTHORIZED"}, 401)
return
match = re.fullmatch(r"/v1/read-only-tasks/([^/?#]+)", self.path)
if match is None:
self.respond({"error": "NOT_FOUND"}, 404)
return
try:
if self.service is None:
raise IngressError("RUNTIME_UNAVAILABLE", 503)
self.respond(self.service.status(match.group(1)), 200)
except IngressError as exc:
self.respond({"error": exc.code}, exc.status)
def do_POST(self) -> None:
if self.path != "/v1/read-only-tasks":
self.respond({"error": "NOT_FOUND"}, 404)
return
if not self.authorized():
self.respond({"error": "UNAUTHORIZED"}, 401)
return
try:
if self.service is None:
raise IngressError("RUNTIME_UNAVAILABLE", 503)
value = self.service.create(self.body())
self.respond(value, 200)
except IngressError as exc:
self.respond({"error": exc.code}, exc.status)
def serve() -> None:
ingress_token_path = Path(
os.environ.get(
"AIOS_GPT_INGRESS_TOKEN_FILE",
str(
Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
/ "gpt_ingress_token"
),
)
)
control_token_path = Path(
os.environ.get(
"AIOS_AGENT_TASK_CONTROL_TOKEN_FILE",
str(
Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
/ "agent_task_control_token"
),
)
)
ingress_token = load_secret(
ingress_token_path,
code="GPT_INGRESS_CREDENTIAL",
)
control_token = load_secret(
control_token_path,
code="CONTROL_PLANE_CREDENTIAL",
)
state_root = Path(
os.environ.get(
"AIOS_GPT_INGRESS_STATE_ROOT",
"/var/lib/aios-gpt-online-task-ingress",
)
)
host = validated_bind_host(
os.environ.get("AIOS_GPT_INGRESS_HOST", "127.0.0.1")
)
port = int(os.environ.get("AIOS_GPT_INGRESS_PORT", "18945"))
if not 1024 <= port <= 65535:
raise RuntimeError("GPT_INGRESS_PORT_INVALID")
Handler.service = IngressService(
Upstream(
bridge_base=os.environ.get(
"AIOS_GOVERNED_WRITE_API_URL",
"http://127.0.0.1:18892",
),
task_base=os.environ.get(
"AIOS_AGENT_TASK_API_URL",
"http://127.0.0.1:18940",
),
control_token=control_token,
),
Store(state_root / "state.sqlite3"),
)
Handler.ingress_token = ingress_token
ThreadingHTTPServer((host, port), Handler).serve_forever()
if __name__ == "__main__":
serve()
@@ -1,306 +0,0 @@
"""Uncredentialed deterministic executor for governed Online Agent changes.
This process never receives Agent/Owner/Bridge credentials and never writes Git refs.
It operates only on an exchange bundle and returns hash-bound changed files + test evidence.
"""
from __future__ import annotations
import base64
from dataclasses import dataclass
import errno
import hashlib
import io
import json
import os
from pathlib import Path
import re
import shutil
import socket
import subprocess
import tarfile
import tempfile
import time
from typing import Any, Mapping, Sequence
ALLOWED_PROGRAMS = {"/usr/bin/python3", "/usr/bin/git", "/bin/true", "/usr/bin/true"}
MAX_ARCHIVE_BYTES = 64 * 1024 * 1024
MAX_FILE_BYTES = 4 * 1024 * 1024
SECRET_ENV_TOKENS = ("TOKEN", "SECRET", "PASSWORD", "CREDENTIAL", "AUTHORIZATION", "PRIVATE_KEY")
REQUEST_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{15,254}$")
SENSITIVE_VALUE_RE = re.compile(
r"(?i)(authorization\s*[:=]\s*bearer\s+\S+|authorization\s+bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*\S+)"
)
class IsolatedExecutorError(RuntimeError):
pass
def sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def safe_relative(value: object) -> str:
if not isinstance(value, str) or not value or value.startswith("/"):
raise IsolatedExecutorError("PATH_INVALID")
path = Path(value)
if ".." in path.parts or "." in path.parts or any(part in {"", ".git"} for part in path.parts):
raise IsolatedExecutorError("PATH_INVALID")
return path.as_posix()
def resolve_under(root: Path, relative: str) -> Path:
root = root.resolve()
path = (root / safe_relative(relative)).resolve(strict=False)
if path == root or root not in path.parents:
raise IsolatedExecutorError("PATH_ESCAPE")
return path
def _credential_exposure(directory: Path) -> str | None:
"""Return an exposure code; EACCES is the intended credential isolation proof."""
try:
if not directory.exists():
return None
if directory.is_symlink():
return "CREDENTIAL_MOUNT_SYMLINK"
if os.access(directory, os.R_OK):
return "CREDENTIAL_MOUNT_ACCESSIBLE"
if directory.is_dir():
for child in directory.iterdir():
if child.is_file() and os.access(child, os.R_OK):
return "CREDENTIAL_FILE_ACCESSIBLE"
except PermissionError:
return None
return None
def assert_uncredentialed() -> None:
credentials = Path("/run/credentials")
candidates = {credentials}
credentials_directory = Path(os.environ.get("CREDENTIALS_DIRECTORY", "/run/credentials"))
candidates.add(credentials_directory)
leaked = sorted(
key for key, value in os.environ.items()
if value
and (any(token in key.upper() for token in SECRET_ENV_TOKENS) or SENSITIVE_VALUE_RE.search(str(value)))
)
if leaked:
raise IsolatedExecutorError("CREDENTIAL_ENV_EXPOSED:" + ",".join(leaked))
for directory in candidates:
exposure = _credential_exposure(directory)
if exposure is not None:
raise IsolatedExecutorError(exposure)
# In the intended PrivateNetwork namespace host loopback services are unreachable.
try:
probe = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except OSError as exc:
if exc.errno in {errno.EACCES, errno.EPERM, errno.EAFNOSUPPORT}:
return
raise IsolatedExecutorError("NETWORK_ISOLATION_PROBE_FAILED") from exc
probe.settimeout(0.15)
try:
if probe.connect_ex(("127.0.0.1", 18892)) == 0:
raise IsolatedExecutorError("GOVERNED_BRIDGE_REACHABLE")
finally:
probe.close()
def extract_source(archive: Path, root: Path) -> None:
if archive.is_symlink() or not archive.is_file() or archive.stat().st_size > MAX_ARCHIVE_BYTES:
raise IsolatedExecutorError("SOURCE_ARCHIVE_INVALID")
with tarfile.open(archive, "r:gz") as tf:
for member in tf.getmembers():
path = Path(member.name)
if path.is_absolute() or ".." in path.parts or ".git" in path.parts:
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
if member.issym() or member.islnk():
raise IsolatedExecutorError("SOURCE_ARCHIVE_LINK_DENY")
tf.extractall(root, filter="data")
def run_command(item: Mapping[str, Any], root: Path) -> dict[str, Any]:
argv = item.get("argv")
if not isinstance(argv, Sequence) or isinstance(argv, (str, bytes)) or not argv:
raise IsolatedExecutorError("COMMAND_INVALID")
argv = [str(part) for part in argv]
if argv[0] not in ALLOWED_PROGRAMS:
raise IsolatedExecutorError("COMMAND_PROGRAM_DENY")
if argv[0] == "/usr/bin/python3" and "-m" in argv:
index = argv.index("-m")
if index + 1 >= len(argv) or argv[index + 1] not in {"pytest", "py_compile"}:
raise IsolatedExecutorError("PYTHON_MODULE_DENY")
if argv[0] == "/usr/bin/git" and any(
token in {"push", "update-ref", "commit", "merge", "rebase", "reset", "clean", "branch"}
for token in argv[1:]
):
raise IsolatedExecutorError("GIT_MUTATION_DENY")
timeout = int(item.get("timeout_seconds") or 300)
if not 1 <= timeout <= 1800:
raise IsolatedExecutorError("COMMAND_TIMEOUT_INVALID")
cwd = root if item.get("cwd", ".") == "." else resolve_under(root, str(item.get("cwd")))
started = time.monotonic()
completed = subprocess.run(
argv,
cwd=cwd,
capture_output=True,
text=True,
check=False,
timeout=timeout,
env={"PATH": "/usr/bin:/bin", "LC_ALL": "C.UTF-8"},
)
result = {
"label": str(item.get("label") or "command"),
"exit_code": completed.returncode,
"stdout_sha256": sha256(completed.stdout.encode("utf-8", "replace")),
"stderr_sha256": sha256(completed.stderr.encode("utf-8", "replace")),
"duration_ms": int((time.monotonic() - started) * 1000),
}
if completed.returncode != 0:
raise IsolatedExecutorError(
"COMMAND_FAILED:" + result["label"] + ":" + str(completed.returncode)
)
return result
def apply_changes(request: Mapping[str, Any], root: Path) -> list[str]:
changes = request.get("change_set")
if not isinstance(changes, list) or not changes:
raise IsolatedExecutorError("CHANGE_SET_INVALID")
changed: list[str] = []
for change in changes:
if not isinstance(change, Mapping):
raise IsolatedExecutorError("CHANGE_INVALID")
relative = safe_relative(change.get("path"))
path = resolve_under(root, relative)
current = path.read_bytes() if path.is_file() else None
expected = change.get("expected_sha256")
if expected is not None and (current is None or sha256(current) != expected):
raise IsolatedExecutorError("SOURCE_DRIFT:" + relative)
operation = change.get("operation")
if operation == "write_file":
content = change.get("content")
if not isinstance(content, str):
raise IsolatedExecutorError("WRITE_CONTENT_INVALID")
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content, encoding="utf-8")
elif operation == "replace_text":
if current is None:
raise IsolatedExecutorError("REPLACE_SOURCE_MISSING")
old = str(change.get("old"))
new = str(change.get("new"))
count = int(change.get("count", 1))
text = current.decode("utf-8")
if text.count(old) != count:
raise IsolatedExecutorError("REPLACE_COUNT_INVALID:" + relative)
path.write_text(text.replace(old, new), encoding="utf-8")
elif operation == "delete_file":
if path.exists():
path.unlink()
else:
raise IsolatedExecutorError("CHANGE_OPERATION_INVALID")
changed.append(relative)
return sorted(set(changed))
def changed_bundle(root: Path, changed: list[str]) -> list[dict[str, Any]]:
files = []
for relative in changed:
path = resolve_under(root, relative)
if path.exists():
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
raise IsolatedExecutorError("CHANGED_FILE_TOO_LARGE:" + relative)
files.append({
"path": relative,
"deleted": False,
"sha256": sha256(data),
"content_b64": base64.b64encode(data).decode("ascii"),
})
else:
files.append({"path": relative, "deleted": True, "sha256": None, "content_b64": None})
return files
def execute_request(request_path: Path, result_path: Path) -> None:
assert_uncredentialed()
request = json.loads(request_path.read_text(encoding="utf-8"))
if request.get("schema") != "aios.isolated-execution.request.v1":
raise IsolatedExecutorError("REQUEST_SCHEMA_INVALID")
request_id = request.get("request_id")
if not isinstance(request_id, str) or not REQUEST_ID_RE.fullmatch(request_id):
raise IsolatedExecutorError("REQUEST_ID_INVALID")
source_archive = Path(str(request.get("source_archive") or ""))
if not source_archive.is_absolute():
raise IsolatedExecutorError("SOURCE_ARCHIVE_PATH_INVALID")
expected_archive_sha = str(request.get("source_archive_sha256") or "")
if sha256(source_archive.read_bytes()) != expected_archive_sha:
raise IsolatedExecutorError("SOURCE_ARCHIVE_SHA_MISMATCH")
with tempfile.TemporaryDirectory(prefix="aios-isolated-executor-") as temporary:
root = Path(temporary) / "workspace"
root.mkdir()
extract_source(source_archive, root)
changed = apply_changes(request, root)
commands = [
run_command(item, root)
for item in request.get("verification", [])
if isinstance(item, Mapping)
]
result = {
"schema": "aios.isolated-execution.result.v1",
"request_id": request_id,
"request_sha256": sha256(canonical(request)),
"changed_files": changed_bundle(root, changed),
"verification": commands,
"credential_isolation": {
"credential_env_absent": True,
"credential_mount_inaccessible": True,
"governed_bridge_unreachable": True,
},
}
result["result_sha256"] = sha256(canonical(result))
temporary_result = result_path.with_suffix(".tmp")
temporary_result.write_text(json.dumps(result, sort_keys=True, separators=(",", ":")), encoding="utf-8")
os.replace(temporary_result, result_path)
def serve(root: Path) -> None:
assert_uncredentialed()
requests = root / "requests"
results = root / "results"
failed = root / "failed"
for path in (requests, results, failed):
path.mkdir(parents=True, exist_ok=True)
while True:
for request in sorted(requests.glob("*.json")):
if request.is_symlink():
continue
result = results / request.name
try:
execute_request(request, result)
request.unlink()
except Exception as exc:
(failed / request.name).write_text(
json.dumps({"error": str(exc) or type(exc).__name__}, sort_keys=True),
encoding="utf-8",
)
request.unlink(missing_ok=True)
time.sleep(0.5)
if __name__ == "__main__":
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--root", default="/var/lib/aios-online-agent-executor")
parser.add_argument("--request")
parser.add_argument("--result")
args = parser.parse_args()
if args.request and args.result:
execute_request(Path(args.request), Path(args.result))
else:
serve(Path(args.root))
+59 -2
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import hashlib, hmac, json, os, re, subprocess, sys, time, urllib.parse, urllib.request, uuid
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from dataclasses import replace
from pathlib import Path
from baseline_authority import BaselineAuthority, BaselineAuthorityError
from silent_release_controller import ReleaseCandidate
@@ -114,6 +115,9 @@ TRUSTED_BUILDER_SOCKET = Path(os.getenv(
TRUSTED_BUILDER_PUBLIC_KEYRING = Path(os.getenv(
'AIOS_TRUSTED_CANDIDATE_BUILDER_PUBLIC_KEYRING', ''
))
TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT = str(os.getenv(
'AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT', ''
)).strip()
TRUSTED_RELEASE_POLICY = Path(os.getenv(
'AIOS_SILENT_RELEASE_TRUSTED_POLICY',
str(WS / 'config' / 'gitops' / 'silent-release-policy.json'),
@@ -233,7 +237,7 @@ class _HostTrustedCandidateBuilderAdapter:
try:
if not isinstance(intent, dict):
raise TypeError('intent must be a mapping')
version = str(intent.get('version') or '')
version = str(intent.get('version') or 'V2R2C2')
# A createReleaseIntent request is intentionally identity-light.
# Candidate identity is resolved by the host builder policy, not by
# the caller. Explicit release-worker requests may still carry it.
@@ -253,6 +257,22 @@ class _HostTrustedCandidateBuilderAdapter:
nonce=secrets.token_hex(32),
)
except BuilderError as exc:
code = str(exc)
if code in {'SOURCE_ADMISSION_REQUIRED', 'RECIPE_UNKNOWN'} or (
code == 'REQUEST_RELEASE_BINDING_INVALID'
and not intent.get('version')
):
raise CapabilityAuthorizationError(
'release source admission required'
) from exc
if code == 'MIGRATION_PLAN_REQUIRED':
raise CapabilityAuthorizationError(
'migration plan required'
) from exc
if code == 'MIGRATION_PLAN_INVALID':
raise CapabilityAuthorizationError(
'migration plan invalid'
) from exc
raise TrustedCandidateBuilderUnavailable(
'trusted builder request failed'
) from exc
@@ -290,6 +310,7 @@ def _build_silent_release_service() -> SilentReleaseActionService:
builder_attestation_verifier = None
trusted_policy = None
builder_readiness_probe = None
source_admission_resolver = None
if TRUSTED_BUILDER_ENABLED:
# Every path is opt-in. In particular, this API receives its own
# client HMAC credential, never the builder's signing private key.
@@ -317,6 +338,33 @@ def _build_silent_release_service() -> SilentReleaseActionService:
client_hmac = (
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY / 'builder_hmac'
).read_bytes().strip()
if TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT:
from release_source_admission import (
open_source_admission_store,
)
try:
source_admission_hmac = (
TRUSTED_BUILDER_CREDENTIALS_DIRECTORY
/ 'source_admission_hmac'
).read_bytes().strip()
except OSError as exc:
raise CapabilityAuthorizationError(
'release source admission credential is unavailable'
) from exc
source_store = open_source_admission_store(
Path(TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT),
repository=builder_config.repository,
policy=TRUSTED_BUILDER_POLICY,
hmac_key=source_admission_hmac,
expected_owner_uid=0,
writable=False,
)
builder_config = replace(
builder_config,
source_admission_resolver=source_store.resolve_recipe,
)
source_admission_resolver = source_store.lookup_source
client_keyring = BuilderKeyring(builder_hmac=client_hmac)
client = UnixTrustedCandidateBuilderClient(
builder_config,
@@ -365,6 +413,7 @@ def _build_silent_release_service() -> SilentReleaseActionService:
builder_attestation_verifier=builder_attestation_verifier,
trusted_policy=trusted_policy,
builder_readiness_probe=builder_readiness_probe,
source_admission_resolver=source_admission_resolver,
allow_legacy_candidate_input=False,
expected_repository=REPOSITORY_IDENTITY,
policy_sha256=policy_sha,
@@ -528,6 +577,15 @@ def publication_request_from_query(q: dict[str, list[str]]) -> PublicationReques
(q['approval_plan_id'])[0],
)
def governed_release_query_payload(q: dict[str, list[str]]) -> dict[str, str]:
request = publication_request_from_query(q)
return {
'version': request.version,
'target_oid': request.target_oid,
'evidence_closure_oid': request.evidence_closure_oid,
'approval_plan_id': request.approval_plan_id,
}
def required_authoritative_git_dir() -> Path:
snapshot = repository_snapshot(
git_dir=AUTHORITATIVE_GIT_DIR,
@@ -1221,4 +1279,3 @@ def main():
print(f'AIOS ChatGPT GitOps adapter listening on {HOST}:{PORT}')
ThreadingHTTPServer((HOST,PORT), H).serve_forever()
if __name__ == '__main__': main()
File diff suppressed because it is too large Load Diff
@@ -1,232 +0,0 @@
#!/usr/bin/env python3
"""Root-owned Model Governance configuration reconciler.
The config contains provider/model/policy metadata only. Raw credentials are rejected by
aios_model_governance and never stored in the governance database.
"""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
from typing import Any, Mapping
from aios_model_governance import (
ModelGovernanceError,
ModelGovernanceStore,
ModelRecord,
ProviderRecord,
ScopePolicy,
)
SCHEMA = "aios.model-governance.bootstrap.v1"
MAX_CONFIG_BYTES = 512 * 1024
def canonical(value: object) -> bytes:
return json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
def fail(code: str) -> None:
raise SystemExit(code)
def _load(path: Path) -> dict[str, Any]:
if path.is_symlink() or not path.is_file() or path.stat().st_size > MAX_CONFIG_BYTES:
fail("MODEL_GOVERNANCE_CONFIG_INVALID")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError:
fail("MODEL_GOVERNANCE_CONFIG_INVALID")
if not isinstance(value, Mapping) or value.get("schema") != SCHEMA:
fail("MODEL_GOVERNANCE_CONFIG_SCHEMA_INVALID")
return dict(value)
def _provider_desired(value: Mapping[str, Any]) -> ProviderRecord:
return ProviderRecord(
provider_id=str(value.get("provider_id") or ""),
credential_ref=value.get("credential_ref"),
enabled=bool(value.get("enabled", True)),
failure_domain=str(value.get("failure_domain") or "default"),
max_failures=int(value.get("max_failures") or 3),
cooldown_seconds=int(value.get("cooldown_seconds") or 60),
)
def _model_desired(value: Mapping[str, Any]) -> ModelRecord:
return ModelRecord(
model_id=str(value.get("model_id") or ""),
provider_id=str(value.get("provider_id") or ""),
version=str(value.get("version") or ""),
capabilities=tuple(value.get("capabilities") or ()),
input_cost_micros_per_1k=int(value.get("input_cost_micros_per_1k") or 0),
output_cost_micros_per_1k=int(value.get("output_cost_micros_per_1k") or 0),
max_input_tokens=int(value.get("max_input_tokens") or 0),
max_output_tokens=int(value.get("max_output_tokens") or 0),
evaluation_state=str(value.get("evaluation_state") or "APPROVED"),
evaluation_sha256=str(value.get("evaluation_sha256") or ""),
enabled=bool(value.get("enabled", True)),
priority=int(value.get("priority") or 100),
)
def _policy_desired(value: Mapping[str, Any]) -> ScopePolicy:
return ScopePolicy(
scope_type=str(value.get("scope_type") or ""),
tenant_id=value.get("tenant_id"),
project_id=value.get("project_id"),
allowed_models=tuple(value.get("allowed_models") or ()),
denied_providers=tuple(value.get("denied_providers") or ()),
required_capabilities=tuple(value.get("required_capabilities") or ()),
pinned_model_id=value.get("pinned_model_id"),
pinned_model_version=value.get("pinned_model_version"),
max_input_tokens_per_request=int(value.get("max_input_tokens_per_request") or 0),
max_output_tokens_per_request=int(value.get("max_output_tokens_per_request") or 0),
daily_token_budget=int(value.get("daily_token_budget") or 0),
daily_cost_budget_micros=int(value.get("daily_cost_budget_micros") or 0),
fallback_limit=int(value.get("fallback_limit") or 0),
)
def _row_dict(row: object) -> dict[str, Any] | None:
return dict(row) if row is not None else None
def _provider_matches(row: Mapping[str, Any], desired: ProviderRecord) -> bool:
return (
row["credential_ref"] == desired.credential_ref
and bool(row["enabled"]) == desired.enabled
and row["failure_domain"] == desired.failure_domain
and int(row["max_failures"]) == desired.max_failures
and int(row["cooldown_seconds"]) == desired.cooldown_seconds
)
def _model_matches(row: Mapping[str, Any], desired: ModelRecord) -> bool:
return (
row["provider_id"] == desired.provider_id
and row["version"] == desired.version
and tuple(json.loads(row["capabilities_json"])) == tuple(sorted(set(desired.capabilities)))
and int(row["input_cost_micros_per_1k"]) == desired.input_cost_micros_per_1k
and int(row["output_cost_micros_per_1k"]) == desired.output_cost_micros_per_1k
and int(row["max_input_tokens"]) == desired.max_input_tokens
and int(row["max_output_tokens"]) == desired.max_output_tokens
and row["evaluation_state"] == desired.evaluation_state
and row["evaluation_sha256"] == desired.evaluation_sha256
and bool(row["enabled"]) == desired.enabled
and int(row["priority"]) == desired.priority
)
def _policy_matches(row: Mapping[str, Any], desired: ScopePolicy) -> bool:
return (
row["scope_type"] == desired.scope_type
and row["tenant_id"] == desired.tenant_id
and row["project_id"] == desired.project_id
and tuple(json.loads(row["allowed_models_json"])) == tuple(sorted(set(desired.allowed_models)))
and tuple(json.loads(row["denied_providers_json"])) == tuple(sorted(set(desired.denied_providers)))
and tuple(json.loads(row["required_capabilities_json"])) == tuple(sorted(set(desired.required_capabilities)))
and row["pinned_model_id"] == desired.pinned_model_id
and row["pinned_model_version"] == desired.pinned_model_version
and int(row["max_input_tokens_per_request"]) == desired.max_input_tokens_per_request
and int(row["max_output_tokens_per_request"]) == desired.max_output_tokens_per_request
and int(row["daily_token_budget"]) == desired.daily_token_budget
and int(row["daily_cost_budget_micros"]) == desired.daily_cost_budget_micros
and int(row["fallback_limit"]) == desired.fallback_limit
)
def apply_config(db_path: Path, config_path: Path) -> dict[str, Any]:
config = _load(config_path)
config_sha = hashlib.sha256(canonical(config)).hexdigest()
providers = config.get("providers") or []
models = config.get("models") or []
policies = config.get("policies") or []
if not all(isinstance(value, list) for value in (providers, models, policies)):
fail("MODEL_GOVERNANCE_CONFIG_COLLECTION_INVALID")
store = ModelGovernanceStore(db_path)
changes = {"providers": 0, "models": 0, "policies": 0, "unchanged": 0}
try:
for item in providers:
if not isinstance(item, Mapping):
fail("MODEL_GOVERNANCE_PROVIDER_CONFIG_INVALID")
desired = _provider_desired(item)
row = store.conn.execute(
"SELECT * FROM model_governance_providers WHERE provider_id=?",
(desired.provider_id,),
).fetchone()
if row is not None and _provider_matches(row, desired):
changes["unchanged"] += 1
continue
expected = int(row["revision"]) if row is not None else 0
store.register_provider(desired, expected_revision=expected)
changes["providers"] += 1
for item in models:
if not isinstance(item, Mapping):
fail("MODEL_GOVERNANCE_MODEL_CONFIG_INVALID")
desired = _model_desired(item)
row = store.conn.execute(
"SELECT * FROM model_governance_models WHERE model_id=?",
(desired.model_id,),
).fetchone()
if row is not None and _model_matches(row, desired):
changes["unchanged"] += 1
continue
expected = int(row["revision"]) if row is not None else 0
store.register_model(desired, expected_revision=expected)
changes["models"] += 1
for item in policies:
if not isinstance(item, Mapping):
fail("MODEL_GOVERNANCE_POLICY_CONFIG_INVALID")
desired = _policy_desired(item)
row = store.conn.execute(
"SELECT * FROM model_governance_scope_policies WHERE scope_key=?",
(desired.scope_key,),
).fetchone()
if row is not None and _policy_matches(row, desired):
changes["unchanged"] += 1
continue
expected = int(row["revision"]) if row is not None else 0
store.set_scope_policy(desired, expected_revision=expected)
changes["policies"] += 1
if not store.verify_audit_chain():
fail("MODEL_GOVERNANCE_AUDIT_CHAIN_INVALID")
return {
"schema": "aios.model-governance.bootstrap-result.v1",
"database": str(db_path),
"config_sha256": config_sha,
"changes": changes,
"audit_chain_valid": True,
}
finally:
store.close()
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--db", required=True)
parser.add_argument("--config", required=True)
parser.add_argument("--output")
args = parser.parse_args()
try:
result = apply_config(Path(args.db), Path(args.config))
except ModelGovernanceError as exc:
print(json.dumps({"status": "FAILED", "error": exc.code}, sort_keys=True))
return 20
data = json.dumps(result, indent=2, sort_keys=True) + "\n"
if args.output:
Path(args.output).write_text(data, encoding="utf-8")
print(data, end="")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -1,302 +0,0 @@
"""Durable checkpoint journal for the generic Online Agent worker.
The JSON task state remains a fast cache. This SQLite journal is the durable,
append-only evidence used for restart/resume/replay decisions.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
from pathlib import Path
import sqlite3
from typing import Any, Mapping
PHASE_ORDER = {
"claimed": 10,
"workspace_prepared": 20,
"verification_passed": 30,
"agent_runtime_done": 40,
"source_capture_done": 40,
"commit_done": 40,
"reconciliation_required": 45,
"result_pending": 50,
"result_submitted": 60,
}
class CheckpointError(RuntimeError):
pass
def canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str).encode("utf-8")
def sha256_json(value: object) -> str:
return hashlib.sha256(canonical(value)).hexdigest()
def now() -> str:
return datetime.now(timezone.utc).isoformat()
@dataclass(frozen=True)
class CheckpointBinding:
task_id: str
attempt: int
execution_digest: str
job_id: str
repository: str
base_oid: str
expected_branch_oid: str
target_branch: str
generation: int
def material(self) -> dict[str, Any]:
return {
"task_id": self.task_id,
"attempt": self.attempt,
"execution_digest": self.execution_digest,
"job_id": self.job_id,
"repository": self.repository,
"base_oid": self.base_oid,
"expected_branch_oid": self.expected_branch_oid,
"target_branch": self.target_branch,
"generation": self.generation,
}
@property
def binding_sha256(self) -> str:
return sha256_json(self.material())
class CheckpointStore:
def __init__(self, path: Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as connection:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS checkpoints(
task_id TEXT NOT NULL,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
job_id TEXT NOT NULL,
repository TEXT NOT NULL,
base_oid TEXT NOT NULL,
expected_branch_oid TEXT NOT NULL,
target_branch TEXT NOT NULL,
generation INTEGER NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
created_at TEXT NOT NULL,
PRIMARY KEY(task_id, attempt, sequence)
);
CREATE INDEX IF NOT EXISTS idx_checkpoints_task_latest
ON checkpoints(task_id, attempt DESC, sequence DESC);
CREATE UNIQUE INDEX IF NOT EXISTS idx_checkpoints_exact_replay
ON checkpoints(task_id, attempt, phase, payload_sha256);
CREATE TABLE IF NOT EXISTS checkpoint_heads(
task_id TEXT PRIMARY KEY,
attempt INTEGER NOT NULL,
sequence INTEGER NOT NULL,
phase TEXT NOT NULL,
phase_order INTEGER NOT NULL,
execution_digest TEXT NOT NULL,
binding_sha256 TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
updated_at TEXT NOT NULL
);
"""
)
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA journal_mode=WAL")
connection.execute("PRAGMA synchronous=FULL")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _phase_order(phase: str) -> int:
try:
return PHASE_ORDER[phase]
except KeyError as exc:
raise CheckpointError("CHECKPOINT_PHASE_INVALID") from exc
def append(
self,
binding: CheckpointBinding,
phase: str,
payload: Mapping[str, Any],
) -> dict[str, Any]:
order = self._phase_order(phase)
payload_value = dict(payload)
payload_sha = sha256_json(payload_value)
with self._connect() as connection:
connection.execute("BEGIN IMMEDIATE")
replay = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=? AND phase=? AND payload_sha256=?
""",
(binding.task_id, binding.attempt, phase, payload_sha),
).fetchone()
if replay is not None:
connection.commit()
return dict(replay)
head = connection.execute(
"SELECT * FROM checkpoint_heads WHERE task_id=?",
(binding.task_id,),
).fetchone()
if head is not None:
head_attempt = int(head["attempt"])
head_order = int(head["phase_order"])
if binding.attempt < head_attempt:
raise CheckpointError("CHECKPOINT_ATTEMPT_REGRESSION")
if binding.attempt == head_attempt:
if head["execution_digest"] != binding.execution_digest:
raise CheckpointError("CHECKPOINT_EXECUTION_DIGEST_CONFLICT")
if head["binding_sha256"] != binding.binding_sha256:
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
if order < head_order and phase != "reconciliation_required":
raise CheckpointError("CHECKPOINT_PHASE_REGRESSION")
sequence = int(head["sequence"]) + 1
else:
sequence = 1
else:
sequence = 1
created_at = now()
connection.execute(
"""
INSERT INTO checkpoints(
task_id,attempt,sequence,phase,phase_order,execution_digest,job_id,
repository,base_oid,expected_branch_oid,target_branch,generation,
binding_sha256,payload_json,payload_sha256,created_at
) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.job_id,
binding.repository,
binding.base_oid,
binding.expected_branch_oid,
binding.target_branch,
binding.generation,
binding.binding_sha256,
canonical(payload_value).decode("utf-8"),
payload_sha,
created_at,
),
)
connection.execute(
"""
INSERT INTO checkpoint_heads(
task_id,attempt,sequence,phase,phase_order,execution_digest,
binding_sha256,payload_sha256,updated_at
) VALUES (?,?,?,?,?,?,?,?,?)
ON CONFLICT(task_id) DO UPDATE SET
attempt=excluded.attempt,
sequence=excluded.sequence,
phase=excluded.phase,
phase_order=excluded.phase_order,
execution_digest=excluded.execution_digest,
binding_sha256=excluded.binding_sha256,
payload_sha256=excluded.payload_sha256,
updated_at=excluded.updated_at
""",
(
binding.task_id,
binding.attempt,
sequence,
phase,
order,
binding.execution_digest,
binding.binding_sha256,
payload_sha,
created_at,
),
)
connection.commit()
row = connection.execute(
"SELECT * FROM checkpoints WHERE task_id=? AND attempt=? AND sequence=?",
(binding.task_id, binding.attempt, sequence),
).fetchone()
return dict(row)
def latest(self, task_id: str) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=?
ORDER BY attempt DESC, sequence DESC
LIMIT 1
""",
(task_id,),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def latest_for_attempt(self, task_id: str, attempt: int) -> dict[str, Any] | None:
with self._connect() as connection:
row = connection.execute(
"""
SELECT * FROM checkpoints
WHERE task_id=? AND attempt=?
ORDER BY sequence DESC
LIMIT 1
""",
(task_id, attempt),
).fetchone()
if row is None:
return None
value = dict(row)
value["payload"] = json.loads(value.pop("payload_json"))
return value
def verify_latest_binding(self, binding: CheckpointBinding) -> dict[str, Any] | None:
value = self.latest(binding.task_id)
if value is None:
return None
if int(value["attempt"]) > binding.attempt:
raise CheckpointError("CHECKPOINT_FUTURE_ATTEMPT")
# A previous attempt may be reused only when the immutable execution/scope binding matches.
if value["execution_digest"] != binding.execution_digest:
return None
expected = CheckpointBinding(
task_id=value["task_id"],
attempt=int(value["attempt"]),
execution_digest=value["execution_digest"],
job_id=value["job_id"],
repository=value["repository"],
base_oid=value["base_oid"],
expected_branch_oid=value["expected_branch_oid"],
target_branch=value["target_branch"],
generation=int(value["generation"]),
)
current_scope = binding.material()
historical_scope = expected.material()
current_scope.pop("attempt", None)
historical_scope.pop("attempt", None)
if sha256_json(current_scope) != sha256_json(historical_scope):
raise CheckpointError("CHECKPOINT_SCOPE_MISMATCH_DENY")
return value
@@ -1,463 +0,0 @@
"""Bounded read-only repository interface map for governed Online Agent analysis.
The compact map intentionally returns symbols and short source excerpts rather than a
source archive. It is designed for control-plane consumers that cannot safely ingest a
multi-megabyte repository snapshot.
"""
from __future__ import annotations
import ast
import hashlib
import json
from pathlib import Path
import re
from typing import Any, Iterable, Mapping, Sequence
SCHEMA = "aios.engineering.interface-map.v1"
MAX_RESULT_BYTES = 120 * 1024
TARGET_RESULT_BYTES = 108 * 1024
MAX_MODULES = 40
MAX_SYMBOLS = 80
MAX_TESTS = 40
MAX_EXCERPTS = 25
MAX_EXCERPT_CHARS = 1200
MAX_FILE_BYTES = 2 * 1024 * 1024
ROOTS = (
"active-runtime/app",
"runtime/agent-os",
"tests",
"db/migrations",
"migrations",
"deploy/systemd",
"docs/task-plane",
"docs/superpowers/specs",
)
SUFFIXES = {".py", ".sql", ".service", ".md", ".json", ".yaml", ".yml"}
DEFAULT_TERMS = {
"agent", "worker", "fleet", "scheduler", "scheduling", "task", "claim", "lease",
"heartbeat", "capacity", "concurrency", "priority", "fairness", "queue", "retry",
"backoff", "circuit", "health", "liveness", "scope", "tenant", "project", "trace",
"message", "checkpoint", "recovery", "idempotency", "dispatch", "assignment",
}
FLEET_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"FLEET_AGENT_REGISTRATION": ("agent", "register", "worker", "capability", "version"),
"FLEET_CAPACITY_SCHEDULING": ("capacity", "slot", "concurrency", "scheduler", "queue"),
"FLEET_FAIRNESS": ("fair", "priority", "queue", "tenant", "project", "starvation"),
"FLEET_FAULT_ISOLATION": ("failure", "health", "liveness", "timeout", "circuit", "poison"),
"FLEET_WORKER_REASSIGNMENT": ("reassign", "assignment", "claim", "lease", "retry", "worker"),
"FLEET_SCOPE_ISOLATION": ("tenant", "project", "scope", "conversation", "task", "run"),
"FLEET_IDEMPOTENT_DISPATCH": ("idempot", "dispatch", "claim", "lease", "result", "attempt"),
"FLEET_FAILURE_RECOVERY": ("checkpoint", "resume", "recover", "retry", "cleanup", "receipt"),
}
MODEL_GOVERNANCE_CONTRACT_TERMS: dict[str, tuple[str, ...]] = {
"MODEL_PROVIDER_REGISTRY": ("model", "provider", "registry", "endpoint", "capability"),
"MODEL_SELECTION_POLICY": ("model", "select", "route", "policy", "capability"),
"MODEL_VERSION_PINNING": ("model", "version", "pin", "revision", "snapshot"),
"MODEL_BUDGET_ACCOUNTING": ("budget", "token", "cost", "quota", "usage"),
"MODEL_FALLBACK_POLICY": ("fallback", "retry", "route", "provider", "model"),
"MODEL_SCOPE_ISOLATION": ("tenant", "project", "scope", "policy", "model"),
"MODEL_CREDENTIAL_BOUNDARY": ("credential", "secret", "provider", "tool", "token"),
"MODEL_TRACE_AUDIT": ("trace", "audit", "model", "provider", "request"),
"MODEL_EVALUATION_GATE": ("eval", "evaluation", "approval", "quality", "gate"),
"MODEL_ROUTING_RESILIENCE": ("route", "retry", "fallback", "timeout", "health"),
}
PROFILE_CLASSIFICATION = {
"fleet": "P2_INTERFACE_MAP_CAPTURED",
"model_governance": "MODEL_GOVERNANCE_INTERFACE_MAP_CAPTURED",
}
MODEL_PROFILE_MARKERS = {
"model", "provider", "llm", "inference", "routing", "router", "budget",
"token", "cost", "quota", "fallback", "evaluation", "eval", "approval",
"version", "pinning",
}
SENSITIVE_RE = re.compile(
r"(?i)(authorization\s*:\s*bearer\s+\S+|"
r"(?:password|passwd|secret|token|api[_-]?key)\s*[:=]\s*['\"]?[^'\"\s,;]+)"
)
SQL_TABLE_RE = re.compile(
r"(?is)\bcreate\s+table\s+(?:if\s+not\s+exists\s+)?[\"`[]?([A-Za-z_][A-Za-z0-9_.-]*)"
)
class CompactInterfaceMapError(RuntimeError):
pass
def _sha(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _bounded_excerpt(text: str, start_line: int, end_line: int) -> str:
lines = text.splitlines()
start = max(0, start_line - 2)
end = min(len(lines), max(end_line + 1, start + 1))
excerpt = "\n".join(lines[start:end])
excerpt = SENSITIVE_RE.sub("<redacted-sensitive-source-pattern>", excerpt)
if len(excerpt) > MAX_EXCERPT_CHARS:
excerpt = excerpt[: MAX_EXCERPT_CHARS - 1] + "…"
return excerpt
def _signal_terms(signals: Sequence[str]) -> set[str]:
terms = set(DEFAULT_TERMS)
for signal in signals:
if not isinstance(signal, str):
continue
terms.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
return set(sorted(terms)[:96])
def _profile(source_signals: Sequence[str]) -> str:
tokens: set[str] = set()
for signal in source_signals:
if not isinstance(signal, str):
continue
tokens.update(
token.lower()
for token in re.findall(r"[A-Za-z][A-Za-z0-9_.-]{2,63}", signal)
)
model_score = len(tokens & MODEL_PROFILE_MARKERS)
return "model_governance" if model_score >= 2 else "fleet"
def _contract_terms(profile: str) -> Mapping[str, tuple[str, ...]]:
if profile == "model_governance":
return MODEL_GOVERNANCE_CONTRACT_TERMS
if profile == "fleet":
return FLEET_CONTRACT_TERMS
raise CompactInterfaceMapError("INTERFACE_MAP_PROFILE_INVALID")
def _candidate_files(root: Path, terms: set[str]) -> list[dict[str, Any]]:
root = root.resolve()
candidates: list[dict[str, Any]] = []
for relative_root in ROOTS:
directory = (root / relative_root).resolve(strict=False)
if directory == root or root not in directory.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_SCOPE_INVALID")
if not directory.is_dir() or directory.is_symlink():
continue
for path in sorted(directory.rglob("*")):
if path.is_symlink() or not path.is_file() or path.suffix.lower() not in SUFFIXES:
continue
resolved = path.resolve(strict=True)
if root not in resolved.parents:
raise CompactInterfaceMapError("INTERFACE_MAP_PATH_ESCAPE")
data = path.read_bytes()
if len(data) > MAX_FILE_BYTES:
continue
relative = path.relative_to(root).as_posix()
text = data.decode("utf-8", "ignore")
searchable = (relative + "\n" + text[:384000]).lower()
matched = sorted(term for term in terms if term in searchable)
if not matched:
continue
path_bonus = sum(3 for term in terms if term in relative.lower())
test_bonus = 3 if relative.startswith("tests/") else 0
score = min(250, len(matched) + path_bonus + test_bonus)
candidates.append(
{
"path": relative,
"sha256": _sha(data),
"bytes": len(data),
"score": score,
"text": text,
"suffix": path.suffix.lower(),
}
)
candidates.sort(key=lambda item: (-item["score"], item["path"]))
return candidates
def _python_symbols(item: Mapping[str, Any], terms: set[str]) -> list[dict[str, Any]]:
try:
tree = ast.parse(str(item["text"]), filename=str(item["path"]))
except SyntaxError:
return []
result: list[dict[str, Any]] = []
class Visitor(ast.NodeVisitor):
def __init__(self) -> None:
self.stack: list[str] = []
def _record(self, node: ast.AST, kind: str, name: str) -> None:
qualified = ".".join([*self.stack, name]) if self.stack else name
haystack = f"{item['path']} {qualified}".lower()
score = sum(3 if term in haystack else 0 for term in terms)
if name.startswith("test_"):
score += 12
result.append(
{
"path": item["path"],
"symbol": qualified,
"kind": kind,
"line": int(getattr(node, "lineno", 1)),
"end_line": int(getattr(node, "end_lineno", getattr(node, "lineno", 1))),
"score": score + int(item["score"]),
}
)
def visit_ClassDef(self, node: ast.ClassDef) -> Any:
self._record(node, "class", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_FunctionDef(self, node: ast.FunctionDef) -> Any:
self._record(node, "function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> Any:
self._record(node, "async_function", node.name)
self.stack.append(node.name)
self.generic_visit(node)
self.stack.pop()
Visitor().visit(tree)
return result
def _surface_ref(symbol: Mapping[str, Any]) -> str:
return f"{symbol['path']}::{symbol['symbol']}"
def _contract_map(
symbols: Sequence[Mapping[str, Any]],
modules: Sequence[Mapping[str, Any]],
contract_terms: Mapping[str, tuple[str, ...]],
) -> dict[str, Any]:
result: dict[str, Any] = {}
module_paths = [str(item["path"]) for item in modules]
for contract, terms in contract_terms.items():
scored: list[tuple[int, str]] = []
for symbol in symbols:
haystack = _surface_ref(symbol).lower()
score = sum(1 for term in terms if term in haystack)
if score:
scored.append((score, _surface_ref(symbol)))
scored.sort(key=lambda item: (-item[0], item[1]))
surfaces = [value for _, value in scored[:6]]
if not surfaces:
for path in module_paths:
score = sum(1 for term in terms if term in path.lower())
if score:
surfaces.append(path)
if len(surfaces) == 6:
break
result[contract] = {
"status": "PARTIAL" if surfaces else "GAP",
"surfaces": surfaces,
}
return result
def _gap_evidence(
symbols: Sequence[Mapping[str, Any]],
profile: str,
) -> tuple[list[str], list[str], list[str]]:
haystack = "\n".join(_surface_ref(item).lower() for item in symbols)
if profile == "model_governance":
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Provider/model credentials remain outside uncredentialed planner/executor processes.",
"P0-A tenant/project scope and P0-C trace/audit lineage must constrain model routing.",
"P1 retry/checkpoint semantics remain authoritative for model-call execution recovery.",
]
failure_windows: list[str] = []
if "provider" not in haystack and "model" not in haystack:
failure_windows.append("No first-class model/provider registry surface detected.")
if "budget" not in haystack and "cost" not in haystack and "quota" not in haystack:
failure_windows.append("No explicit durable model budget/cost/quota accounting surface detected.")
if "fallback" not in haystack and "route" not in haystack:
failure_windows.append("No explicit model fallback/routing policy surface detected.")
if "eval" not in haystack and "approval" not in haystack:
failure_windows.append("No explicit evaluation/approval gate surface detected for model changes.")
if "version" not in haystack and "pin" not in haystack:
failure_windows.append("No explicit model-version pinning surface detected.")
desired = (
("trace", "P0-C trace/audit lineage"),
("scope", "P0-A tenant/project scope isolation"),
("credential", "P0-C credential governance"),
("checkpoint", "P1 durable execution recovery"),
("message", "P0-B MessageEnvelope/handoff"),
("retry", "P1 retry/idempotency"),
)
else:
invariants = [
"Online Task claim/heartbeat remains the authoritative worker ownership boundary.",
"Governed mutation remains separated from read-only analysis and requires WriterLease.",
"P1 attempt-aware result idempotency and durable checkpoint recovery are reusable fleet primitives.",
"Scope/trace/message lineage must remain attached when work is scheduled or reassigned.",
]
failure_windows = []
if "scheduler" not in haystack:
failure_windows.append("No first-class durable fleet scheduler symbol detected; worker-local polling may still be the scheduling mechanism.")
if "capacity" not in haystack and "slot" not in haystack:
failure_windows.append("No explicit durable worker-capacity/slot primitive detected.")
if "fair" not in haystack and "starvation" not in haystack:
failure_windows.append("No explicit fairness/starvation policy symbol detected.")
if "circuit" not in haystack:
failure_windows.append("No explicit circuit-breaker primitive detected for repeated worker/task failures.")
if "reassign" not in haystack:
failure_windows.append("No explicit worker-reassignment primitive detected beyond lease expiry/retry semantics.")
desired = (
("checkpoint", "P1 durable checkpoint/resume"),
("message", "P0-B MessageEnvelope/handoff"),
("trace", "P0-C trace correlation"),
("scope", "P0-A scope isolation"),
("lease", "claim/WriterLease ownership"),
("heartbeat", "worker liveness"),
("idempot", "retry-safe dispatch/result"),
)
reuse_points: list[str] = []
for token, label in desired:
match = next((item for item in symbols if token in _surface_ref(item).lower()), None)
if match is not None:
reuse_points.append(f"{label}: {_surface_ref(match)}")
return invariants, failure_windows, reuse_points
def _trim_to_budget(value: dict[str, Any]) -> dict[str, Any]:
def size() -> int:
return len(json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")).encode("utf-8"))
while size() > TARGET_RESULT_BYTES and value["bounded_excerpts"]:
value["bounded_excerpts"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["functions"]) > 20:
value["implementation_map"]["functions"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["classes"]) > 15:
value["implementation_map"]["classes"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["tests"]) > 15:
value["implementation_map"]["tests"].pop()
while size() > TARGET_RESULT_BYTES and len(value["implementation_map"]["modules"]) > 20:
value["implementation_map"]["modules"].pop()
final_size = size()
if final_size > MAX_RESULT_BYTES:
raise CompactInterfaceMapError("INTERFACE_MAP_RESULT_TOO_LARGE")
value["result_bytes"] = final_size
return value
def build_compact_interface_map(
worktree: Path,
source_signals: Sequence[str],
*,
source_commit: str,
) -> dict[str, Any]:
root = Path(worktree)
if not root.is_dir() or root.is_symlink():
raise CompactInterfaceMapError("INTERFACE_MAP_WORKTREE_INVALID")
if re.fullmatch(r"[0-9a-f]{40}", source_commit) is None:
raise CompactInterfaceMapError("INTERFACE_MAP_SOURCE_COMMIT_INVALID")
profile = _profile(source_signals)
terms = _signal_terms(source_signals)
candidates = _candidate_files(root, terms)
if not candidates:
raise CompactInterfaceMapError("INTERFACE_MAP_EMPTY")
module_candidates = candidates[:MAX_MODULES]
modules = [
{
"path": item["path"],
"sha256": item["sha256"],
"bytes": item["bytes"],
"score": item["score"],
}
for item in module_candidates
]
symbols: list[dict[str, Any]] = []
tables: list[dict[str, Any]] = []
systemd_units: list[dict[str, Any]] = []
for item in candidates:
if item["suffix"] == ".py":
symbols.extend(_python_symbols(item, terms))
elif item["suffix"] == ".sql":
for table in SQL_TABLE_RE.findall(str(item["text"])):
tables.append({"path": item["path"], "table": table, "sha256": item["sha256"]})
elif item["suffix"] == ".service":
systemd_units.append({"path": item["path"], "unit": Path(str(item["path"])).name, "sha256": item["sha256"]})
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
all_symbols = list(symbols)
test_symbols = [
item for item in all_symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][: min(MAX_TESTS, 24)]
non_test_symbols = [
item for item in all_symbols
if item not in test_symbols
][: max(0, MAX_SYMBOLS - len(test_symbols))]
symbols = [*test_symbols, *non_test_symbols]
symbols.sort(key=lambda item: (-int(item["score"]), str(item["path"]), str(item["symbol"])))
classes = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] == "class"
]
functions = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if item["kind"] in {"function", "async_function"} and not str(item["symbol"]).split(".")[-1].startswith("test_")
]
tests = [
{k: item[k] for k in ("path", "symbol", "line", "end_line")}
for item in symbols
if str(item["path"]).startswith("tests/") or str(item["symbol"]).split(".")[-1].startswith("test_")
][:MAX_TESTS]
excerpts: list[dict[str, Any]] = []
by_path = {str(item["path"]): item for item in candidates}
for symbol in symbols:
if len(excerpts) == MAX_EXCERPTS:
break
item = by_path.get(str(symbol["path"]))
if item is None:
continue
excerpt = _bounded_excerpt(str(item["text"]), int(symbol["line"]), int(symbol["end_line"]))
if not excerpt.strip():
continue
excerpts.append(
{
"path": symbol["path"],
"symbol": symbol["symbol"],
"sha256": item["sha256"],
"excerpt": excerpt,
}
)
invariants, failure_windows, reuse_points = _gap_evidence(all_symbols, profile)
result = {
"schema": SCHEMA,
"profile": profile,
"classification": PROFILE_CLASSIFICATION[profile],
"source_commit": source_commit,
"implementation_map": {
"modules": modules,
"classes": classes[:MAX_SYMBOLS],
"functions": functions[:MAX_SYMBOLS],
"tables": tables[:40],
"systemd_units": systemd_units[:20],
"tests": tests,
"current_invariants": invariants,
"failure_windows": failure_windows,
"reuse_points": reuse_points,
},
"bounded_excerpts": excerpts,
"contracts": _contract_map(all_symbols, modules, _contract_terms(profile)),
}
return _trim_to_budget(result)
@@ -1,187 +0,0 @@
"""Durable, authority-bound deterministic execution plans for Online Agent Tasks."""
from __future__ import annotations
from contextlib import contextmanager
from dataclasses import dataclass
import fcntl
import hashlib
import json
from pathlib import Path
from typing import Any, Mapping
from aios_online_agent_execution import (
GovernedExecutionBinding,
project_execution_envelope,
)
class ExecutionPlanError(RuntimeError):
pass
def _canonical(value: object) -> bytes:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
def _digest(value: object) -> str:
return hashlib.sha256(_canonical(value)).hexdigest()
def _task_key(task_id: str) -> str:
readable = "".join(ch if ch.isalnum() or ch in "._-" else "-" for ch in task_id).strip("-")[:72] or "task"
return readable + "-" + hashlib.sha256(task_id.encode("utf-8")).hexdigest()[:24]
@dataclass(frozen=True)
class ExecutionPlanRecord:
task_id: str
job_id: str
source_head_oid: str
generation: int
idempotency_key: str
plan_digest: str
execution: dict[str, Any]
def public(self) -> dict[str, Any]:
return {
"schema": "aios.online-agent.plan-record.v1",
"task_id": self.task_id,
"job_id": self.job_id,
"source_head_oid": self.source_head_oid,
"generation": self.generation,
"idempotency_key": self.idempotency_key,
"plan_digest": self.plan_digest,
"execution": dict(self.execution),
}
class ExecutionPlanStore:
def __init__(self, root: Path) -> None:
self.root = Path(root)
self.root.mkdir(parents=True, exist_ok=True)
def _path(self, task_id: str) -> Path:
return self.root / (_task_key(task_id) + ".json")
@contextmanager
def _lock(self, task_id: str):
path = self.root / (_task_key(task_id) + ".lock")
with path.open("a+b") as handle:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX)
try:
yield
finally:
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
@staticmethod
def _source_head(binding: GovernedExecutionBinding) -> str:
return binding.expected_branch_oid if binding.branch_mode == "CONTINUE_EXISTING" else binding.base_oid
def _read(self, task_id: str) -> ExecutionPlanRecord | None:
path = self._path(task_id)
if not path.is_file() or path.is_symlink():
return None
value = json.loads(path.read_text(encoding="utf-8"))
if not isinstance(value, Mapping) or value.get("schema") != "aios.online-agent.plan-record.v1":
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
execution = value.get("execution")
if not isinstance(execution, Mapping):
raise ExecutionPlanError("EXECUTION_PLAN_RECORD_INVALID")
return ExecutionPlanRecord(
task_id=str(value.get("task_id") or ""),
job_id=str(value.get("job_id") or ""),
source_head_oid=str(value.get("source_head_oid") or ""),
generation=int(value.get("generation") or 0),
idempotency_key=str(value.get("idempotency_key") or ""),
plan_digest=str(value.get("plan_digest") or ""),
execution=dict(execution),
)
def register(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
with self._lock(binding.task_id):
return self._register_locked(binding, request)
def _register_locked(
self,
binding: GovernedExecutionBinding,
request: Mapping[str, Any],
) -> dict[str, Any]:
if request.get("schema") != "aios.online-agent.plan.v1":
raise ExecutionPlanError("EXECUTION_PLAN_SCHEMA_INVALID")
if request.get("task_id") != binding.task_id or request.get("job_id") != binding.job_id:
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if int(request.get("generation") or 0) != binding.generation:
raise ExecutionPlanError("EXECUTION_PLAN_GENERATION_CONFLICT")
source_head = str(request.get("source_head_oid") or "")
if source_head != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_CONFLICT")
key = str(request.get("idempotency_key") or "")
if len(key) < 8:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_INVALID")
execution_request = request.get("execution")
if not isinstance(execution_request, Mapping) or execution_request.get("action") != "deterministic":
raise ExecutionPlanError("EXECUTION_PLAN_DETERMINISTIC_REQUIRED")
envelope = project_execution_envelope(binding, execution_request)
material = {
"task_id": binding.task_id,
"job_id": binding.job_id,
"source_head_oid": source_head,
"generation": binding.generation,
"idempotency_key": key,
"execution": dict(execution_request),
}
plan_digest = _digest(material)
existing = self._read(binding.task_id)
expected_previous = request.get("expected_previous_plan_digest")
if existing is not None:
if existing.idempotency_key == key:
if existing.plan_digest != plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_IDEMPOTENCY_CONFLICT")
return {**existing.public(), "idempotent": True, "execution_digest": envelope["execution_digest"]}
if expected_previous != existing.plan_digest:
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
if existing.source_head_oid != source_head:
raise ExecutionPlanError("EXECUTION_PLAN_STALE_REPLACEMENT_DENY")
elif expected_previous not in (None, ""):
raise ExecutionPlanError("EXECUTION_PLAN_REPLACEMENT_CAS_MISMATCH")
record = ExecutionPlanRecord(
task_id=binding.task_id,
job_id=binding.job_id,
source_head_oid=source_head,
generation=binding.generation,
idempotency_key=key,
plan_digest=plan_digest,
execution=dict(execution_request),
)
path = self._path(binding.task_id)
temporary = path.with_suffix(".tmp")
temporary.write_text(
json.dumps(record.public(), sort_keys=True, separators=(",", ":")),
encoding="utf-8",
)
temporary.replace(path)
return {**record.public(), "idempotent": False, "execution_digest": envelope["execution_digest"]}
def resolve(self, binding: GovernedExecutionBinding) -> dict[str, Any] | None:
record = self._read(binding.task_id)
if record is None:
return None
if (
record.task_id != binding.task_id
or record.job_id != binding.job_id
or record.generation != binding.generation
):
raise ExecutionPlanError("EXECUTION_PLAN_BINDING_CONFLICT")
if record.source_head_oid != self._source_head(binding):
raise ExecutionPlanError("EXECUTION_PLAN_SOURCE_HEAD_DRIFT")
envelope = project_execution_envelope(binding, record.execution)
return {
"execution": dict(record.execution),
"plan_digest": record.plan_digest,
"execution_digest": envelope["execution_digest"],
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,527 @@
"""Owner-final, structured authorization for one governed pull-request merge.
The Owner approval is the policy decision. Signing and public verification
prove that decision but cannot change its repository, pull request, CAS base,
technical-gate contract, or lifetime. This module deliberately exposes no
generic sign-bytes API and does not publish or alter technical check results.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import hashlib
import os
from pathlib import Path
import re
import secrets
import sqlite3
import time
from typing import Any, Callable, Mapping, Protocol
from aios_single_host_authority import (
OWNER_NAMESPACE,
LocalFileSigner,
SignedArtifact,
canonical_json_bytes,
sha256_json,
)
POLICY_SCHEMA = "aios.owner-merge-approval-policy.v2"
APPROVAL_SCHEMA = "aios.owner-merge-approval.v2"
RECEIPT_SCHEMA = "aios.owner-merge-admission-receipt.v2"
OPERATION = "merge.pull-request"
OID_RE = re.compile(r"^[0-9a-f]{40}$")
DIGEST_RE = re.compile(r"^[0-9a-f]{64}$")
REPOSITORY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}/[A-Za-z0-9][A-Za-z0-9._-]{0,127}$")
IDENTITY_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/+:-]{2,255}$")
CHECK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{1,127}$")
TOKEN_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_-]{11,255}$")
class OwnerMergeApprovalError(RuntimeError):
"""Stable fail-closed error for the Owner merge approval contract."""
def __init__(self, code: str):
super().__init__(code)
self.code = code
def _fail(code: str) -> None:
raise OwnerMergeApprovalError(code)
def _require_oid(value: object, code: str) -> str:
if not isinstance(value, str) or OID_RE.fullmatch(value) is None:
_fail(code)
return value
def _require_digest(value: object, code: str) -> str:
if not isinstance(value, str) or DIGEST_RE.fullmatch(value) is None:
_fail(code)
return value
@dataclass(frozen=True, slots=True)
class OwnerMergeApprovalPolicy:
"""Immutable policy input for one exact protected merge approval."""
repository: str
pr_number: int
pr_head_oid: str
expected_main_oid: str
operation: str
required_check_contract_digest: str
required_checks: tuple[str, ...]
owner_identity: str
audit_lineage_digest: str
maximum_ttl_seconds: int
policy_version: str = "v2"
schema: str = POLICY_SCHEMA
def validate(self) -> None:
if (
self.schema != POLICY_SCHEMA
or self.policy_version != "v2"
or not isinstance(self.repository, str)
or REPOSITORY_RE.fullmatch(self.repository) is None
or type(self.pr_number) is not int
or not 1 <= self.pr_number < 2**31
or self.operation != OPERATION
or not isinstance(self.owner_identity, str)
or IDENTITY_RE.fullmatch(self.owner_identity) is None
or type(self.maximum_ttl_seconds) is not int
or not 60 <= self.maximum_ttl_seconds <= 900
):
_fail("OWNER_MERGE_POLICY_INVALID")
_require_oid(self.pr_head_oid, "OWNER_MERGE_POLICY_HEAD_INVALID")
_require_oid(self.expected_main_oid, "OWNER_MERGE_POLICY_BASE_INVALID")
_require_digest(
self.required_check_contract_digest,
"OWNER_MERGE_POLICY_CHECK_CONTRACT_INVALID",
)
_require_digest(self.audit_lineage_digest, "OWNER_MERGE_POLICY_LINEAGE_INVALID")
if (
not isinstance(self.required_checks, tuple)
or not self.required_checks
or len(set(self.required_checks)) != len(self.required_checks)
or any(not isinstance(item, str) or CHECK_RE.fullmatch(item) is None for item in self.required_checks)
):
_fail("OWNER_MERGE_POLICY_REQUIRED_CHECKS_INVALID")
def as_dict(self) -> dict[str, Any]:
self.validate()
value = asdict(self)
value["required_checks"] = list(self.required_checks)
return value
@property
def digest(self) -> str:
return sha256_json(self.as_dict())
@dataclass(frozen=True, slots=True)
class OwnerMergeLiveState:
"""Read-only live evidence; this object has no status publication method."""
repository: str
pr_number: int
pr_head_oid: str
main_oid: str
required_check_contract_digest: str
check_statuses: Mapping[str, bool]
@dataclass(frozen=True, slots=True)
class OwnerMergeOidReadback:
authority_main_oid: str
gitea_main_oid: str
api_main_oid: str
@dataclass(frozen=True, slots=True)
class OwnerMergeAdmissionReceipt:
approval_id: str
nonce: str
owner_identity: str
policy_digest: str
proof_path: str
merge_oid: str
schema: str = RECEIPT_SCHEMA
class ApprovalProofVerifier(Protocol):
"""Public proof verifier; it is intentionally unable to issue approvals."""
def verify(
self,
*,
namespace: str,
payload: bytes,
signature: str,
key_id: str,
) -> bool:
...
class GovernedMergeExecutor(Protocol):
def execute(self, approval: Mapping[str, object]) -> str:
...
class MergeOidReconciler(Protocol):
def readback(self, merge_oid: str) -> OwnerMergeOidReadback:
...
APPROVAL_FIELDS = frozenset(
{
"schema",
"approval_id",
"nonce",
"repository",
"pr_number",
"pr_head_oid",
"expected_main_oid",
"operation",
"required_check_contract_digest",
"required_checks",
"owner_identity",
"audit_lineage_digest",
"policy_version",
"policy_digest",
"issued_at",
"not_before",
"expires_at",
"signature_namespace",
"signature_key_id",
"canonical_payload_digest",
}
)
class OwnerMergeApprovalAuthority:
"""Structured Owner signer for an immutable merge policy.
``issue`` has no caller-controlled payload parameter. The immutable policy
selects the complete authorization scope; only approval IDs and nonces are
generated inside the authority.
"""
def __init__(
self,
key_path: Path,
policy: OwnerMergeApprovalPolicy,
*,
require_root_owner: bool = True,
clock: Callable[[], float] = time.time,
token_urlsafe: Callable[[int], str] = secrets.token_urlsafe,
) -> None:
policy.validate()
self.policy = policy
self.signer = LocalFileSigner(
Path(key_path),
signer_id="local-owner-authority",
namespace=OWNER_NAMESPACE,
require_root_owner=require_root_owner,
)
self.clock = clock
self.token_urlsafe = token_urlsafe
def _token(self, prefix: str) -> str:
value = prefix + self.token_urlsafe(24)
if TOKEN_RE.fullmatch(value) is None:
_fail("OWNER_MERGE_AUTHORITY_GENERATED_TOKEN_INVALID")
return value
def issue(self) -> SignedArtifact:
now = int(self.clock())
approval_id = self._token("owner-merge-approval-")
nonce = self._token("owner-merge-nonce-")
policy = self.policy
payload: dict[str, Any] = {
"schema": APPROVAL_SCHEMA,
"approval_id": approval_id,
"nonce": nonce,
"repository": policy.repository,
"pr_number": policy.pr_number,
"pr_head_oid": policy.pr_head_oid,
"expected_main_oid": policy.expected_main_oid,
"operation": policy.operation,
"required_check_contract_digest": policy.required_check_contract_digest,
"required_checks": list(policy.required_checks),
"owner_identity": policy.owner_identity,
"audit_lineage_digest": policy.audit_lineage_digest,
"policy_version": policy.policy_version,
"policy_digest": policy.digest,
"issued_at": now,
"not_before": max(0, now - 5),
"expires_at": now + policy.maximum_ttl_seconds,
"signature_namespace": OWNER_NAMESPACE,
"signature_key_id": self.signer.fingerprint,
}
payload["canonical_payload_digest"] = sha256_json(payload)
return self.signer._sign_canonical(payload)
class OwnerMergeApprovalVerifier:
"""Verify Owner proof, preferring normal verification then recovery proof.
The recovery verifier validates a pre-existing approval with the registered
public Owner root. It cannot issue an approval and is therefore not a
second policy principal.
"""
def __init__(
self,
*,
primary: ApprovalProofVerifier,
recovery: ApprovalProofVerifier | None = None,
clock: Callable[[], float] = time.time,
) -> None:
self.primary = primary
self.recovery = recovery
self.clock = clock
@staticmethod
def _artifact(artifact: SignedArtifact | Mapping[str, Any]) -> tuple[dict[str, Any], str, str, str]:
if isinstance(artifact, SignedArtifact):
return dict(artifact.payload), artifact.signature, artifact.key_id, artifact.namespace
if not isinstance(artifact, Mapping) or set(artifact) != {"payload", "signature", "key_id", "namespace"}:
_fail("OWNER_MERGE_APPROVAL_ARTIFACT_INVALID")
payload = artifact.get("payload")
signature = artifact.get("signature")
key_id = artifact.get("key_id")
namespace = artifact.get("namespace")
if not isinstance(payload, Mapping) or not all(isinstance(item, str) for item in (signature, key_id, namespace)):
_fail("OWNER_MERGE_APPROVAL_ARTIFACT_INVALID")
return dict(payload), signature, key_id, namespace
@staticmethod
def _validate_scope(payload: Mapping[str, Any], policy: OwnerMergeApprovalPolicy, *, now: int, key_id: str, namespace: str) -> None:
if set(payload) != APPROVAL_FIELDS:
_fail("OWNER_MERGE_APPROVAL_FIELDS_INVALID")
expected = {
"schema": APPROVAL_SCHEMA,
"repository": policy.repository,
"pr_number": policy.pr_number,
"pr_head_oid": policy.pr_head_oid,
"expected_main_oid": policy.expected_main_oid,
"operation": policy.operation,
"required_check_contract_digest": policy.required_check_contract_digest,
"required_checks": list(policy.required_checks),
"owner_identity": policy.owner_identity,
"audit_lineage_digest": policy.audit_lineage_digest,
"policy_version": policy.policy_version,
"policy_digest": policy.digest,
"signature_namespace": OWNER_NAMESPACE,
"signature_key_id": key_id,
}
if payload.get("pr_head_oid") != policy.pr_head_oid:
_fail("OWNER_MERGE_APPROVAL_HEAD_BINDING_INVALID")
if payload.get("expected_main_oid") != policy.expected_main_oid:
_fail("OWNER_MERGE_APPROVAL_BASE_BINDING_INVALID")
if payload.get("policy_digest") != policy.digest:
_fail("OWNER_MERGE_POLICY_DIGEST_DRIFT")
if namespace != OWNER_NAMESPACE or any(payload.get(key) != value for key, value in expected.items()):
_fail("OWNER_MERGE_APPROVAL_SCOPE_INVALID")
for field in ("approval_id", "nonce"):
if not isinstance(payload.get(field), str) or TOKEN_RE.fullmatch(payload[field]) is None:
_fail("OWNER_MERGE_APPROVAL_TOKEN_INVALID")
for field in ("issued_at", "not_before", "expires_at"):
if type(payload.get(field)) is not int:
_fail("OWNER_MERGE_APPROVAL_TIME_INVALID")
if not payload["not_before"] <= now < payload["expires_at"]:
_fail("OWNER_MERGE_APPROVAL_EXPIRED")
if payload["issued_at"] > now + 5 or payload["expires_at"] - payload["issued_at"] != policy.maximum_ttl_seconds:
_fail("OWNER_MERGE_APPROVAL_TIME_INVALID")
supplied = payload.get("canonical_payload_digest")
body = dict(payload)
body.pop("canonical_payload_digest", None)
if supplied != sha256_json(body):
_fail("OWNER_MERGE_APPROVAL_CANONICAL_DIGEST_INVALID")
def verify(
self,
artifact: SignedArtifact | Mapping[str, Any],
policy: OwnerMergeApprovalPolicy,
) -> tuple[dict[str, Any], str]:
policy.validate()
payload, signature, key_id, namespace = self._artifact(artifact)
canonical = canonical_json_bytes(payload)
proof_path: str | None = None
for path, verifier in (("primary", self.primary), ("recovery", self.recovery)):
if verifier is None:
continue
try:
verified = verifier.verify(
namespace=namespace,
payload=canonical,
signature=signature,
key_id=key_id,
)
except Exception:
verified = False
if verified is True:
proof_path = path
break
if proof_path is None:
_fail("OWNER_MERGE_APPROVAL_SIGNATURE_INVALID")
self._validate_scope(payload, policy, now=int(self.clock()), key_id=key_id, namespace=namespace)
return payload, proof_path
class SQLiteOwnerMergeNonceLedger:
"""Durable one-shot fence. Unknown effects are intentionally not retried."""
def __init__(self, path: Path) -> None:
self.path = Path(path)
def _connection(self) -> sqlite3.Connection:
if self.path.is_symlink() or self.path.parent.is_symlink():
_fail("OWNER_MERGE_NONCE_LEDGER_PATH_INVALID")
self.path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
connection = sqlite3.connect(self.path, timeout=5, isolation_level=None)
connection.execute(
"CREATE TABLE IF NOT EXISTS owner_merge_nonce ("
"nonce TEXT PRIMARY KEY, approval_digest TEXT NOT NULL, state TEXT NOT NULL, claimed_at INTEGER NOT NULL)"
)
return connection
def claim(self, *, nonce: str, approval_digest: str, now: int) -> None:
connection = self._connection()
try:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT state FROM owner_merge_nonce WHERE nonce=?", (nonce,)
).fetchone()
if existing is not None:
connection.rollback()
_fail("OWNER_MERGE_NONCE_REPLAY")
connection.execute(
"INSERT INTO owner_merge_nonce(nonce,approval_digest,state,claimed_at) VALUES(?,?,?,?)",
(nonce, approval_digest, "PENDING", now),
)
connection.commit()
except OwnerMergeApprovalError:
raise
except sqlite3.Error as exc:
try:
connection.rollback()
except sqlite3.Error:
pass
raise OwnerMergeApprovalError("OWNER_MERGE_NONCE_LEDGER_UNAVAILABLE") from exc
finally:
connection.close()
def mark(self, *, nonce: str, state: str) -> None:
if state not in {"COMPLETED", "UNKNOWN"}:
_fail("OWNER_MERGE_NONCE_STATE_INVALID")
connection = self._connection()
try:
connection.execute("BEGIN IMMEDIATE")
updated = connection.execute(
"UPDATE owner_merge_nonce SET state=? WHERE nonce=? AND state='PENDING'",
(state, nonce),
).rowcount
if updated != 1:
connection.rollback()
_fail("OWNER_MERGE_NONCE_STATE_CONFLICT")
connection.commit()
except OwnerMergeApprovalError:
raise
except sqlite3.Error as exc:
try:
connection.rollback()
except sqlite3.Error:
pass
raise OwnerMergeApprovalError("OWNER_MERGE_NONCE_LEDGER_UNAVAILABLE") from exc
finally:
connection.close()
class GovernedOwnerMergeAdmission:
"""Admit one protected merge without owning any policy or technical gate."""
def __init__(
self,
*,
policy: OwnerMergeApprovalPolicy,
verifier: OwnerMergeApprovalVerifier,
live_state_reader: Callable[[], OwnerMergeLiveState],
executor: GovernedMergeExecutor,
reconciler: MergeOidReconciler,
nonce_ledger: SQLiteOwnerMergeNonceLedger,
clock: Callable[[], float] = time.time,
) -> None:
policy.validate()
self.policy = policy
self.verifier = verifier
self.live_state_reader = live_state_reader
self.executor = executor
self.reconciler = reconciler
self.nonce_ledger = nonce_ledger
self.clock = clock
def _validate_live_state(self, live: OwnerMergeLiveState) -> None:
if not isinstance(live, OwnerMergeLiveState):
_fail("OWNER_MERGE_LIVE_STATE_INVALID")
policy = self.policy
if live.repository != policy.repository:
_fail("OWNER_MERGE_REPOSITORY_DRIFT")
if live.pr_number != policy.pr_number:
_fail("OWNER_MERGE_PR_BINDING_DRIFT")
if live.pr_head_oid != policy.pr_head_oid:
_fail("OWNER_MERGE_HEAD_DRIFT")
if live.main_oid != policy.expected_main_oid:
_fail("OWNER_MERGE_BASE_DRIFT")
if live.required_check_contract_digest != policy.required_check_contract_digest:
_fail("OWNER_MERGE_REQUIRED_CHECK_CONTRACT_DRIFT")
if not isinstance(live.check_statuses, Mapping) or set(live.check_statuses) != set(policy.required_checks):
_fail("OWNER_MERGE_REQUIRED_CHECK_CONTRACT_DRIFT")
if any(value is not True for value in live.check_statuses.values()):
_fail("OWNER_MERGE_REQUIRED_CHECKS_FAILED")
def admit(self, artifact: SignedArtifact | Mapping[str, Any]) -> OwnerMergeAdmissionReceipt:
approval, proof_path = self.verifier.verify(artifact, self.policy)
self._validate_live_state(self.live_state_reader())
nonce = str(approval["nonce"])
approval_digest = hashlib.sha256(canonical_json_bytes(approval)).hexdigest()
self.nonce_ledger.claim(nonce=nonce, approval_digest=approval_digest, now=int(self.clock()))
try:
merge_oid = self.executor.execute(dict(approval))
_require_oid(merge_oid, "OWNER_MERGE_EXECUTOR_RESULT_INVALID")
readback = self.reconciler.readback(merge_oid)
if (
not isinstance(readback, OwnerMergeOidReadback)
or readback.authority_main_oid != merge_oid
or readback.gitea_main_oid != merge_oid
or readback.api_main_oid != merge_oid
):
self.nonce_ledger.mark(nonce=nonce, state="UNKNOWN")
_fail("OWNER_MERGE_OID_RECONCILIATION_REQUIRED")
except OwnerMergeApprovalError:
# Every failure after a durable claim represents an operation that
# is either known failed or needs reconciliation. Preserve the
# one-shot fence rather than exposing a retryable PENDING state.
try:
self.nonce_ledger.mark(nonce=nonce, state="UNKNOWN")
except OwnerMergeApprovalError:
pass
raise
except Exception as exc:
try:
self.nonce_ledger.mark(nonce=nonce, state="UNKNOWN")
except OwnerMergeApprovalError:
pass
raise OwnerMergeApprovalError("OWNER_MERGE_EXECUTION_UNKNOWN") from exc
self.nonce_ledger.mark(nonce=nonce, state="COMPLETED")
return OwnerMergeAdmissionReceipt(
approval_id=str(approval["approval_id"]),
nonce=nonce,
owner_identity=str(approval["owner_identity"]),
policy_digest=str(approval["policy_digest"]),
proof_path=proof_path,
merge_oid=merge_oid,
)
@@ -0,0 +1,287 @@
"""Restricted issuance service for ``aios.owner-merge-approval.v2``.
The service has one Owner-only operation. It accepts an empty request payload,
loads one immutable policy, generates approval identifiers and nonces itself,
and stores one exact issuance response. It deliberately does not expose a
generic signing, payload, or policy-selection API.
"""
from __future__ import annotations
import json
import os
from pathlib import Path
import sqlite3
import stat
import time
from typing import Any, Callable, Mapping
from aios_owner_merge_authorization import (
OwnerMergeApprovalAuthority,
OwnerMergeApprovalError,
OwnerMergeApprovalPolicy,
OwnerMergeApprovalVerifier,
)
from aios_single_host_authority import RegistryOwnerApprovalVerifier, SignedArtifact
ISSUE_OPERATION = "issue_owner_merge_approval"
def _fail(code: str) -> None:
raise OwnerMergeApprovalError(code)
def _secure_regular(path: Path, *, require_root_owner: bool) -> None:
try:
metadata = path.lstat()
except OSError as exc:
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_FILE_MISSING") from exc
if path.is_symlink() or not stat.S_ISREG(metadata.st_mode):
_fail("OWNER_MERGE_ISSUER_FILE_INVALID")
if stat.S_IMODE(metadata.st_mode) & 0o022:
_fail("OWNER_MERGE_ISSUER_FILE_WRITABLE")
if require_root_owner and metadata.st_uid != 0:
_fail("OWNER_MERGE_ISSUER_FILE_NOT_ROOT_OWNED")
def _read_policy(path: Path, *, require_root_owner: bool) -> OwnerMergeApprovalPolicy:
_secure_regular(path, require_root_owner=require_root_owner)
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_POLICY_INVALID") from exc
if not isinstance(value, Mapping):
_fail("OWNER_MERGE_ISSUER_POLICY_INVALID")
expected = set(OwnerMergeApprovalPolicy.__dataclass_fields__)
if set(value) != expected or not isinstance(value.get("required_checks"), list):
_fail("OWNER_MERGE_ISSUER_POLICY_INVALID")
try:
policy = OwnerMergeApprovalPolicy(
repository=value["repository"],
pr_number=value["pr_number"],
pr_head_oid=value["pr_head_oid"],
expected_main_oid=value["expected_main_oid"],
operation=value["operation"],
required_check_contract_digest=value["required_check_contract_digest"],
required_checks=tuple(value["required_checks"]),
owner_identity=value["owner_identity"],
audit_lineage_digest=value["audit_lineage_digest"],
maximum_ttl_seconds=value["maximum_ttl_seconds"],
policy_version=value["policy_version"],
schema=value["schema"],
)
policy.validate()
except (KeyError, TypeError, OwnerMergeApprovalError) as exc:
if isinstance(exc, OwnerMergeApprovalError):
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_POLICY_INVALID") from exc
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_POLICY_INVALID") from exc
return policy
def _artifact_dict(artifact: SignedArtifact) -> dict[str, Any]:
return {
"payload": dict(artifact.payload),
"signature": artifact.signature,
"key_id": artifact.key_id,
"namespace": artifact.namespace,
}
class SQLiteOwnerMergeIssuanceLedger:
"""Durably replay a current approval and retain expired issuance history."""
def __init__(self, path: Path) -> None:
self.path = Path(path)
def _connect(self) -> sqlite3.Connection:
if self.path.is_symlink() or self.path.parent.is_symlink():
_fail("OWNER_MERGE_ISSUER_LEDGER_PATH_INVALID")
self.path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
connection = sqlite3.connect(self.path, timeout=5, isolation_level=None)
connection.execute(
"CREATE TABLE IF NOT EXISTS owner_merge_issuance ("
"nonce TEXT PRIMARY KEY, policy_digest TEXT NOT NULL, response_json TEXT NOT NULL, "
"issued_at INTEGER NOT NULL, expires_at INTEGER NOT NULL)"
)
connection.execute(
"CREATE INDEX IF NOT EXISTS owner_merge_issuance_policy_expiry "
"ON owner_merge_issuance(policy_digest, expires_at)"
)
return connection
def issue_once(
self,
*,
policy_digest: str,
now: int,
issuer: Callable[[], dict[str, Any]],
validate_current: Callable[[dict[str, Any]], None],
expiry_of: Callable[[dict[str, Any]], int],
) -> tuple[dict[str, Any], bool]:
connection = self._connect()
try:
connection.execute("BEGIN IMMEDIATE")
rows = connection.execute(
"SELECT response_json, expires_at FROM owner_merge_issuance "
"WHERE policy_digest=? AND expires_at>? ORDER BY issued_at DESC",
(policy_digest, now),
).fetchall()
if len(rows) > 1:
connection.rollback()
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
if rows:
try:
response = json.loads(rows[0][0])
except (TypeError, json.JSONDecodeError) as exc:
connection.rollback()
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_LEDGER_INVALID") from exc
if not isinstance(response, dict):
connection.rollback()
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
if type(rows[0][1]) is not int or expiry_of(response) != rows[0][1]:
connection.rollback()
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
validate_current(response)
connection.rollback()
return response, True
response = issuer()
expires_at = expiry_of(response)
if expires_at <= now:
connection.rollback()
_fail("OWNER_MERGE_ISSUER_RESPONSE_EXPIRED")
artifact = response.get("artifact")
if not isinstance(artifact, Mapping):
connection.rollback()
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
payload = artifact.get("payload")
if not isinstance(payload, Mapping) or not isinstance(payload.get("nonce"), str):
connection.rollback()
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
encoded = json.dumps(
response,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
allow_nan=False,
)
connection.execute(
"INSERT INTO owner_merge_issuance(nonce,policy_digest,response_json,issued_at,expires_at) "
"VALUES(?,?,?,?,?)",
(payload["nonce"], policy_digest, encoded, now, expires_at),
)
connection.commit()
return response, False
except OwnerMergeApprovalError:
raise
except sqlite3.IntegrityError as exc:
try:
connection.rollback()
except sqlite3.Error:
pass
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_NONCE_CONFLICT") from exc
except sqlite3.Error as exc:
try:
connection.rollback()
except sqlite3.Error:
pass
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_LEDGER_UNAVAILABLE") from exc
finally:
connection.close()
class OwnerMergeApprovalService:
"""A formal structured adapter, not an arbitrary signing oracle."""
def __init__(
self,
*,
policy_path: Path,
registry_path: Path,
owner_key_path: Path,
issuance_ledger_path: Path,
allowed_uids: tuple[int, ...],
require_root_owner: bool = True,
clock: Callable[[], float] = time.time,
token_urlsafe: Callable[[int], str] | None = None,
) -> None:
if not allowed_uids or any(type(uid) is not int or uid < 0 for uid in allowed_uids):
_fail("OWNER_MERGE_ISSUER_ALLOWED_UIDS_INVALID")
self.policy_path = Path(policy_path)
self.registry_path = Path(registry_path)
self.owner_key_path = Path(owner_key_path)
self.allowed_uids = tuple(allowed_uids)
self.require_root_owner = require_root_owner
self.clock = clock
self.token_urlsafe = token_urlsafe
self.ledger = SQLiteOwnerMergeIssuanceLedger(issuance_ledger_path)
def handle(self, request: Mapping[str, Any], *, peer_uid: int) -> dict[str, Any]:
if peer_uid not in self.allowed_uids:
_fail("OWNER_MERGE_ISSUER_PEER_UID_FORBIDDEN")
if not isinstance(request, Mapping) or set(request) != {"operation", "payload"}:
_fail("OWNER_MERGE_ISSUER_REQUEST_INVALID")
if request.get("operation") != ISSUE_OPERATION:
_fail("OWNER_MERGE_ISSUER_OPERATION_FORBIDDEN")
if request.get("payload") != {}:
_fail("OWNER_MERGE_ISSUER_REQUEST_FIELDS_INVALID")
policy = _read_policy(
self.policy_path,
require_root_owner=self.require_root_owner,
)
_secure_regular(
self.owner_key_path,
require_root_owner=self.require_root_owner,
)
primary = RegistryOwnerApprovalVerifier(
self.registry_path,
require_root_owner=self.require_root_owner,
)
verifier = OwnerMergeApprovalVerifier(primary=primary, clock=self.clock)
def expiry_of(response: dict[str, Any]) -> int:
if set(response) != {"status", "artifact"} or response.get("status") != "PASS":
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
artifact = response.get("artifact")
if not isinstance(artifact, Mapping):
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
payload = artifact.get("payload")
if not isinstance(payload, Mapping) or type(payload.get("expires_at")) is not int:
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
return payload["expires_at"]
def validate_current(response: dict[str, Any]) -> None:
expiry_of(response)
artifact = response["artifact"]
if not isinstance(artifact, Mapping):
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
try:
verifier.verify(artifact, policy)
except OwnerMergeApprovalError as exc:
raise OwnerMergeApprovalError("OWNER_MERGE_ISSUER_LEDGER_INVALID") from exc
def issue() -> dict[str, Any]:
kwargs: dict[str, Any] = {}
if self.token_urlsafe is not None:
kwargs["token_urlsafe"] = self.token_urlsafe
artifact = OwnerMergeApprovalAuthority(
self.owner_key_path,
policy,
require_root_owner=self.require_root_owner,
clock=self.clock,
**kwargs,
).issue()
value = _artifact_dict(artifact)
verifier.verify(value, policy)
return {"status": "PASS", "artifact": value}
response, idempotent = self.ledger.issue_once(
policy_digest=policy.digest,
now=int(self.clock()),
issuer=issue,
validate_current=validate_current,
expiry_of=expiry_of,
)
if set(response) != {"status", "artifact"} or response.get("status") != "PASS":
_fail("OWNER_MERGE_ISSUER_LEDGER_INVALID")
return {**response, "idempotent": idempotent}
@@ -1,82 +0,0 @@
"""Immutable release-lineage contracts used by activation and recovery."""
from __future__ import annotations
from dataclasses import asdict, dataclass
from typing import Any, Mapping
@dataclass(frozen=True)
class ReleaseLineage:
task_id: str
candidate_id: str
source_commit: str
version: str
required_capabilities: tuple[str, ...]
def as_dict(self) -> dict[str, Any]:
return asdict(self)
FIXED_V2R2C2 = ReleaseLineage(
task_id="r42-release-candidate-v2r2c2-live-20260819",
candidate_id="r42-release-candidate-v2r2c2-20260819",
source_commit="71e736b05fab13efc06af7a4fe2dc022c1cf9a85",
version="V2R2C2",
required_capabilities=("release:candidate",),
)
class ReleaseLineageError(RuntimeError):
pass
def validate_fixed_v2r2c2_task(task: Mapping[str, Any]) -> dict[str, Any]:
"""Validate identity/source/version without mutating Task state.
This contract intentionally does not bind status/attempt/revision, because
those are lifecycle state and may change through governed retry/recovery.
"""
if str(task.get("task_id") or "") != FIXED_V2R2C2.task_id:
raise ReleaseLineageError("FIXED_TASK_ID_DRIFT")
capabilities = task.get("required_capabilities")
if (
not isinstance(capabilities, list)
or tuple(capabilities) != FIXED_V2R2C2.required_capabilities
):
raise ReleaseLineageError("FIXED_TASK_CAPABILITY_DRIFT")
constraints = task.get("constraints")
if not isinstance(constraints, Mapping):
raise ReleaseLineageError("FIXED_TASK_CONSTRAINTS_MISSING")
release = constraints.get("release_candidate")
if not isinstance(release, Mapping):
raise ReleaseLineageError("FIXED_RELEASE_BINDING_MISSING")
actual = {
"candidate_id": str(release.get("candidate_id") or ""),
"source_commit": str(release.get("source_commit") or ""),
"version": str(release.get("version") or ""),
}
expected = {
"candidate_id": FIXED_V2R2C2.candidate_id,
"source_commit": FIXED_V2R2C2.source_commit,
"version": FIXED_V2R2C2.version,
}
if actual != expected:
raise ReleaseLineageError(
"FIXED_RELEASE_LINEAGE_DRIFT:"
+ repr({"expected": expected, "actual": actual})
)
return {
"schema": "aios.release-lineage.validation.v1",
"status": "VERIFIED",
"lineage": FIXED_V2R2C2.as_dict(),
"task_status": task.get("status"),
"attempt": task.get("attempt"),
"max_attempts": task.get("max_attempts"),
"revision": task.get("revision"),
}
@@ -1,714 +0,0 @@
#!/usr/bin/env python3
"""Persistent release-preparation Online Agent worker.
The worker is intentionally narrower than the generic engineering worker:
it can only consume tasks whose required capability set is exactly
{"release:candidate"} and whose payload asks for immutable release-candidate
creation. It never receives Owner signing, deployment, tag, baseline or Git
write credentials.
"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import socket
import threading
import time
import urllib.error
import urllib.request
from typing import Any, Mapping
from aios_autonomous_control_plane import (
classify_failure,
repair_for_failure,
runtime_admission,
)
AGENT_ID = os.environ.get(
"AIOS_RELEASE_WORKER_AGENT_ID",
"online-release-worker-cn-network-mkt-20260819",
).strip()
AGENT_VERSION = os.environ.get(
"AIOS_RELEASE_WORKER_AGENT_VERSION",
"2026-08-19.1-release-candidate",
).strip()
TASK_API = os.environ.get(
"AIOS_AGENT_TASK_API_URL",
"http://127.0.0.1:18940",
).rstrip("/")
RELEASE_API = os.environ.get("AIOS_RELEASE_API_URL", "").rstrip("/")
POLL_SECONDS = int(os.environ.get("AIOS_RELEASE_WORKER_POLL_SECONDS", "3"))
LEASE_SECONDS = int(os.environ.get("AIOS_RELEASE_WORKER_LEASE_SECONDS", "60"))
HEARTBEAT_SECONDS = int(
os.environ.get("AIOS_RELEASE_WORKER_HEARTBEAT_SECONDS", "15")
)
STATE_ROOT = Path(
os.environ.get(
"AIOS_RELEASE_WORKER_STATE_ROOT",
"/var/lib/aios-online-agent-release",
)
)
EXPECTED_CAPABILITIES = frozenset({"release:candidate"})
LIVENESS_PATH = Path(
os.environ.get(
"AIOS_WORKER_LIVENESS_PATH",
str(STATE_ROOT / "worker-liveness.json"),
)
)
CANDIDATE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{7,255}$")
OID_RE = re.compile(r"^[0-9a-f]{40}$")
VERSION_RE = re.compile(r"^V[0-9]+R[0-9]+C[0-9]+$")
TASK_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{2,179}$")
SECRET_MARKERS = (
"authorization",
"token",
"secret",
"credential",
"password",
"private_key",
"signature",
)
class ReleaseWorkerError(RuntimeError):
def __init__(self, code: str, *, phase: str) -> None:
super().__init__(code)
self.code = code
self.phase = phase
def now() -> str:
return datetime.now(timezone.utc).isoformat()
def canonical(value: object) -> bytes:
return json.dumps(
value,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def sha256(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def sanitized(value: Any) -> Any:
if isinstance(value, Mapping):
result: dict[str, Any] = {}
for key, inner in value.items():
lowered = str(key).lower()
if any(marker in lowered for marker in SECRET_MARKERS):
result[str(key)] = "[REDACTED]"
else:
result[str(key)] = sanitized(inner)
return result
if isinstance(value, list):
return [sanitized(item) for item in value]
return value
def audit(event: str, **fields: Any) -> None:
STATE_ROOT.mkdir(parents=True, exist_ok=True, mode=0o750)
path = STATE_ROOT / "events.jsonl"
record = {
"schema": "aios.release-worker.event.v1",
"event": event,
"created_at": now(),
"agent_id": AGENT_ID,
**sanitized(fields),
}
with path.open("a", encoding="utf-8") as handle:
handle.write(json.dumps(record, sort_keys=True) + "\n")
os.chmod(path, 0o640)
def sd_notify(message: str) -> None:
target = os.environ.get("NOTIFY_SOCKET", "")
if not target:
return
if target.startswith("@"):
address = "\0" + target[1:]
else:
address = target
sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
try:
sock.connect(address)
sock.sendall(message.encode("utf-8"))
except OSError:
# Liveness JSON remains the diagnostic source; systemd will enforce
# WatchdogSec if notify transport is persistently broken.
pass
finally:
sock.close()
def write_liveness(
state: str,
*,
task_id: str | None = None,
stage: str | None = None,
error: str | None = None,
visible_count: int | None = None,
) -> None:
STATE_ROOT.mkdir(parents=True, exist_ok=True, mode=0o750)
value = {
"schema": "aios.online-agent.worker-liveness.v1",
"agent_id": AGENT_ID,
"agent_type": "release-preparation-worker",
"agent_version": AGENT_VERSION,
"pid": os.getpid(),
"state": state,
"last_seen_epoch": time.time(),
"last_seen_at": now(),
"poll_seconds": POLL_SECONDS,
"heartbeat_seconds": HEARTBEAT_SECONDS,
"api_base": TASK_API,
"capabilities": sorted(EXPECTED_CAPABILITIES),
"task_id": task_id,
"stage": stage,
"error": error,
"visible_count": visible_count,
}
LIVENESS_PATH.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
temporary = LIVENESS_PATH.with_name(
"." + LIVENESS_PATH.name + f".{os.getpid()}.tmp"
)
temporary.write_text(
json.dumps(value, ensure_ascii=False, sort_keys=True) + "\n",
encoding="utf-8",
)
os.chmod(temporary, 0o640)
os.replace(temporary, LIVENESS_PATH)
def load_credential(name: str) -> str:
directory = Path(os.environ.get("CREDENTIALS_DIRECTORY", ""))
path = directory / name
if path.is_symlink() or not path.is_file():
raise ReleaseWorkerError(
"CREDENTIAL_UNAVAILABLE:" + name,
phase="startup",
)
value = path.read_text(encoding="utf-8").strip()
if len(value) < 24 or any(ch.isspace() for ch in value):
raise ReleaseWorkerError(
"CREDENTIAL_INVALID:" + name,
phase="startup",
)
return value
def http_json(
base: str,
method: str,
route: str,
token: str,
body: Mapping[str, Any] | None = None,
*,
timeout: int = 30,
) -> tuple[int, dict[str, Any]]:
data = None if body is None else canonical(body)
headers = {"Accept": "application/json"}
if token:
headers["Authorization"] = "Bearer " + token
if data is not None:
headers["Content-Type"] = "application/json"
request = urllib.request.Request(
base + route,
method=method,
data=data,
headers=headers,
)
try:
with urllib.request.urlopen(request, timeout=timeout) as response:
raw, status = response.read(), response.status
except urllib.error.HTTPError as exc:
raw, status = exc.read(), exc.code
except (urllib.error.URLError, TimeoutError) as exc:
raise ReleaseWorkerError(
"HTTP_TRANSPORT_UNAVAILABLE",
phase="network",
) from exc
try:
value = json.loads(raw.decode())
except Exception as exc:
raise ReleaseWorkerError(
"HTTP_RESPONSE_INVALID",
phase="network",
) from exc
return status, value if isinstance(value, dict) else {}
@dataclass(frozen=True)
class ReleaseRequest:
candidate_id: str
source_commit: str
version: str
projection_id: str | None
@classmethod
def from_task(cls, task: Mapping[str, Any]) -> "ReleaseRequest":
if not TASK_RE.fullmatch(str(task.get("task_id") or "")):
raise ReleaseWorkerError("TASK_ID_INVALID", phase="validation")
caps = task.get("required_capabilities")
if (
not isinstance(caps, list)
or frozenset(caps) != EXPECTED_CAPABILITIES
or len(caps) != 1
):
raise ReleaseWorkerError(
"TASK_CAPABILITY_SCOPE_INVALID",
phase="validation",
)
metadata = task.get("metadata")
if not isinstance(metadata, Mapping):
metadata = {}
constraints = task.get("constraints")
if not isinstance(constraints, Mapping):
constraints = {}
release = constraints.get("release_candidate")
if not isinstance(release, Mapping):
release = metadata.get("release_candidate")
if not isinstance(release, Mapping):
release = task.get("release_candidate")
if not isinstance(release, Mapping):
raise ReleaseWorkerError(
"RELEASE_CANDIDATE_PAYLOAD_MISSING",
phase="validation",
)
allowed = {
"candidate_id",
"source_commit",
"version",
"projection_id",
}
if set(release) - allowed:
raise ReleaseWorkerError(
"RELEASE_CANDIDATE_PAYLOAD_FIELDS_INVALID",
phase="validation",
)
candidate_id = str(release.get("candidate_id") or "")
source_commit = str(release.get("source_commit") or "")
version = str(release.get("version") or "")
projection_id = release.get("projection_id")
if CANDIDATE_RE.fullmatch(candidate_id) is None:
raise ReleaseWorkerError(
"RELEASE_CANDIDATE_ID_INVALID",
phase="validation",
)
if OID_RE.fullmatch(source_commit) is None:
raise ReleaseWorkerError(
"RELEASE_SOURCE_COMMIT_INVALID",
phase="validation",
)
if VERSION_RE.fullmatch(version) is None:
raise ReleaseWorkerError(
"RELEASE_VERSION_INVALID",
phase="validation",
)
if projection_id is not None:
projection_id = str(projection_id)
if not re.fullmatch(r"projection-[0-9a-f]{32}", projection_id):
raise ReleaseWorkerError(
"RELEASE_PROJECTION_ID_INVALID",
phase="validation",
)
return cls(
candidate_id=candidate_id,
source_commit=source_commit,
version=version,
projection_id=projection_id,
)
def body(self) -> dict[str, str]:
return {
"candidate_id": self.candidate_id,
"source_commit": self.source_commit,
"version": self.version,
}
class Heartbeat:
def __init__(
self,
agent_token: str,
task_id: str,
lease_token: str,
) -> None:
self.agent_token = agent_token
self.task_id = task_id
self.lease_token = lease_token
self.stage = "claimed"
self.failure: str | None = None
self.stop = threading.Event()
self.thread = threading.Thread(
target=self._run,
name="release-worker-heartbeat",
daemon=True,
)
def first(self) -> None:
status, value = http_json(
TASK_API,
"POST",
f"/v1/tasks/{self.task_id}/heartbeat",
self.agent_token,
{
"lease_token": self.lease_token,
"lease_seconds": LEASE_SECONDS,
"progress": {
"stage": self.stage,
"worker_version": AGENT_VERSION,
},
},
)
if status != 200:
raise ReleaseWorkerError(
"TASK_HEARTBEAT_FAILED",
phase="heartbeat",
)
audit(
"TASK_HEARTBEAT",
task_id=self.task_id,
stage=self.stage,
api_status=status,
)
write_liveness(
"BUSY",
task_id=self.task_id,
stage=self.stage,
)
sd_notify("WATCHDOG=1")
def _run(self) -> None:
while not self.stop.wait(HEARTBEAT_SECONDS):
try:
status, value = http_json(
TASK_API,
"POST",
f"/v1/tasks/{self.task_id}/heartbeat",
self.agent_token,
{
"lease_token": self.lease_token,
"lease_seconds": LEASE_SECONDS,
"progress": {
"stage": self.stage,
"worker_version": AGENT_VERSION,
},
},
)
if status != 200:
self.failure = str(value.get("error") or status)
return
audit(
"TASK_HEARTBEAT",
task_id=self.task_id,
stage=self.stage,
api_status=status,
)
write_liveness(
"BUSY",
task_id=self.task_id,
stage=self.stage,
)
sd_notify("WATCHDOG=1")
except Exception as exc:
self.failure = type(exc).__name__
return
def ensure(self) -> None:
if self.failure:
raise ReleaseWorkerError(
"TASK_LEASE_LOST",
phase="heartbeat",
)
def close(self) -> None:
self.stop.set()
if self.thread.ident is not None:
self.thread.join(timeout=2)
def release_admission(
task: Mapping[str, Any],
*,
release_token: str,
):
required = task.get("required_capabilities")
if not isinstance(required, list):
required = []
def release_api_health() -> bool:
status, value = http_json(
RELEASE_API,
"GET",
"/health",
"",
timeout=10,
)
return status == 200 and str(value.get("status") or "").lower() in {"ok", "healthy"}
def authenticated_probe() -> bool:
status, _ = http_json(
RELEASE_API,
"POST",
"/createReleaseCandidate",
release_token,
{},
timeout=10,
)
# Authentication + credential projection succeeded; the empty release
# request is intentionally rejected by request validation.
return status == 400
probes = {
"credential_projection": authenticated_probe,
"trusted_builder": release_api_health,
"authenticated_probe": authenticated_probe,
"release_api_health": release_api_health,
"worker_liveness": lambda: True,
}
return runtime_admission(
worker_capabilities=EXPECTED_CAPABILITIES,
required_capabilities=required,
requested_operations=("prepare_release",),
probes=probes,
)
def result_payload(
task_id: str,
release: ReleaseRequest,
builder: Mapping[str, Any],
) -> dict[str, Any]:
builder_candidate = builder.get("candidate_id")
if builder_candidate not in {None, release.candidate_id}:
raise ReleaseWorkerError(
"BUILDER_CANDIDATE_BINDING_INVALID",
phase="release",
)
immutable = {
"candidate_id": release.candidate_id,
"source_commit": release.source_commit,
"version": release.version,
}
return {
"status": "SUCCEEDED",
"classification": "RELEASE_CANDIDATE_CREATED",
"execution_mode": "release_candidate",
"adapter": "release_candidate",
"task_id": task_id,
"agent_id": AGENT_ID,
"release_candidate": immutable,
"release_candidate_request_sha256": sha256(canonical(immutable)),
"builder_response": sanitized(dict(builder)),
"projection_id": release.projection_id,
"writer_lease": "NONE",
"changed_files": [],
"direct_main_write": False,
"production_write": False,
"tag_write": False,
"baseline_write": False,
"runtime_switch": False,
"owner_approval_submitted": False,
"credential_material_exposed": False,
}
def execute_task(
task: Mapping[str, Any],
*,
agent_token: str,
release_token: str,
) -> bool:
task_id = str(task.get("task_id") or "")
release = ReleaseRequest.from_task(task)
admission = release_admission(
task,
release_token=release_token,
)
if not admission.admitted:
audit(
"TASK_ADMISSION_BLOCKED",
task_id=task_id,
admission=admission.as_dict(),
)
write_liveness(
"BLOCKED",
task_id=task_id,
stage="execution_ready",
error=admission.failure_class.value if admission.failure_class else "ADMISSION_FAILED",
)
return False
audit(
"TASK_ADMISSION_PASSED",
task_id=task_id,
admission=admission.as_dict(),
)
status, claimed = http_json(
TASK_API,
"POST",
f"/v1/tasks/{task_id}/claim",
agent_token,
{"lease_seconds": LEASE_SECONDS},
)
if status in {409, 423}:
return False
if status != 200:
raise ReleaseWorkerError(
"TASK_CLAIM_FAILED",
phase="claim",
)
lease_token = str(claimed.get("lease_token") or "")
if len(lease_token) < 16:
raise ReleaseWorkerError(
"TASK_LEASE_TOKEN_INVALID",
phase="claim",
)
attempt = int(claimed.get("attempt") or 0)
audit(
"TASK_CLAIMED",
task_id=task_id,
attempt=attempt,
)
heartbeat = Heartbeat(agent_token, task_id, lease_token)
heartbeat.first()
heartbeat.thread.start()
try:
heartbeat.stage = "release_candidate"
heartbeat.ensure()
status, builder = http_json(
RELEASE_API,
"POST",
"/createReleaseCandidate",
release_token,
release.body(),
timeout=120,
)
if status not in {200, 201, 202}:
raise ReleaseWorkerError(
"RELEASE_CANDIDATE_CREATE_FAILED:"
+ str(builder.get("error") or status),
phase="release",
)
heartbeat.ensure()
result = result_payload(task_id, release, builder)
idempotency_key = (
f"{task_id}-attempt-{attempt}-release-result-v1"
)
status, submitted = http_json(
TASK_API,
"POST",
f"/v1/tasks/{task_id}/result",
agent_token,
{
"lease_token": lease_token,
"idempotency_key": idempotency_key,
"result": result,
},
)
if status not in {200, 201, 202}:
raise ReleaseWorkerError(
"TASK_RESULT_SUBMIT_FAILED",
phase="result",
)
audit(
"RESULT_SUBMITTED",
task_id=task_id,
attempt=attempt,
release_candidate_id=release.candidate_id,
)
return True
finally:
heartbeat.close()
def main() -> int:
if not RELEASE_API:
raise SystemExit("FATAL:AIOS_RELEASE_API_URL_REQUIRED")
agent_token = load_credential("agent_token")
release_token = load_credential("release_candidate_token")
STATE_ROOT.mkdir(parents=True, exist_ok=True, mode=0o750)
audit(
"WORKER_STARTED",
capabilities=sorted(EXPECTED_CAPABILITIES),
release_api=RELEASE_API,
)
write_liveness("STARTING", stage="startup")
sd_notify("READY=1\nSTATUS=release worker ready")
while True:
try:
status, response = http_json(
TASK_API,
"POST",
"/v1/tasks/poll",
agent_token,
{"limit": 100},
)
if status != 200:
raise ReleaseWorkerError(
"TASK_POLL_FAILED",
phase="poll",
)
tasks = response.get("tasks")
if not isinstance(tasks, list):
tasks = []
write_liveness(
"IDLE",
stage="poll",
visible_count=len(tasks),
)
sd_notify("WATCHDOG=1\nSTATUS=release worker poll ok")
handled = False
for raw in tasks:
if not isinstance(raw, Mapping):
continue
caps = raw.get("required_capabilities")
if (
not isinstance(caps, list)
or frozenset(caps) != EXPECTED_CAPABILITIES
):
continue
try:
handled = execute_task(
raw,
agent_token=agent_token,
release_token=release_token,
) or handled
except ReleaseWorkerError as exc:
failure = classify_failure(exc.code, phase=exc.phase)
repair = repair_for_failure(failure)
audit(
"TASK_EXECUTION_FAILED",
task_id=raw.get("task_id"),
phase=exc.phase,
error=exc.code,
failure_class=failure.value,
repair=repair.as_dict(),
)
time.sleep(0 if handled else POLL_SECONDS)
except KeyboardInterrupt:
return 0
except Exception as exc:
audit(
"WORKER_LOOP_ERROR",
error=type(exc).__name__,
)
write_liveness(
"DEGRADED",
stage="loop",
error=type(exc).__name__,
)
time.sleep(POLL_SECONDS)
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,967 @@
"""Bounded, read-only discovery over a verified immutable Git repository.
The module is deliberately independent from Task, lease, credential, worktree,
and write code. Every public operation verifies the exact repository identity,
resolves a floating ref once to a full commit OID, and then reads only that
immutable snapshot.
"""
from __future__ import annotations
import base64
from collections.abc import Iterable, Iterator, Mapping
import fnmatch
import hashlib
import hmac
import json
import os
from pathlib import Path
import re
import subprocess
import time
from authoritative_git import AuthoritativeGitError, verify_repository
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_REF_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,239}$")
_IDENTITY_RE = re.compile(
r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}/[A-Za-z0-9][A-Za-z0-9._-]{0,127}$"
)
_READ_ONLY_COMMANDS = frozenset({"cat-file", "diff", "ls-tree", "rev-parse"})
_MAX_COMMAND_OUTPUT = 1_048_576
_MAX_RECORD_BYTES = 1_048_576
_MAX_LIMIT = 1_000
_MAX_DEPTH = 32
_MAX_BYTES = 1_048_576
_MAX_GLOBS = 16
_MAX_QUERY_LENGTH = 512
_MAX_CURSOR_LENGTH = 4_096
_GIT_TIMEOUT_SECONDS = 8.0
class RepositoryDiscoveryError(RuntimeError):
"""A fail-closed repository discovery request error."""
def __init__(self, code: str, *, status_code: int | None = None) -> None:
self.code = str(code)
self.status_code = (
status_code if status_code is not None else self._status_for(self.code)
)
super().__init__(self.code)
@staticmethod
def _status_for(code: str) -> int:
if "OUTPUT_LIMIT" in code:
return 413
if code.endswith("_NOT_FOUND"):
return 404
if code in {"REPOSITORY_NOT_ALLOWED", "REPOSITORY_IDENTITY_MISMATCH"}:
return 403
if code in {
"REPOSITORY_DISCOVERY_UNAVAILABLE",
"REPOSITORY_DISCOVERY_GIT_FAILED",
"REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID",
"REPOSITORY_DISCOVERY_CURSOR_SECRET_REQUIRED",
}:
return 503
return 400
class RepositoryDiscoveryNotFound(RepositoryDiscoveryError):
"""A validated immutable object or path was not found."""
class RepositoryDiscoveryOutputTooLarge(RepositoryDiscoveryError):
"""A fixed read-only Git query exceeded its collection boundary."""
def _error(code: str) -> RepositoryDiscoveryError:
return RepositoryDiscoveryError(code)
def _request_mapping(request: Mapping[str, object]) -> Mapping[str, object]:
if not isinstance(request, Mapping):
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return request
def _required_request_value(request: Mapping[str, object], field: str) -> object:
value = request.get(field)
if value is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return value
def _canonical(value: Mapping[str, object]) -> bytes:
return json.dumps(
value, sort_keys=True, separators=(",", ":"), ensure_ascii=True
).encode("ascii")
def _positive_int(value: object, field: str, maximum: int) -> int:
if isinstance(value, bool) or not isinstance(value, int) or not 1 <= value <= maximum:
raise _error(f"REPOSITORY_DISCOVERY_INVALID_{field.upper()}")
return value
def _safe_relative_path(
value: object,
field: str,
*,
allow_empty: bool,
allow_glob: bool = False,
) -> str:
if not isinstance(value, str) or value != value.strip() or "\x00" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if "\\" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if value in ("", "."):
if allow_empty:
return ""
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if value.startswith(("/", "-")) or "//" in value or ":" in value:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if not allow_glob and any(character in value for character in "*?["):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
parts = value.split("/")
if any(part in ("", ".", "..") for part in parts):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
if len(value) > 512:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
return value
def _safe_glob(value: object) -> str:
return _safe_relative_path(value, "path_glob", allow_empty=False, allow_glob=True)
def _safe_ref(value: str) -> bool:
return bool(
_REF_RE.fullmatch(value)
and ".." not in value
and "//" not in value
and "@{" not in value
and not value.endswith(("/", ".", ".lock"))
and not any(part in ("", ".", "..") for part in value.split("/"))
)
def _parse_tree_record(record: bytes) -> dict[str, str] | None:
try:
header, raw_path = record.split(b"\t", 1)
mode, kind, oid = header.decode("ascii").split(" ", 2)
path = raw_path.decode("utf-8")
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if kind not in {"blob", "tree", "commit"} or not _OID_RE.fullmatch(oid.lower()):
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
try:
_safe_relative_path(path, "tree_path", allow_empty=False)
except RepositoryDiscoveryError:
# Git permits filenames outside the public POSIX-relative contract.
# They are not exposed as usable discovery paths.
return None
return {"mode": mode, "type": kind, "object_oid": oid.lower(), "path": path}
def _decode_output_path(raw: bytes) -> str:
try:
value = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
return _safe_relative_path(value, "git_output_path", allow_empty=False)
class RepositoryDiscovery:
"""Repository-scoped immutable Git discovery with signed cursors."""
def __init__(
self,
*,
allowed_repositories: Iterable[str] | None = None,
git_dir: str | Path | None = None,
cursor_secret: str | bytes | None = None,
) -> None:
configured = allowed_repositories
if configured is None:
configured = (os.getenv("AIOS_GITOPS_REPOSITORY_IDENTITY", ""),)
identities = tuple(configured)
if not identities or any(
not isinstance(item, str) or not _IDENTITY_RE.fullmatch(item)
for item in identities
):
raise _error("REPOSITORY_DISCOVERY_ALLOWLIST_INVALID")
self._allowed_repositories = frozenset(identities)
self._git_dir = git_dir
secret = (
cursor_secret
if cursor_secret is not None
else os.getenv("AIOS_REPOSITORY_DISCOVERY_CURSOR_SECRET", "")
)
if isinstance(secret, str):
secret = secret.encode("utf-8")
if not isinstance(secret, bytes) or len(secret) < 16:
raise _error("REPOSITORY_DISCOVERY_CURSOR_SECRET_REQUIRED")
self._cursor_secret = secret
def _repository_path(self, repository: object) -> Path:
if not isinstance(repository, str) or repository not in self._allowed_repositories:
raise _error("REPOSITORY_NOT_ALLOWED")
try:
path, actual_identity = verify_repository(
git_dir=self._git_dir,
repository_identity=repository,
)
except AuthoritativeGitError as exc:
code = (
"REPOSITORY_IDENTITY_MISMATCH"
if "REPOSITORY_IDENTITY_MISMATCH" in str(exc)
else "REPOSITORY_DISCOVERY_UNAVAILABLE"
)
raise _error(code) from exc
if actual_identity != repository:
raise _error("REPOSITORY_IDENTITY_MISMATCH")
return path
@staticmethod
def _git_env() -> dict[str, str]:
return {
**os.environ,
"GIT_OPTIONAL_LOCKS": "0",
"GIT_CONFIG_NOSYSTEM": "1",
"GIT_TERMINAL_PROMPT": "0",
}
@staticmethod
def _run(
path: Path,
args: list[str],
*,
max_output: int = _MAX_COMMAND_OUTPUT,
) -> bytes:
if not args or args[0] not in _READ_ONLY_COMMANDS:
raise _error("REPOSITORY_DISCOVERY_GIT_COMMAND_REJECTED")
output_limit = min(max_output, _MAX_COMMAND_OUTPUT)
process: subprocess.Popen[bytes] | None = None
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=RepositoryDiscovery._git_env(),
)
assert process.stdout is not None
output = process.stdout.read(output_limit + 1)
if len(output) > output_limit:
process.kill()
process.wait(timeout=1)
raise RepositoryDiscoveryOutputTooLarge(
"REPOSITORY_DISCOVERY_OUTPUT_LIMIT"
)
returncode = process.wait(timeout=_GIT_TIMEOUT_SECONDS)
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
if process is not None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
if returncode != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
return output
@staticmethod
def _read_blob_prefix(path: Path, oid: str, maximum: int) -> tuple[bytes, bool]:
"""Read at most maximum bytes plus one sentinel byte."""
process: subprocess.Popen[bytes] | None = None
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), "cat-file", "blob", oid],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=RepositoryDiscovery._git_env(),
)
assert process.stdout is not None
content = process.stdout.read(maximum + 1)
truncated = len(content) > maximum
if truncated:
process.kill()
process.wait(timeout=1)
return content[:maximum], True
if process.wait(timeout=_GIT_TIMEOUT_SECONDS) != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
return content, False
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
if process is not None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
def _iter_nul_records(self, path: Path, args: list[str]) -> Iterator[bytes]:
if not args or args[0] not in _READ_ONLY_COMMANDS:
raise _error("REPOSITORY_DISCOVERY_GIT_COMMAND_REJECTED")
process: subprocess.Popen[bytes] | None = None
buffer = b""
started = time.monotonic()
try:
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
shell=False,
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
env=self._git_env(),
)
assert process.stdout is not None
while True:
if time.monotonic() - started > _GIT_TIMEOUT_SECONDS:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
chunk = process.stdout.read(64 * 1024)
if not chunk:
break
buffer += chunk
if len(buffer) > _MAX_RECORD_BYTES and b"\0" not in buffer:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
while b"\0" in buffer:
record, buffer = buffer.split(b"\0", 1)
if len(record) > _MAX_RECORD_BYTES:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
yield record
if buffer:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
if process.wait(
timeout=max(0.1, _GIT_TIMEOUT_SECONDS - (time.monotonic() - started))
) != 0:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
except RepositoryDiscoveryError:
raise
except (OSError, subprocess.TimeoutExpired) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED") from exc
finally:
if process is not None and process.poll() is None:
try:
process.kill()
process.wait(timeout=1)
except (OSError, subprocess.TimeoutExpired):
pass
def _resolve_at(self, path: Path, revision: object, field: str) -> str:
if (
not isinstance(revision, str)
or revision != revision.strip()
or not revision
or revision.startswith("-")
):
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
if _OID_RE.fullmatch(revision.lower()):
candidates = [revision.lower()]
elif revision == "HEAD":
candidates = ["HEAD"]
elif _safe_ref(revision) and not any(
token in revision for token in ("^", "~", ":", "\\")
):
if revision.startswith("refs/"):
if not revision.startswith(("refs/heads/", "refs/tags/")):
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
candidates = [revision]
else:
candidates = [f"refs/heads/{revision}", f"refs/tags/{revision}"]
else:
raise _error("REPOSITORY_DISCOVERY_INVALID_REVISION")
resolved: list[str] = []
for candidate in candidates:
try:
output = self._run(
path,
["rev-parse", "--verify", f"{candidate}^{{commit}}"],
max_output=256,
)
except RepositoryDiscoveryError:
continue
oid = output.decode("ascii", errors="ignore").strip().lower()
if _OID_RE.fullmatch(oid):
resolved.append(oid)
if not resolved:
raise RepositoryDiscoveryNotFound(
f"REPOSITORY_DISCOVERY_{field.upper()}_NOT_FOUND"
)
if len(set(resolved)) != 1:
raise _error("REPOSITORY_DISCOVERY_AMBIGUOUS_REVISION")
return resolved[0]
def resolve_revision(self, repository: str, revision: str) -> str:
return self._resolve_at(self._repository_path(repository), revision, "revision")
def _cursor_offset(
self,
cursor: str | None,
*,
operation: str,
repository: str,
resolved_oid: str,
query: Mapping[str, object],
) -> int:
if cursor is None:
return 0
if (
not isinstance(cursor, str)
or not cursor
or len(cursor) > _MAX_CURSOR_LENGTH
or "." not in cursor
):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
encoded, supplied_signature = cursor.split(".", 1)
try:
padded = encoded + "=" * (-len(encoded) % 4)
payload = json.loads(
base64.urlsafe_b64decode(padded.encode("ascii")).decode("ascii")
)
except (UnicodeError, ValueError, json.JSONDecodeError) as exc:
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID") from exc
expected_signature = hmac.new(
self._cursor_secret, encoded.encode("ascii"), hashlib.sha256
).hexdigest()
if not hmac.compare_digest(supplied_signature, expected_signature):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
expected = {
"operation": operation,
"repository": repository,
"resolved_oid": resolved_oid,
"query": dict(query),
}
if (
not isinstance(payload, dict)
or payload.get("v") != 1
or any(payload.get(key) != value for key, value in expected.items())
):
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
offset = payload.get("offset")
if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0:
raise _error("REPOSITORY_DISCOVERY_CURSOR_INVALID")
return offset
def _next_cursor(
self,
*,
operation: str,
repository: str,
resolved_oid: str,
query: Mapping[str, object],
offset: int,
) -> str:
payload = {
"v": 1,
"operation": operation,
"repository": repository,
"resolved_oid": resolved_oid,
"query": dict(query),
"offset": offset,
}
encoded = base64.urlsafe_b64encode(_canonical(payload)).decode("ascii").rstrip("=")
signature = hmac.new(
self._cursor_secret, encoded.encode("ascii"), hashlib.sha256
).hexdigest()
return f"{encoded}.{signature}"
def _iter_tree_entries(
self,
path: Path,
oid: str,
prefix: str,
*,
include_trees: bool,
) -> Iterator[dict[str, str]]:
args = ["ls-tree", "-z", "-r", "--full-tree"]
if include_trees:
args.append("-t")
args.append(oid)
if prefix:
args.extend(["--", prefix])
for record in self._iter_nul_records(path, args):
entry = _parse_tree_record(record)
if entry is None:
continue
if not prefix or entry["path"] == prefix or entry["path"].startswith(prefix + "/"):
yield entry
def list_tree(
self,
*,
repository: str,
revision: str,
prefix: str = "",
depth: int = 8,
limit: int = 100,
cursor: str | None = None,
) -> dict[str, object]:
prefix = _safe_relative_path(prefix, "prefix", allow_empty=True)
depth = _positive_int(depth, "depth", _MAX_DEPTH)
limit = _positive_int(limit, "limit", _MAX_LIMIT)
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
query = {"prefix": prefix, "depth": depth, "limit": limit}
offset = self._cursor_offset(
cursor,
operation="list_tree",
repository=repository,
resolved_oid=resolved_oid,
query=query,
)
page: list[dict[str, str]] = []
matched = 0
for entry in self._iter_tree_entries(path, resolved_oid, prefix, include_trees=True):
relative = entry["path"][len(prefix):].lstrip("/") if prefix else entry["path"]
if relative.count("/") + 1 > depth:
continue
if matched >= offset and len(page) < limit:
page.append(entry)
matched += 1
has_more = offset + len(page) < matched
return {
"repository": repository,
"resolved_oid": resolved_oid,
"entries": page,
"truncated": has_more,
"next_cursor": (
self._next_cursor(
operation="list_tree",
repository=repository,
resolved_oid=resolved_oid,
query=query,
offset=offset + len(page),
)
if has_more
else None
),
}
def list_repository_tree(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
return self.list_tree(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
prefix=request.get("prefix", ""), # type: ignore[arg-type]
depth=request.get("depth", 8), # type: ignore[arg-type]
limit=request.get("limit", 100), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def _blob_size(self, path: Path, oid: str) -> int:
output = self._run(path, ["cat-file", "-s", oid], max_output=128)
try:
size = int(output.decode("ascii").strip())
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if size < 0:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
return size
def search_source(
self,
*,
repository: str,
revision: str,
query: str,
path_globs: Iterable[str] | None = None,
max_matches: int = 100,
max_bytes: int = 262_144,
cursor: str | None = None,
) -> dict[str, object]:
if (
not isinstance(query, str)
or not query
or "\x00" in query
or len(query) > _MAX_QUERY_LENGTH
):
raise _error("REPOSITORY_DISCOVERY_QUERY_INVALID")
max_matches = _positive_int(max_matches, "max_matches", _MAX_LIMIT)
max_bytes = _positive_int(max_bytes, "max_bytes", _MAX_BYTES)
if path_globs is None:
globs = ("*",)
else:
if isinstance(path_globs, (str, bytes)) or not isinstance(path_globs, Iterable):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
globs = tuple(_safe_glob(item) for item in path_globs)
if not globs or len(globs) > _MAX_GLOBS:
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
query_binding = {
"query": query,
"path_globs": list(globs),
"max_matches": max_matches,
"max_bytes": max_bytes,
}
offset = self._cursor_offset(
cursor,
operation="search_source",
repository=repository,
resolved_oid=resolved_oid,
query=query_binding,
)
scanned_bytes = 0
skipped_for_bound = False
binary_blobs = 0
matched = 0
matches: list[dict[str, object]] = []
has_more = False
for entry in self._iter_tree_entries(path, resolved_oid, "", include_trees=False):
if entry["type"] != "blob" or not any(
fnmatch.fnmatchcase(entry["path"], pattern) for pattern in globs
):
continue
blob_size = self._blob_size(path, entry["object_oid"])
remaining = max_bytes - scanned_bytes
if blob_size > remaining:
skipped_for_bound = True
continue
raw, clipped = self._read_blob_prefix(path, entry["object_oid"], blob_size)
if clipped:
raise _error("REPOSITORY_DISCOVERY_GIT_FAILED")
scanned_bytes += len(raw)
if b"\0" in raw:
binary_blobs += 1
continue
text = raw.decode("utf-8", errors="replace")
for number, line in enumerate(text.splitlines(), start=1):
if query not in line:
continue
if matched < offset:
matched += 1
continue
if len(matches) >= max_matches:
has_more = True
break
matches.append(
{
"path": entry["path"],
"line_start": number,
"line_end": number,
"blob_oid": entry["object_oid"],
"snippet": line[:512],
}
)
matched += 1
if has_more:
break
return {
"repository": repository,
"resolved_oid": resolved_oid,
"matches": matches,
"scanned_bytes": scanned_bytes,
"max_bytes": max_bytes,
"binary_blobs_skipped": binary_blobs,
"truncated": bool(has_more or skipped_for_bound),
"next_cursor": (
self._next_cursor(
operation="search_source",
repository=repository,
resolved_oid=resolved_oid,
query=query_binding,
offset=offset + len(matches),
)
if has_more
else None
),
}
def search_repository_source(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
path_globs = request.get("path_globs")
if path_globs is not None and (
isinstance(path_globs, (str, bytes)) or not isinstance(path_globs, Iterable)
):
raise _error("REPOSITORY_DISCOVERY_PATH_INVALID")
return self.search_source(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
query=_required_request_value(request, "query"), # type: ignore[arg-type]
path_globs=path_globs, # type: ignore[arg-type]
max_matches=request.get("max_matches", 100), # type: ignore[arg-type]
max_bytes=request.get("max_bytes", 262_144), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def _iter_diff_name_entries(
self,
path: Path,
from_oid: str,
to_oid: str,
prefix: str,
) -> Iterator[dict[str, object]]:
suffix = ["--", prefix] if prefix else []
records = self._iter_nul_records(
path,
[
"diff",
"--no-ext-diff",
"--no-textconv",
"--find-renames=50%",
"--name-status",
"-z",
from_oid,
to_oid,
*suffix,
],
)
iterator = iter(records)
while True:
try:
status_raw = next(iterator)
except StopIteration:
return
try:
status = status_raw.decode("ascii")
except UnicodeDecodeError as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if not status or status[0] not in {"A", "C", "D", "M", "R", "T", "U", "X"}:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
try:
if status[0] in {"R", "C"}:
old_path = _decode_output_path(next(iterator))
new_path = _decode_output_path(next(iterator))
yield {
"status": status,
"path": new_path,
"old_path": old_path,
}
else:
yield {
"status": status,
"path": _decode_output_path(next(iterator)),
"old_path": None,
}
except StopIteration as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
@staticmethod
def _parse_numstat_count(value: bytes) -> int | None:
if value == b"-":
return None
try:
parsed = int(value.decode("ascii"), 10)
except (UnicodeDecodeError, ValueError) as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if parsed < 0:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
return parsed
def _diff_stats(
self,
path: Path,
from_oid: str,
to_oid: str,
prefix: str,
wanted_paths: set[str],
) -> dict[str, tuple[int | None, int | None]]:
suffix = ["--", prefix] if prefix else []
records = self._iter_nul_records(
path,
[
"diff",
"--no-ext-diff",
"--no-textconv",
"--find-renames=50%",
"--numstat",
"-z",
from_oid,
to_oid,
*suffix,
],
)
iterator = iter(records)
stats: dict[str, tuple[int | None, int | None]] = {}
for row in iterator:
if not row:
continue
fields = row.split(b"\t", 2)
if len(fields) != 3:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID")
additions = self._parse_numstat_count(fields[0])
deletions = self._parse_numstat_count(fields[1])
if fields[2]:
stat_path = _decode_output_path(fields[2])
else:
try:
next(iterator) # old path
stat_path = _decode_output_path(next(iterator))
except StopIteration as exc:
raise _error("REPOSITORY_DISCOVERY_GIT_OUTPUT_INVALID") from exc
if stat_path in wanted_paths:
stats[stat_path] = (additions, deletions)
if wanted_paths <= set(stats):
break
return stats
def diff_index(
self,
*,
repository: str,
from_revision: str,
to_revision: str,
prefix: str = "",
limit: int = 100,
cursor: str | None = None,
) -> dict[str, object]:
prefix = _safe_relative_path(prefix, "prefix", allow_empty=True)
limit = _positive_int(limit, "limit", _MAX_LIMIT)
path = self._repository_path(repository)
from_oid = self._resolve_at(path, from_revision, "from")
to_oid = self._resolve_at(path, to_revision, "to")
query = {
"from_oid": from_oid,
"to_oid": to_oid,
"prefix": prefix,
"limit": limit,
}
offset = self._cursor_offset(
cursor,
operation="diff_index",
repository=repository,
resolved_oid=to_oid,
query=query,
)
page: list[dict[str, object]] = []
total = 0
for entry in self._iter_diff_name_entries(path, from_oid, to_oid, prefix):
if total >= offset and len(page) < limit:
page.append(entry)
total += 1
wanted_paths = {str(entry["path"]) for entry in page}
stat_by_path = (
self._diff_stats(path, from_oid, to_oid, prefix, wanted_paths)
if page
else {}
)
for entry in page:
additions, deletions = stat_by_path.get(
str(entry["path"]), (None, None)
)
entry["additions"] = additions
entry["deletions"] = deletions
has_more = offset + len(page) < total
return {
"repository": repository,
"resolved_from_oid": from_oid,
"resolved_to_oid": to_oid,
"entries": page,
"truncated": has_more,
"next_cursor": (
self._next_cursor(
operation="diff_index",
repository=repository,
resolved_oid=to_oid,
query=query,
offset=offset + len(page),
)
if has_more
else None
),
}
def get_diff_index(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
from_revision = request.get("from_revision", request.get("from"))
to_revision = request.get("to_revision", request.get("to"))
prefix = request.get("prefix", request.get("path_prefix", ""))
if from_revision is None or to_revision is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return self.diff_index(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
from_revision=from_revision, # type: ignore[arg-type]
to_revision=to_revision, # type: ignore[arg-type]
prefix=prefix, # type: ignore[arg-type]
limit=request.get("limit", 100), # type: ignore[arg-type]
cursor=request.get("cursor"), # type: ignore[arg-type]
)
def read_file(
self,
*,
repository: str,
revision: str,
file_path: str,
max_bytes: int = 262_144,
) -> dict[str, object]:
file_path = _safe_relative_path(file_path, "file_path", allow_empty=False)
max_bytes = _positive_int(max_bytes, "max_bytes", _MAX_BYTES)
path = self._repository_path(repository)
resolved_oid = self._resolve_at(path, revision, "revision")
exact: list[dict[str, str]] = []
for entry in self._iter_tree_entries(
path, resolved_oid, file_path, include_trees=False
):
if entry["path"] == file_path and entry["type"] == "blob":
exact.append(entry)
if len(exact) != 1:
raise RepositoryDiscoveryNotFound("REPOSITORY_DISCOVERY_FILE_NOT_FOUND")
blob_oid = exact[0]["object_oid"]
content, truncated = self._read_blob_prefix(path, blob_oid, max_bytes)
return {
"repository": repository,
"resolved_oid": resolved_oid,
"blob_oid": blob_oid,
"path": file_path,
"content": content.decode("utf-8", errors="replace"),
"content_base64": base64.b64encode(content).decode("ascii"),
"content_bytes": len(content),
"truncated": truncated,
}
def read_repository_file(self, request: Mapping[str, object]) -> dict[str, object]:
request = _request_mapping(request)
file_path = request.get("file_path", request.get("path"))
if file_path is None:
raise _error("REPOSITORY_DISCOVERY_REQUEST_INVALID")
return self.read_file(
repository=_required_request_value(request, "repository"), # type: ignore[arg-type]
revision=request.get("revision", request.get("ref", "HEAD")), # type: ignore[arg-type]
file_path=file_path, # type: ignore[arg-type]
max_bytes=request.get("max_bytes", 262_144), # type: ignore[arg-type]
)
def create_repository_discovery(
*,
allowed_repositories: Iterable[str] | None = None,
git_dir: str | Path | None = None,
cursor_secret: str | bytes | None = None,
) -> RepositoryDiscovery:
return RepositoryDiscovery(
allowed_repositories=allowed_repositories,
git_dir=git_dir,
cursor_secret=cursor_secret,
)
def list_repository_tree(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).list_repository_tree(request)
def search_repository_source(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).search_repository_source(request)
def get_diff_index(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).get_diff_index(request)
def read_repository_file(
request: Mapping[str, object], *, discovery: RepositoryDiscovery | None = None
) -> dict[str, object]:
return (discovery or create_repository_discovery()).read_repository_file(request)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,707 @@
"""Single-host operational delegation for AIOS migration.
This module is deliberately narrow:
* Owner signs only OperationalDelegation and ExpiredTaskRecoveryDecision.
* Machine signer signs only WorkerRuntimeBinding and WorkerRuntimeProof.
* Every object is bound to one immutable Task identity and a narrow capability set.
* There is no arbitrary sign-bytes API.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import hashlib
import json
from pathlib import Path
import re
import time
from typing import Any, Mapping, Sequence
from aios_single_host_authority import (
DEFAULT_MACHINE_SIGNER_ID,
DEFAULT_OWNER_SIGNER_ID,
MACHINE_NAMESPACE,
OWNER_IDENTITY,
OWNER_NAMESPACE,
LocalFileSigner,
RegistryOwnerApprovalVerifier,
SignerRegistry,
SingleHostAuthorityError,
canonical_json_bytes,
sha256_json,
verify_sshsig,
)
OPERATIONAL_POLICY_SCHEMA = "aios.single-host-operational-policy.v1"
OPERATIONAL_DELEGATION_SCHEMA = "aios.owner-operational-delegation.v1"
RECOVERY_DECISION_SCHEMA = "aios.expired-task-recovery-decision.v1"
WORKER_BINDING_SCHEMA = "aios.worker-runtime-binding.v1"
WORKER_PROOF_SCHEMA = "aios.worker-runtime-proof.v1"
REVOCATION_SCHEMA = "aios.operational-revocations.v1"
ALLOWED_OPERATIONAL_CAPABILITIES = frozenset({
"task.recovery",
"worker.runtime.bind",
"worker.runtime.proof",
"task.claim.guard",
})
ALLOWED_EXPECTED_STATE = "EXPIRED"
HEX40 = re.compile(r"^[0-9a-f]{40}$")
HEX64 = re.compile(r"^[0-9a-f]{64}$")
SAFE_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/+:-]{2,511}$")
CAP = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:-]{1,127}$")
class OperationalAuthorityError(RuntimeError):
def __init__(self, code: str):
super().__init__(code)
self.code = code
def fail(code: str) -> None:
raise OperationalAuthorityError(code)
def _exact_fields(value: Mapping[str, Any], fields: set[str], code: str) -> None:
if not isinstance(value, Mapping) or set(value) != fields:
fail(code)
def _safe_id(value: str, code: str) -> str:
if not isinstance(value, str) or not SAFE_ID.fullmatch(value):
fail(code)
return value
def _hex(value: str, n: int, code: str) -> str:
regex = HEX40 if n == 40 else HEX64
if not isinstance(value, str) or not regex.fullmatch(value):
fail(code)
return value
def _caps(value: Sequence[str], code: str) -> tuple[str, ...]:
if not isinstance(value, (list, tuple)) or not value:
fail(code)
out = tuple(sorted(set(value)))
if len(out) != len(value) or any(not isinstance(x, str) or not CAP.fullmatch(x) for x in out):
fail(code)
return out
def task_identity_sha256(task: Mapping[str, Any]) -> str:
"""Digest only immutable recovery/claim identity fields.
Revision, state, attempt and deadline are intentionally excluded.
"""
fields = (
"task_id", "repository", "target_branch", "base_oid", "generation",
"execution_plan_sha256", "external_job_identity",
)
try:
value = {name: task[name] for name in fields}
except KeyError as exc:
raise OperationalAuthorityError("TASK_IDENTITY_FIELD_MISSING") from exc
_safe_id(str(value["task_id"]), "TASK_ID_INVALID")
_safe_id(str(value["repository"]), "REPOSITORY_INVALID")
_safe_id(str(value["target_branch"]), "TARGET_BRANCH_INVALID")
_hex(str(value["base_oid"]), 40, "BASE_OID_INVALID")
if type(value["generation"]) is not int or value["generation"] < 1:
fail("GENERATION_INVALID")
_hex(str(value["execution_plan_sha256"]), 64, "EXECUTION_PLAN_DIGEST_INVALID")
_safe_id(str(value["external_job_identity"]), "EXTERNAL_JOB_IDENTITY_INVALID")
return hashlib.sha256(canonical_json_bytes(value)).hexdigest()
@dataclass(frozen=True, slots=True)
class OperationalPolicy:
repository: str
task_id: str
task_identity_sha256: str
target_branch: str
base_oid: str
generation: int
execution_plan_sha256: str
external_job_identity: str
allowed_capabilities: tuple[str, ...]
allowed_worker_ids: tuple[str, ...]
maximum_delegation_ttl_seconds: int
maximum_recovery_decision_ttl_seconds: int
maximum_binding_ttl_seconds: int
maximum_proof_ttl_seconds: int
maximum_heartbeat_age_seconds: int
maximum_task_deadline_extension_seconds: int
schema: str = OPERATIONAL_POLICY_SCHEMA
def validate(self) -> None:
if self.schema != OPERATIONAL_POLICY_SCHEMA:
fail("OPERATIONAL_POLICY_SCHEMA_INVALID")
_safe_id(self.repository, "OPERATIONAL_POLICY_REPOSITORY_INVALID")
_safe_id(self.task_id, "OPERATIONAL_POLICY_TASK_ID_INVALID")
_hex(self.task_identity_sha256, 64, "OPERATIONAL_POLICY_TASK_IDENTITY_INVALID")
_safe_id(self.target_branch, "OPERATIONAL_POLICY_BRANCH_INVALID")
_hex(self.base_oid, 40, "OPERATIONAL_POLICY_BASE_OID_INVALID")
if type(self.generation) is not int or self.generation < 1:
fail("OPERATIONAL_POLICY_GENERATION_INVALID")
_hex(self.execution_plan_sha256, 64, "OPERATIONAL_POLICY_PLAN_INVALID")
_safe_id(self.external_job_identity, "OPERATIONAL_POLICY_JOB_INVALID")
caps = _caps(self.allowed_capabilities, "OPERATIONAL_POLICY_CAPABILITIES_INVALID")
if not set(caps).issubset(ALLOWED_OPERATIONAL_CAPABILITIES):
fail("OPERATIONAL_POLICY_CAPABILITY_WIDENING")
if not self.allowed_worker_ids or any(not SAFE_ID.fullmatch(x) for x in self.allowed_worker_ids):
fail("OPERATIONAL_POLICY_WORKERS_INVALID")
bounds = (
(self.maximum_delegation_ttl_seconds, 300, 7 * 24 * 3600),
(self.maximum_recovery_decision_ttl_seconds, 30, 900),
(self.maximum_binding_ttl_seconds, 60, 24 * 3600),
(self.maximum_proof_ttl_seconds, 15, 900),
(self.maximum_heartbeat_age_seconds, 5, 300),
(self.maximum_task_deadline_extension_seconds, 300, 24 * 3600),
)
if any(type(v) is not int or not lo <= v <= hi for v, lo, hi in bounds):
fail("OPERATIONAL_POLICY_TTL_INVALID")
@property
def digest(self) -> str:
self.validate()
return sha256_json(asdict(self))
@classmethod
def from_mapping(cls, value: Mapping[str, Any]) -> "OperationalPolicy":
fields = set(cls.__dataclass_fields__)
_exact_fields(value, fields, "OPERATIONAL_POLICY_FIELDS_INVALID")
try:
policy = cls(
repository=value["repository"],
task_id=value["task_id"],
task_identity_sha256=value["task_identity_sha256"],
target_branch=value["target_branch"],
base_oid=value["base_oid"],
generation=value["generation"],
execution_plan_sha256=value["execution_plan_sha256"],
external_job_identity=value["external_job_identity"],
allowed_capabilities=tuple(value["allowed_capabilities"]),
allowed_worker_ids=tuple(value["allowed_worker_ids"]),
maximum_delegation_ttl_seconds=value["maximum_delegation_ttl_seconds"],
maximum_recovery_decision_ttl_seconds=value["maximum_recovery_decision_ttl_seconds"],
maximum_binding_ttl_seconds=value["maximum_binding_ttl_seconds"],
maximum_proof_ttl_seconds=value["maximum_proof_ttl_seconds"],
maximum_heartbeat_age_seconds=value["maximum_heartbeat_age_seconds"],
maximum_task_deadline_extension_seconds=value["maximum_task_deadline_extension_seconds"],
schema=value["schema"],
)
except Exception as exc:
raise OperationalAuthorityError("OPERATIONAL_POLICY_FIELDS_INVALID") from exc
policy.validate()
return policy
def build_exact_policy(
task: Mapping[str, Any],
*,
worker_id: str,
allowed_capabilities: Sequence[str] = tuple(sorted(ALLOWED_OPERATIONAL_CAPABILITIES)),
) -> OperationalPolicy:
identity = task_identity_sha256(task)
policy = OperationalPolicy(
repository=str(task["repository"]),
task_id=str(task["task_id"]),
task_identity_sha256=identity,
target_branch=str(task["target_branch"]),
base_oid=str(task["base_oid"]),
generation=int(task["generation"]),
execution_plan_sha256=str(task["execution_plan_sha256"]),
external_job_identity=str(task["external_job_identity"]),
allowed_capabilities=tuple(sorted(allowed_capabilities)),
allowed_worker_ids=(worker_id,),
maximum_delegation_ttl_seconds=24 * 3600,
maximum_recovery_decision_ttl_seconds=300,
maximum_binding_ttl_seconds=4 * 3600,
maximum_proof_ttl_seconds=300,
maximum_heartbeat_age_seconds=30,
maximum_task_deadline_extension_seconds=4 * 3600,
)
policy.validate()
return policy
def _signed_payload_dict(payload: Mapping[str, Any], *, key_id: str, namespace: str) -> dict[str, Any]:
value = dict(payload)
value["signature_namespace"] = namespace
value["signature_key_id"] = key_id
value["canonical_payload_digest"] = sha256_json(value)
return value
def _verify_digest_field(payload: Mapping[str, Any]) -> bool:
if not isinstance(payload, Mapping) or "canonical_payload_digest" not in payload:
return False
value = dict(payload)
supplied = value.pop("canonical_payload_digest", None)
return isinstance(supplied, str) and supplied == sha256_json(value)
@dataclass(frozen=True, slots=True)
class SignedOperationalArtifact:
payload: dict[str, Any]
signature: str
key_id: str
namespace: str
@property
def canonical_payload(self) -> bytes:
return canonical_json_bytes(self.payload)
@property
def payload_sha256(self) -> str:
return hashlib.sha256(self.canonical_payload).hexdigest()
def to_dict(self) -> dict[str, Any]:
return {
"payload": dict(self.payload),
"signature": self.signature,
"key_id": self.key_id,
"namespace": self.namespace,
}
class OperationalOwnerAuthority:
def __init__(self, key_path: Path, policy: OperationalPolicy, *, require_root_owner: bool = True, clock: Any = time.time):
policy.validate()
self.policy = policy
self.signer = LocalFileSigner(
Path(key_path),
signer_id=DEFAULT_OWNER_SIGNER_ID,
namespace=OWNER_NAMESPACE,
require_root_owner=require_root_owner,
)
self.clock = clock
@property
def fingerprint(self) -> str:
return self.signer.fingerprint
def _sign(self, payload: Mapping[str, Any]) -> SignedOperationalArtifact:
full = _signed_payload_dict(payload, key_id=self.fingerprint, namespace=OWNER_NAMESPACE)
art = self.signer._sign_canonical(full)
return SignedOperationalArtifact(dict(art.payload), art.signature, art.key_id, art.namespace)
def issue_operational_delegation(self, *, delegation_id: str, nonce: str, ttl_seconds: int) -> SignedOperationalArtifact:
now = int(self.clock())
p = self.policy
if not SAFE_ID.fullmatch(delegation_id) or not SAFE_ID.fullmatch(nonce):
fail("OPERATIONAL_DELEGATION_ID_INVALID")
if type(ttl_seconds) is not int or not 60 <= ttl_seconds <= p.maximum_delegation_ttl_seconds:
fail("OPERATIONAL_DELEGATION_TTL_INVALID")
payload = {
"schema": OPERATIONAL_DELEGATION_SCHEMA,
"delegation_id": delegation_id,
"nonce": nonce,
"owner_identity": OWNER_IDENTITY,
"repository": p.repository,
"task_id": p.task_id,
"task_identity_sha256": p.task_identity_sha256,
"target_branch": p.target_branch,
"base_oid": p.base_oid,
"generation": p.generation,
"execution_plan_sha256": p.execution_plan_sha256,
"external_job_identity": p.external_job_identity,
"allowed_capabilities": list(p.allowed_capabilities),
"allowed_worker_ids": list(p.allowed_worker_ids),
"policy_digest": p.digest,
"issued_at": now,
"not_before": max(0, now - 5),
"expires_at": now + ttl_seconds,
"revocation_epoch": 0,
"status": "ACTIVE",
}
return self._sign(payload)
def issue_recovery_decision(
self,
delegation: Mapping[str, Any],
*,
decision_id: str,
nonce: str,
expected_revision: int,
expected_attempt: int,
original_deadline: int,
new_deadline: int,
ttl_seconds: int,
) -> SignedOperationalArtifact:
now = int(self.clock())
verify_delegation_scope(delegation, self.policy, now=now)
if "task.recovery" not in delegation["allowed_capabilities"]:
fail("RECOVERY_CAPABILITY_NOT_DELEGATED")
if not SAFE_ID.fullmatch(decision_id) or not SAFE_ID.fullmatch(nonce):
fail("RECOVERY_DECISION_ID_INVALID")
if type(expected_revision) is not int or expected_revision < 0 or type(expected_attempt) is not int or expected_attempt < 0:
fail("RECOVERY_CAS_INVALID")
if type(original_deadline) is not int or type(new_deadline) is not int:
fail("RECOVERY_DEADLINE_INVALID")
if new_deadline <= now or new_deadline <= original_deadline:
fail("RECOVERY_DEADLINE_INVALID")
if new_deadline - now > self.policy.maximum_task_deadline_extension_seconds:
fail("RECOVERY_DEADLINE_WIDENING")
if type(ttl_seconds) is not int or not 15 <= ttl_seconds <= self.policy.maximum_recovery_decision_ttl_seconds:
fail("RECOVERY_DECISION_TTL_INVALID")
payload = {
"schema": RECOVERY_DECISION_SCHEMA,
"decision_id": decision_id,
"nonce": nonce,
"parent_delegation_id": delegation["delegation_id"],
"parent_delegation_digest": hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest(),
"task_id": self.policy.task_id,
"task_identity_sha256": self.policy.task_identity_sha256,
"expected_state": ALLOWED_EXPECTED_STATE,
"expected_revision": expected_revision,
"expected_attempt": expected_attempt,
"original_deadline": original_deadline,
"new_deadline": new_deadline,
"repository": self.policy.repository,
"target_branch": self.policy.target_branch,
"base_oid": self.policy.base_oid,
"generation": self.policy.generation,
"execution_plan_sha256": self.policy.execution_plan_sha256,
"external_job_identity": self.policy.external_job_identity,
"policy_digest": self.policy.digest,
"one_time_use": True,
"issued_at": now,
"not_before": max(0, now - 5),
"expires_at": now + ttl_seconds,
}
return self._sign(payload)
class OperationalMachineAuthority:
def __init__(self, key_path: Path, policy: OperationalPolicy, *, require_root_owner: bool = True, clock: Any = time.time):
policy.validate()
self.policy = policy
self.signer = LocalFileSigner(
Path(key_path),
signer_id=DEFAULT_MACHINE_SIGNER_ID,
namespace=MACHINE_NAMESPACE,
require_root_owner=require_root_owner,
)
self.clock = clock
@property
def fingerprint(self) -> str:
return self.signer.fingerprint
def _sign(self, payload: Mapping[str, Any]) -> SignedOperationalArtifact:
full = _signed_payload_dict(payload, key_id=self.fingerprint, namespace=MACHINE_NAMESPACE)
art = self.signer._sign_canonical(full)
return SignedOperationalArtifact(dict(art.payload), art.signature, art.key_id, art.namespace)
def issue_runtime_binding(
self,
delegation: Mapping[str, Any],
*,
binding_id: str,
nonce: str,
worker_id: str,
runtime_id: str,
runtime_sha256: str,
task_contract_version: str,
supported_contract_versions: Sequence[str],
required_capabilities: Sequence[str],
worker_capabilities: Sequence[str],
max_concurrency: int,
ttl_seconds: int,
) -> SignedOperationalArtifact:
now = int(self.clock())
p = self.policy
verify_delegation_scope(delegation, p, now=now)
if "worker.runtime.bind" not in delegation["allowed_capabilities"]:
fail("RUNTIME_BIND_CAPABILITY_NOT_DELEGATED")
if worker_id not in p.allowed_worker_ids:
fail("WORKER_ID_NOT_DELEGATED")
for val, code in [(binding_id, "BINDING_ID_INVALID"), (nonce, "BINDING_NONCE_INVALID"), (worker_id, "WORKER_ID_INVALID"), (runtime_id, "RUNTIME_ID_INVALID"), (task_contract_version, "CONTRACT_VERSION_INVALID")]:
_safe_id(val, code)
_hex(runtime_sha256, 64, "RUNTIME_SHA256_INVALID")
supported = tuple(sorted(set(supported_contract_versions)))
if task_contract_version not in supported or not supported or any(not SAFE_ID.fullmatch(x) for x in supported):
fail("CONTRACT_INCOMPATIBLE")
required = _caps(required_capabilities, "REQUIRED_CAPABILITIES_INVALID")
worker_caps = _caps(worker_capabilities, "WORKER_CAPABILITIES_INVALID")
if not set(required).issubset(set(worker_caps)):
fail("CAPABILITY_MISMATCH")
if type(max_concurrency) is not int or not 1 <= max_concurrency <= 64:
fail("MAX_CONCURRENCY_INVALID")
if type(ttl_seconds) is not int or not 30 <= ttl_seconds <= p.maximum_binding_ttl_seconds:
fail("BINDING_TTL_INVALID")
if now + ttl_seconds > int(delegation["expires_at"]):
fail("BINDING_EXCEEDS_PARENT")
payload = {
"schema": WORKER_BINDING_SCHEMA,
"binding_id": binding_id,
"nonce": nonce,
"parent_delegation_id": delegation["delegation_id"],
"parent_delegation_digest": hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest(),
"repository": p.repository,
"task_id": p.task_id,
"task_identity_sha256": p.task_identity_sha256,
"target_branch": p.target_branch,
"base_oid": p.base_oid,
"generation": p.generation,
"execution_plan_sha256": p.execution_plan_sha256,
"external_job_identity": p.external_job_identity,
"worker_id": worker_id,
"runtime_id": runtime_id,
"runtime_sha256": runtime_sha256,
"task_contract_version": task_contract_version,
"supported_contract_versions": list(supported),
"required_capabilities": list(required),
"worker_capabilities": list(worker_caps),
"max_concurrency": max_concurrency,
"policy_digest": p.digest,
"issued_at": now,
"not_before": max(0, now - 5),
"expires_at": now + ttl_seconds,
}
return self._sign(payload)
def issue_runtime_proof(
self,
delegation: Mapping[str, Any],
binding: Mapping[str, Any],
*,
proof_id: str,
nonce: str,
heartbeat_at: int,
adapter_readiness: bool,
resource_readiness: bool,
active_claim_count: int,
ttl_seconds: int,
) -> SignedOperationalArtifact:
now = int(self.clock())
p = self.policy
verify_delegation_scope(delegation, p, now=now)
verify_binding_scope(binding, delegation, p, now=now)
if "worker.runtime.proof" not in delegation["allowed_capabilities"]:
fail("RUNTIME_PROOF_CAPABILITY_NOT_DELEGATED")
_safe_id(proof_id, "PROOF_ID_INVALID")
_safe_id(nonce, "PROOF_NONCE_INVALID")
if type(heartbeat_at) is not int or heartbeat_at > now + 5 or now - heartbeat_at > p.maximum_heartbeat_age_seconds:
fail("HEARTBEAT_STALE")
if adapter_readiness is not True or resource_readiness is not True:
fail("RUNTIME_NOT_READY")
if type(active_claim_count) is not int or active_claim_count < 0 or active_claim_count >= int(binding["max_concurrency"]):
fail("MAX_CONCURRENCY_REACHED")
if type(ttl_seconds) is not int or not 10 <= ttl_seconds <= p.maximum_proof_ttl_seconds:
fail("PROOF_TTL_INVALID")
if now + ttl_seconds > min(int(binding["expires_at"]), int(delegation["expires_at"])):
fail("PROOF_EXCEEDS_PARENT")
payload = {
"schema": WORKER_PROOF_SCHEMA,
"proof_id": proof_id,
"nonce": nonce,
"parent_delegation_id": delegation["delegation_id"],
"parent_delegation_digest": hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest(),
"binding_id": binding["binding_id"],
"binding_digest": hashlib.sha256(canonical_json_bytes(dict(binding))).hexdigest(),
"repository": p.repository,
"task_id": p.task_id,
"task_identity_sha256": p.task_identity_sha256,
"generation": p.generation,
"execution_plan_sha256": p.execution_plan_sha256,
"worker_id": binding["worker_id"],
"runtime_id": binding["runtime_id"],
"runtime_sha256": binding["runtime_sha256"],
"heartbeat_at": heartbeat_at,
"adapter_readiness": True,
"resource_readiness": True,
"active_claim_count": active_claim_count,
"max_concurrency": binding["max_concurrency"],
"policy_digest": p.digest,
"issued_at": now,
"expires_at": now + ttl_seconds,
}
return self._sign(payload)
DELEGATION_FIELDS = {
"schema","delegation_id","nonce","owner_identity","repository","task_id",
"task_identity_sha256","target_branch","base_oid","generation","execution_plan_sha256",
"external_job_identity","allowed_capabilities","allowed_worker_ids","policy_digest",
"issued_at","not_before","expires_at","revocation_epoch","status",
"signature_namespace","signature_key_id","canonical_payload_digest",
}
BINDING_FIELDS = {
"schema","binding_id","nonce","parent_delegation_id","parent_delegation_digest",
"repository","task_id","task_identity_sha256","target_branch","base_oid","generation",
"execution_plan_sha256","external_job_identity","worker_id","runtime_id","runtime_sha256",
"task_contract_version","supported_contract_versions","required_capabilities","worker_capabilities",
"max_concurrency","policy_digest","issued_at","not_before","expires_at",
"signature_namespace","signature_key_id","canonical_payload_digest",
}
PROOF_FIELDS = {
"schema","proof_id","nonce","parent_delegation_id","parent_delegation_digest",
"binding_id","binding_digest","repository","task_id","task_identity_sha256","generation",
"execution_plan_sha256","worker_id","runtime_id","runtime_sha256","heartbeat_at",
"adapter_readiness","resource_readiness","active_claim_count","max_concurrency",
"policy_digest","issued_at","expires_at","signature_namespace","signature_key_id",
"canonical_payload_digest",
}
RECOVERY_FIELDS = {
"schema","decision_id","nonce","parent_delegation_id","parent_delegation_digest",
"task_id","task_identity_sha256","expected_state","expected_revision","expected_attempt",
"original_deadline","new_deadline","repository","target_branch","base_oid","generation",
"execution_plan_sha256","external_job_identity","policy_digest","one_time_use",
"issued_at","not_before","expires_at","signature_namespace","signature_key_id",
"canonical_payload_digest",
}
def verify_delegation_scope(value: Mapping[str, Any], policy: OperationalPolicy, *, now: int | None = None) -> None:
policy.validate()
_exact_fields(value, DELEGATION_FIELDS, "OPERATIONAL_DELEGATION_FIELDS_INVALID")
now = int(time.time()) if now is None else int(now)
if (
value["schema"] != OPERATIONAL_DELEGATION_SCHEMA
or value["status"] != "ACTIVE"
or value["revocation_epoch"] != 0
or value["owner_identity"] != OWNER_IDENTITY
or value["repository"] != policy.repository
or value["task_id"] != policy.task_id
or value["task_identity_sha256"] != policy.task_identity_sha256
or value["target_branch"] != policy.target_branch
or value["base_oid"] != policy.base_oid
or value["generation"] != policy.generation
or value["execution_plan_sha256"] != policy.execution_plan_sha256
or value["external_job_identity"] != policy.external_job_identity
or value["policy_digest"] != policy.digest
or not set(value["allowed_capabilities"]).issubset(set(policy.allowed_capabilities))
or not set(value["allowed_worker_ids"]).issubset(set(policy.allowed_worker_ids))
or value["signature_namespace"] != OWNER_NAMESPACE
or not _verify_digest_field(value)
or not int(value["not_before"]) <= now < int(value["expires_at"])
):
fail("OPERATIONAL_DELEGATION_SCOPE_INVALID")
def verify_binding_scope(binding: Mapping[str, Any], delegation: Mapping[str, Any], policy: OperationalPolicy, *, now: int | None = None) -> None:
_exact_fields(binding, BINDING_FIELDS, "WORKER_BINDING_FIELDS_INVALID")
now = int(time.time()) if now is None else int(now)
expected_parent = hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest()
if (
binding["schema"] != WORKER_BINDING_SCHEMA
or binding["parent_delegation_id"] != delegation["delegation_id"]
or binding["parent_delegation_digest"] != expected_parent
or binding["repository"] != policy.repository
or binding["task_id"] != policy.task_id
or binding["task_identity_sha256"] != policy.task_identity_sha256
or binding["target_branch"] != policy.target_branch
or binding["base_oid"] != policy.base_oid
or binding["generation"] != policy.generation
or binding["execution_plan_sha256"] != policy.execution_plan_sha256
or binding["external_job_identity"] != policy.external_job_identity
or binding["worker_id"] not in policy.allowed_worker_ids
or binding["policy_digest"] != policy.digest
or binding["signature_namespace"] != MACHINE_NAMESPACE
or not _verify_digest_field(binding)
or not int(binding["not_before"]) <= now < int(binding["expires_at"])
):
fail("WORKER_BINDING_SCOPE_INVALID")
required = set(binding["required_capabilities"])
worker = set(binding["worker_capabilities"])
if not required.issubset(worker) or binding["task_contract_version"] not in binding["supported_contract_versions"]:
fail("WORKER_BINDING_COMPATIBILITY_INVALID")
def verify_proof_scope(proof: Mapping[str, Any], binding: Mapping[str, Any], delegation: Mapping[str, Any], policy: OperationalPolicy, *, now: int | None = None) -> None:
_exact_fields(proof, PROOF_FIELDS, "WORKER_PROOF_FIELDS_INVALID")
now = int(time.time()) if now is None else int(now)
if (
proof["schema"] != WORKER_PROOF_SCHEMA
or proof["parent_delegation_id"] != delegation["delegation_id"]
or proof["parent_delegation_digest"] != hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest()
or proof["binding_id"] != binding["binding_id"]
or proof["binding_digest"] != hashlib.sha256(canonical_json_bytes(dict(binding))).hexdigest()
or proof["repository"] != policy.repository
or proof["task_id"] != policy.task_id
or proof["task_identity_sha256"] != policy.task_identity_sha256
or proof["generation"] != policy.generation
or proof["execution_plan_sha256"] != policy.execution_plan_sha256
or proof["worker_id"] != binding["worker_id"]
or proof["runtime_id"] != binding["runtime_id"]
or proof["runtime_sha256"] != binding["runtime_sha256"]
or proof["max_concurrency"] != binding["max_concurrency"]
or proof["policy_digest"] != policy.digest
or proof["signature_namespace"] != MACHINE_NAMESPACE
or proof["adapter_readiness"] is not True
or proof["resource_readiness"] is not True
or not _verify_digest_field(proof)
or not now < int(proof["expires_at"])
or now - int(proof["heartbeat_at"]) > policy.maximum_heartbeat_age_seconds
or int(proof["active_claim_count"]) >= int(proof["max_concurrency"])
):
fail("WORKER_PROOF_SCOPE_INVALID")
def verify_recovery_scope(decision: Mapping[str, Any], delegation: Mapping[str, Any], policy: OperationalPolicy, *, now: int | None = None) -> None:
_exact_fields(decision, RECOVERY_FIELDS, "RECOVERY_DECISION_FIELDS_INVALID")
now = int(time.time()) if now is None else int(now)
if (
decision["schema"] != RECOVERY_DECISION_SCHEMA
or decision["parent_delegation_id"] != delegation["delegation_id"]
or decision["parent_delegation_digest"] != hashlib.sha256(canonical_json_bytes(dict(delegation))).hexdigest()
or decision["task_id"] != policy.task_id
or decision["task_identity_sha256"] != policy.task_identity_sha256
or decision["expected_state"] != "EXPIRED"
or decision["repository"] != policy.repository
or decision["target_branch"] != policy.target_branch
or decision["base_oid"] != policy.base_oid
or decision["generation"] != policy.generation
or decision["execution_plan_sha256"] != policy.execution_plan_sha256
or decision["external_job_identity"] != policy.external_job_identity
or decision["policy_digest"] != policy.digest
or decision["one_time_use"] is not True
or decision["signature_namespace"] != OWNER_NAMESPACE
or not _verify_digest_field(decision)
or not int(decision["not_before"]) <= now < int(decision["expires_at"])
or int(decision["new_deadline"]) <= now
):
fail("RECOVERY_DECISION_SCOPE_INVALID")
class OperationalVerifier:
"""Verify signatures through the public root-owned SignerRegistry."""
def __init__(self, registry_path: Path, *, require_root_owner: bool = True):
self.registry_path = Path(registry_path)
self.require_root_owner = require_root_owner
def _verify(self, artifact: SignedOperationalArtifact | Mapping[str, Any], *, role: str, signer_id: str, namespace: str) -> dict[str, Any]:
if isinstance(artifact, SignedOperationalArtifact):
payload, signature, key_id, artifact_namespace = artifact.payload, artifact.signature, artifact.key_id, artifact.namespace
else:
_exact_fields(artifact, {"payload","signature","key_id","namespace"}, "SIGNED_ARTIFACT_FIELDS_INVALID")
payload, signature, key_id, artifact_namespace = artifact["payload"], artifact["signature"], artifact["key_id"], artifact["namespace"]
if artifact_namespace != namespace:
fail("SIGNED_ARTIFACT_NAMESPACE_INVALID")
registry = SignerRegistry(self.registry_path, require_root_owner=self.require_root_owner)
rec = registry.require(signer_id, role=role)
if rec.fingerprint != key_id:
fail("SIGNED_ARTIFACT_KEY_ID_INVALID")
if role == "MACHINE_OPERATIONAL":
owner_rec = registry.require(DEFAULT_OWNER_SIGNER_ID, role="OWNER_AUTHORITY")
if rec.parent_signer_id != owner_rec.signer_id or rec.parent_fingerprint != owner_rec.fingerprint:
fail("MACHINE_PARENT_BINDING_INVALID")
# RegistryOwnerApprovalVerifier handles Owner namespace; generic machine uses registry public key.
if role == "OWNER_AUTHORITY":
verifier = RegistryOwnerApprovalVerifier(self.registry_path, require_root_owner=self.require_root_owner, signer_id=signer_id)
ok = verifier.verify(namespace=namespace, payload=canonical_json_bytes(payload), signature=signature, key_id=key_id)
else:
from aios_single_host_authority import _load_public_key_line
pub = _load_public_key_line(rec.public_key)
ok = verify_sshsig(pub, canonical_json_bytes(payload), signature, namespace=namespace, expected_key_id=key_id)
if not ok:
fail("SIGNED_ARTIFACT_SIGNATURE_INVALID")
return dict(payload)
def owner(self, artifact: SignedOperationalArtifact | Mapping[str, Any]) -> dict[str, Any]:
return self._verify(artifact, role="OWNER_AUTHORITY", signer_id=DEFAULT_OWNER_SIGNER_ID, namespace=OWNER_NAMESPACE)
def machine(self, artifact: SignedOperationalArtifact | Mapping[str, Any]) -> dict[str, Any]:
return self._verify(artifact, role="MACHINE_OPERATIONAL", signer_id=DEFAULT_MACHINE_SIGNER_ID, namespace=MACHINE_NAMESPACE)
@@ -0,0 +1,410 @@
"""Unix-domain fixed-schema service for operational migration authority."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
from pathlib import Path
import socket
import sqlite3
import struct
import stat
import sys
import time
from typing import Any, Mapping
from aios_single_host_authority import (
DEFAULT_REGISTRY_PATH,
OWNER_NAMESPACE,
MACHINE_NAMESPACE,
SingleHostAuthorityError,
)
from aios_single_host_operational_authority import (
OperationalAuthorityError,
OperationalMachineAuthority,
OperationalOwnerAuthority,
OperationalPolicy,
OperationalVerifier,
SignedOperationalArtifact,
verify_binding_scope,
verify_delegation_scope,
)
DEFAULT_POLICY = Path("/etc/aios/single-host-operational-policy.json")
DEFAULT_OWNER_KEY = Path("/var/lib/aios-owner-authority/owner-authority.key")
DEFAULT_MACHINE_KEY = Path("/var/lib/aios-machine-signer/machine-operational.key")
DEFAULT_OWNER_SOCKET = Path("/run/aios/owner-authority.sock")
DEFAULT_MACHINE_SOCKET = Path("/run/aios/machine-signer.sock")
DEFAULT_LEDGER = Path("/var/lib/aios-operational-authority/issuance-ledger.sqlite3")
MAX_REQUEST = 128 * 1024
READINESS_SCHEMA = "aios.signer-readiness-challenge.v1"
class OperationalServiceError(RuntimeError):
def __init__(self, code: str):
super().__init__(code)
self.code = code
def fail(code: str) -> None:
raise OperationalServiceError(code)
def _read_policy(path: Path, *, require_root_owner: bool = True) -> OperationalPolicy:
path=Path(path)
if not path.is_absolute():
fail("OPERATIONAL_POLICY_PATH_INVALID")
flags=os.O_RDONLY
if hasattr(os,"O_NOFOLLOW"):
flags |= os.O_NOFOLLOW
try:
fd=os.open(path,flags)
except OSError as exc:
raise OperationalServiceError("OPERATIONAL_POLICY_UNAVAILABLE") from exc
try:
st=os.fstat(fd)
if not stat.S_ISREG(st.st_mode):
fail("OPERATIONAL_POLICY_NOT_REGULAR")
mode=stat.S_IMODE(st.st_mode)
if mode & 0o022:
fail("OPERATIONAL_POLICY_WRITABLE_BY_GROUP_OR_OTHER")
if require_root_owner and st.st_uid != 0:
fail("OPERATIONAL_POLICY_NOT_ROOT_OWNED")
raw=bytearray()
while len(raw) <= MAX_REQUEST:
chunk=os.read(fd,min(65536,MAX_REQUEST+1-len(raw)))
if not chunk: break
raw.extend(chunk)
if not raw or len(raw)>MAX_REQUEST:
fail("OPERATIONAL_POLICY_UNAVAILABLE")
finally:
os.close(fd)
try:
obj=json.loads(bytes(raw).decode("utf-8"))
except Exception as exc:
raise OperationalServiceError("OPERATIONAL_POLICY_INVALID") from exc
return OperationalPolicy.from_mapping(obj)
def _artifact(value: Mapping[str, Any]) -> SignedOperationalArtifact:
if not isinstance(value, Mapping) or set(value) != {"payload","signature","key_id","namespace"}:
fail("SIGNED_ARTIFACT_INVALID")
if not isinstance(value["payload"], Mapping):
fail("SIGNED_ARTIFACT_INVALID")
return SignedOperationalArtifact(
payload=dict(value["payload"]),
signature=str(value["signature"]),
key_id=str(value["key_id"]),
namespace=str(value["namespace"]),
)
def _peer_uid(conn: socket.socket) -> int:
if not hasattr(socket, "SO_PEERCRED"):
fail("PEER_CREDENTIALS_UNAVAILABLE")
pid, uid, gid = struct.unpack("3i", conn.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED, struct.calcsize("3i")))
return uid
class DurableIssuanceLedger:
"""Root-owned exact-idempotent structured signing ledger."""
def __init__(self, path: Path):
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
if os.name == "posix": os.chmod(self.path.parent, 0o700)
if self.path.exists() and self.path.is_symlink():
fail("ISSUANCE_LEDGER_SYMLINK_FORBIDDEN")
conn = sqlite3.connect(self.path)
try:
conn.execute("PRAGMA journal_mode=DELETE")
conn.execute("""
CREATE TABLE IF NOT EXISTS issuance (
role TEXT NOT NULL,
operation TEXT NOT NULL,
nonce TEXT NOT NULL,
request_sha256 TEXT NOT NULL,
response_json TEXT NOT NULL,
created_at INTEGER NOT NULL,
PRIMARY KEY(role, operation, nonce)
)
""")
conn.commit()
finally:
conn.close()
os.chmod(self.path, 0o600)
def issue(self, *, role: str, operation: str, nonce: str, request: Mapping[str, Any], issuer: Any) -> dict[str, Any]:
request_sha = hashlib.sha256(
json.dumps(request, sort_keys=True, separators=(",",":"), ensure_ascii=False).encode("utf-8")
).hexdigest()
conn = sqlite3.connect(self.path, timeout=5, isolation_level=None)
try:
conn.execute("BEGIN IMMEDIATE")
row = conn.execute(
"SELECT request_sha256,response_json FROM issuance WHERE role=? AND operation=? AND nonce=?",
(role, operation, nonce),
).fetchone()
if row is not None:
if row[0] != request_sha:
conn.rollback()
fail("NONCE_REUSE_WITH_DIFFERENT_REQUEST")
result = json.loads(row[1])
conn.commit()
return result
result = issuer()
encoded = json.dumps(result, sort_keys=True, separators=(",",":"), ensure_ascii=False)
conn.execute(
"INSERT INTO issuance(role,operation,nonce,request_sha256,response_json,created_at) VALUES(?,?,?,?,?,?)",
(role, operation, nonce, request_sha, encoded, int(time.time())),
)
conn.commit()
return result
except sqlite3.IntegrityError:
conn.rollback()
fail("ISSUANCE_LEDGER_RACE")
finally:
conn.close()
class OperationalAuthorityService:
def __init__(
self,
*,
role: str,
policy_path: Path = DEFAULT_POLICY,
registry_path: Path = DEFAULT_REGISTRY_PATH,
owner_key_path: Path = DEFAULT_OWNER_KEY,
machine_key_path: Path = DEFAULT_MACHINE_KEY,
socket_path: Path | None = None,
ledger_path: Path = DEFAULT_LEDGER,
allowed_uids: tuple[int, ...] = (0,),
clock: Any = time.time,
require_root_owner: bool = True,
):
if role not in {"owner","machine"}:
fail("ROLE_INVALID")
self.role = role
self.policy_path = Path(policy_path)
self.registry_path = Path(registry_path)
self.owner_key_path = Path(owner_key_path)
self.machine_key_path = Path(machine_key_path)
self.socket_path = Path(socket_path or (DEFAULT_OWNER_SOCKET if role == "owner" else DEFAULT_MACHINE_SOCKET))
self.allowed_uids = tuple(allowed_uids)
self.ledger = DurableIssuanceLedger(ledger_path)
self.clock = clock
self.require_root_owner = require_root_owner
def _components(self):
policy = _read_policy(self.policy_path, require_root_owner=self.require_root_owner)
verifier = OperationalVerifier(self.registry_path, require_root_owner=self.require_root_owner)
if self.role == "owner":
signer = OperationalOwnerAuthority(self.owner_key_path, policy, require_root_owner=self.require_root_owner, clock=self.clock)
else:
signer = OperationalMachineAuthority(self.machine_key_path, policy, require_root_owner=self.require_root_owner, clock=self.clock)
return policy, verifier, signer
def handle(self, request: Mapping[str, Any], *, peer_uid: int) -> dict[str, Any]:
if peer_uid not in self.allowed_uids:
fail("PEER_UID_FORBIDDEN")
if not isinstance(request, Mapping) or set(request) != {"operation","payload"} or not isinstance(request["payload"], Mapping):
fail("REQUEST_INVALID")
op = request["operation"]
p = request["payload"]
policy, verifier, signer = self._components()
if op == "identity":
return {
"status":"PASS",
"role":self.role,
"policy_digest":policy.digest,
"key_id":signer.fingerprint,
"socket":str(self.socket_path),
"arbitrary_signing":False,
}
if op == "readiness_challenge":
if set(p) != {"challenge_id","challenge_sha256"}:
fail("REQUEST_FIELDS_INVALID")
challenge_id = str(p["challenge_id"])
challenge_sha256 = str(p["challenge_sha256"])
if not challenge_id or len(challenge_id) > 160:
fail("READINESS_CHALLENGE_ID_INVALID")
if not isinstance(challenge_sha256, str) or not re.fullmatch(r"[0-9a-f]{64}", challenge_sha256):
fail("READINESS_CHALLENGE_SHA256_INVALID")
namespace = OWNER_NAMESPACE if self.role == "owner" else MACHINE_NAMESPACE
def issue():
art = signer._sign({
"schema": READINESS_SCHEMA,
"role": self.role,
"challenge_id": challenge_id,
"challenge_sha256": challenge_sha256,
"policy_digest": policy.digest,
"issued_at": int(self.clock()),
})
return {"status":"PASS","artifact":art.to_dict()}
return self.ledger.issue(
role=self.role,
operation=op,
nonce=challenge_id,
request=p,
issuer=issue,
)
if self.role == "owner":
if op == "issue_operational_delegation":
if set(p) != {"delegation_id","nonce","ttl_seconds"}:
fail("REQUEST_FIELDS_INVALID")
def issue():
art = signer.issue_operational_delegation(
delegation_id=str(p["delegation_id"]),
nonce=str(p["nonce"]),
ttl_seconds=p["ttl_seconds"],
)
return {"status":"PASS","artifact":art.to_dict()}
return self.ledger.issue(role=self.role, operation=op, nonce=str(p["nonce"]), request=p, issuer=issue)
if op == "issue_expired_task_recovery":
if set(p) != {"delegation","decision_id","nonce","expected_revision","expected_attempt","original_deadline","new_deadline","ttl_seconds"}:
fail("REQUEST_FIELDS_INVALID")
parent_art = _artifact(p["delegation"])
parent = verifier.owner(parent_art)
verify_delegation_scope(parent, policy, now=int(self.clock()))
def issue():
art = signer.issue_recovery_decision(
parent,
decision_id=str(p["decision_id"]),
nonce=str(p["nonce"]),
expected_revision=p["expected_revision"],
expected_attempt=p["expected_attempt"],
original_deadline=p["original_deadline"],
new_deadline=p["new_deadline"],
ttl_seconds=p["ttl_seconds"],
)
return {"status":"PASS","artifact":art.to_dict()}
return self.ledger.issue(role=self.role, operation=op, nonce=str(p["nonce"]), request=p, issuer=issue)
fail("OWNER_OPERATION_FORBIDDEN")
if op == "issue_worker_runtime_binding":
if set(p) != {
"delegation","binding_id","nonce","worker_id","runtime_id","runtime_sha256",
"task_contract_version","supported_contract_versions","required_capabilities",
"worker_capabilities","max_concurrency","ttl_seconds"
}:
fail("REQUEST_FIELDS_INVALID")
parent_art = _artifact(p["delegation"])
parent = verifier.owner(parent_art)
verify_delegation_scope(parent, policy, now=int(self.clock()))
def issue():
art = signer.issue_runtime_binding(
parent,
binding_id=str(p["binding_id"]), nonce=str(p["nonce"]),
worker_id=str(p["worker_id"]), runtime_id=str(p["runtime_id"]),
runtime_sha256=str(p["runtime_sha256"]),
task_contract_version=str(p["task_contract_version"]),
supported_contract_versions=tuple(p["supported_contract_versions"]),
required_capabilities=tuple(p["required_capabilities"]),
worker_capabilities=tuple(p["worker_capabilities"]),
max_concurrency=p["max_concurrency"], ttl_seconds=p["ttl_seconds"],
)
return {"status":"PASS","artifact":art.to_dict()}
return self.ledger.issue(role=self.role, operation=op, nonce=str(p["nonce"]), request=p, issuer=issue)
if op == "issue_worker_runtime_proof":
if set(p) != {
"delegation","binding","proof_id","nonce","heartbeat_at","adapter_readiness",
"resource_readiness","active_claim_count","ttl_seconds"
}:
fail("REQUEST_FIELDS_INVALID")
parent_art = _artifact(p["delegation"])
binding_art = _artifact(p["binding"])
parent = verifier.owner(parent_art)
binding = verifier.machine(binding_art)
verify_delegation_scope(parent, policy, now=int(self.clock()))
verify_binding_scope(binding, parent, policy, now=int(self.clock()))
def issue():
art = signer.issue_runtime_proof(
parent, binding,
proof_id=str(p["proof_id"]), nonce=str(p["nonce"]),
heartbeat_at=p["heartbeat_at"],
adapter_readiness=p["adapter_readiness"],
resource_readiness=p["resource_readiness"],
active_claim_count=p["active_claim_count"],
ttl_seconds=p["ttl_seconds"],
)
return {"status":"PASS","artifact":art.to_dict()}
return self.ledger.issue(role=self.role, operation=op, nonce=str(p["nonce"]), request=p, issuer=issue)
fail("MACHINE_OPERATION_FORBIDDEN")
def serve_forever(self) -> None:
path = self.socket_path
path.parent.mkdir(parents=True, exist_ok=True)
if path.exists() or path.is_symlink():
if path.is_socket():
path.unlink()
else:
fail("SOCKET_PATH_OCCUPIED")
sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
try:
sock.bind(str(path))
os.chmod(path, 0o600)
sock.listen(16)
while True:
conn, _ = sock.accept()
with conn:
try:
uid = _peer_uid(conn)
chunks = []
total = 0
while True:
data = conn.recv(65536)
if not data:
break
total += len(data)
if total > MAX_REQUEST:
fail("REQUEST_TOO_LARGE")
chunks.append(data)
if b"\n" in data:
break
raw = b"".join(chunks)
line = raw.split(b"\n",1)[0]
req = json.loads(line.decode("utf-8"))
result = self.handle(req, peer_uid=uid)
out = {"ok":True,"result":result}
except (OperationalServiceError, OperationalAuthorityError, SingleHostAuthorityError) as exc:
out = {"ok":False,"error":getattr(exc,"code",str(exc))}
except Exception:
out = {"ok":False,"error":"INTERNAL_FAIL_CLOSED"}
conn.sendall(json.dumps(out, sort_keys=True, separators=(",",":")).encode("utf-8")+b"\n")
finally:
sock.close()
try:
path.unlink()
except FileNotFoundError:
pass
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--role", choices=("owner","machine"), required=True)
ap.add_argument("--policy", default=str(DEFAULT_POLICY))
ap.add_argument("--registry", default=str(DEFAULT_REGISTRY_PATH))
ap.add_argument("--owner-key", default=str(DEFAULT_OWNER_KEY))
ap.add_argument("--machine-key", default=str(DEFAULT_MACHINE_KEY))
ap.add_argument("--socket")
ns = ap.parse_args()
service = OperationalAuthorityService(
role=ns.role,
policy_path=Path(ns.policy),
registry_path=Path(ns.registry),
owner_key_path=Path(ns.owner_key),
machine_key_path=Path(ns.machine_key),
socket_path=Path(ns.socket) if ns.socket else None,
)
service.serve_forever()
return 0
if __name__ == "__main__":
raise SystemExit(main())
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,87 @@
"""Bridge contract surface for Task Admission Bootstrap.
This module is intentionally transport-agnostic. The live Bridge may merge
these routes/operations into its existing authenticated/HTTPS surface, but the
authorization proof for the authorize operation is the verified delegation
epoch or Owner signature carried by the request itself.
No handler here can issue TaskPrincipal assertions or WriterLeases. Those remain
strictly downstream of a real admitted/claimed Agent Task.
"""
from __future__ import annotations
from typing import Any, Mapping
from aios_task_admission_bootstrap import TaskAdmissionBootstrapService
TASK_ADMISSION_ROUTES = {
"/v2/task-admission/envelope": "getTaskAdmissionEnvelope",
"/v2/task-admission/authorize": "submitTaskAdmissionAuthorization",
"/v2/task-admission/status": "getTaskAdmissionStatus",
}
TASK_ADMISSION_OPERATION_REQUIRED_FIELDS = {
"getTaskAdmissionEnvelope": (
"idempotency_key",
"task_id",
"repository",
"authority_base_oid",
"target_branch",
"generation",
"execution_digest",
"requested_capabilities",
"file_allowlist",
"risk",
),
"submitTaskAdmissionAuthorization": (
"envelope_id",
"authorization_kind",
),
"getTaskAdmissionStatus": ("envelope_id",),
}
TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS = {
"getTaskAdmissionEnvelope": ("expires_in_seconds",),
"submitTaskAdmissionAuthorization": (
"delegation_epoch_id",
"owner_key_id",
"signature_base64",
),
"getTaskAdmissionStatus": (),
}
class TaskAdmissionBridgeContract:
def __init__(self, service: TaskAdmissionBootstrapService) -> None:
self.service = service
@staticmethod
def _validate_fields(operation: str, request: Mapping[str, Any]) -> None:
required = set(TASK_ADMISSION_OPERATION_REQUIRED_FIELDS[operation])
allowed = required | set(TASK_ADMISSION_OPERATION_OPTIONAL_FIELDS[operation])
if not required.issubset(request):
raise ValueError("TASK_ADMISSION_BRIDGE_REQUIRED_FIELDS_MISSING")
if set(request) - allowed:
raise ValueError("TASK_ADMISSION_BRIDGE_UNKNOWN_FIELDS")
def dispatch(self, operation: str, request: Mapping[str, Any]) -> dict[str, Any]:
if operation not in TASK_ADMISSION_OPERATION_REQUIRED_FIELDS:
raise ValueError("TASK_ADMISSION_BRIDGE_OPERATION_NOT_FOUND")
self._validate_fields(operation, request)
if operation == "getTaskAdmissionEnvelope":
return self.service.create_envelope(request)
if operation == "submitTaskAdmissionAuthorization":
return self.service.authorize_and_admit(request)
if operation == "getTaskAdmissionStatus":
return self.service.status(str(request["envelope_id"]))
raise ValueError("TASK_ADMISSION_BRIDGE_OPERATION_NOT_FOUND")
def handlers(self) -> dict[str, Any]:
return {
operation: (
lambda request, operation=operation: self.dispatch(operation, request)
)
for operation in TASK_ADMISSION_OPERATION_REQUIRED_FIELDS
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,394 @@
"""Append-only ClaimGuard binding for the superseded Single-Host child Task."""
from __future__ import annotations
from datetime import datetime, timezone
import hashlib
import json
import re
import sqlite3
from typing import Any, Callable, Mapping
from aios_task_lineage_supersession_initial_projection import (
PROJECTION_RECEIPT_SCHEMA,
PROJECTION_TABLE,
SOURCE_BUSINESS_EXECUTION_DIGEST,
)
SOURCE_TASK_ID = "TASK-SINGLEHOST-TRUSTROOT-P0-20260904"
CHILD_TASK_ID = SOURCE_TASK_ID + "-S1"
WORKER_ID = "p0-task-principal-v15-20260830"
REPOSITORY = "aios/aios"
TARGET_BRANCH = "agent/single-host-trust-root-p0-20260904"
GENERATION = 1
EXTERNAL_JOB_ID = "job-4e10f92a6a5742518a9f1d8745c845ab"
BINDING_SCHEMA = "aios.task-claim-guard-binding.v2"
RECEIPT_SCHEMA = "aios.task-claim-guard-binding-receipt.v2"
HEX64 = re.compile(r"^[0-9a-f]{64}$")
class ClaimGuardBindingError(RuntimeError):
def __init__(self, code: str, http_status: int = 409):
super().__init__(code)
self.code = code
self.http_status = int(http_status)
def _fail(code: str, status: int = 409) -> None:
raise ClaimGuardBindingError(code, status)
def canonical_json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
def _sha_text(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def _sha_json(value: Any) -> str:
return _sha_text(canonical_json(value))
def _now() -> str:
return datetime.now(timezone.utc).isoformat()
def _row_dict(row: Any) -> dict[str, Any]:
if row is None:
return {}
if hasattr(row, "keys"):
return {str(key): row[key] for key in row.keys()}
_fail("TASK_ROW_MAPPING_UNAVAILABLE", 500)
def _json(raw: Any, code: str) -> Any:
if not isinstance(raw, str):
_fail(code)
try:
return json.loads(raw)
except (TypeError, ValueError) as exc:
raise ClaimGuardBindingError(code) from exc
def _default_policy_factory(value: Mapping[str, Any]):
try:
from aios_single_host_operational_authority import OperationalPolicy
except ImportError as exc:
raise ClaimGuardBindingError("OPERATIONAL_POLICY_RUNTIME_UNAVAILABLE", 503) from exc
return OperationalPolicy.from_mapping(value)
def _policy_value(policy: Any, name: str) -> Any:
try:
return getattr(policy, name)
except AttributeError as exc:
raise ClaimGuardBindingError("OPERATIONAL_POLICY_FIELDS_INVALID") from exc
def _load_projection_evidence(connection, relation) -> dict[str, Any]:
try:
row = connection.execute(
f"SELECT * FROM {PROJECTION_TABLE} WHERE task_id=? AND source_task_id=? "
"AND supersession_ordinal=?",
(CHILD_TASK_ID, SOURCE_TASK_ID, 1),
).fetchone()
except sqlite3.OperationalError as exc:
raise ClaimGuardBindingError(
"FORMAL_CHILD_INITIAL_PROJECTION_SCHEMA_UNAVAILABLE", 503
) from exc
if row is None:
_fail("FORMAL_CHILD_INITIAL_PROJECTION_REQUIRED")
value = _row_dict(row)
receipt = _json(value.get("receipt_json"), "CHILD_PROJECTION_RECEIPT_INVALID")
canonical_digest = value.get("canonical_child_execution_digest")
source_digest = value.get("source_business_execution_digest")
exact = {
"task_id": CHILD_TASK_ID,
"source_task_id": SOURCE_TASK_ID,
"supersession_ordinal": 1,
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"external_job_id": EXTERNAL_JOB_ID,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"owner_decision_sha256": relation["owner_decision_sha256"],
}
if any(value.get(key) != expected for key, expected in exact.items()):
_fail("CHILD_PROJECTION_LINEAGE_MISMATCH")
if (
not isinstance(canonical_digest, str)
or HEX64.fullmatch(canonical_digest) is None
or canonical_digest == SOURCE_BUSINESS_EXECUTION_DIGEST
or source_digest != SOURCE_BUSINESS_EXECUTION_DIGEST
or _sha_text(value["receipt_json"]) != value.get("receipt_sha256")
or not isinstance(receipt, Mapping)
or receipt.get("schema") != PROJECTION_RECEIPT_SCHEMA
or receipt.get("canonical_child_execution_digest") != canonical_digest
or receipt.get("source_business_execution_digest") != source_digest
or receipt.get("owner_supersession_decision_sha256")
!= relation["owner_decision_sha256"]
):
_fail("CHILD_PROJECTION_RECEIPT_MISMATCH")
return {
**value,
"canonical_child_execution_digest": canonical_digest,
"source_business_execution_digest": source_digest,
}
def install_child_binding(
connection,
*,
policy: Mapping[str, Any],
runtime_id: str,
runtime_observation_digest: str,
task_contract_version: str,
policy_factory: Callable[[Mapping[str, Any]], Any] = _default_policy_factory,
now: str | None = None,
) -> dict[str, Any]:
"""Generate and persist the only permitted child binding from current DB state."""
if not isinstance(policy, Mapping):
_fail("OPERATIONAL_POLICY_INVALID")
if not isinstance(runtime_id, str) or not runtime_id or len(runtime_id) > 512:
_fail("RUNTIME_ID_INVALID")
if not isinstance(runtime_observation_digest, str) or HEX64.fullmatch(runtime_observation_digest) is None:
_fail("RUNTIME_OBSERVATION_DIGEST_INVALID")
if not isinstance(task_contract_version, str) or not task_contract_version or len(task_contract_version) > 128:
_fail("TASK_CONTRACT_VERSION_INVALID")
relation = connection.execute(
"SELECT * FROM task_lineage_supersessions WHERE source_task_id=? AND supersession_ordinal=1 AND child_task_id=?",
(SOURCE_TASK_ID, CHILD_TASK_ID),
).fetchone()
if relation is None:
_fail("FORMAL_SUPERSESSION_REQUIRED")
projection = _load_projection_evidence(connection, relation)
canonical_execution_digest = projection["canonical_child_execution_digest"]
task_row = connection.execute("SELECT * FROM tasks WHERE task_id=?", (CHILD_TASK_ID,)).fetchone()
if task_row is None:
_fail("CHILD_TASK_NOT_FOUND", 404)
task = _row_dict(task_row)
if (
task.get("status") != "QUEUED" or int(task.get("revision", -1)) < 2
or int(task.get("attempt", -1)) != 0 or task.get("claimed_by") is not None
or task.get("lease_token_hash") is not None or task.get("lease_expires_at") is not None
or task.get("result_json") is not None or task.get("verification_json") is not None
):
_fail("CHILD_PROJECTED_CLAIMABLE_STATE_REQUIRED")
event_rows = connection.execute(
"SELECT event_type,payload_json FROM events WHERE task_id=? ORDER BY event_id",
(CHILD_TASK_ID,),
).fetchall()
if [row[0] for row in event_rows] != [
"TASK_CREATED",
"TASK_INITIAL_EXECUTION_PROJECTION_AUTHORIZED",
]:
_fail("CANONICAL_INITIAL_PROJECTION_REQUIRED")
projection_event = _json(
event_rows[-1][1], "CANONICAL_INITIAL_PROJECTION_EVENT_INVALID"
)
if (
projection_event.get("source_business_execution_digest")
!= SOURCE_BUSINESS_EXECUTION_DIGEST
or projection_event.get("canonical_child_execution_digest")
!= canonical_execution_digest
or projection_event.get("owner_supersession_decision_sha256")
!= relation["owner_decision_sha256"]
or _sha_json(projection_event) != projection["projection_event_sha256"]
):
_fail("CANONICAL_INITIAL_PROJECTION_EVENT_MISMATCH")
metadata_raw = task.get("metadata_json")
capabilities_raw = task.get("required_capabilities_json")
metadata = _json(metadata_raw, "CHILD_METADATA_INVALID")
capabilities = _json(capabilities_raw, "CHILD_CAPABILITIES_INVALID")
if (
_sha_text(metadata_raw) != projection["metadata_sha256"]
or _sha_text(capabilities_raw)
!= projection["required_capabilities_sha256"]
):
_fail("CHILD_PROJECTION_TASK_DIGEST_MISMATCH")
if not isinstance(metadata, Mapping) or not isinstance(capabilities, list) or not capabilities:
_fail("CHILD_PROJECTION_INVALID")
if any(not isinstance(value, str) or not value for value in capabilities) or len(set(capabilities)) != len(capabilities):
_fail("CHILD_CAPABILITIES_INVALID")
execution = metadata.get("execution")
if not isinstance(execution, Mapping):
_fail("CHILD_EXECUTION_PROJECTION_MISSING")
exact_execution = {
"task_id": CHILD_TASK_ID, "repository": REPOSITORY,
"target_branch": TARGET_BRANCH, "generation": GENERATION,
"execution_digest": canonical_execution_digest, "job_id": EXTERNAL_JOB_ID,
}
if any(execution.get(key) != value for key, value in exact_execution.items()):
_fail("CHILD_EXECUTION_BINDING_MISMATCH")
parsed_policy = policy_factory(dict(policy))
if hasattr(parsed_policy, "validate"):
parsed_policy.validate()
policy_exact = {
"task_id": CHILD_TASK_ID, "repository": REPOSITORY,
"target_branch": TARGET_BRANCH, "generation": GENERATION,
"execution_plan_sha256": canonical_execution_digest,
"external_job_identity": EXTERNAL_JOB_ID,
}
if any(_policy_value(parsed_policy, key) != value for key, value in policy_exact.items()):
_fail("OPERATIONAL_POLICY_SCOPE_MISMATCH")
if tuple(_policy_value(parsed_policy, "allowed_worker_ids")) != (WORKER_ID,):
_fail("OPERATIONAL_POLICY_WORKER_MISMATCH")
expected_oid = execution.get("expected_branch_oid", execution.get("base_oid"))
if expected_oid is not None and _policy_value(parsed_policy, "base_oid") != expected_oid:
_fail("OPERATIONAL_POLICY_BASE_OID_MISMATCH")
policy_json = canonical_json(dict(policy))
policy_sha = _sha_text(policy_json)
policy_digest = str(_policy_value(parsed_policy, "digest"))
if policy_digest != policy_sha:
_fail("OPERATIONAL_POLICY_DIGEST_MISMATCH")
binding = {
"schema": BINDING_SCHEMA, "task_id": CHILD_TASK_ID,
"source_task_id": SOURCE_TASK_ID, "supersession_ordinal": 1,
"expected_revision": int(task["revision"]),
"metadata_sha256": _sha_text(metadata_raw),
"required_capabilities_sha256": _sha_json(sorted(capabilities)),
"worker_id": WORKER_ID,
"execution_digest": canonical_execution_digest,
"canonical_child_execution_digest": canonical_execution_digest,
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"repository": REPOSITORY, "target_branch": TARGET_BRANCH, "generation": GENERATION,
"external_job_id": EXTERNAL_JOB_ID, "policy_sha256": policy_sha,
"policy_digest": policy_digest, "runtime_id": runtime_id,
"runtime_observation_digest": runtime_observation_digest,
"task_contract_version": task_contract_version,
"owner_supersession_decision_sha256": relation["owner_decision_sha256"],
}
binding_json = canonical_json(binding)
binding_sha = _sha_text(binding_json)
binding_id = "sha256:" + binding_sha
existing = connection.execute(
"SELECT binding_sha256 FROM task_claim_guard_bindings_v2 WHERE task_id=?", (CHILD_TASK_ID,)
).fetchone()
if existing is not None:
if existing[0] != binding_sha:
_fail("CLAIM_GUARD_BINDING_ALREADY_EXISTS")
receipt = connection.execute(
"SELECT receipt_json FROM task_claim_guard_binding_receipts_v2 WHERE binding_id=?", (binding_id,)
).fetchone()
if receipt is None:
_fail("CLAIM_GUARD_BINDING_RECEIPT_MISSING", 500)
return {**json.loads(receipt[0]), "idempotent": True}
created_at = now or _now()
connection.execute(
"""INSERT INTO task_claim_guard_bindings_v2(
binding_id,task_id,source_task_id,supersession_ordinal,expected_revision,
metadata_sha256,required_capabilities_sha256,worker_id,execution_digest,
repository,target_branch,generation,policy_json,policy_sha256,binding_json,
binding_sha256,created_at) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
(binding_id, CHILD_TASK_ID, SOURCE_TASK_ID, 1, binding["expected_revision"],
binding["metadata_sha256"], binding["required_capabilities_sha256"], WORKER_ID,
canonical_execution_digest, REPOSITORY, TARGET_BRANCH, GENERATION, policy_json, policy_sha,
binding_json, binding_sha, created_at),
)
receipt = {
"schema": RECEIPT_SCHEMA, "status": "BOUND", "binding_id": binding_id,
"binding_sha256": binding_sha, "task_id": CHILD_TASK_ID,
"expected_revision": binding["expected_revision"],
"metadata_sha256": binding["metadata_sha256"], "worker_id": WORKER_ID,
"execution_digest": canonical_execution_digest,
"canonical_child_execution_digest": canonical_execution_digest,
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH, "generation": GENERATION,
"owner_supersession_decision_sha256": relation["owner_decision_sha256"],
}
receipt_sha = _sha_json(receipt)
receipt_id = "sha256:" + receipt_sha
connection.execute(
"INSERT INTO task_claim_guard_binding_receipts_v2 VALUES(?,?,?,?,?,?)",
(receipt_id, binding_id, CHILD_TASK_ID, canonical_json(receipt), receipt_sha, created_at),
)
return {**receipt, "receipt_id": receipt_id, "receipt_sha256": receipt_sha, "idempotent": False}
def _load_child_binding(connection) -> tuple[dict[str, Any], dict[str, Any]]:
row = connection.execute(
"SELECT * FROM task_claim_guard_bindings_v2 WHERE task_id=?", (CHILD_TASK_ID,)
).fetchone()
if row is None:
_fail("CHILD_CLAIM_GUARD_BINDING_MISSING")
value = _row_dict(row)
binding = _json(value["binding_json"], "CLAIM_GUARD_BINDING_INVALID")
policy = _json(value["policy_json"], "OPERATIONAL_POLICY_INVALID")
relation = connection.execute(
"SELECT * FROM task_lineage_supersessions WHERE source_task_id=? "
"AND supersession_ordinal=1 AND child_task_id=?",
(SOURCE_TASK_ID, CHILD_TASK_ID),
).fetchone()
if relation is None:
_fail("FORMAL_SUPERSESSION_REQUIRED")
projection = _load_projection_evidence(connection, relation)
canonical_execution_digest = projection["canonical_child_execution_digest"]
if (
_sha_text(value["binding_json"]) != value["binding_sha256"]
or binding.get("schema") != BINDING_SCHEMA
or value.get("execution_digest") != canonical_execution_digest
or binding.get("execution_digest") != canonical_execution_digest
or binding.get("canonical_child_execution_digest")
!= canonical_execution_digest
or binding.get("source_business_execution_digest")
!= SOURCE_BUSINESS_EXECUTION_DIGEST
or binding.get("owner_supersession_decision_sha256")
!= relation["owner_decision_sha256"]
or policy.get("execution_plan_sha256") != canonical_execution_digest
):
_fail("CLAIM_GUARD_BINDING_DIGEST_MISMATCH")
task = connection.execute("SELECT * FROM tasks WHERE task_id=?", (CHILD_TASK_ID,)).fetchone()
task = _row_dict(task)
if (
task.get("status") != "QUEUED" or int(task.get("revision", -1)) != int(binding["expected_revision"])
or int(task.get("attempt", -1)) != 0 or task.get("claimed_by") is not None
or task.get("lease_token_hash") is not None or task.get("lease_expires_at") is not None
or _sha_text(task.get("metadata_json", "")) != binding["metadata_sha256"]
or task.get("result_json") is not None or task.get("verification_json") is not None
):
_fail("CHILD_CLAIM_GUARD_TASK_CAS_MISMATCH")
return binding, policy
def _run_child_guard(connection, binding, policy_value, agent_id):
try:
from aios_single_host_operational_authority import OperationalPolicy, OperationalVerifier
from aios_singlehost_claim_guard_stage1 import _LiveReader, REGISTRY_PATH
from aios_task_claim_guard import ClaimGuard, ClaimGuardError as CoreError
except ImportError as exc:
raise ClaimGuardBindingError("CLAIM_GUARD_RUNTIME_UNAVAILABLE", 503) from exc
policy = OperationalPolicy.from_mapping(policy_value)
cfg = {
"task_id": CHILD_TASK_ID, "expected_claim_revision": binding["expected_revision"],
"task_metadata_sha256": binding["metadata_sha256"],
"required_capabilities_sha256": binding["required_capabilities_sha256"],
"task_contract_version": binding["task_contract_version"], "worker_id": WORKER_ID,
"policy_digest": binding["policy_digest"], "runtime_id": binding["runtime_id"],
"runtime_observation_digest": binding["runtime_observation_digest"],
}
reader = _LiveReader(connection, policy, cfg, now=int(datetime.now(timezone.utc).timestamp()))
try:
ClaimGuard(policy, OperationalVerifier(REGISTRY_PATH, require_root_owner=True)).validate_in_transaction(
reader, task_id=CHILD_TASK_ID,
expected_revision=int(binding["expected_revision"]), worker_id=agent_id,
)
except CoreError as exc:
raise ClaimGuardBindingError(exc.code) from exc
def enforce_claim_guard(connection, *, task_id: str, agent_id: str) -> None:
"""Exact two-lineage dispatcher; unrelated Tasks retain their prior behavior."""
if task_id == SOURCE_TASK_ID:
try:
from aios_singlehost_claim_guard_stage1 import enforce_claim_guard as frozen_guard
except ImportError as exc:
raise ClaimGuardBindingError("FROZEN_SOURCE_CLAIM_GUARD_UNAVAILABLE", 503) from exc
frozen_guard(connection, task_id=task_id, agent_id=agent_id)
return
if task_id != CHILD_TASK_ID:
return
if agent_id != WORKER_ID:
_fail("WORKER_ID_MISMATCH", 403)
binding, policy = _load_child_binding(connection)
_run_child_guard(connection, binding, policy, agent_id)
@@ -0,0 +1,148 @@
"""Fail-closed signed deterministic execution amendment for R7."""
from datetime import datetime, timezone
import hashlib, json, sqlite3
from pathlib import Path
from typing import Mapping
from aios_task_terminal_recovery import DeterministicPlan, RecoveryError, canonical_json
AMENDMENT_SCHEMA="aios.deterministic-execution-amendment.v1"
APPLY_SCHEMA="aios.deterministic-execution-amendment-apply.v1"
NAMESPACE="aios-owner-approval-v1"
EVENT="TASK_RECOVERY_EXECUTION_AMENDMENT_APPLIED"
APPLY_STATES=frozenset({"FAILED", "QUEUED"})
def _err(code): raise RecoveryError(code)
def _sha(v): return hashlib.sha256(canonical_json(v).encode()).hexdigest()
def amendment_body(x):
return {k:v for k,v in x.items() if k not in {"signature","amendment_id"}}
def amendment_id(x): return "sha256:"+_sha(amendment_body(x))
def signed_payload(x):
return canonical_json({"amendment_id":x["amendment_id"],**amendment_body(x)}).encode()
def parse_amendment(raw):
x=dict(raw)
required={"schema","amendment_id","nonce","task_id","job_id","generation",
"repository","branch","expected_branch_oid","recovery_epoch","decision_id",
"plan_id","plan_sha256","expected_apply_status","expected_apply_revision",
"expected_apply_attempt","expected_current_capabilities","envelope",
"owner_identity","key_id","issued_at","not_before","expires_at",
"one_time_use","signature"}
if set(x)<required or x.get("schema")!=AMENDMENT_SCHEMA: _err("AMENDMENT_SCHEMA_INVALID")
if x["expected_apply_status"] not in APPLY_STATES: _err("AMENDMENT_APPLY_STATUS_INVALID")
if x.get("one_time_use") is not True: _err("AMENDMENT_ONE_TIME_REQUIRED")
if not isinstance(x["envelope"],Mapping): _err("AMENDMENT_ENVELOPE_INVALID")
if not isinstance(x["expected_current_capabilities"],list): _err("AMENDMENT_CAPABILITIES_INVALID")
for k in ("generation","expected_apply_revision","expected_apply_attempt"):
if isinstance(x[k],bool) or not isinstance(x[k],int) or x[k]<0: _err("AMENDMENT_CAS_INVALID")
for k in ("issued_at","not_before","expires_at"):
try:d=datetime.fromisoformat(str(x[k]).replace("Z","+00:00"))
except ValueError as e: raise RecoveryError("AMENDMENT_TIME_INVALID") from e
if d.tzinfo is None:_err("AMENDMENT_TIME_INVALID")
x[k]=d.astimezone(timezone.utc).isoformat().replace("+00:00","Z")
x["expected_current_capabilities"]=sorted(set(x["expected_current_capabilities"]))
x["envelope"]=dict(x["envelope"])
return x
def _validate_envelope(env,plan,x):
try:
from aios_online_agent_execution import GovernedExecutionBinding,validate_execution_envelope
binding=GovernedExecutionBinding.from_mapping(env)
e=dict(validate_execution_envelope(env,binding=binding,worker_capabilities=plan.required_capabilities))
except Exception as z: raise RecoveryError("AMENDMENT_EXECUTION_ENVELOPE_INVALID") from z
if e.get("action")!="deterministic" or e.get("adapter")!="deterministic":_err("AMENDMENT_EXECUTION_ACTION_INVALID")
if tuple(sorted(e.get("capability_scope") or ()))!=plan.required_capabilities:_err("AMENDMENT_EXECUTION_CAPABILITIES_MISMATCH")
if not e.get("change_set") or not e.get("verification"):_err("AMENDMENT_EXECUTION_OPERATIONS_REQUIRED")
if e.get("runtime_input") not in (None,{}) or e.get("source_signals") not in (None,[]):_err("AMENDMENT_STALE_RUNTIME_FIELDS_FORBIDDEN")
for k,w in (("task_id",x["task_id"]),("job_id",x["job_id"]),("repository",x["repository"]),
("generation",x["generation"]),("target_branch",x["branch"]),
("expected_branch_oid",x["expected_branch_oid"]),("base_oid",x["expected_branch_oid"]),
("plan_id",x["plan_id"]),("plan_sha256",x["plan_sha256"]),("recovery_epoch",x["recovery_epoch"])):
if e.get(k)!=w:_err("AMENDMENT_EXECUTION_BINDING_MISMATCH")
g=e.get("governance") or {}
if g.get("writer_lease") is not True:_err("AMENDMENT_WRITER_LEASE_REQUIRED")
if any(g.get(k,False) for k in ("main_write","production_write","tag_write","baseline_write","force_push","protected_ref_bypass")):_err("AMENDMENT_PROTECTED_WRITE_FORBIDDEN")
return e
class DeterministicExecutionAmendmentController:
def __init__(self,db_path,verifier,*,clock=None,envelope_validator=None):
self.db_path=Path(db_path); self.verifier=verifier
self.clock=clock or (lambda:datetime.now(timezone.utc)); self._init()
def _db(self):
c=sqlite3.connect(self.db_path,timeout=10,isolation_level=None); c.row_factory=sqlite3.Row
c.execute("PRAGMA busy_timeout=10000"); return c
def _now(self): return self.clock().astimezone(timezone.utc).isoformat().replace("+00:00","Z")
def _init(self):
with self._db() as c:
c.execute("DROP TRIGGER IF EXISTS aios_r7_recovery_capability_projection")
c.executescript("""
CREATE TABLE IF NOT EXISTS deterministic_execution_amendments(
amendment_id TEXT PRIMARY KEY,nonce TEXT UNIQUE NOT NULL,task_id TEXT NOT NULL,
decision_id TEXT NOT NULL,plan_id TEXT NOT NULL,plan_sha256 TEXT NOT NULL,
envelope_digest TEXT NOT NULL,envelope_json TEXT NOT NULL,payload_json TEXT NOT NULL,
created_at TEXT NOT NULL,UNIQUE(task_id,decision_id));
DROP TRIGGER IF EXISTS aios_r7_recovery_requires_executable_amendment;
CREATE TRIGGER aios_r7_recovery_requires_executable_amendment
BEFORE UPDATE OF status ON tasks
WHEN OLD.status='FAILED' AND NEW.status='QUEUED' BEGIN
SELECT CASE WHEN NOT EXISTS(
SELECT 1 FROM deterministic_execution_amendments a
WHERE a.task_id=OLD.task_id
AND a.decision_id=json_extract(NEW.metadata_json,'$.recovery.decision_id')
AND a.amendment_id=json_extract(OLD.metadata_json,'$.recovery.execution_amendment_id')
AND a.envelope_digest=json_extract(OLD.metadata_json,'$.execution.execution_digest')
) THEN RAISE(ABORT,'SIGNED_EXECUTION_AMENDMENT_REQUIRED') END;
END;""")
def _plan(self,c,x):
r=c.execute("SELECT * FROM deterministic_plans WHERE plan_id=?",(x["plan_id"],)).fetchone()
if r is None or r["plan_sha256"]!=x["plan_sha256"]:_err("AMENDMENT_PLAN_NOT_FOUND_OR_MISMATCH")
p=DeterministicPlan.from_dict(json.loads(r["payload_json"]))
if (p.task_id,p.job_id,p.generation,p.repository,p.branch,p.expected_branch_oid,p.recovery_epoch)!=(x["task_id"],x["job_id"],x["generation"],x["repository"],x["branch"],x["expected_branch_oid"],x["recovery_epoch"]):_err("AMENDMENT_PLAN_BINDING_MISMATCH")
return p
def persist(self,raw):
x=parse_amendment(raw)
if x["amendment_id"]!=amendment_id(x):_err("AMENDMENT_ID_DIGEST_MISMATCH")
now=self._now()
if now<x["not_before"] or now>=x["expires_at"]:_err("AMENDMENT_EXPIRED_OR_NOT_YET_VALID")
if not self.verifier.verify(namespace=NAMESPACE,payload=signed_payload(x),signature=x["signature"],key_id=x["key_id"]):_err("AMENDMENT_SIGNATURE_INVALID")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); e=_validate_envelope(x["envelope"],self._plan(c,x),x)
ej=canonical_json(e); ed=e["execution_digest"]; stored=dict(x); stored["signature"]="<present>"
old=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=?",(x["amendment_id"],)).fetchone()
if old:
if old["envelope_json"]!=ej:_err("AMENDMENT_ID_REUSE_MISMATCH")
c.commit(); return {"status":"PRESENT","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
if c.execute("SELECT 1 FROM deterministic_execution_amendments WHERE nonce=? OR (task_id=? AND decision_id=?)",(x["nonce"],x["task_id"],x["decision_id"])).fetchone():_err("AMENDMENT_REPLAY_OR_DECISION_REUSE")
c.execute("INSERT INTO deterministic_execution_amendments VALUES(?,?,?,?,?,?,?,?,?,?)",(x["amendment_id"],x["nonce"],x["task_id"],x["decision_id"],x["plan_id"],x["plan_sha256"],ed,ej,canonical_json(stored),now)); c.commit()
return {"status":"PERSISTED","amendment_id":x["amendment_id"],"task_id":x["task_id"],"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":ed}
def apply(self,raw):
if raw.get("schema")!=APPLY_SCHEMA:_err("AMENDMENT_APPLY_SCHEMA_INVALID")
aid=raw.get("amendment_id"); tid=raw.get("task_id")
with self._db() as c:
c.execute("BEGIN IMMEDIATE"); a=c.execute("SELECT * FROM deterministic_execution_amendments WHERE amendment_id=? AND task_id=?",(aid,tid)).fetchone()
if a is None:_err("AMENDMENT_NOT_FOUND")
x=json.loads(a["payload_json"]); d=json.loads(a["envelope_json"])
t=c.execute("SELECT * FROM tasks WHERE task_id=?",(tid,)).fetchone()
if t is None:_err("TASK_NOT_FOUND")
m=json.loads(t["metadata_json"] or "{}"); cp=tuple(sorted(json.loads(t["required_capabilities_json"] or "[]"))); dc=tuple(sorted(d.get("capability_scope") or ()))
rec=m.get("recovery"); rec=dict(rec) if isinstance(rec,Mapping) else {}
old=m.get("execution")
if isinstance(old,Mapping) and old.get("execution_digest")==a["envelope_digest"] and cp==dc and rec.get("execution_amendment_id")==aid:
c.commit(); return {"status":"PRESENT","task_id":tid,"amendment_id":aid,"execution_digest":a["envelope_digest"],"revision":int(t["revision"]),"attempt":int(t["attempt"])}
if t["status"]!=x["expected_apply_status"] or int(t["revision"])!=x["expected_apply_revision"] or int(t["attempt"])!=x["expected_apply_attempt"] or cp!=tuple(x["expected_current_capabilities"]) or t["claimed_by"] is not None or t["lease_token_hash"] is not None:_err("AMENDMENT_APPLY_TASK_CAS_MISMATCH")
if t["status"]=="QUEUED" and rec.get("decision_id")!=x["decision_id"]:_err("AMENDMENT_DECISION_BINDING_MISMATCH")
m["execution"]=d; m["execution_projection"]={"schema":"aios.online-agent.task-projection.v1","task_id":tid,"job_id":d.get("job_id"),"execution_digest":a["envelope_digest"]}
rec["execution_amendment_id"]=aid; m["recovery"]=rec; now=self._now()
q=c.execute("""UPDATE tasks SET required_capabilities_json=?,metadata_json=?,revision=revision+1,updated_at=? WHERE task_id=? AND status=? AND revision=? AND attempt=? AND claimed_by IS NULL AND lease_token_hash IS NULL""",(canonical_json(list(dc)),canonical_json(m),now,tid,x["expected_apply_status"],x["expected_apply_revision"],x["expected_apply_attempt"]))
if q.rowcount!=1:_err("AMENDMENT_APPLY_TASK_CAS_FAILED")
c.execute("INSERT INTO events(task_id,agent_id,event_type,payload_json,created_at) VALUES(?,?,?,?,?)",(tid,None,EVENT,canonical_json({"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc)}),now)); c.commit()
return {"status":"APPLIED","task_id":tid,"amendment_id":aid,"decision_id":x["decision_id"],"plan_id":x["plan_id"],"execution_digest":a["envelope_digest"],"required_capabilities":list(dc),"revision":x["expected_apply_revision"]+1,"attempt":x["expected_apply_attempt"]}
@@ -0,0 +1,339 @@
"""One-time, Owner-signed, append-only supersession for one exact Task lineage."""
from __future__ import annotations
from contextlib import closing
from datetime import datetime, timedelta, timezone
import hashlib
import json
from pathlib import Path
import re
import sqlite3
from typing import Any, Callable, Mapping, Protocol
DECISION_SCHEMA = "aios.task-lineage-supersession-decision.v1"
RECEIPT_SCHEMA = "aios.task-lineage-supersession-receipt.v1"
OWNER_NAMESPACE = "aios-owner-approval-v1"
OWNER_PRINCIPAL = "aios-task-grant-signer"
OWNER_KEY_ID = "SHA256:rPqpxfPHtaZPM3zSWojX8Q7VxzCknsYqOokyo/aAM3A"
SOURCE_TASK_ID = "TASK-SINGLEHOST-TRUSTROOT-P0-20260904"
CHILD_TASK_ID = SOURCE_TASK_ID + "-S1"
REPOSITORY = "aios/aios"
TARGET_BRANCH = "agent/single-host-trust-root-p0-20260904"
GENERATION = 1
EXECUTION_DIGEST = "e99bda6a251832eb3865271eb5687af7eea44dd261144cdcdfa57104c0780536"
EXTERNAL_JOB_ID = "job-4e10f92a6a5742518a9f1d8745c845ab"
SUPERSESSION_ORDINAL = 1
CANONICAL_PROFILE = "aios.task-initial-execution.v1"
CANONICAL_PROFILE_VERSION = 1
SUPERSESSION_REASON = "formal-lineage-supersession-after-expired-unclaimed-source"
MAX_TTL = timedelta(minutes=5)
HEX64 = re.compile(r"^[0-9a-f]{64}$")
SAFE_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/+:-]{2,511}$")
class TaskLineageSupersessionError(RuntimeError):
def __init__(self, code: str, http_status: int = 409):
super().__init__(code)
self.code = code
self.http_status = int(http_status)
class SignatureVerifier(Protocol):
def verify(self, *, namespace: str, payload: bytes, signature: str, key_id: str) -> bool: ...
def canonical_json(value: Any) -> str:
return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":"), allow_nan=False)
def sha256_text(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def _fail(code: str, status: int = 409) -> None:
raise TaskLineageSupersessionError(code, status)
def _utc(value: Any, field: str) -> datetime:
if not isinstance(value, str) or not value or len(value) > 80:
_fail(field.upper() + "_INVALID", 400)
try:
result = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise TaskLineageSupersessionError(field.upper() + "_INVALID", 400) from exc
if result.tzinfo is None:
_fail(field.upper() + "_INVALID", 400)
return result.astimezone(timezone.utc)
def _text(value: Any, field: str, *, maximum: int = 1024) -> str:
if not isinstance(value, str) or not value or len(value) > maximum:
_fail(field.upper() + "_INVALID", 400)
return value
def decision_payload(raw: Mapping[str, Any]) -> dict[str, Any]:
return {key: raw[key] for key in raw if key != "signature"}
DECISION_FIELDS = {
"schema", "decision_id", "nonce", "source_task_id",
"source_task_identity_sha256", "source_metadata_sha256",
"source_expected_status", "source_expected_revision", "source_expected_attempt",
"source_expected_claimed_by", "source_expected_lease_token_hash",
"source_expected_lease_expires_at", "repository", "target_branch", "generation",
"execution_digest", "external_job_id", "child_task_id", "supersession_ordinal",
"canonical_projection_profile", "canonical_projection_version", "supersession_reason",
"owner_identity", "signature_namespace", "signature_principal", "key_id",
"issued_at", "not_before", "expires_at", "one_time_use", "signature",
}
def validate_decision(raw: Mapping[str, Any], *, now: datetime) -> dict[str, Any]:
if not isinstance(raw, Mapping) or set(raw) != DECISION_FIELDS:
_fail("DECISION_FIELDS_INVALID", 400)
exact = {
"schema": DECISION_SCHEMA,
"source_task_id": SOURCE_TASK_ID,
"source_expected_status": "EXPIRED",
"source_expected_attempt": 0,
"source_expected_claimed_by": None,
"source_expected_lease_token_hash": None,
"source_expected_lease_expires_at": None,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"execution_digest": EXECUTION_DIGEST,
"external_job_id": EXTERNAL_JOB_ID,
"child_task_id": CHILD_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
"supersession_reason": SUPERSESSION_REASON,
"owner_identity": OWNER_PRINCIPAL,
"signature_namespace": OWNER_NAMESPACE,
"signature_principal": OWNER_PRINCIPAL,
"key_id": OWNER_KEY_ID,
"one_time_use": True,
}
for field, expected in exact.items():
if raw[field] != expected:
_fail(field.upper() + "_MISMATCH", 403 if field.startswith(("owner", "signature", "key")) else 409)
for field in ("decision_id", "nonce"):
value = _text(raw[field], field, maximum=256)
if SAFE_ID.fullmatch(value) is None:
_fail(field.upper() + "_INVALID", 400)
for field in ("source_task_identity_sha256", "source_metadata_sha256"):
if not isinstance(raw[field], str) or HEX64.fullmatch(raw[field]) is None:
_fail(field.upper() + "_INVALID", 400)
if type(raw["source_expected_revision"]) is not int or raw["source_expected_revision"] < 1:
_fail("SOURCE_EXPECTED_REVISION_INVALID", 400)
_text(raw["signature"], "signature", maximum=32768)
issued_at = _utc(raw["issued_at"], "issued_at")
not_before = _utc(raw["not_before"], "not_before")
expires_at = _utc(raw["expires_at"], "expires_at")
if issued_at > expires_at or not_before > expires_at:
_fail("DECISION_TIME_ORDER_INVALID", 403)
if expires_at - issued_at > MAX_TTL or expires_at <= issued_at:
_fail("DECISION_TTL_INVALID", 403)
now = now.astimezone(timezone.utc)
if now < not_before or now >= expires_at:
_fail("DECISION_EXPIRED_OR_NOT_YET_VALID", 403)
if issued_at > now + timedelta(seconds=5) or now - issued_at > MAX_TTL:
_fail("DECISION_ISSUED_AT_INVALID", 403)
return {**dict(raw), "issued_at_dt": issued_at, "not_before_dt": not_before, "expires_at_dt": expires_at}
def _default_identity_digest(row: sqlite3.Row) -> str:
try:
from aios_expired_task_recovery import task_identity_digest
except ImportError as exc:
raise TaskLineageSupersessionError("FORMAL_TASK_IDENTITY_HELPER_UNAVAILABLE", 503) from exc
return str(task_identity_digest(row))
def _json_object(row: sqlite3.Row, field: str) -> dict[str, Any]:
try:
value = json.loads(row[field])
except (TypeError, ValueError, KeyError) as exc:
raise TaskLineageSupersessionError("SOURCE_TASK_JSON_INVALID", 409) from exc
if not isinstance(value, Mapping):
_fail("SOURCE_TASK_JSON_INVALID")
return dict(value)
def _verify_source(row: sqlite3.Row, d: Mapping[str, Any], identity_digest: Callable[[sqlite3.Row], str]) -> None:
if row["task_id"] != SOURCE_TASK_ID:
_fail("SOURCE_TASK_ID_MISMATCH")
cas = (
row["status"] == "EXPIRED"
and int(row["revision"]) == d["source_expected_revision"]
and int(row["attempt"]) == 0
and row["claimed_by"] is None
and row["lease_token_hash"] is None
and row["lease_expires_at"] is None
)
if not cas:
_fail("SOURCE_TASK_EXACT_CAS_MISMATCH")
metadata_raw = row["metadata_json"]
if not isinstance(metadata_raw, str) or sha256_text(metadata_raw) != d["source_metadata_sha256"]:
_fail("SOURCE_METADATA_DIGEST_MISMATCH")
if identity_digest(row) != d["source_task_identity_sha256"]:
_fail("SOURCE_TASK_IDENTITY_DIGEST_MISMATCH")
metadata = _json_object(row, "metadata_json")
execution = metadata.get("execution")
constraints = _json_object(row, "constraints_json")
if not isinstance(execution, Mapping):
_fail("SOURCE_EXECUTION_BINDING_INVALID")
expected = {
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"execution_digest": EXECUTION_DIGEST,
"external_job_id": EXTERNAL_JOB_ID,
"task_id": SOURCE_TASK_ID,
}
if any(execution.get(key) != value for key, value in expected.items()):
_fail("SOURCE_EXECUTION_BINDING_MISMATCH")
if constraints.get("generation") != GENERATION:
_fail("SOURCE_GENERATION_CONSTRAINT_MISMATCH")
class TaskLineageSupersessionController:
def __init__(self, db_path: str | Path, verifier: SignatureVerifier, *, clock=None, identity_digest=None):
self.db_path = Path(db_path)
self.verifier = verifier
self.clock = clock or (lambda: datetime.now(timezone.utc))
self.identity_digest = identity_digest or _default_identity_digest
def _connect(self) -> sqlite3.Connection:
connection = sqlite3.connect(self.db_path, timeout=30, isolation_level=None)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
def _now(self) -> datetime:
value = self.clock()
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
@staticmethod
def _assert_schema(connection: sqlite3.Connection) -> None:
required = {"tasks", "events", "task_lineage_supersessions", "task_lineage_supersession_receipts"}
actual = {row[0] for row in connection.execute("SELECT name FROM sqlite_master WHERE type='table'")}
if not required.issubset(actual):
_fail("SUPERSESSION_SCHEMA_UNAVAILABLE", 503)
columns = {row[1] for row in connection.execute("PRAGMA table_info(tasks)")}
task_columns = {
"task_id", "goal", "instructions", "success_criteria_json",
"required_capabilities_json", "constraints_json", "metadata_json", "deadline",
"priority", "max_attempts", "attempt", "status", "claimed_by",
"lease_token_hash", "lease_expires_at", "result_json", "result_digest",
"result_idempotency_key", "verification_json", "revision", "created_at", "updated_at",
}
if not task_columns.issubset(columns):
_fail("TASK_SCHEMA_UNSUPPORTED", 503)
def supersede(self, raw: Mapping[str, Any]) -> dict[str, Any]:
now = self._now()
decision = validate_decision(raw, now=now)
payload_bytes = canonical_json(decision_payload(raw)).encode("utf-8")
if not self.verifier.verify(
namespace=OWNER_NAMESPACE, payload=payload_bytes,
signature=decision["signature"], key_id=decision["key_id"],
):
_fail("DECISION_SIGNATURE_INVALID", 403)
decision_sha = hashlib.sha256(payload_bytes).hexdigest()
now_text = now.isoformat()
with closing(self._connect()) as connection:
connection.execute("BEGIN IMMEDIATE")
try:
self._assert_schema(connection)
replay = connection.execute(
"SELECT owner_decision_sha256,receipt_json FROM task_lineage_supersession_receipts WHERE decision_id=? OR nonce=?",
(decision["decision_id"], decision["nonce"]),
).fetchone()
if replay is not None:
if replay["owner_decision_sha256"] != decision_sha:
_fail("DECISION_REPLAY_MISMATCH")
result = json.loads(replay["receipt_json"])
connection.commit()
return {**result, "idempotent": True}
source = connection.execute("SELECT * FROM tasks WHERE task_id=?", (SOURCE_TASK_ID,)).fetchone()
if source is None:
_fail("SOURCE_TASK_NOT_FOUND", 404)
_verify_source(source, decision, self.identity_digest)
if connection.execute(
"SELECT 1 FROM task_lineage_supersessions WHERE source_task_id=? AND supersession_ordinal=?",
(SOURCE_TASK_ID, SUPERSESSION_ORDINAL),
).fetchone():
_fail("SUPERSESSION_ALREADY_EXISTS")
if connection.execute("SELECT 1 FROM tasks WHERE task_id=?", (CHILD_TASK_ID,)).fetchone():
_fail("CHILD_TASK_ALREADY_EXISTS")
connection.execute(
"""INSERT INTO tasks(
task_id,goal,instructions,success_criteria_json,required_capabilities_json,
constraints_json,metadata_json,deadline,priority,max_attempts,attempt,status,
claimed_by,lease_token_hash,lease_expires_at,result_json,result_digest,
result_idempotency_key,verification_json,revision,created_at,updated_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
(
CHILD_TASK_ID, source["goal"], source["instructions"], source["success_criteria_json"],
"[]", source["constraints_json"], "{}", None, source["priority"], source["max_attempts"],
0, "QUEUED", None, None, None, None, None, None, None, 1, now_text, now_text,
),
)
connection.execute(
"""INSERT INTO task_lineage_supersessions(
source_task_id,supersession_ordinal,child_task_id,source_task_identity_sha256,
source_metadata_sha256,owner_decision_sha256,decision_id,created_at)
VALUES(?,?,?,?,?,?,?,?)""",
(SOURCE_TASK_ID, SUPERSESSION_ORDINAL, CHILD_TASK_ID,
decision["source_task_identity_sha256"], decision["source_metadata_sha256"],
decision_sha, decision["decision_id"], now_text),
)
event_payload = {
"task_id": CHILD_TASK_ID, "supersedes_task_id": SOURCE_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"source_task_identity_sha256": decision["source_task_identity_sha256"],
"source_metadata_sha256": decision["source_metadata_sha256"],
"owner_supersession_decision_sha256": decision_sha,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
}
connection.execute(
"INSERT INTO events(task_id,agent_id,event_type,payload_json,created_at) VALUES(?,?,?,?,?)",
(CHILD_TASK_ID, None, "TASK_CREATED", canonical_json(event_payload), now_text),
)
receipt = {
"schema": RECEIPT_SCHEMA, "status": "SUPERSEDED",
"decision_id": decision["decision_id"], "owner_decision_sha256": decision_sha,
"source_task_id": SOURCE_TASK_ID, "source_task_identity_sha256": decision["source_task_identity_sha256"],
"source_metadata_sha256": decision["source_metadata_sha256"],
"source_status": "EXPIRED", "source_revision": decision["source_expected_revision"],
"source_attempt": 0, "source_history_preserved": True,
"child_task_id": CHILD_TASK_ID, "supersession_ordinal": SUPERSESSION_ORDINAL,
"child_status": "QUEUED", "child_revision": 1, "child_attempt": 0,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
"exact_identity_cas": True, "one_time_use": True,
"owner_signature_namespace": OWNER_NAMESPACE,
"owner_signature_principal": OWNER_PRINCIPAL,
"owner_signature_key_id": OWNER_KEY_ID,
"owner_signature_sha256": sha256_text(decision["signature"]),
}
connection.execute(
"""INSERT INTO task_lineage_supersession_receipts(
decision_id,nonce,source_task_id,supersession_ordinal,child_task_id,
owner_decision_sha256,decision_json,receipt_json,consumed_at)
VALUES(?,?,?,?,?,?,?,?,?)""",
(decision["decision_id"], decision["nonce"], SOURCE_TASK_ID, SUPERSESSION_ORDINAL,
CHILD_TASK_ID, decision_sha, canonical_json(dict(raw)), canonical_json(receipt), now_text),
)
connection.commit()
return {**receipt, "idempotent": False}
except Exception:
connection.rollback()
raise
@@ -0,0 +1,548 @@
"""Canonical initial projection for the exact Single-Host supersession child.
The source business execution digest is lineage evidence. It is deliberately
not used as the child execution-envelope digest: the canonical envelope binds
the child Task id and therefore has its own server-derived digest.
"""
from __future__ import annotations
from dataclasses import dataclass
import hashlib
import json
import sqlite3
from typing import Any, Mapping
from aios_agent_task_initial_execution_projection import (
InitialExecutionProjectionError,
InitialExecutionProjectionRequest,
InitialExecutionProjectionStore,
RUNTIME_CAPABILITY_RE,
_now,
_require_string,
_required_authorization_id,
)
from aios_online_agent_execution import (
ExecutionProjectionError,
GovernedExecutionBinding,
validate_execution_envelope,
)
from aios_task_lineage_supersession import (
CANONICAL_PROFILE,
CANONICAL_PROFILE_VERSION,
CHILD_TASK_ID,
DECISION_SCHEMA,
EXECUTION_DIGEST as SOURCE_BUSINESS_EXECUTION_DIGEST,
EXTERNAL_JOB_ID,
GENERATION,
OWNER_KEY_ID,
OWNER_NAMESPACE,
OWNER_PRINCIPAL,
RECEIPT_SCHEMA as SUPERSESSION_RECEIPT_SCHEMA,
REPOSITORY,
SOURCE_TASK_ID,
SUPERSESSION_ORDINAL,
TARGET_BRANCH,
canonical_json,
decision_payload,
)
PROJECTION_INTENT_SCHEMA = "aios.task-lineage-supersession-initial-projection-intent.v1"
PROJECTION_RECEIPT_SCHEMA = "aios.task-lineage-supersession-initial-projection-receipt.v1"
PROJECTION_IDEMPOTENCY_KEY = "task-lineage-supersession-s1-initial-projection-v1"
PROJECTION_REASON = "canonical-initial-projection-for-formal-supersession-child"
PROJECTION_TABLE = "task_lineage_supersession_initial_projections"
def _sha_text(value: str) -> str:
return hashlib.sha256(value.encode("utf-8")).hexdigest()
def _sha_json(value: Any) -> str:
return _sha_text(canonical_json(value))
def _json_object(raw: Any, code: str) -> dict[str, Any]:
if not isinstance(raw, str):
raise InitialExecutionProjectionError(code, 409)
try:
value = json.loads(raw)
except (TypeError, ValueError) as exc:
raise InitialExecutionProjectionError(code, 409) from exc
if not isinstance(value, Mapping):
raise InitialExecutionProjectionError(code, 409)
return dict(value)
@dataclass(frozen=True)
class SupersessionChildProjectionIntent:
"""Caller-selectable portion of the exact child projection.
Task identity, CAS values, Job identity, projection profile, reason and
idempotency are intentionally absent from the request and server-owned.
"""
adapter: str
required_capability: str
authorization_id: str
@classmethod
def from_mapping(
cls, value: Mapping[str, Any]
) -> "SupersessionChildProjectionIntent":
if not isinstance(value, Mapping) or set(value) != {
"adapter",
"required_capability",
"authorization_id",
}:
raise InitialExecutionProjectionError(
"SUPERSESSION_INITIAL_PROJECTION_FIELDS_INVALID", 400
)
adapter = _require_string(value["adapter"], "RUNTIME_ADAPTER_INVALID")
capability = _require_string(
value["required_capability"],
"RUNTIME_CAPABILITY_INVALID",
RUNTIME_CAPABILITY_RE,
)
if adapter not in {"hermes", "openclaw"} or capability != f"agent.runtime.{adapter}":
raise InitialExecutionProjectionError("RUNTIME_CAPABILITY_MISMATCH", 400)
authorization_id = _require_string(
value["authorization_id"], "AUTHORIZATION_ID_INVALID"
)
return cls(adapter, capability, authorization_id)
def public(self) -> dict[str, Any]:
return {
"schema": PROJECTION_INTENT_SCHEMA,
"source_task_id": SOURCE_TASK_ID,
"child_task_id": CHILD_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
"expected_status": "QUEUED",
"expected_revision": 1,
"expected_attempt": 0,
"expected_metadata_sha256": _sha_json({}),
"expected_required_capabilities_sha256": _sha_json([]),
"expected_job_id": EXTERNAL_JOB_ID,
"adapter": self.adapter,
"required_capability": self.required_capability,
"authorization_id": self.authorization_id,
"idempotency_key": PROJECTION_IDEMPOTENCY_KEY,
"reason": PROJECTION_REASON,
}
def initial_request(self, canonical_execution_digest: str) -> InitialExecutionProjectionRequest:
return InitialExecutionProjectionRequest.from_mapping(
{
"expected_status": "QUEUED",
"expected_revision": 1,
"expected_attempt": 0,
"expected_metadata_sha256": _sha_json({}),
"expected_required_capabilities_sha256": _sha_json([]),
"expected_job_id": EXTERNAL_JOB_ID,
"expected_execution_digest": canonical_execution_digest,
"required_capability": self.required_capability,
"adapter": self.adapter,
"authorization_id": self.authorization_id,
"idempotency_key": PROJECTION_IDEMPOTENCY_KEY,
"reason": PROJECTION_REASON,
}
)
def _table_names(connection: sqlite3.Connection) -> set[str]:
return {
str(row[0])
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table'"
)
}
def _load_lineage_context(
connection: sqlite3.Connection, *, expect_projected: bool
) -> dict[str, Any]:
required_tables = {
"tasks",
"events",
"task_lineage_supersessions",
"task_lineage_supersession_receipts",
PROJECTION_TABLE,
}
if not required_tables.issubset(_table_names(connection)):
raise InitialExecutionProjectionError(
"SUPERSESSION_INITIAL_PROJECTION_SCHEMA_UNAVAILABLE", 503
)
relation = connection.execute(
"SELECT * FROM task_lineage_supersessions "
"WHERE source_task_id=? AND supersession_ordinal=? AND child_task_id=?",
(SOURCE_TASK_ID, SUPERSESSION_ORDINAL, CHILD_TASK_ID),
).fetchone()
if relation is None:
raise InitialExecutionProjectionError("FORMAL_SUPERSESSION_REQUIRED", 409)
receipt_row = connection.execute(
"SELECT * FROM task_lineage_supersession_receipts "
"WHERE decision_id=? AND source_task_id=? AND supersession_ordinal=? "
"AND child_task_id=?",
(
relation["decision_id"],
SOURCE_TASK_ID,
SUPERSESSION_ORDINAL,
CHILD_TASK_ID,
),
).fetchone()
if receipt_row is None:
raise InitialExecutionProjectionError("SUPERSESSION_RECEIPT_REQUIRED", 409)
decision = _json_object(receipt_row["decision_json"], "SUPERSESSION_DECISION_INVALID")
receipt = _json_object(receipt_row["receipt_json"], "SUPERSESSION_RECEIPT_INVALID")
decision_sha256 = _sha_text(canonical_json(decision_payload(decision)))
exact_decision = {
"schema": DECISION_SCHEMA,
"source_task_id": SOURCE_TASK_ID,
"child_task_id": CHILD_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"external_job_id": EXTERNAL_JOB_ID,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
"owner_identity": OWNER_PRINCIPAL,
"signature_namespace": OWNER_NAMESPACE,
"signature_principal": OWNER_PRINCIPAL,
"key_id": OWNER_KEY_ID,
"one_time_use": True,
}
if any(decision.get(key) != expected for key, expected in exact_decision.items()):
raise InitialExecutionProjectionError(
"SUPERSESSION_DECISION_SCOPE_MISMATCH", 409
)
if (
decision_sha256 != relation["owner_decision_sha256"]
or decision_sha256 != receipt_row["owner_decision_sha256"]
or decision.get("source_task_identity_sha256")
!= relation["source_task_identity_sha256"]
or decision.get("source_metadata_sha256") != relation["source_metadata_sha256"]
or receipt.get("schema") != SUPERSESSION_RECEIPT_SCHEMA
or receipt.get("owner_decision_sha256") != decision_sha256
or receipt.get("source_task_id") != SOURCE_TASK_ID
or receipt.get("child_task_id") != CHILD_TASK_ID
or receipt.get("supersession_ordinal") != SUPERSESSION_ORDINAL
):
raise InitialExecutionProjectionError(
"SUPERSESSION_LINEAGE_EVIDENCE_MISMATCH", 409
)
events = connection.execute(
"SELECT event_type,payload_json FROM events WHERE task_id=? ORDER BY event_id",
(CHILD_TASK_ID,),
).fetchall()
expected_events = ["TASK_CREATED"]
if expect_projected:
expected_events.append("TASK_INITIAL_EXECUTION_PROJECTION_AUTHORIZED")
if [str(row["event_type"]) for row in events] != expected_events:
raise InitialExecutionProjectionError(
"SUPERSESSION_CHILD_EVENT_CHAIN_INVALID", 409
)
created = _json_object(events[0]["payload_json"], "SUPERSESSION_CREATED_EVENT_INVALID")
exact_created = {
"task_id": CHILD_TASK_ID,
"supersedes_task_id": SOURCE_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"source_task_identity_sha256": relation["source_task_identity_sha256"],
"source_metadata_sha256": relation["source_metadata_sha256"],
"owner_supersession_decision_sha256": decision_sha256,
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
}
if any(created.get(key) != expected for key, expected in exact_created.items()):
raise InitialExecutionProjectionError(
"SUPERSESSION_CREATED_EVENT_SCOPE_MISMATCH", 409
)
return {
"source_task_identity_sha256": relation["source_task_identity_sha256"],
"source_metadata_sha256": relation["source_metadata_sha256"],
"owner_decision_sha256": decision_sha256,
}
def _validate_canonical_child_execution(
resolved: Mapping[str, Any],
*,
intent: SupersessionChildProjectionIntent,
runtime_input: Mapping[str, Any],
) -> dict[str, Any]:
try:
binding = GovernedExecutionBinding.from_mapping(resolved)
execution = validate_execution_envelope(resolved, binding=binding)
except ExecutionProjectionError as exc:
raise InitialExecutionProjectionError(
"SUPERSESSION_CANONICAL_EXECUTION_INVALID", 409
) from exc
exact_binding = {
"task_id": CHILD_TASK_ID,
"job_id": EXTERNAL_JOB_ID,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
}
if any(execution.get(key) != expected for key, expected in exact_binding.items()):
raise InitialExecutionProjectionError(
"SUPERSESSION_CANONICAL_EXECUTION_BINDING_MISMATCH", 409
)
if (
execution.get("action") != "agent_runtime"
or execution.get("adapter") != intent.adapter
or execution.get("capability_scope") != [intent.required_capability]
or execution.get("runtime_input") != dict(runtime_input)
or execution.get("change_set") != []
or execution.get("verification") != []
or execution.get("governance", {}).get("writer_lease") is not False
):
raise InitialExecutionProjectionError(
"SUPERSESSION_AGENT_RUNTIME_ENVELOPE_INVALID", 409
)
if execution["execution_digest"] == SOURCE_BUSINESS_EXECUTION_DIGEST:
raise InitialExecutionProjectionError(
"CHILD_CANONICAL_DIGEST_COLLIDES_WITH_SOURCE_BUSINESS_DIGEST", 409
)
return execution
class _SupersessionProjectionTransactionStore(InitialExecutionProjectionStore):
"""Per-invocation store that adds immutable lineage evidence atomically."""
allows_supersession_child = True
def __init__(self, db_path, resolved: Mapping[str, Any], intent):
self._resolved = dict(resolved)
self.intent = intent
super().__init__(db_path, self._resolve_once)
def _resolve_once(self, *_args):
return dict(self._resolved)
def _projection_event_extension(
self, connection, task_id, request, execution, applied_revision
):
if task_id != CHILD_TASK_ID:
raise InitialExecutionProjectionError(
"SUPERSESSION_CHILD_TASK_ID_MISMATCH", 409
)
context = _load_lineage_context(connection, expect_projected=False)
return {
"source_task_id": SOURCE_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"source_task_identity_sha256": context["source_task_identity_sha256"],
"source_metadata_sha256": context["source_metadata_sha256"],
"owner_supersession_decision_sha256": context["owner_decision_sha256"],
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"canonical_child_execution_digest": execution["execution_digest"],
"canonical_projection_profile": CANONICAL_PROFILE,
"canonical_projection_version": CANONICAL_PROFILE_VERSION,
}
def _insert_projection_extension(
self,
connection,
task_id,
request,
execution,
event_payload,
applied_revision,
):
context = _load_lineage_context(connection, expect_projected=True)
task = connection.execute(
"SELECT metadata_json,required_capabilities_json FROM tasks WHERE task_id=?",
(CHILD_TASK_ID,),
).fetchone()
if task is None:
raise InitialExecutionProjectionError("TASK_NOT_FOUND", 404)
intent_json = canonical_json(self.intent.public())
intent_sha256 = _sha_text(intent_json)
receipt = {
"schema": PROJECTION_RECEIPT_SCHEMA,
"projection_id": "sha256:" + intent_sha256,
"task_id": CHILD_TASK_ID,
"source_task_id": SOURCE_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"source_task_identity_sha256": context["source_task_identity_sha256"],
"source_metadata_sha256": context["source_metadata_sha256"],
"owner_supersession_decision_sha256": context["owner_decision_sha256"],
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"canonical_child_execution_digest": execution["execution_digest"],
"external_job_id": EXTERNAL_JOB_ID,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"adapter": self.intent.adapter,
"required_capability": self.intent.required_capability,
"expected_revision": 1,
"applied_revision": applied_revision,
"attempt": 0,
"metadata_sha256": _sha_text(task["metadata_json"]),
"required_capabilities_sha256": _sha_text(
task["required_capabilities_json"]
),
"projection_event_sha256": _sha_json(dict(event_payload)),
}
receipt_json = canonical_json(receipt)
receipt_sha256 = _sha_text(receipt_json)
connection.execute(
f"""INSERT INTO {PROJECTION_TABLE}(
projection_id,task_id,source_task_id,supersession_ordinal,
intent_sha256,intent_json,owner_decision_sha256,
source_task_identity_sha256,source_metadata_sha256,
source_business_execution_digest,canonical_child_execution_digest,
external_job_id,repository,target_branch,generation,adapter,
required_capability,expected_revision,applied_revision,metadata_sha256,
required_capabilities_sha256,projection_event_sha256,receipt_json,
receipt_sha256,created_at)
VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)""",
(
receipt["projection_id"],
CHILD_TASK_ID,
SOURCE_TASK_ID,
SUPERSESSION_ORDINAL,
intent_sha256,
intent_json,
context["owner_decision_sha256"],
context["source_task_identity_sha256"],
context["source_metadata_sha256"],
SOURCE_BUSINESS_EXECUTION_DIGEST,
execution["execution_digest"],
EXTERNAL_JOB_ID,
REPOSITORY,
TARGET_BRANCH,
GENERATION,
self.intent.adapter,
self.intent.required_capability,
1,
applied_revision,
receipt["metadata_sha256"],
receipt["required_capabilities_sha256"],
receipt["projection_event_sha256"],
receipt_json,
receipt_sha256,
_now(),
),
)
class SupersessionChildInitialProjectionStore:
"""Server-owned adapter from exact supersession lineage to current ABI."""
def __init__(self, db_path, resolver):
self.db_path = db_path
self.resolver = resolver
def _connect(self):
connection = sqlite3.connect(self.db_path, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA foreign_keys=ON")
connection.execute("PRAGMA busy_timeout=30000")
return connection
@staticmethod
def _replay(connection, intent: SupersessionChildProjectionIntent):
if PROJECTION_TABLE not in _table_names(connection):
raise InitialExecutionProjectionError(
"SUPERSESSION_INITIAL_PROJECTION_SCHEMA_UNAVAILABLE", 503
)
row = connection.execute(
f"SELECT * FROM {PROJECTION_TABLE} WHERE task_id=?", (CHILD_TASK_ID,)
).fetchone()
if row is None:
tables = _table_names(connection)
if "task_initial_execution_projections" in tables:
generic = connection.execute(
"SELECT 1 FROM task_initial_execution_projections "
"WHERE task_id=? AND idempotency_key=?",
(CHILD_TASK_ID, PROJECTION_IDEMPOTENCY_KEY),
).fetchone()
if generic is not None:
raise InitialExecutionProjectionError(
"SUPERSESSION_PROJECTION_RECEIPT_MISSING", 500
)
return None
intent_sha256 = _sha_json(intent.public())
if row["intent_sha256"] != intent_sha256:
raise InitialExecutionProjectionError(
"SUPERSESSION_INITIAL_PROJECTION_INTENT_CONFLICT", 409
)
receipt = _json_object(
row["receipt_json"], "SUPERSESSION_PROJECTION_RECEIPT_INVALID"
)
if (
_sha_text(row["receipt_json"]) != row["receipt_sha256"]
or receipt.get("schema") != PROJECTION_RECEIPT_SCHEMA
or receipt.get("canonical_child_execution_digest")
!= row["canonical_child_execution_digest"]
):
raise InitialExecutionProjectionError(
"SUPERSESSION_PROJECTION_RECEIPT_DIGEST_MISMATCH", 500
)
_load_lineage_context(connection, expect_projected=True)
task = connection.execute(
"SELECT * FROM tasks WHERE task_id=?", (CHILD_TASK_ID,)
).fetchone()
if (
task is None
or task["status"] != "QUEUED"
or int(task["revision"]) != int(row["applied_revision"])
or int(task["attempt"]) != 0
or task["claimed_by"] is not None
or task["lease_token_hash"] is not None
or task["lease_expires_at"] is not None
or _sha_text(task["metadata_json"]) != row["metadata_sha256"]
or _sha_text(task["required_capabilities_json"])
!= row["required_capabilities_sha256"]
):
raise InitialExecutionProjectionError(
"SUPERSESSION_PROJECTED_TASK_CAS_MISMATCH", 409
)
return {**receipt, "idempotent": True}
def project(self, value: Mapping[str, Any]) -> dict[str, Any]:
intent = SupersessionChildProjectionIntent.from_mapping(value)
if intent.authorization_id != _required_authorization_id():
raise InitialExecutionProjectionError("AUTHORIZATION_ID_MISMATCH", 403)
with self._connect() as connection:
replay = self._replay(connection, intent)
if replay is not None:
return replay
probe_request = intent.initial_request("0" * 64)
probe_store = _SupersessionProjectionTransactionStore(
self.db_path, {}, intent
)
task, _, _, _ = probe_store._load_and_validate(
connection, CHILD_TASK_ID, probe_request
)
_load_lineage_context(connection, expect_projected=False)
runtime_input = probe_store._runtime_input(task)
resolved = dict(
self.resolver(
CHILD_TASK_ID,
EXTERNAL_JOB_ID,
intent.required_capability,
intent.adapter,
runtime_input,
)
)
execution = _validate_canonical_child_execution(
resolved, intent=intent, runtime_input=runtime_input
)
store = _SupersessionProjectionTransactionStore(
self.db_path, resolved, intent
)
result = store.project(
CHILD_TASK_ID, intent.initial_request(execution["execution_digest"])
)
with self._connect() as connection:
receipt = self._replay(connection, intent)
if receipt is None:
raise InitialExecutionProjectionError(
"SUPERSESSION_PROJECTION_RECEIPT_MISSING", 500
)
return {**receipt, "idempotent": bool(result["idempotent"])}
@@ -0,0 +1,221 @@
"""Fixed Owner operation for one exact Task-lineage supersession.
This staging overlay deliberately extends the installed single-host operational
authority instead of exposing a generic signing primitive. The caller can
only provide the three live CAS values which cannot be known at build time.
Every authority, lineage and execution field is constructed server-side.
"""
from __future__ import annotations
import re
import secrets
from typing import Any, Callable, Mapping
from aios_single_host_authority import OWNER_IDENTITY, OWNER_NAMESPACE, sha256_json
from aios_single_host_operational_authority import (
OperationalAuthorityError,
OperationalOwnerAuthority,
OperationalPolicy,
SignedOperationalArtifact,
)
TASK_LINEAGE_SUPERSESSION_OPERATION = "issue_task_lineage_supersession"
TASK_LINEAGE_SUPERSESSION_SCHEMA = "aios.task-lineage-supersession-decision.v1"
SOURCE_TASK_ID = "TASK-SINGLEHOST-TRUSTROOT-P0-20260904"
CHILD_TASK_ID = SOURCE_TASK_ID + "-S1"
SUPERSESSION_ORDINAL = 1
REPOSITORY = "aios/aios"
TARGET_BRANCH = "agent/single-host-trust-root-p0-20260904"
GENERATION = 1
SOURCE_BUSINESS_EXECUTION_DIGEST = (
"e99bda6a251832eb3865271eb5687af7eea44dd261144cdcdfa57104c0780536"
)
EXTERNAL_JOB_IDENTITY = "job-4e10f92a6a5742518a9f1d8745c845ab"
CANONICAL_PROJECTION_PROFILE = "aios.task-initial-execution.v1"
CANONICAL_PROJECTION_VERSION = 1
SUPERSESSION_REASON = "formal-lineage-supersession-after-expired-unclaimed-source"
DECISION_TTL_SECONDS = 300
HEX64 = re.compile(r"^[0-9a-f]{64}$")
SAFE_TOKEN = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/+:-]{2,511}$")
def _fail(code: str) -> None:
raise OperationalAuthorityError(code)
def _assert_fixed_policy(policy: OperationalPolicy) -> None:
"""Refuse issuance if the live policy is not the approved source scope."""
policy.validate()
exact = {
"task_id": SOURCE_TASK_ID,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"execution_plan_sha256": SOURCE_BUSINESS_EXECUTION_DIGEST,
"external_job_identity": EXTERNAL_JOB_IDENTITY,
}
if any(getattr(policy, field) != expected for field, expected in exact.items()):
_fail("TASK_LINEAGE_SUPERSESSION_POLICY_MISMATCH")
if policy.maximum_recovery_decision_ttl_seconds < DECISION_TTL_SECONDS:
_fail("TASK_LINEAGE_SUPERSESSION_POLICY_TTL_TOO_NARROW")
TASK_LINEAGE_SUPERSESSION_FIELDS = {
"schema",
"decision_id",
"nonce",
"source_task_id",
"source_task_identity_sha256",
"source_metadata_sha256",
"source_expected_status",
"source_expected_revision",
"source_expected_attempt",
"source_expected_claimed_by",
"source_expected_lease_token_hash",
"source_expected_lease_expires_at",
"repository",
"target_branch",
"generation",
"source_business_execution_digest",
"external_job_identity",
"child_task_id",
"supersession_ordinal",
"canonical_projection_profile",
"canonical_projection_version",
"supersession_reason",
"owner_identity",
"policy_digest",
"one_time_use",
"issued_at",
"not_before",
"expires_at",
"signature_namespace",
"signature_key_id",
"canonical_payload_digest",
}
def verify_task_lineage_supersession_scope(
decision: Mapping[str, Any],
policy: OperationalPolicy,
*,
now: int,
) -> None:
"""Validate a signature-verified decision against the fixed source scope."""
_assert_fixed_policy(policy)
if not isinstance(decision, Mapping) or set(decision) != TASK_LINEAGE_SUPERSESSION_FIELDS:
_fail("TASK_LINEAGE_SUPERSESSION_FIELDS_INVALID")
exact = {
"schema": TASK_LINEAGE_SUPERSESSION_SCHEMA,
"source_task_id": SOURCE_TASK_ID,
"source_expected_status": "EXPIRED",
"source_expected_attempt": 0,
"source_expected_claimed_by": None,
"source_expected_lease_token_hash": None,
"source_expected_lease_expires_at": None,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"external_job_identity": EXTERNAL_JOB_IDENTITY,
"child_task_id": CHILD_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"canonical_projection_profile": CANONICAL_PROJECTION_PROFILE,
"canonical_projection_version": CANONICAL_PROJECTION_VERSION,
"supersession_reason": SUPERSESSION_REASON,
"owner_identity": OWNER_IDENTITY,
"policy_digest": policy.digest,
"one_time_use": True,
"signature_namespace": OWNER_NAMESPACE,
}
if any(decision.get(field) != expected for field, expected in exact.items()):
_fail("TASK_LINEAGE_SUPERSESSION_SCOPE_INVALID")
if decision.get("source_task_identity_sha256") != policy.task_identity_sha256:
_fail("TASK_LINEAGE_SUPERSESSION_SOURCE_IDENTITY_INVALID")
for field in ("source_task_identity_sha256", "source_metadata_sha256"):
if not isinstance(decision.get(field), str) or HEX64.fullmatch(decision[field]) is None:
_fail("TASK_LINEAGE_SUPERSESSION_DIGEST_INVALID")
if type(decision.get("source_expected_revision")) is not int or not 1 <= decision["source_expected_revision"] < 2**63:
_fail("TASK_LINEAGE_SUPERSESSION_REVISION_INVALID")
for field in ("decision_id", "nonce"):
if not isinstance(decision.get(field), str) or SAFE_TOKEN.fullmatch(decision[field]) is None:
_fail("TASK_LINEAGE_SUPERSESSION_GENERATED_ID_INVALID")
for field in ("issued_at", "not_before", "expires_at"):
if type(decision.get(field)) is not int:
_fail("TASK_LINEAGE_SUPERSESSION_TIME_INVALID")
if not decision["not_before"] <= now < decision["expires_at"]:
_fail("TASK_LINEAGE_SUPERSESSION_TIME_INVALID")
if decision["issued_at"] > now + 5 or now - decision["issued_at"] > DECISION_TTL_SECONDS:
_fail("TASK_LINEAGE_SUPERSESSION_TIME_INVALID")
if decision["expires_at"] - decision["issued_at"] != DECISION_TTL_SECONDS:
_fail("TASK_LINEAGE_SUPERSESSION_TIME_INVALID")
digest_value = dict(decision)
supplied_digest = digest_value.pop("canonical_payload_digest", None)
if supplied_digest != sha256_json(digest_value):
_fail("TASK_LINEAGE_SUPERSESSION_CANONICAL_DIGEST_INVALID")
class TaskLineageSupersessionOwnerAuthority(OperationalOwnerAuthority):
"""Owner signer with one bounded constructor and no caller-defined message."""
def __init__(
self,
*args: Any,
token_hex: Callable[[int], str] = secrets.token_hex,
**kwargs: Any,
) -> None:
super().__init__(*args, **kwargs)
self._token_hex = token_hex
def issue_task_lineage_supersession(
self,
*,
source_expected_revision: int,
source_task_identity_sha256: str,
source_metadata_sha256: str,
) -> SignedOperationalArtifact:
_assert_fixed_policy(self.policy)
if type(source_expected_revision) is not int or not 1 <= source_expected_revision < 2**63:
_fail("TASK_LINEAGE_SUPERSESSION_REVISION_INVALID")
if source_task_identity_sha256 != self.policy.task_identity_sha256:
_fail("TASK_LINEAGE_SUPERSESSION_SOURCE_IDENTITY_INVALID")
if not isinstance(source_metadata_sha256, str) or HEX64.fullmatch(source_metadata_sha256) is None:
_fail("TASK_LINEAGE_SUPERSESSION_METADATA_DIGEST_INVALID")
decision_id = "task-lineage-supersession-" + self._token_hex(16)
nonce = "task-lineage-supersession-nonce-" + self._token_hex(16)
if SAFE_TOKEN.fullmatch(decision_id) is None or SAFE_TOKEN.fullmatch(nonce) is None:
_fail("TASK_LINEAGE_SUPERSESSION_GENERATED_ID_INVALID")
now = int(self.clock())
payload = {
"schema": TASK_LINEAGE_SUPERSESSION_SCHEMA,
"decision_id": decision_id,
"nonce": nonce,
"source_task_id": SOURCE_TASK_ID,
"source_task_identity_sha256": source_task_identity_sha256,
"source_metadata_sha256": source_metadata_sha256,
"source_expected_status": "EXPIRED",
"source_expected_revision": source_expected_revision,
"source_expected_attempt": 0,
"source_expected_claimed_by": None,
"source_expected_lease_token_hash": None,
"source_expected_lease_expires_at": None,
"repository": REPOSITORY,
"target_branch": TARGET_BRANCH,
"generation": GENERATION,
"source_business_execution_digest": SOURCE_BUSINESS_EXECUTION_DIGEST,
"external_job_identity": EXTERNAL_JOB_IDENTITY,
"child_task_id": CHILD_TASK_ID,
"supersession_ordinal": SUPERSESSION_ORDINAL,
"canonical_projection_profile": CANONICAL_PROJECTION_PROFILE,
"canonical_projection_version": CANONICAL_PROJECTION_VERSION,
"supersession_reason": SUPERSESSION_REASON,
"owner_identity": OWNER_IDENTITY,
"policy_digest": self.policy.digest,
"one_time_use": True,
"issued_at": now,
"not_before": max(0, now - 5),
"expires_at": now + DECISION_TTL_SECONDS,
}
return self._sign(payload)
@@ -0,0 +1,190 @@
"""Staging-only server overlay for the fixed lineage Owner operation."""
from __future__ import annotations
import argparse
import hashlib
import json
from pathlib import Path
import sqlite3
import time
from typing import Any, Callable, Mapping
from aios_single_host_authority import DEFAULT_REGISTRY_PATH
from aios_single_host_operational_authority import (
OperationalMachineAuthority,
OperationalVerifier,
)
from aios_single_host_operational_authority_server import (
DEFAULT_LEDGER,
DEFAULT_MACHINE_KEY,
DEFAULT_OWNER_KEY,
DEFAULT_POLICY,
DurableIssuanceLedger,
OperationalAuthorityService as BaseOperationalAuthorityService,
_read_policy,
fail,
)
from aios_task_lineage_supersession_owner_authority import (
TASK_LINEAGE_SUPERSESSION_OPERATION,
TaskLineageSupersessionOwnerAuthority,
)
REQUEST_FIELDS = {
"source_expected_revision",
"source_task_identity_sha256",
"source_metadata_sha256",
}
class TaskLineageSupersessionIssuanceLedger(DurableIssuanceLedger):
"""Operation-scoped one-time fence layered on the existing durable ledger."""
def issue_once(
self,
*,
role: str,
operation: str,
request: Mapping[str, Any],
issuer: Callable[[], dict[str, Any]],
) -> dict[str, Any]:
request_sha = hashlib.sha256(
json.dumps(
request,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
).encode("utf-8")
).hexdigest()
connection = sqlite3.connect(self.path, timeout=5, isolation_level=None)
try:
connection.execute("BEGIN IMMEDIATE")
existing = connection.execute(
"SELECT 1 FROM issuance WHERE role=? AND operation=? LIMIT 1",
(role, operation),
).fetchone()
if existing is not None:
connection.rollback()
fail("TASK_LINEAGE_SUPERSESSION_ALREADY_ISSUED")
result = issuer()
artifact = result.get("artifact")
nonce = artifact.get("payload", {}).get("nonce") if isinstance(artifact, Mapping) else None
if not isinstance(nonce, str) or not nonce:
connection.rollback()
fail("TASK_LINEAGE_SUPERSESSION_ISSUER_RESULT_INVALID")
encoded = json.dumps(
result,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
allow_nan=False,
)
connection.execute(
"INSERT INTO issuance(role,operation,nonce,request_sha256,response_json,created_at) "
"VALUES(?,?,?,?,?,?)",
(role, operation, nonce, request_sha, encoded, int(time.time())),
)
connection.commit()
return result
except sqlite3.IntegrityError:
connection.rollback()
fail("ISSUANCE_LEDGER_RACE")
except Exception:
connection.rollback()
raise
finally:
connection.close()
class OperationalAuthorityService(BaseOperationalAuthorityService):
"""Preserve all existing operations and add one exact Owner-only operation."""
def __init__(self, *args: Any, token_hex=None, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self.ledger = TaskLineageSupersessionIssuanceLedger(self.ledger.path)
self._token_hex = token_hex
def _components(self):
policy = _read_policy(
self.policy_path,
require_root_owner=self.require_root_owner,
)
verifier = OperationalVerifier(
self.registry_path,
require_root_owner=self.require_root_owner,
)
if self.role == "owner":
kwargs = {}
if self._token_hex is not None:
kwargs["token_hex"] = self._token_hex
signer = TaskLineageSupersessionOwnerAuthority(
self.owner_key_path,
policy,
require_root_owner=self.require_root_owner,
clock=self.clock,
**kwargs,
)
else:
signer = OperationalMachineAuthority(
self.machine_key_path,
policy,
require_root_owner=self.require_root_owner,
clock=self.clock,
)
return policy, verifier, signer
def handle(self, request: Mapping[str, Any], *, peer_uid: int) -> dict[str, Any]:
if peer_uid not in self.allowed_uids:
fail("PEER_UID_FORBIDDEN")
if not isinstance(request, Mapping) or set(request) != {"operation", "payload"}:
fail("REQUEST_INVALID")
if request.get("operation") != TASK_LINEAGE_SUPERSESSION_OPERATION:
return super().handle(request, peer_uid=peer_uid)
if self.role != "owner":
fail("MACHINE_OPERATION_FORBIDDEN")
payload = request.get("payload")
if not isinstance(payload, Mapping) or set(payload) != REQUEST_FIELDS:
fail("REQUEST_FIELDS_INVALID")
_policy, _verifier, signer = self._components()
def issue() -> dict[str, Any]:
artifact = signer.issue_task_lineage_supersession(
source_expected_revision=payload["source_expected_revision"],
source_task_identity_sha256=payload["source_task_identity_sha256"],
source_metadata_sha256=payload["source_metadata_sha256"],
)
return {"status": "PASS", "artifact": artifact.to_dict()}
return self.ledger.issue_once(
role=self.role,
operation=TASK_LINEAGE_SUPERSESSION_OPERATION,
request=payload,
issuer=issue,
)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--role", choices=("owner", "machine"), required=True)
parser.add_argument("--policy", default=str(DEFAULT_POLICY))
parser.add_argument("--registry", default=str(DEFAULT_REGISTRY_PATH))
parser.add_argument("--owner-key", default=str(DEFAULT_OWNER_KEY))
parser.add_argument("--machine-key", default=str(DEFAULT_MACHINE_KEY))
parser.add_argument("--socket")
parser.add_argument("--ledger", default=str(DEFAULT_LEDGER))
arguments = parser.parse_args()
service = OperationalAuthorityService(
role=arguments.role,
policy_path=Path(arguments.policy),
registry_path=Path(arguments.registry),
owner_key_path=Path(arguments.owner_key),
machine_key_path=Path(arguments.machine_key),
socket_path=Path(arguments.socket) if arguments.socket else None,
ledger_path=Path(arguments.ledger),
)
service.serve_forever()
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -15,7 +15,7 @@ import struct
import subprocess
import time
from datetime import datetime, timezone
from dataclasses import dataclass
from dataclasses import dataclass, replace
from pathlib import Path
from typing import Any, Callable, Mapping, Protocol
@@ -133,6 +133,8 @@ class TrustedCandidateBuilderConfig:
recipes: Mapping[str, Mapping[str, str]]
governed_recipes: Mapping[str, Mapping[str, str]]
policy_sha256: str
source_admission_resolver: Callable[[str, str | None, str], Mapping[str, str]] | None = None
migration_plan_resolver: Callable[[str, str, str], Any] | None = None
@classmethod
def from_policy(cls, policy: Mapping[str, Any]) -> "TrustedCandidateBuilderConfig":
@@ -227,14 +229,63 @@ class TrustedCandidateBuilderConfig:
return recipe
def resolve_recipe(self, version: str, candidate_id: str | None, source_commit: str) -> Mapping[str, str]:
recipe = self.fixed_recipe(version)
# Intent-only callers do not choose a candidate identity. The
# policy-owned recipe supplies it; explicit worker requests are
# still checked against the same immutable recipe.
effective_candidate_id = str(candidate_id or recipe.get("candidate_id") or "")
if {"version": recipe.get("version"), "candidate_id": recipe.get("candidate_id"), "source_commit": recipe.get("source_commit")} != {"version": version, "candidate_id": effective_candidate_id, "source_commit": source_commit}:
recipe = self.recipes.get(version)
if recipe is not None:
resolved = self.fixed_recipe(version)
# Fixed/governed-derived policy recipes remain immutable. A
# source admission can never override one of these identities.
effective_candidate_id = str(candidate_id or resolved.get("candidate_id") or "")
if {
"version": resolved.get("version"),
"candidate_id": resolved.get("candidate_id"),
"source_commit": resolved.get("source_commit"),
} != {
"version": version,
"candidate_id": effective_candidate_id,
"source_commit": source_commit,
}:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return resolved
resolver = self.source_admission_resolver
if resolver is None:
raise BuilderError("RECIPE_UNKNOWN")
try:
admitted = dict(resolver(version, candidate_id, source_commit))
except Exception as exc:
raise BuilderError("SOURCE_ADMISSION_REQUIRED") from exc
required = {
"mode",
"task_id",
"candidate_id",
"base_oid",
"source_commit",
"version",
"purpose",
}
if (
set(admitted) != required
or admitted.get("mode") != "governed-source-admission"
or admitted.get("version") != version
or admitted.get("source_commit") != source_commit
or not isinstance(admitted.get("candidate_id"), str)
or not admitted.get("candidate_id")
):
raise BuilderError("SOURCE_ADMISSION_INVALID")
effective_candidate_id = str(candidate_id or admitted["candidate_id"])
if effective_candidate_id != admitted["candidate_id"]:
raise BuilderError("REQUEST_RELEASE_BINDING_INVALID")
return recipe
for name in ("base_oid", "source_commit"):
value = admitted.get(name)
if (
not isinstance(value, str)
or len(value) != 40
or any(char not in "0123456789abcdef" for char in value.lower())
):
raise BuilderError("SOURCE_ADMISSION_INVALID")
if admitted.get("purpose") != "AIOS_TRUSTED_RELEASE_BUILDER":
raise BuilderError("SOURCE_ADMISSION_INVALID")
return admitted
def _invalid_governed_recipes() -> Mapping[str, Mapping[str, str]]:
@@ -297,6 +348,35 @@ class ServerOwnedEvidenceSource:
raise BuilderError("COLLECTOR_EVIDENCE_OWNER_DENIED")
return resolved
def assert_ready(self) -> None:
"""Validate the server-owned evidence facility without requiring recipe evidence.
Readiness is an infrastructure/property check. Candidate-specific evidence
remains mandatory for ``BuildTrustedCandidateV1`` and is validated by
:meth:`__call__`; an empty evidence root therefore no longer makes the
builder globally unavailable.
"""
for path, code in (
(self._authority_git_dir, "COLLECTOR_AUTHORITY_GIT_UNAVAILABLE"),
(self._evidence_root, "COLLECTOR_EVIDENCE_ROOT_UNAVAILABLE"),
):
try:
metadata = path.stat(follow_symlinks=False)
except OSError as exc:
raise BuilderError(code) from exc
if not stat.S_ISDIR(metadata.st_mode) or path.is_symlink():
raise BuilderError(code)
if path == self._evidence_root:
if os.name != "nt" and metadata.st_mode & 0o022:
raise BuilderError("COLLECTOR_EVIDENCE_ROOT_MODE_DENIED")
if (
os.name != "nt"
and self._expected_owner_uid is not None
and metadata.st_uid != self._expected_owner_uid
):
raise BuilderError("COLLECTOR_EVIDENCE_ROOT_OWNER_DENIED")
def _digest(self, path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
@@ -436,9 +516,17 @@ class DeterministicEvidenceCollector:
self._config = config
self._evidence_source = evidence_source
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
def assert_ready(self) -> None:
"""Fail closed on collector configuration, not on recipe evidence presence."""
if self._evidence_source is None:
raise BuilderError("COLLECTOR_UNCONFIGURED")
checker = getattr(self._evidence_source, "assert_ready", None)
if checker is not None:
checker()
def collect(self, recipe_name: str, recipe: Mapping[str, str]) -> Mapping[str, Any]:
self.assert_ready()
bundle = self._evidence_source(recipe_name, recipe)
if not isinstance(bundle, Mapping) or set(bundle) != self._BUNDLE_FIELDS:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID")
@@ -464,11 +552,32 @@ class DeterministicEvidenceCollector:
try:
from silent_release_controller import ReleaseCandidate
from silent_release_hardening import ReleasePlan, TrustedCandidateBundle
from release_migration_plan import (
MigrationPlanError,
MigrationPlanRequired,
)
candidate = ReleaseCandidate.from_dict(dict(bundle["candidate"]))
release_plan = ReleasePlan.from_dict(dict(bundle["release_plan"]))
release_plan_payload = dict(bundle["release_plan"])
migration_resolver = self._config.migration_plan_resolver
if migration_resolver is not None:
try:
migration = migration_resolver(
recipe_name,
recipe["candidate_id"],
recipe["source_commit"],
)
except MigrationPlanRequired as exc:
raise BuilderError("MIGRATION_PLAN_REQUIRED") from exc
except MigrationPlanError as exc:
raise BuilderError("MIGRATION_PLAN_INVALID") from exc
if migration is not None:
release_plan_payload["migration"] = migration.to_dict()
release_plan = ReleasePlan.from_dict(release_plan_payload)
checks = dict(raw_checks)
TrustedCandidateBundle(candidate, release_plan, checks).validate()
except BuilderError:
raise
except (ImportError, KeyError, TypeError, ValueError, RuntimeError) as exc:
raise BuilderError("COLLECTOR_EVIDENCE_INVALID") from exc
if candidate.source_commit != recipe["source_commit"] or candidate.candidate_id != recipe["candidate_id"]:
@@ -622,14 +731,12 @@ class TrustedCandidateBuilder:
bound = {"repository": self.config.repository, "task_id": recipe["task_id"], "candidate_id": recipe["candidate_id"], "base_oid": recipe["base_oid"], "source_commit": recipe["source_commit"], "version": recipe["version"], "purpose": recipe["purpose"]}
if {key: request.get(key) for key in bound} != bound:
raise BuilderError("REQUEST_SCOPE_INVALID")
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
if request["operation"] == self.config.readiness_operation:
self.collector.assert_ready()
# Accessing the identity properties validates that the signer is
# configured and usable without invoking the private key.
builder_key_id = self.signer.key_id
builder_key_generation = self.signer.key_generation
result = {
"schema": self.config.response_schema,
"status": "READY",
@@ -637,10 +744,10 @@ class TrustedCandidateBuilder:
"recipe": recipe_name,
"candidate_id": recipe["candidate_id"],
"source_commit": recipe["source_commit"],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"builder_key_id": builder_key_id,
"builder_key_generation": builder_key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "builder_attestation",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"policy_sha256": self.config.policy_sha256,
"collector_ready": True,
"signer_ready": True,
@@ -653,13 +760,22 @@ class TrustedCandidateBuilder:
)
self._persist_nonce_ledger()
return result
bundle = dict(self.collector.collect(recipe_name, recipe))
candidate = bundle["candidate"]
release_plan = bundle["release_plan"]
checks = bundle.get("checks")
evidence_digests = bundle["evidence_digests"]
if not isinstance(checks, Mapping) or set(checks) != set(self.config.required_checks) or not all(value is True for value in checks.values()):
raise BuilderError("COLLECTOR_CHECKS_INCOMPLETE")
issued = datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
attestation = {
"builder_attestation_id": "builder-attestation-" + _sha256(recipe_name, recipe["task_id"], candidate["artifact_sha256"])[:24],
"builder_key_id": self.signer.key_id,
"builder_key_generation": self.signer.key_generation,
"signature_algorithm": "Ed25519",
"signature_purpose": "builder_attestation",
"signature_purpose": "AIOS_TRUSTED_RELEASE_BUILDER",
"source_commit": candidate["source_commit"], "tree_oid": candidate["tree_oid"],
"artifact_sha256": candidate["artifact_sha256"], "manifest_sha256": candidate["manifest_sha256"],
"package_closure_sha256": release_plan["deployment"]["package_closure_sha256"],
@@ -673,7 +789,7 @@ class TrustedCandidateBuilder:
}
canonical_attestation = _canonical_json(attestation)
fingerprint = hashlib.sha256(canonical_attestation).hexdigest()
signature = base64.b64encode(self.signer.sign_for_purpose("builder_attestation", canonical_attestation)).decode("ascii")
signature = base64.b64encode(self.signer.sign_for_purpose("AIOS_TRUSTED_RELEASE_BUILDER", canonical_attestation)).decode("ascii")
result = {
"schema": self.config.response_schema, "status": "BUILT", "recipe": recipe_name,
"repository": self.config.repository, "task_id": recipe["task_id"], "base_oid": recipe["base_oid"],
@@ -776,11 +892,55 @@ def _main() -> None:
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
keyring = BuilderKeyring.from_credentials_directory(Path(credentials))
source_admission_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_SOURCE_ADMISSION_ROOT", ""
).strip()
if source_admission_root:
from release_source_admission import open_source_admission_store
try:
source_admission_hmac = (
Path(credentials) / "source_admission_hmac"
).read_bytes().strip()
except OSError as exc:
raise BuilderError("SOURCE_ADMISSION_HMAC_UNAVAILABLE") from exc
source_store = open_source_admission_store(
Path(source_admission_root),
repository=config.repository,
policy=arguments.policy,
hmac_key=source_admission_hmac,
expected_owner_uid=0,
writable=False,
)
config = replace(
config,
source_admission_resolver=source_store.resolve_recipe,
)
migration_plan_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_MIGRATION_PLAN_ROOT", ""
).strip()
migration_profile_root = os.environ.get(
"AIOS_TRUSTED_BUILDER_MIGRATION_PROFILE_ROOT", ""
).strip()
if bool(migration_plan_root) != bool(migration_profile_root):
raise BuilderError("MIGRATION_PLAN_CONFIGURATION_INVALID")
if migration_plan_root:
from release_migration_plan import FileMigrationPlanResolver
migration_resolver = FileMigrationPlanResolver(
Path(migration_plan_root),
profile_root=Path(migration_profile_root),
expected_owner_uid=0,
)
config = replace(
config,
migration_plan_resolver=migration_resolver.resolve,
)
signer = CredentialPathEd25519Signer(
keyring.signing_private_key_path,
key_id=os.environ["AIOS_TRUSTED_BUILDER_KEY_ID"],
key_generation=int(os.environ["AIOS_TRUSTED_BUILDER_KEY_GENERATION"]),
allowed_purpose="builder_attestation",
allowed_purpose="AIOS_TRUSTED_RELEASE_BUILDER",
)
listener = socket.fromfd(arguments.socket_fd, socket.AF_UNIX, socket.SOCK_STREAM)
collector = DeterministicEvidenceCollector(
@@ -1,462 +0,0 @@
"""Self-healing availability guard for persistent Online Agent workers.
The guard never claims Tasks and never writes Task/Job/Git databases. It only:
- ensures required worker/executor systemd units are enabled + active;
- checks worker-owned liveness JSON written by the real poll/heartbeat loop;
- restarts an active worker only when its liveness is stale;
- honors an explicit maintenance inhibit file;
- writes a bounded status JSON under /run for observability.
Idle liveness is the successful poll loop itself. Busy liveness is refreshed by the
Task lease-heartbeat thread, so long-running Tasks are not mistaken for hung workers.
"""
from __future__ import annotations
from dataclasses import asdict, dataclass
import argparse
import json
import os
from pathlib import Path
import shlex
import subprocess
import tempfile
import time
from typing import Any, Callable, Mapping, Sequence
SCHEMA = "aios.worker-availability.status.v1"
DEFAULT_WORKER_UNITS = (
"aios-online-agent-worker-20260815.service",
"aios-hermes-online-agent-worker.service",
"aios-openclaw-online-agent-worker.service",
)
DEFAULT_DEPENDENCY_UNITS = (
"aios-external-agent-executor.service",
)
DEFAULT_RUN_ROOT = Path("/run/aios-worker-availability")
DEFAULT_STATUS = DEFAULT_RUN_ROOT / "status.json"
DEFAULT_RECOVERY = DEFAULT_RUN_ROOT / "recovery.json"
DEFAULT_INHIBIT = DEFAULT_RUN_ROOT / "maintenance.lock"
DEFAULT_MAX_STALE_SECONDS = 90
DEFAULT_RESTART_COOLDOWN_SECONDS = 60
DEFAULT_STARTUP_WAIT_SECONDS = 10
ACCEPTED_ENABLED_STATES = frozenset({"enabled", "enabled-runtime", "static", "indirect", "generated"})
class AvailabilityGuardError(RuntimeError):
pass
@dataclass(frozen=True)
class UnitSnapshot:
unit: str
role: str
load_state: str
active_state: str
unit_file_state: str
enabled_ok: bool
liveness_path: str | None = None
liveness_state: str | None = None
last_seen_epoch: float | None = None
liveness_age_seconds: float | None = None
fresh: bool | None = None
action: str = "NONE"
detail: str | None = None
class Systemd:
def __init__(
self,
runner: Callable[..., subprocess.CompletedProcess[str]] | None = None,
) -> None:
self.runner = runner or subprocess.run
def _run(self, args: Sequence[str], *, check: bool = False) -> subprocess.CompletedProcess[str]:
result = self.runner(
["/usr/bin/systemctl", *args],
text=True,
capture_output=True,
check=False,
timeout=20,
)
if check and result.returncode != 0:
raise AvailabilityGuardError(
"SYSTEMD_COMMAND_FAILED:"
+ ":".join(args)
+ ":"
+ str(result.returncode)
)
return result
def property(self, unit: str, name: str) -> str:
result = self._run(["show", unit, f"--property={name}", "--value"])
if result.returncode != 0:
return ""
return result.stdout.strip()
def environment(self, unit: str) -> dict[str, str]:
text = self.property(unit, "Environment")
result: dict[str, str] = {}
for item in shlex.split(text):
if "=" not in item:
continue
key, value = item.split("=", 1)
result[key] = value
return result
def enable(self, unit: str) -> None:
self._run(["enable", unit], check=True)
def start(self, unit: str) -> None:
self._run(["start", unit], check=True)
def restart(self, unit: str) -> None:
self._run(["restart", unit], check=True)
def _atomic_json(path: Path, value: Mapping[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o755)
if path.parent.is_symlink() or path.is_symlink():
raise AvailabilityGuardError("STATUS_PATH_SYMLINK_REJECTED")
fd, temporary = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
try:
os.fchmod(fd, 0o644)
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(value, handle, ensure_ascii=False, sort_keys=True, indent=2)
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
os.chmod(path, 0o644)
finally:
if os.path.exists(temporary):
os.unlink(temporary)
def _load_json(path: Path) -> dict[str, Any] | None:
if path.is_symlink() or not path.is_file():
return None
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
return dict(value) if isinstance(value, Mapping) else None
def _state_root(environment: Mapping[str, str], unit: str) -> Path:
explicit = environment.get("AIOS_WORKER_STATE_ROOT")
if explicit:
return Path(explicit)
fallbacks = {
"aios-online-agent-worker-20260815.service": Path(
"/var/lib/aios-online-agent-worker-20260815"
),
"aios-hermes-online-agent-worker.service": Path(
"/var/lib/aios-online-agent-hermes"
),
"aios-openclaw-online-agent-worker.service": Path(
"/var/lib/aios-online-agent-openclaw"
),
}
if unit not in fallbacks:
raise AvailabilityGuardError("WORKER_STATE_ROOT_UNAVAILABLE:" + unit)
return fallbacks[unit]
def _liveness_path(environment: Mapping[str, str], unit: str) -> Path:
explicit = environment.get("AIOS_WORKER_LIVENESS_PATH")
if explicit:
return Path(explicit)
return _state_root(environment, unit) / "worker-liveness.json"
class AvailabilityGuard:
def __init__(
self,
*,
systemd: Systemd | None = None,
worker_units: Sequence[str] = DEFAULT_WORKER_UNITS,
dependency_units: Sequence[str] = DEFAULT_DEPENDENCY_UNITS,
status_path: Path = DEFAULT_STATUS,
recovery_path: Path = DEFAULT_RECOVERY,
inhibit_path: Path = DEFAULT_INHIBIT,
max_stale_seconds: int = DEFAULT_MAX_STALE_SECONDS,
restart_cooldown_seconds: int = DEFAULT_RESTART_COOLDOWN_SECONDS,
startup_wait_seconds: int = DEFAULT_STARTUP_WAIT_SECONDS,
now_fn: Callable[[], float] = time.time,
sleep_fn: Callable[[float], None] = time.sleep,
) -> None:
self.systemd = systemd or Systemd()
self.worker_units = tuple(worker_units)
self.dependency_units = tuple(dependency_units)
self.status_path = Path(status_path)
self.recovery_path = Path(recovery_path)
self.inhibit_path = Path(inhibit_path)
self.max_stale_seconds = int(max_stale_seconds)
self.restart_cooldown_seconds = int(restart_cooldown_seconds)
self.startup_wait_seconds = int(startup_wait_seconds)
self.now_fn = now_fn
self.sleep_fn = sleep_fn
if self.max_stale_seconds < 15:
raise AvailabilityGuardError("MAX_STALE_SECONDS_TOO_SMALL")
if self.restart_cooldown_seconds < 15:
raise AvailabilityGuardError("RESTART_COOLDOWN_TOO_SMALL")
if self.startup_wait_seconds < 0 or self.startup_wait_seconds > 60:
raise AvailabilityGuardError("STARTUP_WAIT_INVALID")
def _recovery_state(self) -> dict[str, float]:
value = _load_json(self.recovery_path) or {}
raw = value.get("last_restart_epoch")
if not isinstance(raw, Mapping):
return {}
result: dict[str, float] = {}
for unit, epoch in raw.items():
try:
result[str(unit)] = float(epoch)
except (TypeError, ValueError):
continue
return result
def _save_recovery_state(self, state: Mapping[str, float]) -> None:
_atomic_json(
self.recovery_path,
{
"schema": "aios.worker-availability.recovery.v1",
"last_restart_epoch": dict(state),
},
)
def _basic_snapshot(self, unit: str, role: str) -> tuple[str, str, str, bool]:
load = self.systemd.property(unit, "LoadState")
active = self.systemd.property(unit, "ActiveState")
enabled = self.systemd.property(unit, "UnitFileState")
return load, active, enabled, enabled in ACCEPTED_ENABLED_STATES
def _ensure_enabled_active(self, unit: str, role: str) -> tuple[str, bool]:
load, active, enabled, enabled_ok = self._basic_snapshot(unit, role)
if load != "loaded":
raise AvailabilityGuardError("REQUIRED_UNIT_NOT_LOADED:" + unit)
action = "NONE"
if not enabled_ok:
self.systemd.enable(unit)
action = "ENABLE"
if active != "active":
self.systemd.start(unit)
action = "START" if action == "NONE" else "ENABLE_START"
return action, active == "active"
def _wait_for_liveness(self, path: Path) -> dict[str, Any] | None:
deadline = self.now_fn() + self.startup_wait_seconds
while True:
value = _load_json(path)
if value is not None:
try:
epoch = float(value.get("last_seen_epoch"))
except (TypeError, ValueError):
epoch = 0.0
if epoch > 0 and self.now_fn() - epoch <= self.max_stale_seconds:
return value
if self.now_fn() >= deadline:
return value
self.sleep_fn(0.5)
def run(self, *, heal: bool) -> dict[str, Any]:
now = self.now_fn()
if self.inhibit_path.exists():
result = {
"schema": SCHEMA,
"status": "MAINTENANCE_INHIBITED",
"generated_at_epoch": now,
"maintenance_inhibit": str(self.inhibit_path),
"workers": [],
"dependencies": [],
"all_fresh": False,
"actions": [],
}
_atomic_json(self.status_path, result)
return result
actions: list[dict[str, str]] = []
dependencies: list[UnitSnapshot] = []
for unit in self.dependency_units:
load, active, enabled, enabled_ok = self._basic_snapshot(unit, "dependency")
action = "NONE"
if heal:
if load != "loaded":
raise AvailabilityGuardError("REQUIRED_UNIT_NOT_LOADED:" + unit)
if not enabled_ok:
self.systemd.enable(unit)
action = "ENABLE"
if active != "active":
self.systemd.start(unit)
action = "START" if action == "NONE" else "ENABLE_START"
if action != "NONE":
actions.append({"unit": unit, "action": action})
load, active, enabled, enabled_ok = self._basic_snapshot(unit, "dependency")
dependencies.append(
UnitSnapshot(
unit=unit,
role="dependency",
load_state=load,
active_state=active,
unit_file_state=enabled,
enabled_ok=enabled_ok,
fresh=(active == "active"),
action=action,
)
)
recovery = self._recovery_state()
workers: list[UnitSnapshot] = []
for unit in self.worker_units:
load, active, enabled, enabled_ok = self._basic_snapshot(unit, "worker")
environment = self.systemd.environment(unit) if load == "loaded" else {}
path = _liveness_path(environment, unit) if load == "loaded" else None
action = "NONE"
if heal:
if load != "loaded":
raise AvailabilityGuardError("REQUIRED_UNIT_NOT_LOADED:" + unit)
if not enabled_ok:
self.systemd.enable(unit)
action = "ENABLE"
if active != "active":
self.systemd.start(unit)
action = "START" if action == "NONE" else "ENABLE_START"
actions.append({"unit": unit, "action": action})
if path is not None:
self._wait_for_liveness(path)
load, active, enabled, enabled_ok = self._basic_snapshot(unit, "worker")
live = _load_json(path) if path is not None else None
last_seen: float | None = None
state: str | None = None
age: float | None = None
fresh = False
detail: str | None = None
if live is not None:
state = str(live.get("state") or "")
try:
last_seen = float(live.get("last_seen_epoch"))
except (TypeError, ValueError):
last_seen = None
if last_seen is not None:
age = max(0.0, now - last_seen)
fresh = age <= self.max_stale_seconds
if live.get("agent_id") in (None, ""):
fresh = False
detail = "LIVENESS_AGENT_ID_MISSING"
else:
detail = "LIVENESS_MISSING"
if heal and active == "active" and not fresh:
previous = float(recovery.get(unit, 0.0))
if now - previous >= self.restart_cooldown_seconds:
self.systemd.restart(unit)
recovery[unit] = now
action = "RESTART_STALE"
actions.append({"unit": unit, "action": action})
if path is not None:
live = self._wait_for_liveness(path)
load, active, enabled, enabled_ok = self._basic_snapshot(unit, "worker")
if live is not None:
state = str(live.get("state") or "")
try:
last_seen = float(live.get("last_seen_epoch"))
except (TypeError, ValueError):
last_seen = None
if last_seen is not None:
age = max(0.0, self.now_fn() - last_seen)
fresh = age <= self.max_stale_seconds
detail = None if fresh else "LIVENESS_STALE_AFTER_RESTART"
else:
detail = "RESTART_COOLDOWN_ACTIVE"
workers.append(
UnitSnapshot(
unit=unit,
role="worker",
load_state=load,
active_state=active,
unit_file_state=enabled,
enabled_ok=enabled_ok,
liveness_path=str(path) if path is not None else None,
liveness_state=state,
last_seen_epoch=last_seen,
liveness_age_seconds=age,
fresh=bool(active == "active" and enabled_ok and fresh),
action=action,
detail=detail,
)
)
if heal:
self._save_recovery_state(recovery)
all_fresh = (
all(item.active_state == "active" and item.enabled_ok and item.fresh for item in workers)
and all(item.active_state == "active" for item in dependencies)
)
result = {
"schema": SCHEMA,
"status": "HEALTHY" if all_fresh else "DEGRADED",
"generated_at_epoch": self.now_fn(),
"max_stale_seconds": self.max_stale_seconds,
"maintenance_inhibit": str(self.inhibit_path),
"workers": [asdict(item) for item in workers],
"dependencies": [asdict(item) for item in dependencies],
"all_fresh": all_fresh,
"actions": actions,
}
_atomic_json(self.status_path, result)
return result
def main() -> int:
parser = argparse.ArgumentParser()
mode = parser.add_mutually_exclusive_group(required=True)
mode.add_argument("--heal", action="store_true")
mode.add_argument("--check", action="store_true")
parser.add_argument("--status-file", default=str(DEFAULT_STATUS))
parser.add_argument("--recovery-file", default=str(DEFAULT_RECOVERY))
parser.add_argument("--inhibit-file", default=str(DEFAULT_INHIBIT))
parser.add_argument(
"--max-stale-seconds",
type=int,
default=int(
os.environ.get(
"AIOS_WORKER_AVAILABILITY_MAX_STALE_SECONDS",
str(DEFAULT_MAX_STALE_SECONDS),
)
),
)
parser.add_argument(
"--restart-cooldown-seconds",
type=int,
default=int(
os.environ.get(
"AIOS_WORKER_AVAILABILITY_RESTART_COOLDOWN_SECONDS",
str(DEFAULT_RESTART_COOLDOWN_SECONDS),
)
),
)
args = parser.parse_args()
guard = AvailabilityGuard(
status_path=Path(args.status_file),
recovery_path=Path(args.recovery_file),
inhibit_path=Path(args.inhibit_file),
max_stale_seconds=args.max_stale_seconds,
restart_cooldown_seconds=args.restart_cooldown_seconds,
)
try:
result = guard.run(heal=args.heal)
except AvailabilityGuardError as exc:
print(json.dumps({"status": "FAILED", "error": str(exc)}, sort_keys=True))
return 20
print(json.dumps(result, ensure_ascii=False, sort_keys=True))
if args.check and not result.get("all_fresh"):
return 10
return 0
if __name__ == "__main__":
raise SystemExit(main())
+56 -21
View File
@@ -31,6 +31,8 @@ import time
from pathlib import Path
from typing import Any, Mapping, Sequence
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -73,6 +75,24 @@ class Binding:
production_port: int
repository_identity: str = "aios/aios"
candidate_id: str | None = None
release_plan_sha256: str | None = None
@classmethod
def from_trusted_plan(cls, plan: TrustedReleasePlan) -> "Binding":
value = plan.value
adapter = value["adapter_policy"]
binding = cls(
version=str(value["version"]), target_oid=str(value["source_commit"]),
evidence_closure_oid=str(value["evidence_closure_oid"]), artifact_sha256=str(value["artifact_sha256"]),
approval_plan_id=str(value["approval_plan_id"]), artifact_path=str(value["artifact"]["path"]),
artifact_size=int(value["artifact"]["size"]), production_service=str(adapter["service"]),
production_port=int(adapter["port"]), candidate_id=str(value["candidate_id"]),
release_plan_sha256=plan.release_plan_sha256,
)
plan.assert_binding(dataclasses.asdict(binding))
return binding
def __post_init__(self) -> None:
if not self.version or not OID_RE.fullmatch(self.target_oid):
raise ValueError("invalid release target binding")
@@ -107,6 +127,8 @@ class Binding:
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
return hashlib.sha256(canonical_json(payload)).hexdigest()
@@ -126,9 +148,12 @@ class Binding:
"production_service": self.production_service,
"production_port": self.production_port,
"repository_identity": self.repository_identity,
"candidate_id": self.candidate_id,
"release_plan_sha256": self.release_plan_sha256,
}
def canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
@@ -282,6 +307,12 @@ def verify_authorization(receipt: Mapping[str, Any], binding: Binding) -> dict[s
"artifact_sha256": binding.artifact_sha256,
"approval_plan_id": binding.approval_plan_id,
}
if binding.candidate_id is not None:
expected["candidate_id"] = binding.candidate_id
if binding.release_plan_sha256 is not None:
expected["release_plan_sha256"] = binding.release_plan_sha256
if receipt.get("candidate_binding_sha256") is None:
raise ManualOwnerReviewRequired("Owner authorization missing candidate binding")
for key, value in expected.items():
if receipt.get(key) != value:
raise ManualOwnerReviewRequired(f"Owner authorization mismatch: {key}")
@@ -498,7 +529,14 @@ def execute(
deploy_command: Sequence[str],
registry_command: Sequence[str],
baseline_command: Sequence[str],
release_plan_path: Path | None = None,
) -> dict[str, Any]:
plan = TrustedReleasePlan.load(release_plan_path) if release_plan_path is not None else None
if plan is not None:
try:
plan.assert_binding(dataclasses.asdict(binding))
except PlanError as exc:
raise ReleaseBlocked(f"trusted release plan mismatch: {exc}") from exc
authorization = verify_authorization(secure_json(authorization_path), binding)
lineage = verify_git_lineage(repository, binding)
artifact = verify_artifact(binding)
@@ -591,17 +629,18 @@ def execute(
def parser() -> argparse.ArgumentParser:
p = argparse.ArgumentParser()
p.add_argument("--authorization", required=True, type=Path)
p.add_argument("--release-plan", type=Path)
p.add_argument("--repository", required=True, type=Path)
p.add_argument("--state-root", required=True, type=Path)
p.add_argument("--artifact", required=True, type=Path)
p.add_argument("--artifact-size", required=True, type=int)
p.add_argument("--version", required=True)
p.add_argument("--target-oid", required=True)
p.add_argument("--evidence-closure-oid", required=True)
p.add_argument("--artifact-sha256", required=True)
p.add_argument("--approval-plan-id", required=True)
p.add_argument("--production-service", required=True)
p.add_argument("--production-port", required=True, type=int)
p.add_argument("--artifact", type=Path)
p.add_argument("--artifact-size", type=int)
p.add_argument("--version")
p.add_argument("--target-oid")
p.add_argument("--evidence-closure-oid")
p.add_argument("--artifact-sha256")
p.add_argument("--approval-plan-id")
p.add_argument("--production-service")
p.add_argument("--production-port", type=int)
p.add_argument("--repository-identity", default="aios/aios")
p.add_argument("--tag-adapter", required=True, nargs="+")
p.add_argument("--deploy-adapter", required=True, nargs="+")
@@ -612,18 +651,13 @@ def parser() -> argparse.ArgumentParser:
def main() -> int:
args = parser().parse_args()
binding = Binding(
version=args.version,
target_oid=args.target_oid,
evidence_closure_oid=args.evidence_closure_oid,
artifact_sha256=args.artifact_sha256,
approval_plan_id=args.approval_plan_id,
artifact_path=str(args.artifact),
artifact_size=args.artifact_size,
production_service=args.production_service,
production_port=args.production_port,
repository_identity=args.repository_identity,
)
plan = TrustedReleasePlan.load(args.release_plan) if args.release_plan else None
if plan is not None:
binding = Binding.from_trusted_plan(plan)
else:
legacy=(args.artifact,args.artifact_size,args.version,args.target_oid,args.evidence_closure_oid,args.artifact_sha256,args.approval_plan_id,args.production_service,args.production_port)
if any(value is None for value in legacy): raise SystemExit("legacy binding is incomplete")
binding = Binding(version=args.version,target_oid=args.target_oid,evidence_closure_oid=args.evidence_closure_oid,artifact_sha256=args.artifact_sha256,approval_plan_id=args.approval_plan_id,artifact_path=str(args.artifact),artifact_size=args.artifact_size,production_service=args.production_service,production_port=args.production_port,repository_identity=args.repository_identity)
try:
result = execute(
binding=binding,
@@ -634,6 +668,7 @@ def main() -> int:
deploy_command=parse_command(args.deploy_adapter),
registry_command=parse_command(args.registry_adapter),
baseline_command=parse_command(args.baseline_adapter),
release_plan_path=args.release_plan,
)
print(json.dumps(result, sort_keys=True))
return 0
+65 -2
View File
@@ -4,6 +4,7 @@ from __future__ import annotations
import os
import re
import subprocess
import time
import urllib.parse
from pathlib import Path
from typing import Iterable
@@ -306,14 +307,76 @@ def authoritative_diff(
from_oid = validate_git_revision(from_revision, "from", git_dir=path)
to_oid = validate_git_revision(to_revision, "to", git_dir=path)
args = ["diff", "--no-ext-diff", "--no-textconv", from_oid, to_oid]
result = run_authoritative_git(args, git_dir=path)
patch_limit = max(1, min(_MAX_OUTPUT, 256 * 1024))
process = subprocess.Popen(
["git", "--git-dir", str(path), *args],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env={**os.environ, "GIT_OPTIONAL_LOCKS": "0"},
)
chunks: list[bytes] = []
stored_bytes = 0
total_bytes = 0
started = time.monotonic()
try:
while process.stdout is not None:
chunk = process.stdout.read(64 * 1024)
if not chunk:
break
total_bytes += len(chunk)
if stored_bytes < patch_limit:
remaining = patch_limit - stored_bytes
selected = chunk[:remaining]
chunks.append(selected)
stored_bytes += len(selected)
if time.monotonic() - started > _TIMEOUT:
process.kill()
raise AuthoritativeGitError("GIT_REF_QUERY_TIMEOUT")
stderr = (
process.stderr.read(8192).decode("utf-8", errors="replace")
if process.stderr is not None
else ""
)
returncode = process.wait(timeout=max(0.1, _TIMEOUT - (time.monotonic() - started)))
except (OSError, subprocess.TimeoutExpired) as exc:
process.kill()
process.wait()
raise AuthoritativeGitError("GIT_REF_QUERY_FAILED") from exc
if returncode != 0:
raise AuthoritativeGitError(_redact(stderr.strip() or "GIT_REF_QUERY_FAILED")[:200])
try:
names_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--name-status", from_oid, to_oid],
git_dir=path,
)
changed_paths = [
line.split("\t", 1)[-1]
for line in names_result.stdout.splitlines()
if line.strip()
]
except AuthoritativeGitOutputTooLarge:
changed_paths = []
try:
stats_result = run_authoritative_git(
["diff", "--no-ext-diff", "--no-textconv", "--stat", from_oid, to_oid],
git_dir=path,
)
stats = stats_result.stdout.strip()
except AuthoritativeGitOutputTooLarge:
stats = "GIT_STAT_OUTPUT_TRUNCATED"
patch = b"".join(chunks).decode("utf-8", errors="replace")
return {
"code": 0,
"stdout": result.stdout,
"stdout": patch,
"stderr": "",
"cmd": ["git", f"--git-dir={path}", *args],
"from_oid": from_oid,
"to_oid": to_oid,
"changed_paths": changed_paths,
"stats": stats,
"total_size_bytes": total_bytes,
"truncated": total_bytes > patch_limit,
"patch_limit_bytes": patch_limit,
}
@@ -964,7 +964,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
required = {
"schema",
"task_id",
"job_id",
"repository",
"branch_mode",
"base_oid",
@@ -977,7 +976,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
raise ProvisioningError("LEASE_REQUEST_V2_FIELDS_REJECTED", 409)
request = WriterLeaseRequestV2(
task_id=str(raw["task_id"]),
job_id=str(raw["job_id"]),
repository=str(raw["repository"]),
branch_mode=str(raw["branch_mode"]),
base_oid=str(raw["base_oid"]),
@@ -990,7 +988,6 @@ def validate_writer_lease_request_v2(raw: Mapping[str, Any], probe: Any) -> dict
return {
"schema": WriterLeaseRequestV2.schema,
"task_id": request.task_id,
"job_id": request.job_id,
"repository": request.repository,
"branch_mode": request.branch_mode,
"base_oid": request.base_oid,
@@ -12,6 +12,7 @@ import tempfile
from typing import Any, Mapping
from release_campaign_authorization import CampaignAuthorization, CampaignError
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
class AcceptanceError(RuntimeError):
@@ -61,6 +62,7 @@ def evaluate(
staging: Mapping[str, Any],
main: Mapping[str, Any],
tag: Mapping[str, Any],
release_plan: Mapping[str, Any] | None = None,
) -> dict[str, Any]:
required_binding = {
"version",
@@ -98,8 +100,27 @@ def evaluate(
raise AcceptanceError("rollback_readiness_failed")
if tag.get("conflict") is not False:
raise AcceptanceError("tag_conflict")
trusted_plan: TrustedReleasePlan | None = None
if release_plan is not None:
try:
trusted_plan = TrustedReleasePlan.from_mapping(release_plan)
trusted_plan.assert_binding(binding)
except PlanError as exc:
raise AcceptanceError(f"trusted_release_plan_invalid:{exc}") from exc
if trusted_plan.value["campaign_authorization_sha256"] != campaign.digest:
raise AcceptanceError("campaign_authorization_digest_mismatch")
if int(trusted_plan.value["campaign_generation"]) != int(campaign.data["generation"]):
raise AcceptanceError("campaign_generation_mismatch")
if main.get("tree_oid") not in (None, trusted_plan.value["tree_oid"]):
raise AcceptanceError("tree_oid_mismatch")
if evidence.get("release_plan_sha256") not in (None, trusted_plan.release_plan_sha256):
raise AcceptanceError("evidence_release_plan_mismatch")
if artifact.get("manifest_sha256") not in (None, trusted_plan.value["manifest_sha256"]):
raise AcceptanceError("manifest_digest_mismatch")
if artifact.get("package_closure_sha256") not in (None, trusted_plan.value["package_closure_sha256"]):
raise AcceptanceError("package_closure_digest_mismatch")
campaign.assert_operation("derive_campaign_acceptance")
return {
result = {
"schema": "aios.campaign-release-acceptance.v1",
"accepted": True,
"binding_replaced": False,
@@ -108,6 +129,22 @@ def evaluate(
**dict(binding),
"owner_interaction_required": False,
}
if trusted_plan is not None:
result.update({
"candidate_id": trusted_plan.value["candidate_id"],
"release_plan_sha256": trusted_plan.release_plan_sha256,
"candidate_binding_sha256": trusted_plan.candidate_binding_sha256,
"campaign_authorization_sha256": trusted_plan.value["campaign_authorization_sha256"],
"campaign_generation": trusted_plan.value["campaign_generation"],
"tree_oid": trusted_plan.value["tree_oid"],
"manifest_sha256": trusted_plan.value["manifest_sha256"],
"package_closure_sha256": trusted_plan.value["package_closure_sha256"],
"deployment_plan_sha256": trusted_plan.value["deployment_plan_sha256"],
"policy_sha256": trusted_plan.value["policy_sha256"],
"rollback": trusted_plan.value["rollback"],
"baseline": trusted_plan.value["baseline"],
})
return result
def main() -> int:
@@ -120,6 +157,7 @@ def main() -> int:
p.add_argument("--staging", type=Path, required=True)
p.add_argument("--main-facts", type=Path, required=True)
p.add_argument("--tag-facts", type=Path, required=True)
p.add_argument("--release-plan", type=Path)
p.add_argument("--output", type=Path, required=True)
a = p.parse_args()
@@ -136,6 +174,7 @@ def main() -> int:
secure_json(a.staging),
secure_json(a.main_facts),
secure_json(a.tag_facts),
secure_json(a.release_plan) if a.release_plan else None,
)
atomic_new(a.output, receipt)
print(json.dumps(receipt, sort_keys=True))
+16 -7
View File
@@ -8,6 +8,7 @@ import json
import os
import re
import subprocess
import sys
import tempfile
import time
import urllib.error
@@ -23,10 +24,15 @@ try:
from authoritative_git import AuthoritativeGitError, run_authoritative_git
except ModuleNotFoundError:
from .authoritative_git import AuthoritativeGitError, run_authoritative_git
try:
from release_version import RELEASE_VERSION_RE, validate_release_version
except ModuleNotFoundError:
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
from release_version import RELEASE_VERSION_RE, validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
VERSION_RE = re.compile(r"^V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?$")
VERSION_RE = RELEASE_VERSION_RE
DEPLOYMENT_EVIDENCE_RE = re.compile(r"^(?:sha256:[0-9a-f]{64}|git:[0-9a-f]{40})$")
FINGERPRINT_FIELDS = (
"version",
@@ -83,9 +89,14 @@ class PublicationManualOwnerReviewRequired(
def validate_version(version: str) -> str:
value = str(version or "")
if not VERSION_RE.fullmatch(value):
raise ValueError("invalid release version")
return validate_release_version(version)
def validate_gitea_api_url(api_url: str) -> str:
value = str(api_url or "").strip().rstrip("/")
parsed = urllib.parse.urlsplit(value)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise ValueError("Gitea API URL must be HTTP(S)")
return value
@@ -411,13 +422,11 @@ class GiteaTagClient:
timeout: float = 15,
token_line: int = 1,
):
self.api_url = api_url.rstrip("/")
self.api_url = validate_gitea_api_url(api_url)
self.repository = repository.strip("/")
self.token_file = token_file
self.timeout = timeout
self.token_line = token_line
if not self.api_url.startswith(("https://", "http://")):
raise ValueError("Gitea API URL must be HTTP(S)")
if self.repository.count("/") != 1:
raise ValueError("Gitea repository must be owner/name")
@@ -0,0 +1,349 @@
from __future__ import annotations
"""Authenticated Gitea consistency provider using the real governed Git transport.
The provider proves Gitea consistency without inventing a REST API:
* local HEAD == source_oid,
* local source_oid^{tree} == tree_oid,
* origin identifies the governed repository,
* exact checked-out branch exists remotely, and
* git ls-remote reports exactly source_oid for that branch.
It then writes an append-only root-owned v2 provider receipt using only governed
policy for provider identity, operation, store root, store id and modes.
The result explicitly records ``transport=git-ls-remote`` and
``api_verified=false``. It must never be described as REST/API evidence.
"""
from dataclasses import dataclass
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
import stat
import subprocess
from typing import Callable, Mapping
from urllib.parse import urlsplit, urlunsplit
RECEIPT_SCHEMA = "aios.trusted-evidence-provider-receipt.v2"
RECEIPT_VERSION = 2
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_REF_RE = re.compile(r"^refs/heads/[A-Za-z0-9._/-]+$")
_REPO_RE = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$")
class GiteaProviderError(RuntimeError):
pass
@dataclass(frozen=True)
class GitCommandResult:
returncode: int
stdout: str = ""
stderr: str = ""
GitRunner = Callable[[tuple[str, ...], Path], GitCommandResult]
def canonical_json_bytes(payload: Mapping) -> bytes:
return json.dumps(
dict(payload),
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
).encode("utf-8")
def canonical_sha256(payload: Mapping) -> str:
return hashlib.sha256(canonical_json_bytes(payload)).hexdigest()
def _runner(args: tuple[str, ...], cwd: Path) -> GitCommandResult:
proc = subprocess.run(
list(args),
cwd=str(cwd),
text=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
timeout=45,
check=False,
)
return GitCommandResult(proc.returncode, proc.stdout, proc.stderr)
def _git(run: GitRunner, cwd: Path, *args: str, error: str) -> str:
result = run(("git", *args), cwd)
if result.returncode != 0:
raise GiteaProviderError(f"{error}: {result.stderr.strip()[:300]}")
return result.stdout.strip()
def _oid(value: str, field: str) -> str:
value = str(value).strip().lower()
if not _OID_RE.fullmatch(value):
raise GiteaProviderError(f"{field.upper()}_INVALID")
return value
def _mode(value) -> int:
if isinstance(value, int):
return value
text = str(value).strip()
if text.isdigit():
return int(text, 8)
raise GiteaProviderError("PROVIDER_MODE_INVALID")
def _safe_ref(value: str) -> Path:
raw = str(value).strip()
p = Path(raw)
if not raw or p.is_absolute() or any(part in ("", ".", "..") for part in p.parts):
raise GiteaProviderError("PROVIDER_RECEIPT_REF_INVALID")
return p
def _safe_remote_url(url: str) -> str:
url = str(url).strip()
if "://" in url:
parts = urlsplit(url)
host = parts.hostname or ""
port = f":{parts.port}" if parts.port else ""
return urlunsplit((parts.scheme, host + port, parts.path, "", ""))
if ":" in url and not url.startswith(("/", "./", "../")):
left, right = url.split(":", 1)
return f"{left.rsplit('@', 1)[-1]}:{right}"
return url
def repository_identity_from_remote(url: str) -> str:
safe = _safe_remote_url(url)
if "://" in safe:
path = urlsplit(safe).path
elif ":" in safe and not safe.startswith("/"):
path = safe.split(":", 1)[1]
else:
path = safe
parts = [x for x in path.strip("/").split("/") if x]
if parts and parts[0] == "git":
parts = parts[1:]
if len(parts) < 2:
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_UNAVAILABLE")
owner, repo = parts[-2], parts[-1]
if repo.endswith(".git"):
repo = repo[:-4]
identity = f"{owner}/{repo}".lower()
if not _REPO_RE.fullmatch(identity):
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_INVALID")
return identity
def load_gitea_policy(policy_path: str | Path) -> dict:
raw = json.loads(Path(policy_path).read_text(encoding="utf-8"))
defaults = dict(raw.get("defaults") or {})
providers = raw.get("providers")
if not isinstance(providers, dict) or not isinstance(providers.get("gitea"), dict):
raise GiteaProviderError("GITEA_PROVIDER_POLICY_MISSING")
cfg = dict(providers["gitea"])
return {
"provider_identity": str(cfg["provider_identity"]),
"store_id": str(cfg["store_id"]),
"store_root": Path(str(cfg["store_root"])),
"provider_operation": str(cfg["allowed_operations"][0]),
"expected_owner_uid": int(cfg.get("expected_owner_uid", defaults.get("expected_owner_uid", 0))),
"directory_mode": min(
_mode(x) for x in cfg.get(
"allowed_directory_modes",
defaults.get("allowed_directory_modes", ["0700", "0750"]),
)
),
"receipt_mode": min(
_mode(x) for x in cfg.get(
"allowed_receipt_modes",
defaults.get("allowed_receipt_modes", ["0600", "0640"]),
)
),
"authentication_kind": str(
cfg.get(
"authentication_kind",
defaults.get("authentication_kind", "root-owned-store-v1"),
)
),
}
def collect_gitea_transport_consistency(
*,
repo_root: str | Path,
source_oid: str,
tree_oid: str,
expected_repository: str = "aios/aios",
remote: str = "origin",
git_runner: GitRunner | None = None,
) -> dict:
repo_root = Path(repo_root).resolve()
if not repo_root.is_dir():
raise GiteaProviderError("GITEA_CHECKOUT_UNAVAILABLE")
source_oid = _oid(source_oid, "source_oid")
tree_oid = _oid(tree_oid, "tree_oid")
expected_repository = str(expected_repository).strip().lower()
if not _REPO_RE.fullmatch(expected_repository):
raise GiteaProviderError("GITEA_EXPECTED_REPOSITORY_INVALID")
if not re.fullmatch(r"[A-Za-z0-9._/-]+", remote) or remote.startswith(("/", "-")):
raise GiteaProviderError("GITEA_REMOTE_NAME_INVALID")
run = git_runner or _runner
if _git(run, repo_root, "rev-parse", "--is-inside-work-tree",
error="GITEA_LOCAL_REPOSITORY_REQUIRED").lower() != "true":
raise GiteaProviderError("GITEA_LOCAL_REPOSITORY_REQUIRED")
actual_source = _oid(
_git(run, repo_root, "rev-parse", "HEAD^{commit}",
error="GITEA_LOCAL_SOURCE_UNAVAILABLE"),
"actual_source_oid",
)
if actual_source != source_oid:
raise GiteaProviderError("GITEA_LOCAL_SOURCE_OID_MISMATCH")
actual_tree = _oid(
_git(run, repo_root, "rev-parse", f"{source_oid}^{{tree}}",
error="GITEA_LOCAL_TREE_UNAVAILABLE"),
"actual_tree_oid",
)
if actual_tree != tree_oid:
raise GiteaProviderError("GITEA_LOCAL_TREE_OID_MISMATCH")
branch = _git(
run, repo_root, "symbolic-ref", "--quiet", "HEAD",
error="GITEA_GOVERNED_BRANCH_REQUIRED",
)
if not _REF_RE.fullmatch(branch):
raise GiteaProviderError("GITEA_GOVERNED_BRANCH_INVALID")
remote_url = _git(
run, repo_root, "remote", "get-url", remote,
error="GITEA_REMOTE_UNAVAILABLE",
)
identity = repository_identity_from_remote(remote_url)
if identity != expected_repository:
raise GiteaProviderError("GITEA_REPOSITORY_IDENTITY_MISMATCH")
output = _git(
run, repo_root, "ls-remote", "--exit-code", remote, branch,
error="GITEA_REMOTE_REF_UNAVAILABLE",
)
rows = []
for line in output.splitlines():
parts = line.split()
if len(parts) >= 2 and parts[1] == branch:
rows.append((parts[0].lower(), parts[1]))
if len(rows) != 1:
raise GiteaProviderError("GITEA_REMOTE_REF_AMBIGUOUS")
remote_oid = _oid(rows[0][0], "remote_ref_oid")
if remote_oid != source_oid:
raise GiteaProviderError("GITEA_REMOTE_REF_OID_MISMATCH")
return {
"consistent": True,
"repository": identity,
"source_commit": source_oid,
"remote_ref_oid": remote_oid,
"tree_oid": tree_oid,
"remote_ref": branch,
"remote_url": _safe_remote_url(remote_url),
"transport": "git-ls-remote",
"api_verified": False,
}
def _ensure_root_directory(path: Path, mode: int, expected_uid: int) -> None:
if os.geteuid() != expected_uid:
raise GiteaProviderError("GITEA_PROVIDER_ROOT_REQUIRED")
path.mkdir(parents=True, exist_ok=True, mode=mode)
st = os.lstat(path)
if stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode):
raise GiteaProviderError("GITEA_PROVIDER_STORE_INVALID")
if st.st_uid != expected_uid:
raise GiteaProviderError("GITEA_PROVIDER_STORE_OWNER_INVALID")
if stat.S_IMODE(st.st_mode) & 0o022:
raise GiteaProviderError("GITEA_PROVIDER_STORE_MODE_INVALID")
def _write_receipt_exclusive(path: Path, payload: Mapping, mode: int) -> str:
data = canonical_json_bytes(payload)
if path.exists() or path.is_symlink():
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_EXISTS")
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
flags |= getattr(os, "O_NOFOLLOW", 0)
flags |= getattr(os, "O_BINARY", 0)
fd = os.open(path, flags, mode)
try:
view = memoryview(data)
while view:
written = os.write(fd, view)
if written <= 0:
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_WRITE_FAILED")
view = view[written:]
os.fsync(fd)
finally:
os.close(fd)
os.chmod(path, mode)
return hashlib.sha256(data).hexdigest()
def produce_gitea_consistency_receipt(
*,
repo_root: str | Path,
source_oid: str,
tree_oid: str,
policy_path: str | Path,
receipt_ref: str,
expected_repository: str = "aios/aios",
) -> tuple[str, dict]:
cfg = load_gitea_policy(policy_path)
if cfg["authentication_kind"] != "root-owned-store-v1":
raise GiteaProviderError("GITEA_PROVIDER_AUTH_KIND_UNSUPPORTED")
result = collect_gitea_transport_consistency(
repo_root=repo_root,
source_oid=source_oid,
tree_oid=tree_oid,
expected_repository=expected_repository,
)
now = datetime.now(timezone.utc).isoformat()
run_id = f"gitea-git-transport-{source_oid[:12]}-{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}"
rel = _safe_ref(receipt_ref)
target = cfg["store_root"].joinpath(rel)
root = cfg["store_root"].resolve()
if not str(target.resolve(strict=False)).startswith(str(root) + os.sep):
raise GiteaProviderError("GITEA_PROVIDER_RECEIPT_REF_ESCAPE")
_ensure_root_directory(root, cfg["directory_mode"], cfg["expected_owner_uid"])
parent = root
for part in rel.parts[:-1]:
parent = parent / part
_ensure_root_directory(parent, cfg["directory_mode"], cfg["expected_owner_uid"])
payload = {
"schema": RECEIPT_SCHEMA,
"version": RECEIPT_VERSION,
"provider_identity": cfg["provider_identity"],
"provider_operation": cfg["provider_operation"],
"operation_run_id": run_id,
"result_reference": f"git:{result['remote_url']}#{result['remote_ref']}",
"content_sha256": canonical_sha256(result),
"source_oid": _oid(source_oid, "source_oid"),
"tree_oid": _oid(tree_oid, "tree_oid"),
"environment": "hw-isolated-linux-git-transport",
"issued_at": now,
"result": result,
"authentication": {
"kind": "root-owned-store-v1",
"store_id": cfg["store_id"],
},
}
sha = _write_receipt_exclusive(target, payload, cfg["receipt_mode"])
return sha, payload
@@ -0,0 +1,58 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
from governed_provider_auth_adapter import (
ALL_RELEASE_EVIDENCE_KINDS,
authenticate_build_artifacts,
gate_w_authentication_status,
)
from trusted_provider_receipts import (
ProviderAuthenticationError,
TrustedProviderReceiptResolver,
)
def produce_authenticated_evidence_closure(
request: Any,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
source_root: Path,
builder_policy: Path,
legacy_producer=None,
):
"""Production wrapper for the existing evidence producer.
This is the governed operation entrypoint. It refuses to call the legacy
closure assembler until *all* final-release provider receipts are resolved
from authenticated stores. For Gate-W, use `validate_gate_w_provider_auth`
instead; canary/rollback remain pending until Gate-R.
"""
if provider_receipt_resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
if legacy_producer is None:
from governed_release_evidence_producer import produce_evidence_closure as legacy_producer
authenticated_request = authenticate_build_artifacts(
request,
resolver=provider_receipt_resolver,
required_kinds=ALL_RELEASE_EVIDENCE_KINDS,
)
return legacy_producer(
authenticated_request,
source_root=source_root,
builder_policy=builder_policy,
)
def validate_gate_w_provider_auth(
request: Any,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Gate-W authentication check without pretending Gate-R evidence exists."""
return gate_w_authentication_status(
request,
resolver=provider_receipt_resolver,
)
@@ -0,0 +1,323 @@
#!/usr/bin/env python3
"""Governed evidence runtime registration chain (V2R2C2 P0).
Establishes the REAL runtime chain for the governed release evidence producer:
Inventory -> Resolver -> operation -> Producer -> closure
The Producer alone (governed_release_evidence_producer) is not "runtime
registered" just because its OperationDefinition exists — registration only
holds when a runtime can actually discover the operation in an inventory,
resolve its ``handler_ref`` to the real callable, and invoke it to produce a
closure. That full path is what this module wires and what the
``PRODUCER_RUNTIME_REGISTERED_PASS`` verdict proves.
- Inventory : EvidenceOperationInventory — the set of governed evidence
operations the authority can execute (unions the producer
registry; extensible to future governed operations).
- Resolver : resolve_handler(handler_ref) maps the operation's declared
handler reference to the actual ``produce_evidence_closure``
callable. Resolution is exact and restrictive — it never
imports by arbitrary string eval.
- Producer : the resolved callable executes with validated, receipt-bound
build artifacts and materializes the evidence closure.
- closure : the produced source_root/<version>/ closure.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any, Callable, Iterable, Mapping
from aios_baseline.operations import OperationDefinition, OperationRegistry
from governed_authenticated_evidence_operation import (
produce_authenticated_evidence_closure,
)
from governed_provider_auth_adapter import gate_w_authentication_status
from governed_release_evidence_producer import (
EvidenceProduceError,
GovernedBuildArtifacts,
ProduceResult,
TRUSTED_PROVIDER_EVIDENCE_TYPES,
AUTHENTICATED_EVIDENCE_HANDLER_REF,
produce_evidence_closure,
producer_operation_definition,
producer_registry,
)
from trusted_provider_receipts import (
ProviderAuthenticationError,
TrustedProviderPolicy,
TrustedProviderReceiptResolver,
)
# Exact, restrictive handler_ref -> callable map. This is the ONLY way
# handler_refs are resolved; no arbitrary import/eval.
_HANDLERS: Mapping[str, Callable[..., ProduceResult]] = {
"governed_release_evidence_producer.produce_evidence_closure": produce_evidence_closure,
AUTHENTICATED_EVIDENCE_HANDLER_REF: produce_authenticated_evidence_closure,
}
class EvidenceOperationInventory:
"""Inventory of governed evidence operations available to the authority."""
def __init__(self, *registries: OperationRegistry) -> None:
definitions: list[OperationDefinition] = []
seen: set[str] = set()
for reg in registries:
for definition in reg.definitions:
if definition.operation_id in seen:
continue
seen.add(definition.operation_id)
definitions.append(definition)
self._registry = OperationRegistry(definitions)
self._by_id = {d.operation_id: d for d in definitions}
@classmethod
def default(cls) -> "EvidenceOperationInventory":
"""The standard inventory: the governed release evidence producer."""
return cls(producer_registry())
def contains(self, operation_id: str) -> bool:
return operation_id in self._by_id
def operation_ids(self) -> tuple[str, ...]:
return tuple(self._by_id)
def get(self, operation_id: str) -> OperationDefinition | None:
return self._by_id.get(operation_id)
def resolve_handler(handler_ref: str) -> Callable[..., ProduceResult]:
"""Resolver: turn an OperationDefinition.handler_ref into the real callable.
Exact lookup in a fixed allowlist. Unknown or non-producer references raise.
"""
handler = _HANDLERS.get(handler_ref)
if handler is None:
raise EvidenceProduceError("HANDLER_REF_UNRESOLVED")
return handler
def _payload_to_artifacts(payload: Mapping[str, Any]) -> GovernedBuildArtifacts:
try:
return GovernedBuildArtifacts(
version=payload["version"],
candidate_id=payload["candidate_id"],
source_commit=payload["source_commit"],
tree_oid=payload["tree_oid"],
artifact=bytes.fromhex(payload["artifact_hex"]),
manifest=payload["manifest"],
package_closure=bytes.fromhex(payload["package_closure_hex"]),
policy=payload["policy"],
tests=payload["tests"],
canary=payload["canary"],
gitea=payload["gitea"],
rollback=payload["rollback"],
secret_scan=payload["secret_scan"],
deployment=payload["deployment"],
tag=payload["tag"],
finalization=payload["finalization"],
rollback_sha256=payload.get("rollback_sha256", ""),
issued_at=payload.get("issued_at", ""),
)
except (KeyError, TypeError, ValueError) as exc:
raise EvidenceProduceError("PAYLOAD_SCHEMA_INVALID") from exc
def invoke_governed_produce(
*,
builder_policy: Path,
source_root: Path,
payload: Mapping[str, Any],
expected_owner_uid: int | None = 0,
) -> dict[str, Any]:
"""Real runtime chain: Inventory -> Resolver -> operation -> Producer -> closure.
Returns a verdict dict. Only a genuine end-to-end invocation that reaches
evidence production reports ``status == "PRODUCER_RUNTIME_REGISTERED_PASS"``.
"""
inventory = EvidenceOperationInventory.default()
operation_id = "governed.release.evidence.produce"
if not inventory.contains(operation_id):
raise EvidenceProduceError("OPERATION_NOT_IN_INVENTORY")
definition = inventory.get(operation_id)
assert definition is not None # guaranteed by contains()
handler = resolve_handler(definition.handler_ref)
request = _payload_to_artifacts(payload)
result = handler(
builder_policy=builder_policy,
source_root=source_root,
request=request,
expected_owner_uid=expected_owner_uid,
)
# The resolved handler returned a real ProduceResult from a real closure;
# the registered operation genuinely produced evidence.
return {
"status": "PRODUCER_RUNTIME_REGISTERED_PASS",
"operation_id": operation_id,
"handler_ref": definition.handler_ref,
"handler_resolved": handler.__module__ + "." + handler.__name__,
"evidence_types_bound": list(TRUSTED_PROVIDER_EVIDENCE_TYPES),
"result": {
"produce_status": result.status,
"idempotent": result.idempotent,
"receipt_sha256": result.receipt_sha256,
"closure_dir": result.closure_dir,
"evidence_digests": dict(result.evidence_digests),
},
}
# ---------------------------------------------------------------------------
# V2R2C2 Round-3: Provider-Authenticated runtime chain.
#
# Inventory -> Resolver -> authenticated operation -> TrustedProviderReceiptResolver
# -> Producer assembler (second-layer bound)
#
# The Round-3 runtime builds TrustedProviderReceiptResolver from the governed
# evidence-provider policy (never from request-supplied store paths, provider
# identities, key ids, or HMAC material). Gate-W authenticates only the
# provider classes that can exist before staging (tests/Gitea/secret-scan);
# canary/rollback remain pending until Gate-R.
# ---------------------------------------------------------------------------
# Sequence used for a full final release closure (Gate-R).
ALL_RELEASE_EVIDENCE_KINDS = ("tests", "canary", "gitea", "rollback", "secret_scan")
# Provider classes that are real before staging (Gate-W).
GATE_W_EVIDENCE_KINDS = ("tests", "gitea", "secret_scan")
_DEFAULT_PROVIDER_POLICY = Path(__file__).resolve().parents[2] / "config" / "gitops" / "trusted-evidence-providers.json"
def _load_provider_policy(path: Path) -> Mapping[str, Any]:
try:
decoded = json.loads(Path(path).read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceProduceError("PROVIDER_POLICY_UNAVAILABLE") from exc
if not isinstance(decoded, Mapping) or decoded.get("schema") != "aios.trusted-evidence-providers.v1":
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
return decoded
def _policy_from_config(name: str, raw: Mapping[str, Any], defaults: Mapping[str, Any]) -> TrustedProviderPolicy:
def _mode(value: object, field: str) -> int:
if isinstance(value, int):
return value
if isinstance(value, str) and value.isdigit():
return int(value, 8)
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
try:
store_root = Path(str(raw["store_root"])).expanduser()
allowed_ops = frozenset(str(op) for op in raw["allowed_operations"])
return TrustedProviderPolicy(
evidence_kind=name,
provider_identity=str(raw["provider_identity"]),
store_id=str(raw["store_id"]),
store_root=store_root,
allowed_operations=allowed_ops,
expected_owner_uid=_mode(raw.get("expected_owner_uid", defaults.get("expected_owner_uid", 0)), "expected_owner_uid"),
allowed_directory_modes=frozenset(
_mode(v, "allowed_directory_modes")
for v in raw.get("allowed_directory_modes", defaults.get("allowed_directory_modes", ["0700", "0750"]))
),
allowed_receipt_modes=frozenset(
_mode(v, "allowed_receipt_modes")
for v in raw.get("allowed_receipt_modes", defaults.get("allowed_receipt_modes", ["0600", "0640"]))
),
authentication_kind=str(raw.get("authentication_kind", defaults.get("authentication_kind", "root-owned-store-v1"))),
max_age_seconds=raw.get("max_age_seconds", defaults.get("max_age_seconds")),
)
except (KeyError, TypeError, ValueError) as exc:
raise EvidenceProduceError("PROVIDER_POLICY_INVALID") from exc
def build_provider_receipt_resolver(
provider_policy_path: Path = _DEFAULT_PROVIDER_POLICY,
*,
hmac_key_resolver: Callable[[str], bytes] | None = None,
now_fn: Callable[[], Any] | None = None,
) -> TrustedProviderReceiptResolver:
"""Build TrustedProviderReceiptResolver from the governed evidence-provider policy.
Store paths, provider identities, key ids and HMAC material come only from
the governed policy/secret resolver — never from a request payload.
"""
policy = _load_provider_policy(provider_policy_path)
defaults = dict(policy.get("defaults") or {})
providers = policy.get("providers")
if not isinstance(providers, Mapping) or not providers:
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
policies: dict[str, TrustedProviderPolicy] = {}
for name, raw in providers.items():
if not isinstance(raw, Mapping):
raise EvidenceProduceError("PROVIDER_POLICY_INVALID")
policies[name] = _policy_from_config(name, raw, defaults)
return TrustedProviderReceiptResolver(policies, hmac_key_resolver=hmac_key_resolver, now_fn=now_fn)
def _resolver_guard(resolver: TrustedProviderReceiptResolver | None) -> TrustedProviderReceiptResolver:
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
return resolver
def validate_gate_w_provider_auth(
request: GovernedBuildArtifacts,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Gate-W: authenticate only the provider classes real before staging.
Returns PRODUCER_RUNTIME_REGISTRATION_PASS / PROVIDER_AUTH_ENFORCEMENT_PASS /
FINAL_RELEASE_CLOSURE_PENDING_GATE_R. Canary/rollback are intentionally not
required here — they remain pending until Gate-R.
"""
resolver = _resolver_guard(provider_receipt_resolver)
gate_w_authentication_status(request, resolver=resolver)
return {
"producer_runtime": "PRODUCER_RUNTIME_REGISTRATION_PASS",
"provider_auth": "PROVIDER_AUTH_ENFORCEMENT_PASS",
"final_release_closure": "FINAL_RELEASE_CLOSURE_PENDING_GATE_R",
}
def _legacy_closure_adapter(authenticated_request: GovernedBuildArtifacts, *, source_root: Path, builder_policy: Path) -> ProduceResult:
"""Invoke the legacy closure assembler on the resolver-authenticated request.
The classified evidence fields now carry ``provider_receipt`` provenance;
the producer's second-layer validator re-asserts content/OID binding on it.
"""
return produce_evidence_closure(
builder_policy=builder_policy,
source_root=source_root,
request=authenticated_request,
expected_owner_uid=0,
)
def produce_authenticated_evidence(
request: GovernedBuildArtifacts,
*,
provider_receipt_resolver: TrustedProviderReceiptResolver | None,
source_root: Path,
builder_policy: Path,
required_kinds: Iterable[str] = ALL_RELEASE_EVIDENCE_KINDS,
) -> ProduceResult:
"""Gate-R production path: authenticate all five final-release provider
receipts, then invoke the legacy assembler only after authentication."""
resolver = _resolver_guard(provider_receipt_resolver)
try:
return produce_authenticated_evidence_closure(
request,
provider_receipt_resolver=resolver,
source_root=source_root,
builder_policy=builder_policy,
legacy_producer=_legacy_closure_adapter,
)
except ProviderAuthenticationError as exc:
raise EvidenceProduceError(str(exc)) from exc
@@ -1,4 +1,7 @@
"""Governed V2R2C1 production release orchestration.
"""Governed release orchestration with a trusted-plan V2R2C2 path.
The legacy V2R2C1 implementation remains available only for compatibility;
new release requests are dispatched to the plan-driven executor below.
This module is deliberately scoped to the Owner-approved V2R2C1 binding. It
uses the existing TagPublisher and metadata adapters; it does not introduce a
@@ -20,6 +23,8 @@ from dataclasses import dataclass
from pathlib import Path
from typing import Any, Mapping
from trusted_release_execution_plan import PlanError, TrustedReleasePlan
from gitea_tag_publication import (
GiteaTagClient,
GitTagBackend,
@@ -406,6 +411,14 @@ class GovernedProductionReleaseExecutor:
config.remote_name,
state_root=self.root / "tag-publication",
)
# Release checks must use the same explicitly configured staging baseline
# as the Action/model runtime; do not fall back to the publication workspace.
self.publisher.baseline_path = Path(
os.environ.get("AIOS_GITOPS_BASELINE_AUTHORITY", str(config.baseline_path))
)
# The immutable runtime package is the source of the version projection;
# the mutable state workspace only stores publication state.
self.publisher.versions_root = Path(__file__).resolve().parents[1] / "versions"
def _request(self, request: Mapping[str, Any]) -> PublicationRequest:
binding = self.binding
@@ -546,7 +559,139 @@ def _health(url: str, binding: ReleaseBinding, artifact: Mapping[str, Any]) -> d
return {"status": "ACTIVE_VERIFIED" if exact else "CONFLICT", "http_status": status, **body}
def build_executor_config(config: Any) -> GovernedProductionReleaseExecutor:
class TrustedPlanReleaseExecutor:
"""Release handler whose identity comes only from an immutable trusted plan.
The bridge may prepare/read evidence and request tag publication, but it does
not perform production side effects. The privileged approved-release
service consumes the same plan and exact Owner approval envelope.
"""
def __init__(self, config: Any):
self.config = config
configured = getattr(config, "release_plan_path", None)
self.plan_path = Path(configured) if configured else None
self.state_root = Path(config.state_root) / "trusted-release"
self.state_root.mkdir(parents=True, exist_ok=True)
self._approval: dict[str, Any] | None = None
self._tag: dict[str, Any] | None = None
def _plan(self) -> TrustedReleasePlan:
path = self.plan_path
if path is None:
env_path = os.environ.get("AIOS_RELEASE_PLAN_PATH", "")
path = Path(env_path) if env_path else None
if path is None:
raise ReleaseBlocked("TRUSTED_RELEASE_PLAN_REQUIRED")
try:
return TrustedReleasePlan.load(path)
except PlanError as exc:
raise ReleaseBlocked(f"TRUSTED_RELEASE_PLAN_INVALID:{exc}") from exc
@staticmethod
def _binding(plan: TrustedReleasePlan) -> dict[str, Any]:
value = plan.value
return {
"version": value["version"],
"target_oid": value["source_commit"],
"evidence_closure_oid": value["evidence_closure_oid"],
"artifact_sha256": value["artifact_sha256"],
"artifact_size": value["artifact"]["size"],
"approval_plan_id": value["approval_plan_id"],
"candidate_id": value["candidate_id"],
"release_plan_sha256": plan.release_plan_sha256,
}
def _request(self, request: Mapping[str, Any]) -> tuple[TrustedReleasePlan, dict[str, Any]]:
plan = self._plan()
binding = self._binding(plan)
for key, expected in binding.items():
if key in request and request[key] != expected:
raise ManualOwnerReviewRequired("trusted release request binding mismatch:" + key)
return plan, binding
def approval_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not False:
raise ReleaseBlocked("TAG_APPROVAL_REQUEST_MUST_BE_DRY_RUN")
plan, binding = self._request(request)
approval = request.get("owner_approval")
if not isinstance(approval, Mapping) or approval.get("approved") is not True:
raise ReleaseBlocked("OWNER_RELEASE_APPROVAL_REQUIRED")
for key in ("candidate_id", "release_plan_sha256", "approval_plan_id"):
if approval.get(key) != binding[key]:
raise ManualOwnerReviewRequired("OWNER_APPROVAL_BINDING_MISMATCH:" + key)
record = {"plan": plan.as_dict(), "owner_approval": dict(approval)}
digest = hashlib.sha256(canonical(record)).hexdigest()
if self._approval is not None and self._approval != record:
raise ManualOwnerReviewRequired("OWNER_APPROVAL_REPLAY_CONFLICT")
self._approval = record
return {"status": "RECORDED", "execute": False, "release_plan_sha256": plan.release_plan_sha256, "approval_digest_sha256": digest}
def _require_approval(self, plan: TrustedReleasePlan) -> None:
if self._approval is None or self._approval.get("plan") != plan.as_dict():
raise ReleaseBlocked("OWNER_RELEASE_APPROVAL_NOT_RECORDED")
def build_artifact(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
artifact = dict(plan.value["artifact"])
path = Path(str(artifact["path"]))
if path.is_symlink() or not path.is_file() or path.stat().st_size != artifact["size"]:
raise ReleaseBlocked("TRUSTED_ARTIFACT_NOT_PRESENT")
actual = hashlib.sha256(path.read_bytes()).hexdigest()
if actual != plan.value["artifact_sha256"]:
raise ReleaseBlocked("TRUSTED_ARTIFACT_DIGEST_MISMATCH")
return {"status": "VERIFIED", "version": plan.value["version"], "source_commit": plan.value["source_commit"], "tree_oid": plan.value["tree_oid"], "artifact": str(path), "artifact_size": artifact["size"], "artifact_sha256": actual, "manifest_sha256": plan.value["manifest_sha256"], "package_closure_sha256": plan.value["package_closure_sha256"], "release_plan_sha256": plan.release_plan_sha256}
def _publisher(self, plan: TrustedReleasePlan) -> TagPublisher:
root = self.state_root / plan.value["candidate_id"]
root.mkdir(parents=True, exist_ok=True)
publisher = TagPublisher(root / "publication-workspace", GitTagBackend(root / "publication-workspace", authoritative_repo=Path(self.config.authoritative_git_dir)), GiteaTagClient(self.config.api_base, "aios/aios", Path(self.config.token_path)), self.config.remote_name, state_root=root / "tag-publication")
publisher.baseline_path = Path(os.environ.get("AIOS_GITOPS_BASELINE_AUTHORITY", str(self.config.baseline_path)))
publisher.versions_root = Path(__file__).resolve().parents[1] / "versions"
return publisher
def publish_tag(self, request: Mapping[str, Any]) -> dict[str, Any]:
if request.get("execute") is not True:
raise ReleaseBlocked("TAG_PUBLICATION_REQUIRES_EXECUTE_TRUE")
plan, _ = self._request(request)
self._require_approval(plan)
publication = PublicationRequest(plan.value["version"], plan.value["source_commit"], plan.value["evidence_closure_oid"], plan.value["approval_plan_id"])
try:
result = self._publisher(plan)._verify_publication(publication, allow_push=True, previously_deployed=False)
except (PublicationManualOwnerReviewRequired, PublicationPending, PublicationReconciliationRequired) as exc:
raise ReconciliationRequired(str(exc)) from exc
self._tag = dict(result)
return {"status": "REMOTE_VERIFIED", **result, "release_plan_sha256": plan.release_plan_sha256}
def tag_status(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
return self._publisher(plan).get_tag_publication_status(plan.value["version"])
def execute_request(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, _ = self._request(request)
self._require_approval(plan)
raise ReleaseBlocked("PRODUCTION_EXECUTION_REQUIRES_APPROVED_RELEASE_EXECUTOR")
def consistency(self, request: Mapping[str, Any]) -> dict[str, Any]:
plan, binding = self._request(request)
result = {"status": "PASS", "release_binding": binding, "trusted_plan": plan.as_dict(), "production_mutation": False}
result["baseline"] = _read_json(Path(self.config.baseline_path))
result["registry"] = _read_json(Path(self.config.registry_root) / f"{plan.value['version']}.json")
return result
def acceptance(self, request: Mapping[str, Any]) -> dict[str, Any]:
result = self.consistency(request)
result["accepted"] = result["status"] == "PASS"
return result
def build_executor_config(config: Any) -> Any:
"""Select legacy compatibility only when no trusted new-release plan exists."""
plan_path = getattr(config, "release_plan_path", None) or os.environ.get("AIOS_RELEASE_PLAN_PATH", "")
target_version = getattr(config, "target_version", "")
if plan_path or target_version != VERSION:
return TrustedPlanReleaseExecutor(config)
return GovernedProductionReleaseExecutor(config)
@@ -0,0 +1,87 @@
from __future__ import annotations
from dataclasses import is_dataclass, replace
from typing import Any, Iterable, Mapping
from trusted_provider_receipts import (
ProviderAuthenticationError,
ResolvedEvidence,
TrustedProviderReceiptResolver,
)
ALL_RELEASE_EVIDENCE_KINDS = ("tests", "canary", "gitea", "rollback", "secret_scan")
GATE_W_EVIDENCE_KINDS = ("tests", "gitea", "secret_scan")
def authenticate_evidence_request(
evidence_kind: str,
evidence_request: Mapping[str, Any],
*,
source_oid: str,
tree_oid: str,
resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, Any]:
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
resolved: ResolvedEvidence = resolver.resolve_evidence(
evidence_kind,
evidence_request,
source_oid=source_oid,
tree_oid=tree_oid,
)
return {
**dict(resolved.result),
"provider_receipt": dict(resolved.provider_receipt),
}
def authenticate_build_artifacts(
request: Any,
*,
resolver: TrustedProviderReceiptResolver | None,
required_kinds: Iterable[str] = ALL_RELEASE_EVIDENCE_KINDS,
) -> Any:
"""Authenticate receipt_ref-only evidence and return a replaced request.
The input is expected to be the existing GovernedBuildArtifacts dataclass.
The function deliberately relies only on field names already used by the
producer (`source_commit`, `tree_oid`, and the evidence kind fields), so it
can be integrated without changing the deployment/release-plan contract.
"""
if resolver is None:
raise ProviderAuthenticationError("PROVIDER_AUTHENTICATION_REQUIRED")
if not is_dataclass(request):
raise TypeError("request must be a dataclass instance")
source_oid = getattr(request, "source_commit")
tree_oid = getattr(request, "tree_oid")
replacements: dict[str, Any] = {}
for kind in tuple(required_kinds):
evidence_request = getattr(request, kind, None)
if evidence_request is None:
raise ProviderAuthenticationError(f"EVIDENCE_{kind.upper()}_REQUIRED")
replacements[kind] = authenticate_evidence_request(
kind,
evidence_request,
source_oid=source_oid,
tree_oid=tree_oid,
resolver=resolver,
)
return replace(request, **replacements)
def gate_w_authentication_status(
request: Any,
*,
resolver: TrustedProviderReceiptResolver | None,
) -> dict[str, str]:
"""Authenticate Gate-W evidence without pretending final closure exists."""
authenticate_build_artifacts(
request,
resolver=resolver,
required_kinds=GATE_W_EVIDENCE_KINDS,
)
return {
"producer_runtime": "PRODUCER_RUNTIME_REGISTRATION_PASS",
"provider_auth": "PROVIDER_AUTH_ENFORCEMENT_PASS",
"final_release_closure": "FINAL_RELEASE_CLOSURE_PENDING_GATE_R",
}
@@ -0,0 +1,634 @@
#!/usr/bin/env python3
"""Governed release evidence producer (V2R2C2 P0, operation governed.release.evidence.produce).
This module is the *producer* half of the governed evidence pipeline. It never
invents candidate/release evidence: every digest and pass/fail flag in the closure
is derived from real build artifacts and real verification executions supplied as
input. It assembles a self-consistent, root-owned evidence closure below
``source_root/<version>`` that ``TrustedBuilderEvidenceMaterializer`` (the projection
half) can re-validate and project byte-for-byte.
Contract (see docs/V2R2C2-P0-convergence-20260909.md section 1):
- operation_id : governed.release.evidence.produce
- input : governed build artifacts + deployment conventions (no hand-write)
- output : source_root/<version>/ full evidence_layout closure + receipt.json
- fail-closed : mismatched secret-scan digest, tampered artifact, non-passing
tests/canary/rollback/secret-scan, or inconsistent gitea all abort
without writing a closure.
- idempotent : an existing identical closure is accepted; a conflicting one fails.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import secrets
from dataclasses import dataclass, field
from pathlib import Path
import shutil
import stat
from typing import Any, Mapping
from aios_baseline.operations import OperationDefinition, OperationRegistry
from aios_trusted_candidate_builder import (
load_builder_policy,
TrustedCandidateBuilderConfig,
)
from release_evidence_materializer import EvidenceMaterializationError
from release_version import validate_release_version
PRODUCE_SCHEMA = "aios.governed-release-evidence-producer.v1"
EVIDENCE_RECEIPT_SCHEMA = "aios.trusted-candidate-builder.evidence.v1"
# Each governed evidence type (tests/canary/gitea/rollback/secret_scan) must be
# bound to a trusted-provider receipt. A plain caller-supplied {"passed": true}
# is never accepted on its own: the producer requires the receipt that proves a
# trusted provider actually ran the check and produced the referenced result.
TRUSTED_PROVIDER_RECEIPT_SCHEMA = "aios.trusted-provider.evidence.v1"
TRUSTED_PROVIDER_EVIDENCE_TYPES = ("tests", "canary", "gitea", "rollback", "secret_scan")
class EvidenceProduceError(RuntimeError):
"""Stable, non-sensitive producer failure."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _sha256_bytes(value: bytes) -> str:
return hashlib.sha256(value).hexdigest()
def _validate_oid(value: object, name: str) -> str:
normalized = str(value or "").strip().lower()
if (
len(normalized) != 40
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
def _validate_identifier(value: object, name: str) -> str:
normalized = str(value or "").strip()
if (
not normalized
or len(normalized) > 255
or normalized[0] in ".-"
or any(
char not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._:-"
for char in normalized
)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
def _validate_sha256(value: object, name: str, *, allow_empty: bool = False) -> str:
normalized = str(value or "").strip().lower()
if allow_empty and not normalized:
return ""
if (
len(normalized) != 64
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceProduceError(f"{name.upper()}_INVALID")
return normalized
@dataclass(frozen=True)
class GovernedBuildArtifacts:
"""Real build artifacts + real verification results. Producer never invents these."""
version: str
candidate_id: str
source_commit: str
tree_oid: str
artifact: bytes
manifest: Mapping[str, Any]
package_closure: bytes
policy: Mapping[str, Any]
tests: Mapping[str, Any]
canary: Mapping[str, Any]
gitea: Mapping[str, Any]
rollback: Mapping[str, Any]
secret_scan: Mapping[str, Any]
# Deployment conventions (supplied by release planning, not evidence invention).
deployment: Mapping[str, Any]
tag: Mapping[str, Any]
finalization: Mapping[str, Any]
rollback_sha256: str = ""
issued_at: str = ""
@dataclass(frozen=True)
class ProduceResult:
status: str
idempotent: bool
receipt_sha256: str
closure_dir: str
evidence_digests: Mapping[str, str] = field(default_factory=dict)
def _require_trusted_provider_receipt(
name: str,
evidence: Mapping[str, Any],
source_commit: str,
tree_oid: str,
) -> None:
"""Require a trusted-provider binding for `evidence` (second-layer check).
A caller may never pass a bare ``{"passed": true}``: each governed evidence
type must be bound to a trusted provider. Two accepted forms, both
fail-closed on any missing/structurally invalid/digest-mismatched binding:
1. Round-2 native ``receipt`` (schema ``aios.trusted-provider.evidence.v1``)
— a caller-carried receipt.
2. Round-3 resolver-authenticated ``provider_receipt`` provenance (schema
``aios.trusted-evidence-provider-receipt.v2``) — produced by
``TrustedProviderReceiptResolver`` from an authenticated store. Its input
is the resolver-authenticated provenance, never a caller-supplied body.
This helper is a *second-layer* binding validator: for the Round-3 path the
authoritative proof is the resolver's authenticated provenance embedded as
``provider_receipt``, and this check only re-asserts content/OID binding on
top of it.
"""
receipt = evidence.get("receipt")
if isinstance(receipt, Mapping) and receipt:
body = {k: v for k, v in dict(evidence).items() if k != "receipt"}
expected = _sha256_bytes(_canonical_json(body))
if receipt.get("schema") != TRUSTED_PROVIDER_RECEIPT_SCHEMA:
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if receipt.get("content_sha256") != expected:
raise EvidenceProduceError("EVIDENCE_RECEIPT_CONTENT_MISMATCH")
for f in ("provider_identity", "operation_run_id", "result_reference",
"environment", "issued_at"):
if not str(receipt.get(f, "") or "").strip():
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if receipt.get("source_oid") not in (None, "", source_commit):
raise EvidenceProduceError("EVIDENCE_RECEIPT_SOURCE_OID_MISMATCH")
if receipt.get("tree_oid") not in (None, "", tree_oid):
raise EvidenceProduceError("EVIDENCE_RECEIPT_TREE_OID_MISMATCH")
return
provider_receipt = evidence.get("provider_receipt")
if isinstance(provider_receipt, Mapping) and provider_receipt:
if (
provider_receipt.get("schema")
!= "aios.trusted-evidence-provider-receipt.v2"
):
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
for f in ("provider_identity", "provider_operation", "operation_run_id",
"result_reference", "content_sha256", "source_oid", "tree_oid",
"issued_at"):
if not str(provider_receipt.get(f, "") or "").strip():
raise EvidenceProduceError("EVIDENCE_RECEIPT_INVALID")
if provider_receipt.get("source_oid") != source_commit:
raise EvidenceProduceError("EVIDENCE_RECEIPT_SOURCE_OID_MISMATCH")
if provider_receipt.get("tree_oid") != tree_oid:
raise EvidenceProduceError("EVIDENCE_RECEIPT_TREE_OID_MISMATCH")
return
raise EvidenceProduceError("EVIDENCE_RECEIPT_MISSING")
def _verify_artifacts(
config: TrustedCandidateBuilderConfig,
request: GovernedBuildArtifacts,
) -> dict[str, str]:
"""Validate input consistency (fail-closed) and return the evidence digests."""
try:
version = validate_release_version(request.version)
except ValueError as exc:
raise EvidenceProduceError("VERSION_INVALID") from exc
if version != request.version:
raise EvidenceProduceError("VERSION_INVALID")
candidate_id = _validate_identifier(request.candidate_id, "candidate_id")
source_commit = _validate_oid(request.source_commit, "source_commit")
tree_oid = _validate_oid(request.tree_oid, "tree_oid")
if (
set(request.manifest) != {
"artifact_sha256", "package_closure_sha256",
}
):
raise EvidenceProduceError("MANIFEST_SCHEMA_INVALID")
closure_sha = _validate_sha256(
request.manifest.get("package_closure_sha256"), "package_closure_sha256"
)
artifact_sha = _sha256_bytes(request.artifact)
if request.manifest["artifact_sha256"] != artifact_sha:
raise EvidenceProduceError("MANIFEST_ARTIFACT_DIGEST_MISMATCH")
# manifest_sha256 is the digest of the manifest evidence file itself. It is
# derived (never invented) from the manifest content; the manifest does not
# embed its own digest to avoid a self-referential fixed point.
manifest_sha = _sha256_bytes(_canonical_json(dict(request.manifest)))
if _sha256_bytes(request.package_closure) != closure_sha:
raise EvidenceProduceError("PACKAGE_CLOSURE_DIGEST_MISMATCH")
# policy_sha256 is the digest of the policy evidence file content; derived
# here (never invented, never self-referential).
policy_sha = _sha256_bytes(_canonical_json(dict(request.policy)))
# Secret scan must cover the exact artifact bytes.
secret_scan_artifact = _validate_sha256(
request.secret_scan.get("artifact_sha256"), "secret_scan.artifact_sha256"
)
if secret_scan_artifact != artifact_sha:
raise EvidenceProduceError("SECRET_SCAN_ARTIFACT_DIGEST_MISMATCH")
# Every governed evidence type must be bound to a trusted-provider receipt.
# A bare caller-supplied {"passed": true} (no receipt) fails closed here;
# the producer never accepts an invented pass flag.
for etype in TRUSTED_PROVIDER_EVIDENCE_TYPES:
_require_trusted_provider_receipt(
etype, getattr(request, etype), source_commit, tree_oid
)
if request.secret_scan.get("passed") is not True:
raise EvidenceProduceError("SECRET_SCAN_NOT_PASSED")
if request.tests.get("passed") is not True:
raise EvidenceProduceError("TESTS_NOT_PASSED")
if request.canary.get("passed") is not True:
raise EvidenceProduceError("CANARY_NOT_PASSED")
if request.rollback.get("ready") is not True:
raise EvidenceProduceError("ROLLBACK_NOT_READY")
if (
request.gitea.get("consistent") is not True
or request.gitea.get("repository") != config.repository
or request.gitea.get("source_commit") != source_commit
or request.gitea.get("remote_ref_oid") != source_commit
or request.gitea.get("tree_oid") != tree_oid
):
raise EvidenceProduceError("GITEA_INCONSISTENT")
digests = {
"artifact": artifact_sha,
"manifest": manifest_sha,
"package_closure": closure_sha,
"policy": policy_sha,
"tests": _sha256_bytes(_canonical_json(dict(request.tests))),
"canary": _sha256_bytes(_canonical_json(dict(request.canary))),
"gitea": _sha256_bytes(_canonical_json(dict(request.gitea))),
"rollback": _sha256_bytes(_canonical_json(dict(request.rollback))),
"secret_scan": _sha256_bytes(_canonical_json(dict(request.secret_scan))),
}
return digests
def _build_receipt(
config: TrustedCandidateBuilderConfig,
request: GovernedBuildArtifacts,
) -> Mapping[str, Any]:
"""Assemble the self-consistent receipt from verified artifacts."""
from silent_release_controller import ReleaseCandidate, ReleaseState
artifact_sha = _sha256_bytes(request.artifact)
manifest_sha = _sha256_bytes(_canonical_json(dict(request.manifest)))
closure_sha = _validate_sha256(
request.manifest.get("package_closure_sha256"), "package_closure_sha256"
)
policy_sha = _sha256_bytes(_canonical_json(dict(request.policy)))
candidate_kwargs: dict[str, Any] = {}
issued_at = str(request.issued_at or "").strip()
if issued_at:
# The evidence was produced at the build execution instant; bind it so the
# same verified artifacts reproduce an identical, idempotent closure.
candidate_kwargs["created_at"] = issued_at
candidate_kwargs["updated_at"] = issued_at
candidate = ReleaseCandidate(
candidate_id=request.candidate_id,
source_commit=request.source_commit,
tree_oid=request.tree_oid,
artifact_sha256=artifact_sha,
manifest_sha256=manifest_sha,
tests=dict(request.tests),
canary=dict(request.canary),
rollback_ready=request.rollback.get("ready") is True,
git_consistency={
"consistent": True,
"repository": config.repository,
"source_commit": request.source_commit,
"tree_oid": request.tree_oid,
},
gitea_consistency=dict(request.gitea),
state=ReleaseState.READY_FOR_APPROVAL,
**candidate_kwargs,
)
deployment_payload = dict(request.deployment)
deployment_payload.update(
{
"service": request.deployment["service"],
"source_commit": request.source_commit,
"tree_oid": request.tree_oid,
"artifact_sha256": artifact_sha,
"manifest_sha256": manifest_sha,
"package_closure_sha256": closure_sha,
"policy_sha256": policy_sha,
"rollback_sha256": _validate_sha256(
request.rollback_sha256, "rollback_sha256"
)
or request.rollback.get("rollback_sha256", ""),
"repository_identity": config.repository,
"authoritative_git_dir": request.deployment["authoritative_git_dir"],
}
)
if not deployment_payload["rollback_sha256"]:
raise EvidenceProduceError("ROLLBACK_SHA256_REQUIRED")
from silent_release_hardening import (
DeploymentPlan,
FinalizationPlan,
ReleasePlan,
TagPlan,
TrustedCandidateBundle,
)
release_plan = ReleasePlan(
deployment=DeploymentPlan.from_dict(deployment_payload),
tag=TagPlan.from_dict(dict(request.tag)),
finalization=FinalizationPlan.from_dict(dict(request.finalization)),
policy_sha256=policy_sha,
)
checks = {name: True for name in config.required_checks}
TrustedCandidateBundle(candidate, release_plan, checks).validate()
return {
"schema": EVIDENCE_RECEIPT_SCHEMA,
"version": request.version,
"candidate": candidate.to_dict(),
"release_plan": release_plan.to_dict(),
}
def _safe_target(source_root: Path, version: str) -> Path:
relative = Path(version)
if relative.is_absolute() or any(
part in {"", ".", ".."} for part in relative.parts
):
raise EvidenceProduceError("CLOSURE_PATH_INVALID")
candidate = Path(source_root) / version
try:
metadata = Path(source_root).stat(follow_symlinks=False)
except OSError as exc:
raise EvidenceProduceError("SOURCE_ROOT_UNAVAILABLE") from exc
if not stat.S_ISDIR(metadata.st_mode) or Path(source_root).is_symlink():
raise EvidenceProduceError("SOURCE_ROOT_INVALID")
return candidate
def _write_durable_json(path: Path, payload: Mapping[str, Any], mode: int = 0o600) -> str:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
# The deterministic collector compares file digests against the canonical
# payload hash (no trailing newline); write the exact canonical bytes so the
# producer's closure re-validates under TrustedBuilderEvidenceMaterializer.
raw = _canonical_json(payload)
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
os.write(descriptor, raw)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, mode)
return hashlib.sha256(raw).hexdigest()
def _write_durable_bytes(path: Path, value: bytes, mode: int = 0o600) -> str:
path.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
os.write(descriptor, value)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, mode)
return hashlib.sha256(value).hexdigest()
def produce_evidence_closure(
*,
builder_policy: Path,
source_root: Path,
request: GovernedBuildArtifacts,
expected_owner_uid: int | None = 0,
) -> ProduceResult:
"""Produce a self-consistent, root-owned evidence closure, fail-closed and idempotent."""
try:
config = TrustedCandidateBuilderConfig.from_policy(
load_builder_policy(builder_policy)
)
except Exception as exc:
raise EvidenceProduceError(str(exc)) from exc
try:
version = validate_release_version(request.version)
except ValueError as exc:
raise EvidenceProduceError("VERSION_INVALID") from exc
digests = _verify_artifacts(config, request)
receipt = _build_receipt(config, request)
# Resolve layout paths from policy (aligned with evidence_layout; never invented).
target = _safe_target(source_root, version)
layout_paths = {
name: target / Path(template.format(version=version)).relative_to(version)
for name, template in config.evidence_layout.items()
}
if target.exists() or target.is_symlink():
# Idempotency: an existing identical closure (same receipt digest + same
# artifact bytes) is accepted; anything else is a conflict.
existing_receipt = target / "receipt.json"
if not existing_receipt.is_file():
raise EvidenceProduceError("CLOSURE_CONFLICT")
try:
existing = json.loads(existing_receipt.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceProduceError("CLOSURE_CONFLICT") from exc
if _canonical_json(dict(existing)) != _canonical_json(dict(receipt)):
raise EvidenceProduceError("CLOSURE_CONFLICT")
for name, path in layout_paths.items():
if not (path.is_file() and _file_sha256(path) == digests[name]):
raise EvidenceProduceError("CLOSURE_CONFLICT")
return ProduceResult(
status="EVIDENCE_PRODUCED",
idempotent=True,
receipt_sha256=_sha256_bytes(_canonical_json(dict(receipt))),
closure_dir=str(target),
evidence_digests=dict(digests),
)
temporary = source_root / (
f".{version}.{os.getpid()}.{secrets.token_hex(8)}.prod.tmp"
)
if temporary.exists() or temporary.is_symlink():
raise EvidenceProduceError("PRODUCE_TEMP_CONFLICT")
temporary.mkdir(mode=0o700)
# The closure directory mirrors source_root/<version>/; every written file uses
# the policy-declared basename so the layout is aligned with evidence_layout.
try:
_write_durable_json(temporary / "receipt.json", receipt)
_write_durable_bytes(temporary / "artifact.tar.gz", request.artifact)
_write_durable_json(temporary / "manifest.json", request.manifest)
_write_durable_bytes(temporary / "checksums.sha256", request.package_closure)
_write_durable_json(temporary / "trusted-release-policy.json", request.policy)
_write_durable_json(temporary / "tests.json", request.tests)
_write_durable_json(temporary / "canary.json", request.canary)
_write_durable_json(temporary / "gitea.json", request.gitea)
_write_durable_json(temporary / "rollback.json", request.rollback)
_write_durable_json(temporary / "secret-scan.json", request.secret_scan)
os.replace(temporary, target)
except Exception:
shutil.rmtree(temporary, ignore_errors=True)
raise
return ProduceResult(
status="EVIDENCE_PRODUCED",
idempotent=False,
receipt_sha256=_sha256_bytes(_canonical_json(dict(receipt))),
closure_dir=str(target),
evidence_digests=dict(digests),
)
def producer_operation_definition() -> OperationDefinition:
"""Register the governed.release.evidence.produce operation (lookup-only registry).
This is the *legacy closure assembler* entrypoint. V2R2C2 Round-3 keeps it
registered for backward compatibility with the existing runtime chain; the
governed production path now resolves
``governed_authenticated_evidence_operation.produce_authenticated_evidence_closure``
(see ``authenticated_producer_operation_definition``).
"""
return OperationDefinition.from_dict(
{
"operation_id": "governed.release.evidence.produce",
"version": 1,
"effect_class": "RECONCILABLE",
"handler_ref": "governed_release_evidence_producer.produce_evidence_closure",
"input_schema": PRODUCE_SCHEMA,
"required_permissions": ["governed.release.evidence.produce"],
"timeout_seconds": 60.0,
"readback": {"mode": "STATE", "required": True},
"retry": {"mode": "NEVER"},
"reconcile": {"mode": "EXACT_READBACK"},
"rollback": {"mode": "COMPENSATE"},
"owner_boundary": {"mode": "WITHIN_GRANT"},
}
)
AUTHENTICATED_EVIDENCE_HANDLER_REF = (
"governed_authenticated_evidence_operation.produce_authenticated_evidence_closure"
)
def authenticated_producer_operation_definition() -> OperationDefinition:
"""Register the governed, provider-authenticated release evidence operation.
V2R2C2 Round-3 production entrypoint. Its ``handler_ref`` points at the
authenticated wrapper, which refuses to invoke the legacy closure assembler
until every final-release provider receipt has been resolved from an
authenticated store (Gate-W only authenticates tests/Gitea/secret-scan;
canary/rollback require Gate-R). See Round-3 Provider Authentication.
"""
return OperationDefinition.from_dict(
{
"operation_id": "governed.authenticated.release.evidence.produce",
"version": 1,
"effect_class": "RECONCILABLE",
"handler_ref": AUTHENTICATED_EVIDENCE_HANDLER_REF,
"input_schema": PRODUCE_SCHEMA,
"required_permissions": ["governed.release.evidence.produce"],
"timeout_seconds": 60.0,
"readback": {"mode": "STATE", "required": True},
"retry": {"mode": "NEVER"},
"reconcile": {"mode": "EXACT_READBACK"},
"rollback": {"mode": "COMPENSATE"},
"owner_boundary": {"mode": "WITHIN_GRANT"},
}
)
def producer_registry() -> OperationRegistry:
"""Union of the legacy assembler and the authenticated production operation."""
return OperationRegistry(
[producer_operation_definition(), authenticated_producer_operation_definition()]
)
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("operation", choices=("produce",))
parser.add_argument("--version", required=True)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--source-root", required=True, type=Path)
parser.add_argument("--artifacts-json", required=True, type=Path)
arguments = parser.parse_args()
try:
payload = json.loads(arguments.artifacts_json.read_text(encoding="utf-8"))
request = GovernedBuildArtifacts(
version=arguments.version,
candidate_id=payload["candidate_id"],
source_commit=payload["source_commit"],
tree_oid=payload["tree_oid"],
artifact=bytes.fromhex(payload["artifact_hex"]),
manifest=payload["manifest"],
package_closure=bytes.fromhex(payload["package_closure_hex"]),
policy=payload["policy"],
tests=payload["tests"],
canary=payload["canary"],
gitea=payload["gitea"],
rollback=payload["rollback"],
secret_scan=payload["secret_scan"],
deployment=payload["deployment"],
tag=payload["tag"],
finalization=payload["finalization"],
rollback_sha256=payload.get("rollback_sha256", ""),
)
result = produce_evidence_closure(
builder_policy=arguments.builder_policy,
source_root=arguments.source_root,
request=request,
)
except (EvidenceProduceError, KeyError, ValueError, OSError, json.JSONDecodeError) as exc:
raise SystemExit(str(exc)) from exc
from dataclasses import asdict
print(json.dumps(asdict(result), sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
+6 -2
View File
@@ -34,6 +34,8 @@ from gitea_tag_publication import (
TagPublisher,
)
from release_version import validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -44,8 +46,10 @@ def load_request() -> dict[str, Any]:
raise ValueError("request_object_required")
if value.get("schema") != "aios.production-executor-request.v1":
raise ValueError("request_schema_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
try:
validate_release_version(value.get("version", ""))
except ValueError as exc:
raise ValueError("version_invalid") from exc
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
-456
View File
@@ -1,456 +0,0 @@
from __future__ import annotations
import dataclasses
import datetime as dt
import hashlib
import json
import sqlite3
import re
from pathlib import Path
from typing import Any, Iterable, Mapping
UTC = dt.timezone.utc
_ALLOWED_CONFIDENTIALITY = frozenset({"public", "tenant", "project", "private", "run"})
_PRIVILEGED = frozenset({"*", "admin", "system.admin", "message.admin", "task.admin"})
class HandoffError(RuntimeError):
code = "HANDOFF_ERROR"
class MessageScopeMismatch(HandoffError):
code = "MESSAGE_SCOPE_MISMATCH_DENY"
class ConfidentialityScopeDenied(HandoffError):
code = "MESSAGE_CONFIDENTIALITY_SCOPE_ENFORCE"
class MessageExpired(HandoffError):
code = "MESSAGE_EXPIRED"
class SequenceGap(HandoffError):
code = "MESSAGE_SEQUENCE_GAP_DETECT"
class SequenceConflict(HandoffError):
code = "MESSAGE_SEQUENCE_CONFLICT"
class IdempotencyConflict(HandoffError):
code = "MESSAGE_IDEMPOTENCY_CONFLICT"
def _utcnow() -> dt.datetime:
return dt.datetime.now(UTC)
def _iso(value: dt.datetime | str | None) -> str | None:
if value is None:
return None
parsed = dt.datetime.fromisoformat(value.replace("Z", "+00:00")) if isinstance(value, str) else value
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC).isoformat()
def _canonical(value: Any) -> str:
return json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, default=str)
def _sha(value: Any) -> str:
return hashlib.sha256(_canonical(value).encode("utf-8")).hexdigest()
def _clean(value: Any) -> str | None:
if value is None:
return None
text = str(value).strip()
return text or None
def _tuple_scope(value: Iterable[str] | str | None) -> tuple[str, ...]:
if value is None:
return ()
if isinstance(value, str):
value = value.split(",")
return tuple(sorted({str(item).strip() for item in value if str(item).strip()}))
_SECRET_KEY = re.compile(r"(?i)(?:token|secret|password|private[_-]?key|api[_-]?key)")
_SECRET_VALUE = re.compile(r"(?i)bearer\s+[A-Za-z0-9._~+/-]{8,}|(?:token|secret|password|api[_-]?key)\s*[:=]\s*\S+")
_SAFE_REFERENCE = re.compile(r"^(?:systemd-credential|vault|secret-manager):[A-Za-z0-9][A-Za-z0-9._:/#-]{0,255}$")
def assert_payload_secret_safe(value: Any, *, key: str = "") -> None:
if isinstance(value, Mapping):
for child_key, child in value.items():
assert_payload_secret_safe(child, key=str(child_key))
return
if isinstance(value, (list, tuple)):
for child in value:
assert_payload_secret_safe(child, key=key)
return
if isinstance(value, (bytes, bytearray)):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if isinstance(value, str):
if _SECRET_VALUE.search(value):
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
if _SECRET_KEY.search(key) and value and _SAFE_REFERENCE.fullmatch(value) is None:
raise ValueError("CREDENTIAL_SECRET_EXPOSURE_DENY")
@dataclasses.dataclass(frozen=True)
class DeliveryContext:
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
run_id: str | None = None
capabilities: tuple[str, ...] = ()
@classmethod
def from_actor(cls, actor: Any) -> "DeliveryContext":
def get(name: str) -> Any:
if isinstance(actor, Mapping):
return actor.get(name)
return getattr(actor, name, None)
return cls(
tenant_id=_clean(get("tenant_id")),
user_id=_clean(get("user_id")),
project_id=_clean(get("project_id")),
conversation_id=_clean(get("conversation_id")),
run_id=_clean(get("run_id")),
capabilities=_tuple_scope(get("capabilities") or get("capability_scope")),
)
@property
def privileged(self) -> bool:
return bool(set(self.capabilities) & _PRIVILEGED)
@dataclasses.dataclass(frozen=True)
class MessageEnvelope:
source: str
target: str
message_type: str
payload: Mapping[str, Any]
sequence: int
idempotency_key: str
tenant_id: str | None = None
user_id: str | None = None
project_id: str | None = None
conversation_id: str | None = None
task_id: str | None = None
run_id: str | None = None
correlation_id: str | None = None
causation_id: str | None = None
capability_scope: tuple[str, ...] = ()
confidentiality_scope: str = "private"
expires_at: str | None = None
created_at: str = dataclasses.field(default_factory=lambda: _iso(_utcnow()) or "")
envelope_id: str = ""
def __post_init__(self) -> None:
assert_payload_secret_safe(self.payload)
for field in ("source", "target", "message_type", "idempotency_key", "tenant_id", "user_id",
"project_id", "conversation_id", "task_id", "run_id", "correlation_id", "causation_id"):
object.__setattr__(self, field, _clean(getattr(self, field)) if field not in {
"source", "target", "message_type", "idempotency_key"
} else (_clean(getattr(self, field)) or ""))
object.__setattr__(self, "capability_scope", _tuple_scope(self.capability_scope))
object.__setattr__(self, "confidentiality_scope", (_clean(self.confidentiality_scope) or "private").lower())
object.__setattr__(self, "expires_at", _iso(self.expires_at))
object.__setattr__(self, "created_at", _iso(self.created_at) or "")
if not self.source or not self.target or not self.message_type or not self.idempotency_key:
raise ValueError("MESSAGE_ENVELOPE_IDENTITY_REQUIRED")
if not isinstance(self.sequence, int) or self.sequence < 1:
raise ValueError("MESSAGE_SEQUENCE_MUST_BE_POSITIVE")
if self.confidentiality_scope not in _ALLOWED_CONFIDENTIALITY:
raise ValueError("MESSAGE_CONFIDENTIALITY_SCOPE_INVALID")
if self.confidentiality_scope != "public" and self.tenant_id is None:
raise ValueError("MESSAGE_TENANT_REQUIRED")
if self.confidentiality_scope in {"project", "private", "run"} and self.project_id is None:
raise ValueError("MESSAGE_PROJECT_REQUIRED")
if self.confidentiality_scope in {"private", "run"} and self.user_id is None:
raise ValueError("MESSAGE_USER_REQUIRED")
if self.confidentiality_scope == "run" and self.run_id is None:
raise ValueError("MESSAGE_RUN_REQUIRED")
expected = stable_event_id(
source=self.source, target=self.target, message_type=self.message_type, payload=self.payload,
tenant_id=self.tenant_id, user_id=self.user_id, project_id=self.project_id,
conversation_id=self.conversation_id, task_id=self.task_id, run_id=self.run_id,
correlation_id=self.correlation_id, causation_id=self.causation_id,
sequence=self.sequence, idempotency_key=self.idempotency_key,
)
if self.envelope_id and self.envelope_id != expected:
raise ValueError("MESSAGE_ENVELOPE_ID_MISMATCH")
object.__setattr__(self, "envelope_id", expected)
@property
def payload_sha256(self) -> str:
return _sha(self.payload)
@property
def stream_key(self) -> str:
return _sha({
"tenant_id": self.tenant_id, "source": self.source, "target": self.target,
"conversation_id": self.conversation_id, "task_id": self.task_id, "run_id": self.run_id,
})
def is_expired(self, *, now: dt.datetime | None = None) -> bool:
if self.expires_at is None:
return False
return dt.datetime.fromisoformat(self.expires_at) <= (now or _utcnow()).astimezone(UTC)
def as_record(self) -> dict[str, Any]:
return {
"envelope_id": self.envelope_id, "stream_key": self.stream_key, "source": self.source,
"target": self.target, "message_type": self.message_type, "payload_json": _canonical(self.payload),
"payload_sha256": self.payload_sha256, "tenant_id": self.tenant_id, "user_id": self.user_id,
"project_id": self.project_id, "conversation_id": self.conversation_id, "task_id": self.task_id,
"run_id": self.run_id, "correlation_id": self.correlation_id,
"causation_id": self.causation_id, "sequence": self.sequence,
"capability_scope_json": _canonical(self.capability_scope),
"confidentiality_scope": self.confidentiality_scope, "idempotency_key": self.idempotency_key,
"expires_at": self.expires_at, "created_at": self.created_at,
}
def stable_event_id(*, source: Any, target: Any, message_type: Any, payload: Any,
tenant_id: Any = None, user_id: Any = None, project_id: Any = None,
conversation_id: Any = None, task_id: Any = None, run_id: Any = None,
correlation_id: Any = None, causation_id: Any = None,
sequence: Any = None, idempotency_key: Any = None) -> str:
assert_payload_secret_safe(payload)
material = {
"source": _clean(source), "target": _clean(target), "message_type": _clean(message_type),
"payload_sha256": _sha(payload), "tenant_id": _clean(tenant_id), "user_id": _clean(user_id),
"project_id": _clean(project_id), "conversation_id": _clean(conversation_id),
"task_id": _clean(task_id), "run_id": _clean(run_id), "causation_id": _clean(causation_id),
"sequence": sequence, "idempotency_key": _clean(idempotency_key),
}
# Preserve historical envelope IDs for legacy rows that predate correlation_id.
if _clean(correlation_id) is not None:
material["correlation_id"] = _clean(correlation_id)
return _sha(material)
def bind_envelope_scope(envelope: MessageEnvelope, actor: Any) -> MessageEnvelope:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return envelope
if ctx.tenant_id is None:
if envelope.tenant_id is not None:
raise MessageScopeMismatch(MessageScopeMismatch.code)
return envelope
updates: dict[str, Any] = {}
for field in ("tenant_id", "user_id", "project_id", "conversation_id", "run_id"):
trusted = getattr(ctx, field)
requested = getattr(envelope, field)
if requested is not None and requested != trusted:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:{field}")
if requested is None and trusted is not None:
updates[field] = trusted
if not updates:
return envelope
data = dataclasses.asdict(envelope)
data.update(updates)
data["envelope_id"] = ""
return MessageEnvelope(**data)
def authorize_delivery(actor: Any, envelope: MessageEnvelope) -> None:
ctx = DeliveryContext.from_actor(actor)
if ctx.privileged:
return
if not set(envelope.capability_scope).issubset(set(ctx.capabilities)):
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:capability")
if envelope.confidentiality_scope == "public":
return
if ctx.tenant_id != envelope.tenant_id:
raise MessageScopeMismatch(f"{MessageScopeMismatch.code}:tenant_id")
if envelope.confidentiality_scope in {"project", "private", "run"} and ctx.project_id != envelope.project_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:project_id")
if envelope.confidentiality_scope in {"private", "run"} and ctx.user_id != envelope.user_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:user_id")
if envelope.confidentiality_scope == "run" and ctx.run_id != envelope.run_id:
raise ConfidentialityScopeDenied(f"{ConfidentialityScopeDenied.code}:run_id")
_SCHEMA = """
CREATE TABLE IF NOT EXISTS message_handoff (
envelope_id TEXT PRIMARY KEY,
stream_key TEXT NOT NULL,
source TEXT NOT NULL,
target TEXT NOT NULL,
message_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
payload_sha256 TEXT NOT NULL,
tenant_id TEXT,
user_id TEXT,
project_id TEXT,
conversation_id TEXT,
task_id TEXT,
run_id TEXT,
correlation_id TEXT,
causation_id TEXT,
sequence INTEGER NOT NULL CHECK(sequence >= 1),
capability_scope_json TEXT NOT NULL,
confidentiality_scope TEXT NOT NULL,
idempotency_key TEXT NOT NULL,
expires_at TEXT,
created_at TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'PENDING',
delivered_at TEXT,
acked_at TEXT,
UNIQUE(stream_key, idempotency_key),
UNIQUE(stream_key, sequence)
);
CREATE INDEX IF NOT EXISTS idx_message_handoff_target_status
ON message_handoff(target, status, created_at);
CREATE INDEX IF NOT EXISTS idx_message_handoff_scope
ON message_handoff(tenant_id, project_id, user_id, conversation_id, run_id);
"""
class DurableHandoffStore:
def __init__(self, path: str | Path) -> None:
self.path = Path(path)
self.path.parent.mkdir(parents=True, exist_ok=True)
with self._connect() as conn:
conn.executescript(_SCHEMA)
columns = {str(row[1]) for row in conn.execute("PRAGMA table_info(message_handoff)")}
if "correlation_id" not in columns:
conn.execute("ALTER TABLE message_handoff ADD COLUMN correlation_id TEXT")
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_message_handoff_lineage "
"ON message_handoff(correlation_id, causation_id)"
)
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.path, timeout=30.0)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys=ON")
conn.execute("PRAGMA busy_timeout=30000")
return conn
@staticmethod
def _from_row(row: sqlite3.Row) -> MessageEnvelope:
return MessageEnvelope(
source=row["source"], target=row["target"], message_type=row["message_type"],
payload=json.loads(row["payload_json"]), tenant_id=row["tenant_id"], user_id=row["user_id"],
project_id=row["project_id"], conversation_id=row["conversation_id"], task_id=row["task_id"],
run_id=row["run_id"], correlation_id=row["correlation_id"],
causation_id=row["causation_id"], sequence=int(row["sequence"]),
capability_scope=tuple(json.loads(row["capability_scope_json"])),
confidentiality_scope=row["confidentiality_scope"], idempotency_key=row["idempotency_key"],
expires_at=row["expires_at"], created_at=row["created_at"], envelope_id=row["envelope_id"],
)
def put(self, envelope: MessageEnvelope) -> tuple[str, MessageEnvelope]:
if envelope.is_expired():
raise MessageExpired(MessageExpired.code)
record = envelope.as_record()
with self._connect() as conn:
conn.execute("BEGIN IMMEDIATE")
replay = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND idempotency_key=?",
(envelope.stream_key, envelope.idempotency_key),
).fetchone()
if replay is not None:
existing = self._from_row(replay)
if existing.envelope_id != envelope.envelope_id or existing.payload_sha256 != envelope.payload_sha256:
raise IdempotencyConflict(IdempotencyConflict.code)
conn.commit()
return "REPLAY", existing
seq = conn.execute(
"SELECT * FROM message_handoff WHERE stream_key=? AND sequence=?",
(envelope.stream_key, envelope.sequence),
).fetchone()
if seq is not None:
existing = self._from_row(seq)
if existing.envelope_id == envelope.envelope_id:
conn.commit()
return "REPLAY", existing
raise SequenceConflict(SequenceConflict.code)
maximum = int(conn.execute(
"SELECT COALESCE(MAX(sequence),0) FROM message_handoff WHERE stream_key=?",
(envelope.stream_key,),
).fetchone()[0])
expected = maximum + 1
if envelope.sequence != expected:
raise SequenceGap(f"{SequenceGap.code}:expected={expected}:actual={envelope.sequence}")
columns = tuple(record)
placeholders = ",".join("?" for _ in columns)
conn.execute(
f"INSERT INTO message_handoff ({','.join(columns)}) VALUES ({placeholders})",
tuple(record[column] for column in columns),
)
conn.commit()
return "CREATED", envelope
def get(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
with self._connect() as conn:
row = conn.execute("SELECT * FROM message_handoff WHERE envelope_id=?", (envelope_id,)).fetchone()
if row is None:
raise KeyError(envelope_id)
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope_id,))
conn.commit()
raise MessageExpired(MessageExpired.code)
authorize_delivery(actor, envelope)
return envelope
def pending(self, *, target: str, actor: Any, limit: int = 100) -> list[MessageEnvelope]:
if not 1 <= limit <= 1000:
raise ValueError("MESSAGE_PENDING_LIMIT_INVALID")
visible: list[MessageEnvelope] = []
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM message_handoff WHERE target=? AND status='PENDING' ORDER BY created_at, sequence LIMIT ?",
(target, limit),
).fetchall()
for row in rows:
envelope = self._from_row(row)
if envelope.is_expired():
conn.execute("UPDATE message_handoff SET status='EXPIRED' WHERE envelope_id=?", (envelope.envelope_id,))
continue
try:
authorize_delivery(actor, envelope)
except HandoffError:
continue
visible.append(envelope)
conn.commit()
return visible
def mark_delivered(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='DELIVERED', delivered_at=? "
"WHERE envelope_id=? AND status='PENDING'",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
def acknowledge(self, envelope_id: str, *, actor: Any) -> MessageEnvelope:
envelope = self.get(envelope_id, actor=actor)
with self._connect() as conn:
conn.execute(
"UPDATE message_handoff SET status='ACKED', acked_at=? WHERE envelope_id=?",
(_iso(_utcnow()), envelope_id),
)
conn.commit()
return envelope
class HandoffFabric:
def __init__(self, store: DurableHandoffStore) -> None:
self.store = store
def publish(self, envelope: MessageEnvelope, *, actor: Any | None = None) -> tuple[str, MessageEnvelope]:
return self.store.put(bind_envelope_scope(envelope, actor) if actor is not None else envelope)
@@ -1,68 +0,0 @@
from __future__ import annotations
from typing import Any, Mapping
from message_handoff import MessageEnvelope
def _get(value: Any, *names: str, default: Any = None) -> Any:
for name in names:
if isinstance(value, Mapping) and name in value:
return value[name]
if hasattr(value, name):
return getattr(value, name)
return default
def agent_event_to_envelope(event: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(event, "payload", "data", default={}) or {}
target = target or _get(event, "target", "recipient")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
return MessageEnvelope(
source=str(_get(event, "source", "producer", "agent_id", default="agent-event")),
target=str(target),
message_type=str(_get(event, "event_type", "type", "message_type", default="agent.event")),
payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or str(_get(event, "id", "event_id", default="")) or f"agent-event:{sequence}",
causation_id=_get(event, "causation_id"),
**scope,
)
def taskplane_event_to_envelope(event: Any, *, target: str, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
data = _get(event, "data", "payload", default={}) or {}
event_id = str(_get(event, "id", "event_id", default=""))
return MessageEnvelope(
source=str(_get(event, "source", default="task-plane")),
target=target,
message_type=str(_get(event, "type", "event_type", default="task.event")),
payload=dict(data),
sequence=sequence,
idempotency_key=idempotency_key or event_id or f"task-event:{sequence}",
causation_id=event_id or None,
**scope,
)
def legacy_message_to_envelope(message: Any, *, target: str | None = None, sequence: int = 1,
idempotency_key: str | None = None, **scope: Any) -> MessageEnvelope:
payload = _get(message, "payload", "data", "body", "content", default={})
if not isinstance(payload, Mapping):
payload = {"value": payload}
target = target or _get(message, "target", "recipient", "receiver")
if not target:
raise ValueError("MESSAGE_TARGET_REQUIRED")
source = _get(message, "source", "sender", "agent_id", default="legacy-message-bus")
kind = _get(message, "message_type", "type", "kind", default="legacy.message")
identifier = _get(message, "id", "message_id", "event_id")
return MessageEnvelope(
source=str(source), target=str(target), message_type=str(kind), payload=dict(payload),
sequence=sequence,
idempotency_key=idempotency_key or (str(identifier) if identifier else f"legacy:{sequence}"),
causation_id=str(identifier) if identifier else None,
**scope,
)
@@ -25,8 +25,17 @@ class ReconciliationController:
def reconcile(self, actual: Mapping[str, Any]) -> ReconciliationReport:
drift: list[str] = []
for left, right, label in self.CHECKS:
if actual.get(left) != actual.get(right):
left_value = actual.get(left)
right_value = actual.get(right)
expected_type = bool if left == "declared_production_state" else str
complete = (
left in actual
and right in actual
and isinstance(left_value, expected_type)
and isinstance(right_value, expected_type)
and (expected_type is bool or bool(left_value.strip()) and bool(right_value.strip()))
)
if not complete or left_value != right_value:
drift.append(label)
actions = () if not drift else ("open_reconciliation_task", "open_reconciliation_pr", "attach_evidence", "notify_owner")
return ReconciliationReport(not drift, tuple(drift), actions)
@@ -0,0 +1,531 @@
#!/usr/bin/env python3
"""Governed, fail-closed projection of verified release evidence into Builder storage.
This module never invents candidate/release evidence. A separate governed release
producer must first write a complete, root-owned evidence closure below
``source_root/<version>``. The materializer re-validates that closure with the
same ``ServerOwnedEvidenceSource``/``DeterministicEvidenceCollector`` code used by
Trusted Candidate Builder and atomically projects the exact bytes into the
Builder-owned evidence root.
Readiness is deliberately orthogonal: an empty Builder evidence root does not make
Builder infrastructure globally unavailable, but every BuildTrustedCandidateV1
request still requires a valid materialized closure.
"""
from __future__ import annotations
import argparse
from dataclasses import replace
import hashlib
import json
import os
from pathlib import Path
import secrets
import shutil
import stat
from typing import Any, Callable, Mapping
import subprocess
from aios_trusted_candidate_builder import (
BuilderError,
DeterministicEvidenceCollector,
ServerOwnedEvidenceSource,
TrustedCandidateBuilderConfig,
load_builder_policy,
)
from release_migration_plan import FileMigrationPlanResolver
from release_source_admission import (
ReleaseSourceAdmissionStore,
open_source_admission_store,
source_admission_policy_sha256,
)
from release_version import validate_release_version
MATERIALIZATION_SCHEMA = "aios.trusted-builder-evidence-materialization.v1"
SOURCE_ADMISSION_CREDENTIAL = "source_admission_hmac"
class EvidenceMaterializationError(RuntimeError):
"""Stable, non-sensitive materialization failure."""
def _canonical_json(value: Mapping[str, Any]) -> bytes:
return json.dumps(
value,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _file_sha256(path: Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def _validate_oid(value: object, name: str) -> str:
normalized = str(value or "").strip().lower()
if (
len(normalized) != 40
or any(char not in "0123456789abcdef" for char in normalized)
):
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return normalized
def _validate_identifier(value: object, name: str) -> str:
normalized = str(value or "").strip()
if (
not normalized
or len(normalized) > 255
or normalized[0] in ".-"
or any(
char not in "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789._:-"
for char in normalized
)
):
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return normalized
def _safe_directory(path: Path, *, expected_owner_uid: int | None) -> Path:
candidate = Path(path)
try:
metadata = candidate.stat(follow_symlinks=False)
resolved = candidate.resolve(strict=True)
except OSError as exc:
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_UNAVAILABLE") from exc
if candidate.is_symlink() or not stat.S_ISDIR(metadata.st_mode):
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_INVALID")
if os.name != "nt" and metadata.st_mode & 0o022:
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_PERMISSIONS_INVALID")
if (
os.name != "nt"
and expected_owner_uid is not None
and metadata.st_uid != expected_owner_uid
):
raise EvidenceMaterializationError("EVIDENCE_DIRECTORY_OWNER_INVALID")
return resolved
def _safe_regular(
root: Path,
relative: str,
*,
expected_owner_uid: int | None,
) -> Path:
root = _safe_directory(root, expected_owner_uid=expected_owner_uid)
relative_path = Path(relative)
if relative_path.is_absolute() or any(
part in {"", ".", ".."} for part in relative_path.parts
):
raise EvidenceMaterializationError("EVIDENCE_PATH_INVALID")
current = root
for part in relative_path.parts:
current = current / part
if current.is_symlink():
raise EvidenceMaterializationError("EVIDENCE_SYMLINK_DENIED")
try:
resolved = (root / relative_path).resolve(strict=True)
resolved.relative_to(root)
metadata = resolved.stat(follow_symlinks=False)
except (OSError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_INPUT_UNAVAILABLE") from exc
if not stat.S_ISREG(metadata.st_mode):
raise EvidenceMaterializationError("EVIDENCE_SPECIAL_FILE_DENIED")
if os.name != "nt" and metadata.st_mode & 0o022:
raise EvidenceMaterializationError("EVIDENCE_INPUT_PERMISSIONS_INVALID")
if (
os.name != "nt"
and expected_owner_uid is not None
and metadata.st_uid != expected_owner_uid
):
raise EvidenceMaterializationError("EVIDENCE_INPUT_OWNER_INVALID")
return resolved
def _fsync_directory(path: Path) -> None:
if os.name == "nt":
return
descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def _copy_exact(source: Path, target: Path, *, mode: int = 0o640) -> None:
target.parent.mkdir(parents=True, exist_ok=True, mode=0o750)
descriptor = os.open(
target,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
mode,
)
try:
with source.open("rb") as src:
while True:
chunk = src.read(1024 * 1024)
if not chunk:
break
os.write(descriptor, chunk)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(target, mode)
def _write_json_exclusive(path: Path, payload: Mapping[str, Any]) -> str:
raw = _canonical_json(payload) + b"\n"
descriptor = os.open(
path,
os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
| getattr(os, "O_BINARY", 0),
0o640,
)
try:
os.write(descriptor, raw)
os.fsync(descriptor)
finally:
os.close(descriptor)
os.chmod(path, 0o640)
return hashlib.sha256(raw).hexdigest()
class TrustedBuilderEvidenceMaterializer:
"""Verify a governed evidence closure and atomically project exact bytes."""
def __init__(
self,
*,
builder_policy: Path,
authority_git_dir: Path,
source_root: Path,
output_root: Path,
source_admission_root: Path | None = None,
source_admission_hmac: bytes | None = None,
migration_plan_root: Path | None = None,
migration_profile_root: Path | None = None,
expected_owner_uid: int | None = 0,
git_runner: Callable[..., Any] = subprocess.run,
source_admission_resolver: Callable[[str, str | None, str], Mapping[str, str]] | None = None,
migration_plan_resolver: Callable[[str, str, str], Any] | None = None,
) -> None:
self.builder_policy = Path(builder_policy)
self.authority_git_dir = Path(authority_git_dir)
self.source_root = Path(source_root)
self.output_root = Path(output_root)
self.expected_owner_uid = expected_owner_uid
self.git_runner = git_runner
try:
policy = load_builder_policy(self.builder_policy)
config = TrustedCandidateBuilderConfig.from_policy(policy)
except BuilderError as exc:
raise EvidenceMaterializationError(str(exc)) from exc
self.source_admission_store: ReleaseSourceAdmissionStore | None = None
if source_admission_resolver is not None:
config = replace(
config,
source_admission_resolver=source_admission_resolver,
)
elif source_admission_root is not None:
if not source_admission_hmac or len(source_admission_hmac) < 32:
raise EvidenceMaterializationError("SOURCE_ADMISSION_HMAC_UNAVAILABLE")
self.source_admission_store = open_source_admission_store(
Path(source_admission_root),
repository=config.repository,
policy=self.builder_policy,
hmac_key=source_admission_hmac,
expected_owner_uid=expected_owner_uid,
writable=False,
)
config = replace(
config,
source_admission_resolver=self.source_admission_store.resolve_recipe,
)
if migration_plan_resolver is not None:
config = replace(config, migration_plan_resolver=migration_plan_resolver)
elif migration_plan_root is not None or migration_profile_root is not None:
if migration_plan_root is None or migration_profile_root is None:
raise EvidenceMaterializationError(
"MIGRATION_PLAN_CONFIGURATION_INVALID"
)
resolver = FileMigrationPlanResolver(
Path(migration_plan_root),
profile_root=Path(migration_profile_root),
expected_owner_uid=expected_owner_uid,
)
config = replace(config, migration_plan_resolver=resolver.resolve)
self.config = config
@property
def policy_file_sha256(self) -> str:
return source_admission_policy_sha256(self.builder_policy)
def _collector_for(self, evidence_root: Path) -> DeterministicEvidenceCollector:
source = ServerOwnedEvidenceSource(
self.config,
self.authority_git_dir,
evidence_root,
expected_owner_uid=self.expected_owner_uid,
git_runner=self.git_runner,
)
return DeterministicEvidenceCollector(
self.config,
evidence_source=source,
)
def _relative_files(self, version: str) -> tuple[str, ...]:
relative = [f"{version}/receipt.json"]
relative.extend(
template.format(version=version)
for template in self.config.evidence_layout.values()
)
return tuple(dict.fromkeys(relative))
def _validate_existing(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
expected_bundle: Mapping[str, Any],
) -> dict[str, Any]:
try:
observed = self._collector_for(self.output_root).collect(
version,
self.config.resolve_recipe(version, candidate_id, source_commit),
)
except (BuilderError, RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT") from exc
if _canonical_json(dict(observed)) != _canonical_json(dict(expected_bundle)):
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
record_path = self.output_root / version / "materialization.json"
if not record_path.is_file():
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
try:
record = json.loads(record_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT") from exc
if not isinstance(record, Mapping):
raise EvidenceMaterializationError("EVIDENCE_DESTINATION_CONFLICT")
return dict(record)
def materialize(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
) -> dict[str, Any]:
try:
version = validate_release_version(version)
except ValueError as exc:
raise EvidenceMaterializationError("VERSION_INVALID") from exc
candidate_id = _validate_identifier(candidate_id, "candidate_id")
source_commit = _validate_oid(source_commit, "source_commit")
_safe_directory(
self.source_root,
expected_owner_uid=self.expected_owner_uid,
)
_safe_directory(
self.output_root,
expected_owner_uid=self.expected_owner_uid,
)
try:
recipe = self.config.resolve_recipe(
version,
candidate_id,
source_commit,
)
bundle = self._collector_for(self.source_root).collect(
version,
recipe,
)
except BuilderError as exc:
raise EvidenceMaterializationError(str(exc)) from exc
except (RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError("EVIDENCE_SOURCE_INVALID") from exc
admission_id = ""
if self.source_admission_store is not None:
try:
admission_id = str(
self.source_admission_store.lookup_source(source_commit)[
"admission_id"
]
)
except Exception as exc:
raise EvidenceMaterializationError(
"SOURCE_ADMISSION_INVALID"
) from exc
receipt_path = _safe_regular(
self.source_root,
f"{version}/receipt.json",
expected_owner_uid=self.expected_owner_uid,
)
receipt_sha256 = _file_sha256(receipt_path)
release_plan = dict(bundle["release_plan"])
migration = release_plan.get("migration")
migration_sha256 = ""
if isinstance(migration, Mapping):
migration_sha256 = str(migration.get("sha256") or "")
materialization_payload = {
"schema": MATERIALIZATION_SCHEMA,
"repository": self.config.repository,
"version": version,
"candidate_id": candidate_id,
"source_commit": source_commit,
"tree_oid": str(bundle["candidate"]["tree_oid"]),
"source_admission_id": admission_id,
"builder_policy_file_sha256": self.policy_file_sha256,
"receipt_sha256": receipt_sha256,
"migration_sha256": migration_sha256,
"evidence_digests": dict(bundle["evidence_digests"]),
}
materialization_id = hashlib.sha256(
_canonical_json(materialization_payload)
).hexdigest()
materialization_payload["materialization_id"] = (
"evidence-materialization-" + materialization_id[:24]
)
target = self.output_root / version
if target.exists() or target.is_symlink():
existing = self._validate_existing(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
expected_bundle=bundle,
)
if (
existing.get("materialization_id")
!= materialization_payload["materialization_id"]
):
raise EvidenceMaterializationError(
"EVIDENCE_DESTINATION_CONFLICT"
)
return {
**materialization_payload,
"status": "EVIDENCE_MATERIALIZED",
"idempotent": True,
}
temporary = self.output_root / (
f".{version}.{os.getpid()}.{secrets.token_hex(8)}.tmp"
)
if temporary.exists() or temporary.is_symlink():
raise EvidenceMaterializationError("EVIDENCE_TEMP_CONFLICT")
temporary.mkdir(mode=0o750)
try:
for relative in self._relative_files(version):
source = _safe_regular(
self.source_root,
relative,
expected_owner_uid=self.expected_owner_uid,
)
destination_relative = Path(relative).relative_to(version)
destination = temporary / destination_relative
_copy_exact(source, destination)
if _file_sha256(destination) != _file_sha256(source):
raise EvidenceMaterializationError(
"EVIDENCE_COPY_DIGEST_MISMATCH"
)
materialization_sha256 = _write_json_exclusive(
temporary / "materialization.json",
materialization_payload,
)
_fsync_directory(temporary)
os.replace(temporary, target)
_fsync_directory(self.output_root)
except Exception:
shutil.rmtree(temporary, ignore_errors=True)
raise
try:
observed = self._collector_for(self.output_root).collect(
version,
recipe,
)
except (BuilderError, RuntimeError, ValueError) as exc:
raise EvidenceMaterializationError(
"EVIDENCE_POST_WRITE_VERIFY_FAILED"
) from exc
if _canonical_json(dict(observed)) != _canonical_json(dict(bundle)):
raise EvidenceMaterializationError(
"EVIDENCE_POST_WRITE_VERIFY_FAILED"
)
return {
**materialization_payload,
"materialization_sha256": materialization_sha256,
"status": "EVIDENCE_MATERIALIZED",
"idempotent": False,
}
def _credential(name: str) -> bytes:
root = os.environ.get("CREDENTIALS_DIRECTORY", "").strip()
if not root:
raise EvidenceMaterializationError("CREDENTIALS_DIRECTORY_REQUIRED")
path = Path(root) / name
try:
value = path.read_bytes().strip()
except OSError as exc:
raise EvidenceMaterializationError(
f"{name.upper()}_UNAVAILABLE"
) from exc
if len(value) < 32:
raise EvidenceMaterializationError(f"{name.upper()}_INVALID")
return value
def _main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("operation", choices=("materialize",))
parser.add_argument("--version", required=True)
parser.add_argument("--candidate-id", required=True)
parser.add_argument("--source-commit", required=True)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--authority-git-dir", required=True, type=Path)
parser.add_argument("--source-root", required=True, type=Path)
parser.add_argument("--output-root", required=True, type=Path)
parser.add_argument("--source-admission-root", required=True, type=Path)
parser.add_argument("--migration-plan-root", required=True, type=Path)
parser.add_argument("--migration-profile-root", required=True, type=Path)
arguments = parser.parse_args()
try:
result = TrustedBuilderEvidenceMaterializer(
builder_policy=arguments.builder_policy,
authority_git_dir=arguments.authority_git_dir,
source_root=arguments.source_root,
output_root=arguments.output_root,
source_admission_root=arguments.source_admission_root,
source_admission_hmac=_credential(SOURCE_ADMISSION_CREDENTIAL),
migration_plan_root=arguments.migration_plan_root,
migration_profile_root=arguments.migration_profile_root,
).materialize(
version=arguments.version,
candidate_id=arguments.candidate_id,
source_commit=arguments.source_commit,
)
except EvidenceMaterializationError as exc:
raise SystemExit(str(exc)) from exc
print(json.dumps(result, sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
@@ -27,6 +27,8 @@ import urllib.request
from pathlib import Path
from typing import Any, Mapping
from release_version import validate_release_version
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
PLAN_RE = re.compile(r"^sha256:[0-9a-f]{64}$")
@@ -86,8 +88,10 @@ def load_request() -> dict[str, Any]:
raise ValueError("request_schema_invalid")
if value.get("repository_identity") != "aios/aios":
raise ValueError("repository_identity_invalid")
if not re.fullmatch(r"V\d+R\d+C\d+(?:-[A-Za-z0-9_.-]+)?", str(value.get("version", ""))):
raise ValueError("version_invalid")
try:
validate_release_version(value.get("version", ""))
except ValueError as exc:
raise ValueError("version_invalid") from exc
for name in ("target_oid", "evidence_closure_oid"):
if not OID_RE.fullmatch(str(value.get(name, ""))):
raise ValueError(f"{name}_invalid")
@@ -0,0 +1,989 @@
#!/usr/bin/env python3
"""Fail-closed cross-code-line release migration planning.
The migration plan is data, not a shell script. It binds an admitted source to
the exact currently observed staging code lines, the compatible state contract,
and the rollback/verification contract. The plan can then be embedded in the
existing signed ReleasePlan so Owner approval covers migration semantics without
creating a second release authority.
"""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import sqlite3
import stat
import subprocess
import tempfile
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any, Callable, Mapping, Sequence
OID_RE = re.compile(r"^[0-9a-f]{40}$")
SHA_RE = re.compile(r"^[0-9a-f]{64}$")
IDENT_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:@/-]{2,255}$")
PLAN_SCHEMA = "aios.release-migration-plan.v1"
REQUIREMENT_SCHEMA = "aios.release-migration-requirement.v1"
PROFILE_SCHEMA = "aios.release-migration-profile.v1"
ALLOWED_STATE_MODES = frozenset({"REUSE_IN_PLACE_WITH_SNAPSHOT", "PRESERVE_UNUSED"})
REQUIRED_PRECONDITIONS = (
"writer_disabled",
"task_quiescent",
"state_snapshot_verified",
"credentials_ready",
"rollback_ready",
)
REQUIRED_VERIFICATION_CONTRACTS = (
"runtime_identity_exact",
"task_principal_no_bearer_e2e",
"restart_recovery",
"credential_projection",
"release_consistency_no_unknown",
)
class MigrationPlanError(RuntimeError):
"""Migration evidence is missing, malformed, stale, or unsafe."""
class MigrationPlanRequired(MigrationPlanError):
"""A cross-code-line release cannot proceed without an exact migration plan."""
def _canonical_json(value: object) -> bytes:
return (json.dumps(value, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
def _sha(value: object, label: str) -> str:
text = str(value or "").lower()
if not SHA_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _oid(value: object, label: str) -> str:
text = str(value or "").lower()
if not OID_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _ident(value: object, label: str) -> str:
text = str(value or "")
if not IDENT_RE.fullmatch(text):
raise MigrationPlanError(f"{label}_invalid")
return text
def _secure_json(path: Path, *, expected_owner_uid: int | None = 0) -> dict[str, Any]:
try:
metadata = path.lstat()
except OSError as exc:
raise MigrationPlanError(f"secure_json_unavailable:{path}") from exc
if path.is_symlink() or not stat.S_ISREG(metadata.st_mode):
raise MigrationPlanError("secure_json_regular_file_required")
if expected_owner_uid is not None and os.name != "nt" and metadata.st_uid != expected_owner_uid:
raise MigrationPlanError("secure_json_owner_invalid")
if os.name != "nt" and metadata.st_mode & 0o022:
raise MigrationPlanError("secure_json_mode_invalid")
try:
value = json.loads(path.read_text(encoding="utf-8"))
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise MigrationPlanError("secure_json_invalid") from exc
if not isinstance(value, dict):
raise MigrationPlanError("secure_json_object_required")
return value
def _durable_json(path: Path, value: Mapping[str, Any], *, mode: int = 0o640) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
encoded = _canonical_json(dict(value))
if path.exists():
current = path.read_bytes()
if current != encoded:
raise MigrationPlanError("migration_record_conflict")
return
temporary = path.with_name(path.name + ".tmp")
descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, mode)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, path)
if os.name != "nt":
os.chmod(path, mode)
directory_fd = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
try:
temporary.unlink()
except FileNotFoundError:
pass
def _hash_files(root: Path, files: Sequence[str]) -> str:
payload: dict[str, str] = {}
resolved_root = root.resolve(strict=True)
for relative in files:
rel = Path(relative)
if rel.is_absolute() or ".." in rel.parts:
raise MigrationPlanError("interface_path_invalid")
path = (resolved_root / rel).resolve(strict=True)
try:
path.relative_to(resolved_root)
except ValueError as exc:
raise MigrationPlanError("interface_path_escape") from exc
if path.is_symlink() or not path.is_file():
raise MigrationPlanError("interface_file_invalid")
payload[relative] = hashlib.sha256(path.read_bytes()).hexdigest()
return hashlib.sha256(_canonical_json(payload)).hexdigest()
def _git_blob(git_dir: Path, source_oid: str, relative: str) -> bytes:
rel = Path(relative)
if rel.is_absolute() or ".." in rel.parts:
raise MigrationPlanError("target_interface_path_invalid")
completed = subprocess.run(
["git", f"--git-dir={git_dir}", "show", f"{source_oid}:{relative}"],
check=False,
capture_output=True,
timeout=30,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise MigrationPlanError("target_interface_unavailable")
return bytes(completed.stdout)
def _hash_git_files(git_dir: Path, source_oid: str, files: Sequence[str]) -> str:
payload = {
relative: hashlib.sha256(_git_blob(git_dir, source_oid, relative)).hexdigest()
for relative in files
}
return hashlib.sha256(_canonical_json(payload)).hexdigest()
def _systemd_properties(service: str) -> dict[str, Any]:
_ident(service, "service")
completed = subprocess.run(
[
"systemctl",
"show",
service,
"--property=WorkingDirectory",
"--property=Environment",
"--property=FragmentPath",
"--property=ActiveState",
],
check=False,
capture_output=True,
text=True,
timeout=20,
env={"PATH": os.environ.get("PATH", "")},
)
if completed.returncode != 0:
raise MigrationPlanError("systemd_observation_unavailable")
result: dict[str, Any] = {}
for line in completed.stdout.splitlines():
if "=" in line:
key, value = line.split("=", 1)
result[key] = value
if result.get("ActiveState") != "active":
raise MigrationPlanError("component_not_active")
environment: dict[str, str] = {}
for token in str(result.get("Environment") or "").split():
if "=" in token:
key, value = token.split("=", 1)
environment[key] = value.strip('"')
result["environment"] = environment
return result
def _sqlite_schema(connection: sqlite3.Connection) -> dict[str, Any]:
tables = [
row[0]
for row in connection.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name"
)
]
result: dict[str, Any] = {}
for table in tables:
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", table):
raise MigrationPlanError("sqlite_table_name_invalid")
columns = []
for row in connection.execute(f"PRAGMA table_info({table})"):
columns.append(
{
"cid": int(row[0]),
"name": str(row[1]),
"type": str(row[2]),
"notnull": int(row[3]),
"default": row[4],
"pk": int(row[5]),
}
)
result[table] = {
"columns": columns,
"row_count": int(connection.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]),
}
return result
def _schema_digest(schema: Mapping[str, Any]) -> str:
shape = {
table: {"columns": value["columns"]}
for table, value in sorted(schema.items())
}
return hashlib.sha256(_canonical_json(shape)).hexdigest()
def _assert_additive_sqlite(before: Mapping[str, Any], after: Mapping[str, Any]) -> None:
for table, old in before.items():
if table not in after:
raise MigrationPlanError("sqlite_table_removed")
old_columns = {column["name"]: column for column in old["columns"]}
new_columns = {column["name"]: column for column in after[table]["columns"]}
for name, column in old_columns.items():
if new_columns.get(name) != column:
raise MigrationPlanError("sqlite_column_changed")
for name, column in new_columns.items():
if name in old_columns:
continue
# A new NOT NULL column is backward compatible only with a default.
if column["notnull"] and column["default"] is None:
raise MigrationPlanError("sqlite_new_column_unsafe")
if int(after[table]["row_count"]) != int(old["row_count"]):
raise MigrationPlanError("sqlite_row_count_changed_during_probe")
@dataclass(frozen=True)
class RuntimeComponentBinding:
component_id: str
service: str
code_line_id: str
source_oid: str
artifact_sha256: str | None
interface_sha256: str
def __post_init__(self) -> None:
_ident(self.component_id, "component_id")
_ident(self.service, "service")
_ident(self.code_line_id, "code_line_id")
_oid(self.source_oid, "source_oid")
if self.artifact_sha256 is not None:
_sha(self.artifact_sha256, "artifact_sha256")
_sha(self.interface_sha256, "interface_sha256")
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "RuntimeComponentBinding":
return cls(**dict(value))
@dataclass(frozen=True)
class StateMigrationContract:
state_id: str
state_ref: str
mode: str
before_schema_sha256: str
after_schema_sha256: str
snapshot_name: str
snapshot_sha256: str
snapshot_required: bool
rollback_compatible: bool
def __post_init__(self) -> None:
_ident(self.state_id, "state_id")
_ident(self.state_ref, "state_ref")
if self.mode not in ALLOWED_STATE_MODES:
raise MigrationPlanError("state_mode_invalid")
_sha(self.before_schema_sha256, "before_schema_sha256")
_sha(self.after_schema_sha256, "after_schema_sha256")
_sha(self.snapshot_sha256, "snapshot_sha256")
snapshot = Path(self.snapshot_name)
if (
snapshot.is_absolute()
or ".." in snapshot.parts
or not snapshot.parts
or str(snapshot).startswith(".")
):
raise MigrationPlanError("snapshot_name_invalid")
if self.snapshot_required is not True or self.rollback_compatible is not True:
raise MigrationPlanError("state_recovery_contract_invalid")
def to_dict(self) -> dict[str, Any]:
return asdict(self)
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "StateMigrationContract":
return cls(**dict(value))
@dataclass(frozen=True)
class MigrationPlan:
migration_id: str
version: str
candidate_id: str
repository_identity: str
source_admission_id: str
source_commit: str
tree_oid: str
profile_id: str
profile_sha256: str
from_components: tuple[RuntimeComponentBinding, ...]
to_components: tuple[RuntimeComponentBinding, ...]
state_contracts: tuple[StateMigrationContract, ...]
activation_schema: str
activation_sha256: str
rollback_window_seconds: int
preconditions: tuple[str, ...] = REQUIRED_PRECONDITIONS
verification_contracts: tuple[str, ...] = REQUIRED_VERIFICATION_CONTRACTS
runtime_switch_capability: str = "runtime_switch"
rollback_contract: str = "restore_previous_runtime_and_state_snapshot.v1"
forward_recovery_contract: str = "forward_recovery_after_verified_rollback_block.v1"
writer_expected: bool = False
direct_db_allowed: bool = False
direct_runtime_overwrite_allowed: bool = False
def __post_init__(self) -> None:
_ident(self.migration_id, "migration_id")
_ident(self.version, "version")
_ident(self.candidate_id, "candidate_id")
if self.repository_identity != "aios/aios":
raise MigrationPlanError("repository_identity_invalid")
_ident(self.source_admission_id, "source_admission_id")
_oid(self.source_commit, "source_commit")
_oid(self.tree_oid, "tree_oid")
_ident(self.profile_id, "profile_id")
_sha(self.profile_sha256, "profile_sha256")
_ident(self.activation_schema, "activation_schema")
_sha(self.activation_sha256, "activation_sha256")
if not 60 <= int(self.rollback_window_seconds) <= 86400:
raise MigrationPlanError("rollback_window_invalid")
if tuple(self.preconditions) != REQUIRED_PRECONDITIONS:
raise MigrationPlanError("migration_preconditions_invalid")
if tuple(self.verification_contracts) != REQUIRED_VERIFICATION_CONTRACTS:
raise MigrationPlanError("migration_verification_contracts_invalid")
if self.runtime_switch_capability != "runtime_switch":
raise MigrationPlanError("migration_capability_invalid")
if self.writer_expected or self.direct_db_allowed or self.direct_runtime_overwrite_allowed:
raise MigrationPlanError("migration_unsafe_authority")
if not self.from_components or not self.to_components or not self.state_contracts:
raise MigrationPlanError("migration_contract_incomplete")
from_ids = {item.component_id for item in self.from_components}
to_ids = {item.component_id for item in self.to_components}
if from_ids != to_ids or len(from_ids) != len(self.from_components) or len(to_ids) != len(self.to_components):
raise MigrationPlanError("migration_component_binding_invalid")
if all(
next(item.code_line_id for item in self.from_components if item.component_id == target.component_id)
== target.code_line_id
for target in self.to_components
):
raise MigrationPlanError("migration_not_required")
@property
def canonical_payload(self) -> dict[str, Any]:
return {
"schema": PLAN_SCHEMA,
"migration_id": self.migration_id,
"version": self.version,
"candidate_id": self.candidate_id,
"repository_identity": self.repository_identity,
"source_admission_id": self.source_admission_id,
"source_commit": self.source_commit,
"tree_oid": self.tree_oid,
"profile_id": self.profile_id,
"profile_sha256": self.profile_sha256,
"from_components": [item.to_dict() for item in self.from_components],
"to_components": [item.to_dict() for item in self.to_components],
"state_contracts": [item.to_dict() for item in self.state_contracts],
"activation_schema": self.activation_schema,
"activation_sha256": self.activation_sha256,
"rollback_window_seconds": self.rollback_window_seconds,
"preconditions": list(self.preconditions),
"verification_contracts": list(self.verification_contracts),
"runtime_switch_capability": self.runtime_switch_capability,
"rollback_contract": self.rollback_contract,
"forward_recovery_contract": self.forward_recovery_contract,
"writer_expected": self.writer_expected,
"direct_db_allowed": self.direct_db_allowed,
"direct_runtime_overwrite_allowed": self.direct_runtime_overwrite_allowed,
}
@property
def sha256(self) -> str:
return hashlib.sha256(_canonical_json(self.canonical_payload)).hexdigest()
def to_dict(self) -> dict[str, Any]:
return {**self.canonical_payload, "sha256": self.sha256}
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "MigrationPlan":
raw = dict(value)
if raw.pop("schema", None) != PLAN_SCHEMA:
raise MigrationPlanError("migration_plan_schema_invalid")
claimed = raw.pop("sha256", None)
raw["from_components"] = tuple(RuntimeComponentBinding.from_dict(item) for item in raw["from_components"])
raw["to_components"] = tuple(RuntimeComponentBinding.from_dict(item) for item in raw["to_components"])
raw["state_contracts"] = tuple(StateMigrationContract.from_dict(item) for item in raw["state_contracts"])
raw["preconditions"] = tuple(raw.get("preconditions") or ())
raw["verification_contracts"] = tuple(raw.get("verification_contracts") or ())
plan = cls(**raw)
if claimed is not None and claimed != plan.sha256:
raise MigrationPlanError("migration_plan_sha256_mismatch")
return plan
def assert_release_binding(
self,
*,
version: str,
candidate_id: str,
source_commit: str,
tree_oid: str,
repository_identity: str,
) -> None:
expected = (
self.version,
self.candidate_id,
self.source_commit,
self.tree_oid,
self.repository_identity,
)
actual = (version, candidate_id, source_commit, tree_oid, repository_identity)
if actual != expected:
raise MigrationPlanError("migration_release_binding_mismatch")
@dataclass(frozen=True)
class MigrationRequirement:
version: str
candidate_id: str
source_commit: str
repository_identity: str
required: bool
reason_code: str
profile_id: str
profile_sha256: str
migration_plan_sha256: str | None
@property
def canonical_payload(self) -> dict[str, Any]:
return {
"schema": REQUIREMENT_SCHEMA,
**asdict(self),
}
@property
def sha256(self) -> str:
return hashlib.sha256(_canonical_json(self.canonical_payload)).hexdigest()
def to_dict(self) -> dict[str, Any]:
return {**self.canonical_payload, "sha256": self.sha256}
@classmethod
def from_dict(cls, value: Mapping[str, Any]) -> "MigrationRequirement":
raw = dict(value)
if raw.pop("schema", None) != REQUIREMENT_SCHEMA:
raise MigrationPlanError("migration_requirement_schema_invalid")
claimed = raw.pop("sha256", None)
requirement = cls(**raw)
_ident(requirement.version, "version")
_ident(requirement.candidate_id, "candidate_id")
_oid(requirement.source_commit, "source_commit")
if requirement.repository_identity != "aios/aios":
raise MigrationPlanError("migration_requirement_repository_invalid")
_ident(requirement.reason_code, "reason_code")
_ident(requirement.profile_id, "profile_id")
_sha(requirement.profile_sha256, "profile_sha256")
if requirement.required:
_sha(requirement.migration_plan_sha256, "migration_plan_sha256")
elif requirement.migration_plan_sha256 is not None:
raise MigrationPlanError("unexpected_migration_plan")
if claimed is not None and claimed != requirement.sha256:
raise MigrationPlanError("migration_requirement_sha256_mismatch")
return requirement
class FileMigrationPlanResolver:
"""Read-only resolver used by the trusted builder.
A profile declares that a release version is cross-code-line. Once a
version is declared, absence of requirement/plan state is fail-closed.
"""
def __init__(
self,
root: Path,
*,
profile_root: Path,
expected_owner_uid: int | None = 0,
):
self.root = Path(root)
self.profile_root = Path(profile_root)
self.expected_owner_uid = expected_owner_uid
def _profile_for_version(self, version: str) -> dict[str, Any] | None:
matches = []
if not self.profile_root.is_dir():
return None
for path in sorted(self.profile_root.glob("*.json")):
profile = _secure_json(path, expected_owner_uid=self.expected_owner_uid)
if profile.get("schema") != PROFILE_SCHEMA:
raise MigrationPlanError("migration_profile_schema_invalid")
if profile.get("version") == version:
matches.append(profile)
if len(matches) > 1:
raise MigrationPlanError("migration_profile_ambiguous")
return matches[0] if matches else None
def resolve(self, version: str, candidate_id: str, source_commit: str) -> MigrationPlan | None:
profile = self._profile_for_version(version)
if profile is None:
return None
base = self.root / source_commit
requirement_path = base / "requirement.json"
plan_path = base / "plan.json"
if not requirement_path.is_file():
raise MigrationPlanRequired("MIGRATION_PLAN_REQUIRED")
requirement = MigrationRequirement.from_dict(
_secure_json(requirement_path, expected_owner_uid=self.expected_owner_uid)
)
expected = (version, candidate_id, source_commit, "aios/aios")
actual = (
requirement.version,
requirement.candidate_id,
requirement.source_commit,
requirement.repository_identity,
)
if actual != expected:
raise MigrationPlanError("migration_requirement_binding_mismatch")
if requirement.profile_id != profile.get("profile_id"):
raise MigrationPlanError("migration_profile_binding_mismatch")
profile_sha = hashlib.sha256(_canonical_json(profile)).hexdigest()
if requirement.profile_sha256 != profile_sha:
raise MigrationPlanError("migration_profile_digest_mismatch")
if not requirement.required:
return None
if not plan_path.is_file():
raise MigrationPlanRequired("MIGRATION_PLAN_REQUIRED")
plan = MigrationPlan.from_dict(
_secure_json(plan_path, expected_owner_uid=self.expected_owner_uid)
)
plan.assert_release_binding(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
tree_oid=plan.tree_oid,
repository_identity="aios/aios",
)
if requirement.migration_plan_sha256 != plan.sha256:
raise MigrationPlanError("migration_requirement_plan_mismatch")
return plan
class MigrationPreSwitchValidator:
"""Validates a root-owned live proof against the signed migration plan."""
def __init__(self, proof_resolver: Callable[[MigrationPlan], Mapping[str, Any]]):
self.proof_resolver = proof_resolver
def validate(self, migration_plan: MigrationPlan | None) -> dict[str, Any]:
if migration_plan is None:
return {"required": False, "verified": True}
proof = dict(self.proof_resolver(migration_plan))
snapshot_digests = sorted(
contract.snapshot_sha256 for contract in migration_plan.state_contracts
)
required = {
"migration_plan_sha256": migration_plan.sha256,
"migration_id": migration_plan.migration_id,
"source_commit": migration_plan.source_commit,
"tree_oid": migration_plan.tree_oid,
"activation_sha256": migration_plan.activation_sha256,
"state_snapshot_sha256s": snapshot_digests,
"writer_disabled": True,
"task_quiescent": True,
"state_snapshot_verified": True,
"credentials_ready": True,
"rollback_ready": True,
}
failed = [key for key, expected in required.items() if proof.get(key) != expected]
if failed:
raise MigrationPlanError(
"MIGRATION_PRE_SWITCH_PROOF_INVALID:" + ",".join(sorted(failed))
)
return {"required": True, "verified": True, **proof}
class HostMigrationCompiler:
def __init__(
self,
*,
authority_git_dir: Path,
profile_path: Path,
output_root: Path,
expected_owner_uid: int | None = 0,
):
self.authority_git_dir = Path(authority_git_dir)
self.profile_path = Path(profile_path)
self.output_root = Path(output_root)
self.expected_owner_uid = expected_owner_uid
def _profile(self) -> dict[str, Any]:
profile = _secure_json(self.profile_path, expected_owner_uid=self.expected_owner_uid)
required = {
"schema",
"profile_id",
"version",
"repository",
"current_components",
"target_components",
"activation_file",
"state_ref",
"state_db",
"state_mode",
"rollback_window_seconds",
}
if set(profile) != required or profile.get("schema") != PROFILE_SCHEMA:
raise MigrationPlanError("migration_profile_fields_invalid")
if profile.get("repository") != "aios/aios":
raise MigrationPlanError("migration_profile_repository_invalid")
_ident(profile.get("profile_id"), "profile_id")
_ident(profile.get("version"), "version")
if profile.get("state_mode") not in ALLOWED_STATE_MODES:
raise MigrationPlanError("migration_profile_state_mode_invalid")
return profile
@staticmethod
def _component_from_live(spec: Mapping[str, Any]) -> RuntimeComponentBinding:
required = {
"component_id",
"service",
"code_line_id",
"expected_source_oid",
"source_oid_env",
"artifact_sha_env",
"interface_files",
"marker_files",
}
if set(spec) != required:
raise MigrationPlanError("current_component_profile_invalid")
properties = _systemd_properties(str(spec["service"]))
environment = properties["environment"]
source_oid = _oid(environment.get(str(spec["source_oid_env"])), "current_source_oid")
expected_oid = _oid(spec["expected_source_oid"], "expected_source_oid")
if source_oid != expected_oid:
raise MigrationPlanError("CURRENT_RUNTIME_DRIFT")
working = Path(str(properties.get("WorkingDirectory") or ""))
if not working.is_absolute() or not working.is_dir():
raise MigrationPlanError("current_working_directory_invalid")
for marker in spec["marker_files"]:
if not (working / str(marker)).is_file():
raise MigrationPlanError("current_code_line_marker_missing")
interface = _hash_files(working, tuple(str(item) for item in spec["interface_files"]))
raw_artifact = environment.get(str(spec["artifact_sha_env"]))
artifact = str(raw_artifact or "").lower()
artifact_value = artifact if SHA_RE.fullmatch(artifact) else None
return RuntimeComponentBinding(
component_id=str(spec["component_id"]),
service=str(spec["service"]),
code_line_id=str(spec["code_line_id"]),
source_oid=source_oid,
artifact_sha256=artifact_value,
interface_sha256=interface,
)
def _component_from_target(
self, spec: Mapping[str, Any], *, source_commit: str
) -> RuntimeComponentBinding:
required = {"component_id", "service", "code_line_id", "interface_files"}
if set(spec) != required:
raise MigrationPlanError("target_component_profile_invalid")
interface_files = tuple(str(item) for item in spec["interface_files"])
interface = _hash_git_files(self.authority_git_dir, source_commit, interface_files)
return RuntimeComponentBinding(
component_id=str(spec["component_id"]),
service=str(spec["service"]),
code_line_id=str(spec["code_line_id"]),
source_oid=source_commit,
artifact_sha256=None,
interface_sha256=interface,
)
@staticmethod
def _state_compatibility(
state_db: Path,
state_ref: str,
*,
snapshot_path: Path,
snapshot_name: str,
) -> StateMigrationContract:
from aios_task_admission_bootstrap import TaskAdmissionStore
if state_db.is_symlink() or not state_db.is_file():
raise MigrationPlanError("state_db_invalid")
snapshot_path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="aios-migration-probe-") as temp_dir:
temp_root = Path(temp_dir)
consistent_snapshot = temp_root / "pre-migration.sqlite3"
probe_path = temp_root / "schema-probe.sqlite3"
with sqlite3.connect(
f"file:{state_db}?mode=ro", uri=True
) as live, sqlite3.connect(consistent_snapshot) as copy:
live.backup(copy)
with sqlite3.connect(consistent_snapshot) as frozen:
before = _sqlite_schema(frozen)
# Probe target initialization only on a second private copy. The
# rollback snapshot remains byte-for-byte pre-migration state.
with sqlite3.connect(consistent_snapshot) as source, sqlite3.connect(
probe_path
) as probe:
source.backup(probe)
TaskAdmissionStore(probe_path)
with sqlite3.connect(probe_path) as migrated:
after = _sqlite_schema(migrated)
required_tables = {
"task_admission_envelopes",
"task_admission_grants",
"task_admission_receipts",
}
if not required_tables.issubset(after):
raise MigrationPlanError("INCOMPATIBLE_STATE_SCHEMA")
_assert_additive_sqlite(before, after)
snapshot_bytes = consistent_snapshot.read_bytes()
snapshot_sha = hashlib.sha256(snapshot_bytes).hexdigest()
if snapshot_path.exists():
if snapshot_path.is_symlink() or not snapshot_path.is_file():
raise MigrationPlanError("migration_snapshot_invalid")
if hashlib.sha256(snapshot_path.read_bytes()).hexdigest() != snapshot_sha:
raise MigrationPlanError("migration_snapshot_conflict")
else:
temporary = snapshot_path.with_name(snapshot_path.name + ".tmp")
descriptor = os.open(
temporary,
os.O_WRONLY | os.O_CREAT | os.O_EXCL,
0o640,
)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(snapshot_bytes)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary, snapshot_path)
if os.name != "nt":
os.chmod(snapshot_path, 0o640)
directory_fd = os.open(snapshot_path.parent, os.O_RDONLY)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
finally:
try:
temporary.unlink()
except FileNotFoundError:
pass
return StateMigrationContract(
state_id="task-admission",
state_ref=state_ref,
mode="REUSE_IN_PLACE_WITH_SNAPSHOT",
before_schema_sha256=_schema_digest(before),
after_schema_sha256=_schema_digest(after),
snapshot_name=snapshot_name,
snapshot_sha256=snapshot_sha,
snapshot_required=True,
rollback_compatible=True,
)
def compile(
self,
*,
source_admission: Mapping[str, Any],
candidate_id: str,
) -> tuple[MigrationRequirement, MigrationPlan]:
profile = self._profile()
version = str(source_admission.get("version") or "")
source_commit = _oid(source_admission.get("source_commit"), "source_commit")
tree_oid = _oid(source_admission.get("tree_oid"), "tree_oid")
admission_id = _ident(source_admission.get("admission_id"), "source_admission_id")
if (
version != profile["version"]
or source_admission.get("repository") != "aios/aios"
):
raise MigrationPlanError("source_admission_profile_mismatch")
_ident(candidate_id, "candidate_id")
profile_sha = hashlib.sha256(_canonical_json(profile)).hexdigest()
from_components = tuple(
self._component_from_live(item) for item in profile["current_components"]
)
to_components = tuple(
self._component_from_target(item, source_commit=source_commit)
for item in profile["target_components"]
)
if {item.component_id for item in from_components} != {item.component_id for item in to_components}:
raise MigrationPlanError("migration_component_profile_mismatch")
required = any(
next(old.code_line_id for old in from_components if old.component_id == new.component_id)
!= new.code_line_id
for new in to_components
)
if not required:
raise MigrationPlanError("migration_profile_does_not_require_transition")
activation_path = Path(str(profile["activation_file"]))
activation = _secure_json(activation_path, expected_owner_uid=self.expected_owner_uid)
if activation.get("schema") != "aios.task-admission-host-activation.v1":
raise MigrationPlanError("activation_schema_invalid")
state_path = Path(str(profile["state_db"]))
if str(activation.get("state_db") or "") != str(state_path):
raise MigrationPlanError("activation_state_db_mismatch")
root = self.output_root / source_commit
snapshot_name = "snapshots/task-admission.sqlite3"
state_contract = self._state_compatibility(
state_path,
str(profile["state_ref"]),
snapshot_path=root / snapshot_name,
snapshot_name=snapshot_name,
)
migration_id = (
"migration-"
+ hashlib.sha256(
f"{version}\0{candidate_id}\0{source_commit}\0{profile_sha}".encode("utf-8")
).hexdigest()[:24]
)
plan = MigrationPlan(
migration_id=migration_id,
version=version,
candidate_id=candidate_id,
repository_identity="aios/aios",
source_admission_id=admission_id,
source_commit=source_commit,
tree_oid=tree_oid,
profile_id=str(profile["profile_id"]),
profile_sha256=profile_sha,
from_components=from_components,
to_components=to_components,
state_contracts=(state_contract,),
activation_schema=str(activation["schema"]),
activation_sha256=hashlib.sha256(_canonical_json(activation)).hexdigest(),
rollback_window_seconds=int(profile["rollback_window_seconds"]),
)
requirement = MigrationRequirement(
version=version,
candidate_id=candidate_id,
source_commit=source_commit,
repository_identity="aios/aios",
required=True,
reason_code="CROSS_CODE_LINE",
profile_id=str(profile["profile_id"]),
profile_sha256=profile_sha,
migration_plan_sha256=plan.sha256,
)
_durable_json(root / "requirement.json", requirement.to_dict())
_durable_json(root / "plan.json", plan.to_dict())
return requirement, plan
def _verified_source_admission(
*,
source_admission_root: Path,
source_commit: str,
builder_policy: Path,
credential_name: str = "source_admission_hmac",
) -> dict[str, Any]:
"""Resolve one source admission through its existing HMAC verifier.
The migration planner never trusts a caller-selected admission JSON path.
It consumes only a root-owned admission store and the independent
``source_admission_hmac`` credential projected by systemd.
"""
from release_source_admission import open_source_admission_store
credentials = os.environ.get("CREDENTIALS_DIRECTORY", "").strip()
if not credentials:
raise MigrationPlanError("source_admission_credentials_unavailable")
credential_path = Path(credentials) / credential_name
try:
hmac_key = credential_path.read_bytes().strip()
except OSError as exc:
raise MigrationPlanError("source_admission_hmac_unavailable") from exc
try:
store = open_source_admission_store(
source_admission_root,
repository="aios/aios",
policy=builder_policy,
hmac_key=hmac_key,
expected_owner_uid=0 if os.name != "nt" else None,
writable=False,
)
record = store.lookup_source(source_commit)
except Exception as exc:
raise MigrationPlanError("source_admission_verification_failed") from exc
if record is None:
raise MigrationPlanRequired("SOURCE_ADMISSION_REQUIRED")
return dict(record)
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--authority-git-dir", required=True, type=Path)
parser.add_argument("--profile", required=True, type=Path)
parser.add_argument("--output-root", required=True, type=Path)
parser.add_argument("--source-admission-root", required=True, type=Path)
parser.add_argument("--builder-policy", required=True, type=Path)
parser.add_argument("--source-commit", required=True)
parser.add_argument("--candidate-id", required=True)
args = parser.parse_args()
source_commit = _oid(args.source_commit, "source_commit")
compiler = HostMigrationCompiler(
authority_git_dir=args.authority_git_dir,
profile_path=args.profile,
output_root=args.output_root,
expected_owner_uid=0 if os.name != "nt" else None,
)
requirement, plan = compiler.compile(
source_admission=_verified_source_admission(
source_admission_root=args.source_admission_root,
source_commit=source_commit,
builder_policy=args.builder_policy,
),
candidate_id=args.candidate_id,
)
print(
json.dumps(
{
"status": "MIGRATION_PLAN_READY",
"migration_plan_sha256": plan.sha256,
"requirement_sha256": requirement.sha256,
"source_commit": plan.source_commit,
"version": plan.version,
},
sort_keys=True,
)
)
return 0
if __name__ == "__main__":
try:
raise SystemExit(main())
except Exception as exc:
print(
json.dumps(
{
"status": "FAIL_CLOSED",
"error": type(exc).__name__,
"reason": str(exc),
},
sort_keys=True,
)
)
raise SystemExit(2)
@@ -0,0 +1,894 @@
#!/usr/bin/env python3
"""Governed admission of an exact remote Git commit into the release source closure.
The admitter is intentionally separate from the trusted candidate builder:
it may use a narrowly-scoped Gitea read/fetch credential, while the builder
remains PrivateNetwork=true and reads only the authoritative Git object store
plus immutable admission records.
An admission never moves ``main`` and never treats an arbitrary caller-supplied
OID as trusted. It binds:
remote branch ref -> exact full commit OID -> authoritative Git object/tree
-> current authoritative main ancestry -> immutable release recipe metadata.
"""
from __future__ import annotations
import argparse
import hashlib
import hmac
import json
import os
import re
import tempfile
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Mapping, Protocol
from authoritative_git import (
AuthoritativeGitError,
authoritative_head_oid,
authoritative_lineage,
run_authoritative_git,
)
from aios_gitea_machine_transport import GiteaMachineTransport, GiteaTransportError
from release_version import validate_release_version
SCHEMA = "aios.release-source-admission.v1"
RECIPE_MODE = "governed-source-admission"
_OID_RE = re.compile(r"^[0-9a-f]{40}$")
_DIGEST_RE = re.compile(r"^[0-9a-f]{64}$")
_IDENTIFIER_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._:@/-]{0,199}$")
_BRANCH_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,239}$")
_REF_PREFIX = "refs/heads/"
class ReleaseSourceAdmissionError(RuntimeError):
"""A stable fail-closed source-admission error code."""
def _canonical(value: Mapping[str, Any]) -> bytes:
return json.dumps(
dict(value),
sort_keys=True,
separators=(",", ":"),
ensure_ascii=True,
).encode("utf-8")
def _oid(value: object, field: str) -> str:
text = str(value or "").strip().lower()
if not _OID_RE.fullmatch(text):
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _digest(value: object, field: str) -> str:
text = str(value or "").strip().lower()
if not _DIGEST_RE.fullmatch(text):
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _identifier(value: object, field: str) -> str:
text = str(value or "").strip()
if not _IDENTIFIER_RE.fullmatch(text) or ".." in text or "\x00" in text:
raise ReleaseSourceAdmissionError(f"{field.upper()}_INVALID")
return text
def _source_ref(value: object) -> str:
text = str(value or "").strip()
if not text.startswith(_REF_PREFIX):
raise ReleaseSourceAdmissionError("SOURCE_REF_INVALID")
branch = text[len(_REF_PREFIX):]
if (
not _BRANCH_RE.fullmatch(branch)
or ".." in branch
or "//" in branch
or "@{" in branch
or branch.endswith(("/", ".", ".lock"))
or any(part in {"", ".", ".."} for part in branch.split("/"))
):
raise ReleaseSourceAdmissionError("SOURCE_REF_INVALID")
return text
def _version(value: object) -> str:
try:
return validate_release_version(value)
except (TypeError, ValueError) as exc:
raise ReleaseSourceAdmissionError("VERSION_INVALID") from exc
def _version_slug(version: str) -> str:
value = re.sub(r"[^A-Za-z0-9]+", "-", version.lower()).strip("-")
if not value:
raise ReleaseSourceAdmissionError("VERSION_INVALID")
return value[:80]
def _now() -> str:
return datetime.now(timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
def _secure_stat(path: Path, *, expected_owner_uid: int | None, directory: bool) -> None:
try:
info = path.lstat()
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNAVAILABLE") from exc
if path.is_symlink():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if directory and not path.is_dir():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if not directory and not path.is_file():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNSAFE")
if expected_owner_uid is not None and info.st_uid != expected_owner_uid:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_OWNER_MISMATCH")
# Group/other write would allow an untrusted process to forge release input.
if info.st_mode & 0o022:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_PERMISSIONS_INVALID")
def _validate_record(
value: Mapping[str, Any],
*,
repository: str,
policy_sha256: str,
hmac_key: bytes,
) -> dict[str, Any]:
if not isinstance(value, Mapping):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
required = {
"schema",
"admission_id",
"repository",
"task_id",
"version",
"candidate_id",
"base_oid",
"source_ref",
"source_commit",
"remote_ref_oid",
"tree_oid",
"purpose",
"policy_sha256",
"admitted_at",
"mac",
}
if set(value) != required or value.get("schema") != SCHEMA:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
record = dict(value)
if _identifier(record["repository"], "repository") != repository:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REPOSITORY_MISMATCH")
_identifier(record["task_id"], "task_id")
record["version"] = _version(record["version"])
_identifier(record["candidate_id"], "candidate_id")
_identifier(record["purpose"], "purpose")
_identifier(record["admission_id"], "admission_id")
record["base_oid"] = _oid(record["base_oid"], "base_oid")
record["source_commit"] = _oid(record["source_commit"], "source_commit")
record["remote_ref_oid"] = _oid(record["remote_ref_oid"], "remote_ref_oid")
record["tree_oid"] = _oid(record["tree_oid"], "tree_oid")
record["source_ref"] = _source_ref(record["source_ref"])
record["policy_sha256"] = _digest(record["policy_sha256"], "policy_sha256")
if record["policy_sha256"] != policy_sha256:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_POLICY_MISMATCH")
if record["remote_ref_oid"] != record["source_commit"]:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REMOTE_MISMATCH")
admitted_at = record.get("admitted_at")
if not isinstance(admitted_at, str) or not admitted_at.endswith("Z") or len(admitted_at) < 20:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_TIME_INVALID")
supplied_mac = record.get("mac")
if (
not isinstance(supplied_mac, str)
or not re.fullmatch(r"[0-9a-f]{64}", supplied_mac)
):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_MAC_INVALID")
unsigned = {key: value for key, value in record.items() if key != "mac"}
expected_mac = hmac.new(hmac_key, _canonical(unsigned), hashlib.sha256).hexdigest()
if not hmac.compare_digest(supplied_mac, expected_mac):
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_MAC_INVALID")
return record
class ReleaseSourceBackend(Protocol):
def authoritative_main_oid(self) -> str: ...
def remote_ref_oid(self, source_ref: str) -> str: ...
def fetch_exact_ref(self, source_ref: str, source_commit: str) -> str: ...
def tree_oid(self, source_commit: str) -> str: ...
def require_lineage(self, base_oid: str, source_commit: str) -> None: ...
class ReleaseSourceAdmissionStore:
"""Durable immutable admission records keyed by exact source commit."""
def __init__(
self,
root: str | os.PathLike[str],
*,
repository: str,
policy_sha256: str,
hmac_key: bytes,
expected_owner_uid: int | None = None,
writable: bool = False,
) -> None:
self.root = Path(root)
self.repository = _identifier(repository, "repository")
self.policy_sha256 = _digest(policy_sha256, "policy_sha256")
if not isinstance(hmac_key, (bytes, bytearray)) or len(hmac_key) < 32:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_HMAC_INVALID")
self.hmac_key = bytes(hmac_key)
self.expected_owner_uid = expected_owner_uid
self.writable = bool(writable)
if self.writable and not self.root.exists():
self.root.mkdir(parents=True, mode=0o750)
if self.root.exists():
_secure_stat(
self.root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
def _path(self, source_commit: str) -> Path:
return self.root / f"{_oid(source_commit, 'source_commit')}.json"
def register(self, record: Mapping[str, Any]) -> dict[str, Any]:
if not self.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
supplied = dict(record)
if "mac" not in supplied:
supplied["mac"] = hmac.new(
self.hmac_key,
_canonical(supplied),
hashlib.sha256,
).hexdigest()
checked = _validate_record(
supplied,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
path = self._path(checked["source_commit"])
encoded = _canonical(checked) + b"\n"
if path.exists():
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
existing = path.read_bytes()
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_UNAVAILABLE") from exc
if existing == encoded:
return dict(checked)
try:
previous = _validate_record(
json.loads(existing),
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID") from exc
comparable_previous = {
key: value
for key, value in previous.items()
if key not in {"admitted_at", "mac"}
}
comparable_checked = {
key: value
for key, value in checked.items()
if key not in {"admitted_at", "mac"}
}
if comparable_previous == comparable_checked:
return dict(previous)
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_CONFLICT")
self.root.mkdir(parents=True, exist_ok=True, mode=0o750)
_secure_stat(
self.root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
fd, raw_name = tempfile.mkstemp(
prefix=f".{checked['source_commit']}.",
suffix=".tmp",
dir=self.root,
)
temp_path = Path(raw_name)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
os.replace(temp_path, path)
os.chmod(path, 0o640)
directory_fd = os.open(self.root, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
try:
os.close(fd)
except OSError:
pass
temp_path.unlink(missing_ok=True)
raise
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
return dict(checked)
def rebind_policy(
self,
source_commit: str,
*,
previous_policy_sha256: str,
reason: str,
expected_admission_id: str | None = None,
) -> dict[str, Any]:
"""Rebind one immutable source identity to an explicitly authorized policy.
Only ``policy_sha256`` and ``mac`` change in the admission record.
``admission_id`` and every source/candidate/tree binding remain immutable.
A separate HMAC-authenticated history record makes the temporary §8.1
trust expansion and its restoration auditable and idempotent.
"""
if not self.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
previous_policy = _digest(
previous_policy_sha256,
"previous_policy_sha256",
)
reason_value = _identifier(reason, "reason")
path = self._path(source_commit)
if not path.exists():
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_REQUIRED")
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
raw = path.read_bytes()
decoded = json.loads(raw)
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_RECORD_INVALID"
) from exc
try:
previous = _validate_record(
decoded,
repository=self.repository,
policy_sha256=previous_policy,
hmac_key=self.hmac_key,
)
except ReleaseSourceAdmissionError:
# Idempotent replay after a successful rebind validates under the
# current policy instead of silently weakening the old-policy gate.
current = _validate_record(
decoded,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
if expected_admission_id and current["admission_id"] != expected_admission_id:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_ID_MISMATCH"
)
return dict(current)
if expected_admission_id and previous["admission_id"] != expected_admission_id:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_ID_MISMATCH")
rebound = {
key: value
for key, value in previous.items()
if key != "mac"
}
rebound["policy_sha256"] = self.policy_sha256
rebound["mac"] = hmac.new(
self.hmac_key,
_canonical(rebound),
hashlib.sha256,
).hexdigest()
checked = _validate_record(
rebound,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
if checked["admission_id"] != previous["admission_id"]:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_ID_CHANGED"
)
for field in (
"repository",
"version",
"candidate_id",
"base_oid",
"source_ref",
"source_commit",
"remote_ref_oid",
"tree_oid",
"purpose",
"task_id",
):
if checked[field] != previous[field]:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_REBIND_SCOPE_CHANGED"
)
encoded = _canonical(checked) + b"\n"
history_payload = {
"schema": "aios.release-source-admission-policy-rebind.v1",
"admission_id": checked["admission_id"],
"source_commit": checked["source_commit"],
"previous_policy_sha256": previous_policy,
"policy_sha256": self.policy_sha256,
"reason": reason_value,
"record_sha256": hashlib.sha256(encoded).hexdigest(),
"rebound_at": _now(),
}
history_payload["mac"] = hmac.new(
self.hmac_key,
_canonical(history_payload),
hashlib.sha256,
).hexdigest()
history_root = self.root / "rebind-history"
if not history_root.exists():
history_root.mkdir(mode=0o750)
_secure_stat(
history_root,
expected_owner_uid=self.expected_owner_uid,
directory=True,
)
history_name = (
f"{checked['source_commit']}."
f"{previous_policy[:12]}-{self.policy_sha256[:12]}."
f"{hashlib.sha256(reason_value.encode()).hexdigest()[:12]}.json"
)
history_path = history_root / history_name
history_bytes = _canonical(history_payload) + b"\n"
fd, raw_name = tempfile.mkstemp(
prefix=f".{checked['source_commit']}.rebind.",
suffix=".tmp",
dir=self.root,
)
temp_path = Path(raw_name)
try:
os.fchmod(fd, 0o640)
with os.fdopen(fd, "wb", closefd=True) as handle:
handle.write(encoded)
handle.flush()
os.fsync(handle.fileno())
if history_path.exists():
_secure_stat(
history_path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
if history_path.read_bytes() != history_bytes:
raise ReleaseSourceAdmissionError(
"SOURCE_ADMISSION_REBIND_HISTORY_CONFLICT"
)
else:
history_fd = os.open(
history_path,
os.O_WRONLY
| os.O_CREAT
| os.O_EXCL
| getattr(os, "O_NOFOLLOW", 0),
0o640,
)
try:
os.write(history_fd, history_bytes)
os.fsync(history_fd)
finally:
os.close(history_fd)
os.replace(temp_path, path)
os.chmod(path, 0o640)
directory_fd = os.open(
self.root,
os.O_RDONLY | getattr(os, "O_DIRECTORY", 0),
)
try:
os.fsync(directory_fd)
finally:
os.close(directory_fd)
except Exception:
try:
os.close(fd)
except OSError:
pass
temp_path.unlink(missing_ok=True)
raise
return dict(checked)
def lookup_source(self, source_commit: str) -> dict[str, Any] | None:
path = self._path(source_commit)
if not path.exists():
return None
_secure_stat(
path,
expected_owner_uid=self.expected_owner_uid,
directory=False,
)
try:
raw = path.read_bytes()
if len(raw) > 64 * 1024:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID")
decoded = json.loads(raw)
except ReleaseSourceAdmissionError:
raise
except (OSError, UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_RECORD_INVALID") from exc
return _validate_record(
decoded,
repository=self.repository,
policy_sha256=self.policy_sha256,
hmac_key=self.hmac_key,
)
def resolve_recipe(
self,
version: str,
candidate_id: str | None,
source_commit: str,
) -> dict[str, str]:
record = self.lookup_source(source_commit)
if record is None:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_NOT_FOUND")
if record["version"] != version:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_VERSION_MISMATCH")
supplied_candidate = str(candidate_id or "")
if supplied_candidate and supplied_candidate != record["candidate_id"]:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_CANDIDATE_MISMATCH")
return {
"mode": RECIPE_MODE,
"task_id": record["task_id"],
"candidate_id": record["candidate_id"],
"base_oid": record["base_oid"],
"source_commit": record["source_commit"],
"version": record["version"],
"purpose": record["purpose"],
}
@dataclass(slots=True)
class GiteaReleaseSourceBackend:
transport: GiteaMachineTransport
git_dir: Path
def authoritative_main_oid(self) -> str:
return authoritative_head_oid(git_dir=self.git_dir)
def remote_ref_oid(self, source_ref: str) -> str:
ref = _source_ref(source_ref)
branch = ref[len(_REF_PREFIX):]
value = self.transport.read_branch(branch)
if value is None:
raise ReleaseSourceAdmissionError("SOURCE_REF_NOT_FOUND")
return _oid(value, "remote_ref_oid")
def fetch_exact_ref(self, source_ref: str, source_commit: str) -> str:
ref = _source_ref(source_ref)
branch = ref[len(_REF_PREFIX):]
try:
value = self.transport.fetch_branch_exact(
branch,
_oid(source_commit, "source_commit"),
)
except GiteaTransportError as exc:
raise ReleaseSourceAdmissionError("SOURCE_FETCH_FAILED") from exc
return _oid(value, "fetched_source_commit")
def tree_oid(self, source_commit: str) -> str:
source = _oid(source_commit, "source_commit")
try:
result = run_authoritative_git(
["rev-parse", "--verify", f"{source}^{{tree}}"],
git_dir=self.git_dir,
)
except AuthoritativeGitError as exc:
raise ReleaseSourceAdmissionError("SOURCE_COMMIT_NOT_FOUND") from exc
return _oid(result.stdout.strip().lower(), "tree_oid")
def require_lineage(self, base_oid: str, source_commit: str) -> None:
try:
authoritative_lineage(
_oid(base_oid, "base_oid"),
_oid(source_commit, "source_commit"),
git_dir=self.git_dir,
)
except AuthoritativeGitError as exc:
raise ReleaseSourceAdmissionError("SOURCE_LINEAGE_MISMATCH") from exc
class ReleaseSourceAdmitter:
def __init__(
self,
store: ReleaseSourceAdmissionStore,
backend: ReleaseSourceBackend,
*,
purpose: str = "AIOS_TRUSTED_RELEASE_BUILDER",
) -> None:
if not store.writable:
raise ReleaseSourceAdmissionError("SOURCE_ADMISSION_STORE_READ_ONLY")
self.store = store
self.backend = backend
self.purpose = _identifier(purpose, "purpose")
def admit(
self,
*,
version: str,
task_id: str,
source_ref: str,
source_commit: str,
) -> dict[str, Any]:
version_value = _version(version)
task_value = _identifier(task_id, "task_id")
ref_value = _source_ref(source_ref)
source_value = _oid(source_commit, "source_commit")
remote_before = self.backend.remote_ref_oid(ref_value)
if remote_before != source_value:
raise ReleaseSourceAdmissionError("SOURCE_REF_MOVED")
base_oid = _oid(self.backend.authoritative_main_oid(), "base_oid")
fetched = _oid(
self.backend.fetch_exact_ref(ref_value, source_value),
"fetched_source_commit",
)
if fetched != source_value:
raise ReleaseSourceAdmissionError("SOURCE_FETCH_MISMATCH")
remote_after = self.backend.remote_ref_oid(ref_value)
if remote_after != source_value:
raise ReleaseSourceAdmissionError("SOURCE_REF_MOVED")
self.backend.require_lineage(base_oid, source_value)
tree_oid = _oid(self.backend.tree_oid(source_value), "tree_oid")
candidate_id = f"rc-{_version_slug(version_value)}-{source_value[:12]}"
admission_id = "source-admission-" + hashlib.sha256(
f"{self.store.repository}\0{version_value}\0{source_value}\0{ref_value}".encode("utf-8")
).hexdigest()[:24]
record = {
"schema": SCHEMA,
"admission_id": admission_id,
"repository": self.store.repository,
"task_id": task_value,
"version": version_value,
"candidate_id": candidate_id,
"base_oid": base_oid,
"source_ref": ref_value,
"source_commit": source_value,
"remote_ref_oid": remote_after,
"tree_oid": tree_oid,
"purpose": self.purpose,
"policy_sha256": self.store.policy_sha256,
"admitted_at": _now(),
}
return self.store.register(record)
def _sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def source_admission_policy_sha256(path: Path) -> str:
"""Canonical Source Admission policy binding: exact policy file bytes.
Builder/controller readiness may use a canonical JSON digest internally,
but Source Admission records are intentionally bound to the immutable policy
file bytes. Every admission producer/consumer must call this helper.
"""
try:
return _sha256_file(Path(path))
except OSError as exc:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_UNAVAILABLE") from exc
def open_source_admission_store(
root: str | os.PathLike[str],
*,
repository: str,
policy: Path,
hmac_key: bytes,
expected_owner_uid: int | None,
writable: bool,
) -> ReleaseSourceAdmissionStore:
"""The only supported constructor for policy-bound Source Admission stores."""
return ReleaseSourceAdmissionStore(
root,
repository=repository,
policy_sha256=source_admission_policy_sha256(policy),
hmac_key=hmac_key,
expected_owner_uid=expected_owner_uid,
writable=writable,
)
def validate_isolated_peer_policy_delta(
previous_policy: Path,
policy: Path,
isolated_peer_cgroup: str,
) -> tuple[str, str, str]:
"""Allow exactly one temporary isolated peer add/remove and no other drift."""
peer = str(isolated_peer_cgroup or "").strip()
if (
not peer.startswith("/system.slice/")
or not peer.endswith(".service")
or "*" in peer
or ".." in peer
):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
try:
from aios_trusted_candidate_builder import load_builder_policy
before = dict(load_builder_policy(Path(previous_policy)))
after = dict(load_builder_policy(Path(policy)))
except Exception as exc:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_INVALID") from exc
before_cgroups = tuple(before.get("allowed_peer_cgroups") or ())
after_cgroups = tuple(after.get("allowed_peer_cgroups") or ())
if len(set(before_cgroups)) != len(before_cgroups) or len(set(after_cgroups)) != len(after_cgroups):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
if any("*" in value or ".." in value for value in (*before_cgroups, *after_cgroups)):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_INVALID")
before_without = dict(before)
after_without = dict(after)
before_without["allowed_peer_cgroups"] = []
after_without["allowed_peer_cgroups"] = []
if _canonical(before_without) != _canonical(after_without):
raise ReleaseSourceAdmissionError("SOURCE_POLICY_REBIND_SCOPE_CHANGED")
before_set, after_set = set(before_cgroups), set(after_cgroups)
added = after_set - before_set
removed = before_set - after_set
if added == {peer} and not removed:
direction = "ADD_ISOLATED_PEER"
elif removed == {peer} and not added:
direction = "REMOVE_ISOLATED_PEER"
else:
raise ReleaseSourceAdmissionError("SOURCE_POLICY_PEER_DELTA_INVALID")
return (
source_admission_policy_sha256(previous_policy),
source_admission_policy_sha256(policy),
direction,
)
def _main() -> None:
parser = argparse.ArgumentParser()
subparsers = parser.add_subparsers(dest="command", required=True)
admit = subparsers.add_parser("admit")
admit.add_argument("--version", required=True)
admit.add_argument("--task-id", required=True)
admit.add_argument("--source-ref", required=True)
admit.add_argument("--source-commit", required=True)
admit.add_argument("--policy", required=True, type=Path)
admit.add_argument("--store-root", required=True, type=Path)
rebind = subparsers.add_parser("rebind-policy")
rebind.add_argument("--source-commit", required=True)
rebind.add_argument("--previous-policy", required=True, type=Path)
rebind.add_argument("--policy", required=True, type=Path)
rebind.add_argument("--isolated-peer-cgroup", required=True)
rebind.add_argument("--expected-admission-id", required=True)
rebind.add_argument("--reason", required=True)
rebind.add_argument("--store-root", required=True, type=Path)
arguments = parser.parse_args()
repository = os.environ.get("AIOS_GITOPS_REPOSITORY_IDENTITY", "")
if not repository:
raise SystemExit("AIOS_GITOPS_REPOSITORY_IDENTITY is required")
credentials = os.environ.get("CREDENTIALS_DIRECTORY", "")
if not credentials:
raise SystemExit("CREDENTIALS_DIRECTORY is required")
try:
admission_hmac = (
Path(credentials) / "source_admission_hmac"
).read_bytes().strip()
except OSError as exc:
raise SystemExit(
"source_admission_hmac credential is required"
) from exc
if arguments.command == "rebind-policy":
try:
previous_sha, policy_sha, direction = (
validate_isolated_peer_policy_delta(
arguments.previous_policy,
arguments.policy,
arguments.isolated_peer_cgroup,
)
)
store = open_source_admission_store(
arguments.store_root,
repository=repository,
policy=arguments.policy,
hmac_key=admission_hmac,
expected_owner_uid=(
os.geteuid() if hasattr(os, "geteuid") else None
),
writable=True,
)
result = store.rebind_policy(
arguments.source_commit,
previous_policy_sha256=previous_sha,
reason=f"{arguments.reason}:{direction}",
expected_admission_id=arguments.expected_admission_id,
)
except ReleaseSourceAdmissionError as exc:
raise SystemExit(str(exc)) from exc
print(
json.dumps(
{
**result,
"rebind_direction": direction,
"previous_policy_sha256": previous_sha,
"policy_sha256": policy_sha,
},
sort_keys=True,
separators=(",", ":"),
)
)
return
api_base = os.environ.get("AIOS_GITEA_API_BASE", "")
remote_url = os.environ.get("AIOS_GITEA_REMOTE_URL", "")
raw_git_dir = os.environ.get("AIOS_GITOPS_AUTHORITATIVE_GIT_DIR", "")
if not api_base or not remote_url or not raw_git_dir:
raise SystemExit("release source admission environment is incomplete")
git_dir = Path(raw_git_dir)
transport = GiteaMachineTransport(
api_base=api_base,
repository_id=repository,
remote_url=remote_url,
authority_git_dir=git_dir,
credential_name=os.environ.get(
"AIOS_GITEA_MACHINE_CREDENTIAL_NAME",
"gitea-machine-token",
),
)
store = open_source_admission_store(
arguments.store_root,
repository=repository,
policy=arguments.policy,
hmac_key=admission_hmac,
expected_owner_uid=os.geteuid() if hasattr(os, "geteuid") else None,
writable=True,
)
result = ReleaseSourceAdmitter(
store,
GiteaReleaseSourceBackend(transport=transport, git_dir=git_dir),
).admit(
version=arguments.version,
task_id=arguments.task_id,
source_ref=arguments.source_ref,
source_commit=arguments.source_commit,
)
print(json.dumps(result, sort_keys=True, separators=(",", ":")))
if __name__ == "__main__":
_main()
+1 -9
View File
@@ -11,7 +11,6 @@ from dataclasses import asdict, dataclass, field
from typing import Any
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
@dataclass(frozen=True)
@@ -44,9 +43,6 @@ class SearchGatewayAgentAdapter:
domain_profiles: list[str] | None = None,
user_scope: str = "public",
correlation_id: str | None = None,
*,
actor: GovernanceActor | dict[str, Any] | None = None,
trace: TraceContext | None = None,
) -> dict[str, Any]:
payload = {
"query": query,
@@ -63,11 +59,7 @@ class SearchGatewayAgentAdapter:
"session_profile": context.session_profile,
"correlation_id": correlation_id,
}
if actor is None and trace is None:
# Preserve adapter compatibility; the governed orchestrator itself
# still fails closed when trusted context is absent.
return self.orchestrator.execute(payload)
return self.orchestrator.execute(payload, actor=actor, trace=trace)
return self.orchestrator.execute(payload)
class OpenClawSearchClient(SearchGatewayAgentAdapter):
+4 -70
View File
@@ -13,18 +13,10 @@ from __future__ import annotations
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
import hashlib
import json
from typing import Any, Protocol
from uuid import uuid4
from .search_profile_resolver import DomainQueryProfileRegistry, SearchProfile, SearchProfileResolver
from .trace_tool_credential_governance import (
GovernanceActor,
GovernanceError,
GovernanceStore,
TraceContext,
)
def utc_now() -> str:
@@ -164,25 +156,13 @@ class SearchPolicyGuard:
class SearchGateway:
def __init__(self, providers: list[SearchProvider] | None = None, *, governance_store: GovernanceStore | None = None):
def __init__(self, providers: list[SearchProvider] | None = None):
self.providers = providers or []
self.governance_store = governance_store
self.profile_resolver = SearchProfileResolver()
self.planner = QueryPlanner()
self.guard = SearchPolicyGuard()
def search(self, request: SearchRequest, *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> SearchResponse:
trusted_actor: GovernanceActor | None = None
if self.governance_store is not None:
if actor is None or trace is None:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY")
trusted_actor = GovernanceActor.from_value(actor)
if request.requester != trusted_actor.identity:
raise GovernanceError("TOOL_MCP_AUTHZ_DENY", field="requester")
if request.correlation_id is not None and request.correlation_id not in {
trace.trace_id, trace.correlation_id
}:
raise GovernanceError("TOOL_SCOPE_PROPAGATION", field="correlation_id")
def search(self, request: SearchRequest) -> SearchResponse:
request_id = request.correlation_id or f"search_{uuid4().hex[:12]}"
profile = self.profile_resolver.resolve(request.group_profile, request.user_profile, request.session_profile)
expanded_queries = self.planner.expand(request, profile)
@@ -204,60 +184,14 @@ class SearchGateway:
attempts: list[SearchProviderAttempt] = []
providers = self._ordered_providers(profile)
for provider in providers:
if request.authorized_sources and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_DENY"))
continue
if request.user_scope != "public" and provider.name not in request.authorized_sources:
attempts.append(SearchProviderAttempt(provider=provider.name, status="denied", reason="SOURCE_AUTHZ_REQUIRED"))
continue
try:
if self.governance_store is not None:
assert trusted_actor is not None and trace is not None
input_digest = hashlib.sha256(
json.dumps(
{"query": request.query, "intent": request.intent, "provider": provider.name},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
invocation_key = hashlib.sha256(
f"{trace.trace_id}:{trace.span_id}:{provider.name}:{input_digest}".encode("utf-8")
).hexdigest()
authorization = self.governance_store.authorize_tool(
actor=trusted_actor,
tool_name=f"search.{provider.name}",
trace=trace,
requested_scope=trace.scope,
input_digest=input_digest,
invocation_key=invocation_key,
)
if authorization.get("idempotent"):
raise GovernanceError("TOOL_INVOCATION_REPLAY")
provider_evidence, attempt = provider.search(request, expanded_queries)
if self.governance_store is not None:
result_digest = hashlib.sha256(
json.dumps(
{
"evidence": [item.to_dict() for item in provider_evidence],
"attempt": attempt.to_dict(),
},
sort_keys=True, separators=(",", ":"), ensure_ascii=False,
).encode("utf-8")
).hexdigest()
self.governance_store.record_tool_receipt(
invocation_key=invocation_key,
result_digest=result_digest,
actor=trusted_actor,
trace=trace,
)
evidence.extend(provider_evidence)
attempts.append(attempt)
if provider_evidence:
break
except GovernanceError:
raise
except Exception:
# Provider exceptions may contain URLs, tokens, or local paths.
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason="PROVIDER_FAILURE"))
except Exception as exc:
attempts.append(SearchProviderAttempt(provider=getattr(provider, "name", "unknown"), status="failed", reason=str(exc)))
if evidence:
return SearchResponse(
+6 -8
View File
@@ -7,29 +7,27 @@ try:
except Exception:
APIRouter = None
from .search_gateway import SearchRequest
from .search_gateway_orchestrator import SearchGatewayOrchestrator
from .trace_tool_credential_governance import GovernanceActor, TraceContext
from .search_gateway import SearchGateway, SearchRequest
from .search_gateway_dashboard import SearchGatewayDashboard
orchestrator = SearchGatewayOrchestrator()
gateway = SearchGateway()
dashboard = SearchGatewayDashboard()
def search_gateway_request(payload: dict[str, Any], *, actor: GovernanceActor | dict[str, Any] | None = None, trace: TraceContext | None = None) -> dict[str, Any]:
return orchestrator.execute(payload, actor=actor, trace=trace)
def search_gateway_request(payload: dict[str, Any]) -> dict[str, Any]:
request = SearchRequest(**payload)
return gateway.search(request).to_dict()
def get_search_gateway_status() -> dict[str, Any]:
return {
"module": "search-gateway",
"status": "ready",
"providers_configured": len(orchestrator.gateway.providers),
"providers_configured": len(gateway.providers),
"policy_guard": "enabled",
"profile_customization": "enabled",
"fallback_required_when_provider_unavailable": True,
"governance": "required",
}

Some files were not shown because too many files have changed in this diff Show More