Switch runtime to cloud config bundle#24622
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
|
recheck |
f9a036b to
7f490c5
Compare
33f45ee to
78a0d84
Compare
9f9cea6 to
d38594c
Compare
78a0d84 to
438e65e
Compare
438e65e to
194afd0
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 438e65ef09
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
dc90f07 to
da8a6d7
Compare
cc9ec27 to
fae0096
Compare
da8a6d7 to
4b922aa
Compare
fae0096 to
ccc0c6a
Compare
4b922aa to
b14d255
Compare
ccc0c6a to
ee9df4f
Compare
b14d255 to
81a9b2c
Compare
ee9df4f to
1f13fcf
Compare
2d4d4ea to
f2b36ab
Compare
1f13fcf to
4333622
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f2b36ab920
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
4333622 to
be7117f
Compare
f2b36ab to
bd5f010
Compare
be7117f to
c4f0868
Compare
ce00443 to
9256aaa
Compare
a919f8f to
7be8ebe
Compare
9256aaa to
662d8ba
Compare
7be8ebe to
7fe4768
Compare
662d8ba to
cd05603
Compare
7fe4768 to
3bd41f0
Compare
fb6482e to
2396470
Compare
3bd41f0 to
bbf7c52
Compare
2396470 to
8d67707
Compare
bbf7c52 to
fbfc181
Compare
8d67707 to
42f2427
Compare
fbfc181 to
06b8025
Compare
1e185ff to
a1729b9
Compare
06b8025 to
486805b
Compare
ea0c385 to
3a5ae03
Compare
3a5ae03 to
4a1e94d
Compare
| } | ||
| Err(err) => { | ||
| warn!(error = %err, "Failed to preload config for cloud requirements"); | ||
| // TODO(gt): Make cloud requirements preload failures blocking once we can fail-closed. |
There was a problem hiding this comment.
Is this TODO still relevant?
There was a problem hiding this comment.
This TODO isn't worded quite right, but the general theme is still relevant. The TUI codepath fails closed in this case, where the app server doesn't and is left in a state that can't fetch config bundles. Let me replace this with a more accurate TODO comment, and can address it in a follow up PR.
| const CLOUD_REQUIREMENTS_FETCH_ATTEMPT_METRIC: &str = "codex.cloud_config_bundle.fetch_attempt"; | ||
| const CLOUD_REQUIREMENTS_FETCH_FINAL_METRIC: &str = "codex.cloud_config_bundle.fetch_final"; | ||
| const CLOUD_REQUIREMENTS_LOAD_METRIC: &str = "codex.cloud_config_bundle.load"; |
There was a problem hiding this comment.
Is this change going to mess up any reporting?
There was a problem hiding this comment.
There's just an internal developer dashboard for this by a member on our team - don't think anyone is actively looking at it. Would rather have this metric have a correct name long term though. Will update the dash accordingly.
| const CLOUD_REQUIREMENTS_AUTH_RECOVERY_FAILED_MESSAGE: &str = concat!( | ||
| "Your authentication session could not be refreshed automatically. ", | ||
| "Please log out and sign in again." | ||
| ); | ||
| const CLOUD_REQUIREMENTS_CACHE_WRITE_HMAC_KEY: &[u8] = | ||
| b"codex-cloud-requirements-cache-v3-064f8542-75b4-494c-a294-97d3ce597271"; | ||
| b"codex-cloud-config-bundle-cache-v1-6160ae70-bcfd-4ca8-a99b-40f73b3b072e"; |
There was a problem hiding this comment.
I assume this was done in coordination with some backend system?
There was a problem hiding this comment.
No, this hash is purely local. There's a slack thread back in January started by George Thomas where you, Gav, and George discussed this and agreed on this repo local HMAC approach.
| cache_path: PathBuf, | ||
| codex_home: PathBuf, |
There was a problem hiding this comment.
Prefer AbsolutePathBuf to PathBuf.
| @@ -614,6 +674,11 @@ impl CloudRequirementsService { | |||
| if !verify_cache_signature(&payload_bytes, &cache_file.signature) { | |||
| return Err(CacheLoadStatus::CacheSignatureInvalid); | |||
| } | |||
| if cache_file.signed_payload.version != CLOUD_CONFIG_BUNDLE_CACHE_VERSION { | |||
There was a problem hiding this comment.
Are we confident that when CLOUD_CONFIG_BUNDLE_CACHE_VERSION inevitably changes, we'll be sure to support both old and new versions? Should we be embedding v1 into any type names?
There was a problem hiding this comment.
We just need to be able to signal incompatibility - backwards compatibility is not necessary. Since this is just locally caching a remote payload, a version mismatch means the cache file on disk is generated from a different codex harness that uses an incompatible format, and we should just refetch and store a new cache file.
| /// any unset fields. | ||
| /// If available, load requirements from the platform system `requirements.toml` | ||
| /// location as a requirements layer. | ||
| #[doc(hidden)] |
There was a problem hiding this comment.
Can you delete this #[doc(hidden)] thing while you're here?
| /// If available, apply requirements from the platform system | ||
| /// `requirements.toml` location to `config_requirements_toml` by filling in | ||
| /// any unset fields. | ||
| /// If available, load requirements from the platform system `requirements.toml` |
|
|
||
| for (source, config) in managed_config_from_mdm | ||
| .map(|config| { | ||
| let mut layers = Vec::new(); |
There was a problem hiding this comment.
If you want to get fancy:
| let mut layers = Vec::new(); | |
| let mut layers = Vec::with_capacity(managed_config.len() + managed_config_from_mdm.map_or(0, |_| 1)); |
|
|
||
| #[derive(Clone, Debug, Default, Deserialize, Eq, PartialEq, Serialize)] | ||
| pub struct CloudConfigBundle { | ||
| pub config_toml: CloudConfigTomlBundle, |
There was a problem hiding this comment.
Should all these fields be pub?
There was a problem hiding this comment.
codex-cloud-config uses these fields, so they need to be public with how the code is currently structured.
| mod skills_config; | ||
| mod state; | ||
| mod strict_config; | ||
| #[doc(hidden)] |
Adapt the moved codex-cloud-config crate from the legacy cloud requirements endpoint to the new config bundle endpoint, while keeping the implementation in one file so the prior PR's move remains easy to review. The loader still preserves the old cloud requirements cache/fetch semantics where intentional: account-scoped signed cache, 30 minute TTL, 5 minute background refresh cadence, auth recovery, retry/backoff, and fail-closed startup loading. The cached payload changes from a single requirements TOML string to the backend-delivered config bundle, and validation now ensures malformed config or requirements fragments are rejected before cache write/use.\n\nSwitch runtime consumers from CloudRequirementsLoader to CloudConfigBundleLoader so config construction uses one shared bundle for both cloud-delivered config and requirements. This removes the legacy cloud requirements domain loader path, wires the bundle through app-server, TUI, exec, core config construction, hooks/network tests, and updates diagnostics/error handling to report cloud config bundle failures.\n\nKeep codex-cloud-config monolithic in this PR for review lineage. The follow-up PR is a pure module split that moves the same implementation back into focused files once the behavior diff has been reviewed.\n\nVerification: just fmt; just test -p codex-cloud-config; git diff --cached --check.
Clarify the app-server preload TODO now that bootstrap config preload failures can leave non-strict startup without installed cloud/thread config loaders. Store cloud config service paths as AbsolutePathBuf, use whole-object assertions in cache/auth tests, rename the cache test helper to create_test_service, and clean up the requirements loader docs and allocation per review feedback.
Update the remaining core connector test to use CloudConfigBundleFixture instead of the removed CloudRequirementsLoader path. Add the cache-test-local Path import after moving cloud-config service paths to AbsolutePathBuf so Bazel test compilation sees the type in module scope.
Summary
codex-cloud-configcrate from the legacy cloud requirements endpoint to the new config bundle endpoint.CloudRequirementsLoadertoCloudConfigBundleLoaderso one shared bundle supplies cloud-delivered config and requirements.Details
This intentionally keeps
codex-cloud-configmonolithic for review lineage: the previous PR establishes the crate move, and this PR shows the behavior change against that moved implementation. A follow-up PR splits the module back into focused files.The new bundle path preserves the important cloud requirements loader semantics where intended: account-scoped signed cache, 30 minute TTL, 5 minute refresh cadence, retry/backoff, auth recovery, and fail-closed startup loading. The cached payload changes from a single requirements TOML string to the backend-delivered bundle, and validation rejects malformed config or requirements fragments before cache write/use.