feat(azure): Add Azure Ignition fragment generation#1264
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for generating Azure-specific Ignition fragments. The implementation is well-structured, adding a new CLI command, a systemd service for execution, and the core logic for fetching data from IMDS and OVF. The refactoring to share SSH key parsing logic is a good improvement.
I've found one critical issue regarding the systemd service activation which would cause the feature to not work as intended, and one medium-severity suggestion to improve code efficiency and readability. Please see the detailed comments.
| mkdir -p "$initdir/$systemdsystemunitdir/ignition-complete.target.requires" | ||
| ln -s "../afterburn-hostname.service" "$initdir/$systemdsystemunitdir/ignition-complete.target.requires/afterburn-hostname.service" | ||
| ln -s "../afterburn-network-kargs.service" "$initdir/$systemdsystemunitdir/ignition-complete.target.requires/afterburn-network-kargs.service" | ||
| ln -s "../afterburn-ignition-fragment.service" "$initdir/$systemdsystemunitdir/ignition-complete.target.requires/afterburn-ignition-fragment.service" |
There was a problem hiding this comment.
The afterburn-ignition-fragment.service is being linked into ignition-complete.target.requires. This will cause it to run after all main Ignition stages are complete, which is too late for the generated fragment to be used by Ignition.
According to its own unit file, this service must run before ignition-kargs.service. To ensure correct ordering and activation, it should be made a requirement of ignition-kargs.service instead of ignition-complete.target.
| ln -s "../afterburn-ignition-fragment.service" "$initdir/$systemdsystemunitdir/ignition-complete.target.requires/afterburn-ignition-fragment.service" | |
| mkdir -p "$initdir/$systemdsystemunitdir/ignition-kargs.service.requires" | |
| ln -s "../afterburn-ignition-fragment.service" "$initdir/$systemdsystemunitdir/ignition-kargs.service.requires/afterburn-ignition-fragment.service" |
There was a problem hiding this comment.
Hmm, the ordering concern is valid but ignition-kargs.service.requires might also be too late. Looking at the Ignition boot flow (coreos/ignition#2195), both ignition-fetch-offline.service and ignition-fetch.service read from base.platform.d/{platform}/ before ignition-kargs.service runs.
Maybe we should use Before=ignition-fetch-offline.service or similar to ensure the fragments are written before Ignition discovers them?
There was a problem hiding this comment.
@peytonr18 how are we going to handle the ordering of this?
There was a problem hiding this comment.
You're right, the ordering we had was likely too late.
I don't think we can go all the way to Before=ignition-fetch-offline.service because the service needs network-online.target for the IMDS call, and the network isn't up yet at that point.
For Azure that's not really a big deal: the provider always requires the network, so fetch-offline exits early with neednet and ignition-fetch.service is the stage that matters.
With that being said, I think switching the ordering to Before=ignition-fetch.service is the best approach? Let me know your thoughts!
There was a problem hiding this comment.
That's a great point, yeah I think that should work for this use case.
| let key_data = item | ||
| .key_data | ||
| .replace("\r\n", "") | ||
| .replace('\n', "") | ||
| .trim() | ||
| .to_string(); |
There was a problem hiding this comment.
The current string manipulation for cleaning key_data involves a chain of operations that results in an unnecessary String allocation because .to_string() is called on a temporary &str. This can be made more efficient and readable by separating the steps.
| let key_data = item | |
| .key_data | |
| .replace("\r\n", "") | |
| .replace('\n', "") | |
| .trim() | |
| .to_string(); | |
| let key_data_cleaned = item.key_data.replace("\r\n", "").replace('\n', ""); | |
| let key_data = key_data_cleaned.trim(); |
6af9c2e to
50a81df
Compare
|
@peytonr18, I have started looking into the changes, I will add my feedback in the next day or so. |
prestist
left a comment
There was a problem hiding this comment.
Overall this looks good! just a few concerns around the CLI design and how the IMDS client is built.
|
@prestist I should have addressed all of your comments - thank you so much for these! I agreed with all of your points and adjusted the logic accordingly. |
|
@peytonr18 I will take another pass today! thank you for the quick changes! |
|
@peytonr18 we do have some failing lints in the CI and you would need to add an entry to the release notes file in the "upcoming" release notes section. https://github.com/coreos/afterburn/blob/main/docs/release-notes.md |
18deb87 to
b2da779
Compare
…larize ignition config fragments
…tandalone IMDS client Route admin username and SSH key fetches through the Azure provider's existing client, remove duplicate IMDS_ENDPOINT and helpers from config.rs, and restrict the subcommand to the azure provider only.
Overlay a tmpfs on /usr/lib/ignition/base.platform.d in the dracut service so afterburn can write .ign fragments on the read-only OSTree filesystem. Move the test module after all impl blocks in azure/mod.rs to fix the clippy items_after_test_module lint, and add a release notes entry for the render-ignition feature.
Use Ignition’s writable local config directory for generated Azure fragments, and translate OVF adminPassword into an Ignition passwordHash instead of rejecting it.
b2da779 to
af9bd12
Compare
Replace the sha-crypt crate with an inline SHA-512 crypt implementation backed by the sha2 crate (packaged in Fedora as rust-sha2). This removes a dependency not available in upstream Fedora packaging. Replace naive string-based XML namespace stripping with structural parsing via xml-rs EventReader and EventWriter.
9d71b9b to
aef7dca
Compare
prestist
left a comment
There was a problem hiding this comment.
Quick look over, a few small questions/nits
| serde-xml-rs = ">= 0.4, < 0.9" | ||
| serde_json = "1.0" | ||
| serde_yaml = ">= 0.8, < 0.10" | ||
| sha-crypt = "0.5" |
There was a problem hiding this comment.
We likely can not do that as this will not be FIPS compliant. Is there something in the openssl crate that we can use instead?
There was a problem hiding this comment.
I swapped the sha-crypt crate for a small FFI call to the system's crypt(3) (libxcrypt) — same function passwd and PAM use to write shadow, so there isn't a new dependency to worry about.
By my searching, openssl doesn't have a crypt equivalent, so going that route would mean re-implementing the password hashing scheme by hand, which I'd like to avoid.
Let me know your thoughts on this, I was trying to find the middle ground in best practices here but I'm open to whatever implementation feels best for you all.
3786e89 to
c555f66
Compare
Building on Ubuntu baked NEEDED libcrypt.so.1 into the binary, which the Fedora dynamic loader could not satisfy (it ships libcrypt.so.2). Drop the #[link] attribute and look up crypt_r at runtime, trying libcrypt.so.2 then libcrypt.so.1, so the same binary works on both distros.
…orrupt rand 0.9.4 checksum from merge
| # before ignition-fetch.service, which produces /run/ignition.json from | ||
| # /etc/ignition/base.platform.d/{platform}/. | ||
| Wants=network-online.target | ||
| After=network-online.target |
There was a problem hiding this comment.
to clarify, the current ignition-fetch.service is After=network.target. So this does somewhat indirectly change behavior if we order before fetch.service (it has been seen that fetch will fail while DHCP is ongoing). There may be some implications to waiting for online target in multi-nic scenarios where maybe a subset don't dhcp [fast enough].
@prestist what ordering issue specifically are you referring to?
There was a problem hiding this comment.
Hmm, so the concern I had was about module-setup.sh — the fragment service gets symlinked into ignition-complete.target.requires. That would mean the unit only gets pulled into the transaction when ignition-complete.target is being assembled, which is after Ignition has already read from base.platform.d/ and produced /run/ignition.json. The fragments wouldn't be consumed at that point.
Its entirely possible I have a misunderstanding here, but I think it might make sense to ensure this service runs before ignition-fetch-offline.service or ignition-fetch.service, since those are what read base.platform.d/{platform}/.
On the network-online.target vs network.target point — good point. If we add Before=ignition-fetch.service here while keeping After=network-online.target, that would transitively force fetch to also wait for network-online.target, which could be a problem in multi-NIC setups where not all interfaces come up quickly. IMDS is a link-local endpoint right? So it should work once the primary interface has an address — would network.target be sufficient here instead?
There was a problem hiding this comment.
I think they read base.platform.d entries (it's certainly logged), but I'm unsure that it does anything with it. Can you look into if these parameters are used at all @peytonr18?
There was a problem hiding this comment.
Looking at ignition the fragments are actively merged, not just logged.
FetchBaseConfig() reads .ign files from base.platform.d/{platform}/ across all system config dirs (/run/ignition, /etc/ignition, /usr/lib/ignition), parses them as Ignition configs, and merges them via latest.Merge().
engine.go:Run() calls FetchBaseConfig at the start of every stage, then merges the result into the final config at line 127: fullConfig := latest.Merge(baseConfig, latest.Merge(systemBaseConfig, cfg))
Since FetchBaseConfig runs at the start of every stage including fetch-offline, the afterburn fragment service would need to complete before ignition-fetch-offline.service to ensure the fragments are present when Ignition first reads them.
There was a problem hiding this comment.
In practice though, that fullConfig has little or no impact to behavior.
used to determine if it needs networking at this stage (which results in offline bailing).
no-op.
There was a problem hiding this comment.
Very good point, you're right that the fetch stages don't act on the merged config in practice.
I think the stage that actually matters is files, where createPasswd() and createFilesystemsEntries() run. So the fragments just need to exist before ignition-files.service.
On the network-online.target vs network.target question — since IMDS is link-local (169.254.169.254) and afterburn already has retry logic, would network.target be sufficient here? That'd be consistent with how ignition-fetch.service handles it and would avoid the multi-NIC stalling issue.
I think I confused this thread a bit, I want to make sure so going back to the ordering concern I raised earlier (#1264 (comment)) — I think the module-setup.sh symlink is still relevant here. The fragment service is currently linked into ignition-complete.target.requires, but ignition-files.service has Before=ignition-complete.target. Wouldn't that mean the files stage finishes before systemd even pulls in the fragment service? If so, would switching the symlink to ignition-files.service.requires make more sense?
There was a problem hiding this comment.
Stage order: fetch-offline [-> fetch] [-> kargs] -> disks -> mount -> files.
Given this, we will want to order before disks. ignition-complete.target requires would be fine, but it would be redundant once we order before ignition-disks.
I would take the above bot suggestion and just switch from kargs -> disks, because kargs is optional and we don't actually care to order before that either (unless someone knows a good reason to). We do need to order before disks (and implicitly, mount and files).
There was a problem hiding this comment.
That makes sense; Before=ignition-disks.service covers disks, mount, and files implicitly through the chain. And agreed, no reason to order before kargs since it's optional and the fragments don't contain kargs config anyway.
Oook, So to summarize, the service unit would need Before=ignition-disks.service instead of Before=ignition-kargs.service, and it might make sense to switch After=network-online.target to After=network.target to stay consistent with how ignition-fetch.service handles it and avoid the multi-NIC stalling concern.
There was a problem hiding this comment.
That's what my takeaways were as well - will get the files updated asap!
d2bb19f to
9d6be5c
Compare
| ln -s "../afterburn-network-kargs.service" "$initdir/$systemdsystemunitdir/ignition-complete.target.requires/afterburn-network-kargs.service" | ||
|
|
||
| mkdir -p "$initdir/$systemdsystemunitdir/ignition-files.service.requires" | ||
| ln -s "../afterburn-ignition-fragment.service" "$initdir/$systemdsystemunitdir/ignition-files.service.requires/afterburn-ignition-fragment.service" |
There was a problem hiding this comment.
keep this tied to ignition-complete.target. the before/after config will order it correctly. this is just to enable the unit as being enabled if ignition-complete.target is enabled.
There was a problem hiding this comment.
(just verify that this unit runs parallel to ignition-fetch now)
| # Order before ignition-disks.service so fragments exist before the | ||
| # disks/mount/files stages run (files is where they're consumed). | ||
| # ignition-kargs is optional and does not consume these fragments. | ||
| After=network.target |
There was a problem hiding this comment.
upon further consideration, if we are in parallel with other ignition services, is that are in the middle of writing the config fragment when an ignition service is loading the configs and breaks the parser. unlikely race condition but I suspect possible nonetheless. we would want to be sure the config write is atomic (file written before moved into target location), or maybe order in between fetch.service and disks.service.
There was a problem hiding this comment.
sorry, let's include after=ignition-kargs.service too.
Summary
This PR adds a
render-ignitionCLI subcommand to Afterburn that generates Ignition config fragment files from Azure IMDS and OVF metadata. These fragments are written to/etc/ignition/base.platform.d/azure/during initrd, allowing Ignition to natively merge platform-provided user data (hostname, SSH keys, password) without requiring a custom Ignition config.Changes
New
render-ignitionsubcommandsrc/cli/render_ignition.rswith clap-derived argument parsing.--provider/--cmdline,--render-ignition-dir, and per-feature flags (--hostname,--platform-user,--platform-extensions).Azure fragment generation (
src/providers/microsoft/azure/config.rs)generate_hostname_fragment(): fetches hostname viaMetadataProvider::hostname(), writeshostname.ignwith astorage.filesentry for/etc/hostnameusing adata:,<hostname>URI.generate_user_fragment(): fetches admin username and SSH keys from IMDS viaMetadataProvider, optionally readsadminPasswordfrom OVF (ovf-env.xmlon CD-ROM), hashes it with SHA-512, and writesuser.ignwith apasswd.usersentry.OVF / IMDS source rules
adminPasswordis the only field read from OVF. If present and non-empty, it is SHA-512 hashed and included aspasswordHashin the Ignition user fragment. If absent or empty, it is omitted.Systemd / Dracut integration
afterburn-ignition-fragment.serviceunit (dracut module) runs during initrd.ignition.platform.id=azure.network-online.target, beforeignition-kargs.service.afterburn render-ignition --cmdline --render-ignition-dir /etc/ignition/base.platform.d/azure/ --platform-extensions.CLI usage
--hostnamehostname.ign(storage.filesentry for/etc/hostname)--platform-useruser.ign(passwd.usersentry with username + SSH keys + optional password hash)--platform-extensions--hostname --platform-userEach feature writes its own standalone fragment file. Missing data (e.g., no hostname available) logs a warning and skips, matching existing Afterburn behavior.
Existing functionality is untouched
The
multisubcommand's--hostname=<path>direct-file-write path is unchanged. Theafterburn-hostname.service(including its Azure/AzureStack conditions) is not modified.The CLI design is up for discussion — would love feedback on the flag names, subcommand structure, and overall approach.