nutanix AHV support

[tchernomax]

add nutanix AHV support

ignition support nutanix AHV, the only thing to do is to add flatcar.oem.id=nutanix in the kernel cmdline.
However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

I managed to create a nutanix AHV flatcar image (without nutanix guest tools... but that doesn't forbid the VM to run) by following this path:

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

(it tested it on a Nutanix AHV infra, also tried a flatcar update (via nebraska) -> it works)

And creating the VM with the ignition config provided as "cloud_init" "guest_customization":

...
        "guest_customization": {
          "cloud_init": {
            "user_data": "<base64 ignition conf>"
          }
          "is_overridable": false
        }
...

https://www.nutanix.dev/api_reference/apis/prism_v3.html#tag/vms/paths/~1vms/post

I tried to reproduce this path in this PR, but I can't try the resulting image myself.
It build on my desktop, but I can't upload it to a nutanix AHV platform due to my poor home network connection.
And I failed to build the image on the remote host (where I updated the flatcar_production_image.bin.bz2 official image).

How to use

If what I changed in the code is in fact the equivalent of the adding oem_id="nutanix" in the grub.cfg of the OEM partition.
This PR should be ok.

Otherwise, please guide me (I am a noob in flatcar dev).

Testing done

I created an image :

wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_image.bin.bz2
bunzip2 flatcar_production_image.bin.bz2
PART=6
LOOP=$(sudo losetup --partscan --find --show flatcar_production_image.bin)
TARGET=$(sudo mktemp -d -p /mnt --suffix -flatcar)
sudo mount "${LOOP}p${PART}" "$TARGET"
echo 'set oem_id="nutanix"' | sudo tee "$TARGET"/grub.cfg
sudo umount "${TARGET}"
sudo rmdir "${TARGET}"
sudo losetup -d "${LOOP}"
qemu-img convert -f raw -O qcow2 flatcar_production_image.bin flatcar_production_image_nutanix.qcow2
rm flatcar_production_image.bin

• upload it to a nutanix ahv cluster
• created multiple hosts using this image via terraform restapi provider (the nutanix official provider is currently unable to create VM with ignition configuration as "cloudinit" guest info).
• created a k8s cluster on thoses hosts via rke
• tried an update via nebraska (flatcar downloaded the new image, I reboot the hosts, it booted on the new version).

I tried this ignition config:

{"ignition":{"version":"3.3.0"},"storage":{"files":[{"path":"/toto","contents":{"compression":"","source":"data:,toto"},"mode":292}]}}

and checked that /toto was actually created at boot.
(I also tried some more advanced ignition config).

CI: http://localhost:8080/job/container/job/packages_all_arches/6848/cldsv/

[tormath1]

Hi @tchernomax and thanks for your contribution, two questions:

• About the naming, could we use only nutanix averywhere to match the OEM id? Or _ahv is required because multiple versions of Nutanix images might exist?

• However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.

  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

[github-actions]

Build action triggered: https://github.com/flatcar/scripts/actions/runs/19302545998

[tchernomax]

* About the naming, could we use only `nutanix` averywhere to match the OEM id? Or `_ahv` is required because multiple versions of Nutanix images might exist?

I don't really know.
I know that nutanix provide an interface which can be plugged on top off VMWare ESX hypervisors (but in that case we use the vmware flatcar image).
There is also nutanix AHV which use a nutanix branded hypervisor (what this MR is for).
While googling I also found nutanix cloud platform and nutanix kubernetes platform but I don't really what it is.

I think the best curse of action would be to ask nutanix themself.
Tomorrow I will ask a colleague (I don't have access myself) to create an issue on nutanix support to ask them to reply on this PR.

(if there is no need for _ahv I will update this PR)

* > However, Nutanix doesn't provide its "Nutanix Guest Tools" in any other format than .deb or .rpm -> so I didn't port it to flatcar.
  Do you know if it's open-source? If yes, we can investigate to package this to a Gentoo format.

I didn't find any source online.
And the content of LICENSE file on hosts where the software is already install suggest it's not.

[aurelienlm]

I’ve asked Nutanix Support to take a look at this PR and provide feedback, just in case it helps

[aurelienlm]

the answer :
If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource.
2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

[tormath1]

the answer : If i understand correctly, you are trying to integrate nutanix guest tools in flatcar ?

1 - Nutanix guest tools isn't opensource. 2 - I will check internally whether we can raise an internal request to our engineering for this

I will provide an update tomorrow or Monday on this.

Great thanks for asking. That'd be nice to have those guest tools to get the full support. Do you know what they add? I guess it's some helpers for network, system information, etc. ?

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

[tchernomax]

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine.
(though we only tests flatcar on nutanix (without tools) on a simple POC)

[tormath1]

Do you know what they add? I guess it's some helpers for network, system information, etc. ?

From what I understand (@nutanix please correct me if I am wrong) it's for sending system information from the VM to the host (like ip and such).

Do you folks think that'd make sense to wait for this tools or Flatcar can be used without on Nutanix?

We plan on using flatcar without nutanix tools, from the nutanix support told us it's not ideal but the system/flatcar should be fine. (though we only tests flatcar on nutanix (without tools) on a simple POC)

Ok then, we can start with an image without guest tools and revisit this later.

To get back on the first point (nutanix_ahv vs nutanix) that'd be nice to know which one we can use, to have some consistency with the OEM ID. Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

[aurelienlm]

sorry the Nutanix support team can't help us on this. They have close the case.

I have reach out to our account team

[tchernomax]

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

[tormath1]

Upstream Ignition is using nutanix so I'd be in favor of using nutanix everywhere.

I changed this PR to remove "AHV" reference and only use nutanix.

Thanks! I'll setup our CI system to build nutanix images then. Once the image will be built, will you be available for running test for it?

[tchernomax]

Once the image will be built, will you be available for running test for it?

yes (I might not be very responsive but I will make time to tests the images you provide)

[tchernomax]

I'll setup our CI system to build nutanix images then

@tormath1 any news ?

[tormath1]

I'll setup our CI system to build nutanix images then

@tormath1 any news ?

@tchernomax thanks for the heads-up! I started a build, images should be available here: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/ in a few hours.

EDIT: Now available: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2

[tchernomax]

EDIT: Now available: https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2

@tormath1
Sorry for the delay.

I just tried the image https://bincache.flatcar-linux.net/images/amd64/9999.0.100+tormath1-nutanix/flatcar_production_nutanix_image.img.bz2
with this butane configuration:

	variant: flatcar
	version: 1.0.0
	kernel_arguments:
	  should_exist:
	    - flatcar.autologin
	storage:
	  files:
	    - path: /toto
	      overwrite: true
	      mode: 0644
	      contents:
		inline: toto

And it's working properly.

I am autologged and the file /toto is created.

No service in "failing" state.
Nothing special in the journal or dmesg.

Everything seems fine.

localhost ~ # tail -n+l1 /etc/*relea*

==> /etc/lsb-release <==
DISTRIB_ID="Flatcar Container Linux by Kinvolk"
DISTRIB_RELEASE=9999.0.100+tormath1-nutanix
DISTRIB_CODENAME="Oklo"
DISTRIB_DESCRIPTION="Flatcar Container Linux by Kinvolk 9999.0.100+tormath1-nutanix (Oklo)"

==> /etc/os-release <==
NAME="Flatcar Container Linux by Kinvolk"
ID=flatcar
ID_LIKE=coreos
VERSION=9999.0.100+tormath1-nutanix
VERSION_ID=9999.0.100
BUILD_ID=tormath1-nutanix
SYSEXT_LEVEL=1.0
PRETTY_NAME="Flatcar Container Linux by Kinvolk 9999.0.100+tormath1-nutanix (Oklo)"
ANSI_COLOR="38;5;75"
HOME_URL="https://flatcar.org/"
BUG_REPORT_URL="https://issues.flatcar.org"
FLATCAR_BOARD="‘amd64-usr"
CPE_NAME="cpe:2.3:o:flatcar-linux:flatcar_linux:9999.0.100+tormath1-nutanix:*:*:*:*:*:*:*"

Regards

[tchernomax]

@tormath1 any news ?

[tormath1]

LGTM, we can just add one last thing: a changelog entry to be added (similar to https://github.com/flatcar/scripts/blob/main/changelog/changes/2025-06-30-stackit.md)

As this is a new platform, I think we can directly provide images for all channels (alpha, beta, stable, LTS)

[tchernomax]

we can just add one last thing: a changelog entry to be added (similar to https://github.com/flatcar/scripts/blob/main/changelog/changes/2025-06-30-stackit.md)

I just amend the commit to add changelog/changes/2025-11-05-nutanix.md.

As this is a new platform, I think we can directly provide images for all channels (alpha, beta, stable, LTS)

I will let you be judge on that.

[tormath1]

Two things:

• as part of a CNCF project, all the commits must be signed-off (see instructions: https://github.com/flatcar/scripts/pull/3311/checks?check_run_id=54561187192)
• next week there is the monthly Flatcar office hours, would you be interested to present your work and a quick demo of Nutanix support? (Agenda for the Flatcar Office Hours on Wednesday, 12th of November 2025, 2:30pm UTC Flatcar#1907)

[tchernomax]

* as part of a CNCF project, all the commits must be signed-off (see instructions: https://github.com/flatcar/scripts/pull/3311/checks?check_run_id=54561187192)

done

* next week there is the monthly Flatcar office hours, would you be interested to present your work and a quick demo of Nutanix support? ([Agenda for the Flatcar Office Hours on Wednesday, 12th of November 2025, 2:30pm UTC Flatcar#1907](https://github.com/flatcar/Flatcar/discussions/1907))

wow… I didn't expected that one…
First : I'm not very confident with my spoken English (I can manage, but I have a tendency to look for my words)
Second: I already have an appointment 14:00→15:30 that day.

If you think the first point shouldn't be a problem, I can ask my boss if he could move my appointment and free me for the "Flatcar Office Hours".

[tchernomax]

I can ask my boss if he could move my appointment and free me for the "Flatcar Office Hours".

I just asked and it's ok → I am free next wednesday 14:30 UTC.

So if you think my spoken English shouldn't be a problem I can do a quick presentation of the new nutanix image (dev + live demo).

[tormath1]

Cherry-picked to:

• flatcar-4487
• flatcar-4459
• flatcar-4230

[tormath1]

@tchernomax many thanks for the contribution, this will be available in the next set of releases (alpha, beta and stable) around next week.

For the Office Hours, it's really open to contributors to show case this kind of contribution. There are usually around 8/9 contributors / maintainers and there's nothing formal (you can prepare some slides or just share your screen without support, you decide!)
We can discuss the next steps (e.g write some documentation to use Flatcar on Nutanix) as well.

If you're OK to join, feel free to drop a comment in the discussion to mention a Flatcar / Nutanix demo: flatcar/Flatcar#1907

[tormath1]

@tchernomax thanks for joining the office-hours. To follow-up on the discussion we had about the documentation: here's the place where Nutanix documentation should land: https://github.com/flatcar/flatcar-website/tree/fa4b1f33bc3cb9fe9de9682733bae8d5af1cf5ca/content/docs/latest/installing/community-platforms

That might be interesting to provide a small example on how to start a Flatcar image and mentioning the current Nutanix limitation (about the user-data). This can be a good source of inspiration: https://github.com/flatcar/flatcar-website/blob/main/content/docs/latest/installing/community-platforms/proxmoxve.md

I will mention here as well that we decided to postpone the Nutanix image publication to the next release (December) to not interfere with on-going security release.

[invidian]

To have complete log in this PR, here is link to the recording of community call https://youtu.be/DV0-WNpe-KA?t=707 ;)

[tormath1]

To have complete log in this PR, here is link to the recording of community call https://youtu.be/DV0-WNpe-KA?t=707 ;)

Thanks a lot! And to have it even more complete, the Nutanix images are now available on Alpha, Beta and Stable:
https://www.flatcar.org/releases#stable-release (cc @tchernomax)