openstack: use config-drive by default and metadata API as fallback

[zonggen]

This changes afterburn to use config-drive as default for scraping metadata and then try metadata API if config-drive is not available. Also changes the name of the platform from openstack-metadata to openstack.

Closes: coreos/fedora-coreos-tracker#422
Signed-off-by: Allen Bai abai@redhat.com

[zonggen]

Some potential points of interest:

• Now Afterburn requires super user permission since it might need to mount/umount the config-drive to read the metadata
• Not entirely sure if the public key API is always available. Tried to boot up an FCOS instance on OpenStack but there's no public key data in either metadata API or config-drive files.

Except the public keys, other metadata can be successfully fetched:

core@ip_addr $ AFTERBURN_OPT_PROVIDER=--cmdline sudo ./afterburn ${AFTERBURN_OPT_PROVIDER} --attributes=./metadata

AFTERBURN_OPENSTACK_INSTANCE_TYPE=m1.small
AFTERBURN_OPENSTACK_IPV4_LOCAL=ip_addr
AFTERBURN_OPENSTACK_INSTANCE_ID=i-022da7a2
AFTERBURN_OPENSTACK_IPV4_PUBLIC=
AFTERBURN_OPENSTACK_HOSTNAME=abai-fcos-afterburn-test

[cgwalters]

Hmm...don't we basically want to try them in parallel?

[zonggen]

don't we basically want to try them in parallel?

@cgwalters By parallel do you mean literally running both config-drive and metadata API at the same time, or running the metadata API as a fallback if any of the method in config-drive version fails?

I was wondering too, since right now it won't check the metadata API unless something at the initialization step went wrong (like a missing config-drive device). For example, if the config-drive is mounted correctly, Afterburn won't check metadata API for ssh public keys even if public keys is missing in the meta_data.json filr in config-drive.

[jlebon]

There's some more details in the chat logs from the community meeting where this was discussed, but the TL;DR IIUC was: we keep openstack-metadata for backwards compatibility, and we add a new openstack provider which does config-drive first, then falls back to metadata service.

The reason we do config drive first is to try to be consistent with Ignition's auto-discovery behaviour (which is not great, but we have to live with for now). If it used config-drive, then Afterburn should use config-drive too. Because we run so much later Ignition, we don't need to wait for the device to show up like Ignition does. We can just check for it and if it doesn't exist, fallback to the metadata server.

[zonggen]

Updated with followings:

• support both openstack-metadata and openstack platform id
• add unit tests for attributes and ssh_keys
• create explicit structs for the JSON files
• add fixtures for the ec2/meta_data.json and openstack/meta_data.json
• update docs in README.md and attributes.md
• minor adjustments according to comments / reviews

Also ran on OpenStack to make sure it worked as intended.. and it worked as expected

[zonggen]

Ready for another round of review!

[zonggen]

@lucab Thanks for the review! Updated regarding the reviews.

[lucab]

LGTM, thanks for implementing this!