| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move isc-dhclient code to dhcp.py
In support of the upcoming deprecation of
isc-dhcp-client, this code refactors current
dhcp code into classes in dhcp.py. The
primary user-visible change should be the
addition of the following log:
dhcp.py[DEBUG]: DHCP client selected: dhclient
This code lays groundwork to enable
alternate implementations to live side by
side in the codebase to be selected with
distro-defined priority fallback. Note that
maybe_perform_dhcp_discovery() now selects
which dhcp client to call, and then runs the
corresponding client's dhcp_discovery()
method. Currently only class IscDhclient is
implemented, however a yet-to-be-implemented
class Dhcpcd exists to test fallback behavior
and this will be implemented in part two of
this series.
Part of this refactor includes shifting
dhclient service management from hardcoded
calls to the distro-defined manage_service()
method in the *BSDs. Future work is required
in this area to support multiple clients via
select_dhcp_client().
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Running on OpenStack Ironic was broken in 1efa8a0a0,
which prevented a system configured to run on only
Openstack from actually running this ds. This change
also prevents the kernel commandline definition from
working. This change was required to prevent
unnecessarily probing OpenStack on Ec2, and is
therefore still required.
This commit reverts an earlier attempt[1][2] to
automatically detect OpenStack, due to regression
it caused. Additionally, this change allows a
system that defines a datasource list containing
only [OpenStack] or [OpenStack, None] to attempt
running on OpenStack, overriding ds_detect(). A
datasource list that defines [OpenStack, None]
still falls back to DataSourceNone if OpenStack
fails to reach the IMDS.
This change also lays groundwork for the following
future work:
1. Add support for other datasources
2. Also override datasource checking when the kernel
command line defines a datasource. This work needs
to be done manually to support non-systemd systems.
Besides forcing OpenStack to run when it is the only
datasource in the datasource list, this commit also:
[1] 0220295 (it breaks some use cases)
[2] 29faf66 (no longer used)
LP: #2008727
|
| |
|
| |
Usage was dropped in de7851b93c5a2d4658.
|
| |
|
| |
LP: #1815990
|
| |
|
|
|
|
|
|
|
| |
- Add openstack IPv6 metadata url fe80::a9fe:a9fe
- Enable requesting multiple metadata sources in parallel
This PR is very similar to #1160, reusing the provided `url_heper` logic.
LP: #1906849
|
| |
|
|
|
|
|
|
| |
In the case cloudinit.temp_utils points to a fs mounted as noexec
and needs_exe=True, fallback to use
os.join.path(Distro.usr_lib_exec, "cloud-init/clouddir) that
will be mounted with exec perms.
LP: #1962343
|
| |
|
|
|
|
|
| |
Huawei Cloud is a international cloud provider and it runs OpenStack.
Due to the commit 1efa8a0, we're not able to force the use of the
OpenStack datasource.
Detect OpenStack when DMI chassis_asset_tag is HUAWEICLOUD
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also refactor network context managers into net.ephemeral
Currently EC2 is the only IMDS to make use of this.
IPv6 requires a link local address on interfaces. A
link local address is sufficient for the EC2 IMDS,
so no dhcp6 assignment is required for early boot
IMDS queries.
The kernel assigns this address using RFC 4291 [1]
during link initialization, so all cloud-init needs
to do is ensure that link is up.
This means that even if dhcp4 fails, an ipv6-enabled
instance may still succeed at crawling metadata.
[1] https://datatracker.ietf.org/doc/html/rfc4291#section-2.5.6
|
| |
|
|
|
| |
Applied Black and isort, fixed any linting issues, updated tox.ini
and CI.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a udev script which will invoke a hotplug hook script on all net
add events. The script will write some udev arguments to a systemd FIFO
socket (to ensure we have only instance of cloud-init running at a
time), which is then read by a new service that calls a new 'cloud-init
devel hotplug-hook' command to handle the new event.
This hotplug-hook command will:
- Fetch the pickled datsource
- Verify that the hotplug event is supported/enabled
- Update the metadata for the datasource
- Ensure the hotplugged device exists within the datasource
- Apply the config change on the datasource metadata
- Bring up the new interface (or apply global network configuration)
- Save the updated metadata back to the pickle cache
Also scattered in some unrelated typing where helpful
|
| |
|
|
|
|
|
| |
Add support for openstack's dynamic vendor data, which appears under openstack/latest/vendor_data2.json
This adds vendor_data2 to all pathways; it should be a no-op for non-OpenStack providers.
LP: #1841104
|
| |
|
|
|
|
|
|
|
| |
This just separates the reading of dmi values into its own file.
Some things of note:
* left import of util in dmi.py only for 'is_container'
It'd be good if is_container was not in util.
* just the use of 'util.is_x86' to dmi.py
* open() is used directly rather than load_file.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Consider valid product names as valid chassis asset tags when detecting
OpenStack platform before crawling for OpenStack metadata.
As `ds-identify` tool uses product name as valid chassis asset tags,
let's replicate the behaviour in the OpenStack platform detection too.
This change should be backwards compatible and a temporary fix for the
current limitations on the OpenStack platform detection.
LP: #1895976
|
| |
|
|
|
|
| |
Changes:
tox: bump the pylint version to 2.6.0 in the default run
Fix pylint 2.6.0 W0707 warnings (raise-missing-from)
|
| |
|
| |
add SAP Converged Cloud as cloud provider
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* ec2: Add support for AWS IMDS v2 (session-oriented)
AWS now supports a new version of fetching Instance Metadata[1].
Update cloud-init's ec2 utility functions and update ec2 derived
datasources accordingly. For DataSourceEc2 (versus ec2-look-alikes)
cloud-init will issue the PUT request to obtain an API token for
the maximum lifetime and then all subsequent interactions with the
IMDS will include the token in the header.
If the API token endpoint is unreachable on Ec2 platform, log a
warning and fallback to using IMDS v1 and which does not use
session tokens when communicating with the Instance metadata
service.
We handle read errors, typically seen if the IMDS is beyond one
etwork hop (IMDSv2 responses have a ttl=1), by setting the api token
to a disabled value and then using IMDSv1 paths.
To support token-based headers, ec2_utils functions were updated
to support custom headers_cb and exception_cb callback functions
so Ec2 could store, or refresh API tokens in the event of token
becoming stale.
[1] https://docs.aws.amazon.com/AWSEC2/latest/ \
UserGuide/ec2-instance-metadata.html \
#instance-metadata-v2-how-it-works
|
| |
|
|
|
|
|
|
|
|
|
| |
This adds a Oracle specific datasource that functions with OCI.
It is a simplified version of the OpenStack metadata server
with support for vendor-data.
It does not support the OCI-C (classic) platform.
Also here is a move of BrokenMetadata to common 'sources'
as this was the third occurrence of that class.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The OpenStack datasource in 18.3 changed to detect data in the
init-local stage instead of init-network and attempted to redetect
OpenStackLocal datasource on Oracle across reboots. The function
detect_openstack was added to quickly detect whether a platform is
OpenStack based on dmi product_name or chassis_asset_tag and it was
a bit too strict for Oracle in checking for 'OpenStack Nova'/'Compute'
DMI product_name.
Oracle's DMI product_name reports 'SAtandard PC (i440FX + PIIX, 1996)'
and DMI chassis_asset_tag is 'OracleCloud.com'.
detect_openstack function now adds 'OracleCloud.com' as a supported value
'OracleCloud.com' to valid chassis-asset-tags for the OpenStack
datasource.
LP: #1784685
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
OpenStack datasource is now discovered in init-local stage. In order to
probe whether OpenStack metadata is present, it performs a costly
sandboxed dhclient setup and metadata probe against http://169.254.169.254
for openstack data.
Cloud-init properly detects non-OpenStack on EC2, but it spends precious
time probing the metadata service also resulting in a confusing WARNING
log about 'metadata not present'. To avoid the wasted cycles, and
confusing warning, get_data will call a detect_openstack function to
quickly determine whether the platform looks like OpenStack before trying
to setup network to probe and crawl the metadata service.
LP: #1776701
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Network has not yet been configured in the init-local stage so the
openstack datasource will use dhcp-client to temporarily obtain an ipv4
address and query the metadata service at http://169.254.169.254 to get
network_data.json configuration. If present, the datasource will return
network_config version 1 config based on that network_data.json content.
Previously OpenStack datasource only setup dhcp on the fallback interface
so this represents a change in behavior to react to the full config
provided by openstack.
Also significant to OpenStack is the separation of a _crawl_data operation
from get_data(). crawl_data walks the available metadata services and
returns a dict of discovered content. get_data consumes the crawled_data,
caches it in the datasource and reacts to that data.
/run/cloud-init/instance-data.json now published network_data.json or
ec2_metadata key if that data is present on any datasource.
The main reasons for the separation of crawl from get_data:
* Enable performance metrics of cloud-init's metadata crawls on each
* Enable cloud-init modules and scripts to query and consume metadata
content which may have updated/changed after cloud-init's initial cache
during instance boot. (Think hotplug)
Also generalize common logic to base DataSource class/module:
* Move to a common UNSET variable up into base datasource module fix EC2,
ConfigDrive, OpenStack, SmartOS to use the global.
* Drop get_url_settings from Ec2, CloudStack and OpenStack and generalize
DataSource.get_url_params(). Allow subclasses to override url_max_wait,
url_timeout and url_retries params.
* Rename get_network_metadata bool to perform_dhcp_setup as it designates
whether EphemeralDHCPv4 setup is required before crawling metadata.
LP: #1749717
|
| |
|
|
|
| |
This enables warnings produced by pylint for unused variables (W0612),
and fixes the existing errors.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Each DataSource subclass must define its own get_data method. This branch
formalizes our DataSource class to require that subclasses define an
explicit dsname for sourcing cloud-config datasource configuration.
Subclasses must also override the _get_data method or a
NotImplementedError is raised.
The branch also writes /run/cloud-init/instance-data.json. This file
contains all meta-data, user-data and vendor-data and a standardized set
of metadata keys in a json blob which other utilities with root-access
could make use of. Because some meta-data or user-data is potentially
sensitive the file is only readable by root.
Generally most metadata content types should be json serializable. If
specific keys or values are not serializable, those specific values will
be base64encoded and the key path will be listed under the top-level key
'base64-encoded-keys' in instance-data.json. If json writing fails due to
other TypeErrors or UnicodeDecodeErrors, a warning log will be emitted to
/var/log/cloud-init.log and no instance-data.json will be created.
|
| |
|
|
|
|
| |
There was a copy/paste error in _get_url_settings such that the error
message would complain about max wait when in fact it was talking
about retries.
|
| |
|
|
|
|
|
| |
This will change all instances of LOG.warn to LOG.warning as warn
is now a deprecated method. It will also make sure any logging
uses lazy logging by passing string format arguments as function
parameters.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This modifies get_data in DataSourceOpenStack.py to get the timeout
and retries values from the data source configuration, rather than
from keyword arguments. This permits get_data to use the same timeout
as other methods, and allows an operator to increase the timeout in
environments where the metadata service takes longer than five seconds
to respond.
LP: #1657130
Resolves: rhbz#1408589
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has been a recurring ask and we had initially just made the change to
the cloud-init 2.0 codebase. As the current thinking is we'll just
continue to enhance the current codebase, its desirable to relicense to
match what we'd intended as part of the 2.0 plan here.
- put a brief description of license in LICENSE file
- put full license versions in LICENSE-GPLv3 and LICENSE-Apache2.0
- simplify the per-file header to reference LICENSE
- tox: ignore H102 (Apache License Header check)
Add license header to files that ship.
Reformat headers, make sure everything has vi: at end of file.
Non-shipping files do not need the copyright header,
but at the moment tests/ have it.
|
| |
|
|
|
|
|
|
|
|
| |
Add vendor-data support to maas which will behave like the openstack
vendor-data does. Data returned from maas must be yaml loadable.
Also update the main in DataSourceMAAS to "just work" on a maas
deployed system.
LP: #1612313
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
== background ==
DataSource Mode (dsmode) is present in many datasources in cloud-init.
dsmode was originally added to cloud-init to specify when this datasource
should be 'realized'.
cloud-init has 4 stages of boot.
a.) cloud-init --local . network is guaranteed not present.
b.) cloud-init (--network). network is guaranteed present.
c.) cloud-config
d.) cloud-init final
'init_modules' [1] are run "as early as possible". And as such, are executed
in either 'a' or 'b' based on the datasource. However, executing them means
that user-data has been fully consumed. User-data and vendor-data may have
'#include http://...' which then rely on the network being present. boothooks
are an example of the things run in init_modules.
The 'dsmode' was a way for a user to indicate that init_modules
should run at 'a' (dsmode=local) or 'b' (dsmode=net) directly.
Things were further confused when a datasource could provide networking
configuration. Then, we needed to apply the networking config at 'a'
but if the user had provided boothooks that expected networking, then the
init_modules would need to be executed at 'b'. The config drive datasource
hacked its way through this and applies networking if *it* detects it is
a new instance.
== Suggested Change ==
The plan is to
1. incorporate 'dsmode' into DataSource superclass
2. make all existing datasources default to network
3. apply any networking configuration from a datasource on first boot only
apply_networking will always rename network devices when it runs.
for bug 1579130.
4. run init_modules at cloud-init (network) time frame unless datasource
is 'local'.
5. Datasources can provide a 'first_boot' method that will be called when
a new instance_id is found. This will allow the config drive's write_files
to be applied once.
Over all, this will very much simplify things. We'll no longer have
2 sources like DataSourceNoCloud and DataSourceNoCloudNet, but would just
have one source with a dsmode.
== Concerns ==
Some things have odd reliance on dsmode. For example, OpenNebula's get_hostname
uses it to determine if it should do a lookup of an ip address.
== Bugs to fix here ==
http://pad.lv/1577982 ConfigDrive: cloud-init fails to configure network from network_data.json
http://pad.lv/1579130 need to support systemd.link renaming of devices in container
http://pad.lv/1577844 Drop unnecessary blocking of all net udev rules
|
| |
|
|
|
|
| |
Timeouts and retries were triggering so make it so
that tests do not use the typical timesouts and retries
so that the tests finish faster.
|
| |
|
|
|
|
| |
if the Datasource does not have an entry in config, then
set it to be a empty dictionary rather than None.
Also remove places that did this elsewhere.
|
| |
|
|
|
|
|
|
|
| |
Changing this interface to allow for easy change later.
The thing that this will enable is:
a.) maas datasource to look at the system config and see if it
is configured with the same consumer_key
b.) datasource config could allow setting a variable that it
would look at.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a check in cloud-init to see if the existing (cached)
datasource is still valid. It relies on support from the Datasource
to implement 'check_instance_id'. That method should quickly determine
(if possible) if the instance id found in the datasource is still valid.
This means that we can still notice new instance ids without
depending on a network datasource on every boot.
I've also implemented check_instance_id for the superclass and for
3 classes:
DataSourceAzure (check dmi data)
DataSourceOpenstack (check dmi data)
DataSourceNocloud (check the seeded data or kernel command line)
LP: #1553815
|
| |
|
|
|
|
|
|
|
|
| |
For now, this vendor data handling is just added to openstack.
However, in an effort to allow sanely handling of multi-part vendor-data
that is namespaced, we add openstack.convert_vendordata_json .
That basically takes whatever was loaded from vendordata and takes
the 'cloud-init' key if it is a dict. This way the author can
namespace cloud-init, basically telling it to ignore everything else.
|
| |
|
|
|
|
|
| |
We were checking for presense of meta_data.json for each supported
metadata version. Instead just check that /openstack is there.
This reduces the time to check on EC2 or any other cloud.
|
| |
|
|
|
|
|
|
| |
instead of taking a version that they should look for,
the readers now just select the highest supported version.
definitely a use case later for having version= but nothing
is using it now.
|
| | |
|
| | |
|
| |
|
|
|
|
|
| |
There might be multiple things to put inside a vendor-data.
So, if it is a dict and that dict has 'cloud-init', assume that the whole
thing was not meant for cloud-init, and set vendordata_raw to the specific
item.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Waiting around for a metadata service in a given datasource means that if its
not there all the subsequent datasources have to wait, and boot is slowed down.
As it is right now, EC2 is the only one that has the right to wait. In the
past, we had to wait around for the EC2 metadata service. I really do not want
to extend that courtesy to other cloud platforms. A network based metadata
service should be up as soon as networking is up.
|
| | |
|
| | |
|
| | |
|
| |
|
