| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix cloud-init schema --system being unable to find merged
userdata stored at /var/lib/cloud/instance/cloud_config.txt.
Init.paths.get_ipath only has visibility to merged cloud config in
/var/lib/cloud/<instance_id>/cloud-config.txt after fetching the
existing cached datasource which provides instance-id from metadata
in order to determine the unique instance-id which represents the
path to the cloud-config.txt.
To support reuse of read_cfg_paths helper function, add an optional
parameter fetch_existing_datasource which indicates whether reading
the existing datasource is necessary for this helper function.
cloud-init schema --system calls read_cfg_paths providing
fetch_existing_datasource="trust" prior to calls to
paths.get_ipath().
|
| | |
|
| |
|
|
| |
Use the module level Log instances instead of passing log instances
to the cc_* handlers
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With systemd-networkd, the removal of /etc/machine-id can
result in DHCP errors such as:
DHCP4 CLIENT: Failed to set IAID+DUID: No such file or directory
Adapt cloud-init clean --machine-id to be systemd-aware.
systemd v. 237 and later will trigger /etc/machine-id regeneration
on next boot when the value of /etc/machine-id file is
'uninitialized\n'.
On systems without systemd, continue to remove /etc/machine-id when
cloud-init clean --machine-id is provided as there are triggers in
place to regenerate on absence of this file.
LP: #1999680
|
| |
|
|
|
|
|
| |
Handles any situation where `status.json` does not yet contain
datasource information, by gracefully fulfilling a
`datasource=None` in `StatusDetails`.
LP: #1997559
|
| |
|
| |
When a NIC appears, check for a cloud-init.network-config and apply it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
cli/apport: collect-logs include subiquity logs config when present
Add support for both cloud-init collect-logs and apport to
include subiquity live installer artifacts if present to aid in
bug triage.
Apport integration to attach subiquity, curtin and
ubuntu-desktop-installer report keys when logs or
config files are present.
`ubuntu-bug cloud-init` will also automatically tag the
bug as curtin, subiquity or ubuntu-desktop-installer if
related logs are present in the bug report.
Additional collect-logs support to collect
/var/lib/cloud/data in the event that cloud-init is disabled
by systemd generator with /run/cloud-init/disabled flag.
In these situations we want to collect /var/lib/cloud/data
dir for more context on prior cloud-init behavior.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Provide --format option for cloud-init status and cloud-id to
surface machine-readable output and detailed boot_status_codes.
Provide new boot_code_code values in json/yaml output
which will surface unique codes for the current disabled
status reasons which gives visibility to whether cloud-init
may become active on next boot or later in the current boot.
The boot-status-code can be:
- 'unknown': systemd generators and ds-identify haven't run yet to
determine if cloud-init should be run during this boot
- 'disabled-by-marker-file': /etc/cloud/cloud-init.disabled exists
which prevents cloud-init from ever running
- 'disabled-by-generator': systemd generator ran ds-identify and
determined no applicable cloud-init datasources
- 'disabled-by-kernel-cmdline': kernel cmdline contained
cloud-init=disabled
- 'enabled-by-kernel-cmdline': kernel cmdline contained
cloud-init=enabled
- 'enabled-by-generator': ds-identify detected possible cloud-init
datasources
- 'enabled-by-sysvinit': enabled by default in SysV init environment
LP: #1883122
|
| |
|
| |
As we do not support python2 anymore, we can remove those 8 chars.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
There have been multiple requests to allow jinja templating in
/etc/cloud configs the same way we allow jinja templating in vendordata
and userdata. This commit allows for templating both
/etc/cloud/cloud.cfg and any file in /etc/cloud/cloud.cfg.d. The same
instance data used for substitution in vendordata and userdata will be
used here.
Note that these configs get loaded multiple times during the lifetime
of cloud-init, and during cloud-init's earlier loads, instance data
is not yet available.
LP: #1913461
|
| |
|
|
|
|
|
| |
We have multiple areas of the codebase where we fetch the rundir path
and join it with the filename for instance-data.json or
instance-data-sensitive.json. This commit refactors those calls to use a
'Paths' lookup instead.
|
| |
|
|
|
|
|
| |
Adhere to Netplan Passthrough documented behavior,
not limiting v2 netplan configs to the subset of
props that cloud-init supports.
LP: #1978543
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Support overriding network activators in cloud.cfg system_info
on disk.
Default cloud-init activators are used either during hot-plug or
for datasources that are detected during init-network stage when
basic networking has already been setup by the OS.a
Activators are discovered in the following priority order and
determined based on the presence of related network tools:
system_info:
network:
activators: [eni, netplan, network-manager, networkd]
On some systems where multiple network config tools are installed
it is necessary to override the priority order of detection to ensure
one activator is chosen over the other. This is done by providing
the a custom cloud config snippet in a /etc/cloud/cloud.cfg.d/*cfg.
Surface configured Distro.network_activator from system_info
so that the hotplug hook has visibility to the configured
distribution network activator during interface bring_up and
tear_down.
Add typing hints to net.activators search_activator and
select_activator functions.
LP: #1958377
|
| |
|
|
|
|
|
|
|
| |
Typically when a cloud-image boots /etc/machine-id should be
auto-generated if /etc/machine-id does not exist. Some cloud images
appropriately generate /etc/machine-id if the image is recognized
as a clone, in most other cases it is a best practice to remove
/etc/machine-id and let DBUS or systemd do the work of generating
a new UUID on next boot so the cloned VMs are represented
uniquely to any monitoring/management software.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
cloud-init clean command provides a run-parts /etc/cloud/clean.d
directory to allow third party apps to deliver supplemental cleanup
scripts to aid in golden image creation.
Any executable scripts in /etc/cloud/clean.d will be run in
lexicographical order via subp.runparts.
This aids customers creating golden images, where the typical process
an image creator performs is to reset cloud-init on that system so
the next boot of this cloned image will be treated as first-boot and
perform all early system configuration.
|
| |
|
|
|
| |
- add a tox target for running unit tests in parallel
- add missing mocks
- make parametrized gzip bytes deterministic
|
| |
|
|
|
|
|
|
|
|
|
| |
Ensure cloud_dir setting is respected rather than hardcoding
"/var/lib/cloud"
- Modules affected: cmd.main, apport, devel.logs (collect-logs),
cc_snap, sources.DataSourceAzure, sources.DataSourceBigstep,
util:fetch_ssl_details.
- testing: Extend and port to pytest unit tests, add integration test.
LP: #1976564
|
| |
|
|
|
|
|
|
|
|
|
|
| |
When debugging python's NetworkState intance we cannot use
safeyaml.dumps because that leverages the yaml.SafeDumper which
does not allow rendering python objects.
Use yamls.dump instead.
* Additional fix for networkd renderer to create /etc/systemd/network if
it does not exist
LP: #1975907
|
| |
|
|
|
|
|
|
|
| |
- Add types to let mypy pass.
- Add mypy flags:
- detect unused ignores
- redundant casts
- Drop support of `ConfigParser` in Python 2
- Harden DataSourceLXD.network_config
- Convert old-style commented types to proper types.
|
| |
|
|
| |
caplog.text is giving different results on focal. Use caplog.records
instead on failing test.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
For non-root users, emit warnings and redact on any
/etc/cloud/cloud.cfg.d files which raise permissions errors.
Add tests covering this behavior for query, status and render
cmds.
Migrate `test_render.py` and `test_status.py` to Pytest.
LP: #1953430
SC-658
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
upstream cloud-init not longer publishes to distribution
versions which support upstart.
Trusty has not been receiving cloud-init updates since
0.7.5 and Xenial stopped getting updates as version 21.1.
Even though Ubuntu Xenial's default init system is systemd
there was still an option for folks to launch amd maintain
images which relied on upstart. Now that Xenial no longer has
updates from upstream cloud-init we can drop all upstart
support.
Old Fedora and RHEL releases which use upstart are also not
receiving cloud-init updates from tip of main.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
snapd currrently looks for 'not run' from cloud-init status[1].
Avoid changing this behavior and revert "not-run" value to
"not run".
This avoids having to get snapd to change implementation and
release updates as far back as Bionic to handle a hyphenated
not-run string.
[1]: https://github.com/snapcore/snapd/blob/master/sysconfig/\
cloudinit.go#L802
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This fix has two elements:
- cloud-init status will not correctly report 'not-run' prior to systemd
generator running. Only report "disabled" when generator has run
and /run/cloud-init/disabled exists.
- Expose not-run and disabled state in cloud-id responses
- Add unique error codes from cloud-id for error, disabled and not-run.
The new cloud-id exit codes:
0: success
1: error
2: cloud-init is in disabled state
3: cloud-init generator has not run yet
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Includes:
- Update tox.ini and .travis.yml accordingly
- Cleanup tox.ini with new tox syntax and cloud-init dependencies
- Update documentation accordingly
- Replace/remove xenial references where additional testing isn't required
- Remove xenial checks in integration tests
- Replace yield_fixture with fixture in pytest tests
Sections of code commented with lines like "Remove when Xenial is no
longer supported" still exist as they're require additional testing.
|
| |
|
|
|
| |
Applied Black and isort, fixed any linting issues, updated tox.ini
and CI.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
test_handle_args_error_on_invalid_vaname_paths() would fail when run
as root due to invocation of load_userdata() on the default user/vendor
data locations under the instance link.
- Mock load_userdata() for this test case to avoid loads.
- Update _setup_paths() to configure cloud_dir in temporary location.
- Add new test case to verify that the default locations are loaded
when unspecified.
LP: #1825027
Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Improve schema validation.
This adds strict validation of config module definitions at testing
time, with plumbing included for future runtime validation. This
eliminates a class of bugs resulting from schemas that have definitions
that are incorrect, but get interpreted by jsonschema as
"additionalProperties" that are therefore ignored.
- Add strict meta-schema for jsonschema unit test validation
- Separate schema from module metadata structure
- Improve type annotations for various functions and data types
Cleanup:
- Remove unused jsonschema "required" elements
- Eliminate manual memoization in schema.py:get_schema(),
reference module.__doc__ directly
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This attempts to standardize unit test file location under test/unittests/
such that any source file located at cloudinit/path/to/file.py may have a
corresponding unit test file at test/unittests/path/to/test_file.py.
Noteworthy Comments:
====================
Four different duplicate test files existed:
test_{gpg,util,cc_mounts,cc_resolv_conf}.py
Each of these duplicate file pairs has been merged together. This is a
break in git history for these files.
The test suite appears to have a dependency on test order. Changing test
order causes some tests to fail. This should be rectified, but for now
some tests have been modified in
tests/unittests/config/test_set_passwords.py.
A helper class name starts with "Test" which causes pytest to try
executing it as a test case, which then throws warnings "due to Class
having __init__()". Silence by changing the name of the class.
# helpers.py is imported in many test files, import paths change
cloudinit/tests/helpers.py -> tests/unittests/helpers.py
# Move directories:
cloudinit/distros/tests -> tests/unittests/distros
cloudinit/cmd/devel/tests -> tests/unittests/cmd/devel
cloudinit/cmd/tests -> tests/unittests/cmd/
cloudinit/sources/helpers/tests -> tests/unittests/sources/helpers
cloudinit/sources/tests -> tests/unittests/sources
cloudinit/net/tests -> tests/unittests/net
cloudinit/config/tests -> tests/unittests/config
cloudinit/analyze/tests/ -> tests/unittests/analyze/
# Standardize tests already in tests/unittests/
test_datasource -> sources
test_distros -> distros
test_vmware -> sources/vmware
test_handler -> config # this contains cloudconfig module tests
test_runs -> runs
|
| |
|
|
|
|
|
|
|
|
|
| |
This commit removes automatically installing udev rules for hotplug
and adds a module to install them instead.
Automatically including the udev rules and checking if hotplug was
enabled consumed too many resources in certain circumstances. Moving the
rules to a module ensures we don't spend extra extra cycles on hotplug
if hotplug functionality isn't desired.
LP: #1946003
|
|
|
Adds a udev script which will invoke a hotplug hook script on all net
add events. The script will write some udev arguments to a systemd FIFO
socket (to ensure we have only instance of cloud-init running at a
time), which is then read by a new service that calls a new 'cloud-init
devel hotplug-hook' command to handle the new event.
This hotplug-hook command will:
- Fetch the pickled datsource
- Verify that the hotplug event is supported/enabled
- Update the metadata for the datasource
- Ensure the hotplugged device exists within the datasource
- Apply the config change on the datasource metadata
- Bring up the new interface (or apply global network configuration)
- Save the updated metadata back to the pickle cache
Also scattered in some unrelated typing where helpful
|
