summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* run-container: make the container/VM timeout configurable (#2118)Paride Legovini2023-04-101-4/+6
| | | | | | 30 seconds are sometimes not enough when waiting for a VM to be ready. Make the timeout configurable via the command line. Rename the variable from DEFAULT_MAX_WAIT to MAX_WAIT as that is now configurable, and not a default anymore.
* suse: Remove sysvinit files. (#2115)Brett Holman2023-04-105-455/+0
| | | | | | For distros that still use sysvinit, these files are getting updated to support kernel commandline disablement. Suse no longer uses sysvinit, so they may be removed instead.
* test: Backport assert_call_count for old requests (#2119)Brett Holman2023-04-061-3/+35
| | | Fixes test broken by e02c460 on Focal.
* Add "licebmi" as contributor (#2113)Mark Martinez2023-04-061-0/+1
|
* Adapt DataSourceScaleway to upcoming IPv6 support (#2033)Louis Bouchard2023-04-052-167/+751
| | | | | | | | | | | | | | | | | | | Add support for IPv4 and/or IPv6 provided metadata. Upcoming features will allow for Instances to be started with only IPv6 connectivity. The datasource must be able to fetch its metadata from the following endpoints : * IPv4 on http://169.264.42.42/conf * IPv6 on http://[fd00:42::42]/conf This URL may eventually be overridden in the future by a DNS resolvable URL defined in /etc/cloud/config.d/scaleway.conf Add support for the configuration of one or many IP addresses when provided by the `public_ips` metadata key. In such configurations, instances no longer have a `private_ip`. The previous configuration method is kept for backward compatibility. This commit also fixes a bug in the definition of ssh keys from tags.
* rhel: make sure previous-hostname file ends with a new line (#2108)Ani Sinha2023-04-042-1/+5
| | | | | | | | | cloud-init strips new line from "/etc/hostname" on rhel distro when processing "/var/lib/cloud/data/previous-hostname". Although this does not pose a serious issue, it is still better if the behavior is similar to other distros like Ubuntu where /previous-hostname does end with a new line. Fix this issue by using hostname parser in rhel similar to debian. Signed-off-by: Ani Sinha <anisinha@redhat.com>
* Adding contributors for DataSourceAkamai (#2110)acourdavAkamai2023-04-031-0/+3
| | | | | | I signed the Canonical CLA on behalf of Akamai; this change adds myself and two ICs from Akamai who are associated with my organization to the github-cli-signers file to allow them to contribute on the organization's behalf.
* Cleanup ephemeral IP routes on exception (#2100)sxt10012023-04-032-17/+99
| | | | | If an exception occurs during EphemeralIPv4Network setup, any routes that were setup need to be torn down. This wasn't happening, and this commit adds the teardown.
* Fix private key permissions when openssh not earlier than 9.0 #2072sxt10012023-04-023-4/+98
| | | | | | | | Cloud-init's host key generation mimics that of sshd-keygen. It used to generate 640 permissions, but going forward it should be 600. Check sshd version to set the permissions appropriately. LP: #2011291
* Standardize kernel commandline user interface (#2093)Brett Holman2023-03-319-61/+165
| | | | | | - deprecate ci.ds= and ci.datasource= in favor of ds= - enable semi-colon-delimited datasource everywhere - add support for case-insensitive datasource match - add integration tests
* config/cc_resizefs: fix do_resize arguments (#2106)Chris Patterson2023-03-312-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Daily PPA builds were crashing due to refactor: stop passing log instances to cc_* handlers (#2016). When dropping the logger argument, the tuple became a list which then gets expanded in the call to do_resize() and crashes. ``` Traceback (most recent call last):   File "/usr/lib/python3/dist-packages/cloudinit/config/modules.py", line 257, in _run_modules     run_name, mod.handle, func_args, freq=freq   File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 67, in run     return self._runners.run(name, functor, args, freq, clear_on_fail)   File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 172, in run     results = functor(**args)   File "/usr/lib/python3/dist-packages/cloudinit/config/cc_resizefs.py", line 309, in handle     args=(resize_cmd),   File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2722, in log_time     ret = func(*args, **kwargs) TypeError: do_resize() takes 1 positional argument but 2 were given ``` Restore args as a tuple. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* Fix test_dhclient_exits_with_error (#2105)James Falcon2023-03-301-1/+4
| | | | Test was missing a mock of subp.which. This was missed because dhclient is installed on machines it was tested on.
* net/dhcp: catch dhclient failures and raise NoDHCPLeaseError (#2083)Chris Patterson2023-03-292-2/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some variants of dhclient will exit with non-zero codes on lease failure. For example, on RHEL 8.7: ``` [cpatterson@test-rhel87 ~]$ sudo /usr/sbin/dhclient -1 -v -lf /tmp/my.lease -pf /tmp/my.pid bridge2nowhere -sf /bin/true Internet Systems Consortium DHCP Client 4.3.6 Copyright 2004-2017 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/bridge2nowhere/42:ef:d5:38:1d:19 Sending on LPF/bridge2nowhere/42:ef:d5:38:1d:19 Sending on Socket/fallback Created duid "\000\004E<\225X\232\304J\337\243\026T\324\243O\270\177". DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 4 (xid=0x777bc142) DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 7 (xid=0x777bc142) DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 13 (xid=0x777bc142) DHCPDISCOVER on bridge2nowhere to 255.255.255.255 port 67 interval 6 (xid=0x777bc142) No DHCPOFFERS received. Unable to obtain a lease on first try. Exiting. [cpatterson@test-rhel87 ~]$ echo $? 2 ``` This results in an unhandled subp.ProcessExecutionError exception. Catch these failures and re-raise as NoDHCPLeaseError. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* sources/azure: move pps handling out of _poll_imds() (#2075)Chris Patterson2023-03-292-304/+135
| | | | | | | | | | | | | | Pull out remaining PPS handling bits from _poll_imds() and add two explicit methods for the overloaded path: - _wait_for_pps_running_reuse() for running PPS logic. - _wait_for_pps_unknown_reuse() for unknown and recovery PPS logic. For consistency: - Rename _wait_for_all_nics_ready() -> _wait_for_pps_savable_reuse(). - Move reporting ready logic into _wait_for_pps_os_disk_shutdown(). Drop several impacted tests as coverage already exists in TestProvisioning, and update the rest to handle the +/- 1 DHCP attempt due to varying assumptions around PPS state and DHCP.
* tests: bump pycloudlib version (#2102)Alberto Contreras2023-03-291-1/+1
|
* schema: do not manipulate draft4 metaschema for jsonschema 2.6.0 (#2098)Chad Smith2023-03-293-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Only set additionalProperties = False on draft4 schema definition in jsonschema 3.0 or greater because cloud-init globally registers its draft4 extensions as the primary validator for any draft4-based schemas in the same python process. This affects solutions such as subiquity and ubuntu-desktop-installer which invoke jsonschema.validate in the same process at runtime just after calling cloudinit.schema.get_jsonschema_validator. The resulting Tracebacks are seen as something like: jsonschema.exceptions.SchemaError: {'$ref': '#/definitions/ref_id'} is not valid under any of the given schema Background: cloud-init needs to extend draft4 schema to better validate and warn 'deprecated' properties in draft4-based cloud-init schema definitions. Our unittests also attempt to strictly validate any meta schema definitions for the cc_* config modules. To accomplish strict meta schema validation cloud-init makes a copy of the draft4 meta schema and adds an 'additionalProperties' = True to that schema to raise specific errors and catch typos in cc_ module schema definitions. Given that cloud-init at runtime extends and registers a draft4 schema validator, any external consumers of jsonschema.validate with draft4-base schemas are exposed to cloud-init's validator so let's limit our risk exposure. For python 2.6.0, we cannot specify make draft4 schema strict because any "$ref" keys are not yet resolved to their actual #/defintions/<id> values so the traceback above will always be generated in 'strict' mode for complex schemas. This does not affect jsonschema 3.0+ which appears to resolve schema $refs values before schema validation.
* sources/azure/imds: don't count timeout errors as connection errors (#2074)Chris Patterson2023-03-293-7/+7
| | | | | | | | | | | | | | | When fetching metadata in _check_if_nic_is_primary() the retry count is 300, but fails out after 10 connection errors. In some cases, fetching from IMDS may fail with read timeout for more than 10 attempts, far sooner than the desired 300. Keeping the existing max_connection_errors = 10 is fine so long as it is truly a connection error. These generally shouldn't occur when using the primary NIC. Always retry on timeout errors (up until desired limit) and count only connections errors against max_connection_errors. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* Fix Python 3.12 unit test failures (#2099)James Falcon2023-03-272-19/+21
|
* integration tests: Refactor instance checking (#1989)James Falcon2023-03-2750-311/+460
| | | | | | Using individual release and platform marks to specify our test support matrix was leading to too many marks specifying different combinations of things. Rather, we can rely on the "skipif" mark to perform any needed release or platform checks.
* ci: migrate remaining jobs from travis to gh (#2085)Alberto Contreras2023-03-272-43/+17
|
* missing ending quote in instancedata docs(#2094)Hong L2023-03-271-1/+1
|
* refactor: stop passing log instances to cc_* handlers (#2016)d1r3ct0r2023-03-27113-931/+795
| | | | Use the module level Log instances instead of passing log instances to the cc_* handlers
* tests/vmware: fix test_no_data_access_method failure (#2092)Chris Patterson2023-03-231-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Patch is_vmware_platform to return False to avoid failure: ``` def test_no_data_access_method(self): ds = get_ds(self.tmp) ds.vmware_rpctool = None > ret = ds._get_data() tests/unittests/sources/test_vmware.py:104: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cloudinit/sources/DataSourceVMware.py:193: in _get_data if require_vmware_platform and not is_vmware_platform(): cloudinit/sources/DataSourceVMware.py:401: in is_vmware_platform system_type = dmi.read_dmi_data("system-product-name") cloudinit/dmi.py:180: in read_dmi_data return _call_dmidecode(key, dmidecode_path) cloudinit/dmi.py:130: in _call_dmidecode (result, _err) = subp.subp(cmd) E RuntimeError: called subp. set self.allowed_subp=True to allow E subp(['/usr/sbin/dmidecode', '--string', 'system-product-name']) tests/unittests/helpers.py:176: RuntimeError ``` Bypassing is_vmware_platform() avoids the dmi reads. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* Don't change permissions of netrules target (#2076)James Falcon2023-03-233-4/+27
| | | | | | | Set permissions if file doesn't exist. Leave them if it does. LP: #2011783 Co-authored-by: Chad Smith <chad.smith@canonical.com>
* tests/sources: patch util.get_cmdline() for datasource tests (#2091)Chris Patterson2023-03-231-0/+9
| | | | | | | | | | | | | Recent changes to override_ds_detect() triggers a call to get_cmdline(), which invokes subp.subp() for various container checks. This causes tests to fail when running a specific test module instead of the full set. This is because test_smartos.py on module load will trigger these calls and the lru_cache() will retain the results. So if the module does not load, the tests will fail. Patch util.get_cmdline() for all data source tests to avoid this behavior. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* macs: ignore duplicate MAC for devs with driver driver qmi_wwan (#2090)Chad Smith2023-03-232-13/+29
| | | | | | | | | | | | | Another physical modem which has duplicate MAC addresses. Cloud-init needs to ignore the subordinate devices which are associated with the qmi_wwan driver. Fixes network rendering for the following modems: Quectel EG25 Quectel RM510Q-GLHA Sierra Wireless MC7455 LP: #2008888
* Fedora: Enable CA handling (#2086)František Zatloukal2023-03-221-0/+8
| | | | | Fedora wasn't previously supported for CA handling. Enabling this allows the testsuite to pass when ran on a Fedora system. The conf override is the same as for rhel.
* Add frantisekz as contributor (#2087)František Zatloukal2023-03-221-0/+1
|
* Send dhcp-client-identifier for InfiniBand ports (#2043)Waleed Mousa2023-03-224-5/+127
| | | | | | | | | Sending dhclient command failed for InfiniBand ports because dhcp-client-identifier is not specified. So, providing this patch to allow send dhcp-client-identifier hardware with the dhclient command for InfiniBand ports. Signed-off-by: waleedm <waleedm@nvidia.com>
* cc_ansible: complete the examples and doc (#2082)Yves2023-03-212-0/+2
| | | | | Complete the examples that the suggested code work. The examples in this script are not correct and should be adjusted accordingly.
* contributor: add bdrungChad Smith2023-03-201-0/+1
| | | | This contributor was added on a downstream ubuntu/devel branch and should be reflected in main too
* bddeb: for dev package, derive debhelper-compat from host systemChad Smith2023-03-204-110/+169
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running `make deb` or `packages/bddeb` our script uses the host's debuild tooling instead of sbuild. Since we move CI runners to a jammy environment, the host's debhelper-compat levels become incompatible with building a local bionic deb with debuild. Fix package-build .github/workflow/integration.yaml to install all package build dependenies from both requirements.txt an test-requirements.txt using read-depenedencies. Fix bddeb to determine the hosts debhelper-compat level support when generating local deb packages without using sbuild. To migrate to host-based debhelper-compat support the following changes were made: - drop packages/debian/compat file and prefer debian/control debhelper-compat (= ##) definitions - dpkg-query debhelper to check applicable debhelper versions in Provides on the host - Hardcode any `bddeb -d bionic` to a maximum of debhelper-compat( = 11) - Update package/debian/rules with latest published rules file from ubuntu/devel branch Use the latest debhelper-compat level provided by the host's debhelper package by using dpkg-query. Fixes CI on downstream ubuntu/* branches which invoke bddeb directly.
* apport: only prompt for cloud_name when instance-data.json is absentChad Smith2023-03-202-13/+86
| | | | | | | | | | | | | | | Do not prompt for CloudName when instance-data.json exists and is valid YAML. When instance-data.json exists, general-hooks/cloud_init.py will add the following fields to bug reports: CloudName, CloudID, CloudPlatform and CloudSubplatform. Downstream ubuntu packaging braches deliver: debian/apport-general-hook.py to /usr/share/apport/general-hooks/cloud-init.py Only prompt in during apport bug when the general-hook can't process instance-data.json.
* datasource: Optimize datasource detection, fix bugs (#2060)Brett Holman2023-03-1918-148/+244
| | | | | | | | | | | | | | | | | | Commit d1ffbea556a06105 enabled skipping python datasource detection on OpenStack when no other datasources (besides DataSourceNone) can be discovered. This allowed one to override detection, which is a requirement for OpenStack Ironic which does not advertise itself to cloud-init. Since no further datasources can be detected at this stage in the code, this pattern can be generalized to other datasources to facilitate troubleshooting or providing a general workaround to runtime detection bugs. Additionally, this pattern can be extended to kernel commandline datasource definition. Since kernel commandline is highest priority of the configurations, it makes sense to override python code datasource detection as well. Include an integration test on LXD for this behavior that configures kernel commandline and reboots to verify that the specified datasource is forced.
* Handle non existent ca-cert-config situation (#2073)Shreenidhi Shedi2023-03-162-6/+25
| | | | | | | | | Currently if a cert file doesn't exist, cc_ca_certs module crashes This fix makes it possible to handle it gracefully. Also, out_lines variable may not be available if os.stat returns 0. This issue is also taken care of. Added tests for the same.
* sources/azure: add networking check for all source PPS (#2061)Chris Patterson2023-03-162-12/+46
| | | | | | | | | | | | | | | | There is a networking check in _poll_imds() which will attempt DHCP again if networking is not up for source PPS. With the previous change to wait at least 20 minutes during provisioning for DHCP, this additional round is not necessary. Report failure if networking is not up for any mode of source PPS. In practice, this is very unlikely as provisioning will typically timeout within the 20 minute window the VM is attempting DHCP and the source PPS VM will be deleted. This fixes an (unobserved) issue where Savable PPS does not have networking prior to _wait_for_all_nics_ready(). Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* do not attempt dns resolution on ip addresses (#2040)Brett Holman2023-03-144-15/+35
|
* chore: fix style tip (#2071)Alberto Contreras2023-03-1426-55/+59
| | | | - remove too broad exceptions - ignore dynamic base types in templater
* Fix metadata IP in instancedata.rst (#2063)Brian Haley2023-03-102-1/+2
|
* util: Pass deprecation schedule in deprecate_call() (#2064)Brett Holman2023-03-101-0/+1
|
* config: Update grub-dpkg docs (#2058)Brett Holman2023-03-103-6/+6
| | | Co-authored-by: s-makin <sally.makin@canonical.com>
* docs: Cosmetic improvements and styling (#2057)s-makin2023-03-062-5/+285
| | | | | | | | | | | | | | | | Make the docs look more like the documentation produced on Ubuntu.com. Make the default font size larger for accessibility. Code literals are now the same size as regular font for the same reason, but made slightly bolded to help pick them out from text (which should help e.g. when commands are included as inline text and not a code block. Give the sidebar headings a higher weight, for easier navigation. For accessibility, change the weight and spacing between topics in the side bars to help separate them and make it easier to tell where one link ends and the next begins.
* cc_grub_dpkg: Added UEFI support (#2029)Alexander Birkner2023-03-063-49/+150
| | | | | | | | On Debian and Ubuntu based systems the cc_grub_dpkg module handles the needed change of the disk device name / path between the pre created image and the real hardware system. Currently it seems only BIOS mode is supported. This adds UEFI support as well to change the configuration keys for UEFI.
* tests: Write to /var/spool/rsyslog to adhere to apparmor profile (#2059)Chad Smith2023-03-051-2/+4
| | | | | AppArmor is active on 23.04. It prevents writing to /var/tmp. Integration test now writes to /var/spool/rsyslog/cloudinit.log to assert working config.
* oracle-ds: prefer system_cfg over ds network config source (#1998)Alberto Contreras2023-03-043-1/+49
| | | | | | | | | | | Bump system_cfg over ds network_config_source for Oracle DS, so that if network config is defined under /etc/cloud, it will be honored. In a previous change, we moved the initramfs and system_cfg bellow ds to favor ds, but this implied system-wide configs were always not honored. LP: #1956788
* Remove dead code (#2038)Brett Holman2023-03-0333-412/+7
|
* Release 23.1.1 (#2052)James Falcon2023-03-022-1/+6
| | | | Bump the version in cloudinit/version.py to 23.1.1 and update ChangeLog.
* source: Force OpenStack when it is only option (#2045)Brett Holman2023-03-0210-313/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running on OpenStack Ironic was broken in 1efa8a0a0, which prevented a system configured to run on only Openstack from actually running this ds. This change also prevents the kernel commandline definition from working. This change was required to prevent unnecessarily probing OpenStack on Ec2, and is therefore still required. This commit reverts an earlier attempt[1][2] to automatically detect OpenStack, due to regression it caused. Additionally, this change allows a system that defines a datasource list containing only [OpenStack] or [OpenStack, None] to attempt running on OpenStack, overriding ds_detect(). A datasource list that defines [OpenStack, None] still falls back to DataSourceNone if OpenStack fails to reach the IMDS. This change also lays groundwork for the following future work: 1. Add support for other datasources 2. Also override datasource checking when the kernel command line defines a datasource. This work needs to be done manually to support non-systemd systems. Besides forcing OpenStack to run when it is the only datasource in the datasource list, this commit also: [1] 0220295 (it breaks some use cases) [2] 29faf66 (no longer used) LP: #2008727
* cc_ubuntu_advantage: improve UA logs discoveryAlberto Contreras2023-03-012-4/+20
| | | | - Use log_time context manager to wrap log UA-API calls - Add a log msg pointing to UA logs
* sources/azure: fix regressions in IMDS behavior (#2041)Chris Patterson2023-03-014-191/+343
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are effectively two regressions in the recent IMDS refactor: 1. The metadata check len(imds_md["interface"]) in _check_if_nic_is_primary() is no longer correct as the refactor switched URLs and did not update this call to account for the fact that this metadata now lives under "network". 2. Network metadata was fetched with infinite=True and is now limited to ten retries. This callback had the twist of only allowing up to ten connection errors but otherwise would retry indefinetely. For check_if_nic_is_primary(): - Drop the interface count check for _check_if_nic_is_primary(), we don't need it anyways. - Fix/update the unit tests mocks that allowed the tests to pass, adding another test to verify max retries for http and connection errors. - Use 300 retries. We do want to hit a case where we spin forever, but this should be more than enough time for IMDS to respond in the Savable PPS case (~5 minutes). For IMDS: - Consolidate IMDS retry handlers into a new ReadUrlRetryHandler class that supports the options required for each variant of request. - Minor tweaks to log and expand logging checks in unit tests. - Move all unit tests to mocking via mock_requests_session_request and replace mock_readurl fixture with wrapped_readurl to improve consistency between tests. Note that this change drops usage of `retry_on_url_exc` and can probably be removed altogether as it is no longer used AFAICT. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
View Lorry Controller Status