summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Release 23.1.223.1.223.1.xJames Falcon2023-04-202-1/+5
| | | | | Bump the version in cloudinit/version.py to 23.1.2 and update ChangeLog.
* Make user/vendor data sensitive and remove log permissionsJames Falcon2023-04-206-23/+77
| | | | | | | | | | | | | | Because user data and vendor data may contain sensitive information, this commit ensures that any user data or vendor data written to instance-data.json gets redacted and is only available to root user. Also, modify the permissions of cloud-init.log to be 640, so that sensitive data leaked to the log isn't world readable. Additionally, remove the logging of user data and vendor data to cloud-init.log from the Vultr datasource. LP: #2013967 CVE: CVE-2023-1786
* Release 23.1.123.1.1James Falcon2023-03-022-1/+6
| | | | | Bump the version in cloudinit/version.py to 23.1.1 and update ChangeLog.
* source: Force OpenStack when it is only option (#2045)Brett Holman2023-03-0210-312/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Running on OpenStack Ironic was broken in 1efa8a0a0, which prevented a system configured to run on only Openstack from actually running this ds. This change also prevents the kernel commandline definition from working. This change was required to prevent unnecessarily probing OpenStack on Ec2, and is therefore still required. This commit reverts an earlier attempt[1][2] to automatically detect OpenStack, due to regression it caused. Additionally, this change allows a system that defines a datasource list containing only [OpenStack] or [OpenStack, None] to attempt running on OpenStack, overriding ds_detect(). A datasource list that defines [OpenStack, None] still falls back to DataSourceNone if OpenStack fails to reach the IMDS. This change also lays groundwork for the following future work: 1. Add support for other datasources 2. Also override datasource checking when the kernel command line defines a datasource. This work needs to be done manually to support non-systemd systems. Besides forcing OpenStack to run when it is the only datasource in the datasource list, this commit also: [1] 0220295 (it breaks some use cases) [2] 29faf66 (no longer used) LP: #2008727
* sources/azure: fix regressions in IMDS behavior (#2041)Chris Patterson2023-03-024-191/+343
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are effectively two regressions in the recent IMDS refactor: 1. The metadata check len(imds_md["interface"]) in _check_if_nic_is_primary() is no longer correct as the refactor switched URLs and did not update this call to account for the fact that this metadata now lives under "network". 2. Network metadata was fetched with infinite=True and is now limited to ten retries. This callback had the twist of only allowing up to ten connection errors but otherwise would retry indefinetely. For check_if_nic_is_primary(): - Drop the interface count check for _check_if_nic_is_primary(), we don't need it anyways. - Fix/update the unit tests mocks that allowed the tests to pass, adding another test to verify max retries for http and connection errors. - Use 300 retries. We do want to hit a case where we spin forever, but this should be more than enough time for IMDS to respond in the Savable PPS case (~5 minutes). For IMDS: - Consolidate IMDS retry handlers into a new ReadUrlRetryHandler class that supports the options required for each variant of request. - Minor tweaks to log and expand logging checks in unit tests. - Move all unit tests to mocking via mock_requests_session_request and replace mock_readurl fixture with wrapped_readurl to improve consistency between tests. Note that this change drops usage of `retry_on_url_exc` and can probably be removed altogether as it is no longer used AFAICT. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* Release 23.123.1Alberto Contreras2023-02-222-1/+149
| | | | | Bump the version in cloudinit/version.py to 23.1 and update ChangeLog.
* Support transactional-updates for SUSE based distros (#1997)Robert Schweikert2023-02-213-9/+404
| | | | | | openSUSE/SUSE has distros that use read only root and btrfs. To update a running system in such a setup the transactional-update command needs to be used. This change implements support for use of the transactional-update commend when appropriate.
* Set ownership for new folders in Write Files Module (#1980)Jack2023-02-216-4/+90
| | | | | | | The parent directory would be created automatically if it does not exist. But the ownership of newly-created parent directory would always be root. With this change, it would be set the same as `owner`. LP: #1990513
* add OpenCloudOS and TencentOS support (#1964)wynnfeng2023-02-1815-10/+89
|
* lxd: Retry if the server isn't ready (#2025)Brett Holman2023-02-182-2/+57
|
* test: switch pycloudlib source to pypi (#2024)Brett Holman2023-02-171-1/+1
|
* test: Fix integration test deprecation message (#2023)Brett Holman2023-02-171-9/+18
|
* Recognize opensuse-microos, dev tooling fixesRobert Schweikert2023-02-1620-110/+465
| | | | | | | | | | | | Update the distro selection code to recognize opensuse-microos as a SUSE based distribution. Also in this commit: * unittest mock OpenNebula of pwd.getpwnam to avoid test leaks on SuSE * tooling fixes to build and test opensuse - read-dependencies fix jinja2 and PyYAML pkg aliases for opensuse - Consolidate package operations based on OS family instead of distro flavor to cut down on duplication of command definitions. - format read-dependencies and run-container with black
* sources/azure: refactor imds handler into own module (#1977)Chris Patterson2023-02-165-894/+799
| | | | | | | | | | Create new azure package for better organization and move IMDS logic for fetching into it. Future work will clean up the test_azure.py tests a little further thanks to these changes, but wanted to minimize churn here to make changes fairly visible. Signed-off-by: Chris Patterson <cpatterson@microsoft.com>
* docs: deprecation generation support [1/2] (#2013)Brett Holman2023-02-1613-160/+374
| | | | | | | | | | | | | | | | | | docs: deprecation version generation support schema: Add the following metadata keys: - changed_version - deprecated_version - new_version - changed_description - new_description - deprecation_description - changed - new Generate formatted docs from this new metadata. Update current descriptions containing unstructured data. Switch current deprecation from a date to deprecated version. Ensure changed/new/deprecated keys have associated version keys.
* add function is_virtual to distro/FreeBSD (#1957)Mina Galić2023-02-162-1/+80
| | | | | | | | | | | | - is_virtual property identifies identify if the thing we're running is any kind of virtualization - virtual() identifies what kind of virtualisation we're dealing with - is_container() tells us if we're running in a container, or in FreeBSD's case, in a jail. - the helper functions are @lru_cached, since this is very unlikely to change Sponsored by: The FreeBSD Foundation Co-authored-by: Brett Holman <brett.holman@canonical.com>
* cc_ssh: support multiple hostcertificates (#2018)Alberto Contreras2023-02-165-21/+91
| | | LP: #1999164
* Fix minor schema validation regression and fixup typing (#2017)James Falcon2023-02-153-7/+15
| | | | | When annotating a schema, we should print "Valid cloud-config" along with the filename if a file has been used rather than just the config type.
* doc: Reword user data debug section (#2019)Brett Holman2023-02-151-6/+21
|
* Overhaul/rewrite of certificate handling as follows: (#1962)dermotbradley2023-02-144-225/+199
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change "ca-certs" references to "ca_certs". New certificates are written to individual files, with an incrementing number as part of their filename, rather than all being placed in a single file. This resolves issues caused when certificate files containing more than a single certificate are placed in /etc/ssl/certs (by utilities such as "update-ca-certificates" run by ca_certs). Alpine / Debian / Ubuntu: The current behaviour, whilst it works, is incorrect with regard to the design of the underlying OS utilities for managing certificates. For "remove_defaults" the system-installed certificate files should not be actually deleted (otherwise it becomes problematic if someone wishes to later re-enable one or more of them), rather they should be deactivated and these OSes already provide the means to do so - this MR modifies the certificate entries in the /etc/ca-certificates.conf file by prefixing them with "!" - when the update-ca-certificate utility is then run it will *not* place such delimited certificates into either the /etc/ssl/certs/ directory (via symlinks) nor add them to the (re)generated certificates bundle file. Additionally it is incorrect for added certificates to be placed in the /usr/share/ca-certificates directory - this location is intended for standard/"official" certificates, the /usr/local/share/ca-certificates directory is intended for "local" or "site-specific" certificates and so this PR adds them there instead - for certs in /usr/local/share/ca-certificates the update-ca-certificates utility will automatically use them, there is *no* need to add their filenames to the /etc/ca-certificates.conf file. LP: #1931174
* disk_setup: use byte string when purging the partition table (#2012)Stefan Prietl2023-02-132-2/+19
| | | | | | | This writes a byte string to the device instead of a string when purging the partition table. Essentially, this will prevent the error "a bytes-like object is required, not 'str'" from happening.
* cli: schema also validate vendordata*.Chad Smith2023-02-095-129/+195
| | | | | | | | | | | | | | | | cloud-init schema --annotate now walks any valid user-data, vendor-data and vendor2-data to report whether any of these cloud-config files provide invalid or deprecated schema values. Also, fix a bug in nested mapping annotations to properly report the full nested indexed schema path. The following now can be annotated without KeyErrors: users: - default - lock-passwd: false name: root
* ci: sort and add checks for cla signers fileStefan Prietl2023-02-094-2/+27
| | | | | | | This sorts the CLA signers file and adds a convenience script for users to check and sort the file. A workflow job - which uses the script - makes sure that the file does not get merged in an unsorted state.
* Add "ederst" as contributor (#2010)Stefan Prietl2023-02-091-0/+1
|
* readme: add reference to packages dir (#2001)Alberto Contreras2023-02-082-1/+16
|
* docs: update downstream package list (#2002)Brett Holman2023-02-081-0/+4
|
* docs: add google search verification (#2000)s-makin2023-02-082-0/+3
| | | | | | Added google verification file to ensure that the Google bots can find and index the site. Should speed up the listing of our docs in Google's search results.
* docs: fix 404 render use default notfound_urls_prefix in RTD conf (#2004)Chad Smith2023-02-081-1/+0
| | | | | | | 404 styling was off due to invalid config notfound_urls_prefix = "/" Drop custom config. The default "/en/latest" value properly renders the 404 page with style.
* Fix OpenStack datasource detection on bare metal (#1923)Alexander Birkner2023-02-078-52/+239
| | | LP: #1815990
* docs: add themed RTD 404 page and pointer to readthedocs-hosted (#1993)Chad Smith2023-02-072-1/+17
|
* schema: fix gpt labels, use type string for GUID (#1995)Brett Holman2023-02-062-3/+30
| | | LP #2004599
* cc_disk_setup: code cleanup (#1996)Brett Holman2023-02-061-45/+17
|
* netplan: keep custom strict perms when 50-cloud-init.yaml existsChad Smith2023-02-042-32/+55
| | | | | Retain existing config file permissions when those permissions are more strict than the default permissions set on /etc/netplan/50-cloud-init.yaml.
* cloud-id: better handling of change in datasource filesd1r3ct0r2023-02-023-7/+31
| | | | | | Stop deleting cloud-id-<cloudname> unless there is a change in discovered datasource cloud-id. LP: #1998998
* tests: Remove restart check from testJames Falcon2023-02-021-4/+0
| | | | If no SSH restart happens, we get no log. Since this is perfectly normal, we shouldn't be checking for a log.
* Ignore duplicate macs from mscc_felix and fsl_enetcJames Falcon2023-02-022-0/+36
| | | | | | | mscc_felix and fsl_enetc are drivers representing a switch that is expected to have duplicate macs. If we encounter either of these drivers, we should not raise the duplicate mac exception. LP: #1997922
* Warn on empty network key (#1990)Brett Holman2023-02-022-0/+22
| | | | It is possible to create an empty network key that fails over to fallback network config without warning. Fix it.
* Fix Vultr cloud_interfaces usage (#1986)eb30952023-02-013-8/+75
| | | cloud_interfaces is intended to be a netcfg. Change the helper function to reflect this.
* cc_puppet: Update puppet service name (#1970)d1r3ct0r2023-02-013-103/+157
| | | | | | | | | | | | | | | cc_puppet: Update puppet service name to puppet-agent From Lunar, we see that the default puppet version is 7.20 which replaces `puppet.service` with `puppet-agent.service`. Thus, we need to have a way of calling the appropriate service depending on the distribution of puppet installed. Attempt to install, start or enable puppet-agent first and fallback to puppet. Log warnings if neither preferred package names exist or if the package_name in user-data is not able to be configured. LP: #2002969
* docs: Clarify networking docs (#1987)Brett Holman2023-01-312-8/+15
| | | | - Add stronger warnings for resolv_conf module. - Add link to lxd network config docs.
* lint: remove httpretty (#1985)sxt10012023-01-311-1/+0
| | | | | Drop httpretty from list of ignored modules to type-check. This module was dropped in favor of responses.
* cc_set_passwords: Prevent traceback when restarting ssh (#1981)James Falcon2023-01-302-15/+19
| | | | | On Bionic in GCE, when restarting ssh, it's not uncommon to see a traceback with `ssh.service is not active, cannot reload.`. Instead log a warning.
* tests: fix lp1912844 (#1978)Alberto Contreras2023-01-301-1/+1
| | | | Fix test_lp1912844.py where the intance's ip was not correctly detected in pycloudlib
* tests: Skip ansible test on bionic (#1984)James Falcon2023-01-301-0/+3
| | | Bionic will no longer be supported by version in pip
* Wait for NetworkManager (#1983)Robert Schweikert2023-01-301-1/+2
| | | | | Newer SUSE distributions are switching to NetworkManager from wicked. For those distributions we need to wait for NetworkManager before starting the cloud-init service.
* docs: minor polishing (#1979)s-makin2023-01-275-7/+11
| | | | | | | | | | Minor changes for Diataxis: 1) Added "reverse" to front page section blocks so they display correctly on mobile/smaller screens. 2) Added links to security/performance pages from Explanation section. These were linking directly to the pages in the Development section and will eventually become more like signposts as more content is added/revised.
* CI: migrate integration-test to GH actions (#1969)Alberto Contreras2023-01-262-105/+84
| | | | | | | changes: - drop caching machinery for LXD images and chroots - divide the workflow in two steps, one for building packages and other for running the tests so that the second one can be retried in case of error
* Fix permission of SSH host keys (#1971)Ron Gebauer2023-01-253-5/+6
| | | | | | | | | If the host-keys are provided the private key permissions have 0600 which is indeed correct. But the public key has 0600 which should instead be 0644. With this change the public key is always 0644 and the private key is 0600 if provided or 640 if generated (to match sshd-keygen functionality).
* Fix default route rendering on v2 ipv6 (#1973)James Falcon2023-01-242-2/+2
| | | | | | ::/0 would get rendered as ::/64 rather than ::/0 across all renderers using ipv6 in a v2 config. LP: #2003562
* doc: fix path in net_convert command (#1975)Brett Holman2023-01-231-1/+1
|
View Lorry Controller Status