summaryrefslogtreecommitdiff
path: root/tests/unittests/test_util.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove mount NTFS error message (#2134)Ksenija Stanojevic2023-05-091-3/+24
| | | | | | Provide an option to suppress error logging from mount_cb as some errors can be expected error and handled appropriately by DataSources. For example: failure to mount NTFS volumes on VMs that do not have NTFS drivers.
* Do not generate dsa and ed25519 key types when crypto FIPS mode is enabled ↵Ani Sinha2023-05-021-0/+25
| | | | | | | | | | | | (#2142) DSA and ED25519 key types are not supported when FIPS is enabled in crypto. Check if FIPS has been enabled on the system and if so, do not generate those key types. Presently the check is only available on Linux systems. LP: 2017761 RHBZ: 2187164 Signed-off-by: Ani Sinha <anisinha@redhat.com>
* gce: activate network discovery on every boot (#2128)Alberto Contreras2023-04-271-0/+21
| | | | | | | Google wants to allow users to make changes on nics while the instance is stopped. Activate network discovery on every boot. Additionally, skip the call to `netplan generate` if the rendered config is the same on subsequent boots.
* Fix Python 3.12 unit test failures (#2099)James Falcon2023-03-271-18/+20
|
* do not attempt dns resolution on ip addresses (#2040)Brett Holman2023-03-141-0/+15
|
* chore: fix style tip (#2071)Alberto Contreras2023-03-141-1/+1
| | | | - remove too broad exceptions - ignore dynamic base types in templater
* Set ownership for new folders in Write Files Module (#1980)Jack2023-02-211-0/+20
| | | | | | | The parent directory would be created automatically if it does not exist. But the ownership of newly-created parent directory would always be root. With this change, it would be set the same as `owner`. LP: #1990513
* add OpenCloudOS and TencentOS support (#1964)wynnfeng2023-02-181-0/+48
|
* nocloud: add support for dmi variable expansion for seedfrom URLChad Smith2022-12-151-14/+97
| | | | | | | | | | | | | | | | NoCloud meta-data seedfrom (or kernel commandline seedfrom) URL can now provide variable expansion for system-specific DMI values as seen in /sys/class/dmi/id on Linux or kenv on FreeBSD platforms. Variable names of the format __dmi.SOME_VAR__ will be replaced when determining the URL from which NoCloud datasource GETs its user-data and meta-data. This allows for a common templated seedfrom URL which can be reused for mass deployments, but can allow for unique URLs based on classes of DMI system characteristics such as chassis serial, product name, UUID etc. LP: #1994980
* mounts: document weird prefix in schema (#1913)Brett Holman2022-12-151-0/+5
| | | Add test and support for parsing IEC prefix format.
* add utility function test cases (#1910)sxt10012022-12-141-0/+13
| | | | | Add coverage for: - atomic_helper.write_file() - util.human2bytes()
* udevadm settle should handle non-udev system gracefully (#1806)dermotbradley2022-11-071-10/+25
| | | | | | | | | | | | Not all Linux systems use udev (and therefore udevadm). Specifically an Alpine Linux system may have either udev (via "eudev" package) or mdev, or mdevd installed. Change the udev_settle function to check for the presence of the udevadm binary before calling it - if it is not present then silently exit the function. This change will enable cloud-init to run on Alpine systems using mdev/mdevd rather than udev.
* add mariner support (#1780)Minghe Ren2022-11-071-0/+23
|
* add support for Container-Optimized OS (#1748)vteratipally2022-10-201-0/+20
| | | Signed-off-by: varsha teratipally <teratipally@google.com>
* util: Implement __str__ and __iter__ for Version (#1790)Brett Holman2022-10-181-0/+14
|
* Refactor: Drop inheritance from object (#1728)Alberto Contreras2022-09-141-1/+1
| | | As we do not support python2 anymore, we can remove those 8 chars.
* Allow jinja templating in /etc/cloud (SC-1170) (#1722)James Falcon2022-09-141-2/+57
| | | | | | | | | | | | | | There have been multiple requests to allow jinja templating in /etc/cloud configs the same way we allow jinja templating in vendordata and userdata. This commit allows for templating both /etc/cloud/cloud.cfg and any file in /etc/cloud/cloud.cfg.d. The same instance data used for substitution in vendordata and userdata will be used here. Note that these configs get loaded multiple times during the lifetime of cloud-init, and during cloud-init's earlier loads, instance data is not yet available. LP: #1913461
* net: Ensure a tmp with exec permissions for dhcp (#1690)Alberto Contreras2022-09-011-0/+17
| | | | | | | | In the case cloudinit.temp_utils points to a fs mounted as noexec and needs_exe=True, fallback to use os.join.path(Distro.usr_lib_exec, "cloud-init/clouddir) that will be mounted with exec perms. LP: #1962343
* Add Ansible Config Module (#1579)Brett Holman2022-08-051-1/+56
| | | | | | | | | | | | | | | | | | | | | The intended purpose of this commit is to augment cloud-init's configuration capabilities during boot. This allows users to run ansible playbooks against the local node as part of the boot order. Current Ansible workflows on nodes booted by cloud-init requires waiting for the booted node before running the playbook against the online node from the external control node. In the current state this could potentially be automated to using phone-home or runcmd modules, however neither of these options provides an obvious solution for integration and both require an external service to operate. This module enables users to seamlessly integrate cloud-init and Ansible auto-configuration during boot using ansible-pull, a less common mode of operation that differs from the normal mode of operation by installing Ansible locally and running a playbook pulled from a VCS repository. Expected future work in this module includes an option to configure management nodes.
* util: Fix error path and parsing in get_proc_ppid()Brett Holman2022-08-051-0/+41
| | | | | | | | | If IOError was thrown, the variable "contents" was unbound and threw a name error exception. Parsing /proc/<num>/stat was broken in less common cases. Fix both and add parsing tests.
* Resource leak cleanup (#1556)Brett Holman2022-07-131-1/+2
| | | Add tox target for tracing for resource leaks, fix some leaks
* testing: remove appereances of FakeCloud (#1584)Alberto Contreras2022-07-121-35/+28
|
* cloud-config: honor cloud_dir setting (#1523)Alberto Contreras2022-06-221-0/+18
| | | | | | | | | | | Ensure cloud_dir setting is respected rather than hardcoding "/var/lib/cloud" - Modules affected: cmd.main, apport, devel.logs (collect-logs), cc_snap, sources.DataSourceAzure, sources.DataSourceBigstep, util:fetch_ssl_details. - testing: Extend and port to pytest unit tests, add integration test. LP: #1976564
* Add support for OpenMandriva (https://openmandriva.org/) (#1520)Bernhard Rosenkraenzer2022-06-171-0/+27
| | | Signed-off-by: Bernhard Rosenkränzer <bero@lindev.ch>
* Remove xenial references (#1472)Alberto Contreras2022-06-081-3/+1
| | | | | | - Remove references and dead code to Xenial, Eoan, Python < 3.7 - cc_ubuntu_drivers: Use python3-debconf instead of shell script - add integration test for ubuntu_drivers - bump pycloudlib for OCI subnet/jammy fixes
* cc_set_hostname: do not write "localhost" when no hostname is given (#1453)Emanuele Giuseppe Esposito2022-05-311-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Systemd used to sometimes ignore localhost in /etc/hostnames, and many programs like cloud-init used this as a workaround to set a default hostname. From https://github.com/systemd/systemd/commit/d39079fcaa05e23540d2b1f0270fa31c22a7e9f1: We would sometimes ignore localhost-style names in /etc/hostname. That is brittle. If the user configured some hostname, it's most likely because they want to use that as the hostname. If they don't want to use such a hostname, they should just not create the config. Everything becomes simples if we just use the configured hostname as-is. This behaviour seems to have been a workaround for Anaconda installer and other tools writing out /etc/hostname with the default of "localhost.localdomain". Anaconda PR to stop doing that: rhinstaller/anaconda#3040. That might have been useful as a work-around for other programs misbehaving if /etc/hostname was not present, but nowadays it's not useful because systemd mostly controls the hostname and it is perfectly happy without that file. Apart from making things simpler, this allows users to set a hostname like "localhost" and have it honoured, if such a whim strikes them. As also suggested by the Anaconda PR, we need to stop writing default "localhost" in /etc/hostnames, and let the right service (networking, user) do that if they need to. Otherwise, "localhost" will permanently stay as hostname and will prevent other tools like NetworkManager from setting the right one. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> RHBZ: 1980403
* Drop mypy excluded files (#1454)Alberto Contreras2022-05-231-12/+1
| | | | | | | | | - Add types to let mypy pass. - Add mypy flags: - detect unused ignores - redundant casts - Drop support of `ConfigParser` in Python 2 - Harden DataSourceLXD.network_config - Convert old-style commented types to proper types.
* cli: Redact files with permission errors in commands (#1440)Alberto Contreras2022-05-171-67/+152
| | | | | | | | | | | | For non-root users, emit warnings and redact on any /etc/cloud/cloud.cfg.d files which raise permissions errors. Add tests covering this behavior for query, status and render cmds. Migrate `test_render.py` and `test_status.py` to Pytest. LP: #1953430 SC-658
* Return a namedtuple from subp() (#1376)Brett Holman2022-04-121-10/+17
| | | | | | This provides a minor readability improvement. subp.subp(cmd)[0] -> subp.subp(cmd).stdout subp.subp(cmd)[1] -> subp.subp(cmd).stderr
* util: atomically update sym links to avoid Suppress FileNotFoundError when ↵Adam Collard2022-04-081-1/+4
| | | | | | | | | | | | reading status (#1298) Atomically update the desired link file from a temporary file when a stale link already exists. This avoids FileNotFound errors due to races with cloud-init status --wait when the symlink /run/cloud-init/status.json already exists. LP:1962150
* BUG 1473527: module ssh-authkey-fingerprints fails Input/output error… (#1340)Andrew Lee2022-04-071-0/+27
| | | | | | | | | | | | | | | | Don't error if we cannot log to /dev/console We've seen instances on VMware of serial consoles not being set up correctly by the kernel, making /dev/ttyS0 not set up correctly, and hence /dev/console not writeable to. In such circumstances, cloud-init should not fail, instead it should gracefully fall back to logging to stdout. The only time cloud-init tries to write to `/dev/console` is in the `multi_log` command- which is called by the ssh-authkey-fingerprints module LP: #1473527
* check for existing symlink while force creating symlink (#1281)Shreenidhi Shedi2022-02-241-0/+46
| | | | | | If a dead symlink by the same name is present, os.path.exists returns false, use os.path.lexists instead. Signed-off-by: Shreenidhi Shedi <sshedi@vmware.com>
* Remove 3.5 and xenial support (SC-711) (#1167)James Falcon2022-01-101-1/+1
| | | | | | | | | | | | Includes: - Update tox.ini and .travis.yml accordingly - Cleanup tox.ini with new tox syntax and cloud-init dependencies - Update documentation accordingly - Replace/remove xenial references where additional testing isn't required - Remove xenial checks in integration tests - Replace yield_fixture with fixture in pytest tests Sections of code commented with lines like "Remove when Xenial is no longer supported" still exist as they're require additional testing.
* Adopt Black and isort (SC-700) (#1157)James Falcon2021-12-151-479/+455
| | | | | Applied Black and isort, fixed any linting issues, updated tox.ini and CI.
* find_devs_with_openbsd: ensure we return the last entry (#1149)Gonéri Le Bouder2021-12-131-1/+1
| | | | | | | | | | | | | | | `sysctl -n hw.disknames` returns a trailing `\n`. We need to clean this up. In addition, the criteria matching system is a source of problem because: - we don't have a way to look up the label of the partition - we've got situation where an ISO image can be exposed through a virtio block device. So we just totally ignore the value of `criteria`. We end-up with a slightly longer loop of mount-retry. But this way we're sure we don't miss a configuration disk. Tested on Kubvirt with the help of Brady Pratt @jbpratt.
* find_devs/openbsd: accept ISO on disk (#1132)Gonéri Le Bouder2021-12-091-2/+2
| | | | | | | | | | | | When the metadata is an ISO image and is exposed through a disk, the device is called `/dev/sd?a` internally. For instance `/dev/sd1a`. It can then be mounted with `mount_cd9660 /dev/sd1a /mnt`. Metadata in the FAT32 format are exposed as `/dev/sd?i`. With this change, we try to mount `/dev/sd?a` in addition to `/dev/sd?i`. Closes: https://github.com/ContainerCraft/kmi/issues/12
* testing: Add deterministic test id (#1138)Brett Holman2021-12-071-1/+7
| | | | | Parametrized pytest tests get named by on their parameters. If a name has random characters, it can break the test collection of when using pytest-xdist. Replace random name with deterministic name.
* Add miraclelinux support (#1128)Haruki TSURUMOTO2021-12-071-0/+38
|
* Reorganize unit test locations under tests/unittests (#1126)Brett Holman2021-12-031-138/+1522
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This attempts to standardize unit test file location under test/unittests/ such that any source file located at cloudinit/path/to/file.py may have a corresponding unit test file at test/unittests/path/to/test_file.py. Noteworthy Comments: ==================== Four different duplicate test files existed: test_{gpg,util,cc_mounts,cc_resolv_conf}.py Each of these duplicate file pairs has been merged together. This is a break in git history for these files. The test suite appears to have a dependency on test order. Changing test order causes some tests to fail. This should be rectified, but for now some tests have been modified in tests/unittests/config/test_set_passwords.py. A helper class name starts with "Test" which causes pytest to try executing it as a test case, which then throws warnings "due to Class having __init__()". Silence by changing the name of the class. # helpers.py is imported in many test files, import paths change cloudinit/tests/helpers.py -> tests/unittests/helpers.py # Move directories: cloudinit/distros/tests -> tests/unittests/distros cloudinit/cmd/devel/tests -> tests/unittests/cmd/devel cloudinit/cmd/tests -> tests/unittests/cmd/ cloudinit/sources/helpers/tests -> tests/unittests/sources/helpers cloudinit/sources/tests -> tests/unittests/sources cloudinit/net/tests -> tests/unittests/net cloudinit/config/tests -> tests/unittests/config cloudinit/analyze/tests/ -> tests/unittests/analyze/ # Standardize tests already in tests/unittests/ test_datasource -> sources test_distros -> distros test_vmware -> sources/vmware test_handler -> config # this contains cloudconfig module tests test_runs -> runs
* tox: bump the pinned flake8 and pylint version (#1029)Paride Legovini2021-09-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tox: bump the pinned flake8 and pylint version * pylint: fix W1406 (redundant-u-string-prefix) The u prefix for strings is no longer necessary in Python >=3.0. * pylint: disable W1514 (unspecified-encoding) From https://www.python.org/dev/peps/pep-0597/ (Python 3.10): The new warning stems form https://www.python.org/dev/peps/pep-0597, which says: Developers using macOS or Linux may forget that the default encoding is not always UTF-8. [...] Even Python experts may assume that the default encoding is UTF-8. This creates bugs that only happen on Windows. The warning could be fixed by always specifying encoding='utf-8', however we should be careful to not break environments which are not utf-8 (or explicitly state that only utf-8 is supported). Let's silence the warning for now. * _quick_read_instance_id: cover the case where load_yaml() returns None Spotted by pylint: - E1135 (unsupported-membership-test) - E1136 (unsubscriptable-object) LP: #1944414
* add DragonFlyBSD support (#904)Gonéri Le Bouder2021-06-141-0/+18
| | | | | | | - Mostly based on FreeBSD, the main exception is that `find_devs_with_on_freebsd` does not work. - Since we cannot get the CDROM or the partition labels, `find_devs_with_on_dragonflybsd()` has a more naive approach and returns all the block devices.
* write passwords only to serial console, lock down cloud-init-output.log (#847)Daniel Watkins2021-03-191-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Prior to this commit, when a user specified configuration which would generate random passwords for users, cloud-init would cause those passwords to be written to the serial console by emitting them on stderr. In the default configuration, any stdout or stderr emitted by cloud-init is also written to `/var/log/cloud-init-output.log`. This file is world-readable, meaning that those randomly-generated passwords were available to be read by any user with access to the system. This presents an obvious security issue. This commit responds to this issue in two ways: * We address the direct issue by moving from writing the passwords to sys.stderr to writing them directly to /dev/console (via util.multi_log); this means that the passwords will never end up in cloud-init-output.log * To avoid future issues like this, we also modify the logging code so that any files created in a log sink subprocess will only be owner/group readable and, if it exists, will be owned by the adm group. This results in `/var/log/cloud-init-output.log` no longer being world-readable, meaning that if there are other parts of the codebase that are emitting sensitive data intended for the serial console, that data is no longer available to all users of the system. LP: #1918303
* cloudinit: move dmi functions out of util (#622)Scott Moser2020-11-021-123/+0
| | | | | | | | | This just separates the reading of dmi values into its own file. Some things of note: * left import of util in dmi.py only for 'is_container' It'd be good if is_container was not in util. * just the use of 'util.is_x86' to dmi.py * open() is used directly rather than load_file.
* Add vendor-data support to seedfrom parameter for NoCloud and OVF (#570)Johann Queuniet2020-09-181-1/+23
|
* Detect kernel version before swap file creation (#428)Eduardo Otubo2020-08-181-0/+16
| | | | | | | | | | According to man page `man 8 swapon', "Preallocated swap files are supported on XFS since Linux 4.18". This patch checks for kernel version before attepting to create swapfile, using dd for XFS only on kernel versions <= 4.18 or btrfs. Add new func util.kernel_version which returns a tuple of ints (major, minor) Signed-off-by: Eduardo Otubo otubo@redhat.com
* test_util: parametrize devlist tests (#523)James Falcon2020-08-071-32/+24
|
* Recognize LABEL_FATBOOT labels (#513)James Falcon2020-08-071-83/+112
| | | | | | Update DataSourceNoCloud and ds-identify to recognize LABEL_FATBOOT labels from blkid. Also updated associated tests. LP: #1841466
* util: add ensure_dir_exists parameter to write_file (#443)Daniel Watkins2020-06-191-0/+11
| | | | This allows us to disable the `ensure_dir` call when it isn't appropriate.
* util: rename write_file's copy_mode parameter to preserve_mode (#439)Daniel Watkins2020-06-171-6/+6
| | | | | When updating the docstring to include it, I realised that the current name is somewhat misleading; this makes it a little easier to understand, I think.
* Move subp into its own module. (#416)Scott Moser2020-06-081-236/+22
| | | | | | | | | | | | | | | | | This was painful, but it finishes a TODO from cloudinit/subp.py. It moves the following from util to subp: ProcessExecutionError subp which target_path I moved subp_blob_in_tempfile into cc_chef, which is its only caller. That saved us from having to deal with it using write_file and temp_utils from subp (which does not import any cloudinit things now). It is arguable that 'target_path' could be moved to a 'path_utils' or something, but in order to use it from subp and also from utils, we had to get it out of utils.
View Lorry Controller Status