summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* update changelog: add sru bug ref in changelog, remove individual ref.ubuntu/18.3-9-g2e62cb8a-0ubuntu1_17.10.1ubuntu/artfulScott Moser2018-07-101-2/+1
| | | | | | | Just remove the reference to bug 1780481 that was added to debian/changelog. Also, add a reference (re-using) the SRU bug 1777912. (cherry picked from commit b07ebf52e94fa5f94708f86937af4dba382168ba)
* releasing cloud-init version 18.3-9-g2e62cb8a-0ubuntu1~17.10.1Chad Smith2018-07-091-1/+2
|
* update changelog (New upstream snapshot 18.3-9-g2e62cb8a).Chad Smith2018-07-091-0/+16
|
* merge from master at 18.3-9-g2e62cb8aChad Smith2018-07-0920-129/+640
|\
| * docs: note in rtd about avoiding /tmp when writing filesChad Smith2018-07-095-3/+25
| | | | | | | | LP: #1727876
| * ubuntu,centos,debian: get_linux_distro to align with platform.distChad Smith2018-07-093-12/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A recent commit added get_linux_distro to replace the deprecated python platform.dist module behavior before it is dropped from python. It added behavior that was compliant on OpenSuSE and SLES, by returning (<distro_name>, <distro_version>, <cpu-arch>). Fix get_linux_distro to behave more like the specific distribution's platform.dist on ubuntu, centos and debian, which will return the distribution release codename as the third element instead of <cpu-arch>. SLES and OpenSUSE will retain their current behavior. Examples follow: ('sles', '15', 'x86_64') ('opensuse', '42.3', 'x86_64') ('debian', '9', 'stretch') ('ubuntu', '16.04', 'xenial') ('centos', '7', 'Core') LP: #1780481
| * Fix boothook docs on environment variable name (INSTANCE_I -> INSTANCE_ID)Marc Tamsky2018-07-031-1/+1
| |
| * update_metadata: a datasource can support network re-config every bootChad Smith2018-07-015-6/+417
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Very basic type definitions are now defined to distinguish 'boot' events from 'new instance (first boot)'. Event types will now be handed to a datasource.update_metadata method which can determine whether to refresh its metadata and re-render configuration based on that source event. A datasource can 'subscribe' to an event by setting up the update_events attribute on the datasource class which describe what config scope is updated by a list of matching events. By default datasources will have the following update_events: {'network': [EventType.BOOT_NEW_INSTANCE]} This setting says the datasource will re-write network configuration only on first boot of a new instance or when the instance id changes. New methods are now present on the datasource: - clear_cached_attrs: Resets cached datasource attributes to values listed in datasource.cached_attr_defaults. This is performed prior to processing a fresh metadata process to avoid keeping old/invalid cached data around. - update_metadata: accepts source_event_types to determine if the metadata should be crawled again and processed
| * tests: drop salt-minion integration testScott Moser2018-06-282-87/+0
| | | | | | | | | | | | | | | | | | | | | | The salt minion integration test as we had it did not do a whole lot more than the unit tests on that module did. Additionally, it caused some transient failures at least in Ubuntu 18.04. At a future date we may choose to add an integration test that installs salt-minion and salt server and configures it to be a better test. LP: #1778737
| * Retry on failed import of gpg receive keys.Scott Moser2018-06-282-10/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When cloud-init tries to read a key from a keyserver, it will now retry twice with 1 second in between each. Retries of import are done by default because keyservers can be unreliable. Additionally, there is no way to determine the difference between a non-existant key and a failure. In both cases gpg (at least 2.2.4) exits with status 2 and stderr: "keyserver receive failed: No data" It is assumed that a key provided to cloud-init exists on the keyserver so re-trying makes better sense than failing. Examples of things that made receive keys particularly unreliable:   https://bitbucket.org/skskeyserver/sks-keyserver/issues/57   https://bitbucket.org/skskeyserver/sks-keyserver/issues/60 There is also a change here from 'gpg --recv' to the longer 'gpg --recv-keys'. That option is functional and working back to centos 6 (gpg 2.0.14) and ubuntu 14.04 (gpg 1.4.16).
| * tools: Fix run-container when neither source or binary package requested.Scott Moser2018-06-281-8/+10
| | | | | | | | | | | | | | | | If run-container was called without --package or --binary-package, then it would still try to copy out artifacts and would fail doing so as there were no artifacts to collect. Also fix a bug when only --source-package without --package.
| * docs: Fix a small spelling error.Oz N Tiram2018-06-281-1/+1
| | | | | | | | standargs -> standards.
| * tox: use simplestreams from git repository rather than bzr.Scott Moser2018-06-261-1/+1
| | | | | | | | Simplestreams is moving to git. Use the git repo rather than bzr.
* | update changelog: remove refresh of added patchubuntu/18.3-0ubuntu1_17.10.1Scott Moser2018-06-221-2/+0
| |
* | releasing cloud-init version 18.3-0ubuntu1~17.10.1Chad Smith2018-06-211-1/+1
| |
* | update changelog (New upstream release 18.3).Chad Smith2018-06-211-3/+92
| |
* | refresh patches against master commit 2d6e4219Chad Smith2018-06-211-4/+2
| | | | | | | | | | refresh patches against master commit 2d6e4219: debian/patches/openstack-no-network-config.patch
* | merge from master at 18.3Chad Smith2018-06-21199-1959/+7188
|\ \ | |/
| * release 18.318.3Chad Smith2018-06-192-1/+227
| | | | | | | | | | | | Bump the version in cloudinit/version.py to be 18.3 and update ChangeLog. LP: #1777743
| * docs: represent sudo:false in docs for user_groups config moduleChad Smith2018-06-191-2/+6
| |
| * Explicitly prevent `sudo` access for user moduleJacob Bednarz2018-06-194-9/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | To deny a user elevated access, you can omit the `sudo` key from the `users` dictionary. This works fine however it's implicitly defined based on defaults of `cloud-init`. If the project moves to have `sudo` access allowed for all by default (quite unlikely but still possible) this will catch a few people out. This introduces the ability to define an explicit `sudo: False` in the `users` dictionary and it will prevent `sudo` access. The behaviour is identical to omitting the key. LP: #1771468
| * lxd: Delete default network and detach device if lxd-init created them.Scott Moser2018-06-152-24/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Newer versions (3.0.1+) of lxd create the 'lxdbr0' network when 'lxd init --auto' is invoked. When cloud-init is given a network configuration to pass on to lxc and that config had no name specified or 'lxdbr0', then cloud-init would fail to create the network as it already exists. Similarly, we need to remove the device from the default profile so that the attach code can work. Also, add a _lxc method and use it to make sure we're getting the --force-local flag everywhere. LP: #1776958
| * openstack: avoid unneeded metadata probe on non-openstack platformsChad Smith2018-06-155-16/+182
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OpenStack datasource is now discovered in init-local stage. In order to probe whether OpenStack metadata is present, it performs a costly sandboxed dhclient setup and metadata probe against http://169.254.169.254 for openstack data. Cloud-init properly detects non-OpenStack on EC2, but it spends precious time probing the metadata service also resulting in a confusing WARNING log about 'metadata not present'. To avoid the wasted cycles, and confusing warning, get_data will call a detect_openstack function to quickly determine whether the platform looks like OpenStack before trying to setup network to probe and crawl the metadata service. LP: #1776701
| * stages: fix tracebacks if a module stage is undefined or emptyRobert Schweikert2018-06-152-3/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In /etc/cloud/cloud.cfg, users and imagees can configure which modules run during a specific cloud-init stage by modifying one of the following lists: cloud_init_modules, cloud_init_modules, cloud_init_final_modules. If any of the configured module lists are absent or empty, cloud-init will emit the same message it already does for existing lists that only contain modules which are not unsupported on that platform: No 'config' modules to run under section 'cloud_config_modules' LP: #1770462
| * Be more safe on string/bytes when writing multipart user-data to disk.Scott Moser2018-06-132-10/+23
| | | | | | | | | | | | | | | | | | | | | | When creating the multipart mime message that is written as user-data.txt.i, cloud-init losing data on conversion to some things as a string. LP: #1768600 Author: Scott Moser <smoser@ubuntu.com> Co-Authored-By: Chad Smith <chad.smith@canonical.com>
| * Fix get_proc_env for pids that have non-utf8 content in environment.Scott Moser2018-06-132-13/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | There is no requirement that the environment of a process contains only utf-8 data. This modifies get_proc_env to support it reading data as binary and decoding if provided with an encoding. The default case is now that we now do: contents.decode('utf-8', 'replace') rather than contents.decode('utf-8', 'strict') LP: #1775371
| * tests: fix salt_minion integration test on bionic and laterChad Smith2018-06-121-2/+12
| | | | | | | | | | | | | | | | In ubuntu, the salt-minion package version 2017.7.4+dfsg1-1 or later automatically moves any seed keys from /etc/salt/pki/minion/ to /var/lib/salt/pki/minion/. Fix integration tests to collect either files in either /etc/salt/pki/minion/ or /var/lib/salt/pki/minion/.
| * tests: provide human-readable integration test summary when --verboseChad Smith2018-06-121-1/+46
| | | | | | | | | | | | | | | | | | | | Integration tests will now provide a brief summary for test failures listed by platform and distribution. The failure summary will only consist of failed test name and assert error message. Drop the verbose dictionary of all integration test output because this content is unreadable given the large number of integration test results listed within this dictionary.
| * tests: skip chrony integration tests on lxd running artful or olderChad Smith2018-06-121-1/+12
| | | | | | | | | | | | A fix for chrony support per LP: #1589780 is not expected in Artful or older series. Skip the chrony suite of tests when running on a container and ubuntu series represented is <= artful as errors are expected.
| * test: add optional --preserve-instance arg to integraiton testsChad Smith2018-06-124-5/+23
| | | | | | | | | | | | | | By default, integration tests destroy the test instances after each test run. To aid debug and development of integration tests, support a --preserve-instance argument which will leave the modified test instance in a stopped state for further debug.
| * netplan: fix mtu if provided by network config for all rendered typesChad Smith2018-06-126-12/+91
| | | | | | | | | | | | | | | | | | | | | | | | When network configuration for any interface defines maximum transmission values (MTU) the netplan, eni and sysconfig renders will take into account any device-level, or subnet-level mtu values. When network configuration has conflicting device-level and ipv4 subnet mtu values, the subnet-specific value is honored and a warning will be logged about any ignored device-level setting. LP: #1774666
| * tests: remove pip install workarounds for pylxd, take upstream fix.Scott Moser2018-06-082-8/+6
| | | | | | | | | | | | | | | | | | | | pylxd upstream provided a fix for the issue we were seeing, so we can take that fix now rather than having our workarounds to order pip installs. The test is that this continues to work: rm -Rf .tox/citest tox -c tox.ini --recreate --notest -e citest
| * subp: support combine_capture argument.Scott Moser2018-06-082-6/+65
| | | | | | | | | | | | | | | | | | | | | | | | This adds 'combine_capture' argument as was present in curtin's subp. It is useful to get interleaved output of a command. I noticed a need for it when looking at user_data_rhevm in DataSourceAltCloud. That will run a subcommand, logging its stdout but swallowing its stderr. Another thing to change to use this would be in udevadm_settle which currently just returns the subp() call. Also, add the docstring copied from curtin's subp.
| * tests: ordered tox dependencies for pylxd installChad Smith2018-06-072-6/+8
| | | | | | | | | | | | | | | | The pylxd project has a setup.py which defines install dependencies. Those sub-dependendencies include pbr and requests which in turn have package version conflicts. Since tox doesn't order dependencies installed, serially install pinned urllib3 at 1.22 which supports both pbr deps and requests deps of pylxd.
| * util: add get_linux_distro function to replace platform.distRobert Schweikert2018-05-293-7/+127
| | | | | | | | | | | | | | | | | | Allow the user to set the distribution with --distro argument to setup.py. Fall back is to read /etc/os-release. Final backup is to use platform.dist() Python function. The platform.dist() function is deprecated and will be removed in Python 3.7 LP: #1745235
| * pyflakes: fix unused variable references identified by pyflakes 2.0.0.Scott Moser2018-05-296-15/+15
| | | | | | | | | | | | A newer version of pyflakes (2.0.0) was released. It identifed some unused variables that version 1.6.0 did not identify. The change here merely fixes those unused variables.
| * - Do not use the systemd_prefix macro, not available in this environmentRobert Schweikert2018-05-291-42/+21
| |
| * doc: Add config info to ec2, openstack and cloudstack datasource docsChad Smith2018-05-244-8/+153
| | | | | | | | Also document instance-data.json on the top-level datasource topic page.
| * Enable SmartOS network metadata to work with netplan via per-subnet routesDan McDonald2018-05-242-5/+67
| | | | | | | | | | | | | | | | | | | | | | - Updated datadict reference URL - Store sdc:routes metadata in DatasourceSmartOS - Map sdc:routes values to per-interface subnet configuration - Added unittest Co-authored-by: Mike Gerdts <mike.gerdts@joyent.com> LP: #1763512
| * openstack: Allow discovery in init-local using dhclient in a sandbox.Chad Smith2018-05-239-124/+416
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Network has not yet been configured in the init-local stage so the openstack datasource will use dhcp-client to temporarily obtain an ipv4 address and query the metadata service at http://169.254.169.254 to get network_data.json configuration. If present, the datasource will return network_config version 1 config based on that network_data.json content. Previously OpenStack datasource only setup dhcp on the fallback interface so this represents a change in behavior to react to the full config provided by openstack. Also significant to OpenStack is the separation of a _crawl_data operation from get_data(). crawl_data walks the available metadata services and returns a dict of discovered content. get_data consumes the crawled_data,  caches it in the datasource and reacts to that data. /run/cloud-init/instance-data.json now published network_data.json or ec2_metadata key if that data is present on any datasource. The main reasons for the separation of crawl from get_data:  * Enable performance metrics of cloud-init's metadata crawls on each  * Enable cloud-init modules and scripts to query and consume metadata    content which may have updated/changed after cloud-init's initial cache    during instance boot. (Think hotplug) Also generalize common logic to base DataSource class/module:  * Move to a common UNSET variable up into base datasource module fix EC2,    ConfigDrive, OpenStack, SmartOS to use the global.  * Drop get_url_settings from Ec2, CloudStack and OpenStack and generalize    DataSource.get_url_params(). Allow subclasses to override url_max_wait,    url_timeout and url_retries params.  * Rename get_network_metadata bool to perform_dhcp_setup as it designates    whether EphemeralDHCPv4 setup is required before crawling metadata. LP: #1749717
| * tests: Avoid using https in httpretty, improve HttPretty test case.Scott Moser2018-05-239-45/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On OpenSuSE 42.3, we would get errors running tests/unittests/test_handler/test_handler_chef.py  - test_myhttps_nonet raises a UnmockedError    No mocking was registered, and real connections are not allowed  - test_myhttps_net raises SSLError    ("bad handshake: SysCallError(32, 'EPIPE')",) This fixes the errors by just using http instead of https. Also it modifies the HttprettyTestCase to do the httpretty activate and deactivate itself in setUp and tearDown. Then we don't have to decorate individual test_ methods. Also, we set    httpretty.HTTPretty.allow_net_connect = False Test cases here should not reach out to a network resource. LP: #1771659
| * yaml_load/schema: Add invalid line and column nums to error messageChad Smith2018-05-234-27/+118
| | | | | | | | | | | | | | | | | | | | Yaml tracebacks are generally hard to read for average users. Add a bit of logic to util.yaml_load and schema validation to look for YAMLError.context_marker or problem_marker line and column counts. No longer log the full exceeption traceback from the yaml_load error, instead just LOG.warning for the specific error and point to the offending line and column where the problem exists.
| * Azure: Ignore NTFS mount errors when checking ephemeral drivePaul Meyer2018-05-233-35/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | The Azure data source provides a method to check whether a NTFS partition on the ephemeral disk is safe for reformatting to ext4. The method checks to see if there are customer data files on the disk. However, mounting the partition fails on systems that do not have the capability of mounting NTFS. Note that in this case, it is also very unlikely that the NTFS partition would have been used by the system (since it can't mount it). The only case would be where an update to the system removed the capability to mount NTFS, the likelihood of which is also very small. This change allows the reformatting of the ephemeral disk to ext4 on systems where mounting NTFS is not supported.
| * packages/brpm: Get proper dependencies for cmdline distro.Scott Moser2018-05-231-3/+3
| | | | | | | | | | | | When invoked with '--distro=suse', the packages that would be attempted for installation would be from redhat. We just were not pasing the args.distro through. That is fixed here.
| * packages: Make rpm spec files patch in package version like in debs.Scott Moser2018-05-232-0/+14
| | | | | | | | | | This makes the necessary changes to patch the full packaged version into the trunk maintained redhat and suse spec files.
| * tools/run-container: replace tools/run-centos with more generic.Scott Moser2018-05-233-312/+626
| | | | | | | | | | | | | | | | | | | | | | | | | | | | tools/run-container is like tools/run-centos, but currently supports the following images from lxc-images opensuse/42.3 centos/6 centos/7 ubuntu/16.04 debian/10 debian/sid Also here is to make installation via zypper in tools/read-dependencies not prompt user.
| * Update version.version_string to contain packaged version.Scott Moser2018-05-224-14/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This modifies version.version_string to support having the package build write the *packaged* version in with a easy replace. Then, when cloud-init reports its version it will include the full packaged version. Also modified here are upstream package build files to get that done. Note part of the trickery in packages/debian/rules.in was to avoid the 'basic' templater consuming the '$variable' variable names. LP: #1770712
| * cc_mounts: Do not add devices to fstab that are already present.Lars Kellogg-Stedman2018-05-223-25/+136
| | | | | | | | | | | | | | Do not add new entries to /etc/fstab for devices that already have an existing fstab entry. Resolves: rhbz#1542578
| * ds-identify: ensure that we have certain tokens in PATH.Scott Moser2018-05-222-1/+33
| | | | | | | | | | | | | | | | | | | | | | SuSE builds were not getting a PATH set in generator's environment. This may seem like mis-configuration on the system, but caused ds-identify to fail to find blkid (or any other program). The change here just ensures that we get /sbin /usr/sbin /bin /usr/bin into the PATH when main is run. LP: #1771382
| * tests: enable Ubuntu Cosmic in integration testsJoshua Powers2018-05-221-0/+16
| |