Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:

nova/tests/functional/regressions/test_bug_1781286.py
  We need to avoid using 'fixtures.MockPatch' since fixtures is using
  'mock' (the library) under the hood and a call to 'mock.patch.stop'
  found in that test will now "stop" mocks from the wrong library. We
  have discussed making this configurable but the option proposed isn't
  that pretty [1] so this is better.

The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:

  import glob

  for path in glob.glob('nova/tests/**/*.py', recursive=True):
      with open(path) as fh:
          lines = fh.readlines()
      if 'import mock\n' not in lines:
          continue
      import_group_found = False
      create_first_party_group = False
      for num, line in enumerate(lines):
          line = line.strip()
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              for lib in (
                  'ddt', 'six', 'webob', 'fixtures', 'testtools'
                  'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
              ):
                  if lib in tokens[1]:
                      create_first_party_group = True
                      break
              if create_first_party_group:
                  break
              import_group_found = True
          if not import_group_found:
              continue
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              if tokens[1] > 'unittest':
                  break
              elif tokens[1] == 'unittest' and (
                  len(tokens) == 2 or tokens[4] > 'mock'
              ):
                  break
          elif not line:
              break
      if create_first_party_group:
          lines.insert(num, 'from unittest import mock\n\n')
      else:
          lines.insert(num, 'from unittest import mock\n')
      del lines[lines.index('import mock\n')]
      with open(path, 'w+') as fh:
          fh.writelines(lines)

Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.

[1] https://github.com/testing-cabal/fixtures/pull/49

Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This change fixes some of the typos in unit tests as well
as in nova code-base.

Change-Id: I209bbb270baf889fcb2b9a4d1ce0ab4a962d0d0e

The InstanceList.get_by_filters query is joining on
info_cache, security_groups, metadata and system_metadata
because of how instance_get_all_by_filters_sort in the DB API
works if columns_to_join (expected_attrs) is None. The get_hosts
method only cares about the instance.host value of its members
so those joins are unnecessarily expensive.

This change simply passes expected_attrs=[] to get_by_filters
to avoid the joins. A follow up change can further optimize this
code by adding a new query method to just get the host values
for a list of instance uuids, so a TODO is left in place for that.
Note that a new query method would need to be remotable and thus
not something we can backport.

Change-Id: I53d4b38d12404a1641f667c537404effa837a83d
Partial-Bug: #1830234

oslo.utils release 3.37.0 [1] introduced uuidsentinel [2]. This change
rips out nova's uuidsentinel and replaces it with the one from
oslo.utils.

[1] https://review.openstack.org/#/c/599754/
[2] https://review.openstack.org/#/c/594179/

Change-Id: I7f5f08691ca3f73073c66c29dddb996fb2c2b266
Depends-On: https://review.openstack.org/600041

To check affinity and anti-affinity policies for scheduling instances,
we use the RequestSpec.instance_group.hosts field to check the hosts
that have group members on them. Access of the 'hosts' field calls
InstanceGroup.get_hosts during a lazy-load and get_hosts does a query
for all instances that are members of the group and returns their hosts
after removing duplicates. The InstanceList query isn't targeting any
cells, so it will return [] in a multi-cell environment in both the
instance create case and the instance move case. In the move case, we
do have a cell-targeted RequestContext when setup_instance_group is
called *but* the RequestSpec.instance_group object is queried early in
compute/api before we're targeted to a cell, so a call of
RequestSpec.instance_group.get_hosts() will result in [] still, even
for move operations.

This makes setup_instance_group query all cells for instances that are
members of the instance group if the RequestContext is untargeted, else
it queries the targeted cell for the instances.

Closes-Bug: #1746863

Change-Id: Ia5f5a0d75953b1154a8de3e1eaa15f8042e32d77

We use obj_to_primitive() to generate the primitive obj, the
origin obj value is primitive['nova_object.data'], so, we need
transfer primitive to obj data before obj_make_compatible
called.

This patch fixed all invalid obj_make_compatible test cases.

Closes-Bug: #1776373

Change-Id: I048b22b2447b671228a7ba067e06c59b59e81ce5

The change items in this patch as below:
1. Add policy and rules to InstanceGroup, deprecate
the policies field.
2. Also, make _from_db_object and _create_in_db method to
support new "policy" field.

The internal usage of InstanceGroup.policies is converted to
use the new model in the REST API and RequestSpec compat code.

Related to blueprint complex-anti-affinity-policies

Change-Id: Ib33719a4b9599d86848c618a6e142c71ece79ca5

A group can only have 1 policy associated it, but now we have
ONE to MANY model in group model, this patch change it to ONE
to ONE Model.

We also adapt create method to new model, and remove policy
update support in group.save because now we don't support to
update a policy of the group.

Related to blueprint complex-anti-affinity-policies

Change-Id: I95eb1c8470a567408696132ae656305be53dc5bb

The /345_require_online_migration_completion cell database
migration added in Ocata prevents anyone from upgrading
the cell database schema until they have performed the online
instance group data migrations via the
"nova-manage db online_data_migrations" command.

This means we can remove the compatibility code in the InstanceGroup
object which performs a lookup routine in the API database and
if not found it falls back to the cell database, which at this
point should be empty, therefore making that dead code now.

The test_get_by_name and test_get_by_hint tests are moved
from _TestInstanceGroupListObject to _TestInstanceGroupObject
since they don't have anything to do with InstanceGroupList.

A follow up change will remove the now unused main DB API
methods for instance groups.

Change-Id: Ifbd53b13fa0fef62e0329283b73d587f367e46c2

The servergroup.addmember notification has been transformed
to the versioned notification framework.

Change-Id: I5291e453ba06f0f26c4928e4f7482c40e725a781
Implements: bp versioned-notification-transformation-rocky