Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:

nova/tests/functional/regressions/test_bug_1781286.py
  We need to avoid using 'fixtures.MockPatch' since fixtures is using
  'mock' (the library) under the hood and a call to 'mock.patch.stop'
  found in that test will now "stop" mocks from the wrong library. We
  have discussed making this configurable but the option proposed isn't
  that pretty [1] so this is better.

The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:

  import glob

  for path in glob.glob('nova/tests/**/*.py', recursive=True):
      with open(path) as fh:
          lines = fh.readlines()
      if 'import mock\n' not in lines:
          continue
      import_group_found = False
      create_first_party_group = False
      for num, line in enumerate(lines):
          line = line.strip()
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              for lib in (
                  'ddt', 'six', 'webob', 'fixtures', 'testtools'
                  'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
              ):
                  if lib in tokens[1]:
                      create_first_party_group = True
                      break
              if create_first_party_group:
                  break
              import_group_found = True
          if not import_group_found:
              continue
          if line.startswith('import ') or line.startswith('from '):
              tokens = line.split()
              if tokens[1] > 'unittest':
                  break
              elif tokens[1] == 'unittest' and (
                  len(tokens) == 2 or tokens[4] > 'mock'
              ):
                  break
          elif not line:
              break
      if create_first_party_group:
          lines.insert(num, 'from unittest import mock\n\n')
      else:
          lines.insert(num, 'from unittest import mock\n')
      del lines[lines.index('import mock\n')]
      with open(path, 'w+') as fh:
          fh.writelines(lines)

Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.

[1] https://github.com/testing-cabal/fixtures/pull/49

Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

This counts instance mappings for the purpopse of counting quota
usage for instances. By counting instances via instance mappings, the
count is resilient to down cells in a multi-cell environment.

Part of blueprint count-quota-usage-from-placement

Change-Id: I7c1ae466a6a6b6d82548396e0c94b1cdab600b5c

This adds an online data migration for the user_id field on
InstanceMapping. It does this by processing instance mappings that do
not have a value set for the field (i.e. are NULL in the database) and
queries instances in each cell that need to be updated.

Part of blueprint count-quota-usage-from-placement

Change-Id: I8cc873ba63db7b806ab1de0a88fe8a87d4baeea9

This adds the new user_id column from the instance_mappings table as
a field in the InstanceMapping object. There is already a project_id
field containing the project_id for the instance. The user_id field
will contain the corresponding user_id for the instance.

Part of blueprint count-quota-usage-from-placement

Change-Id: I0f523b2a2e09e1ece9e1911325e55cffd183a9d5

The data migration was needlessly querying the cell database
for instances even if there were no instance mappings in that
database that needed to be migrated. This simply continues to
the next cell if the instance mappings in the current cell are
migrated.

While we're in here, the joinedload on 'cell_mapping' can be
removed since it's not used.

Closes-Bug: #1817961

Change-Id: Idf35ed9d57945bc80fbd47393b7de076330160e6

1) Change it to accommodate querying it for 'None' project_ids
   in the "--all-tenats" case.
2) If the online data migration for populating queued_for_delete
   has not been run for some reason, the the values could be NULL
   in the database for instance_mapping.queued_for_delete. Under
   such circumstances, we assume that such mappings with NULL
   queued_for_delete have *not* being queued_for_deletion.

Related to blueprint handling-down-cell

Change-Id: I80a65bc026e26a272a9dc041b27f9839511db765

InstanceMappingList.get_by_cell_and_project() is being changed to
get_not_deleted_by_cell_and_project() by which we change the method to
filter out not queued_for_delete instance_mappings and also for
the method to accept cell_uuid as the input instead of the cell_id.
This also adds an optional limit parameter. This is more useful than
the ealier approach since it would increase the performance slightly
by not having to query for the cell_mappings explicitly to get the
cell ids from the cell uuids obtained from scatter gather utility.

Note that this method was added relatively recently and hence is not
used anywhere which is why it should be fine to change this, keeping
in mind the intention of having added it in the first place.

Change-Id: I5df9851cce0b6e635c74fd34f5be9c4906dd7406
Related to blueprint handling-down-cell

Adds a new query method to query for instance_mappings in the provided
cell for the provided project_id.

Related to blueprint handling-down-cell

Change-Id: I4b4466de9160267c23b88c2b35673745fd989f4d

oslo.utils release 3.37.0 [1] introduced uuidsentinel [2]. This change
rips out nova's uuidsentinel and replaces it with the one from
oslo.utils.

[1] https://review.openstack.org/#/c/599754/
[2] https://review.openstack.org/#/c/594179/

Change-Id: I7f5f08691ca3f73073c66c29dddb996fb2c2b266
Depends-On: https://review.openstack.org/600041

This adds an online data migration for the queued_for_delete flag
on InstanceMapping. It does this by processing instance mappings that
do not have a value set for the field (i.e. are set to NULL) and queries
instances in each cell that need to be updated.

Note that this has what looks like a scary schema change in it. When the
column was added, it was specified as default=False and this changes that
value. This is not actually a schema change, but rather just changes what
we are asking SQLAlchemy to do for us when we do an INSERT of that object
if/when a value for that field is not specified. If we had used the
'server_default' flag, that would have been encoded into the schema and
thus would require a migration to change. This patch removes it because
we need to actually be able to see those NULL values in the database in
order to efficiently use them to query for records needing to be migrated.

Related to blueprint handling-down-cell

Change-Id: I0bdadec6f6b9dd7382247213a08fa139329a0683