The stats module is used to decide if the InstancePCIRequests of a boot
request can fit to a given compute host and to decide which PCI device
pool can fulfill the requests. It is used both during scheduling and also
during the PCI claim code.

PCI devices now modelled in placement and the allocation_candidate query
now requests PCI resources therefore each allocation candidate returned
from placement already restricts which PCI devices can be used during
the PciPassthroughFilter, the NumaTopologyFilter, and PCI claim code
paths. This patch adapts the stats module to consider the PCI
allocation candidate or the already made placement PCI allocation when
filtering the PCI device pools.

blueprint: pci-device-tracking-in-placement
Change-Id: If363981c4aeeb09a96ee94b140070d3d0f6af48f

This patch extends the HostState object with an allocation_candidates
list populated by the scheduler manager. Also this changes the generic
scheduler logic to allocate the candidate of the selected host based on
the candidates in the host state.

So after this patch scheduler filters can be extended to filter the
allocation_candidates list of the HostState object while processing a
host and restrict which candidate can be allocated if the host passes
the all the filters. Potentially all candidates can be removed by
multiple consecutive filters making the host as a non viable scheduling
target.

blueprint: pci-device-tracking-in-placement
Change-Id: Id0afff271d345a94aa83fc886e9c3231c3ff2570

Change-Id: I64810898cd9126cf619df0b8f60e6fa01958943e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.

This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.

This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.

The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.

Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d

Follow-up on change I64dc67e2bacd7a6c86153db5ae983dfb54bd40eb by
removing additional code paths that are no longer relevant following the
removal of the 'USES_ALLOCATION_CANDIDATES' option. This is kept
separate from the aforementioned change to help keep both changes
readable.

Change-Id: I1d2b51f5dd2ca75eb565ca5242cfdb938868bff9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>

For the upcoming `socket` affinity, PCI stats needs to know the
host's NUMA topology in order to calculate the socket affinity of PCI
devices (based on their 'numa_node'). In this patch, the PCI manager
starts using its compute_node parameter to give a NUMA topology to PCI
stats.

PCI stats needs to track the NUMA topology at the object (not the
class) level, so all the '_filter_*()' classmethods that, in a
subsequent patch, will either use 'self.numa_topology' directly or
call a method that does, are moved to the object level. They were only
ever called via self anyways.

Implements: blueprint pci-socket-affinity
Change-Id: I1d270cc3e88a74097eefe3b887106222ae06fe1c

Replace the following items with Python 3 style code.

- six.iteritems
- six.itervalues
- six.iterkeys

Subsequent patches will replace other six usages.

Change-Id: Id55de3a105bedcf61bcfc797aabe86d6e75709c8
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>

get_compute_nodes_by_host_or_node uses the scatter_gather_cells
function but was not handling the case that a failure result
was returned, which could be the called function raising some
exception or the cell timing out. This causes issues when the
caller of get_compute_nodes_by_host_or_node expects to get a
ComputeNodeList back and can do something like len(nodes) on it
which fails when the result is not iterable.

To be clear, if a cell is down there are going to be problems
which likely result in a NoValidHost error during scheduling, but
this avoids an ugly TypeError traceback in the scheduler logs.

Change-Id: Ia54b5adf0a125ae1f9b86887a07dd1d79821dd54
Closes-Bug: #1857139

As part of the cross-cell resize work, we need to be able
to tell the scheduler that it's OK to select destinations
from cells in which the instance does not currently live.
This happens via the "allow_cross_cell_move" flag in the
RequestSpec.requested_destination field, which is not set
at this time (this is just plumbing which will be used later)
and defaults to False for backward compatibility.

Part of blueprint cross-cell-resize

Change-Id: Ib29a17f7fcc3280871d5c9ba31e1374eff98cf82