| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The nova.context should be a super set of oslo.context in all cases,
especially after we've deserialized on the other side of the RPC
bus. Our current code actually has a completely decoupled from_dict,
which means that as fields are added to oslo.context we don't pick up
any of those, and end up with a potentially broken and fragmented
context on the workers.
This fixes that by using the parent constructor for all the fields we
can, and only explicitly load in a few fields that we also need.
It also simplifies the testing so that we're just testing that our
extra fields end up in the context, and not exact matching everything
in the context, as oslo.context may add important things over time.
Change-Id: Ie683adb36d5e2a736ddbf714524c9c18f3c0d69c
|
|
|
|
|
|
|
|
|
|
|
| |
This makes target_cell() do the thing it was originally intended to do,
which is yield a new context that is targeted at the proper cell. With this,
any objects you create inside the target context manager (assuming you use
the yielded context) will remain targeted after the thing returns.
Related to blueprint cells-aware-api
Change-Id: Iccdf6b80f5fc8adcc8a89ce6ece3f37b6cbcaee2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Recently in the gate we have seen a trace on some work-in-progress
patches:
OperationalError: (pymysql.err.OperationalError)
(1040, u'Too many connections')
and at least one operator has reported that the number of database
connections increased significantly going from Mitaka to Newton.
It was suspected that the increase was caused by creating new oslo.db
transaction context managers on-the-fly when switching database
connections for cells. Comparing the dstat --tcp output of runs of the
gate-tempest-dsvm-neutron-full-ubuntu-xenial job with and without
caching of the database connections showed a difference of 445 active
TCP connections and 1495 active TCP connections, respectively [1].
This adds caching of the oslo.db transaction context managers and the
oslo.messaging transports to avoid creating a large number of objects
that are not being garbage-collected as expected.
Closes-Bug: #1691545
[1] https://docs.google.com/spreadsheets/d/1DIfFfX3kaA_SRoCM-aO7BN4IBEShChXLztOBFeKryt4/edit?usp=sharing
Change-Id: I17e0eb836dd87aac5859f506e7d771d42753d31a
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When deleting an instance we look it up in the _get_instance
method and if it's in cell0 then the context is permanently
targeted to that cell via the set_target_cell method.
When we delete the instance in _delete we need to temporarily
untarget the context when we decrement the quota usage otherwise
the quota usage gets decremented in the cell0 database rather than
the cell database. Once the instance is deleted then we
re-apply the target cell on the context.
Change-Id: I7de87dce216835729283bca69f0eff59a679b624
Closes-Bug: #1670627
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously we had aimed to make things like compute RPC automatically
look up the InstanceMapping or HostMapping for the call being performed
to target appropriately. However, we cannot do that within the cell,
and even trying incurs some overhead. For now, just deprecate the
by_instance() and by_host() methods and honor what is in the context
(if set) and otherwise fall back to the default client. Make the context
target routines create and store the RPC transport and remove the caching
logic from the ClientRouter since we're removing its ability to do that.
Related to blueprint cells-aware-api
Change-Id: I10f374adca672576058c4dbab708c040d166df47
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This makes the servers API look up the CellMapping for the instance
in question and target the context to the appropriate cell for the
subsequent call into compute/api.
Note that just as with get() and get_all(), we need to avoid going
direct to the cell database and thus short-circuiting the replication
in the case of cellsv1.
Related to blueprint cells-aware-api
Change-Id: If63a4fa7349890a6e7ac0fa3fe12917dc53334d0
|
|
|
|
|
|
|
|
| |
Nova creates a limited service catalog using just the endpoints
required for Nova. This needs to include the new Cinder v3.
Change-Id: I06f9f2885bb1748897332cb15d4aafe4a99aa9dc
Closes-Bug: #1657496
|
|
|
|
|
|
|
|
|
| |
this function was used to put into db layer to ensure
the request has admin context or a unauthorized exception
will be raised, this is not needed as we already move
all this kind of check into api layer and policy functions.
Change-Id: Ia411e6715e4c584952a275a001f1791ca1352d97
|
|
|
|
|
|
|
|
|
| |
The context.get_admin_context is used in places that it's not necessary,
and that's likely because there's no helper method to retrieve a non
admin context. This adds such a helper method and adds a note to
get_admin_context warning that it's not usually the right choice.
Change-Id: I2e6a2efa4bcdf3f8688897972a6cf8a5af3f90d6
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The base oslo.context defines to_policy_values with all the information
that it expects a service to require to enforce policy. Use that instead
of throwing everything in to_dict at policy enforcement.
Change-Id: I0a42b4425e9dd1bd062c48792c4d116dd370afe3
Closes-Bug: #1602081
|
|/
|
|
|
|
|
|
| |
context.can is used to verify that the provided action is valid.
This patch changes the argument name from 'rule' to 'action' to
avoid misunderstanding.
Change-Id: I315df6d1d29966d6554b48335a99bd9f826a8c6b
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This moves the cell message queue switching code from the
RequestContext to the RPC layer where it's ultimately used.
Originally, it was thought that a separate DB query for
a CellMapping would occur per compute RPC API call in the
API cell and the context manager would be invoked to inject
the cell message queue transport to use for the RPC call.
Since compute RPC calls are based on the CellMapping of an
instance or a host, we could instead have generic functions
that take an instance or host and look up InstanceMapping or
HostMapping to get the CellMapping and return the
corresponding RPC client.
The RPC client objects are cached by CellMapping uuid and
expired clients are removed using a periodic task.
Co-Authored-By: Brian Elliott <bdelliott@gmail.com>
Depends-On: I6f211e9102f79418f9f94a15784f91c4150ab8a7
Change-Id: I96849888087f4b09433cb683a9eb4719d1c35c4c
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds functionality to the scheduler "report client" to ensure
that the client calls the placement API to create a resource provider
record for the local compute host managed by the Nova resource tracker.
The report client keeps a cache of resource provider objects, keyed by
resource provider UUID and constructed from the results of placement
REST API calls to get information about a resource provider. If a
resource provider matching a UUID was not found in the placement REST
API, the report client automatically creates the resource provider
record via the placement REST API. These resource provider objects will
be used in followup patches that add creation of inventory and
allocation records to the scheduler report client.
Included in this patch is a new [placement] nova.conf configuration
section with a single os_region_name configuration option that allows
Nova to grab the placement API endpoint URL for the particular OpenStack
region that it is in. We do not support endpoint URL overrides for the
placement API service. We only use the Keystone service catalog for
finding the endpoint for the placement service. We intentionally modeled
the determination of the placement endpoint URL after similar code that
determines the volume endpoint URL in /nova/volume/cinder.py.
This redoes the placement API using keystone session, and stubs out
where we can do more reasonable handling of errors. This works if we
fill out the right credentials in the placement section of the config
file.
Co-Authored-By: Sean Dague <sean@dague.net>
Change-Id: I9d28b51da25c523d22c373039e6d8b36fd96eba6
blueprint: generic-resource-pools
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The user_id and project_id are being provided to the base oslo.context
class which sets them on the user and tenant attributes. In future
oslo.context will provide the user_id and project_id parameters itself.
By force setting user_id and project_id on the context we override the
user and project params that were set in the base class with None
values.
For now make the user_id and project_id parameters renames of the base
class properties and then when oslo.context catches up we can simply
remove these properties. Having these as properties is consistent with
most other OpenStack projects.
Change-Id: I9cd3c1b1121011156e78388b6beaac0253324e22
|
|
|
|
|
|
|
|
|
| |
The 2.6.0 release of oslo.context adds a new attribute
'is_admin_project'. Always include that attribute when serializing the
object, and expect it in our tests.
Change-Id: I563cf810385e84ab30d49ef079b75df279006f0d
Related-Bug: 1602081
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The from_environ method is designed to pick up all the parameters set
from auth_token middleware and other oslo middlewares and create a
context with them.
By doing this there will be information available to libraries like
oslo.policy and oslo.logging without nova having to track each change to
the base library.
There is ongoing work here to move more values to the base class that
will be cleaned up in future.
Change-Id: I6b61028fcecb86cc6c25fb69977774e266a8ea5b
Related-Bug: #1602081
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The unknown keyword arguments that were passed to
RequestContext.__init__ were being dropped so that unknown parameters
coming back from from_dict didn't cause errors. This means however that
newer properties like RequestContext.from_environ might be passing
values to this context and nova's context drop them.
To work around this instead of dropping values at __init__ parse the
incoming values in from_dict so that only known arguments are passed
into the constructor. We can then assume that all values that context
doesn't handle are intended for the base class.
Change-Id: I393c51e5319773420f944bb85fd215270565fcbb
Related-Bug: #1602081
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adds 'fatal' argument (with default value True) to context.
If the argument is False, if a nova.exception.Forbidden
is raised, False is returned instead. This is the equivalent of
soft_authorize.
Adds docstring to nova.context.RequestContext.can.
Adds unit tests for nova.context.RequestContext.can.
Partially-Implements: bp policy-in-code
Change-Id: Ib6388822da3d3eb9b8fbd9b5c341cd8efeb790fe
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds the basic framework for registering and using default policy
rules. Rules should be defined and returned from a module in
nova/policies/, and then added to the list in nova/policies/__init__.py.
A new context.can() method has been added for policy enforcement of
registered rules. It has the same parameters as the enforce() method
currently being used.
To establish the full pattern for usage the policy checks in the servers
API module have been registered and converted to the new usage.
Now that some policy checks are registered they're being used properly
by tests. Some tests have been updated so that the instance project_id
matches the context project_id in order to pass the 'admin_or_owner'
check.
Change-Id: I71b3d1233255125cb280a000b990329f5b03fdfd
Partially-Implements: bp policy-in-code
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds message queue connection information to the RequestContext
which can be used by nova-api to communicate with a targeted cell
message queue with each query.
A function 'get_cell_client' can be called in rpc functions to enable them
to use message queue transport information from a RequestContext. The
function creates a rpc client object dynamically if message queue
connection information is found in the RequestContext and falls back on
the default rpc client.
Example usage:
def get_cell_client(self, context):
return rpc.get_cell_client(context, self.client)
def build_and_run_instances(self, ctxt, instance, host, image, ...)
cctxt = self.get_cell_client(ctxt).prepare(...)
cctxt.cast(...)
Implements blueprint cells-mq-connection-switching
Change-Id: Idef670d5b73c9cef8501a0593eccd785b708bd2b
|
|
|
|
|
|
|
|
|
| |
In Ia575ba803a0fb70f39146bd75d381ed19414bd23, oslo.context (2.2.0)
added roles support in the context itself. This passing roles to
super()__init__.
Depends-On: I74168f8aa7d4875b160e44ff53b9abd6083d1fe4
Change-Id: Idcf2acd020634881f0fd91d666fd0d0de56042de
|
|
|
|
|
|
|
|
|
| |
Skip creating the formatted log message
if the message is not going to be emitted
because of the log level.
TrivialFix
Change-Id: Iba9f47163a0ac3aca612818272db6d536b238975
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This augments the RequestContext with connection information for a
target database to query. This will allow nova-api to issue queries to
a targeted cell database with each query.
Example usages:
ctxt = context.RequestContext()
cell_mapping = CellMapping()
with context.target_cell(ctxt, cell_mapping):
instance = objects.Instance.get_by_uuid(ctxt, uuid)
with context.target_cell(ctxt, cell_mapping):
with instance.obj_alternate_context(ctxt):
instance.save()
Implements blueprint cells-db-connection-switching
Co-Authored-By: melanie witt<melwitt@yahoo-inc.com>
Change-Id: I9e2363a35b58ae55bd40194c1f8bfb89b599bf04
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
keystoneauth was split out last cycle as a library specifically to deal
with doing auth functions so that people who do not need to do keystone
CRUD operations can just consume only the auth session parts. As part
of modernizing keystone interactions, use keystoneauth instead of
keystoneclient.
A change to tests was made to stop checking how often the keystone auth
session is called. This could be broken in the future depending on how
the keystoneauth1 loader works. It is incorrect to mock out and check
how often Session is called when nova has no direct control over this.
The number of times barbican client is called is correctly in the
control of Nova and will continue to be tested.
bp: keystoneclient-to-keystoneauth
Co-Authored-By: Morgan Fainberg <morgan.fainberg@gmail.com>
Depends-On: I1f754a9a949ef92f4e427a91bbd1b1e73e86c8c4
Change-Id: I09a5da761bdc02c83b087f3cec40b7fa022a7a63
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instance of RequestContext contains many objects and some of them like
mutexes could not be copied. Also a deepcopy of the entire
RequestContext wastes CPU time.
To avoid problems with deepcopy and avoid performance overhead this
patch changes deepcopy of RequestContext to shallow copy and makes
deepcopy of only the 'roles' member because of security issue
LP #1386932.
Closes-Bug: #1506958
Related-Bug: #1386932
Change-Id: I1e2c00e95e1c4bcd0ec7bf075458789d6fb06e99
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Since oslo 1.6.0 timeutils.isotime() and timeutils.strtime() methods
are deprecated.
DeprecationWarning: Using function/method
'oslo_utils.timeutils.strtime()' is deprecated in version '1.6' and
will be removed in a future version: use either
datetime.datetime.isoformat() or datetime.datetime.strftime() instead
Change-Id: If69bd8a6bee052556ba8853afef3941bcd1e7b13
Closes-Bug: 1479056
Co-Authored-By: Diana Clarke <diana.joan.clarke@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
'get_session' and 'get_api_session' methods are still
needed for backward compatibility.
Implements: blueprint new-oslodb-enginefacade
Closes-Bug: #1502104
Co-Authored-By: Pavel Kholkin <pkholkin@mirantis.com>
Change-Id: I8ceb9f939470f744f0d616d7db065a4d5d80202b
|
|
|
|
|
|
|
|
|
| |
Nova uses overwrite in constructor of RequestContext to
update thread's local context or not, we don't need it
in method elevated, due to we never replace thread's local
context when get a copy with current context.
Change-Id: Ia85dfc4ddf4d4d31a7367c1ff90da118f503ba09
|
|
|
|
|
|
|
| |
The docstring for the require_admin_context method is stating the
opposite of what the method actually does.
Change-Id: Ifcb0af582acaa553feb8ab52bc4dd5529affcaaf
|
|
|
|
|
|
|
|
|
|
| |
Pass through the rest of the arguments for the oslo_context's
RequestContext class if they are in our constructor's kwargs.
This will prevent us from logging them as arguments that we
drop explicitly.
Closes-Bug: #1431549
Change-Id: Ie804a6117accc0e35ef5cfd31360a79636630e21
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Conversion of nova.context.RequestContext to and from a
dictionary was not working properly. There is duplicate
information as well in the serialized dict. As a first
step i've removed request_id as it dups the one in
oslo_context.RequestContext. However changing project_id
and user_id to a @property is challenging at this late
juncture. So leaving them as-is with a FIXME. The other
issue related to the roundtrip is the stack traces that
show up in screen-n-cpu log as mentioned in the bug
referenced below. To avoid that added some defensive
code ('has_attr') for now and a FIXME.
Partial-Bug: #1427665
Change-Id: Ia47d4909d2656d6fc4c1179659b8098bba3235d3
|
|
|
|
|
|
|
|
|
| |
in the cases where context manages to make it's way into a log message
we have no sane representation of it. The to_dict method provides the
relevant information, so we can use a dump of that as our human
friendly version.
Change-Id: I394260c6957ba5c4f2b134e75cf8e7bd1d1b59c2
|
|
|
|
|
|
|
|
|
|
|
|
| |
oslo.log uses oslo.context to get the context information.
nove.context does not use the RequestContext from oslo_context,
so when oslo.log tries to find the current context using
oslo_context.get_current() it fails. We should get nova's
RequestContext to inherit from oslo.context's RequestContext
and add test cases to makes sure this works properly.
Closes-Bug: #1427209
Change-Id: I7a02e646b7cebdb0eb72be2d5ad7bfe0743c8f4d
|
|
|
|
|
|
|
|
|
|
| |
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
|
|
|
|
|
|
|
|
| |
openstack.common.context is currently moved out of the incubator
to a standalone project, hence replace it with oslo_context in nova.
Change-Id: I6cbf5d719f33072a5a871c250760512a439a7daf
Closes-Bug: #1210261
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
|
|
|
|
|
|
|
|
|
|
|
| |
Adds a barbican keymgr wrapper to the key manager interface in
nova. This allows barbican to be configured as the key manager
for encryption keys in nova. The wrapper translates calls from
the existing key manager interface to python-barbicanclient.
Change-Id: I110c7ceada48de28cee1169b643b12407f21b36c
Implements: blueprint encryption-with-barbican
DocImpact
|
|
|
|
|
|
|
|
|
| |
context.elevated was creating a copy of the current context then adding
'admin' to the roles of that context. This should be a deepcopy, otherwise
'admin' is added to the original context too.
Change-Id: I8ab00c88a8e76a14fb9f4ae96dfdb5f018fc2d0f
Closes-bug: 1386932
|
|\ |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Added a hacking rule for the warning about checking
translation for it and checking logging level `warning` instead
alias `warn`.
Change-Id: I2bced49dc5a0408a94d5d20d85b20c682886edbe
|
|/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the common session loading parameters and the session object for
talking to cinder.
There are some related changes in this patch.
Firstly auth_token middleware now provides an authentication plugin that
can be used along with the session object to make requests under the
user's authentication. This will largely replace the information
required on the context object.
This authentication plugin is not serializable though and so it cannot
be transferred over RPC so we introduce a simple authentication plugin
that reconstructs the required information from the context.
When talking to cinder we now create a global session object (think of
this like keeping open a connection pool object) and use the
authentication plugin to send requests to cinder.
I also condense the cinder tests as they are largely copied and pasted
between v1 and v2 and this solves fixing them in two places.
DocImpact: Renames cinder's timeout, insecure and CA certificates
parameters to the parameters used by the common session object. Adds
options for using client certificates with connection.
Change-Id: I7afe604503b8597c16be61d2a66a10b94269a219
|
|
|
|
|
|
|
|
|
|
| |
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some modules already moved to oslo.utils
will still be around since other code in nova/openstack/common/
are using it and will be removed in a subsequent commit.
Change-Id: Idc716342535fdfa680963e0e073ddb46f5f1eb34
|
|
|
|
|
|
| |
It's part of oslo context module, so let's use it instead.
Change-Id: I5227a99abc711c468a7178ee19332657004f4661
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use v2 volume endpoint to attach/detach volume would be failed,
due to nova not supporting cinder client v2 yet. This patch is for
nova support cinder client v2.
Implements bp support-cinderclient-v2
Closes-Bug: #1215772
Co-Authored-By: Mike Perez <thingee@gmail.com>
Co-Authored-By: Yaguang Tang <yaguang.tang@canonical.com>
Change-Id: Id8abbbb4d9b0c8c49ab51fc3d958ef0d487467f8
|
|
|
|
|
|
|
|
| |
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This reverts commit e25316ecb9c879cd1fe2de0c8a4c4d5a424324ab.
The quota_classes API was used to set default quota values
so it shouldn't have been removed, so reverting a series
of changes that removed the API and it's internal code.
Related mailing list thread on the topic:
http://lists.openstack.org/pipermail/openstack-dev/2014-May/035383.html
Partial-Bug: #1299517
Change-Id: If903863d28fbab74cfe571709a01cd2cdd46d174
|
|
|
|
|
|
|
|
|
|
|
| |
Continuation of quota-class removal started in
I1110022d6f628d03aaf363da707f2d2ef1600437.
Since we don't have an API to set quota-class any more, we don't need to
store it inside of the context. Also begin removing quota-class logic from
nova.quota to make test_quota pass with changes to context
Change-Id: I85dd24121a434b95dc6f0af180444d4258b50765
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The NotAuthorized NovaException has an internal code of 403 which is
actually Forbidden, so rename it appropriately.
This patch doesn't change the external behavior, the status code in
responses will still be 403 but the exception is just named properly.
This is also necessary to create an actual Unauthorized NovaException
with code 401 for use in some Neutron API bug fixes for more granular
error handling from python-neutronclient.
Related-Bug: #1298075
Change-Id: I691fac2e2c797f47c04da7965d7b1c8685c74edb
|