From 256160b849e49d819f6abd492a99989dec285ae7 Mon Sep 17 00:00:00 2001 From: Ghanshyam Mann Date: Wed, 25 Nov 2020 14:04:22 -0600 Subject: [goal] Deprecate the JSON formatted policy file As per the community goal of migrating the policy file the format from JSON to YAML[1], we need to do two things: 1. Change the default value of '[oslo_policy] policy_file'' config option from 'policy.json' to 'policy.yaml' with upgrade checks. 2. Deprecate the JSON formatted policy file on the project side via warning in doc and releasenotes. Also replace policy.json to policy.yaml ref from doc and tests. [1]https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html Change-Id: Ic65d2fd6ce7215b4a47a6fb41b9cbf991f27773b --- .../admin/cli-manage-projects-users-and-roles.rst | 4 ++-- doc/source/admin/identity-concepts.rst | 18 +++++++++--------- doc/source/admin/upgrading.rst | 2 +- doc/source/configuration/policy.rst | 9 +++++++++ doc/source/getting-started/policy_mapping.rst | 2 +- keystone/cmd/status.py | 3 +++ keystone/common/rbac_enforcer/enforcer.py | 8 ++++++++ keystone/conf/__init__.py | 7 +++++++ keystone/tests/unit/test_policy.py | 2 +- keystone/tests/unit/test_v3.py | 2 +- lower-constraints.txt | 6 +++--- ...-json-formatted-policy-file-95f6307f88358f58.yaml | 20 ++++++++++++++++++++ requirements.txt | 6 +++--- 13 files changed, 68 insertions(+), 21 deletions(-) create mode 100644 releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml diff --git a/doc/source/admin/cli-manage-projects-users-and-roles.rst b/doc/source/admin/cli-manage-projects-users-and-roles.rst index f27979d2c..8d2f83702 100644 --- a/doc/source/admin/cli-manage-projects-users-and-roles.rst +++ b/doc/source/admin/cli-manage-projects-users-and-roles.rst @@ -10,8 +10,8 @@ define which actions users can perform. You assign roles to user-project pairs. You can define actions for OpenStack service roles in the -``/etc/PROJECT/policy.json`` files. For example, define actions for -Compute service roles in the ``/etc/nova/policy.json`` file. +``/etc/PROJECT/policy.yaml`` files. For example, define actions for +Compute service roles in the ``/etc/nova/policy.yaml`` file. You can manage projects, users, and roles independently from each other. diff --git a/doc/source/admin/identity-concepts.rst b/doc/source/admin/identity-concepts.rst index 3d615c0da..0f8cfc51f 100644 --- a/doc/source/admin/identity-concepts.rst +++ b/doc/source/admin/identity-concepts.rst @@ -122,9 +122,9 @@ Identity user management examples: Individual services assign meaning to roles, typically through limiting or granting access to users with the role to the operations that the service supports. Role access is typically - configured in the service's ``policy.json`` file. For example, + configured in the service's ``policy.yaml`` file. For example, to limit Compute access to the ``compute-user`` role, edit the - Compute service's ``policy.json`` file to require this role for + Compute service's ``policy.yaml`` file to require this role for Compute operations. The Identity service assigns a project and a role to a user. You might @@ -139,25 +139,25 @@ A user can have different roles in different projects. For example, Alice might also have the ``admin`` role in the ``Cyberdyne`` project. A user can also have multiple roles in the same project. -The ``/etc/[SERVICE_CODENAME]/policy.json`` file controls the +The ``/etc/[SERVICE_CODENAME]/policy.yaml`` file controls the tasks that users can perform for a given service. For example, the -``/etc/nova/policy.json`` file specifies the access policy for the -Compute service, the ``/etc/glance/policy.json`` file specifies +``/etc/nova/policy.yaml`` file specifies the access policy for the +Compute service, the ``/etc/glance/policy.yaml`` file specifies the access policy for the Image service, and the -``/etc/keystone/policy.json`` file specifies the access policy for +``/etc/keystone/policy.yaml`` file specifies the access policy for the Identity service. -The default ``policy.json`` files in the Compute, Identity, and +The default ``policy.yaml`` files in the Compute, Identity, and Image services recognize only the ``admin`` role. Any user with any role in a project can access all operations that do not require the ``admin`` role. To restrict users from performing operations in, for example, the Compute service, you must create a role in the Identity service and -then modify the ``/etc/nova/policy.json`` file so that this role +then modify the ``/etc/nova/policy.yaml`` file so that this role is required for Compute operations. -For example, the following line in the ``/etc/cinder/policy.json`` +For example, the following line in the ``/etc/cinder/policy.yaml`` file does not restrict which users can create volumes: .. code-block:: none diff --git a/doc/source/admin/upgrading.rst b/doc/source/admin/upgrading.rst index 687dba4e3..709d98dac 100644 --- a/doc/source/admin/upgrading.rst +++ b/doc/source/admin/upgrading.rst @@ -42,7 +42,7 @@ Plan your upgrade: to re-read the release notes for the previous release (or two!). * Prepare your new configuration files, including ``keystone.conf``, - ``logging.conf``, ``policy.json``, ``keystone-paste.ini``, and anything else + ``logging.conf``, ``policy.yaml``, ``keystone-paste.ini``, and anything else in ``/etc/keystone/``, by customizing the corresponding files from the next release. diff --git a/doc/source/configuration/policy.rst b/doc/source/configuration/policy.rst index daafdea51..3f80c5ee2 100644 --- a/doc/source/configuration/policy.rst +++ b/doc/source/configuration/policy.rst @@ -2,6 +2,15 @@ Policy configuration ==================== +.. warning:: + + JSON formatted policy file is deprecated since Keystone 19.0.0 (Wallaby). + This `oslopolicy-convert-json-to-yaml`__ tool will migrate your existing + JSON-formatted policy file to YAML in a backward-compatible way. + +.. __: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html + + Configuration ~~~~~~~~~~~~~ diff --git a/doc/source/getting-started/policy_mapping.rst b/doc/source/getting-started/policy_mapping.rst index 2975b45b3..a7cb27cfa 100644 --- a/doc/source/getting-started/policy_mapping.rst +++ b/doc/source/getting-started/policy_mapping.rst @@ -2,7 +2,7 @@ Mapping of policy target to API =============================== -The following table shows the target in the policy.json file for each API. +The following table shows the target in the policy.yaml file for each API. ========================================================= === Target API diff --git a/keystone/cmd/status.py b/keystone/cmd/status.py index 3585c2e2b..64b2e62f3 100644 --- a/keystone/cmd/status.py +++ b/keystone/cmd/status.py @@ -12,6 +12,7 @@ from oslo_policy import _checks from oslo_policy import policy +from oslo_upgradecheck import common_checks from oslo_upgradecheck import upgradecheck from keystone.common import driver_hints @@ -86,6 +87,8 @@ class Checks(upgradecheck.UpgradeCommands): check_trust_policies_are_not_empty), ("Check default roles are immutable", check_default_roles_are_immutable), + ("Policy File JSON to YAML Migration", + (common_checks.check_policy_json, {'conf': CONF})), ) diff --git a/keystone/common/rbac_enforcer/enforcer.py b/keystone/common/rbac_enforcer/enforcer.py index ca6a8e7b2..7add048ce 100644 --- a/keystone/common/rbac_enforcer/enforcer.py +++ b/keystone/common/rbac_enforcer/enforcer.py @@ -14,6 +14,7 @@ import functools import flask from oslo_log import log +from oslo_policy import opts from oslo_policy import policy as common_policy from oslo_utils import strutils @@ -39,6 +40,13 @@ _POSSIBLE_TARGET_ACTIONS = frozenset([ _ENFORCEMENT_CHECK_ATTR = 'keystone:RBAC:enforcement_called' +# TODO(gmann): Remove setting the default value of config policy_file +# once oslo_policy change the default value to 'policy.yaml'. +# https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 +DEFAULT_POLICY_FILE = 'policy.yaml' +opts.set_defaults(CONF, DEFAULT_POLICY_FILE) + + class RBACEnforcer(object): """Enforce RBAC on API calls.""" diff --git a/keystone/conf/__init__.py b/keystone/conf/__init__.py index 77c26a11d..5de0ec183 100644 --- a/keystone/conf/__init__.py +++ b/keystone/conf/__init__.py @@ -18,6 +18,7 @@ from oslo_log import log from oslo_log import versionutils import oslo_messaging from oslo_middleware import cors +from oslo_policy import opts as policy_opts from osprofiler import opts as profiler from keystone.conf import application_credential @@ -185,6 +186,12 @@ def set_external_opts_defaults(): # configure OSprofiler options profiler.set_defaults(CONF, enabled=False, trace_sqlalchemy=False) + # TODO(gmann): Remove setting the default value of config policy_file + # once oslo_policy change the default value to 'policy.yaml'. + # https://github.com/openstack/oslo.policy/blob/a626ad12fe5a3abd49d70e3e5b95589d279ab578/oslo_policy/opts.py#L49 + DEFAULT_POLICY_FILE = 'policy.yaml' + policy_opts.set_defaults(cfg.CONF, DEFAULT_POLICY_FILE) + # Oslo.cache is always enabled by default for request-local caching # TODO(morganfainberg): Fix this to not use internal interface when # oslo.cache has proper interface to set defaults added. This is diff --git a/keystone/tests/unit/test_policy.py b/keystone/tests/unit/test_policy.py index 8346549ec..d58ea6399 100644 --- a/keystone/tests/unit/test_policy.py +++ b/keystone/tests/unit/test_policy.py @@ -197,7 +197,7 @@ class PolicyJsonTestCase(unit.TestCase): def test_all_targets_documented(self): policy_keys = self._get_default_policy_rules() - # These keys are in the policy.json but aren't targets. + # These keys are in the policy.yaml but aren't targets. policy_rule_keys = [ 'admin_or_owner', 'admin_or_token_subject', 'admin_required', 'owner', 'service_admin_or_token_subject', 'service_or_admin', diff --git a/keystone/tests/unit/test_v3.py b/keystone/tests/unit/test_v3.py index 56ebf4a90..7d6c6b11f 100644 --- a/keystone/tests/unit/test_v3.py +++ b/keystone/tests/unit/test_v3.py @@ -247,7 +247,7 @@ class RestfulTestCase(unit.SQLDriverOverrides, rest.RestfulTestCase, domain_id=DEFAULT_DOMAIN_ID) self.default_domain_user_id = self.default_domain_user['id'] - # create & grant policy.json's default role for admin_required + # create & grant policy.yaml's default role for admin_required self.role = unit.new_role_ref(name='admin') self.role_id = self.role['id'] PROVIDERS.role_api.create_role(self.role_id, self.role) diff --git a/lower-constraints.txt b/lower-constraints.txt index 9e9220199..431589d02 100644 --- a/lower-constraints.txt +++ b/lower-constraints.txt @@ -25,16 +25,16 @@ oauthlib==0.6.2 os-api-ref==1.4.0 oslo.cache==1.26.0 oslo.concurrency==3.26.0 -oslo.config==5.2.0 +oslo.config==6.8.0 oslo.context==2.22.0 oslo.db==6.0.0 oslo.i18n==3.15.3 oslo.log==3.44.0 oslo.messaging==5.29.0 oslo.middleware==3.31.0 -oslo.policy==3.0.2 +oslo.policy==3.6.0 oslo.serialization==2.18.0 -oslo.upgradecheck==0.1.0 +oslo.upgradecheck==1.3.0 oslo.utils==3.33.0 oslotest==3.2.0 osprofiler==1.4.0 diff --git a/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml new file mode 100644 index 000000000..c9c530004 --- /dev/null +++ b/releasenotes/notes/deprecate-json-formatted-policy-file-95f6307f88358f58.yaml @@ -0,0 +1,20 @@ +--- +upgrade: + - | + The default value of ``[oslo_policy] policy_file`` config option has + been changed from ``policy.json`` to ``policy.yaml``. + Operators who are utilizing customized or previously generated + static policy JSON files (which are not needed by default), should + generate new policy files or convert them in YAML format. Use the + `oslopolicy-convert-json-to-yaml + `_ + tool to convert a JSON to YAML formatted policy file in + backward compatible way. +deprecations: + - | + Use of JSON policy files was deprecated by the ``oslo.policy`` library + during the Victoria development cycle. As a result, this deprecation is + being noted in the Wallaby cycle with an anticipated future removal of support + by ``oslo.policy``. As such operators will need to convert to YAML policy + files. Please see the upgrade notes for details on migration of any + custom policy files. diff --git a/requirements.txt b/requirements.txt index 33a2c423e..3464fb3fc 100644 --- a/requirements.txt +++ b/requirements.txt @@ -16,16 +16,16 @@ keystonemiddleware>=7.0.0 # Apache-2.0 bcrypt>=3.1.3 # Apache-2.0 scrypt>=0.8.0 # BSD oslo.cache>=1.26.0 # Apache-2.0 -oslo.config>=5.2.0 # Apache-2.0 +oslo.config>=6.8.0 # Apache-2.0 oslo.context>=2.22.0 # Apache-2.0 oslo.messaging>=5.29.0 # Apache-2.0 oslo.db>=6.0.0 # Apache-2.0 oslo.i18n>=3.15.3 # Apache-2.0 oslo.log>=3.44.0 # Apache-2.0 oslo.middleware>=3.31.0 # Apache-2.0 -oslo.policy>=3.0.2 # Apache-2.0 +oslo.policy>=3.6.0 # Apache-2.0 oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 -oslo.upgradecheck>=0.1.0 # Apache-2.0 +oslo.upgradecheck>=1.3.0 # Apache-2.0 oslo.utils>=3.33.0 # Apache-2.0 oauthlib>=0.6.2 # BSD pysaml2>=5.0.0 -- cgit v1.2.1