Handle ServiceNotFound in DbDriver._report_state

If a service record is gone from the DB the _report_state method
will log a generic traceback every time the report interval runs,
which is every 10 seconds by default. This gets pretty noisy and
the error logged isn't very useful. One could get into this state
by deleting the service record in the API before stopping the actual
process that is running.

This simply handles the ServiceNotFound error and logs a more useful
error message without the noisy traceback.

Change-Id: If0336001fbe39922a199756db0803121cbe560af
Related-Bug: #1813147

1 files changed, 11 insertions, 0 deletions

diff --git a/nova/servicegroup/drivers/db.py b/nova/servicegroup/drivers/db.py
index 98a6c435e1..9b134ffebb 100644
--- a/nova/servicegroup/drivers/db.py
+++ b/nova/servicegroup/drivers/db.py
@@ -19,6 +19,7 @@ from oslo_utils import timeutils
 import six
 
 import nova.conf
+from nova import exception
 from nova.i18n import _, _LI, _LW, _LE
 from nova.servicegroup import api
 from nova.servicegroup.drivers import base
@@ -103,6 +104,16 @@ class DbDriver(base.Driver):
                 service.model_disconnected = True
                 LOG.warning(_LW('Lost connection to nova-conductor '
                              'for reporting service status.'))
+        except exception.ServiceNotFound:
+            # The service may have been deleted via the API but the actual
+            # process is still running. Provide a useful error message rather
+            # than the noisy traceback in the generic Exception block below.
+            LOG.error('The services table record for the %s service on '
+                      'host %s is gone. You either need to stop this service '
+                      'if it should be deleted or restart it to recreate the '
+                      'record in the database.',
+                      service.service_ref.binary, service.service_ref.host)
+            service.model_disconnected = True
         except Exception:
             # NOTE(rpodolyaka): we'd like to avoid catching of all possible
             # exceptions here, but otherwise it would become possible for