Resolving the performance issue for image listing of v2 API

According to my diagnosis [1], currently v2 API client consumed a lot
of CPU times by JSON schema validation on image model initialization
stage for image listing case. Because each image entry will triggers
one validation operation which spent most CPU clocks. (My test env has
1002 image entries now, so it's O(N) logic obviously).

This fix changed the original logic, it doesn't validate each image
entry but do it only on first image entry for each page. This approach
is a trade-off, it balanced the data validity and performance.

As comparison [3], under the fixed new logic, the execution time of
image listing is about four times as fast as old logic.

[1]
Clock type: CPU
Ordered by: totaltime, desc
name                                  ncall ttot
..nt/v2/images.py:34 Controller.list  1003  12.08480
../warlock/core.py:30 image.__init__  1002  11.91074
..warlock/model.py:28 image.__init__  1002  11.90463
..arlock/model.py:130 image.validate  1002  11.73838
..nschema/validators.py:388 validate  1002  11.73682

[2]
name                                  ncall ttot
..nt/v2/images.py:35 Controller.list  1003  1.100710
..nceclient/v2/images.py:45 paginate  256.. 1.099965
../warlock/core.py:30 image.__init__  1002  0.944762
..warlock/model.py:28 image.__init__  1002  0.939725
..arlock/model.py:130 image.validate  51    0.779653
..nschema/validators.py:388 validate  51    0.779431

[3]
$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real	0m16.606s
user	0m10.629s
sys	0m2.540s

$ time glance --os-image-api-version 2 image-list | grep 'test-' | wc -l
1000
real	0m4.151s
user	0m1.080s
sys	0m0.092s

Change-Id: Ia6598d3c06a5ff516277053c2b6fa5db744fe9cf
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>

1 files changed, 19 insertions, 5 deletions

diff --git a/glanceclient/v2/images.py b/glanceclient/v2/images.py
index 53b65ef..10b7864 100644
--- a/glanceclient/v2/images.py
+++ b/glanceclient/v2/images.py
@@ -37,10 +37,27 @@ class Controller(object):
         :param page_size: Number of images to request in each paginated request
         :returns generator over list of Images
         """
+
+        ori_validate_fun = self.model.validate
+        empty_fun = lambda *args, **kwargs: None
+
         def paginate(url):
             resp, body = self.http_client.json_request('GET', url)
             for image in body['images']:
-                yield image
+                # NOTE(bcwaldon): remove 'self' for now until we have
+                # an elegant way to pass it into the model constructor
+                # without conflict.
+                image.pop('self', None)
+                yield self.model(**image)
+                # NOTE(zhiyan): In order to resolve the performance issue
+                # of JSON schema validation for image listing case, we
+                # don't validate each image entry but do it only on first
+                # image entry for each page.
+                self.model.validate = empty_fun
+
+            # NOTE(zhiyan); Reset validation function.
+            self.model.validate = ori_validate_fun
+
             try:
                 next_url = body['next']
             except KeyError:
@@ -73,10 +90,7 @@ class Controller(object):
             url = '%s&%s' % (url, parse.urlencode(param))
 
         for image in paginate(url):
-            #NOTE(bcwaldon): remove 'self' for now until we have an elegant
-            # way to pass it into the model constructor without conflict
-            image.pop('self', None)
-            yield self.model(**image)
+            yield image
 
     def get(self, image_id):
         url = '/v2/images/%s' % image_id