summaryrefslogtreecommitdiff
path: root/python/ovs/json.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove dependency on python3-sixTimothy Redaelli2019-12-201-13/+5
| | | | | | | | | | | Since Python 2 support was removed in 1ca0323e7c29 ("Require Python 3 and remove support for Python 2."), python3-six is not needed anymore. Moreover python3-six is not available on RHEL/CentOS7 without using EPEL and so this patch is needed in order to release OVS 2.13 on RHEL7. Signed-off-by: Timothy Redaelli <tredaelli@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* Un-revert Work around Python/C JSON unicode differencesTerry Wilson2019-01-151-4/+6
| | | | | | | | | | | | | | | | | This fix was reverted because it depended on a small bit of code in a patch that was reverted that changed some python/ovs testing and build. The fix is still necessary. The OVS C-based JSON parser operates on bytes, so the parser_feed function returns the number of bytes that are processed. The pure Python JSON parser currently operates on unicode, so it expects that Parser.feed() returns a number of characters. This difference leads to parsing errors when unicode characters are passed to the C JSON parser from Python. Acked-by: Lucas Alvares Gomes <lucasagomes@gmail.com> Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* python: Fix invalid escape sequences.Ben Pfaff2019-01-111-2/+2
| | | | | | | | | | It appears that Python silently treats invalid escape sequences in strings as literals, e.g. "\." is the same as "\\.". Newer versions of checkpatch complain, and it does seem reasonable to me to fix these. Acked-by: Numan Siddique <nusiddiq@redhat.com> Tested-by: Numan Siddique <nusiddiq@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* Revert "Test the Python C JSON extension"Ilya Maximets2018-10-151-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit a7be68a4d77791bbe02c37f7ad8ae60b02e5679e and a subsequent commit 4617d1f6bd24c543f533f6485b42ebca6b0a8371. There are too many issues with these patches. It's better to revert them for now and make a separate fixed versions later if needed. List of issues (maybe not full): 1. 'make clean' removes entire 'python' directory. 2. Fully broken Travis-CI testsuite build: building 'ovs._json' extension creating build/temp.linux-x86_64-2.7 error: could not create 'build/temp.linux-x86_64-2.7': \ Permission denied https://travis-ci.org/openvswitch/ovs/jobs/440693765 3. Broken local testsuite build on Ubuntu 18.04: running build_ext building 'ovs._json' extension creating build/temp.linux-x86_64-3.6 creating build/temp.linux-x86_64-3.6/ovs <...> /usr/bin/ld: .libs/libopenvswitch.a(util.o): \ relocation R_X86_64_TPOFF32 against `var.7749' can not be \ used when making a shared object; recompile with -fPIC <...> collect2: error: ld returned 1 exit status 4. Fedora build failure because of 'setuptools' ('distutils') hard dependency on 'redhat-rpm-config' package: building 'ovs._json' extension <...> gcc: error: <...>/redhat-hardened-cc1: No such file or directory 5. Looks like 'setuptools' also could download and install unwanted python modules during package build. Signed-off-by: Ilya Maximets <i.maximets@samsung.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* Test the Python C JSON extensionTerry Wilson2018-10-111-4/+6
| | | | | | | | | | | | | | | | | | The C JSON parser was added quite a while ago, but unless you configure with --enable-shared and have the Python 2/3 development libraries installed, and the resulting python-ovs module installed, 'make check' won't actually test it. This patch changes Python-based tests to run from the $builddir/python directory and makes the tests configurable to use both JSON backends. There are some unicode failures in the C JSON extension that I left unfixed in this patch to make it easy to show run the new tests on broken code. The next patch in this set works around the issue. Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org> Acked-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
* python: make python idl unicode-tolerantLance Richardson2017-08-091-2/+7
| | | | | | | | | | | | Ensure that JSON is utf-8 encoded and that bytes sent/received on the stream sockets are in utf-8 form. Add a test case to verify that unicode data can be sent/received successfully using Python IDL module. Co-authored-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Lance Richardson <lrichard@redhat.com> Signed-off-by: Russell Bryant <russell@ovn.org>
* Adapt to flake8-import-orderxurong000379972017-03-081-0/+1
| | | | | | | | | | https://review.openstack.org/#/c/432906/ flake8-import-order adds 3 new flake8 warnings: I100: Your import statements are in the wrong order. I101: The names in your from import are in the wrong order. I201: Missing newline between sections or imports. Signed-off-by: Ben Pfaff <blp@ovn.org>
* python: Serial JSON via Python's json lib.Terry Wilson2016-07-261-96/+10
| | | | | | | | | | | | There is no particularly good reason to use our own Python JSON serialization implementation when serialization can be done faster with Python's built-in JSON library. A few tests were changed due to Python's default JSON library returning slightly more precise floating point numbers. Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* Add optional C extension wrapper for Python JSON parsingTerry Wilson2016-06-081-0/+11
| | | | | | | | | | | | | | | | | | | | The pure Python in-tree JSON parser is *much* slower than the in-tree C JSON parser. A local test parsing a 100Mb JSON file showed the Python version taking 270 seconds. With the C wrapper, it took under 4 seconds. The C extension will be used automatically if it can be built. If the extension fails to build, a warning is displayed and the build is restarted without the extension. The Serializer class is replaced with Python's built-in JSON library since the ability to process chunked data is not needed in that case. The extension should work with both Python 2.7 and Python 3.3+. Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* Ensure significand remains an integer in Python3 json parserTerry Wilson2016-06-081-1/+1
| | | | | | | | | | | | | | | | The / operation in Python 2 is "floor division" for int/long types while in Python 3 is "true division". This means that the significand can become a float with the existing code in Python 3. This, in turn, can result in a parse of something like [1.10e1] returning 11 in Python 2 and 11.0 in Python 3. Switching to the // operator resolves this difference. The JSON tests do not catch this difference because the built-in serializer prints floats with the %.15g format which will convert floats with no fractional part to an integer representation. Signed-off-by: Terry Wilson <twilson@redhat.com> Signed-off-by: Ben Pfaff <blp@ovn.org>
* python: Remove reamining direct type comparisons.Russell Bryant2016-02-021-2/+2
| | | | | | | | | | | I've hit several bugs in this Python 3 work where the fix was some code needed to be converted to use isinstance(). This has been primarily around deadling with the changes to unicode handling. Go ahead and convert the rest of the direct type comparisons to use isinstance(), as it could avoid a bug I haven't hit yet and it's more Pythonic, anyway. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Use six.unichr().Russell Bryant2016-02-021-1/+1
| | | | | | | | six.unichr() is equivalent to unichr() in Python 2 and chr() in Python 3. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Drop use of sys.maxint.Russell Bryant2016-02-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | sys.maxint does not exist in Python 3, as an int does not have a max value anymore (except as limited by implementation details and system resources). sys.maxsize works as a reasonable substitute as it's the same as sys.maxint. The Python 3.0 release notes have this to say: The sys.maxint constant was removed, since there is no longer a limit to the value of integers. However, sys.maxsize can be used as an integer larger than any practical list or string index. It conforms to the implementation’s “natural” integer size and is typically the same as sys.maxint in previous releases on the same platform (assuming the same build options). sys.maxsize is documented as: An integer giving the maximum value a variable of type Py_ssize_t can take. It’s usually 2**31 - 1 on a 32-bit platform and 2**63 - 1 on a 64-bit platform. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Don't use StringIO directly.Russell Bryant2016-02-021-2/+1
| | | | | | | | StringIO.StringIO in Python 2 became io.StringIO in Python 3. Use six.StringIO which is an alias for the two cases. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Drop unicode type.Russell Bryant2016-02-021-9/+16
| | | | | | | | | | Python 2 had str and unicode. Python 3 only has str, which is always a unicode string. Drop use of unicode with the help of six.text_type (unicode in py2 and str in py3) and six.string_types ([str, unicode] in py2 and [str] in py3). Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Drop usage of long type.Russell Bryant2016-02-021-9/+12
| | | | | | | | | | | | | | | | | | | | | Python 2 has both long and int types. Python 3 only has int, which behaves like long. In the case of needing a set of integer types, we can use six.integer_types which includes int and long for Python 2 and just int for Python 3. We can convert all cases of long(value) to int(value), because as of Python 2.4, when the result of an operation would be too big for an int, the type is automatically converted to a long. There were several places in this patch doing type comparisons. The preferred way to do this is using the isinstance() or issubclass() built-in functions, so I converted the similar checks nearby while I was at it. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Convert dict iterators.Russell Bryant2016-01-211-1/+2
| | | | | | | | | | | | | | | | In Python 2, dict.items(), dict.keys(), and dict.values() returned a list. dict.iteritems(), dict.iterkeys(), and dict.itervalues() returned an iterator. As of Python 3, dict.iteritems(), dict.itervalues(), and dict.iterkeys() are gone. items(), keys(), and values() now return an iterator. In the case where we want an iterator, we now use the six.iter*() helpers. If we want a list, we explicitly create a list from the iterator. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Stop using xrange().Russell Bryant2016-01-201-0/+2
| | | | | | | | | | | | Python 2 had range() and xrange(). xrange() is more efficient, but behaves differently so range() was retained for compatibility. Python 3 only has range() and it behaves like Python 2's xrange(). Remove explicit use of xrange() and use six.moves.range() to make sure we're using xrange() from Python 2 or range() from Python 3. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Start fixing some Python 3 issues.Terry Wilson2016-01-121-1/+1
| | | | | | | | | | | | | This patch fixes just the Python 3 problems found by running: python3 setup.py install There are still many other issues to be fixed, but this is a start. Signed-off-by: Terry Wilson <twilson@redhat.com> [russell@ovn.org resolved conflicts with current master] Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Fix several pep8 whitespace errors.Russell Bryant2016-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | Fix the following pep8 errors: E201 whitespace after '(' E203 whitespace before ',' E222 multiple spaces after operator E225 missing whitespace around operator E226 missing whitespace around arithmetic operator E231 missing whitespace after ':' E241 multiple spaces after ':' E251 unexpected spaces around keyword / parameter equals E261 at least two spaces before inline comment E262 inline comment should start with '# ' E265 block comment should start with '# ' E271 multiple spaces after keyword Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* python: Resolve pep8 comparison errors.Russell Bryant2016-01-051-1/+1
| | | | | | | | | | | | | | | | | | | | | Resolve pep8 errors: E711 comparison to None should be 'if cond is None:' The reason comparing against None with "is None" is preferred over "== None" is because a class can define its own equality operator and produce bizarre unexpected behavior. Using "is None" has a very explicit meaning that can not be overridden. E721 do not compare types, use 'isinstance()' This one is actually a mistake by the tool in most cases. 'from ovs.db import types' looks just like types from the Python stdlib. In those cases, use the full ovs.db.types name. Fix one case where it actually was types from the stdlib. Signed-off-by: Russell Bryant <russell@ovn.org> Acked-by: Ben Pfaff <blp@ovn.org>
* Global replace of Nicira Networks.Raju Subramanian2012-05-021-1/+1
| | | | | | | | Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc. Feature #10593 Signed-off-by: Raju Subramanian <rsubramanian@nicira.com> Signed-off-by: Ben Pfaff <blp@nicira.com>
* json: Correct position tracking in JSON parser implementations.Ben Pfaff2012-04-261-9/+11
| | | | | | | | | | | When json_lex_input() returns false, the parser does not consume the byte passed in. That byte will get processed again in the next iteration of the json_parser_feed() loop. Therefore, until now, this code has double-counted bytes that cause a false return from json_lex_input(). This fixes the problem. Every input byte is now counted only once. Signed-off-by: Ben Pfaff <blp@nicira.com>
* json.py: Typo in parsing code.Ben Pfaff2011-09-261-1/+1
|
* python: Style cleanup.Ethan Jackson2011-09-241-9/+45
| | | | | | This patch does minor style cleanups to the code in the python and tests directory. There's other code floating around that could use similar treatment, but updating it is not convenient at the moment.
* ovs.json: Remove commented-out debug code.Ben Pfaff2011-09-231-3/+0
| | | | This must have slipped into an old commit by accident.
* ovs.json: Actually implement the "pretty" option for serialization.Ben Pfaff2011-09-231-38/+74
|
* python: Avoid shadowing standard or global names.Ben Pfaff2011-09-231-3/+3
| | | | Found by pychecker.
* python: Avoid "unused parameter" warnings from pychecker.Ben Pfaff2011-09-231-6/+6
| | | | pychecker ignores parameters named "_" or prefixed with "unused_".
* ovs.json: Use Exception, which exists, instead of Error, which doesn't.Ben Pfaff2011-08-251-1/+1
| | | | Found by pychecker.
* python: Use enumerate() builtin function to simplify counted iteration.Ben Pfaff2011-08-241-6/+2
| | | | Suggested-by: Reid Price <reid@nicira.com>
* ovs.json: Optimize __dump_string().Ben Pfaff2011-08-241-9/+1
| | | | Suggested-by: Reid Price <reid@nicira.com>
* python: Make invalid UTF-8 sequence messages consistent across Python versions.Ben Pfaff2011-07-061-1/+2
| | | | | | | | | | | | | | Given the invalid input <C0 22>, some versions of Python report <C0> as the invalid sequence and other versions report <C0 22> as the invalid sequence. Similarly, given input <ED 80 7F>, some report <ED 80> and others report <ED 80 7F> as the invalid sequence. This caused spurious test failures for the test "no invalid UTF-8 sequences in strings - Python", so this commit makes the messages consistent by dropping the extra trailing byte from the message. I first noticed the longer sequences <C0 22> and <ED 80 7F> on Ubuntu 10.04 with python version 2.6.5-0ubuntu1, but undoubtedly it exists elsewhere also.
* tests: Fix the two Python XFAIL tests.Ben Pfaff2011-05-241-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | OVS has two Python tests that have always failed, for reasons not understood, since they were added to the tree. This commit fixes them. One problem was that Python was assuming that stdout was encoded in ASCII. Apparently the only way to "fix" this at runtime is to set PYTHONIOENCODING to utf_8 in the environment, so this change does that. Second, it appears that Python really doesn't like to print invalid UTF-8, so this avoids doing that in python/ovs/json.py, instead just printing the hexadecimal values of the invalid bytes. For consistency, it makes the same change to the C version. Third, the C version of test-ovsdb doesn't check UTF-8 for consistency, it just sends it blindly to the OVSDB server, but Python does check it and so it bails out earlier. This commit changes the Python version of the "no invalid UTF-8 sequences in strings" to allow for the slight difference in output that occurs for that reason. Finally, test-ovsdb.py needs to convert error messages to Unicode explicitly before printing them in the "parse-atoms" function. I don't really understand why, but now it works.
* Implement initial Python bindings for Open vSwitch database.Ben Pfaff2010-08-251-0/+528
These initial bindings pass a few hundred of the corresponding tests for C implementations of various bits of the Open vSwitch library API. The poorest part of them is actually the Python IDL interface in ovs.db.idl, which has not received enough attention yet. It appears to work, but it doesn't yet support writes (transactions) and it is difficult to use. I hope to improve it as it becomes clear what semantics Python applications actually want from an IDL.
View Lorry Controller Status