From 1a5e3e748a5ea4f48e3e88fa3859db4c186d6ba5 Mon Sep 17 00:00:00 2001 From: Richard Maw Date: Fri, 11 Jul 2014 15:09:14 +0000 Subject: Make our use of json binary path safe json only accepts unicode. Various APIs such as file paths and environment variables allow binary data, so we need to support this properly. This patch changes every[1] use of json.load or json.dump to escape non-unicode data strings. This appears exactly as it used to if the input was valid unicode, if it isn't it will insert \xabcd escapes in the place of non-unicode data. When loading back in, if json.load is told to unescape it with `encoding='unicode-escape'` then it will convert it back correctly. This change was primarily to support file paths that weren't valid unicode, where this would choke and die. Now it works, but any tools that parsed the metadata need to unescape the paths. [1]: The interface to the remote repo cache uses json data, but I haven't changes its json.load calls to unescape the data, since the repo caches haven't been made to escape the data. --- scripts/list-overlaps | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'scripts') diff --git a/scripts/list-overlaps b/scripts/list-overlaps index 5fe45a0e..d092ba75 100755 --- a/scripts/list-overlaps +++ b/scripts/list-overlaps @@ -1,6 +1,6 @@ #!/usr/bin/env python # -# Copyright (C) 2011-2012 Codethink Limited +# Copyright (C) 2011-2014 Codethink Limited # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -27,7 +27,7 @@ class ListOverlaps(cliapp.Application): @staticmethod def _load_overlap(filename): - data = json.load(open(filename)) + data = json.load(open(filename), encoding='unicode-escape') overlaps = dict((frozenset(pair[0]), set(pair[1])) for pair in data) return overlaps -- cgit v1.2.1