summaryrefslogtreecommitdiff
path: root/tools/patman
Commit message (Collapse)AuthorAgeFilesLines
* patman: Move to use Python 3Simon Glass2019-11-041-1/+1
| | | | | | Update this tool to use Python 3 to meet the 2020 deadline. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Use unicode for file I/OSimon Glass2019-11-045-11/+11
| | | | | | | At present patman test fail in some environments which don't use utf-8 as the default file encoding. Add this explicitly. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Adjust 'command' to return strings instead of bytesSimon Glass2019-11-042-16/+44
| | | | | | | | | | | | At present all the 'command' methods return bytes. Most of the time we actually want strings, so change this. We still need to keep the internal representation as bytes since otherwise unicode strings might break over a read() boundary (e.g. 4KB), causing errors. But we can convert the end result to strings. Add a 'binary' parameter to cover the few cases where bytes are needed. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: separate emails in CC list with NULsDmitry Torokhov2019-10-273-5/+5
| | | | | | | | | | There is a contributor in Linux kernel with a comma in their name, which confuses patman and results in invalid to- or cc- addresses on some patches. To avoid this, let's use \0 as a separator when generating cc file. Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com> Reviewed-by: Simon Glass <sjg@chromium.org>
* patman: Use the Change-Id, version, and prefix in the Message-IdDouglas Anderson2019-10-154-5/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As per the centithread on ksummit-discuss [1], there are folks who feel that if a Change-Id is present in a developer's local commit that said Change-Id could be interesting to include in upstream posts. Specifically if two commits are posted with the same Change-Id there's a reasonable chance that they are either the same commit or a newer version of the same commit. Specifically this is because that's how gerrit has trained people to work. There is much angst about Change-Id in upstream Linux, but one thing that seems safe and non-controversial is to include the Change-Id as part of the string of crud that makes up a Message-Id. Let's give that a try. In theory (if there is enough adoption) this could help a tool more reliably find various versions of a commit. This actually might work pretty well for U-Boot where (I believe) quite a number of developers use patman, so there could be critical mass (assuming that enough of these people also use a git hook that adds Change-Id to their commits). I was able to find this git hook by searching for "gerrit change id git hook" in my favorite search engine. In theory one could imagine something like this could be integrated into other tools, possibly even git-send-email. Getting it into patman seems like a sane first step, though. NOTE: this patch is being posted using a patman containing this patch, so you should be able to see the Message-Id of this patch and see that it contains my local Change-Id, which ends in 2b9 if you want to check. [1] https://lists.linuxfoundation.org/pipermail/ksummit-discuss/2019-August/006739.html Signed-off-by: Douglas Anderson <dianders@chromium.org>
* binman: Use the Makefile for u_boot_ucode_ptrSimon Glass2019-10-151-1/+1
| | | | | | | | | | Remove this file from git and instead build it using the Makefile. Update tools.GetInputFilename() to support reading files from an absolute path, so that we can read the Elf test files easily. Also make sure that the temp directory is report in ELF tests as this was commented out. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Update command.Run() to handle failure betterSimon Glass2019-10-151-2/+8
| | | | | | | | | | At present tools are not expected to fail. If they do an exception is raised but there is no detail about what went wrong. This makes it hard to debug if something does actually go wrong. Fix this by outputting both stderr and stdout on failure. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Drop binary parameterSimon Glass2019-10-152-10/+8
| | | | | | | Since cros_subprocess use bytestrings now, this feature not needed. Drop it. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Reset the output directory when it is removedSimon Glass2019-07-291-0/+2
| | | | | | | At present outdir remains set ever after the output directory has been removed. Fix this to avoid trying to access it when it is not present. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Support replacing data in a cbfsSimon Glass2019-07-291-2/+9
| | | | | | | | | | | | | | | | | | | At present binman cannot replace data within a CBFS since it does not allow rewriting of the files in that CBFS. Implement this by using the new WriteData() method to handle the case. Add a header to compressed data so that the amount of compressed data can be determined without reference to the size of the containing entry. This allows the entry to be larger that the contents, without causing errors in decompression. This is necessary to cope with a compressed device tree being updated in such a way that it shrinks after the entry size is already set (an obscure case). It is not used with CBFS since it has its own metadata for this. Increase the number of passes allowed to resolve the position of entries, to handle this case. Add a test for this new logic. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Add a bit of logging in entries when packingSimon Glass2019-07-291-0/+16
| | | | | | | | | Use the new logging feature to log information about progress with packing. This is useful to see how binman is figuring things out. Also update elf.py to use the same feature. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Update tout to avoid open-coding the debug levelsSimon Glass2019-07-291-11/+7
| | | | | | Use the debug level constants instead of open-coding them in the file. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Use print() to print outputSimon Glass2019-07-291-1/+3
| | | | | | | | At present tout writes directly to stdout. This is not necessary and it prevents tests from redirecting output. Change it to use print() for the non-progress output. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Allow for logging information to be displayedSimon Glass2019-07-241-1/+9
| | | | | | | | Binman generally operates silently but in some cases it is useful to see what Binman is actually doing at each step. Enable some logging output with different logging levels selectable via the -v flag. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Allow easy importing of entry modulesSimon Glass2019-07-241-0/+1
| | | | | | | | | | | At present entry modules can only be accessed using Entry.Lookup() or Entry.Create(). Most of the time this is fine, but sometimes a module needs to provide constants or helper functions useful to other modules. It is easier in this case to use 'import'. Add an __init__ file to permit this. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Convert to use ArgumentParserSimon Glass2019-07-241-2/+3
| | | | | | | | This class is the new way to handle arguments in Python. Convert binman over to use it. At the same time, introduce commands so that we can separate out the different parts of binman functionality. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Add a function to write ifwitoolSimon Glass2019-07-241-0/+33
| | | | | | | | This tool has quite a few arguments and options, so put the functionality in a function so that we call it from one place and hopefully get it right. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Add functions to compress and decompress dataSimon Glass2019-07-231-0/+66
| | | | | | | | | | | Add utility functions to compress and decompress using lz4 and lzma algorithms. In the latter case these use the legacy lzma support favoured by coreboot's CBFS. No tests are provided as these functions will be tested by the CBFS tests in a separate patch. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Add a way to set the search path for toolsSimon Glass2019-07-231-5/+37
| | | | | | | Sometimes tools can be located by looking in other locations. Add a way to direct the search. Signed-off-by: Simon Glass <sjg@chromium.org>
* tools: Drop duplicate raise_on_error argumentSimon Glass2019-07-231-2/+2
| | | | | | | If kwargs contains raise_on_error then this function generates an error due to a duplicate argument. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Update cover-coverage tests for Python 3Simon Glass2019-07-101-6/+9
| | | | | | | We need slightly different commands to run code coverage with Python 3. Update the RunTestCoverage() function to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Allow reading files in text modeSimon Glass2019-07-101-2/+2
| | | | | | | While reading files in binary mode is the norm, sometimes we want to use text mode. Add an optional parameter to handle this. Signed-off-by: Simon Glass <sjg@chromium.org>
* dtoc: Use byte type instead of str in fdtSimon Glass2019-07-101-0/+25
| | | | | | | In Python 3 bytes and str are separate types. Use bytes to ensure that the code functions correctly with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
* dtoc: Updates BytesToValue() for Python 3Simon Glass2019-07-101-0/+27
| | | | | | | | | The difference between the bytes and str types in Python 3 requires a number of minor changes to this function. Update it to handle the input data using the 'bytes' type. Create two useful helper functions which can be used by other modules too. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Don't require Python 2Simon Glass2019-07-101-1/+1
| | | | | | Update the shebang to allow either Python 2 or Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Tidy up a few more unicode conversionsSimon Glass2019-07-102-6/+3
| | | | | | Use the new functions in the tools module to handle conversion. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Adjust functional tests for Python 3Simon Glass2019-07-101-11/+14
| | | | | | | | | Change the code so that it works on both Python 2 and Python 3. This works by using unicode instead of latin1 for the test input, and ensuring that the output is converted to a string rather than a unicode object on Python 2. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Avoid unicode type in settings unit testsSimon Glass2019-07-101-8/+8
| | | | | | | | | The unicode type does not exist in Python 3 and when displaying strings they do not have the 'u' prefix. Adjusts the settings unit tests to deal with this difference, by converting the comparison value to a string, thus dropping the 'u'. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Sort series output for repeatabilySimon Glass2019-07-102-12/+12
| | | | | | | | We use sets to produce the list of To and Cc lines for a series. This does not result in stable ordering of the recipients. Sort each list to ensure that the output is repeatable. This is necessary for tests. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Move unicode helpers to toolsSimon Glass2019-07-104-16/+39
| | | | | | | | Create helper functions in the tools module to deal with the differences between unicode in Python 2 (where we use the 'unicode' type) and Python 3 (where we use the 'str' type). Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Support use of stringIO in Python 3Simon Glass2019-07-101-1/+6
| | | | | | With Python 3 this class has moved. Update the code to handle both cases. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Handle repeated bytes for Python 3Simon Glass2019-07-101-0/+19
| | | | | | | | | | | | The method of multiplying a character by a number works well for creating a repeated string in Python 2. But in Python 3 we need to use bytes() instead, to avoid unicode problems, since 'bytes' is no-longer just an alias of 'str'. Create a function to handle this detail and call it from the relevant places in binman. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Provide a way to get program output in binary modeSimon Glass2019-07-102-3/+4
| | | | | | | | | At present cros_subprocess and the tools library use a string to obtain stdout from a program. This works fine on Python 2. With Python 3 we end up with unicode errors in some cases. Fix this by providing a binary mode, which returns the data as bytes() instead of a string. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Use items() instead of iteritems()Simon Glass2019-07-101-1/+1
| | | | | | | Python 3 requires this, and Python 2 allows it. Convert the code over to ensure compatibility with Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Convert print statements to Python 3Simon Glass2019-07-103-15/+14
| | | | | | Update all print statements to be functions, as required by Python 3. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Update cros_subprocess to use bytesSimon Glass2019-07-102-20/+29
| | | | | | | | At present this function uses lists and strings. This does not work so well with Python 3, and testing against '' does not work for a bytearray. Update the code to fix these issues. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Run tests concurrentlySimon Glass2018-10-081-1/+1
| | | | | | | | | | | | | | | | | At present the tests run one after the other using a single CPU. This is not very efficient. Bring in the concurrencytest module and run the tests concurrently, using one process for each CPU by default. A -P option allows this to be overridden, which is necessary for code-coverage to function correctly. This requires fixing a few tests which are currently not fully independent. At some point we might consider doing this across all pytests in U-Boot. There is a pytest version that supports specifying the number of processes to use, but it did not work for me. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Don't clear progress in tout unless it was usedSimon Glass2018-10-081-1/+7
| | | | | | | | | | At present calling Uninit() always called ClearProgress() which outputs a \r character as well as spaces to remove any progress information on the line. This can mess up the normal output of binman and other tools. Fix this by outputing this only when progress information has actually been previous written. Signed-off-by: Simon Glass <sjg@chromium.org>
* tools: Set an initial value for indirSimon Glass2018-10-081-0/+3
| | | | | | This variable is not documented or set up in the module. Fix this. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Handle unicode in _ProjectConfigParser testsSimon Glass2018-10-081-7/+20
| | | | | | | | | With Python 2.7.15rc1, ConfigParser.SafeConfigParser has unfortunately started returning unicode, for unknown reasons. Adjust the code to handle this by converting everything to unicode. We cannot convert things to ASCII since email addresses may be encoded with UTF-8. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Support adding filesSimon Glass2018-09-291-0/+18
| | | | | | | | | | | | In some cases it is useful to add a group of files to the image and be able to access them at run-time. Of course it is possible to generate the binman config file with a set of blobs each with a filename. But for convenience, add an entry type which can do this. Add required support (for adding nodes and string properties) into the state module. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Detect missing tools and report themSimon Glass2018-09-281-1/+28
| | | | | | | | | | When tools are needed but not present, at present we just get an error which can be confusing for the user. Try to be helpful by reporting the tool as missing and suggesting a possible remedy. Also update the Run() method to support this. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Correct unit test failureSimon Glass2018-08-081-0/+0
| | | | | | | | A recent rename of the function did not rename the test file. Fix this. Fixes: 12308b128fa (lib: fdtdec: Rename routine fdtdec_setup_memory_size()) Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Show the current directory in GetInputFilename()Simon Glass2018-08-011-2/+2
| | | | | | | | When this fails it is useful to see the current directory, since U-Boot's build system will typically change into the output directory during the build. Add this information to the error. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Add a few more helpers to the tools librarySimon Glass2018-08-011-1/+75
| | | | | | | Add functions to read and write a file, looking through a list of search paths to find it. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Allow test commands to fall back to real onesSimon Glass2018-08-011-2/+6
| | | | | | | | | | | | Tests use the 'test_result' feature to return a predetermined command result for particular commands. The avoids needing to have the real command available just to run a test. It works by calling the function provided by the test, to get the value. However sometimes the test does need to run the real command. Allow it to fall back to do this when the function does not return a result. Signed-off-by: Simon Glass <sjg@chromium.org>
* patman: Split alias entries on arbitrary whitespaceAdam Sampson2018-07-261-1/+1
| | | | | | | | | Previously the first three words in a git-mailrc alias entry could only be separated by spaces. git-send-email and Mutt both allow arbitrary whitespace here. Signed-off-by: Adam Sampson <ats@offog.org> Reviewed-by: Simon Glass <sjg@chromium.org>
* lib: fdtdec: Rename routine fdtdec_setup_memory_size()Siva Durga Prasad Paladugu2018-07-194-5/+5
| | | | | | | | | | | This patch renames the routine fdtdec_setup_memory_size() to fdtdec_setup_mem_size_base() as it now fills the mem base as well along with size. Signed-off-by: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com> Signed-off-by: Michal Simek <michal.simek@xilinx.com> Reviewed-by: Tom Rini <trini@konsulko.com>
* binman: Move capture_sys_output() to test_utilSimon Glass2018-07-091-0/+21
| | | | | | | This function is useful in various tests. Move it into the common test utility module. Signed-off-by: Simon Glass <sjg@chromium.org>
* binman: Move coverage logic into a new test_util fileSimon Glass2018-07-091-0/+64
| | | | | | | | At present only binman has the logic for determining Python test coverage but this is useful for other tools also. Move it out into a separate file so it can be used by other tools. Signed-off-by: Simon Glass <sjg@chromium.org>
View Lorry Controller Status