summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* showjob: Use local variable instead of re-evaluating the same expressionBen Hutchings2020-08-101-1/+1
| | | | | | | | JobShower.get_job_as_json assigns to a local 'path' variable, but doesn't use it. Use it instead of re-evaluating the same expression assigned to it. Caught by pyflakes.
* readconf: Fix handling of invalid time values in configurationBen Hutchings2020-08-101-1/+1
| | | | | | | | | ReadConfiguration.fix_up_interval was returning default_value when the string value did not match the expected format. This variable is not defined, but default_interval is. This bug has been present since the beginning of LC development, and was caught by pyflakes.
* givemejob: Fix upstream host metadata lookupBen Hutchings2020-08-101-2/+2
| | | | | | | | Commit bc7f80d "givemejob: Move upstream host metadata lookup out of get_repo_metadata" failed to pass through the statedb variable into the new function, so it will never work. Caught by pyflakes.
* Merge branch 'robjh/debug-yaml-error' into 'master'Ben Brown2020-08-031-0/+1
|\ | | | | | | | | Rewind lorry FDs before parsing as JSON. See merge request CodethinkLabs/lorry/lorry-controller!21
| * Rewind lorry FDs before parsing as JSON.Robert J. Heywood2020-08-031-0/+1
|/ | | | | | | | | First, the lorry files are parsed as YAML. If the parser fails, then the lorry files are parsed as JSON. If the YAML parser has advanced the Lorry file's file-descriptor, this can lead to JSON parser errors. This change resets the file descriptor to the start of the file before calling the JSON parser.
* Merge branch 'bwh/single-lorry-metadata' into 'master'Ben Brown2020-08-033-25/+124
|\ | | | | | | | | | | | | Add metadata for single repositories Closes #15 See merge request CodethinkLabs/lorry/lorry-controller!17
| * gitlab: Ignore failure to set the default branchBen Hutchings2020-07-301-3/+12
| | | | | | | | | | | | | | | | | | | | In testing the change to define the default branch for single repositories, I found that GitLab will reject a change of default branch if the branch does not yet exist. (It doesn't seem to do this when creating a repository.) Ignore failure to change the default branch, as we will fix it on the next run after the branch has been created.
| * givemejob: Add metadata for single repositoriesBen Hutchings2020-07-302-3/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When mirroring a repository found through a 'lorries' configuration section (instead of 'trove' or 'gitlab'), we currently don't set a description or default branch. * Set the description to the upstream repository path, but allow this to be overridden by a description field in the .lorry file. Prepend the host-name, just as we do when mirroring an Upstream Host. * Set the default branch to: - Bazaar: 'trunk' - Git: upstream default branch, found using 'git ls-remote' - others: 'master' Closes #15.
| * givemejob: Move upstream host metadata lookup out of get_repo_metadataBen Hutchings2020-07-151-10/+18
| | | | | | | | | | | | In preparation for adding metadata for single repositories, move the code for looking up metadata through the Upstream Host connector into a separate function.
| * givemejob: Stop synthesising host information when it's missingBen Hutchings2020-07-151-10/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Any lorry in the database should have been either: 1. Generated based on a 'trove' or 'gitlab' section, with 'from_host' set to the Upstream Host name. 2. Read from a lorry referred to in a 'lorries' section, with 'from_host' set to an empty string. So this error case indicates a dangling reference in the database. The current schema does not use foreign keys so this could happen, but it's reasonable to just give up and return empty metadata.
* | Merge branch 'bwh/local-empty-description' into 'master'Ben Brown2020-07-304-2/+8
|\ \ | | | | | | | | | | | | | | | | | | local: Create repos with empty description and enabled for export Closes #16 See merge request CodethinkLabs/lorry/lorry-controller!20
| * | local: Create repos with empty description and enabled for exportBen Hutchings2020-07-294-2/+8
|/ / | | | | | | | | | | | | | | | | | | | | | | 'git init' copies files into the repository from a template directory, which contains a 'replace me' description that we don't want. Provide and use our own template directory instead. While we're at it, also add the git-daemon-export-ok flag file that git-daemon and git-http-backend may check for (depending on their configuration). Closes #16.
* | Merge branch 'bwh/minion-multiple-lorry-configs' into 'master'Ben Hutchings2020-07-221-3/+3
|\ \ | | | | | | | | | | | | MINION: Allow specifying multiple configuration files for Lorry See merge request CodethinkLabs/lorry/lorry-controller!19
| * | MINION: Allow specifying multiple configuration files for LorryBen Hutchings2020-07-211-3/+3
|/ / | | | | | | | | Change the 'lorry-config' setting from string to string-list type. This will allow separating generic and site configuration files.
* | Merge branch 'bwh/sensible-disk-space' into 'master'Ben Brown2020-07-172-5/+11
|\ \ | |/ |/| | | | | | | | | Improve disk space formatting Closes #14 See merge request CodethinkLabs/lorry/lorry-controller!18
| * status: Show free disk space in MiB when it's < 1 GiBBen Hutchings2020-07-161-1/+7
| | | | | | | | | | | | Showing free disk space as 0 when it drops below 1 GiB may be unncessarily alarming, so show it in units of MiB if that happens. The status dictionary already includes it in both units.
| * status: Use integer division where appropriateBen Hutchings2020-07-161-4/+4
|/ | | | | | | | | | | | | | | In Python 2, the / operator between two integer-typed values produces another integer. In Python 3, it produces a float. We need to use the // operator to get the old behaviour. The change to floats in get_free_disk_space resulted in unnecessarily precise values for free disk space. The change in formats_secs_nicely doesn't seem to make a difference since we format the quotients with '%d', but I'm making it use // so it's clearer what we want. Closes #14.
* Merge branch 'bwh/doc-names' into 'master'Ben Brown2020-07-154-4/+18
|\ | | | | | | | | Use document names expected by GitLab See merge request CodethinkLabs/lorry/lorry-controller!16
| * Add copyright and license statements to a LICENSE fileBen Hutchings2020-07-151-0/+14
| | | | | | | | This is what git hosting services like GitLab expect.
| * Add .md extension to Markdown documentsBen Hutchings2020-07-153-4/+4
|/ | | | | This will cause them to be rendered on GitLab and other git hosts' web interfaces.
* Merge branch 'bwh/update-docs' into 'master'Ben Hutchings2020-06-142-0/+278
|\ | | | | | | | | | | | | Update documentation of dependencies and installation Closes #2 and #1 See merge request CodethinkLabs/lorry-controller!15
| * INSTALL: Document how to install Lorry and Lorry Controller themselvesBen Hutchings2020-06-091-0/+37
| | | | | | | | | | | | | | Installation is mostly covered by setup.py. However the configuration files, many of the directories, and the systemd commands are not. Related to #2.
| * INSTALL: Briefly document creation of CONFGITBen Hutchings2020-06-091-0/+10
| | | | | | | | | | | | | | This is mostly covered in README, but should probably be mentioned here too. Refer to README for the contents of the repository. Related to #2.
| * INSTALL: Document how to configure a front-end web serverBen Hutchings2020-06-091-0/+22
| | | | | | | | | | | | | | This is mostly a generalisation of the example lighttpd configuration. I haven't yet tried to configure any other web server. Relates to #2.
| * INSTALL: Document how to integrate Lorry/LC with each Downstream Host typeBen Hutchings2020-06-091-0/+140
| | | | | | | | | | | | | | | | | | Document how to configure Gerrit, Gitea, and GitLab for use with Lorry and Lorry Controller, and vice versa. For Gitano/Trove, refer to the Baserock documentation. Relates to #2.
| * INSTALL: Document creation of a user account for Lorry and LCBen Hutchings2020-06-081-0/+8
| | | | | | | | Related to #2.
| * INSTALL: List all the known direct dependencies and how to install themBen Hutchings2020-06-082-0/+61
| | | | | | | | | | | | | | | | Add a new documentation file with a full list of dependencies, stating which of them are optional, how to install each of them, and which versions have been tested. Refer to this from README. Closes #1.
* | Merge branch 'bwh/statedb-no-json1' into 'master'Ben Brown2020-06-141-1/+1
|\ \ | | | | | | | | | | | | STATEDB: Avoid depending on SQLite json1 extension See merge request CodethinkLabs/lorry-controller!13
| * | STATEDB: Avoid depending on SQLite json1 extensionBen Hutchings2020-06-081-1/+1
| |/ | | | | | | | | | | | | | | | | | | The migration to a generalised hosts table uses json_object(), which is part of the optional json1 extension to SQLite. This is not enabled by default, and isn't enabled in the Freedesktop-SDK build of SQLite. Since GitLab tokens shouldn't include double-quotes, we can avoid this by using standard SQL concatenation instead.
* | Merge branch 'bwh/update-copyright' into 'master'Ben Brown2020-06-1411-11/+11
|\ \ | |/ |/| | | | | Update copyright dates for files I recently changed See merge request CodethinkLabs/lorry-controller!14
| * Update copyright dates for files I recently changedBen Hutchings2020-06-0811-11/+11
|/
* Merge branch 'bwh/gitea-support' into 'master'Ben Hutchings2020-06-084-5/+146
|\ | | | | | | | | | | | | Add gitea Downstream Host connector Closes #9 See merge request CodethinkLabs/lorry-controller!12
| * Merge branch 'master' into 'bwh/gitea-support'Ben Hutchings2020-06-082-13/+31
| |\ | |/ |/| | | # Conflicts: # lorry-controller-webapp
* | Merge branch 'bwh/gitlab-config' into 'master'Ben Hutchings2020-06-084-10/+44
|\ \ | | | | | | | | | | | | | | | | | | Improve repo/group configuration on GitLab downstream Closes #6 See merge request CodethinkLabs/lorry-controller!10
| * \ Merge branch 'master' into 'bwh/gitlab-config'Ben Hutchings2020-06-085-35/+97
| |\ \ | |/ / |/| | | | | Resolved conflicts (neighbouring insertions) in lorry-controller-webapp and lorrycontroller/gerrit.py.
* | | Merge branch 'bwh/gerrit-config' into 'master'Ben Hutchings2020-06-081-12/+28
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Improve repo configuration on Gerrit downstream Closes #10, #8, #4, and #7 See merge request CodethinkLabs/lorry-controller!11
| * | | GerritDownstream: Pass default branch and description to create-projectBen Hutchings2020-06-011-14/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Lorry's account on Gerrit might not be permitted to change these attributes later, so include them in the create-project command. Closes #7.
| * | | GerritDownstream: Add application setting for parent projectBen Hutchings2020-06-011-1/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GerritDownstream does not specify a parent project for create-project, so it implicitly creates projects directly under 'All-Projects'. This requires that Lorry's user account is given broad permissions on All-Projects, which seems undesirable. Allow specifying an alternate parent project, so that Lorry can be given permissions on only that project.
* | | | Merge branch 'bwh/remote-downstreams' into 'master'Ben Hutchings2020-06-084-29/+72
|\ \ \ \ | |/ / / | | | | | | | | | | | | | | | | | | | | Allow Gerrit and GitLab Downstream Hosts to be remote Closes #10, #8, and #4 See merge request CodethinkLabs/lorry-controller!9
* | | | Merge branch 'bwh/cleanup-host-types' into 'master'Ben Hutchings2020-06-0811-214/+479
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Clean up and document how host type connectors work Closes #10 and #5 See merge request CodethinkLabs/lorry-controller!7
* \ \ \ \ Merge branch 'bwh/gerrit-ssh-quoting' into 'master'Ben Hutchings2020-06-041-1/+2
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | gerrit: Shell-quote ssh command arguments Closes #8 See merge request CodethinkLabs/lorry-controller!8
* \ \ \ \ \ Merge branch 'bwh/update-gitlab' into 'master'Ben Hutchings2020-06-041-56/+29
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update GitLab support See merge request CodethinkLabs/lorry-controller!6
* \ \ \ \ \ \ Merge branch 'bwh/generalise-statedb' into 'master'Ben Hutchings2020-06-045-44/+101
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | STATEDB: Generalise 'troves' table to allow more Upstream Host types Closes #10 See merge request CodethinkLabs/lorry-controller!5
* \ \ \ \ \ \ \ Merge branch 'bwh/de-trove-ify' into 'master'Ben Hutchings2020-06-0416-246/+253
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generalise references to 'Trove' Closes #3 See merge request CodethinkLabs/lorry-controller!4
* \ \ \ \ \ \ \ \ Merge branch 'bwh/fix-lighttpd-path' into 'master'Ben Hutchings2020-06-041-1/+3
|\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | lighttpd: Fix FastCGI configuration to pass full path as PATH_INFO See merge request CodethinkLabs/lorry-controller!3
| * | | | | | | | | lighttpd: Fix FastCGI configuration to pass full path as PATH_INFOBen Hutchings2020-06-011-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The configuration file uses an empty path, which I assume means that the HTTP server root should map to the LC webapp root. So we want the full path of the request to be passed as PATH_INFO. However, with this configuration lighttpd actually seems to pass the first path component as SCRIPT_NAME and the rest as PATH_INFO. We need to set the path to "/" and enable the fix-root-scriptname option to get the full path in PATH_INFO.
* | | | | | | | | | Merge branch 'bwh/python3' into 'master'Ben Hutchings2020-06-0119-114/+128
|\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update LC to run on Python 3 See merge request CodethinkLabs/lorry-controller!2
* \ \ \ \ \ \ \ \ \ \ Merge branch 'bwh/trovekube' into 'master'Ben Hutchings2020-06-015-6/+34
|\ \ \ \ \ \ \ \ \ \ \ | | |/ / / / / / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes to support Trovekube See merge request CodethinkLabs/lorry-controller!1
| | | | | | | | | | * Add gitea Downstream Host connectorBen Hutchings2020-06-013-2/+142
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement organisation and repository creation using the Gitea REST API v1. Add a 'gitea-access-token' application setting for this. Closes #9.
| | | | | | | | | | * lsupstreams: Prefer more specific remote prefixes in prefixmapBen Hutchings2020-06-011-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to support Downstream Hosts with restricted repository hierarchies (like Gitea's 2 levels), it is useful to be able to map both a directory and a subdirectory of it from an Upstream Host that has a deeper hierarchy. Convert the prefixmap into a list of (remote, local) tuples and sort in reverse order, so that a subdirectory prefix will appear before its parent directory prefix if both are present.
View Lorry Controller Status