README: Rename to README.md

This will cause it to be rendered on GitLab and other git hosts'
web interfaces.

1 files changed, 320 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e46408a
--- /dev/null
+++ b/README.md
@@ -0,0 +1,320 @@
+README for Lorry
+================
+
+Lorry is a tool to take upstream source code (in various formats,
+though preferably in version control) and converts it into a git
+repository.
+
+If you want to try this, use `--pull-only` and/or `--mirror-base-url-push`
+so that you do not accidentally overwrite important stuff for Baserock.
+(If you don't have direct commit access to Baserock on git.baserock.org
+then you're not dangerous.)
+
+See the manual page for instructions on using.
+
+The usual use is that a server hosts lorry and checks for updates,
+which it then lorries. Lorry should not generally be run from the
+developer's machine apart from testing. This is because Lorry has to
+keep the git trees to avoid it having to pull everything.
+
+You can find a lot of lorries to crib ideas from at
+<https://git.baserock.org/cgit/baserock/local-config/lorries.git/>.
+
+
+Dependencies
+------------
+
+Required:
+
+* **Python 3**
+
+* **Git**
+
+* **cliapp**: Can be installed as the `python3-cliapp` package in
+  Debian, or with:
+
+        pip3 install https://gitlab.com/trovekube/cliapp/-/archive/cliapp-1.20180812.1/cliapp-cliapp-1.20180812.1.tar.gz
+
+  or from the source at <https://liw.fi/cliapp/>.
+
+* **PyYAML**: Can be installed from PyPI. or as the `python3-yaml`
+  package in Debian.
+
+Optional:
+
+* **bzr-fastimport** or **Breezy**: Needed if you want to import
+  Bazaar (bzr) repositories.  Can be installed as the `bzr-fastimport`
+  or `brz` package in Debian.
+
+* **cmdtest**: Needed if you want to run the test suite.  Can be
+  installed as the `cmdtest` package in Debian, or from the source at
+  <https://liw.fi/cmdtest/>.
+
+* **git-cvsimport**: Needed if you want to import CVS repositories.
+  Can be installed as the `git-cvs` package in Debian.
+
+* **git-svn**: Needed if you want to import Subversion (svn)
+  repositories.  Can be installed as the `git-svn` package in Debian.
+
+* **hg-fast-export**: Needed if you want to import Mercurial (hg)
+  repositories.  Can be installed from the source at
+  <https://repo.or.cz/w/fast-export.git>.
+
+* **OpenSSH**: The OpenSSH client is needed if you want to import or
+  push to repositories using the SSH protocol.
+
+* **Perl**: Needed if you want to import tarballs.
+
+Lorry file specification
+------------------------
+
+Lorry files are json dicts where the repository names are the keys and the
+values are dicts with the data required to mirror it.
+
+So a simple lorry that mirrors a git project looks like
+
+    {
+        "git": {
+            "type": "git",
+            "url": "git://github.com/gitster/git.git"
+        }
+    }
+
+Multiple repositories can be specified in the same .lorry file, in which case
+all of them will be processed by lorry. The following shows two repositories.
+
+    {
+        "git": {
+            "type": "git",
+            "url": "git://github.com/gitster/git.git"
+        },
+        "curl": {
+            "type": "git",
+            "url": "git://github.com/bagder/curl.git"
+        }
+    }
+
+Lorry can import other version control systems into git.
+
+### Mercurial
+Mercurial is very similar to git, just change the type field to "hg"
+
+    {
+        "sudo": {
+            "type": "hg",
+            "url": "http://www.sudo.ws/repos/sudo"
+        }
+    }
+
+### Bazaar
+Repositories and branches in Bazaar mean different things to Git.
+The practical difference for Lorry is that it is not possible to have
+a url for a repository, urls map directly to branches.
+
+    {
+        "libpipeline": {
+            "type": "bzr",
+            "branches": {
+                "trunk": "http://bzr.savannah.gnu.org/r/libpipeline/trunk"
+            }
+        }
+    }
+
+For convenience if the project only needs one branch mirrored, the url
+is assumed to be the master branch.
+
+    {
+        "libpipeline": {
+            "type": "bzr",
+            "url": "http://bzr.savannah.gnu.org/r/libpipeline/trunk"
+        }
+    }
+
+### Subversion
+To support all the branches and tags a layout needs to be specified as svn is
+very flexible with the possible layouts, however the most common is to have the
+working branch in a directory called trunk, and the branches and tags in 
+respectively named subdirectories.
+Because this is so common "standard" can be used as the layout
+
+    {
+        "mpc": {
+            "type": "svn",
+            "url": "svn://scm.gforge.inria.fr/svn/mpc",
+            "layout": "standard"
+        }
+    }
+
+This is equivalent to
+
+    {
+        "mpc": {
+            "type": "svn",
+            "url": "svn://scm.gforge.inria.fr/svn/mpc",
+            "layout": {
+                "trunk": "trunk",
+                "branches": "branches/*",
+                "tags": "tags/*"
+            }
+        }
+    }
+
+Trunk is the path to the directory where the main branch is located.
+Branches and Tags are glob expressions to allow finer control over which paths
+are used.
+Trunk is mandatory, but Branches and Tags are optional.
+Texlive keeps a lot of resources in their svn repository, we are only concerned
+with the source code, so this layout should select the correct subdirectory for
+each branch.
+
+    {
+        "texlive": {
+            "type": "svn",
+            "url": "svn://tug.org/texlive/",
+            "layout": {
+                "trunk": "trunk/Build/source",
+                "branches": "branches/*/Build/source",
+                "tags": "tags/*/Build/source"
+            }
+        }
+    }
+
+Brace expansions can be used to specify subsets of paths.
+Netpbm for example, keeps all its branches in the root directory
+
+    {
+        "netpbm": {
+            "type": "svn",
+            "url": "https://netpbm.svn.sourceforge.net/svnroot/netpbm",
+            "layout": {
+                "trunk": "trunk",
+                "branches": "{advanced,stable,super_stable}",
+                "tags": "release_number/*"
+            }
+        }
+    }
+
+Note that git-svn can provide better history tracking if the url is as close to
+the root of the repository as possible, so it may be more effective if the lorry
+was specified similar to this, assuming svnroot is the real root of the repo
+
+    {
+        "netpbm": {
+            "type": "svn",
+            "url": "https://netpbm.svn.sourceforge.net/svnroot/",
+            "layout": {
+                "trunk": "netpbm/trunk",
+                "branches": "netpbm/{advanced,stable,super_stable}",
+                "tags": "netpbm/release_number/*"
+            }
+        }
+    }
+
+### CVS
+The url for CVS repositories is the CVSROOT string. The module is required as
+cvs repositories usually have multiple modules, the module is usually the same
+as the project name.
+
+    {
+        "openssl": {
+            "type": "cvs",
+            "url": "anonymous@cvs.openssl.org:/openssl-cvs",
+            "module": "openssl"
+        }
+    }
+
+### Tarball
+
+Lorry can import a tarball fetched from a URL. The contents will be
+committed on a branch named after the basename of the tar file (e.g.
+bc-1.06.tar.gz will be imported into a branch named 'bc-1.06'.
+
+The import is done by the `lorry.tar-importer` subprocess. It can detect and
+handle common compression formats including gzip, bz2, xz and lzma. It will
+also detect if there is a 'top directory' that contains the tarball contents
+and strip this out of the imported filenames.
+
+Tarball imports once required 'compression' and 'strip' to be specified.
+These are obsolete now and are ignored by Lorry.
+
+    {
+        "bc": {
+            "type": "tarball",
+            "url": "http://ftp.gnu.org/gnu/bc/bc-1.06.tar.gz"
+        }
+    }
+
+### Zip
+
+Lorry can import a zip file fetched from a URL. The contents will be
+extracted in place and committed directly to master. The new commit will
+be tagged with the basename of the imported zip file (e.g. docbook-xml-4.5.zip
+will be tagged as 'docbook-xml-4.5')
+
+    {
+        "docbook-xml": {
+            "type": "zip",
+            "url": "http://www.docbook.org/xml/4.5/docbook-xml-4.5.zip"
+        }
+    }
+
+
+### Gzip
+
+Lorry can import a gzip file fetched from a URL. The file will be extracted in place
+and committed directly to master after removing the .gz extension from the filename.
+
+The new commit will be tagged with the basename of the imported zip file (e.g. bkai00mp.ttf.gz
+will be tagged as 'bkai00mp.ttf')
+
+    {
+        "ttf-bkai00mp": {
+            "type": "gzip",
+            "url": "ftp://ftp.gnu.org/non-gnu/chinese-fonts-truetype/bkai00mp.ttf.gz"
+        }
+    }
+
+
+Tips
+----
+
+1.  Use upstream's git repository whenever possible
+
+    Importing from foreign version control systems is always slower than a
+    git mirror (with the exception of tarballs because they have less history).
+
+2.  GNU Projects often have a git repository
+
+    Most GNU projects are old compared to git, so were mainly developed in CVS.
+    Many official websites only mention the CVS or SVN repositories.
+    They will tend to have a git repository as well though, especially if they
+    are hosted on savannah.
+
+Legal stuff
+-----------
+
+Copyright (C) 2013-2020  Codethink Limited
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; version 2 of the License.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along
+with this program; if not, write to the Free Software Foundation, Inc.,
+51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+We carry 2 slightly modified git-fast-import frontends:
+
+lorry.tar-importer which is based on
+https://github.com/git/git/blob/master/contrib/fast-import/import-tars.perl
+
+and lorry.zip-importer, which is based on
+https://github.com/git/git/blob/master/contrib/fast-import/import-zips.py
+
+all parts of git are under a GPLv2 compatible license,
+see the git source code for more details.