doc: add "The go command" article based on Russ' mail

This is a philosophical document. We can soup it up at a later stage,
but for now it's nice to have a URL to point to that isn't a mailing
list post.

R=golang-dev, r
CC=golang-dev
http://codereview.appspot.com/5676061

1 files changed, 265 insertions, 0 deletions

diff --git a/doc/articles/go_command.html b/doc/articles/go_command.html
new file mode 100644
index 000000000..343fcbc6d
--- /dev/null
+++ b/doc/articles/go_command.html
@@ -0,0 +1,265 @@
+<!--{
+	"title": "About the go command"
+}-->
+
+<p>The Go distribution includes a command, named
+"<code><a href="/cmd/go/">go</a></code>", that
+automates the downloading, building, installation, and testing of Go packages
+and commands.  This document talks about why we wrote a new command, what it
+is, what it's not, and how to use it.</p>
+
+<h2>Motivation</h2>
+
+<p>You might have seen early Go talks in which Rob Pike jokes that the idea
+for Go arose while waiting for a large Google server to compile.  That
+really was the motivation for Go: to build a language that worked well
+for building the large software that Google writes and runs. It was
+clear from the start that such a language must provide a way to
+express dependencies between code libraries clearly, hence the package
+grouping and the explicit import blocks.  It was also clear from the
+start that you might want arbitrary syntax for describing the code
+being imported; this is why import paths are string literals.</p>
+
+<p>An explicit goal for Go from the beginning was to be able to build Go
+code using only the information found in the source itself, not
+needing to write a makefile or one of the many modern replacements for
+makefiles.  If Go needed a configuration file to explain how to build
+your program, then Go would have failed.</p>
+
+<p>At first, there was no Go compiler, and the initial development
+focused on building one and then building libraries for it. For
+expedience, we postponed the automation of building Go code by using
+make and writing makefiles.  When compiling a single package involved
+multiple invocations of the Go compiler, we even used a program to
+write the makefiles for us.  You can find it if you dig through the
+repository history.</p>
+
+<p>The purpose of the new go command is our return to this ideal, that Go
+programs should compile without configuration or additional effort on
+the part of the developer beyond writing the necessary import
+statements.</p>
+
+<h2>Configuration versus convention</h2>
+
+<p>The way to achieve the simplicity of a configuration-free system is to
+establish conventions. The system works only to the extent that the convention
+is followed. When we first launched Go, many people published packages that
+had to be installed in certain places, under certain names, using certain build
+tools, in order to be used. That's understandable: that's the way it works in
+most other languages. Over the last few years we consistently reminded people
+about the old <code>goinstall</code> command
+(now replaced by <a href="/cmd/go/#Download_and_install_packages_and_dependencies"><code>go get</code></a>)
+and its conventions: first, that the import path is derived in a known way from
+the URL of the source code; second, that that the place to store the sources in
+the local file system is derived in a known way from the import path; third,
+that each directory in a source tree corresponds to a single package; and
+fourth, that the package is built using only information in the source code.
+Today, the vast majority of packages follow these conventions.
+The Go ecosystem is simpler and more powerful for it.</p>
+
+<p>We received many requests to allow a makefile in a package directory to
+provide just a little extra configuration beyond what's in the source code.
+But that would have introduced new rules. Because we did not accede to such
+requests, we were able to write the go command and eliminate our use of make
+or any other build system.</p>
+
+<p>It is important to understand that the go command is not a general
+build tool. It cannot be configured and it does not attempt to build
+anything but Go packages.  These are important simplifying
+assumptions: they simplify not only the implementation but also, more
+important, the use of the tool itself.</p>
+
+<h2>Go's conventions</h2>
+
+<p>The <code>go</code> command requires that code adheres to a few key,
+well-established conventions.</p>
+
+<p>First, the import path is derived in an known way from the URL of the
+source code.  For Bitbucket, GitHub, Google Code, and Launchpad, the
+root directory of the repository is identified by the repository's
+main URL, without the <code>http://</code> prefix.  Subdirectories are named by
+adding to that path.  For example, the supplemental networking
+libraries for Go are obtained by running</p>
+
+<pre>
+hg clone http://code.google.com/p/go.net
+</pre>
+
+<p>and thus the import path for the root directory of that repository is
+"<code>code.google.com/p/go.net</code>".  The websocket package is stored in a
+subdirectory, so its import path is
+"<code>code.google.com/p/go.net/websocket</code>".</p>
+
+<p>These paths are on the long side, but in exchange we get an
+automatically managed name space for import paths and the ability for
+a tool like the go command to look at an unfamiliar import path and
+deduce where to obtain the source code.</p>
+
+<p>Second, the place to store sources in the local file system is derived
+in a known way from the import path.  Specifically, the first choice
+is <code>$GOPATH/src/&lt;import-path&gt;</code>.  If <code>$GOPATH</code> is
+unset, the go command will fall back to storing source code alongside the
+standard Go packages, in <code>$GOROOT/src/pkg/&lt;import-path&gt;</code>.
+If <code>$GOPATH</code> is set to a list of paths, the go command tries
+<code>&lt;dir&gt;/src/&lt;import-path&gt;</code> for each of the directories in
+that list.</p>
+
+<p>Each of those trees contains, by convention, a top-level directory named
+"<code>bin</code>", for holding compiled executables, and a top-level directory
+named "<code>pkg</code>", for holding compiled packages that can be imported,
+and the "<code>src</code>" directory, for holding package source files.
+Imposing this structure lets us keep each of these directory trees
+self-contained: the compiled form and the sources are always near each
+other.</p>
+
+<p>These naming conventions also let us work in the reverse direction,
+from a directory name to its import path. This mapping is important
+for many of the go command's subcommands, as we'll see below.</p>
+
+<p>Third, each directory in a source tree corresponds to a single
+package. By restricting a directory to a single package, we don't have
+to create hybrid import paths that specify first the directory and
+then the package within that directory.  Also, most file management
+tools and UIs work on  directories as fundamental units.  Tying the
+fundamental Go unit&mdash;the package&mdash;to file system structure means
+that file system tools become Go package tools.  Copying, moving, or
+deleting a package corresponds to copying, moving, or deleting a
+directory.</p>
+
+<p>Fourth, each package is built using only the information present in
+the source files.  This makes it much more likely that the tool will
+be able to adapt to changing build environments and conditions. For
+example, if we allowed extra configuration such as compiler flags or
+command line recipes, then that configuration would need to be updated
+each time the build tools changed; it would also be inherently tied
+to the use of a specific tool chain.</p>
+
+<h2>Getting started with the go command</h2>
+
+<p>Finally, a quick tour of how to use the go command, to supplement
+the information in <a href="/doc/code.html">How to Write Go Code</a>,
+which you might want to read first.  Assuming you want
+to keep your source code separate from the Go distribution source
+tree, the first step is to set <code>$GOPATH</code>, the one piece of global
+configuration that the go command needs.  The <code>$GOPATH</code> can be a
+list of directories, but by far the most common usage should be to set it to a
+single directory.  In particular, you do not need a separate entry in
+<code>$GOPATH</code> for each of your projects.  One <code>$GOPATH</code> can
+support many projects.</p>
+
+<p>Here’s an example.  Let’s say we decide to keep our Go code in the directory
+<code>$HOME/mygo</code>.  We need to create that directory and set
+<code>$GOPATH</code> accordingly.</p>
+
+<pre>
+$ mkdir $HOME/mygo
+$ export GOPATH=$HOME/mygo
+$
+</pre>
+
+<p>Into this directory, we now add some source code.  Suppose we want to use
+the indexing library from the codesearch project along with a left-leaning
+red-black tree.  We can install both with the "<code>go get</code>"
+subcommand:</p>
+
+<pre>
+$ go get code.google.com/p/codesearch/index
+$ go get github.com/petar/GoLLRB/llrb
+$
+</pre>
+
+<p>Both of these projects are now downloaded and installed into our
+<code>$GOPATH</code> directory. The one tree now contains the two directories
+<code>src/code.google.com/p/codesearch/index/</code> and
+<code>src/github.com/petar/GoLLRB/llrb/</code>, along with the compiled
+packages (in <code>pkg/</code>) for those libraries and their dependencies.</p>
+
+<p>Because we used version control systems (Mercurial and Git) to check
+out the sources, the source tree also contains the other files in the
+corresponding repositories, such as related packages. The "<code>go list</code>"
+subcommand lists the import paths corresponding to its arguments, and
+the pattern "<code>./...</code>" means start in the current directory
+("<code>./</code>") and find all packages below that directory
+("<code>...</code>"):</p>
+
+<pre>
+$ go list ./...
+code.google.com/p/codesearch/cmd/cgrep
+code.google.com/p/codesearch/cmd/cindex
+code.google.com/p/codesearch/cmd/csearch
+code.google.com/p/codesearch/index
+code.google.com/p/codesearch/regexp
+code.google.com/p/codesearch/sparse
+github.com/petar/GoLLRB/example
+github.com/petar/GoLLRB/llrb
+$
+</pre>
+
+<p>We can also test those packages:</p>
+
+<pre>
+$ go test ./...
+?       code.google.com/p/codesearch/cmd/cgrep   [no test files]
+?       code.google.com/p/codesearch/cmd/cindex  [no test files]
+?       code.google.com/p/codesearch/cmd/csearch [no test files]
+ok      code.google.com/p/codesearch/index       0.239s
+ok      code.google.com/p/codesearch/regexp      0.021s
+?       code.google.com/p/codesearch/sparse      [no test files]
+?       github.com/petar/GoLLRB/example          [no test files]
+ok      github.com/petar/GoLLRB/llrb             0.231s
+$
+</pre>
+
+<p>If a go subcommand is invoked with no paths listed, it operates on the
+current directory:</p>
+
+<pre>
+$ cd $GOPATH/src/code.google.com/p/codesearch/regexp
+$ go list
+code.google.com/p/codesearch/regexp
+$ go test -v
+=== RUN TestNstateEnc
+--- PASS: TestNstateEnc (0.00 seconds)
+=== RUN TestMatch
+--- PASS: TestMatch (0.01 seconds)
+=== RUN TestGrep
+--- PASS: TestGrep (0.00 seconds)
+PASS
+ok      code.google.com/p/codesearch/regexp     0.021s
+$ go install
+$
+</pre>
+
+<p>That "<code>go install</code>" subcommand installs the latest copy of the
+package into the pkg directory. Because the go command can analyze the
+dependency graph, "<code>go install</code>" also installs any packages that
+this package imports but that are out of date, recursively.</p>
+
+<p>Notice that "<code>go install</code>" was able to determine the name of the
+import path for the package in the current directory, because of the convention
+for directory naming.  It would be a little more convenient if we could pick
+the name of the directory where we kept source code, and we probably wouldn't
+pick such a long name, but that ability would require additional configuration
+and complexity in the tool. Typing an extra directory name or two is a small
+price to pay for the increased simplicity and power.</p>
+
+<p>As the example shows, it’s fine to work with packages from many different
+projects at once within a single <code>$GOPATH</code> root directory.</p>
+
+<h2>Limitations</h2>
+
+<p>As mentioned above, the go command is not a general-purpose build
+tool. In particular, it does not have any facility for generating Go
+source files during a build.  Instead, if you want to use a tool like
+yacc or the protocol buffer compiler, you will need to write a
+makefile (or a configuration file for the build tool of your choice)
+to generate the Go files and then check those generated source files
+into your repository. This is more work for you, the package author,
+but it is significantly less work for your users, who can use
+"<code>go get</code>" without needing to obtain and build
+any additional tools.</p>
+
+<h2>More information</h2>
+
+<p>For more information, read <a href="/doc/code.html">How to Write Go Code</a>
+and see the <a href="/cmd/go/">go command documentation</a>.</p>