summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* builtin/receive-pack: fix incorrect pointer arithmeticbc/push-cert-receive-fixbrian m. carlson2017-03-281-1/+1
| | | | | | | | | | | | | | | | | If we had already processed the last newline in a push certificate, we would end up subtracting NULL from the end-of-certificate pointer when computing the length of the line. This would have resulted in an absurdly large length, and possibly a buffer overflow. Instead, subtract the beginning-of-certificate pointer from the end-of-certificate pointer, which is what's expected. Note that this situation should never occur, since not only do we require the certificate to be newline terminated, but the signature will only be read from the beginning of a line. Nevertheless, it seems prudent to correct it. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack::hmac_sha1(): copy the entire SHA-1 hash outBrian Gernhardt2014-09-251-2/+2
| | | | | | | | | | | | | | | clang gives the following warning: builtin/receive-pack.c:327:35: error: sizeof on array function parameter will return size of 'unsigned char *' instead of 'unsigned char [20]' [-Werror,-Wsizeof-array-argument] git_SHA1_Update(&ctx, out, sizeof(out)); ^ builtin/receive-pack.c:292:37: note: declared here static void hmac_sha1(unsigned char out[20], ^ Signed-off-by: Brian Gernhardt <brian@gernhardtsoftware.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* signed push: allow stale nonce in stateless modeJunio C Hamano2014-09-174-12/+112
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When operating with the stateless RPC mode, we will receive a nonce issued by another instance of us that advertised our capability and refs some time ago. Update the logic to check received nonce to detect this case, compute how much time has passed since the nonce was issued and report the status with a new environment variable GIT_PUSH_CERT_NONCE_SLOP to the hooks. GIT_PUSH_CERT_NONCE_STATUS will report "SLOP" in such a case. The hooks are free to decide how large a slop it is willing to accept. Strictly speaking, the "nonce" is not really a "nonce" anymore in the stateless RPC mode, as it will happily take any "nonce" issued by it (which is protected by HMAC and its secret key) as long as it is fresh enough. The degree of this security degradation, relative to the native protocol, is about the same as the "we make sure that the 'git push' decided to update our refs with new objects based on the freshest observation of our refs by making sure the values they claim the original value of the refs they ask us to update exactly match the current state" security is loosened to accomodate the stateless RPC mode in the existing code without this series, so there is no need for those who are already using smart HTTP to push to their repositories to be alarmed any more than they already are. In addition, the server operator can set receive.certnonceslop configuration variable to specify how stale a nonce can be (in seconds). When this variable is set, and if the nonce received in the certificate that passes the HMAC check was less than that many seconds old, hooks are given "OK" in GIT_PUSH_CERT_NONCE_STATUS (instead of "SLOP") and the received nonce value is given in GIT_PUSH_CERT_NONCE, which makes it easier for a simple-minded hook to check if the certificate we received is recent enough. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* signed push: teach smart-HTTP to pass "git push --signed" aroundJunio C Hamano2014-09-176-3/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "--signed" option received by "git push" is first passed to the transport layer, which the native transport directly uses to notice that a push certificate needs to be sent. When the transport-helper is involved, however, the option needs to be told to the helper with set_helper_option(), and the helper needs to take necessary action. For the smart-HTTP helper, the "necessary action" involves spawning the "git send-pack" subprocess with the "--signed" option. Once the above all gets wired in, the smart-HTTP transport now can use the push certificate mechanism to authenticate its pushes. Add a test that is modeled after tests for the native transport in t5534-push-signed.sh to t5541-http-push-smart.sh. Update the test Apache configuration to pass GNUPGHOME environment variable through. As PassEnv would trigger warnings for an environment variable that is not set, export it from test-lib.sh set to a harmless value when GnuPG is not being used in the tests. Note that the added test is deliberately loose and does not check the nonce in this step. This is because the stateless RPC mode is inevitably flaky and a nonce that comes back in the actual push processing is one issued by a different process; if the two interactions with the server crossed a second boundary, the nonces will not match and such a check will fail. A later patch in the series will work around this shortcoming. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* signed push: fortify against replay attacksJunio C Hamano2014-09-177-29/+187
| | | | | | | | | | | | | | | | | | | | | | In order to prevent a valid push certificate for pushing into an repository from getting replayed in a different push operation, send a nonce string from the receive-pack process and have the signer include it in the push certificate. The receiving end uses an HMAC hash of the path to the repository it serves and the current time stamp, hashed with a secret seed (the secret seed does not have to be per-repository but can be defined in /etc/gitconfig) to generate the nonce, in order to ensure that a random third party cannot forge a nonce that looks like it originated from it. The original nonce is exported as GIT_PUSH_CERT_NONCE for the hooks to examine and match against the value on the "nonce" header in the certificate to notice a replay, but returned "nonce" header in the push certificate is examined by receive-pack and the result is exported as GIT_PUSH_CERT_NONCE_STATUS, whose value would be "OK" if the nonce recorded in the certificate matches what we expect, so that the hooks can more easily check. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* signed push: add "pushee" header to push certificateJunio C Hamano2014-09-154-0/+13
| | | | | | | | | | | | | | | | Record the URL of the intended recipient for a push (after anonymizing it if it has authentication material) on a new "pushee URL" header. Because the networking configuration (SSH-tunnels, proxies, etc.) on the pushing user's side varies, the receiving repository may not know the single canonical URL all the pushing users would refer it as (besides, many sites allow pushing over ssh://host/path and https://host/path protocols to the same repository but with different local part of the path). So this value may not be reliably used for replay-attack prevention purposes, but this will still serve as a human readable hint to identify the repository the certificate refers to. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* signed push: remove duplicated protocol infoJunio C Hamano2014-09-154-4/+69
| | | | | | | | | | | | | | | | | | With the interim protocol, we used to send the update commands even though we already send a signed copy of the same information when push certificate is in use. Update the send-pack/receive-pack pair not to do so. The notable thing on the receive-pack side is that it makes sure that there is no command sent over the traditional protocol packet outside the push certificate. Otherwise a pusher can claim to be pushing one set of ref updates in the signed certificate while issuing commands to update unrelated refs, and such an update will evade later audits. Finally, start documenting the protocol. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: send feature request on push-cert packetJunio C Hamano2014-09-152-5/+21
| | | | | | | | | | | | | | | | | | | We would want to update the interim protocol so that we do not send the usual update commands when the push certificate feature is in use, as the same information is in the certificate. Once that happens, the push-cert packet may become the only protocol command, but then there is no packet to put the feature request behind, like we always did. As we have prepared the receiving end that understands the push-cert feature to accept the feature request on the first protocol packet (other than "shallow ", which was an unfortunate historical mistake that has to come before everything else), we can give the feature request on the push-cert packet instead of the first update protocol packet, in preparation for the next step to actually update to the final protocol. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: GPG-validate push certificatesJunio C Hamano2014-09-153-7/+66
| | | | | | | | | | | | Reusing the GPG signature check helpers we already have, verify the signature in receive-pack and give the results to the hooks via GIT_PUSH_CERT_{SIGNER,KEY,STATUS} environment variables. Policy decisions, such as accepting or rejecting a good signature by a key that is not fully trusted, is left to the hook and kept outside of the core. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* push: the beginning of "git push --signed"Junio C Hamano2014-09-1510-2/+253
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While signed tags and commits assert that the objects thusly signed came from you, who signed these objects, there is not a good way to assert that you wanted to have a particular object at the tip of a particular branch. My signing v2.0.1 tag only means I want to call the version v2.0.1, and it does not mean I want to push it out to my 'master' branch---it is likely that I only want it in 'maint', so the signature on the object alone is insufficient. The only assurance to you that 'maint' points at what I wanted to place there comes from your trust on the hosting site and my authentication with it, which cannot easily audited later. Introduce a mechanism that allows you to sign a "push certificate" (for the lack of better name) every time you push, asserting that what object you are pushing to update which ref that used to point at what other object. Think of it as a cryptographic protection for ref updates, similar to signed tags/commits but working on an orthogonal axis. The basic flow based on this mechanism goes like this: 1. You push out your work with "git push --signed". 2. The sending side learns where the remote refs are as usual, together with what protocol extension the receiving end supports. If the receiving end does not advertise the protocol extension "push-cert", an attempt to "git push --signed" fails. Otherwise, a text file, that looks like the following, is prepared in core: certificate version 0.1 pusher Junio C Hamano <gitster@pobox.com> 1315427886 -0700 7339ca65... 21580ecb... refs/heads/master 3793ac56... 12850bec... refs/heads/next The file begins with a few header lines, which may grow as we gain more experience. The 'pusher' header records the name of the signer (the value of user.signingkey configuration variable, falling back to GIT_COMMITTER_{NAME|EMAIL}) and the time of the certificate generation. After the header, a blank line follows, followed by a copy of the protocol message lines. Each line shows the old and the new object name at the tip of the ref this push tries to update, in the way identical to how the underlying "git push" protocol exchange tells the ref updates to the receiving end (by recording the "old" object name, the push certificate also protects against replaying). It is expected that new command packet types other than the old-new-refname kind will be included in push certificate in the same way as would appear in the plain vanilla command packets in unsigned pushes. The user then is asked to sign this push certificate using GPG, formatted in a way similar to how signed tag objects are signed, and the result is sent to the other side (i.e. receive-pack). In the protocol exchange, this step comes immediately before the sender tells what the result of the push should be, which in turn comes before it sends the pack data. 3. When the receiving end sees a push certificate, the certificate is written out as a blob. The pre-receive hook can learn about the certificate by checking GIT_PUSH_CERT environment variable, which, if present, tells the object name of this blob, and make the decision to allow or reject this push. Additionally, the post-receive hook can also look at the certificate, which may be a good place to log all the received certificates for later audits. Because a push certificate carry the same information as the usual command packets in the protocol exchange, we can omit the latter when a push certificate is in use and reduce the protocol overhead. This however is not included in this patch to make it easier to review (in other words, the series at this step should never be released without the remainder of the series, as it implements an interim protocol that will be incompatible with the final one). As such, the documentation update for the protocol is left out of this step. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* pack-protocol doc: typofix for PKT-LINEJunio C Hamano2014-09-151-2/+2
| | | | | | | | | Everywhere else we use PKT-LINE to denote the pkt-line formatted data, but "shallow/deepen" messages are described with PKT_LINE(). Fix them. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* gpg-interface: move parse_signature() to where it should beJunio C Hamano2014-09-154-21/+22
| | | | | | | | | | | | | | | | | | | Our signed-tag objects set the standard format used by Git to store GPG-signed payload (i.e. the payload followed by its detached signature) [*1*], and it made sense to have a helper to find the boundary between the payload and its signature in tag.c back then. Newer code added later to parse other kinds of objects that learned to use the same format to store GPG-signed payload (e.g. signed commits), however, kept using the helper from the same location. Move it to gpg-interface; the helper is no longer about signed tag, but it is how our code and data interact with GPG. [Reference] *1* http://thread.gmane.org/gmane.linux.kernel/297998/focus=1383 Signed-off-by: Junio C Hamano <gitster@pobox.com>
* gpg-interface: move parse_gpg_output() to where it should beJunio C Hamano2014-09-153-41/+47
| | | | | | | | | | | | | | | | Earlier, ffb6d7d5 (Move commit GPG signature verification to commit.c, 2013-03-31) moved this helper that used to be in pretty.c (i.e. the output code path) to commit.c for better reusability. It was a good first step in the right direction, but still suffers from a myopic view that commits will be the only thing we would ever want to sign---we would actually want to be able to reuse it even wider. The function interprets what GPG said; gpg-interface is obviously a better place. Move it there. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: clarify that cmds_sent is a booleanJunio C Hamano2014-09-151-3/+4
| | | | | | | | | | | | | | We use it to make sure that the feature request is sent only once on the very first request packet (ignoring the "shallow " line, which was an unfortunate mistake we cannot retroactively fix with existing receive-pack already deployed in the field) and we set it to "true" with cmds_sent++, not because we care about the actual number of updates sent but because it is merely an idiomatic way. Set it explicitly to one to clarify that the code that uses this variable only cares about its zero-ness. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: refactor inspecting and resetting status and sending commandsJunio C Hamano2014-09-151-19/+30
| | | | | | | | | | | | | | | | The main loop over remote_refs list inspects the ref status to see if we need to generate pack data (i.e. a delete-only push does not need to send any additional data), resets it to "expecting the status report" state, and formats the actual update commands to be sent. Split the former two out of the main loop, as it will become conditional in later steps. Besides, we should have code that does real thing here, before the "Finally, tell the other end!" part ;-) Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: rename "new_refs" to "need_pack_data"Junio C Hamano2014-09-151-4/+3
| | | | | | | | The variable counts how many non-deleting command is being sent, but is only checked with 0-ness to decide if we need to send the pack data. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: factor out capability string generationJunio C Hamano2014-09-151-8/+14
| | | | | | | Similar to the previous one for send-pack, make it easier and cleaner to add to capability advertisement. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: factor out capability string generationJunio C Hamano2014-09-151-8/+13
| | | | | | | | | | | A run of 'var ? " var" : ""' fed to a long printf string in a deeply nested block was hard to read. Move it outside the loop and format it into a strbuf. As an added bonus, the trick to add "agent=<agent-name>" by using two conditionals is replaced by a more readable version. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: always send capabilitiesJunio C Hamano2014-09-151-3/+1
| | | | | | | | We tried to avoid sending one extra byte, NUL and nothing behind it to signal there is no protocol capabilities being sent, on the first command packet on the wire, but it just made the code look ugly. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: refactor decision to send update per refJunio C Hamano2014-09-151-15/+21
| | | | | | | | A new helper function ref_update_to_be_sent() decides for each ref if the update is to be sent based on the status previously set by set_ref_status_for_push() and also if this is a mirrored push. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* send-pack: move REF_STATUS_REJECT_NODELETE logic a bit higherJunio C Hamano2014-09-151-5/+10
| | | | | | | | | | | | | | | | 20e8b465 (refactor ref status logic for pushing, 2010-01-08) restructured the code to set status for each ref to be pushed, but did not quite go far enough. We inspect the status set earlier by set_refs_status_for_push() and then perform yet another update to the status of a ref with an otherwise OK status to be deleted to mark it with REF_STATUS_REJECT_NODELETE when the protocol tells us never to delete. Split the latter into a separate loop that comes before we enter the per-ref loop. This way we would have one less condition to check in the main loop. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: factor out queueing of commandJunio C Hamano2014-09-151-21/+29
| | | | | | | Make a helper function to accept a line of a protocol message and queue an update command out of the code from read_head_info(). Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: do not reuse old_sha1[] for other thingsJunio C Hamano2014-09-151-3/+5
| | | | | | | | This piece of code reads object names of shallow boundaries, not old_sha1[], i.e. the current value the ref points at, which is to be replaced by what is in new_sha1[]. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: parse feature request a bit earlierJunio C Hamano2014-09-151-12/+14
| | | | | | | | | Ideally, we should have also allowed the first "shallow" to carry the feature request trailer, but that is water under the bridge now. This makes the next step to factor out the queuing of commands easier to review. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* receive-pack: do not overallocate command structureJunio C Hamano2014-09-151-2/+3
| | | | | | | | | | | | An "update" command in the protocol exchange consists of 40-hex old object name, SP, 40-hex new object name, SP, and a refname, but the first instance is further followed by a NUL with feature requests. The command structure, which has a flex-array member that stores the refname at the end, was allocated based on the whole length of the update command, without excluding the trailing feature requests. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Sync with 2.0.4Junio C Hamano2014-07-303-1/+8
|\ | | | | | | | | | | * maint: Git 2.0.4 commit --amend: test specifies authorship but forgets to check
| * Git 2.0.4v2.0.4Junio C Hamano2014-07-304-3/+9
| | | | | | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * commit --amend: test specifies authorship but forgets to checkFabian Ruch2014-07-301-0/+1
| | | | | | | | | | | | | | | | | | | | The test case "--amend option copies authorship" specifies that the git-commit option `--amend` uses the authorship of the replaced commit for the new commit. Add the omitted check that this property actually holds. Signed-off-by: Fabian Ruch <bafain@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Update draft release notes to 2.1Junio C Hamano2014-07-301-3/+5
| | | | | | | | Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'jk/more-push-completion'Junio C Hamano2014-07-301-1/+27
|\ \ | | | | | | | | | | | | | | | | | | * jk/more-push-completion: completion: complete `git push --force-with-lease=` completion: add some missing options to `git push` completion: complete "unstuck" `git push --recurse-submodules`
| * | completion: complete `git push --force-with-lease=`jk/more-push-completionJohn Keeping2014-07-221-1/+21
| | | | | | | | | | | | | | | Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | completion: add some missing options to `git push`John Keeping2014-07-221-0/+1
| | | | | | | | | | | | | | | Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | completion: complete "unstuck" `git push --recurse-submodules`John Keeping2014-07-221-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | Since the argument to `--recurse-submodules` is mandatory, it does not need to be stuck to the option with `=`. Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | Merge branch 'sk/mingw-tests-workaround'Junio C Hamano2014-07-308-24/+51
|\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make tests pass on msysgit by mostly disabling ones that are infeasible on that platform. * sk/mingw-tests-workaround: t800[12]: work around MSys limitation t9902: mingw-specific fix for gitfile link files t4210: skip command-line encoding tests on mingw MinGW: disable legacy encoding tests t0110/MinGW: skip tests that pass arbitrary bytes on the command line MinGW: Skip test redirecting to fd 4
| * | | t800[12]: work around MSys limitationsk/mingw-tests-workaroundKarsten Blees2014-07-211-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | MSys works very hard to convert Unix-style paths into DOS-style ones. *Very* hard. So hard, indeed, that git blame -L/hello/,/green/ is translated into something like git blame -LC:/msysgit/hello/,C:/msysgit/green/ As seen in msys_p2w in src\msys\msys\rt\src\winsup\cygwin\path.cc, line 3204ff: case '-': // // here we check for POSIX paths as attributes to a POSIX switch. // ... seemingly absolute POSIX paths in single-letter options get expanded by msys.dll unless they contain '=' or ';'. So a quick and very dirty fix is to use '-L/;*evil/'. (Using an equal sign works only when it is before a comma, so in the above example, /=*green/ would still be converted to a DOS-style path.) The -L mangling can be done by the script, just before the parameter is passed to the executable. This version does not modify the body of the tests and is active on MinGW only. Commit-message-by: Johannes Schindelin <johannes.schindelin@gmx.de> Author: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | t9902: mingw-specific fix for gitfile link filesPat Thoyts2014-07-211-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The path in a .git platform independent link file needs to be absolute and under mingw we need it to be a windows type path, not a unix style path so it should start with a drive letter and not a /. Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | t4210: skip command-line encoding tests on mingwPat Thoyts2014-07-211-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Windows the application command line is provided as unicode and in mingw-git we convert that to utf-8. So these tests that require a iso-8859-1 input are being subverted by the encoding transformations we perform and should be skipped. Signed-off-by: Pat Thoyts <patthoyts@users.sourceforge.net> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | MinGW: disable legacy encoding testsKarsten Blees2014-07-213-15/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Windows, all native APIs are Unicode-based. It is impossible to pass legacy encoded byte arrays to a process via command line or environment variables. Disable the tests that try to do so. In t3901, most tests still work if we don't mess up the repository encoding in setup, so don't switch to ISO-8859-1 on MinGW. Note that i18n tests that do their encoding tricks via encoded files (such as t3900) are not affected by this. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | t0110/MinGW: skip tests that pass arbitrary bytes on the command lineKarsten Blees2014-07-211-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On Windows, the command line is a Unicode string, it is not possible to pass arbitrary bytes to a program. Disable tests that try to do so. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | MinGW: Skip test redirecting to fd 4Johannes Schindelin2014-07-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... because that does not work in MinGW. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | | Merge branch 'sk/mingw-uni-fix-more'Junio C Hamano2014-07-308-150/+184
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most of these are battle-tested in msysgit and are needed to complete what has been merged to 'master' already. * sk/mingw-uni-fix-more: Win32: enable color output in Windows cmd.exe Win32: patch Windows environment on startup Win32: keep the environment sorted Win32: use low-level memory allocation during initialization Win32: reduce environment array reallocations Win32: don't copy the environment twice when spawning child processes Win32: factor out environment block creation Win32: unify environment function names Win32: unify environment case-sensitivity Win32: fix environment memory leaks Win32: Unicode environment (incoming) Win32: Unicode environment (outgoing) Revert "Windows: teach getenv to do a case-sensitive search" tests: do not pass iso8859-1 encoded parameter
| * | | | Win32: enable color output in Windows cmd.exesk/mingw-uni-fix-moreKarsten Blees2014-07-211-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git requires the TERM environment variable to be set for all color* settings. Simulate the TERM variable if it is not set (default on Windows). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: patch Windows environment on startupKarsten Blees2014-07-211-13/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix Windows specific environment settings on startup rather than checking for special values on every getenv call. As a side effect, this makes the patched environment (i.e. with properly initialized TMPDIR and TERM) available to child processes. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: keep the environment sortedKarsten Blees2014-07-211-39/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The Windows environment is sorted, keep it that way for O(log n) environment access. Change compareenv to compare only the keys, so that it can be used to find an entry irrespective of the value. Change lookupenv to binary seach for an entry. Return one's complement of the insert position if not found (libc's bsearch returns NULL). Replace MSVCRT's getenv with a minimal do_getenv based on the binary search function. Change do_putenv to insert new entries at the correct position. Simplify the function by swapping if conditions and using memmove instead of for loops. Move qsort from make_environment_block to mingw_startup. We still need to sort on startup to make sure that the environment is sorted according to our compareenv function (while Win32 / CreateProcess requires the environment block to be sorted case-insensitively, CreateProcess currently doesn't enforce this, and some applications such as bash just don't care). Note that environment functions are _not_ thread-safe and are not required to be so by POSIX, the application is responsible for synchronizing access to the environment. MSVCRT's getenv and our new getenv implementation are better than that in that they are thread-safe with respect to other getenv calls as long as the environment is not modified. Git's indiscriminate use of getenv in background threads currently requires this property. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: use low-level memory allocation during initializationKarsten Blees2014-07-211-15/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As of d41489a6 "Add more large blob test cases", git's high-level memory allocation functions (xmalloc, xmemdupz etc.) access the environment to simulate limited memory in tests (see 'getenv("GIT_ALLOC_LIMIT")' in memory_limit_check()). These functions should not be used before the environment is fully initialized (particularly not to initialize the environment itself). The current solution ('environ = NULL; ALLOC_GROW(environ...)') only works because MSVCRT's getenv() reinitializes environ when it is NULL (i.e. it leaves us with two sets of unusabe (non-UTF-8) and unfreeable (CRT- allocated) environments). Add our own set of malloc-or-die functions to be used in startup code. Also check the result of __wgetmainargs, which may fail if there's not enough memory for wide-char arguments and environment. This patch is in preparation of the sorted environment feature, which completely replaces MSVCRT's getenv() implementation. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: reduce environment array reallocationsKarsten Blees2014-07-211-27/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move environment array reallocation from do_putenv to the respective callers. Keep track of the environment size in a global variable. Use ALLOC_GROW in mingw_putenv to reduce reallocations. Allocate a sufficiently sized environment array in make_environment_block to prevent reallocations. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: don't copy the environment twice when spawning child processesKarsten Blees2014-07-213-64/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When spawning child processes via start_command(), the environment and all environment entries are copied twice. First by make_augmented_environ / copy_environ to merge with child_process.env. Then a second time by make_environment_block to create a sorted environment block string as required by CreateProcess. Move the merge logic to make_environment_block so that we only need to copy the environment once. This changes semantics of the env parameter: it now expects a delta (such as child_process.env) rather than a full environment. This is not a problem as the parameter is only used by start_command() (all other callers previously passed char **environ, and now pass NULL). The merge logic no longer xstrdup()s the environment strings, so do_putenv must not free them. Add a parameter to distinguish this from normal putenv. Remove the now unused make_augmented_environ / free_environ API. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: factor out environment block creationKarsten Blees2014-07-211-23/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: unify environment function namesKarsten Blees2014-07-211-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Environment helper functions use random naming ('env' prefix or suffix or both, with or without '_'). Change to POSIX naming scheme ('env' suffix, no '_'). Env_setenv has more in common with putenv than setenv. Change to do_putenv. Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | | | Win32: unify environment case-sensitivityKarsten Blees2014-07-211-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The environment on Windows is case-insensitive. Some environment functions (such as unsetenv and make_augmented_environ) have always used case- sensitive comparisons instead, while others (getenv, putenv, sorting in spawn*) were case-insensitive. Prevent potential inconsistencies by using case-insensitive comparison in lookup_env (used by putenv, unsetenv and make_augmented_environ). Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Stepan Kasal <kasal@ucw.cz> Signed-off-by: Junio C Hamano <gitster@pobox.com>