summaryrefslogtreecommitdiff
path: root/convert.c
Commit message (Collapse)AuthorAgeFilesLines
* sub-process: refactor handshake to common functionjt/subprocess-handshakeJonathan Tan2017-07-261-68/+7
| | | | | | | | | | | | | | | | | | Refactor, into a common function, the version and capability negotiation done when invoking a long-running process as a clean or smudge filter. This will be useful for other Git code that needs to interact similarly with a long-running process. As you can see in the change to t0021, this commit changes the error message reported when the long-running process does not introduce itself with the expected "server"-terminated line. Originally, the error message reports that the filter "does not support filter protocol version 2", differentiating between the old single-file filter protocol and the new multi-file filter protocol - I have updated it to something more generic and useful. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'ls/filter-process-delayed' into jt/subprocess-handshakeJunio C Hamano2017-07-261-52/+150
|\ | | | | | | | | | | | | | | | | | | | | * ls/filter-process-delayed: convert: add "status=delayed" to filter process protocol convert: refactor capabilities negotiation convert: move multiple file filter error handling to separate function convert: put the flags field before the flag itself for consistent style t0021: write "OUT <size>" only on success t0021: make debug log file name configurable t0021: keep filter log files on comparison
| * convert: add "status=delayed" to filter process protocolLars Schneider2017-06-301-16/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some `clean` / `smudge` filters may require a significant amount of time to process a single blob (e.g. the Git LFS smudge filter might perform network requests). During this process the Git checkout operation is blocked and Git needs to wait until the filter is done to continue with the checkout. Teach the filter process protocol, introduced in edcc8581 ("convert: add filter.<driver>.process option", 2016-10-16), to accept the status "delayed" as response to a filter request. Upon this response Git continues with the checkout operation. After the checkout operation Git calls "finish_delayed_checkout" which queries the filter for remaining blobs. If the filter is still working on the completion, then the filter is expected to block. If the filter has completed all remaining blobs then an empty response is expected. Git has a multiple code paths that checkout a blob. Support delayed checkouts only in `clone` (in unpack-trees.c) and `checkout` operations for now. The optimization is most effective in these code paths as all files of the tree are processed. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: refactor capabilities negotiationLars Schneider2017-06-301-12/+27
| | | | | | | | | | | | | | | | | | | | | | | | The code to negotiate long running filter capabilities was very repetitive for new capabilities. Replace the repetitive conditional statements with a table-driven approach. This is useful for the subsequent patch 'convert: add "status=delayed" to filter process protocol'. Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: move multiple file filter error handling to separate functionLars Schneider2017-06-291-21/+26
| | | | | | | | | | | | | | | | | | | | | | Refactoring the filter error handling is useful for the subsequent patch 'convert: add "status=delayed" to filter process protocol'. In addition, replace the parentheses around the empty "if" block with a single semicolon to adhere to the Git style guide. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert: put the flags field before the flag itself for consistent styleLars Schneider2017-06-291-5/+5
| | | | | | | | | | | | Suggested-by: Jeff King <peff@peff.net> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | hashmap.h: compare function has access to a data fieldStefan Beller2017-06-301-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using the hashmap a common need is to have access to caller provided data in the compare function. A couple of times we abuse the keydata field to pass in the data needed. This happens for example in patch-ids.c. This patch changes the function signature of the compare function to have one more void pointer available. The pointer given for each invocation of the compare function must be defined in the init function of the hashmap and is just passed through. Documentation of this new feature is deferred to a later patch. This is a rather mechanical conversion, just adding the new pass-through parameter. However while at it improve the naming of the fields of all compare functions used by hashmaps by ensuring unused parameters are prefixed with 'unused_' and naming the parameters what they are (instead of 'unused' make it 'unused_keydata'). Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'bw/config-h'Junio C Hamano2017-06-241-0/+1
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix configuration codepath to pay proper attention to commondir that is used in multi-worktree situation, and isolate config API into its own header file. * bw/config-h: config: don't implicitly use gitdir or commondir config: respect commondir setup: teach discover_git_directory to respect the commondir config: don't include config.h by default config: remove git_config_iter config: create config.h
| * | config: don't include config.h by defaultBrandon Williams2017-06-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Stop including config.h by default in cache.h. Instead only include config.h in those files which require use of the config system. Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | convert: convert renormalize_buffer to take an indexBrandon Williams2017-06-131-2/+4
| | | | | | | | | | | | | | | Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | convert: convert convert_to_git to take an indexBrandon Williams2017-06-131-3/+4
| | | | | | | | | | | | | | | Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | convert: convert convert_to_git_filter_fd to take an indexBrandon Williams2017-06-131-2/+3
| | | | | | | | | | | | | | | Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | convert: convert crlf_to_git to take an indexBrandon Williams2017-06-131-6/+8
| | | | | | | | | | | | | | | Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | | convert: convert get_cached_convert_stats_ascii to take an indexBrandon Williams2017-06-131-2/+3
| |/ |/| | | | | | | Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: update subprocess_read_status() to not die on EOFbp/sub-process-convert-filterBen Peart2017-05-151-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | Enable sub-processes to gracefully handle when the process dies by updating subprocess_read_status to return an error on EOF instead of dying. Update apply_multi_file_filter to take advantage of the revised subprocess_read_status. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | sub-process: move sub-process functions into separate filesBen Peart2017-05-151-103/+1
| | | | | | | | | | | | | | | | Move the sub-proces functions into sub-process.h/c. Add documentation for the new module in Documentation/technical/api-sub-process.txt Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: rename reusable sub-process functionsBen Peart2017-05-151-20/+20
| | | | | | | | | | | | | | | | Do a mechanical rename of the functions that will become the reusable sub-process module. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: update generic functions to only use generic data structuresBen Peart2017-05-151-18/+23
| | | | | | | | | | | | | | | | | | Update all functions that are going to be moved into a reusable module so that they only work with the reusable data structures. Move code that is specific to the filter out into the filter specific functions. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: separate generic structures and variables from the filter specific onesBen Peart2017-05-151-15/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | To enable future reuse of the filter.<driver>.process infrastructure, split the cmd2process structure into two separate parts. subprocess_entry will now contain the generic data required to manage the creation and tracking of the child process in a hashmap. cmd2process is a filter protocol specific structure that is used to track the negotiated capabilities of the filter. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: split start_multi_file_filter() into two separate functionsBen Peart2017-05-151-24/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To enable future reuse of the filter.<driver>.process infrastructure, split start_multi_file_filter() into two separate parts. start_multi_file_filter() will now only contain the generic logic to manage the creation and tracking of the child process in a hashmap. start_multi_file_filter_fn() is a protocol specific initialization function that will negotiate the multi-file-filter interface version and capabilities. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: move packet_write_line() into pkt-line as packet_writel()Ben Peart2017-05-081-21/+2
| | | | | | | | | | | | | | | | | | Add packet_writel() which writes multiple lines in a single call and then calls packet_flush_gently(). Update convert.c to use the new packet_writel() function from pkt-line. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | convert: remove erroneous tests for errno == EPIPEBen Peart2017-05-081-2/+2
|/ | | | | | | | | | | start_multi_file_filter() and apply_multi_file_filter() currently test for errno == EPIPE but treating EPIPE as an error is already happening from one of the packet_write() functions. Signed-off-by: Ben Peart <benpeart@microsoft.com> Found/Fixed-by: Jeff King <peff@peff.net> Acked-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* attr: convert git_check_attrs() callers to use the new APIJunio C Hamano2017-02-011-11/+6
| | | | | | | | | | | | | | | The remaining callers are all simple "I have N attributes I am interested in. I'll ask about them with various paths one by one". After this step, no caller to git_check_attrs() remains. After removing it, we can extend "struct attr_check" struct with data that can be used in optimizing the query for the specific N attributes it contains. Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* attr: rename function and struct related to checking attributesJunio C Hamano2017-02-011-6/+6
| | | | | | | | | | | | | | | | The traditional API to check attributes is to prepare an N-element array of "struct git_attr_check" and pass N and the array to the function "git_check_attr()" as arguments. In preparation to revamp the API to pass a single structure, in which these N elements are held, rename the type used for these individual array elements to "struct attr_check_item" and rename the function to "git_check_attrs()". Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Brandon Williams <bmwill@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'jc/renormalize-merge-kill-safer-crlf'Junio C Hamano2016-12-191-6/+7
|\ | | | | | | | | | | | | | | | | | | Fix a corner case in merge-recursive regression that crept in during 2.10 development cycle. * jc/renormalize-merge-kill-safer-crlf: convert: git cherry-pick -Xrenormalize did not work merge-recursive: handle NULL in add_cacheinfo() correctly cherry-pick: demonstrate a segmentation fault
| * convert: git cherry-pick -Xrenormalize did not workjc/renormalize-merge-kill-safer-crlfTorsten Bögershausen2016-12-011-6/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Working with a repo that used to be all CRLF. At some point it was changed to all LF, with `text=auto` in .gitattributes. Trying to cherry-pick a commit from before the switchover fails: $ git cherry-pick -Xrenormalize <commit> fatal: CRLF would be replaced by LF in [path] Commit 65237284 "unify the "auto" handling of CRLF" introduced a regression: Whenever crlf_action is CRLF_TEXT_XXX and not CRLF_AUTO_XXX, SAFE_CRLF_RENORMALIZE was feed into check_safe_crlf(). This is wrong because here everything else than SAFE_CRLF_WARN is treated as SAFE_CRLF_FAIL. Call check_safe_crlf() only if checksafe is SAFE_CRLF_WARN or SAFE_CRLF_FAIL. Reported-by: Eevee (Lexy Munroe) <eevee@veekun.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'ls/filter-process'Junio C Hamano2016-10-311-41/+335
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The smudge/clean filter API expect an external process is spawned to filter the contents for each path that has a filter defined. A new type of "process" filter API has been added to allow the first request to run the filter for a path to spawn a single process, and all filtering need is served by this single process for multiple paths, reducing the process creation overhead. * ls/filter-process: contrib/long-running-filter: add long running filter example convert: add filter.<driver>.process option convert: prepare filter.<driver>.process option convert: make apply_filter() adhere to standard Git error handling pkt-line: add functions to read/write flush terminated packet streams pkt-line: add packet_write_gently() pkt-line: add packet_flush_gently() pkt-line: add packet_write_fmt_gently() pkt-line: extract set_packet_header() pkt-line: rename packet_write() to packet_write_fmt() run-command: add clean_on_exit_handler run-command: move check_pipe() from write_or_die to run_command convert: modernize tests convert: quote filter names in error messages
| * | convert: add filter.<driver>.process optionLars Schneider2016-10-171-6/+295
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Git's clean/smudge mechanism invokes an external filter process for every single blob that is affected by a filter. If Git filters a lot of blobs then the startup time of the external filter processes can become a significant part of the overall Git execution time. In a preliminary performance test this developer used a clean/smudge filter written in golang to filter 12,000 files. This process took 364s with the existing filter mechanism and 5s with the new mechanism. See details here: https://github.com/github/git-lfs/pull/1382 This patch adds the `filter.<driver>.process` string option which, if used, keeps the external filter process running and processes all blobs with the packet format (pkt-line) based protocol over standard input and standard output. The full protocol is explained in detail in `Documentation/gitattributes.txt`. A few key decisions: * The long running filter process is referred to as filter protocol version 2 because the existing single shot filter invocation is considered version 1. * Git sends a welcome message and expects a response right after the external filter process has started. This ensures that Git will not hang if a version 1 filter is incorrectly used with the filter.<driver>.process option for version 2 filters. In addition, Git can detect this kind of error and warn the user. * The status of a filter operation (e.g. "success" or "error) is set before the actual response and (if necessary!) re-set after the response. The advantage of this two step status response is that if the filter detects an error early, then the filter can communicate this and Git does not even need to create structures to read the response. * All status responses are pkt-line lists terminated with a flush packet. This allows us to send other status fields with the same protocol in the future. Helped-by: Martin-Louis Bright <mlbright@gmail.com> Reviewed-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | convert: prepare filter.<driver>.process optionLars Schneider2016-10-171-26/+34
| | | | | | | | | | | | | | | | | | | | | | | | Refactor the existing 'single shot filter mechanism' and prepare the new 'long running filter mechanism'. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | convert: make apply_filter() adhere to standard Git error handlingLars Schneider2016-10-171-9/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | apply_filter() returns a boolean that tells the caller if it "did convert or did not convert". The variable `ret` was used throughout the function to track errors whereas `1` denoted success and `0` failure. This is unusual for the Git source where `0` denotes success. Rename the variable and flip its value to make the function easier readable for Git developers. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * | convert: quote filter names in error messagesLars Schneider2016-10-171-6/+6
| |/ | | | | | | | | | | | | | | Git filter driver commands with spaces (e.g. `filter.sh foo`) are hard to read in error messages. Quote them to improve the readability. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | i18n: convert mark error messages for translationVasco Almeida2016-10-171-4/+8
|/ | | | | | | | | Mark error messages about CRLF for translation. Update test to reflect changes. Signed-off-by: Vasco Almeida <vascomalmeida@sapo.pt> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert: Correct NNO tests and missing `LF will be replaced by CRLF`tb/t0027-raciness-fixTorsten Bögershausen2016-08-141-40/+57
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a non-reversible CRLF conversion is done in "git add", a warning is printed on stderr (or Git dies, depending on checksafe) The function commit_chk_wrnNNO() in t0027 was written to test this, but did the wrong thing: Instead of looking at the warning from "git add", it looked at the warning from "git commit". This is racy because "git commit" may not have to do CRLF conversion at all if it can use the sha1 value from the index (which depends on whether "add" and "commit" run in a single second). Correct t0027 and replace the commit for each and every file with a commit of all files in one go. The function commit_chk_wrnNNO() should be renamed in a separate commit. Now that t0027 does the right thing, it detects a bug in covert.c: This sequence should generate the warning `LF will be replaced by CRLF`, but does not: $ git init $ git config core.autocrlf false $ printf "Line\r\n" >file $ git add file $ git commit -m "commit with CRLF" $ git config core.autocrlf true $ printf "Line\n" >file $ git add file "git add" calls crlf_to_git() in convert.c, which calls check_safe_crlf(). When has_cr_in_index(path) is true, crlf_to_git() returns too early and check_safe_crlf() is not called at all. Factor out the code which determines if "git checkout" converts LF->CRLF into will_convert_lf_to_crlf(). Update the logic around check_safe_crlf() and "simulate" the possible LF->CRLF conversion at "git checkout" with help of will_convert_lf_to_crlf(). Thanks to Jeff King <peff@peff.net> for analyzing t0027. Reported-By: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert: unify the "auto" handling of CRLFTorsten Bögershausen2016-07-061-20/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, $ echo "* text=auto" >.gitattributes $ echo "* eol=crlf" >>.gitattributes would have the same effect as $ echo "* text" >.gitattributes $ git config core.eol crlf Since the 'eol' attribute had higher priority than 'text=auto', this may corrupt binary files and is not what most users expect to happen. Make the 'eol' attribute to obey 'text=auto' and now $ echo "* text=auto" >.gitattributes $ echo "* eol=crlf" >>.gitattributes behaves the same as $ echo "* text=auto" >.gitattributes $ git config core.eol crlf In other words, $ echo "* text=auto eol=crlf" >.gitattributes has the same effect as $ git config core.autocrlf true and $ echo "* text=auto eol=lf" >.gitattributes has the same effect as $ git config core.autocrlf input Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert.c: ident + core.autocrlf didn't worktb/core-eol-fixTorsten Bögershausen2016-04-251-12/+7
| | | | | | | | | | | | | | | When the ident attributes is set, get_stream_filter() did not obey core.autocrlf=true, and the file was checked out with LF. Change the rule when a streaming filter can be used: - if an external filter is specified, don't use a stream filter. - if the worktree eol is CRLF and "auto" is active, don't use a stream filter. - Otherwise the stream filter can be used. Add test cases in t0027. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'tb/conversion'Junio C Hamano2016-02-261-90/+100
|\ | | | | | | | | | | | | | | | | | | | | | | | | Code simplification. * tb/conversion: convert.c: correct attr_action() convert.c: simplify text_stat convert.c: refactor crlf_action convert.c: use text_eol_is_crlf() convert.c: remove input_crlf_action() convert.c: remove unused parameter 'path' t0027: add tests for get_stream_filter()
| * convert.c: correct attr_action()tb/conversionTorsten Bögershausen2016-02-231-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | df747b81 (convert.c: refactor crlf_action, 2016-02-10) introduced a bug to "git ls-files --eol". The "text" attribute was shown as "text eol=lf" or "text eol=crlf", depending on core.autocrlf or core.eol. Correct this and add test cases in t0027. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: simplify text_statTorsten Bögershausen2016-02-101-25/+22
| | | | | | | | | | | | | | | | | | | | | | Simplify the statistics: lonecr counts the CR which is not followed by a LF, lonelf counts the LF which is not preceded by a CR, crlf counts CRLF combinations. This simplifies the evaluation of the statistics. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: refactor crlf_actionTorsten Bögershausen2016-02-101-31/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor the determination and usage of crlf_action. Today, when no "crlf" attribute are set on a file, crlf_action is set to CRLF_GUESS. Use CRLF_UNDEFINED instead, and search for "text" or "eol" as before. After searching for line ending attributes, save the value in struct conv_attrs.crlf_action attr_action, so that get_convert_attr_ascii() is able report the attributes. Replace the old CRLF_GUESS usage: CRLF_GUESS && core.autocrlf=true -> CRLF_AUTO_CRLF CRLF_GUESS && core.autocrlf=false -> CRLF_BINARY CRLF_GUESS && core.autocrlf=input -> CRLF_AUTO_INPUT Save the action in conv_attrs.crlf_action (as before) and change all callers. Make more clear, what is what, by defining: - CRLF_UNDEFINED : No attributes set. Temparally used, until core.autocrlf and core.eol is evaluated and one of CRLF_BINARY, CRLF_AUTO_INPUT or CRLF_AUTO_CRLF is selected - CRLF_BINARY : No processing of line endings. - CRLF_TEXT : attribute "text" is set, line endings are processed. - CRLF_TEXT_INPUT: attribute "input" or "eol=lf" is set. This implies text. - CRLF_TEXT_CRLF : attribute "eol=crlf" is set. This implies text. - CRLF_AUTO : attribute "auto" is set. - CRLF_AUTO_INPUT: core.autocrlf=input (no attributes) - CRLF_AUTO_CRLF : core.autocrlf=true (no attributes) Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: use text_eol_is_crlf()Torsten Bögershausen2016-02-081-6/+14
| | | | | | | | | | | | | | | | | | Add a helper function to find out, which line endings text files should get at checkout, depending on core.autocrlf and core.eol configuration variables. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: remove input_crlf_action()Torsten Bögershausen2016-02-081-23/+14
| | | | | | | | | | | | | | | | | | | | | | | | Integrate the code of input_crlf_action() into convert_attrs(), so that ca.crlf_action is always valid after calling convert_attrs(). Keep a copy of crlf_action in attr_action, this is needed for get_convert_attr_ascii(). Remove eol_attr from struct conv_attrs, as it is now used temporally. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
| * convert.c: remove unused parameter 'path'Torsten Bögershausen2016-02-081-10/+9
| | | | | | | | | | | | | | | | Some functions get a parameter path, but don't use it. Remove the unused parameter. Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'ls/clean-smudge-override-in-config'Junio C Hamano2016-02-101-1/+1
|\ \ | |/ |/| | | | | | | | | | | | | Clean/smudge filters defined in a configuration file of lower precedence can now be overridden to be a pass-through no-op by setting the variable to an empty string. * ls/clean-smudge-override-in-config: convert: treat an empty string for clean/smudge filters as "cat"
| * convert: treat an empty string for clean/smudge filters as "cat"ls/clean-smudge-override-in-configLars Schneider2016-01-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Once a lower-priority configuration file defines a clean or smudge filter, there is no convenient way to override it to produce as-is output. Even though the configuration mechanism implements "the last one wins" semantics, you cannot set them to an empty string and expect them to work, as apply_filter() would try to run the empty string as an external command and fail. The conversion is not done, but the function would still report a failure to convert. Even though resetting the variable to "cat" (i.e. pass the data back as-is and report success) is an obvious and a viable way to solve this, it is wasteful to spawn an external process just as a workaround. Instead, teach apply_filter() to treat an empty string as a no-op filter that always returns successfully its input as-is without conversion. Signed-off-by: Lars Schneider <larsxschneider@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | ls-files: add eol diagnosticstb/ls-files-eolTorsten Bögershausen2016-01-181-28/+91
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When working in a cross-platform environment, a user may want to check if text files are stored normalized in the repository and if .gitattributes are set appropriately. Make it possible to let Git show the line endings in the index and in the working tree and the effective text/eol attributes. The end of line ("eolinfo") are shown like this: "-text" binary (or with bare CR) file "none" text file without any EOL "lf" text file with LF "crlf" text file with CRLF "mixed" text file with mixed line endings. The effective text/eol attribute is one of these: "", "-text", "text", "text=auto", "text eol=lf", "text eol=crlf" git ls-files --eol gives an output like this: i/none w/none attr/text=auto t/t5100/empty i/-text w/-text attr/-text t/test-binary-2.png i/lf w/lf attr/text eol=lf t/t5100/rfc2047-info-0007 i/lf w/crlf attr/text eol=crlf doit.bat i/mixed w/mixed attr/ locale/XX.po to show what eol convention is used in the data in the index ('i'), and in the working tree ('w'), and what attribute is in effect, for each path that is shown. Add test cases in t0027. Helped-By: Eric Sunshine <sunshine@sunshineco.com> Signed-off-by: Torsten Bögershausen <tboegi@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* convert trivial sprintf / strcpy calls to xsnprintfJeff King2015-09-251-1/+2
| | | | | | | | | | | | | | | | | | We sometimes sprintf into fixed-size buffers when we know that the buffer is large enough to fit the input (either because it's a constant, or because it's numeric input that is bounded in size). Likewise with strcpy of constant strings. However, these sites make it hard to audit sprintf and strcpy calls for buffer overflows, as a reader has to cross-reference the size of the array with the input. Let's use xsnprintf instead, which communicates to a reader that we don't expect this to overflow (and catches the mistake in case we do). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* filter_buffer_or_fd(): ignore EPIPEjc/ignore-epipe-in-filterJunio C Hamano2015-05-201-1/+6
| | | | | | | | | | | We are explicitly ignoring SIGPIPE, as we fully expect that the filter program may not read our output fully. Ignore EPIPE that may come from writing to it as well. A new test was stolen from Jeff's suggestion. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'sp/stream-clean-filter'Junio C Hamano2014-10-081-6/+49
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running a required clean filter, we do not have to mmap the original before feeding the filter. Instead, stream the file contents directly to the filter and process its output. * sp/stream-clean-filter: sha1_file: don't convert off_t to size_t too early to avoid potential die() convert: stream from fd to required clean filter to reduce used address space copy_fd(): do not close the input file descriptor mmap_limit: introduce GIT_MMAP_LIMIT to allow testing expected mmap size memory_limit: use git_env_ulong() to parse GIT_ALLOC_LIMIT config.c: add git_env_ulong() to parse environment variable convert: drop arguments other than 'path' from would_convert_to_git()
| * convert: stream from fd to required clean filter to reduce used address spaceSteffen Prohaska2014-08-281-6/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The data is streamed to the filter process anyway. Better avoid mapping the file if possible. This is especially useful if a clean filter reduces the size, for example if it computes a sha1 for binary data, like git media. The file size that the previous implementation could handle was limited by the available address space; large files for example could not be handled with (32-bit) msysgit. The new implementation can filter files of any size as long as the filter output is small enough. The new code path is only taken if the filter is required. The filter consumes data directly from the fd. If it fails, the original data is not immediately available. The condition can easily be handled as a fatal error, which is expected for a required filter anyway. If the filter was not required, the condition would need to be handled in a different way, like seeking to 0 and reading the data. But this would require more restructuring of the code and is probably not worth it. The obvious approach of falling back to reading all data would not help achieving the main purpose of this patch, which is to handle large files with limited address space. If reading all data is an option, we can simply take the old code path right away and mmap the entire file. The environment variable GIT_MMAP_LIMIT, which has been introduced in a previous commit is used to test that the expected code path is taken. A related test that exercises required filters is modified to verify that the data actually has been modified on its way from the file system to the object store. Signed-off-by: Steffen Prohaska <prohaska@zib.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | run-command: introduce CHILD_PROCESS_INITRené Scharfe2014-08-201-2/+1
|/ | | | | | | | | | | | Most struct child_process variables are cleared using memset first after declaration. Provide a macro, CHILD_PROCESS_INIT, that can be used to initialize them statically instead. That's shorter, doesn't require a function call and is slightly more readable (especially given that we already have STRBUF_INIT, ARGV_ARRAY_INIT etc.). Helped-by: Johannes Sixt <j6t@kdbg.org> Signed-off-by: Rene Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
View Lorry Controller Status