summaryrefslogtreecommitdiff
path: root/fast-import.c
Commit message (Collapse)AuthorAgeFilesLines
* Finish current packfile during fast-import crash handlerShawn O. Pearce2008-02-161-0/+7
| | | | | | | | | | | | | | | | | | If fast-import is in the middle of crashing due to a protocol error or something like that then it can be very useful to have the mark table and all objects up until that point be available for a new import to resume from. Currently we just close the active packfile, unkeep all of our newly created packfiles (so they can be deleted), and dump the marks table to a temporary file. We don't attempt to update the refs/tags that the process has in memory as much of that data can be found in the crash report and I'm not sure it would be the right thing to do under every type of crash. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Include the fast-import marks table in crash reportsShawn O. Pearce2008-02-161-0/+10
| | | | | | | | | | | If fast-import was not run with --export-marks but we are crashing the frontend application developer may still benefit from having that information available to them. We now include the marks table as part of the crash report if --export-marks was not supplied on the command line. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Include annotated tags in fast-import crash reportsShawn O. Pearce2008-02-161-0/+13
| | | | | | | | | | | If annotated tags were created they exist in a different namespace within the fast-import process' internal memory tables so we did not export them in the inactive branch table. Now they are written out after the branches, in the order that they were defined by the frontend process. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* fast-import: check return value from unpack_entry()Shawn O. Pearce2008-02-151-0/+2
| | | | | | | | | | | If the tree object we have asked for is deltafied in the packfile and the delta did not apply correctly or was not able to be decompressed from the packfile then we can get back NULL instead of the tree data. This is (part of) the reason why read_sha1_file() can return NULL, so we need to also handle it the same way. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Document the hairy gfi_unpack_entry part of fast-importShawn O. Pearce2008-01-211-0/+33
| | | | | | | | | | | Junio pointed out this part of fast-import wasn't very clear on initial read, and it took some time for someone who was new to fast-import's "dirty little tricks" to understand how this was even working. So a little bit of commentary in the proper place may help future readers. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Teach fast-import to honor pack.compression and pack.depthShawn O. Pearce2008-01-211-3/+29
| | | | | | | | | | | | | | | | | | We now use the configured pack.compression and pack.depth values within fast-import, as like builtin-pack-objects fast-import is generating a packfile for consumption by the Git tools. We use the same behavior as builtin-pack-objects does for these options, allowing core.compression to supply the default value for pack.compression. The default setting for pack.depth within fast-import is still 10 as users will generally repack fast-import generated packfiles by `repack -f`. A large delta depth within the fast-import packfile can significantly slow down such a later repack. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* fast-import: Don't use a maybe-clobbered errno valueJim Meyering2008-01-181-3/+6
| | | | | | | | Without this change, each diagnostic could use an errno value clobbered by the close or unlink in rollback_lock_file. Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Fix random fast-import errors when compiled with NO_MMAPShawn O. Pearce2008-01-171-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fast-import was relying on the fact that on most systems mmap() and write() are synchronized by the filesystem's buffer cache. We were relying on the ability to mmap() 20 bytes beyond the current end of the file, then later fill in those bytes with a future write() call, then read them through the previously obtained mmap() address. This isn't always true with some implementations of NFS, but it is especially not true with our NO_MMAP=YesPlease build time option used on some platforms. If fast-import was built with NO_MMAP=YesPlease we used the malloc()+pread() emulation and the subsequent write() call does not update the trailing 20 bytes of a previously obtained "mmap()" (aka malloc'd) address. Under NO_MMAP that behavior causes unpack_entry() in sha1_file.c to be unable to read an object header (or data) that has been unlucky enough to be written to the packfile at a location such that it is in the trailing 20 bytes of a window previously opened on that same packfile. This bug has gone unnoticed for a very long time as it is highly data dependent. Not only does the object have to be placed at the right position, but it also needs to be positioned behind some other object that has been accessed due to a branch cache invalidation. In other words the stars had to align just right, and if you did run into this bug you probably should also have purchased a lottery ticket. Fortunately the workaround is a lot easier than the bug explanation. Before we allow unpack_entry() to read data from a pack window that has also (possibly) been modified through write() we force all existing windows on that packfile to be closed. By closing the windows we ensure that any new access via the emulated mmap() will reread the packfile, updating to the current file content. This comes at a slight performance degredation as we cannot reuse previously cached windows when we update the packfile. But it is a fairly minor difference as the window closes happen at only two points: - When the packfile is finalized and its .idx is generated: At this stage we are getting ready to update the refs and any data access into the packfile is going to be random, and is going after only the branch tips (to ensure they are valid). Our existing windows (if any) are not likely to be positioned at useful locations to access those final tip commits so we probably were closing them before anyway. - When the branch cache missed and we need to reload: At this point fast-import is getting change commands for the next commit and it needs to go re-read a tree object it previously had written out to the packfile. What windows we had (if any) are not likely to cover the tree in question so we probably were closing them before anyway. We do try to avoid unnecessarily closing windows in the second case by checking to see if the packfile size has increased since the last time we called unpack_entry() on that packfile. If the size has not changed then we have not written additional data, and any existing window is still vaild. This nicely handles the cases where fast-import is going through a branch cache reload and needs to read many trees at once. During such an event we are not likely to be updating the packfile so we do not cycle the windows between reads. With this change in place t9301-fast-export.sh (which was broken by c3b0dec509fe136c5417422f31898b5a4e2d5e02) finally works again. Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* fast-import.c: don't try to commit marks file if write failedBrandon Casey2008-01-171-8/+16
| | | | | | | | | | | We also move the assignment of -1 to the lock file descriptor up, so that rollback_lock_file() can be called safely after a possible attempt to fclose(). This matches the contents of the 'if' statement just above testing success of fdopen(). Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Acked-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Improve use of lockfile APIBrandon Casey2008-01-161-2/+10
| | | | | | | Remove remaining double close(2)'s. i.e. close() before commit_locked_index() or commit_lock_file(). Signed-off-by: Junio C Hamano <gitster@pobox.com>
* bundle, fast-import: detect write failureJim Meyering2008-01-101-2/+3
| | | | | | | | | | I noticed some unchecked writes. This fixes them. * bundle.c (create_bundle): Die upon write failure. * fast-import.c (keep_pack): Die upon write or close failure. Signed-off-by: Jim Meyering <meyering@redhat.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Update callers of check_ref_format()Junio C Hamano2008-01-021-2/+3
| | | | | | | | | | This updates send-pack and fast-import to use symbolic constants for checking the return values from check_ref_format(), and also futureproof the logic in lock_any_ref_for_update() to explicitly name the case that is usually considered an error but is Ok for this particular use. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* fast-import: fix unalinged allocation and accessDavid S. Miller2007-12-141-5/+5
| | | | | | | | | The specialized pool allocator fast-import uses aligned objects on the size of a pointer, which was not sufficient at least on Sparc. Instead, make the alignment for objects of type unitmax_t. Signed-off-by: David S. Miller <davem@davemloft.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Merge branch 'maint'Junio C Hamano2007-11-141-7/+13
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * maint: git-clean: honor core.excludesfile Documentation: Fix man page breakage with DocBook XSL v1.72 git-remote.txt: fix typo core-tutorial.txt: Fix argument mistake in an example. replace reference to git-rm with git-reset in git-commit doc Grammar fixes for gitattributes documentation Don't allow fast-import tree delta chains to exceed maximum depth revert/cherry-pick: allow starting from dirty work tree. t/t3404: fix test for a bogus todo file. Conflicts: fast-import.c
| * Don't allow fast-import tree delta chains to exceed maximum depthShawn O. Pearce2007-11-131-7/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Brian Downing noticed fast-import can produce tree depths of up to 6,035 objects and even deeper. Long delta chains can create very small packfiles but cause problems during repacking as git needs to unpack each tree to count the reachable blobs. What's happening here is the active branch cache isn't big enough. We're swapping out the branch and thus recycling the tree information (struct tree_content) back into the free pool. When we later reload the tree we set the delta_depth to 0 but we kept the tree we just reloaded as a delta base. So if the tree we reloaded was already at the maximum depth we wouldn't know it and make the new tree a delta. Multiply the number of times the branch cache has to swap out the tree times max_depth (10) and you get the maximum delta depth of a tree created by fast-import. In Brian's case above the active branch cache had to swap the branch out 603/604 times during this import to produce a tree with a delta depth of 6035. Acked-by: Brian Downing <bdowning@lavos.net> Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fast-import.c: fix regression due to strbuf conversionPierre Habouzit2007-10-261-0/+1
| | | | | | | | | | | | | | | | Without this strbuf_detach(), it yields a double free later, the command is in fact stashed, and this is not a memory leak. Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Merge branch 'maint'Shawn O. Pearce2007-10-211-1/+1
|\ \ | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * maint: Describe more 1.5.3.5 fixes in release notes Fix diffcore-break total breakage Fix directory scanner to correctly ignore files without d_type Improve receive-pack error message about funny ref creation fast-import: Fix argument order to die in file_change_m git-gui: Don't display CR within console windows git-gui: Handle progress bars from newer gits git-gui: Correctly report failures from git-write-tree gitk.txt: Fix markup. send-pack: respect '+' on wildcard refspecs git-gui: accept versions containing text annotations, like 1.5.3.mingw.1 git-gui: Don't crash when starting gitk from a browser session git-gui: Allow gitk to be started on Cygwin with native Tcl/Tk git-gui: Ensure .git/info/exclude is honored in Cygwin workdirs git-gui: Handle starting on mapped shares under Cygwin git-gui: Display message box when we cannot find git in $PATH git-gui: Avoid using bold text in entire gui for some fonts
| * fast-import: Fix argument order to die in file_change_mJulian Phillips2007-10-201-1/+1
| | | | | | | | | | | | | | | | | | | | The arguments to the "Not a blob" die call in file_change_m were transposed, so that the command was printed as the type, and the type as the command. Switch them around so that the error message comes out correctly. Signed-off-by: Julian Phillips <julian@quantumfyre.co.uk> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | strbuf change: be sure ->buf is never ever NULL.Pierre Habouzit2007-09-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For that purpose, the ->buf is always initialized with a char * buf living in the strbuf module. It is made a char * so that we can sloppily accept things that perform: sb->buf[0] = '\0', and because you can't pass "" as an initializer for ->buf without making gcc unhappy for very good reasons. strbuf_init/_detach/_grow have been fixed to trust ->alloc and not ->buf anymore. as a consequence strbuf_detach is _mandatory_ to detach a buffer, copying ->buf isn't an option anymore, if ->buf is going to escape from the scope, and eventually be free'd. API changes: * strbuf_setlen now always works, so just make strbuf_reset a convenience macro. * strbuf_detatch takes a size_t* optional argument (meaning it can be NULL) to copy the buffer's len, as it was needed for this refactor to make the code more readable, and working like the callers. Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Rework unquote_c_style to work on a strbuf.Pierre Habouzit2007-09-201-25/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | If the gain is not obvious in the diffstat, the resulting code is more readable, _and_ in checkout-index/update-index we now reuse the same buffer to unquote strings instead of always freeing/mallocing. This also is more coherent with the next patch that reworks quoting functions. The quoting function is also made more efficient scanning for backslashes and treating portions of strings without a backslash at once. Signed-off-by: Pierre Habouzit <madcoder@debian.org>
* | strbuf API additions and enhancements.Pierre Habouzit2007-09-201-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add strbuf_remove, change strbuf_insert: As both are special cases of strbuf_splice, implement them as such. gcc is able to do the math and generate almost optimal code this way. Add strbuf_swap: Exchange the values of its arguments. Use it in fast-import.c Also fix spacing issues in strbuf.h Signed-off-by: Pierre Habouzit <madcoder@debian.org>
* | Use xmemdupz() in many places.Pierre Habouzit2007-09-181-3/+1
| | | | | | | | | | Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fast-import optimization:Pierre Habouzit2007-09-181-32/+20
| | | | | | | | | | | | | | | | | | | | Now that cmd_data acts on a strbuf, make last_object stashed buffer be a strbuf as well. On new stash, don't free the last stashed buffer, rather swap it with the one you will stash, this way, callers of store_object can act on static strbufs, and at some point, fast-import won't allocate new memory for objects buffers. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fast-import was using dbuf's, replace them with strbuf's.Pierre Habouzit2007-09-181-109/+68
| | | | | | | | | | Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Drop strbuf's 'eof' marker, and make read_line a first class citizen.Pierre Habouzit2007-09-181-15/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | read_line is now strbuf_getline, and is a first class citizen, it returns 0 when reading a line worked, EOF else. The ->eof marker was used non-locally by fast-import.c, mimic the same behaviour using a static int in "read_next_command", that now returns -1 on EOF, and avoids to call strbuf_getline when it's in EOF state. Also no longer automagically strbuf_release the buffer, it's counter intuitive and breaks fast-import in a very subtle way. Note: being at EOF implies that command_buf.len == 0. Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Now that cache.h needs strbuf.h, remove useless includes.Pierre Habouzit2007-09-161-1/+0
| | | | | | | | | | | | Signed-off-by: Pierre Habouzit <madcoder@debian.org> Acked-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Strbuf API extensions and fixes.Pierre Habouzit2007-09-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add strbuf_rtrim to remove trailing spaces. * Add strbuf_insert to insert data at a given position. * Off-by one fix in strbuf_addf: strbuf_avail() does not counts the final \0 so the overflow test for snprintf is the strict comparison. This is not critical as the growth mechanism chosen will always allocate _more_ memory than asked, so the second test will not fail. It's some kind of miracle though. * Add size extension hints for strbuf_init and strbuf_read. If 0, default applies, else: + initial buffer has the given size for strbuf_init. + first growth checks it has at least this size rather than the default 8192. Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | fast-import: Use strbuf API, and simplify cmd_data()Pierre Habouzit2007-09-061-17/+13
| | | | | | | | | | | | | | | | This patch features the use of strbuf_detach, and prevent the programmer to mess with allocation directly. The code is as efficent as before, just more concise and more straightforward. Signed-off-by: Junio C Hamano <gitster@pobox.com>
* | Rework strbuf API and semantics.Pierre Habouzit2007-09-061-8/+7
|/ | | | | | | | | | | | | | | | | | | | | | | | The gory details are explained in strbuf.h. The change of semantics this patch enforces is that the embeded buffer has always a '\0' character after its last byte, to always make it a C-string. The offs-by-one changes are all related to that very change. A strbuf can be used to store byte arrays, or as an extended string library. The `buf' member can be passed to any C legacy string function, because strbuf operations always ensure there is a terminating \0 at the end of the buffer, not accounted in the `len' field of the structure. A strbuf can be used to generate a string/buffer whose final size is not really known, and then "strbuf_detach" can be used to get the built buffer, and keep the wrapping "strbuf" structure usable for further work again. Other interesting feature: strbuf_grow(sb, size) ensure that there is enough allocated space in `sb' to put `size' new octets of data in the buffer. It helps avoiding reallocating data for nothing when the problem the strbuf helps to solve has a known typical size. Signed-off-by: Pierre Habouzit <madcoder@debian.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Avoid using va_copy in fast-import: it seems to be unportable.Alex Riesen2007-08-201-7/+6
| | | | | | | | [sp: minor change to use fputs, thus reducing the patch size] Signed-off-by: Alex Riesen <raa.lkml@gmail.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* fast-import pull requestJunio C Hamano2007-08-191-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | * skip_optional_lf() decl is old-style -- please say static skip_optional_lf(void) { ... } * t9300 #14 fails, like this: * expecting failure: git-fast-import <input fatal: Branch name doesn't conform to GIT standards: .badbranchname fast-import: dumping crash report to .git/fast_import_crash_14354 ./test-lib.sh: line 143: 14354 Segmentation fault git-fast-import <input -- >8 -- Subject: [PATCH] fastimport: Fix re-use of va_list The va_list is designed to be used only once. The current code reuses va_list argument may cause segmentation fault. Copy and release the arguments to avoid this problem. While we are at it, fix old-style function declaration of skip_optional_lf(). Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Include recent command history in fast-import crash reportsShawn O. Pearce2007-08-191-4/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | When we crash the frontend developer (or end-user) may need to know roughly around what part of the input stream we had a problem with and aborted on. Because line numbers aren't very useful in this sort of application we instead just keep the last 100 commands in a FIFO queue and print them as part of the crash report. Currently one problem with this design is a commit that has more than 100 modified files in it will flood the FIFO and any context regarding branch/from/committer/mark/comments will be lost. We really should save only the last few (10?) file changes for the current commit, ensuring we have some prior higher level commands in the FIFO when we crash on a file M/D/C/R command. Another issue with this approach is the FIFO only includes the commands, it does not include the commit messages. Yet having a commit message may be useful to help locate the relevant change in the source material. In practice I don't think this is going to be a major concern as the frontend can always embed its own source change set identifier as a comment (which will appear in the crash report) and the commit message(s) for the most recent commits of any given branch should be obtainable from the (packed) commit objects. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Generate crash reports on die in fast-importShawn O. Pearce2007-08-191-0/+93
| | | | | | | | | | | | | | | | | | | | | | | As fast-import is quite strict about its input and die()'s anytime something goes wrong it can be difficult for a frontend developer to troubleshoot why fast-import rejected their input, or to even determine what input command it rejected. This change introduces a custom handler for Git's die() routine. When we receive a die() for any reason (fast-import or a lower level core Git routine we called) the error is first dumped onto stderr and then a more extensive crash report file is prepared in GIT_DIR. Finally we exit the process with status 128, just like the stock builtin die handler. An internal flag is set to prevent any further die()'s that may be invoked during the crash report generator from causing us to enter into an infinite loop. We shouldn't die() from our crash report handler, but just in case someone makes a future code change we are prepared to gaurd against small mistakes turning into huge problems for the end-user. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Allow frontends to bidirectionally communicate with fast-importShawn O. Pearce2007-08-191-0/+14
| | | | | | | | | | | | | | | The existing checkpoint command is very useful to force fast-import to dump the branches out to disk so that standard Git tools can access them and the objects they refer to. However there was not a way to know when fast-import had finished executing the checkpoint and it was safe to read those refs. The progress command can be used to make fast-import output any message of the frontend's choosing to standard out. The frontend can scan for these messages using select() or poll() to monitor a pipe connected to the standard output of fast-import. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Make trailing LF optional for all fast-import commandsShawn O. Pearce2007-08-191-14/+20
| | | | | | | | | | | | | | | | | | For the same reasons as the prior change we want to allow frontends to omit the trailing LF that usually delimits commands. In some cases these just make the input stream more verbose looking than it needs to be, and its just simpler for the frontend developer to get started if our parser is slightly more lenient about where an LF is required and where it isn't. To make this optional LF feature work we now have to buffer up to one line of input in command_buf. This buffering can happen if we look at the current input command but don't recognize it at this point in the code. In such a case we need to "unget" the entire line, but we cannot depend upon the stdio library to let us do ungetc() for that many characters at once. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Make trailing LF following fast-import `data` commands optionalShawn O. Pearce2007-08-191-4/+9
| | | | | | | | | | | | A few fast-import frontend developers have found it odd that we require the LF following a `data` command, especially in the exact byte count format. Technically we don't need this LF to parse the stream properly, but having it here does make the stream more readable to humans. We can easily make the LF optional by peeking at the next byte available from the stream and pushing it back into the buffer if its not LF. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Teach fast-import to ignore lines starting with '#'Shawn O. Pearce2007-08-191-2/+15
| | | | | | | | | | | | | | | | | | | | | | Several frontend developers have asked that some form of stream comments be permitted within a fast-import data stream. This way they can include information from their own frontend program about where specific data was taken from in the source system, or about a decision that their frontend may have made while creating the fast-import data stream. This change introduces comments in the Bourne-shell/Tcl/Perl style. Lines starting with '#' are ignored, up to and including the LF. Unlike the above mentioned three languages however we do not look for and ignore leading whitespace. This just simplifies the definition of the comment format and the code that parses them. To make comments work we had to stop using read_next_command() within cmd_data() and directly invoke read_line() during the inline variant of the function. This is necessary to retain any lines of the input data that might otherwise look like a comment to fast-import. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Use handy ALLOC_GROW macro in fast-import when possibleShawn O. Pearce2007-08-191-6/+1
| | | | | | | | Instead of growing our buffer by hand during the inline variant of cmd_data() we can save a few lines of code and just use the nifty new ALLOC_GROW macro already available to us. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Actually allow TAG_FIXUP branches in fast-importShawn O. Pearce2007-08-191-1/+5
| | | | | | | | | | | | | | | | | | | Michael Haggerty <mhagger@alum.mit.edu> noticed while debugging a Git backend for cvs2svn that fast-import was barfing when he tried to use "TAG_FIXUP" as a branch name for temporary work needed to cleanup the tree prior to creating an annotated tag object. The reason we were rejecting the branch name was check_ref_format() returns -2 when there are less than 2 '/' characters in the input name. TAG_FIXUP has 0 '/' characters, but is technically just as valid of a ref as HEAD and MERGE_HEAD, so we really should permit it (and any other similar looking name) during import. New test cases have been added to make sure we still detect very wrong branch names (e.g. containing [ or starting with .) and yet still permit reasonable names (e.g. TAG_FIXUP). Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Fix whitespace in "Format of STDIN stream" of fast-importAlex Riesen2007-08-191-11/+11
| | | | | | | Something probably assumed that HT indentation is 4 characters. Signed-off-by: Alex Riesen <raa.lkml@gmail.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Use xmkstemp() instead of mkstemp()Luiz Fernando N. Capitulino2007-08-141-6/+2
| | | | | | | | xmkstemp() performs error checking and prints a standard error message when an error occur. Signed-off-by: Luiz Fernando N. Capitulino <lcapitulino@mandriva.com.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* Teach fast-import to recursively copy files/directoriesShawn O. Pearce2007-07-151-4/+77
| | | | | | | | | | | | | | | | | | | | | | Some source material (e.g. Subversion dump files) perform directory renames by telling us the directory was copied, then deleted in the same revision. This makes it difficult for a frontend to convert such data formats to a fast-import stream, as all the frontend has on hand is "Copy a/ to b/; Delete a/" with no details about what files are in a/, unless the frontend also kept track of all files. The new 'C' subcommand within a commit allows the frontend to make a recursive copy of one path to another path within the branch, without needing to keep track of the individual file paths. The metadata copy is performed in memory efficiently, but is implemented as a copy-immediately operation, rather than copy-on-write. With this new 'C' subcommand frontends could obviously implement an 'R' (rename) on their own as a combination of 'C' and 'D' (delete), but since we have already offered up 'R' in the past and it is a trivial thing to keep implemented I'm not going to deprecate it. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Support wholesale directory renames in fast-importShawn O. Pearce2007-07-091-16/+75
| | | | | | | | | | | | | | | | | Some source material (e.g. Subversion dump files) perform directory renames without telling us exactly which files in that subdirectory were moved. This makes it hard for a frontend to convert such data formats to a fast-import stream, as all the frontend has on hand is "Rename a/ to b/" with no details about what files are in a/, unless the frontend also kept track of all files. The new 'R' subcommand within a commit allows the frontend to rename either a file or an entire subdirectory, without needing to know the object's SHA-1 or the specific files contained within it. The rename is performed as efficiently as possible internally, making it cheaper than a 'D'/'M' pair for a file rename. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* Merge branch 'maint'Junio C Hamano2007-05-231-30/+37
|\ | | | | | | | | | | | | | | | | * maint: Fix possible coredump with fast-import --import-marks Refactor fast-import branch creation from existing commit fast-import: Fix crash when referencing already existing objects fast-import: Fix uninitialized variable Documentation: fix git-config.xml generation
| * Fix possible coredump with fast-import --import-marksShawn O. Pearce2007-05-241-5/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When e8438420bb7d368bec3647b90c557b9931582267 allowed us to reload the marks table on subsequent runs of fast-import we really broke things, as we set pack_id to MAX_PACK_ID for any objects we imported into the marks table. Creating a branch from that mark should fail as we attempt to read the object through a non-existant packed_git pointer. Instead we have to use the normal Git object system to locate the older commit, as we ourselves do not have a reference to the packed_git it resides in. This bug only occurred because t9300 was not complete enough. When we added the --import-marks feature we didn't actually test its implementation enough to verify the function worked as intended. I have corrected that, and included the changes as part of this fix. Prior versions of fast-import fail the new test(s); this commit allows them to pass. Credit for this bug find goes to Simon Hausmann <simon@lst.de> as he recently identified a similiar bug in the tree lazy-loading path. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * Refactor fast-import branch creation from existing commitShawn O. Pearce2007-05-241-27/+31
| | | | | | | | | | | | | | | | | | | | To resolve a corner case uncovered by Simon Hausmann I need to reuse the logic for the SHA-1 expression version of the 'from ' command within the mark version of the 'from ' command. This change doesn't alter any functionality, but is merely breaking the common code out to a function that I can reuse. Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * fast-import: Fix crash when referencing already existing objectsSimon Hausmann2007-05-231-1/+1
| | | | | | | | | | | | | | | | | | | | Commit a5c1780a0355a71b9fb70f1f1977ce726ee5b8d8 sets the pack_id of existing objects to MAX_PACK_ID. When the same object is referenced later again it is found in the local object hash. With such a pack_id fast-import should not try to locate that object in the newly created pack(s). Signed-off-by: Simon Hausmann <simon@lst.de> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
| * fast-import: Fix uninitialized variableSimon Hausmann2007-05-231-0/+1
| | | | | | | | | | | | | | | | Fix uninitialized last_object->no_free variable that is accessed in store_object. Signed-off-by: Simon Hausmann <simon@lst.de> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
* | git-update-ref: add --no-deref option for overwriting/detaching refSven Verdoolaege2007-05-101-1/+1
|/ | | | | | | | git-checkout is also adapted to make use of this new option instead of the handcrafted command sequence. Signed-off-by: Sven Verdoolaege <skimo@kotnet.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
* Create pack-write.c for common pack writing codeDana L. How2007-05-021-37/+3
| | | | | | | | | | | Include a generalized fixup_pack_header_footer() in this new file. Needed by git-repack --max-pack-size feature in a later patchset. [sp: Moved close(pack_fd) to callers, to support index-pack, and changed name to better indicate it is for packfiles.] Signed-off-by: Dana L. How <danahow@gmail.com> Signed-off-by: Shawn O. Pearce <spearce@spearce.org>