delta/git.git/gitweb, branch jn/gitweb-search-optim

gitweb: Faster project search

2012-02-23T20:53:03+00:00

Before searching by some field the information we search for must be
filled in, but we do not have to fill other fields that are not
involved in the search.

To be able to request filling only specified fields,
fill_project_list_info() was enhanced in previous commit to take
additional parameters which specify part of projects info to fill.
This way we can limit doing expensive calculations (like running
git-for-each-ref to get 'age' / "Last changed" info) to doing those
only for projects which we will show as search results.

This commit actually uses this interface, changing gitweb code from
the following behavior

  fill all project info on all projects
  search projects

to behaving like this pseudocode

  fill search fields on all projects
  search projects
  fill all project info on search results

With this commit the number of git commands used to generate search
results is 2* + 1, and depends on number of matched
projects rather than number of all projects (all repositories).

Note: this is 'git for-each-ref' to find last activity, and 'git config'
for each project, and 'git --version' once.

Example performance improvements, for search that selects 2
repositories out of 12 in total:

* Before (warm cache):
  "This page took 0.867151 seconds  and 27 git commands to generate."

* After (warm cache):
  "This page took 0.673643 seconds  and 5 git commands to generate."

Now imagine that they are 5 repositories out of 5000, and cold or
trashed cache case.

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano

gitweb: Option for filling only specified info in fill_project_list_info

2012-02-23T20:52:16+00:00

Enhance fill_project_list_info() subroutine to accept optional
parameters that specify which fields in project information needs to
be filled.  If none are specified then fill_project_list_info()
behaves as it used to, and ensure that all project info is filled.

This is in preparation of future lazy filling of project info in
project search and pagination of sorted list of projects.

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano

gitweb: Refactor checking if part of project info need filling

2012-02-23T20:49:37+00:00

Extract the check if given keys (given parts) of project info needs to
be filled into project_info_needs_filling() subroutine.  It is for now
a thin wrapper around "!exists $project_info->{$key}".

Note that !defined was replaced by more correct !exists.

While at it uniquify treating of all project info, adding checks for
'age' field before running git_get_last_activity(), and also checking
for all keys filled in code protected by conditional, and not only
one.

The code now looks like this

  foreach my $project (@$project_list) {
  	if (given keys need to be filled) {
  		fill given keys
  	}
  	...
  }

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano

Merge branch 'bl/gitweb-project-filter'

2012-02-14T20:57:17+00:00

* bl/gitweb-project-filter:
  gitweb: Harden and improve $project_filter page title

Merge branch 'jn/gitweb-search-utf-8'

2012-02-13T06:43:24+00:00

* jn/gitweb-search-utf-8:
  gitweb: Allow UTF-8 encoded CGI query parameters and path_info

Conflicts:
	gitweb/gitweb.perl

gitweb: Harden and improve $project_filter page title

2012-02-13T06:11:31+00:00

Commit 19d2d23 (gitweb: add project_filter to limit project list
to a subdirectory, 2012-01-30) added also support for displaying
$project_filter, if present, in page title.

Unfortunately it forgot to treat $project_filter as path, and escape
it using esc_path(), like it is done for $filename.

Also, it was not obvious that "$site_name - $project_filter" is about
project filtering: use "$site_name - projects in '$project_filter'".

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano

Merge branch 'bl/gitweb-project-filter'

2012-02-07T20:57:05+00:00

* bl/gitweb-project-filter:
  gitweb: Make project search respect project_filter
  gitweb: improve usability of projects search form
  gitweb: place links to parent directories in page header
  gitweb: show active project_filter in project_list page header
  gitweb: limit links to alternate forms of project_list to active project_filter
  gitweb: add project_filter to limit project list to a subdirectory
  gitweb: prepare git_get_projects_list for use outside 'forks'.
  gitweb: move hard coded .git suffix out of git_get_projects_list

gitweb: Allow UTF-8 encoded CGI query parameters and path_info

2012-02-03T21:03:08+00:00

Gitweb forgot to turn query parameters into UTF-8. This results in a bug
that one cannot search for a string with characters outside US-ASCII.  For
example searching for "Michał Kiedrowicz" (containing letter 'ł' - LATIN
SMALL LETTER L WITH STROKE, with Unicode codepoint U+0142, represented
with 0xc5 0x82 bytes in UTF-8 and percent-encoded as %C5%82) result in the
following incorrect data in search field

	MichaÅ\202 Kiedrowicz

This is caused by CGI by default treating '0xc5 0x82' bytes as two
characters in Perl legacy encoding latin-1 (iso-8859-1), because 's'
query parameter is not processed explicitly as UTF-8 encoded string.

The solution used here follows "Using Unicode in a Perl CGI script"
article on http://www.lemoda.net/cgi/perl-unicode/index.html:

	use CGI;
	use Encode 'decode_utf8;
	my $value = params('input');
	$value = decode_utf8($value);

Decoding UTF-8 is done when filling %input_params hash and $path_info
variable; the former requires to move from explicit $cgi->param()
to $input_params{} in a few places, which is a good idea anyway.

Also add -override=>1 parameter to $cgi->textfield() invocation in search
form.  Otherwise CGI would use values from query string if it is present,
filling value from $cgi->param... without decode_utf8().  As we are using
value of appropriate parameter anyway, -override=>1 doesn't change the
situation but makes gitweb fill search field correctly.

We could simply use the '-utf8' pragma (via "use CGI '-utf8';") to solve
this, but according to CGI.pm documentation, it may cause problems with
POST requests containing binary files, and it requires CGI 3.31 (I think),
released with perl v5.8.9.

Reported-by: Michał Kiedrowicz 
Signed-off-by: Jakub Narębski 
Tested-by: Michał Kiedrowicz 
Signed-off-by: Junio C Hamano

gitweb: Make project search respect project_filter

2012-02-01T20:52:50+00:00

Make gitweb search within filtered projects (i.e. projects shown), and
change "List all projects" to "List all projects in '$project_filter/'"
if project_filter is used.

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano

gitweb: improve usability of projects search form

2012-02-01T20:52:50+00:00

Refactor generating project search form into git_project_search_form().

Make text field wider and add on mouse over explanation (via "title"
attribute), add an option to use regular expressions, and replace
'Search:' label with [Search] button.

Also add "List all projects" link to make it easier to go back from search
result to list of all projects (note that an empty search term is
disallowed).

Signed-off-by: Jakub Narebski 
Signed-off-by: Junio C Hamano