http: make sure we can consume the data we requestcmn/http-recv-buffer

The recv buffer (parse_buffer) and the buffer have independent sizes and
offsets. We try to fill in parse_buffer as much as possible before
passing it to the http parser. This is fine most of the time, but fails
us when the buffer is almost full.

In those situations, parse_buffer can have more data than we would be
able to put into the buffer (which may be getting full if we're towards
the end of a data sideband packet).

To work around this, we check if the space we have left on our buffer is
smaller than what could come from the network. If this happens, we make
parse_buffer think that it has as much space left as our buffer, so it
won't try to retrieve more data than we can deal with.

As the start of the data may no longer be at the start of the buffer, we
need to keep track of where it really starts (data_offset) and use that
in our calculations for the real size of the data we received from the
network.

This fixes #2518.

1 files changed, 20 insertions, 4 deletions

diff --git a/src/transports/http.c b/src/transports/http.c
index 5c5e5d391..f9df53b71 100644
--- a/src/transports/http.c
+++ b/src/transports/http.c
@@ -607,7 +607,23 @@ replay:
 	}
 
 	while (!*bytes_read && !t->parse_finished) {
-		t->parse_buffer.offset = 0;
+		size_t data_offset;
+
+		/*
+		 * Make the parse_buffer think it's as full of data as
+		 * the buffer, so it won't try to recv more data than
+		 * we can put into it.
+		 *
+		 * data_offset is the actual data offset from which we
+		 * should tell the parser to start reading.
+		 */
+		if (buf_size >= t->parse_buffer.len) {
+			t->parse_buffer.offset = 0;
+		} else {
+			t->parse_buffer.offset = t->parse_buffer.len - buf_size;
+		}
+
+		data_offset = t->parse_buffer.offset;
 
 		if (gitno_recv(&t->parse_buffer) < 0)
 			return -1;
@@ -628,8 +644,8 @@ replay:
 
 		bytes_parsed = http_parser_execute(&t->parser,
 			&t->settings,
-			t->parse_buffer.data,
-			t->parse_buffer.offset);
+			t->parse_buffer.data + data_offset,
+			t->parse_buffer.offset - data_offset);
 
 		t->parser.data = NULL;
 
@@ -647,7 +663,7 @@ replay:
 		if (t->parse_error < 0)
 			return -1;
 
-		if (bytes_parsed != t->parse_buffer.offset) {
+		if (bytes_parsed != t->parse_buffer.offset - data_offset) {
 			giterr_set(GITERR_NET,
 				"HTTP parser error: %s",
 				http_errno_description((enum http_errno)t->parser.http_errno));