From 63ef436ea15dd93343a2040ffb7c3bb2c9681b87 Mon Sep 17 00:00:00 2001 From: Patrick Monnerat Date: Tue, 5 Sep 2017 17:11:59 +0100 Subject: mime: implement encoders. curl_mime_encoder() is operational and documented. curl tool -F option is extended with ";encoder=". curl tool --libcurl option generates calls to curl_mime_encoder(). New encoder tests 648 & 649. Test 1404 extended with an encoder specification. --- docs/cmdline-opts/form.d | 14 ++ docs/libcurl/Makefile.inc | 2 +- docs/libcurl/curl_mime_addpart.3 | 3 +- docs/libcurl/curl_mime_encoder.3 | 97 ++++++++ lib/mime.c | 467 +++++++++++++++++++++++++++++++++++++-- lib/mime.h | 22 +- src/tool_formparse.c | 84 +++++-- src/tool_setopt.c | 8 + tests/data/Makefile.inc | 2 +- tests/data/test1404 | 8 +- tests/data/test648 | 75 +++++++ tests/data/test649 | 72 ++++++ 12 files changed, 809 insertions(+), 45 deletions(-) create mode 100644 docs/libcurl/curl_mime_encoder.3 create mode 100644 tests/data/test648 create mode 100644 tests/data/test649 diff --git a/docs/cmdline-opts/form.d b/docs/cmdline-opts/form.d index 14261d3ad..232d30c07 100644 --- a/docs/cmdline-opts/form.d +++ b/docs/cmdline-opts/form.d @@ -101,6 +101,20 @@ text file: .br -F '=)' -F '=@textfile.txt' ... smtp://example.com +Data can be encoded for transfer using encoder=. Available encodings are +\fIbinary\fP and \fI8bit\fP that do nothing else than adding the corresponding +Content-Transfer-Encoding header, \fI7bit\fP that only rejects 8-bit characters +with a transfer error, \fIquoted-printable\fP and \fIbase64\fP that encodes +data according to the corresponding schemes, limiting lines length to +76 characters. + +Example: send multipart mail with a quoted-printable text message and a +base64 attached file: + + curl -F '=text message;encoder=quoted-printable' \\ +.br + -F '=@localfile;encoder=base64' ... smtp://example.com + See further examples and details in the MANUAL. This option can be used multiple times. diff --git a/docs/libcurl/Makefile.inc b/docs/libcurl/Makefile.inc index 2414ecc12..86a002680 100644 --- a/docs/libcurl/Makefile.inc +++ b/docs/libcurl/Makefile.inc @@ -21,4 +21,4 @@ man_MANS = curl_easy_cleanup.3 curl_easy_getinfo.3 curl_easy_init.3 \ curl_mime_init.3 curl_mime_free.3 curl_mime_addpart.3 curl_mime_name.3 \ curl_mime_data.3 curl_mime_data_cb.3 curl_mime_filedata.3 \ curl_mime_filename.3 curl_mime_subparts.3 \ - curl_mime_type.3 curl_mime_headers.3 + curl_mime_type.3 curl_mime_headers.3 curl_mime_encoder.3 diff --git a/docs/libcurl/curl_mime_addpart.3 b/docs/libcurl/curl_mime_addpart.3 index cbb861852..5ea8171de 100644 --- a/docs/libcurl/curl_mime_addpart.3 +++ b/docs/libcurl/curl_mime_addpart.3 @@ -62,4 +62,5 @@ A mime part structure handle, or NULL upon failure. .BR curl_mime_filename "(3)," .BR curl_mime_subparts "(3)," .BR curl_mime_type "(3)," -.BR curl_mime_headers "(3)" +.BR curl_mime_headers "(3)," +.BR curl_mime_encoder "(3)" diff --git a/docs/libcurl/curl_mime_encoder.3 b/docs/libcurl/curl_mime_encoder.3 new file mode 100644 index 000000000..c17cf25b3 --- /dev/null +++ b/docs/libcurl/curl_mime_encoder.3 @@ -0,0 +1,97 @@ +.\" ************************************************************************** +.\" * _ _ ____ _ +.\" * Project ___| | | | _ \| | +.\" * / __| | | | |_) | | +.\" * | (__| |_| | _ <| |___ +.\" * \___|\___/|_| \_\_____| +.\" * +.\" * Copyright (C) 1998 - 2017, Daniel Stenberg, , et al. +.\" * +.\" * This software is licensed as described in the file COPYING, which +.\" * you should have received as part of this distribution. The terms +.\" * are also available at https://curl.haxx.se/docs/copyright.html. +.\" * +.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell +.\" * copies of the Software, and permit persons to whom the Software is +.\" * furnished to do so, under the terms of the COPYING file. +.\" * +.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY +.\" * KIND, either express or implied. +.\" * +.\" ************************************************************************** +.TH curl_mime_encoder 3 "22 August 2017" "libcurl 7.56.0" "libcurl Manual" +.SH NAME +curl_mime_encoder - set a mime part's encoder and content transfer encoding +.SH SYNOPSIS +.B #include +.sp +.BI "CURLcode curl_mime_encoder(curl_mimepart * " part , +.BI "const char * " encoding ");" +.ad +.SH DESCRIPTION +curl_mime_encoder() requests a mime part's content to be encoded before being +transmitted. + +\fIpart\fP is the part's handle to assign an encoder. +\fIencoding\fP is a pointer to a zero-terminated encoding scheme. It may be +set to NULL to disable an encoder previously attached to the part. The encoding +scheme storage may safely be reused after this function returns. + +Setting a part's encoder twice is valid: only the value set by the last call is +retained. + +Upon multipart rendering, the part's content is encoded according to the +pertaining scheme and a corresponding \fIContent-Transfer-Encoding"\fP header +is added to the part. + +Supported encoding schemes are: +.br +"\fIbinary\fP": the data is left unchanged, the header is added. +.br +"\fI8bit\fP": header added, no data change. +.br +"\fI7bit\fP": the data is unchanged, but is each byte is checked +to be a 7-bit value; if not, a read error occurs. +.br +"\fIbase64\fP": Data is converted to base64 encoding, then split in +CRLF-terminated lines of at most 76 characters. +.br +"\fIquoted-printable\fP": data is encoded in quoted printable lines of +at most 76 characters. Since the resulting size of the final data cannot be +determined prior to reading the original data, it is left as unknown, causing +chunked transfer in HTTP. For the same reason, this encoder may not be used +with IMAP. This encoder targets text data that is mostly ASCII and should +not be used with other types of data. + +If the original data is already encoded in such a scheme, a custom +\fIContent-Transfer-Encoding\fP header should be added with +\FIcurl_mime_headers\fP() instead of setting a part encoder. + +Encoding should not be applied to multiparts, thus the use of this +function on a part with content set with \fIcurl_mime_subparts\fP() is +strongly discouraged. +.SH AVAILABILITY +As long as at least one of HTTP, SMTP or IMAP is enabled. Added in 7.56.0. +.SH RETURN VALUE +CURLE_OK or a CURL error code upon failure. +.SH EXAMPLE +.nf + curl_mime *mime; + curl_mimepart *part; + + /* create a mime handle */ + mime = curl_mime_init(easy); + + /* add a part */ + part = curl_mime_addpart(mime); + + /* send a file */ + curl_mime_filedata(part, "image.png"); + + /* encode file data in base64 for transfer */ + curl_mime_encoder(part, "base64"); +.fi +.SH "SEE ALSO" +.BR curl_mime_addpart "(3)," +.BR curl_mime_headers "(3)," +.BR curl_mime_subparts "(3)" diff --git a/lib/mime.c b/lib/mime.c index f6a626316..38a20ed2d 100644 --- a/lib/mime.c +++ b/lib/mime.c @@ -55,6 +55,73 @@ #define READ_ERROR ((size_t) -1) +/* Encoders. */ +static size_t encoder_nop_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part); +static curl_off_t encoder_nop_size(struct Curl_mimepart *part); +static size_t encoder_7bit_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part); +static size_t encoder_base64_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part); +static curl_off_t encoder_base64_size(struct Curl_mimepart *part); +static size_t encoder_qp_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part); +static curl_off_t encoder_qp_size(struct Curl_mimepart *part); + +static const mime_encoder encoders[] = { + {"binary", encoder_nop_read, encoder_nop_size}, + {"8bit", encoder_nop_read, encoder_nop_size}, + {"7bit", encoder_7bit_read, encoder_nop_size}, + {"base64", encoder_base64_read, encoder_base64_size}, + {"quoted-printable", encoder_qp_read, encoder_qp_size}, + {ZERO_NULL, ZERO_NULL, ZERO_NULL} +}; + +/* Base64 encoding table */ +static const char base64[] = + "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"; + +/* Quoted-printable character class table. + * + * We cannot rely on ctype functions since quoted-printable input data + * is assumed to be ascii-compatible, even on non-ascii platforms. */ +#define QP_OK 1 /* Can be represented by itself. */ +#define QP_SP 2 /* Space or tab. */ +#define QP_CR 3 /* Carriage return. */ +#define QP_LF 4 /* Line-feed. */ +static const unsigned char qp_class[] = { + 0, 0, 0, 0, 0, 0, 0, 0, /* 00 - 07 */ + 0, QP_SP, QP_LF, 0, 0, QP_CR, 0, 0, /* 08 - 0F */ + 0, 0, 0, 0, 0, 0, 0, 0, /* 10 - 17 */ + 0, 0, 0, 0, 0, 0, 0, 0, /* 18 - 1F */ + QP_SP, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 20 - 27 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 28 - 2F */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 30 - 37 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, 0 , QP_OK, QP_OK, /* 38 - 3F */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 40 - 47 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 48 - 4F */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 50 - 57 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 58 - 5F */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 60 - 67 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 68 - 6F */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, /* 70 - 77 */ + QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, QP_OK, 0, /* 78 - 7F */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 80 - 8F */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 90 - 9F */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* A0 - AF */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* B0 - BF */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* C0 - CF */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* D0 - DF */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* E0 - EF */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 /* F0 - FF */ +}; + + +/* Binary --> hexadecimal ASCII table. */ +static const char aschex[] = + "\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x41\x42\x43\x44\x45\x46"; + + #ifndef __VMS #define filesize(name, stat_data) (stat_data.st_size) @@ -277,6 +344,279 @@ static char *strippath(const char *fullfile) return base; /* returns an allocated string or NULL ! */ } +/* Initialize data encoder state. */ +static void cleanup_encoder_state(mime_encoder_state *p) +{ + p->pos = 0; + p->bufbeg = 0; + p->bufend = 0; +} + + +/* Dummy encoder. This is used for 8bit and binary content encodings. */ +static size_t encoder_nop_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part) +{ + mime_encoder_state *st = &part->encstate; + size_t insize = st->bufend - st->bufbeg; + + (void) ateof; + + if(size > insize) + size = insize; + if(size) + memcpy(buffer, st->buf, size); + st->bufbeg += size; + return size; +} + +static curl_off_t encoder_nop_size(struct Curl_mimepart *part) +{ + return part->datasize; +} + + +/* 7bit encoder: the encoder is just a data validity check. */ +static size_t encoder_7bit_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part) +{ + size_t i; + mime_encoder_state *st = &part->encstate; + size_t cursize = st->bufend - st->bufbeg; + + (void) ateof; + + if(size > cursize) + size = cursize; + + for(cursize = 0; cursize < size; cursize++) { + *buffer = st->buf[st->bufbeg]; + if(*buffer++ & 0x80) + return cursize? cursize: READ_ERROR; + st->bufbeg++; + } + + return cursize; +} + + +/* Base64 content encoder. */ +static size_t encoder_base64_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part) +{ + mime_encoder_state *st = &part->encstate; + size_t cursize = 0; + int i; + char *ptr = buffer; + + while(st->bufbeg < st->bufend) { + /* Line full ? */ + if(st->pos >= MAX_ENCODED_LINE_LENGTH - 4) { + /* Yes, we need 2 characters for CRLF. */ + if(size < 2) + break; + *ptr++ = '\r'; + *ptr++ = '\n'; + st->pos = 0; + cursize += 2; + size -= 2; + } + + /* Be sure there is enough space and input data for a base64 group. */ + if(size < 4 || st->bufend - st->bufbeg < 3) + break; + + /* Encode three bytes a four characters. */ + i = st->buf[st->bufbeg++] & 0xFF; + i = (i << 8) | (st->buf[st->bufbeg++] & 0xFF); + i = (i << 8) | (st->buf[st->bufbeg++] & 0xFF); + *ptr++ = base64[(i >> 18) & 0x3F]; + *ptr++ = base64[(i >> 12) & 0x3F]; + *ptr++ = base64[(i >> 6) & 0x3F]; + *ptr++ = base64[i & 0x3F]; + cursize += 4; + st->pos += 4; + size -= 4; + } + + /* If at eof, we have to flush the buffered data. */ + if(ateof && size >= 4) { + /* Buffered data size can only be 0, 1 or 2. */ + ptr[2] = ptr[3] = '='; + i = 0; + switch(st->bufend - st->bufbeg) { + case 2: + i = (st->buf[st->bufbeg + 1] & 0xFF) << 8; + /* FALLTHROUGH */ + case 1: + i |= (st->buf[st->bufbeg] & 0xFF) << 16; + ptr[0] = base64[(i >> 18) & 0x3F]; + ptr[1] = base64[(i >> 12) & 0x3F]; + if(++st->bufbeg != st->bufend) { + ptr[2] = base64[(i >> 6) & 0x3F]; + st->bufbeg++; + } + cursize += 4; + st->pos += 4; + break; + } + } + +#ifdef CURL_DOES_CONVERSIONS + /* This is now textual data, Convert character codes. */ + if(part->easy && cursize) { + CURLcode result = Curl_convert_to_network(part->easy, buffer, cursize); + if(result) + return READ_ERROR; + } +#endif + + return cursize; +} + +static curl_off_t encoder_base64_size(struct Curl_mimepart *part) +{ + curl_off_t size = part->datasize; + + if(size <= 0) + return size; /* Unknown size or no data. */ + + /* Compute base64 character count. */ + size = 4 * (1 + (size - 1) / 3); + + /* Effective character count must include CRLFs. */ + return size + 2 * ((size - 1) / MAX_ENCODED_LINE_LENGTH); +} + + +/* Quoted-printable lookahead. + * + * Check if a CRLF or end of data is in input buffer at current position + n. + * Return -1 if more data needed, 1 if CRLF or end of data, else 0. + */ +static int qp_lookahead_eol(mime_encoder_state *st, int ateof, size_t n) +{ + n += st->bufbeg; + if(n >= st->bufend && ateof) + return 1; + if(n + 2 > st->bufend) + return ateof? 0: -1; + if(qp_class[st->buf[n] & 0xFF] == QP_CR && + qp_class[st->buf[n + 1] & 0xFF] == QP_LF) + return 1; + return 0; +} + +/* Quoted-printable encoder. */ +static size_t encoder_qp_read(char *buffer, size_t size, bool ateof, + struct Curl_mimepart *part) +{ + mime_encoder_state *st = &part->encstate; + char *ptr = buffer; + size_t cursize = 0; + int i; + size_t len; + size_t consumed; + int softlinebreak; + char buf[4]; + + /* On all platforms, input is supposed to be ASCII compatible: for this + reason, we use hexadecimal ASCII codes in this function rather than + character constants that can be interpreted as non-ascii on some + platforms. Preserve ASCII encoding on output too. */ + while(st->bufbeg < st->bufend) { + len = 1; + consumed = 1; + i = st->buf[st->bufbeg]; + buf[0] = (char) i; + buf[1] = aschex[(i >> 4) & 0xF]; + buf[2] = aschex[i & 0xF]; + + switch(qp_class[st->buf[st->bufbeg] & 0xFF]) { + case QP_OK: /* Not a special character. */ + break; + case QP_SP: /* Space or tab. */ + /* Spacing must be escaped if followed by CRLF. */ + switch(qp_lookahead_eol(st, ateof, 1)) { + case -1: /* More input data needed. */ + return cursize; + case 0: /* No encoding needed. */ + break; + default: /* CRLF after space or tab. */ + buf[0] = '\x3D'; /* '=' */ + len = 3; + break; + } + break; + case QP_CR: /* Carriage return. */ + /* If followed by a line-feed, output the CRLF pair. + Else escape it. */ + switch(qp_lookahead_eol(st, ateof, 0)) { + case -1: /* Need more data. */ + return cursize; + case 1: /* CRLF found. */ + buf[len++] = '\x0A'; /* Append '\n'. */ + consumed = 2; + break; + default: /* Not followed by LF: escape. */ + buf[0] = '\x3D'; /* '=' */ + len = 3; + break; + } + break; + default: /* Character must be escaped. */ + buf[0] = '\x3D'; /* '=' */ + len = 3; + break; + } + + /* Be sure the encoded character fits within maximum line length. */ + if(buf[len - 1] != '\x0A') { /* '\n' */ + softlinebreak = st->pos + len > MAX_ENCODED_LINE_LENGTH; + if(!softlinebreak && st->pos + len == MAX_ENCODED_LINE_LENGTH) { + /* We may use the current line only if end of data or followed by + a CRLF. */ + switch(qp_lookahead_eol(st, ateof, consumed)) { + case -1: /* Need more data. */ + return cursize; + break; + case 0: /* Not followed by a CRLF. */ + softlinebreak = 1; + break; + } + } + if(softlinebreak) { + strcpy(buf, "\x3D\x0D\x0A"); /* "=\r\n" */ + len = 3; + consumed = 0; + } + } + + /* If the output buffer would overflow, do not store. */ + if(len > size) + break; + + /* Append to output buffer. */ + memcpy(ptr, buf, len); + cursize += len; + ptr += len; + size -= len; + st->pos += len; + if(buf[len - 1] == '\x0A') /* '\n' */ + st->pos = 0; + st->bufbeg += consumed; + } + + return cursize; +} + +static curl_off_t encoder_qp_size(struct Curl_mimepart *part) +{ + /* Determining the size can only be done by reading the data: unless the + data size is 0, we return it as unknown (-1). */ + return part->datasize? -1: 0; +} + /* In-memory data callbacks. */ /* Argument is a pointer to the mime part. */ @@ -435,6 +775,77 @@ static size_t readback_bytes(struct mime_state *state, return sz; } +/* Read a non-encoded part content. */ +static size_t read_part_content(struct Curl_mimepart *part, + char *buffer, size_t bufsize) +{ + size_t sz = 0; + + if(part->readfunc) + sz = part->readfunc(buffer, 1, bufsize, part->arg); + return sz; +} + +/* Read and encode part content. */ +static size_t read_encoded_part_content(struct Curl_mimepart *part, + char *buffer, size_t bufsize) +{ + mime_encoder_state *st = &part->encstate; + size_t cursize = 0; + size_t sz; + bool ateof = FALSE; + + while(bufsize) { + if(st->bufbeg < st->bufend || ateof) { + /* Encode buffered data. */ + sz = part->encoder->encodefunc(buffer, bufsize, ateof, part); + switch(sz) { + case 0: + if(ateof) + return cursize; + break; + case CURL_READFUNC_ABORT: + case CURL_READFUNC_PAUSE: + case READ_ERROR: + return cursize? cursize: sz; + default: + cursize += sz; + buffer += sz; + bufsize -= sz; + continue; + } + } + + /* We need more data in input buffer. */ + if(st->bufbeg) { + size_t len = st->bufend - st->bufbeg; + + if(len) + memmove(st->buf, st->buf + st->bufbeg, len); + st->bufbeg = 0; + st->bufend = len; + } + if(st->bufend >= sizeof st->buf) + return cursize? cursize: READ_ERROR; /* Buffer full. */ + sz = read_part_content(part, st->buf + st->bufend, + sizeof st->buf - st->bufend); + switch(sz) { + case 0: + ateof = TRUE; + break; + case CURL_READFUNC_ABORT: + case CURL_READFUNC_PAUSE: + case READ_ERROR: + return cursize? cursize: sz; + default: + st->bufend += sz; + break; + } + } + + return cursize; +} + /* Readback a mime part. */ static size_t readback_part(curl_mimepart *part, char *buffer, size_t bufsize) @@ -491,11 +902,14 @@ static size_t readback_part(curl_mimepart *part, convbuf = buffer; } #endif + cleanup_encoder_state(&part->encstate); mimesetstate(&part->state, MIMESTATE_CONTENT, NULL); break; case MIMESTATE_CONTENT: - if(part->readfunc) - sz = part->readfunc(buffer, 1, bufsize, part->arg); + if(part->encoder) + sz = read_encoded_part_content(part, buffer, bufsize); + else + sz = read_part_content(part, buffer, bufsize); switch(sz) { case 0: mimesetstate(&part->state, MIMESTATE_END, NULL); @@ -634,6 +1048,7 @@ static int mime_part_rewind(curl_mimepart *part) if(part->flags & MIME_BODY_ONLY) targetstate = MIMESTATE_BODY; + cleanup_encoder_state(&part->encstate); if(part->state.state > targetstate) { res = CURL_SEEKFUNC_CANTSEEK; if(part->seekfunc) { @@ -643,6 +1058,9 @@ static int mime_part_rewind(curl_mimepart *part) case CURL_SEEKFUNC_FAIL: case CURL_SEEKFUNC_CANTSEEK: break; + case -1: /* For fseek() error. */ + res = CURL_SEEKFUNC_CANTSEEK; + break; default: res = CURL_SEEKFUNC_FAIL; break; @@ -702,6 +1120,8 @@ static void cleanup_part_content(curl_mimepart *part) part->namedfp = NULL; part->origin = 0; part->datasize = (curl_off_t) 0; /* No size yet. */ + part->encoder = NULL; + cleanup_encoder_state(&part->encstate); part->kind = MIMEKIND_NONE; } @@ -976,15 +1396,22 @@ CURLcode curl_mime_type(curl_mimepart *part, const char *mimetype) /* Set mime data transfer encoder. */ CURLcode curl_mime_encoder(curl_mimepart *part, const char *encoding) { - CURLcode result = CURLE_OK; - - /* Encoding feature not yet implemented. */ + CURLcode result = CURLE_BAD_FUNCTION_ARGUMENT; + const mime_encoder *mep; if(!part) - return CURLE_BAD_FUNCTION_ARGUMENT; + return result; - if(encoding) - return CURLE_BAD_FUNCTION_ARGUMENT; + part->encoder = NULL; + + if(!encoding) + return CURLE_OK; /* Removing current encoder. */ + + for(mep = encoders; mep->name; mep++) + if(strcasecompare(encoding, mep->name)) { + part->encoder = mep; + result = CURLE_OK; + } return result; } @@ -1130,6 +1557,10 @@ curl_off_t Curl_mime_size(curl_mimepart *part) part->datasize = multipart_size(part->arg); size = part->datasize; + + if(part->encoder) + size = part->encoder->sizefunc(part); + if(size >= 0 && !(part->flags & MIME_BODY_ONLY)) { /* Compute total part size. */ size += slist_size(part->curlheaders, 2, NULL); @@ -1219,6 +1650,7 @@ CURLcode Curl_mime_prepare_headers(curl_mimepart *part, curl_mime *mime = NULL; const char *boundary = NULL; char *s; + const char *cte = NULL; CURLcode ret = CURLE_OK; /* Get rid of previously prepared headers. */ @@ -1315,13 +1747,18 @@ CURLcode Curl_mime_prepare_headers(curl_mimepart *part, } /* Content-Transfer-Encoding header. */ - if(contenttype && strategy == MIMESTRATEGY_MAIL && - part->kind != MIMEKIND_MULTIPART && - !search_header(part->userheaders, "Content-Transfer-Encoding")) { - ret = Curl_mime_add_header(&part->curlheaders, - "Content-Transfer-Encoding: 8bit"); - if(ret) - return ret; + if(!search_header(part->userheaders, "Content-Transfer-Encoding")) { + if(part->encoder) + cte = part->encoder->name; + else if(contenttype && strategy == MIMESTRATEGY_MAIL && + part->kind != MIMEKIND_MULTIPART) + cte = "8bit"; + if(cte) { + ret = Curl_mime_add_header(&part->curlheaders, + "Content-Transfer-Encoding: %s", cte); + if(ret) + return ret; + } } /* If we were reading curl-generated headers, restart with new ones (this diff --git a/lib/mime.h b/lib/mime.h index 369f9774d..5d2e9477b 100644 --- a/lib/mime.h +++ b/lib/mime.h @@ -23,6 +23,8 @@ ***************************************************************************/ #define MIME_RAND_BOUNDARY_CHARS 16 /* Nb. of random boundary chars. */ +#define MAX_ENCODED_LINE_LENGTH 76 /* Maximum encoded line length. */ +#define ENCODING_BUFFER_SIZE 256 /* Encoding temp buffers size. */ /* Part flags. */ #define MIME_USERHEADERS_OWNER (1 << 0) @@ -60,6 +62,22 @@ enum mimestrategy { MIMESTRATEGY_LAST }; +/* Content transfer encoder. */ +typedef struct { + const char * name; /* Encoding name. */ + size_t (*encodefunc)(char *buffer, size_t size, bool ateof, + curl_mimepart *part); /* Encoded read. */ + curl_off_t (*sizefunc)(curl_mimepart *part); /* Encoded size. */ +} mime_encoder; + +/* Content transfer encoder state. */ +typedef struct { + size_t pos; /* Position on output line. */ + size_t bufbeg; /* Next data index in input buffer. */ + size_t bufend; /* First unused byte index in input buffer. */ + char buf[ENCODING_BUFFER_SIZE]; /* Input buffer. */ +} mime_encoder_state; + /* Mime readback state. */ struct mime_state { enum mimestate state; /* Current state token. */ @@ -67,7 +85,7 @@ struct mime_state { size_t offset; /* State-dependent offset. */ }; -/* A mime context. */ +/* A mime multipart. */ struct curl_mime_s { struct Curl_easy *easy; /* The associated easy handle. */ curl_mimepart *parent; /* Parent part. */ @@ -99,6 +117,8 @@ struct curl_mimepart_s { curl_off_t datasize; /* Expected data size. */ unsigned int flags; /* Flags. */ struct mime_state state; /* Current readback state. */ + const mime_encoder *encoder; /* Content data encoder. */ + mime_encoder_state encstate; /* Data encoder state. */ }; diff --git a/src/tool_formparse.c b/src/tool_formparse.c index de83d6e05..95e0980d1 100644 --- a/src/tool_formparse.c +++ b/src/tool_formparse.c @@ -172,11 +172,12 @@ static int read_field_headers(struct OperationConfig *config, static int get_param_part(struct OperationConfig *config, char **str, char **pdata, char **ptype, char **pfilename, - struct curl_slist **pheaders) + char **pencoder, struct curl_slist **pheaders) { char *p = *str; char *type = NULL; char *filename = NULL; + char *encoder = NULL; char *endpos; char *tp; char sep; @@ -191,6 +192,8 @@ static int get_param_part(struct OperationConfig *config, char **str, *pfilename = NULL; if(pheaders) *pheaders = NULL; + if(pencoder) + *pencoder = NULL; while(ISSPACE(*p)) p++; tp = p; @@ -300,6 +303,22 @@ static int get_param_part(struct OperationConfig *config, char **str, } } } + else if(checkprefix("encoder=", p)) { + if(endct) { + *endct = '\0'; + endct = NULL; + } + for(p += 8; ISSPACE(*p); p++) + ; + tp = p; + encoder = get_param_word(&p, &endpos); + /* If not quoted, strip trailing spaces. */ + if(encoder == tp) + while(endpos > encoder && ISSPACE(endpos[-1])) + endpos--; + sep = *p; + *endpos = '\0'; + } else { /* unknown prefix, skip to next block */ char *unknown = get_param_word(&p, &endpos); @@ -335,6 +354,12 @@ static int get_param_part(struct OperationConfig *config, char **str, warnf(config->global, "Field file name not allowed here: %s\n", filename); + if(pencoder) + *pencoder = encoder; + else if(encoder) + warnf(config->global, + "Field encoder not allowed here: %s\n", encoder); + if(pheaders) *pheaders = headers; else if(headers) { @@ -421,6 +446,7 @@ int formparse(struct OperationConfig *config, char *data; char *type = NULL; char *filename = NULL; + char *encoder = NULL; struct curl_slist *headers = NULL; curl_mimepart *part = NULL; CURLcode res; @@ -454,7 +480,7 @@ int formparse(struct OperationConfig *config, curl_mime *subparts; /* Starting a multipart. */ - sep = get_param_part(config, &contp, &data, &type, NULL, &headers); + sep = get_param_part(config, &contp, &data, &type, NULL, NULL, &headers); if(sep < 0) { Curl_safefree(contents); return 3; @@ -513,8 +539,8 @@ int formparse(struct OperationConfig *config, /* since this was a file, it may have a content-type specifier at the end too, or a filename. Or both. */ ++contp; - sep = get_param_part(config, - &contp, &data, &type, &filename, &headers); + sep = get_param_part(config, &contp, + &data, &type, &filename, &encoder, &headers); if(sep < 0) { if(subparts != *mimecurrent) curl_mime_free(subparts); @@ -577,13 +603,20 @@ int formparse(struct OperationConfig *config, Curl_safefree(contents); return 15; } - if(type && curl_mime_type(part, type)) { + if(curl_mime_type(part, type)) { warnf(config->global, "curl_mime_type failed!\n"); if(subparts != *mimecurrent) curl_mime_free(subparts); Curl_safefree(contents); return 16; } + if(curl_mime_encoder(part, encoder)) { + warnf(config->global, "curl_mime_encoder failed!\n"); + if(subparts != *mimecurrent) + curl_mime_free(subparts); + Curl_safefree(contents); + return 17; + } /* *contp could be '\0', so we just check with the delimiter */ } while(sep); /* loop if there's another file name */ @@ -595,13 +628,13 @@ int formparse(struct OperationConfig *config, warnf(config->global, "curl_mime_addpart failed!\n"); curl_mime_free(subparts); Curl_safefree(contents); - return 17; + return 18; } if(curl_mime_subparts(part, subparts)) { warnf(config->global, "curl_mime_subparts failed!\n"); curl_mime_free(subparts); Curl_safefree(contents); - return 18; + return 19; } } } @@ -611,16 +644,16 @@ int formparse(struct OperationConfig *config, if(!part) { warnf(config->global, "curl_mime_addpart failed!\n"); Curl_safefree(contents); - return 19; + return 20; } if(*contp == '<' && !literal_value) { ++contp; - sep = get_param_part(config, - &contp, &data, &type, &filename, &headers); + sep = get_param_part(config, &contp, + &data, &type, &filename, &encoder, &headers); if(sep < 0) { Curl_safefree(contents); - return 20; + return 21; } /* Set part headers. */ @@ -628,7 +661,7 @@ int formparse(struct OperationConfig *config, warnf(config->global, "curl_mime_headers failed!\n"); curl_slist_free_all(headers); Curl_safefree(contents); - return 21; + return 22; } /* Setup file in part. */ @@ -637,7 +670,7 @@ int formparse(struct OperationConfig *config, warnf(config->global, "setting file %s failed!\n", data); if(res != CURLE_READ_ERROR) { Curl_safefree(contents); - return 22; + return 23; } } } @@ -645,11 +678,11 @@ int formparse(struct OperationConfig *config, if(literal_value) data = contp; else { - sep = get_param_part(config, - &contp, &data, &type, &filename, &headers); + sep = get_param_part(config, &contp, + &data, &type, &filename, &encoder, &headers); if(sep < 0) { Curl_safefree(contents); - return 23; + return 24; } } @@ -658,33 +691,38 @@ int formparse(struct OperationConfig *config, warnf(config->global, "curl_mime_headers failed!\n"); curl_slist_free_all(headers); Curl_safefree(contents); - return 24; + return 25; } #ifdef CURL_DOES_CONVERSIONS if(convert_to_network(data, strlen(data))) { warnf(config->global, "curl_formadd failed!\n"); Curl_safefree(contents); - return 25; + return 26; } #endif if(curl_mime_data(part, data, CURL_ZERO_TERMINATED)) { warnf(config->global, "curl_mime_data failed!\n"); Curl_safefree(contents); - return 26; + return 27; } } if(curl_mime_filename(part, filename)) { warnf(config->global, "curl_mime_filename failed!\n"); Curl_safefree(contents); - return 27; + return 28; } if(curl_mime_type(part, type)) { warnf(config->global, "curl_mime_type failed!\n"); Curl_safefree(contents); - return 28; + return 29; + } + if(curl_mime_encoder(part, encoder)) { + warnf(config->global, "curl_mime_encoder failed!\n"); + Curl_safefree(contents); + return 30; } if(sep) { @@ -698,13 +736,13 @@ int formparse(struct OperationConfig *config, if(name && curl_mime_name(part, name, CURL_ZERO_TERMINATED)) { warnf(config->global, "curl_mime_name failed!\n"); Curl_safefree(contents); - return 29; + return 31; } } else { warnf(config->global, "Illegally formatted input field!\n"); Curl_safefree(contents); - return 30; + return 32; } Curl_safefree(contents); return 0; diff --git a/src/tool_setopt.c b/src/tool_setopt.c index b0f319814..635304a8f 100644 --- a/src/tool_setopt.c +++ b/src/tool_setopt.c @@ -506,6 +506,14 @@ static CURLcode libcurl_generate_mime(curl_mime *mime, int *mimeno) break; } + if(part->encoder) { + Curl_safefree(escaped); + escaped = c_escape(part->encoder->name, CURL_ZERO_TERMINATED); + if(!escaped) + return CURLE_OUT_OF_MEMORY; + CODE2("curl_mime_encoder(part%d, \"%s\");", *mimeno, escaped); + } + if(filename) { Curl_safefree(escaped); escaped = c_escape(filename, CURL_ZERO_TERMINATED); diff --git a/tests/data/Makefile.inc b/tests/data/Makefile.inc index 464faeeaf..d7ea5c951 100644 --- a/tests/data/Makefile.inc +++ b/tests/data/Makefile.inc @@ -78,7 +78,7 @@ test608 test609 test610 test611 test612 test613 test614 test615 test616 \ test617 test618 test619 test620 test621 test622 test623 test624 test625 \ test626 test627 test628 test629 test630 test631 test632 test633 test634 \ test635 test636 test637 test638 test639 test640 test641 test642 \ -test643 test644 test645 test646 test647 \ +test643 test644 test645 test646 test647 test648 test649 \ \ test700 test701 test702 test703 test704 test705 test706 test707 test708 \ test709 test710 test711 test712 test713 test714 test715 \ diff --git a/tests/data/test1404 b/tests/data/test1404 index a3ec66ef7..20dc01333 100644 --- a/tests/data/test1404 +++ b/tests/data/test1404 @@ -27,13 +27,13 @@ Connection: close http ---libcurl for HTTP RFC1867-type formposting - -F with three files, one with explicit type +--libcurl for HTTP RFC1867-type formposting - -F with 3 files, one with explicit type & encoder SSL_CERT_FILE= -http://%HOSTIP:%HTTPPORT/we/want/1404 -F name=value -F 'file=@log/test1404.txt,log/test1404.txt;type=magic/content,log/test1404.txt;headers=X-testheader-1: header 1;headers=X-testheader-2: header 2' --libcurl log/test1404.c +http://%HOSTIP:%HTTPPORT/we/want/1404 -F name=value -F 'file=@log/test1404.txt,log/test1404.txt;type=magic/content;encoder=8bit,log/test1404.txt;headers=X-testheader-1: header 1;headers=X-testheader-2: header 2' --libcurl log/test1404.c # We create this file before the command is invoked! @@ -51,7 +51,7 @@ POST /we/want/1404 HTTP/1.1 User-Agent: curl/7.18.2 (i686-pc-linux-gnu) libcurl/7.18.2 OpenSSL/0.9.7a ipv6 zlib/1.1.4 Host: %HOSTIP:%HTTPPORT Accept: */* -Content-Length: 849 +Content-Length: 882 Content-Type: multipart/form-data; boundary=----------------------------9ef8d6205763 ------------------------------9ef8d6205763 @@ -70,6 +70,7 @@ dummy data ------------------------------9ef8d6205763 Content-Disposition: attachment; filename="test1404.txt" Content-Type: magic/content +Content-Transfer-Encoding: 8bit dummy data @@ -131,6 +132,7 @@ int main(int argc, char *argv[]) curl_mime_filedata(part2, "log/test1404.txt"); part2 = curl_mime_addpart(mime2); curl_mime_filedata(part2, "log/test1404.txt"); + curl_mime_encoder(part2, "8bit"); curl_mime_type(part2, "magic/content"); part2 = curl_mime_addpart(mime2); curl_mime_filedata(part2, "log/test1404.txt"); diff --git a/tests/data/test648 b/tests/data/test648 new file mode 100644 index 000000000..cd8f02085 --- /dev/null +++ b/tests/data/test648 @@ -0,0 +1,75 @@ + + + +SMTP +MULTIPART + + + +# +# Server-side + + + +# +# Client-side + + +smtp + + +SMTP multipart with transfer content encoders + + +From: different +To: another + +body + + +smtp://%HOSTIP:%SMTPPORT/648 --mail-rcpt recipient@example.com --mail-from sender@example.com -F '=This is the e-mail inline text with a very long line containing the special character = and that should be split by encoder.;headers=Content-disposition: "inline";encoder=quoted-printable' -F "=@log/test648.txt;encoder=base64" -H "From: different" -H "To: another" + + +This is an attached file. + +It may contain any type of data and will be encoded in base64 for transfer. + + + +# +# Verify data after the test has been "shot" + + +s/^--------------------------[a-z0-9]*/------------------------------/ +s/boundary=------------------------[a-z0-9]*/boundary=----------------------------/ + + +EHLO 648 +MAIL FROM: +RCPT TO: +DATA +QUIT + + +Content-Type: multipart/mixed; boundary=---------------------------- +Mime-Version: 1.0 +From: different +To: another + +------------------------------ +Content-Transfer-Encoding: quoted-printable +Content-disposition: "inline" + +This is the e-mail inline text with a very long line containing the special= + character =3D and that should be split by encoder. +------------------------------ +Content-Disposition: attachment; filename="test648.txt" +Content-Transfer-Encoding: base64 + +VGhpcyBpcyBhbiBhdHRhY2hlZCBmaWxlLgoKSXQgbWF5IGNvbnRhaW4gYW55IHR5cGUgb2Yg +ZGF0YSBhbmQgd2lsbCBiZSBlbmNvZGVkIGluIGJhc2U2NCBmb3IgdHJhbnNmZXIuCg== +-------------------------------- +. + + + diff --git a/tests/data/test649 b/tests/data/test649 new file mode 100644 index 000000000..46c01cd00 --- /dev/null +++ b/tests/data/test649 @@ -0,0 +1,72 @@ + + + +SMTP +MULTIPART + + + +# +# Server-side + + + +# +# Client-side + + +smtp + + +SMTP multipart with 7bit encoder error + + +From: different +To: another + +body + + +smtp://%HOSTIP:%SMTPPORT/649 --mail-rcpt recipient@example.com --mail-from sender@example.com -F '=This is valid;encoder=7bit' -F "=@log/test649.txt;encoder=7bit" -H "From: different" -H "To: another" + + +This is an attached file (in french: pièce jointe). + +It contains at least an 8-bit byte value. + + + +# +# Verify data after the test has been "shot" + + +s/^--------------------------[a-z0-9]*/------------------------------/ +s/boundary=------------------------[a-z0-9]*/boundary=----------------------------/ + + +EHLO 649 +MAIL FROM: +RCPT TO: +DATA + + +Content-Type: multipart/mixed; boundary=---------------------------- +Mime-Version: 1.0 +From: different +To: another + +------------------------------ +Content-Transfer-Encoding: 7bit + +This is valid +------------------------------ +Content-Disposition: attachment; filename="test649.txt" +Content-Transfer-Encoding: 7bit + +This is an attached file (in french: pi + + +26 + + + -- cgit v1.2.1