From 7c37c6a8e9b7185fe7c7a607fe12931a226faa06 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Mon, 22 May 2000 17:35:35 +0000 Subject: moved here from the root directory --- docs/INTERNALS | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 docs/INTERNALS (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS new file mode 100644 index 000000000..0badf5b29 --- /dev/null +++ b/docs/INTERNALS @@ -0,0 +1,140 @@ + _ _ ____ _ + ___| | | | _ \| | + / __| | | | |_) | | + | (__| |_| | _ <| |___ + \___|\___/|_| \_\_____| + +INTERNALS + + The project is kind of split in two. The library and the client. The client + part uses the library, but the library is meant to be designed to allow other + applications to use it. + + Thus, the largest amount of code and complexity is in the library part. + +Windows vs Unix +=============== + + There are a few differences in how to program curl the unix way compared to + the Windows way. The four most notable details are: + + 1. Different function names for close(), read(), write() + 2. Windows requires a couple of init calls + 3. The file descriptors for network communication and file operations are + not easily interchangable as in unix + 4. When writing data to stdout, Windows makes end-of-lines the DOS way, thus + destroying binary data, although you do want that conversion if it is + text coming through... (sigh) + + In curl, (1) and (2) are done with defines and macros, so that the source + looks the same at all places except for the header file that defines them. + + (3) is simply avoided by not trying any funny tricks on file descriptors. + + (4) is left alone, giving windows users problems when they pipe binary data + through stdout... + + Inside the source code, I do make an effort to avoid '#ifdef WIN32'. All + conditionals that deal with features *should* instead be in the format + '#ifdef HAVE_THAT_WEIRD_FUNCTION'. Since Windows can't run configure scripts, + I maintain two config-win32.h files (one in / and one in src/) that are + supposed to look exactly as a config.h file would have looked like on a + Windows machine! + +Library +======= + + There is a few entry points to the library, namely each publicly defined + function that libcurl offers to applications. All of those functions are + rather small and easy-to-follow, accept the one single and do-it-all named + curl_urlget() (entry point in lib/url.c). + + curl_urlget() takes a variable amount of arguments, and they must all be + passed in pairs, the parameter-ID and the parameter-value. The list of + arguments must be ended with a end-of-arguments parameter-ID. + + The function then continues to analyze the URL, get the different components + and connects to the remote host. This may involve using a proxy and/or using + SSL. The GetHost() function in lib/hostip.c is used for looking up host + names. + + When connected, the proper function is called. The functions are named after + the protocols they handle. ftp(), http(), dict(), etc. They all reside in + their respective files (ftp.c, http.c and dict.c). + + The protocol-specific functions deal with protocol-specific negotiations and + setup. They have access to the sendf() (from lib/sendf.c) function to send + printf-style formatted data to the remote host and when they're ready to make + the actual file transfer they call the Transfer() function (in + lib/download.c) to do the transfer. All printf()-style functions use the + supplied clones in lib/mprintf.c. + + While transfering, the progress functions in lib/progress.c are called at a + frequent interval. The speedcheck functions in lib/speedcheck.c are also used + to verify that the transfer is as fast as required. + + When the operation is done, the writeout() function in lib/writeout.c may be + called to report about the operation as specified previously in the arguments + to curl_urlget(). + + HTTP(S) + + HTTP offers a lot and is the protocol in curl that uses the most lines of + code. There is a special file (lib/formdata.c) that offers all the multipart + post functions. + + base64-functions for user+password stuff is in (lib/base64.c) and all + functions for parsing and sending cookies are found in + (lib/cookie.c). + + HTTPS uses in almost every means the same procedure as HTTP, with only two + exceptions: the connect procedure is different and the function used + + FTP + + The if2ip() function can be used for getting the IP number of a specified + network interface, and it resides in lib/if2ip.c + + TELNET + + Telnet is implemented in lib/telnet.c. + + FILE + + The file:// protocol is dealt with in lib/file.c. + + LDAP + + Everything LDAP is in lib/ldap.c. + + GENERAL + + URL encoding and decoding, called escaping and unescaping in the source code, + is found in lib/escape.c. + + While transfering data in Transfer() a few functions might get + used. get_date() in lib/getdate.c is for HTTP date comparisons. + + lib/getenv.c is for reading environment variables in a neat platform + independent way. That's used in the client, but also in lib/url.c when + checking the PROXY variables. + + lib/netrc.c keeps the .netrc parser + + lib/timeval.c features replacement functions for systems that don't have + + A function named curl_version() that returns the full curl version string is + found in lib/version.c. + +Client +====== + + main() resides in src/main.c together with most of the client + code. src/hugehelp.c is automatically generated by the mkhelp.pl perl script + to display the complete "manual" and the src/urlglob.c file holds the + functions used for the multiple-URL support. + + The client mostly mess around to setup its config struct properly, then it + calls the curl_urlget() function in the library and when it gets back control + it checks status and exits. + -- cgit v1.2.1 From 3b60784f27aec4255d10e5012b321f4e13deee0a Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 14 Jun 2000 09:16:11 +0000 Subject: updated to be more valid for version 7 workings --- docs/INTERNALS | 66 ++++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 43 insertions(+), 23 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 0badf5b29..2a7abff42 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -19,15 +19,18 @@ Windows vs Unix the Windows way. The four most notable details are: 1. Different function names for close(), read(), write() - 2. Windows requires a couple of init calls + 2. Windows requires a couple of init calls for the socket stuff 3. The file descriptors for network communication and file operations are not easily interchangable as in unix 4. When writing data to stdout, Windows makes end-of-lines the DOS way, thus destroying binary data, although you do want that conversion if it is text coming through... (sigh) - In curl, (1) and (2) are done with defines and macros, so that the source - looks the same at all places except for the header file that defines them. + In curl, (1) is made with defines and macros, so that the source looks the + same at all places except for the header file that defines them. + + (2) must be made by the application that uses libcurl, in curl that means + src/main.c has some code #ifdef'ed to do just that. (3) is simply avoided by not trying any funny tricks on file descriptors. @@ -44,19 +47,26 @@ Windows vs Unix Library ======= - There is a few entry points to the library, namely each publicly defined + As described elsewhere, libcurl is meant to get two different "layers" of + interface. At the present point only the high-level, the "easy", interface + has been fully implemented and thus documented. We assume the easy-interface + in this description, the low-level interface will be documented when fully + implemented. + + There are plenty of entry points to the library, namely each publicly defined function that libcurl offers to applications. All of those functions are - rather small and easy-to-follow, accept the one single and do-it-all named - curl_urlget() (entry point in lib/url.c). + rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are + put in the lib/easy.c file. - curl_urlget() takes a variable amount of arguments, and they must all be + curl_easy_setopt() takes a three arguments, where the option stuff must be passed in pairs, the parameter-ID and the parameter-value. The list of - arguments must be ended with a end-of-arguments parameter-ID. + options is documented in the man page. - The function then continues to analyze the URL, get the different components - and connects to the remote host. This may involve using a proxy and/or using - SSL. The GetHost() function in lib/hostip.c is used for looking up host - names. + curl_easy_perform() does a whole lot of things. + + The function analyzes the URL, get the different components and connects to + the remote host. This may involve using a proxy and/or using SSL. The + GetHost() function in lib/hostip.c is used for looking up host names. When connected, the proper function is called. The functions are named after the protocols they handle. ftp(), http(), dict(), etc. They all reside in @@ -70,12 +80,16 @@ Library supplied clones in lib/mprintf.c. While transfering, the progress functions in lib/progress.c are called at a - frequent interval. The speedcheck functions in lib/speedcheck.c are also used - to verify that the transfer is as fast as required. + frequent interval (or at the user's choice, a specified callback might get + called). The speedcheck functions in lib/speedcheck.c are also used to verify + that the transfer is as fast as required. When the operation is done, the writeout() function in lib/writeout.c may be called to report about the operation as specified previously in the arguments - to curl_urlget(). + to curl_easy_setopt(). + + When completed curl_easy_cleanup() should be called to free up used + resources. HTTP(S) @@ -88,12 +102,16 @@ Library (lib/cookie.c). HTTPS uses in almost every means the same procedure as HTTP, with only two - exceptions: the connect procedure is different and the function used + exceptions: the connect procedure is different and the function used to read + or write from the socket is different, although the latter fact is hidden in + the source by the use of curl_read() for reading and curl_write() for writing + data to the remote server. FTP The if2ip() function can be used for getting the IP number of a specified - network interface, and it resides in lib/if2ip.c + network interface, and it resides in lib/if2ip.c. It is only used for the FTP + PORT command. TELNET @@ -113,11 +131,12 @@ Library is found in lib/escape.c. While transfering data in Transfer() a few functions might get - used. get_date() in lib/getdate.c is for HTTP date comparisons. + used. curl_getdate() in lib/getdate.c is for HTTP date comparisons (and + more). - lib/getenv.c is for reading environment variables in a neat platform - independent way. That's used in the client, but also in lib/url.c when - checking the PROXY variables. + lib/getenv.c offers curl_getenv() which is for reading environment variables + in a neat platform independent way. That's used in the client, but also in + lib/url.c when checking the PROXY variables. lib/netrc.c keeps the .netrc parser @@ -135,6 +154,7 @@ Client functions used for the multiple-URL support. The client mostly mess around to setup its config struct properly, then it - calls the curl_urlget() function in the library and when it gets back control - it checks status and exits. + calls the curl_easy_*() functions of the library and when it gets back + control after the curl_easy_perform() it cleans up the library, checks status + and exits. -- cgit v1.2.1 From b2e47dfde4a92a8365ff49e5a0b981dc3e05dab7 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 11 Oct 2000 10:59:36 +0000 Subject: updated to better reflect reality --- docs/INTERNALS | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 2a7abff42..ed79d3853 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -34,8 +34,7 @@ Windows vs Unix (3) is simply avoided by not trying any funny tricks on file descriptors. - (4) is left alone, giving windows users problems when they pipe binary data - through stdout... + (4) we set stdout to binary under windows Inside the source code, I do make an effort to avoid '#ifdef WIN32'. All conditionals that deal with features *should* instead be in the format @@ -84,10 +83,6 @@ Library called). The speedcheck functions in lib/speedcheck.c are also used to verify that the transfer is as fast as required. - When the operation is done, the writeout() function in lib/writeout.c may be - called to report about the operation as specified previously in the arguments - to curl_easy_setopt(). - When completed curl_easy_cleanup() should be called to free up used resources. @@ -158,3 +153,9 @@ Client control after the curl_easy_perform() it cleans up the library, checks status and exits. + When the operation is done, the ourWriteOut() function in src/writeout.c may + be called to report about the operation. That function is using the + curl_easy_getinfo() function to extract useful information from the curl + session. + + -- cgit v1.2.1 From a5217dd10ec0ec0a5f4a7da5c0683ad727c39da6 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 29 Nov 2000 07:47:51 +0000 Subject: minor things about the test suite added --- docs/INTERNALS | 47 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 10 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index ed79d3853..5fe8b29c9 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -12,6 +12,17 @@ INTERNALS Thus, the largest amount of code and complexity is in the library part. +CVS +=== + + All changes to the sources are committed to the CVS repository as soon as + they're somewhat verified to work. Changes shall be commited as independently + as possible so that individual changes can be easier spotted and tracked + afterwards. + + Tagging shall be used extensively, and by the time we release new archives we + should tag the sources with a name similar to the released version number. + Windows vs Unix =============== @@ -47,9 +58,9 @@ Library ======= As described elsewhere, libcurl is meant to get two different "layers" of - interface. At the present point only the high-level, the "easy", interface - has been fully implemented and thus documented. We assume the easy-interface - in this description, the low-level interface will be documented when fully + interfaces. At the present point only the high-level, the "easy", interface + has been fully implemented and documented. We assume the easy-interface in + this description, the low-level interface will be documented when fully implemented. There are plenty of entry points to the library, namely each publicly defined @@ -57,11 +68,14 @@ Library rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are put in the lib/easy.c file. + curl_easy_init() allocates an internal struct and makes some initializations. + The returned handle does not revail internals. + curl_easy_setopt() takes a three arguments, where the option stuff must be passed in pairs, the parameter-ID and the parameter-value. The list of options is documented in the man page. - curl_easy_perform() does a whole lot of things. + curl_easy_perform() does a whole lot of things: The function analyzes the URL, get the different components and connects to the remote host. This may involve using a proxy and/or using SSL. The @@ -131,11 +145,12 @@ Library lib/getenv.c offers curl_getenv() which is for reading environment variables in a neat platform independent way. That's used in the client, but also in - lib/url.c when checking the PROXY variables. + lib/url.c when checking the proxy environment variables. - lib/netrc.c keeps the .netrc parser + lib/netrc.c holds the .netrc parser lib/timeval.c features replacement functions for systems that don't have + gettimeofday(). A function named curl_version() that returns the full curl version string is found in lib/version.c. @@ -143,10 +158,10 @@ Library Client ====== - main() resides in src/main.c together with most of the client - code. src/hugehelp.c is automatically generated by the mkhelp.pl perl script - to display the complete "manual" and the src/urlglob.c file holds the - functions used for the multiple-URL support. + main() resides in src/main.c together with most of the client code. + src/hugehelp.c is automatically generated by the mkhelp.pl perl script to + display the complete "manual" and the src/urlglob.c file holds the functions + used for the multiple-URL support. The client mostly mess around to setup its config struct properly, then it calls the curl_easy_*() functions of the library and when it gets back @@ -158,4 +173,16 @@ Client curl_easy_getinfo() function to extract useful information from the curl session. +Test Suite +========== + + During November 2000, a test suite has evolved. It is placed in its own + subdirectory directly off the root in the curl archive tree, and it contains + a bunch of scripts and a lot of test case data. + + The main test script is runtests.pl that will invoke the two servers + httpserver.pl and ftpserver.pl before all the test cases are performed. The + test suite currently only runs on unix-like platforms. + You'll find a complete description of the test case data files in the README + file in the test directory. -- cgit v1.2.1 From 4031104404c6ceed5e57134125dcdb6cac51c564 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 5 Jan 2001 10:11:41 +0000 Subject: Internal symbols that aren't static are now prefixed with 'Curl_' --- docs/INTERNALS | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 5fe8b29c9..f134b4678 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -12,6 +12,13 @@ INTERNALS Thus, the largest amount of code and complexity is in the library part. +SYMBOLS +======= + All symbols used internally must use a 'Curl_' prefix if they're used in + more than a single file. Single file symbols must be made static. Public + symbols must use a 'curl_' prefix. (There are exceptions, but they are + destined to change to this pattern in the future.) + CVS === -- cgit v1.2.1 From 444024ea1494ceab7747ba0e99ac9382fdbac957 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 17 Jan 2001 14:17:49 +0000 Subject: brought up-to-date and extended --- docs/INTERNALS | 144 ++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 108 insertions(+), 36 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index f134b4678..de86315ac 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -14,14 +14,13 @@ INTERNALS SYMBOLS ======= - All symbols used internally must use a 'Curl_' prefix if they're used in - more than a single file. Single file symbols must be made static. Public - symbols must use a 'curl_' prefix. (There are exceptions, but they are - destined to change to this pattern in the future.) + All symbols used internally must use a 'Curl_' prefix if they're used in more + than a single file. Single-file symbols must be made static. Public + (exported) symbols must use a 'curl_' prefix. (There are exceptions, but they + are destined to be changed to follow this pattern in the future.) CVS === - All changes to the sources are committed to the CVS repository as soon as they're somewhat verified to work. Changes shall be commited as independently as possible so that individual changes can be easier spotted and tracked @@ -34,7 +33,7 @@ Windows vs Unix =============== There are a few differences in how to program curl the unix way compared to - the Windows way. The four most notable details are: + the Windows way. The four perhaps most notable details are: 1. Different function names for close(), read(), write() 2. Windows requires a couple of init calls for the socket stuff @@ -61,6 +60,9 @@ Windows vs Unix supposed to look exactly as a config.h file would have looked like on a Windows machine! + Generally speaking: always remember that this will be compiled on dozens of + operating systems. Don't walk on the edge. + Library ======= @@ -75,6 +77,9 @@ Library rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are put in the lib/easy.c file. + All printf()-style functions use the supplied clones in lib/mprintf.c. This + makes sure we stay absolutely platform independent. + curl_easy_init() allocates an internal struct and makes some initializations. The returned handle does not revail internals. @@ -84,27 +89,31 @@ Library curl_easy_perform() does a whole lot of things: - The function analyzes the URL, get the different components and connects to - the remote host. This may involve using a proxy and/or using SSL. The - GetHost() function in lib/hostip.c is used for looking up host names. - - When connected, the proper function is called. The functions are named after - the protocols they handle. ftp(), http(), dict(), etc. They all reside in - their respective files (ftp.c, http.c and dict.c). - - The protocol-specific functions deal with protocol-specific negotiations and - setup. They have access to the sendf() (from lib/sendf.c) function to send - printf-style formatted data to the remote host and when they're ready to make - the actual file transfer they call the Transfer() function (in - lib/download.c) to do the transfer. All printf()-style functions use the - supplied clones in lib/mprintf.c. - - While transfering, the progress functions in lib/progress.c are called at a + It starts off in the lib/easy.c file by calling curl_transfer(), but the main + work is lib/url.c. The function first analyzes the URL, it separates the + different components and connects to the remote host. This may involve using + a proxy and/or using SSL. The Curl_gethost() function in lib/hostip.c is used + for looking up host names. + + When connected, the proper protocol-specific function is called. The + functions are named after the protocols they handle. Curl_ftp(), Curl_http(), + Curl_dict(), etc. They all reside in their respective files (ftp.c, http.c + and dict.c). + + The protocol-specific functions of course deal with protocol-specific + negotiations and setup. They have access to the Curl_sendf() (from + lib/sendf.c) function to send printf-style formatted data to the remote host + and when they're ready to make the actual file transfer they call the + Curl_Transfer() function (in lib/transfer.c) to setup the transfer and + returns. curl_transfer() then calls _Tranfer() in lib/transfer.c that + performs the entire file transfer. + + During transfer, the progress functions in lib/progress.c are called at a frequent interval (or at the user's choice, a specified callback might get called). The speedcheck functions in lib/speedcheck.c are also used to verify that the transfer is as fast as required. - When completed curl_easy_cleanup() should be called to free up used + When completed, the curl_easy_cleanup() should be called to free up used resources. HTTP(S) @@ -113,9 +122,8 @@ Library code. There is a special file (lib/formdata.c) that offers all the multipart post functions. - base64-functions for user+password stuff is in (lib/base64.c) and all - functions for parsing and sending cookies are found in - (lib/cookie.c). + base64-functions for user+password stuff (and more) is in (lib/base64.c) and + all functions for parsing and sending cookies are found in (lib/cookie.c). HTTPS uses in almost every means the same procedure as HTTP, with only two exceptions: the connect procedure is different and the function used to read @@ -125,9 +133,17 @@ Library FTP - The if2ip() function can be used for getting the IP number of a specified - network interface, and it resides in lib/if2ip.c. It is only used for the FTP - PORT command. + The Curl_if2ip() function can be used for getting the IP number of a + specified network interface, and it resides in lib/if2ip.c. + + Curl_ftpsendf() is used for sending FTP commands to the remote server. It was + made a separate function to prevent us programmers from forgetting that they + must be CRLF terminated. They must also be sent in one single write() to make + firewalls and similar happy. + + Kerberos + + The kerberos support is mainly in lib/krb4.c and lib/security.c. TELNET @@ -146,32 +162,54 @@ Library URL encoding and decoding, called escaping and unescaping in the source code, is found in lib/escape.c. - While transfering data in Transfer() a few functions might get + While transfering data in _Transfer() a few functions might get used. curl_getdate() in lib/getdate.c is for HTTP date comparisons (and more). lib/getenv.c offers curl_getenv() which is for reading environment variables in a neat platform independent way. That's used in the client, but also in - lib/url.c when checking the proxy environment variables. + lib/url.c when checking the proxy environment variables. Note that contrary + to the normal unix getenv(), this returns an allocated buffer that must be + free()ed after use. lib/netrc.c holds the .netrc parser lib/timeval.c features replacement functions for systems that don't have - gettimeofday(). + gettimeofday() and a few support functions for timeval convertions. A function named curl_version() that returns the full curl version string is found in lib/version.c. + If authentication is requested but no password is given, a getpass_r() clone + exists in lib/getpass.c. libcurl offers a custom callback that can be used + instead of this, but it doesn't change much to us. + +Return Codes and Informationals +=============================== + + I've made things simple. Almost every function in libcurl returns a CURLcode, + that must be CURLE_OK if everything is OK or otherwise a suitable error code + as the curl/curl.h include file defines. The very spot that detects an error + must use the Curl_failf() function to set the human-readable error + description. + + In aiding the user to understand what's happening and to debug curl usage, we + must supply a fair amount of informational messages by using the Curl_infof() + function. Those messages are only displayed when the user explicitly asks for + them. They are best used when revealing information that isn't otherwise + obvious. + Client ====== main() resides in src/main.c together with most of the client code. src/hugehelp.c is automatically generated by the mkhelp.pl perl script to display the complete "manual" and the src/urlglob.c file holds the functions - used for the multiple-URL support. + used for the URL-"globbing" support. Globbing in the sense that the {} and [] + expansion stuff is there. - The client mostly mess around to setup its config struct properly, then it - calls the curl_easy_*() functions of the library and when it gets back + The client mostly messes around to setup its 'config' struct properly, then + it calls the curl_easy_*() functions of the library and when it gets back control after the curl_easy_perform() it cleans up the library, checks status and exits. @@ -180,10 +218,30 @@ Client curl_easy_getinfo() function to extract useful information from the curl session. + Recent versions may loop and do all that several times if many URLs were + specified on the command line or config file. + +Memory Debugging +================ + + The file named lib/memdebug.c contains debug-versions of a few + functions. Functions such as malloc, free, fopen, fclose, etc that somehow + deal with resources that might give us problems if we "leak" them. The + functions in the memdebug system do nothing fancy, they do their normal + function and then log information about what they just did. The logged data + is then analyzed after a complete session, + + memanalyze.pl is a perl script present only in CVS (not part of the release + archives) that analyzes a log file generated by the memdebug system. It + detects if resources are allocated but never freed and other kinds of errors + related to resource management. + + Use -DMALLOCDEBUG when compiling to enable memory debugging. + Test Suite ========== - During November 2000, a test suite has evolved. It is placed in its own + Since November 2000, a test suite has evolved. It is placed in its own subdirectory directly off the root in the curl archive tree, and it contains a bunch of scripts and a lot of test case data. @@ -193,3 +251,17 @@ Test Suite You'll find a complete description of the test case data files in the README file in the test directory. + + The test suite automatically detects if curl was built with the memory + debugging enabled, and if it was it will detect memory leaks too. + +Building Releases +================= + + There's no magic to this. When you consider everything stable enough to be + released, run the 'maketgz' script (using 'make distcheck' will give you a + pretty good view on the status of the current sources). maketgz prompts for + version number of the client and the library before it creates a release + archive. + + You must have autoconf installed to build release archives. -- cgit v1.2.1 From 9c695393b28e78890e8228e8ca41048ddf997bf8 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 19 Jan 2001 09:37:39 +0000 Subject: edited the portable code section --- docs/INTERNALS | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index de86315ac..b35298dea 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -36,22 +36,25 @@ Windows vs Unix the Windows way. The four perhaps most notable details are: 1. Different function names for close(), read(), write() + + In curl, this is solved with defines and macros, so that the source looks + the same at all places except for the header file that defines them. + 2. Windows requires a couple of init calls for the socket stuff + + Those must be made by the application that uses libcurl, in curl that means + src/main.c has some code #ifdef'ed to do just that. + 3. The file descriptors for network communication and file operations are not easily interchangable as in unix + + We avoid this by not trying any funny tricks on file descriptors. + 4. When writing data to stdout, Windows makes end-of-lines the DOS way, thus destroying binary data, although you do want that conversion if it is text coming through... (sigh) - In curl, (1) is made with defines and macros, so that the source looks the - same at all places except for the header file that defines them. - - (2) must be made by the application that uses libcurl, in curl that means - src/main.c has some code #ifdef'ed to do just that. - - (3) is simply avoided by not trying any funny tricks on file descriptors. - - (4) we set stdout to binary under windows + We set stdout to binary under windows Inside the source code, I do make an effort to avoid '#ifdef WIN32'. All conditionals that deal with features *should* instead be in the format -- cgit v1.2.1 From e8382ba290025f941eff42d23b6d9237b3fb8c25 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 26 Jan 2001 15:52:51 +0000 Subject: moved the symbols talk to the library part, updated slightly to match --- docs/INTERNALS | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index b35298dea..172eb2cf9 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,3 +1,4 @@ + Updated for curl 7.6 on January 26, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -12,13 +13,6 @@ INTERNALS Thus, the largest amount of code and complexity is in the library part. -SYMBOLS -======= - All symbols used internally must use a 'Curl_' prefix if they're used in more - than a single file. Single-file symbols must be made static. Public - (exported) symbols must use a 'curl_' prefix. (There are exceptions, but they - are destined to be changed to follow this pattern in the future.) - CVS === All changes to the sources are committed to the CVS repository as soon as @@ -35,10 +29,11 @@ Windows vs Unix There are a few differences in how to program curl the unix way compared to the Windows way. The four perhaps most notable details are: - 1. Different function names for close(), read(), write() + 1. Different function names for socket operations. In curl, this is solved with defines and macros, so that the source looks - the same at all places except for the header file that defines them. + the same at all places except for the header file that defines them. The + macros in use are sclose(), sread() and swrite(). 2. Windows requires a couple of init calls for the socket stuff @@ -187,6 +182,15 @@ Library exists in lib/getpass.c. libcurl offers a custom callback that can be used instead of this, but it doesn't change much to us. +Library Symbols +=============== + + All symbols used internally in libcurl must use a 'Curl_' prefix if they're + used in more than a single file. Single-file symbols must be made + static. Public (exported) symbols must use a 'curl_' prefix. (There are + exceptions, but they are destined to be changed to follow this pattern in the + future.) + Return Codes and Informationals =============================== -- cgit v1.2.1 From 7d562bb6859518ee15806b7bdc03053e0380c52b Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Tue, 13 Mar 2001 08:16:54 +0000 Subject: a whole new section on persitant connections and how they're treated internally --- docs/INTERNALS | 46 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 8 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 172eb2cf9..ddba56ee3 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,4 +1,4 @@ - Updated for curl 7.6 on January 26, 2001 + Updated for curl 7.7 on March 13, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -103,8 +103,9 @@ Library lib/sendf.c) function to send printf-style formatted data to the remote host and when they're ready to make the actual file transfer they call the Curl_Transfer() function (in lib/transfer.c) to setup the transfer and - returns. curl_transfer() then calls _Tranfer() in lib/transfer.c that - performs the entire file transfer. + returns. Curl_perform() then calls Transfer() in lib/transfer.c that performs + the entire file transfer. Curl_perform() is what does the main "connect - do + - transfer - done" loop. It loops if there's a Location: to follow. During transfer, the progress functions in lib/progress.c are called at a frequent interval (or at the user's choice, a specified callback might get @@ -160,7 +161,7 @@ Library URL encoding and decoding, called escaping and unescaping in the source code, is found in lib/escape.c. - While transfering data in _Transfer() a few functions might get + While transfering data in Transfer() a few functions might get used. curl_getdate() in lib/getdate.c is for HTTP date comparisons (and more). @@ -182,6 +183,34 @@ Library exists in lib/getpass.c. libcurl offers a custom callback that can be used instead of this, but it doesn't change much to us. +Persistant Connections +====================== + + With curl 7.7, we added persistant connection support to libcurl which has + introduced a somewhat different treatmeant of things inside of libcurl. + + o The 'UrlData' struct returned in the curl_easy_init() call must never + hold connection-oriented data. It is meant to hold the root data as well + as all the options etc that the library-user may choose. + o The 'UrlData' struct holds the cache array of pointers to 'connectdata' + structs. There's one connectdata struct for each connection that libcurl + knows about. + o This also enables the 'curl handle' to be reused on subsequent transfers, + something that was illegal in pre-7.7 versions. + o When we are about to perform a transfer with curl_easy_perform(), we first + check for an already existing connection in the cache that we can use, + otherwise we create a new one and add to the cache. If the cache is full + already when we add a new connection, we close one of the present ones. We + select which one to close dependent on the close policy that may have been + previously set. + o When the tranfer operation is complete, we try to leave the connection open. + Particular options may tell us not to, and protocols may signal closure on + connections and then we don't keep it open of course. + o When curl_easy_cleanup() is called, we close all still opened connections. + + You do realize that the curl handle must be re-used in order for the + persistant connections to work. + Library Symbols =============== @@ -256,8 +285,8 @@ Test Suite httpserver.pl and ftpserver.pl before all the test cases are performed. The test suite currently only runs on unix-like platforms. - You'll find a complete description of the test case data files in the README - file in the test directory. + You'll find a complete description of the test case data files in the + tests/README file. The test suite automatically detects if curl was built with the memory debugging enabled, and if it was it will detect memory leaks too. @@ -269,6 +298,7 @@ Building Releases released, run the 'maketgz' script (using 'make distcheck' will give you a pretty good view on the status of the current sources). maketgz prompts for version number of the client and the library before it creates a release - archive. + archive. maketgz uses 'make dist' for the actual archive building, why you + need to fill in the Makefile.am files properly for which files that should + be included in the release archives. - You must have autoconf installed to build release archives. -- cgit v1.2.1 From ff681f7bfd39df83ebc1e84f468b2a42c243fbcf Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Tue, 13 Mar 2001 15:44:31 +0000 Subject: 7.7 beta 2 fixes --- docs/INTERNALS | 56 +++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 37 insertions(+), 19 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index ddba56ee3..37a388614 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -7,11 +7,11 @@ INTERNALS - The project is kind of split in two. The library and the client. The client - part uses the library, but the library is meant to be designed to allow other - applications to use it. + The project is split in two. The library and the client. The client part uses + the library, but the library is designed to allow other applications to use + it. - Thus, the largest amount of code and complexity is in the library part. + The largest amount of code and complexity is in the library part. CVS === @@ -35,13 +35,13 @@ Windows vs Unix the same at all places except for the header file that defines them. The macros in use are sclose(), sread() and swrite(). - 2. Windows requires a couple of init calls for the socket stuff + 2. Windows requires a couple of init calls for the socket stuff. Those must be made by the application that uses libcurl, in curl that means src/main.c has some code #ifdef'ed to do just that. 3. The file descriptors for network communication and file operations are - not easily interchangable as in unix + not easily interchangable as in unix. We avoid this by not trying any funny tricks on file descriptors. @@ -51,10 +51,10 @@ Windows vs Unix We set stdout to binary under windows - Inside the source code, I do make an effort to avoid '#ifdef WIN32'. All + Inside the source code, We make an effort to avoid '#ifdef [Your OS]'. All conditionals that deal with features *should* instead be in the format '#ifdef HAVE_THAT_WEIRD_FUNCTION'. Since Windows can't run configure scripts, - I maintain two config-win32.h files (one in / and one in src/) that are + we maintain two config-win32.h files (one in / and one in src/) that are supposed to look exactly as a config.h file would have looked like on a Windows machine! @@ -64,12 +64,6 @@ Windows vs Unix Library ======= - As described elsewhere, libcurl is meant to get two different "layers" of - interfaces. At the present point only the high-level, the "easy", interface - has been fully implemented and documented. We assume the easy-interface in - this description, the low-level interface will be documented when fully - implemented. - There are plenty of entry points to the library, namely each publicly defined function that libcurl offers to applications. All of those functions are rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are @@ -115,6 +109,22 @@ Library When completed, the curl_easy_cleanup() should be called to free up used resources. + A quick roundup on internal function sequences (many of these call + protocol-specific function-pointers): + + curl_connect - connects to a remote site and does initial connect fluff + This also checks for an existing connection to the requested site and uses + that one if it is possible. + + curl_do - starts a transfer + curl_transfer() - transfers data + curl_done - ends a transfer + + curl_disconnect - disconnects from a remote site. This is called when the + disconnect is really requested, which doesn't necessarily have to be + exactly after curl_done in case we want to keep the connection open for + a while. + HTTP(S) HTTP offers a lot and is the protocol in curl that uses the most lines of @@ -130,6 +140,14 @@ Library the source by the use of curl_read() for reading and curl_write() for writing data to the remote server. + http_chunks.c contains functions that understands HTTP 1.1 chunked transfer + encoding. + + An interesting detail with the HTTP(S) request, is the add_buffer() series of + functions we use. They append data to one single buffer, and when the + building is done the entire request is sent off in one single write. This is + done this way to overcome problems with flawed firewalls and lame servers. + FTP The Curl_if2ip() function can be used for getting the IP number of a @@ -265,12 +283,12 @@ Memory Debugging deal with resources that might give us problems if we "leak" them. The functions in the memdebug system do nothing fancy, they do their normal function and then log information about what they just did. The logged data - is then analyzed after a complete session, + can then be analyzed after a complete session, - memanalyze.pl is a perl script present only in CVS (not part of the release - archives) that analyzes a log file generated by the memdebug system. It - detects if resources are allocated but never freed and other kinds of errors - related to resource management. + memanalyze.pl is a perl script present only present in CVS (not part of the + release archives) that analyzes a log file generated by the memdebug + system. It detects if resources are allocated but never freed and other kinds + of errors related to resource management. Use -DMALLOCDEBUG when compiling to enable memory debugging. -- cgit v1.2.1 From e83550f51186e264c10de5770f8e4241584caeed Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Mon, 23 Apr 2001 07:09:15 +0000 Subject: persistent is spelled with an 'e', not an 'a' --- docs/INTERNALS | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 37a388614..e0553409c 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -201,10 +201,10 @@ Library exists in lib/getpass.c. libcurl offers a custom callback that can be used instead of this, but it doesn't change much to us. -Persistant Connections +Persistent Connections ====================== - With curl 7.7, we added persistant connection support to libcurl which has + With curl 7.7, we added persistent connection support to libcurl which has introduced a somewhat different treatmeant of things inside of libcurl. o The 'UrlData' struct returned in the curl_easy_init() call must never @@ -227,7 +227,7 @@ Persistant Connections o When curl_easy_cleanup() is called, we close all still opened connections. You do realize that the curl handle must be re-used in order for the - persistant connections to work. + persistent connections to work. Library Symbols =============== -- cgit v1.2.1 From 26d4c80049dd8cb698456a7cabdfd07b02327af1 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 27 Apr 2001 14:46:17 +0000 Subject: clarified and updated --- docs/INTERNALS | 109 ++++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 73 insertions(+), 36 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index e0553409c..0cbd91a98 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,4 +1,4 @@ - Updated for curl 7.7 on March 13, 2001 + Updated for curl 7.7.2 on April 26, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -73,7 +73,7 @@ Library makes sure we stay absolutely platform independent. curl_easy_init() allocates an internal struct and makes some initializations. - The returned handle does not revail internals. + The returned handle does not reveal internals. curl_easy_setopt() takes a three arguments, where the option stuff must be passed in pairs, the parameter-ID and the parameter-value. The list of @@ -81,33 +81,68 @@ Library curl_easy_perform() does a whole lot of things: - It starts off in the lib/easy.c file by calling curl_transfer(), but the main - work is lib/url.c. The function first analyzes the URL, it separates the - different components and connects to the remote host. This may involve using - a proxy and/or using SSL. The Curl_gethost() function in lib/hostip.c is used - for looking up host names. - - When connected, the proper protocol-specific function is called. The - functions are named after the protocols they handle. Curl_ftp(), Curl_http(), - Curl_dict(), etc. They all reside in their respective files (ftp.c, http.c - and dict.c). - - The protocol-specific functions of course deal with protocol-specific - negotiations and setup. They have access to the Curl_sendf() (from - lib/sendf.c) function to send printf-style formatted data to the remote host - and when they're ready to make the actual file transfer they call the - Curl_Transfer() function (in lib/transfer.c) to setup the transfer and - returns. Curl_perform() then calls Transfer() in lib/transfer.c that performs - the entire file transfer. Curl_perform() is what does the main "connect - do - - transfer - done" loop. It loops if there's a Location: to follow. - - During transfer, the progress functions in lib/progress.c are called at a - frequent interval (or at the user's choice, a specified callback might get - called). The speedcheck functions in lib/speedcheck.c are also used to verify - that the transfer is as fast as required. + It starts off in the lib/easy.c file by calling Curl_perform() and the main + work then continues lib/url.c. The flow continues with a call to + Curl_connect() to connect to the remote site. + + o Curl_connect() + + ... analyzes the URL, it separates the different components and connects to + the remote host. This may involve using a proxy and/or using SSL. The + Curl_gethost() function in lib/hostip.c is used for looking up host names. + + When Curl_connect is done, we are connected to the remote site. Then it is + time to tell the server to get a document/file. Curl_do() arranges this. + + o Curl_do() + + Curl_do() makes sure the proper protocol-specific function is called. The + functions are named after the protocols they handle. Curl_ftp(), + Curl_http(), Curl_dict(), etc. They all reside in their respective files + (ftp.c, http.c and dict.c). + + The protocol-specific functions of course deal with protocol-specific + negotiations and setup. They have access to the Curl_sendf() (from + lib/sendf.c) function to send printf-style formatted data to the remote + host and when they're ready to make the actual file transfer they call the + Curl_Transfer() function (in lib/transfer.c) to setup the transfer and + returns. + + o Transfer() + + Curl_perform() then calls Transfer() in lib/transfer.c that performs + the entire file transfer. + + During transfer, the progress functions in lib/progress.c are called at a + frequent interval (or at the user's choice, a specified callback might get + called). The speedcheck functions in lib/speedcheck.c are also used to + verify that the transfer is as fast as required. + + o Curl_done() + + Called after a transfer is done. This function takes care of everything + that has to be done after a transfer. This function attempts to leave + matters in a state so that Curl_do() should be possible to call again on + the same connection (in a persistent connection case). It may also soon be + closed with Curl_disconnect(). + + o Curl_disconnect() + + During normal connection and transfers, no one ever tries to close any + connection so this is not normally called when curl_easy_perform() is + used. This function is only used when we are certain that no more transfers + is going to be made on the connection (it can be also closed by + force). This function can also be called at times to make sure that libcurl + doesn't keep too many connections alive at the same time. + + This function cleans up all resources that are associated with a single + connection. + + Curl_perform() is the function that does the main "connect - do - transfer - + done" loop. It loops if there's a Location: to follow. When completed, the curl_easy_cleanup() should be called to free up used - resources. + resources. It runs Curl_disconnect() on all open connectons. A quick roundup on internal function sequences (many of these call protocol-specific function-pointers): @@ -257,6 +292,7 @@ Client ====== main() resides in src/main.c together with most of the client code. + src/hugehelp.c is automatically generated by the mkhelp.pl perl script to display the complete "manual" and the src/urlglob.c file holds the functions used for the URL-"globbing" support. Globbing in the sense that the {} and [] @@ -272,25 +308,26 @@ Client curl_easy_getinfo() function to extract useful information from the curl session. - Recent versions may loop and do all that several times if many URLs were + Recent versions may loop and do all this several times if many URLs were specified on the command line or config file. Memory Debugging ================ - The file named lib/memdebug.c contains debug-versions of a few - functions. Functions such as malloc, free, fopen, fclose, etc that somehow - deal with resources that might give us problems if we "leak" them. The - functions in the memdebug system do nothing fancy, they do their normal - function and then log information about what they just did. The logged data - can then be analyzed after a complete session, + The file lib/memdebug.c contains debug-versions of a few functions. Functions + such as malloc, free, fopen, fclose, etc that somehow deal with resources + that might give us problems if we "leak" them. The functions in the memdebug + system do nothing fancy, they do their normal function and then log + information about what they just did. The logged data can then be analyzed + after a complete session, - memanalyze.pl is a perl script present only present in CVS (not part of the + memanalyze.pl is the perl script present only present in CVS (not part of the release archives) that analyzes a log file generated by the memdebug system. It detects if resources are allocated but never freed and other kinds of errors related to resource management. - Use -DMALLOCDEBUG when compiling to enable memory debugging. + Use -DMALLOCDEBUG when compiling to enable memory debugging, this is also + switched on by running configure with --enable-debug. Test Suite ========== -- cgit v1.2.1 From de16ddd5b4f9ea8d60b3a621586c568ad42dcdee Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Tue, 29 May 2001 13:23:41 +0000 Subject: updated, improved language at a few places --- docs/INTERNALS | 67 +++++++++++++++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 26 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 0cbd91a98..e40804197 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,4 +1,4 @@ - Updated for curl 7.7.2 on April 26, 2001 + Updated for curl 7.8 on May 29, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -69,20 +69,29 @@ Library rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are put in the lib/easy.c file. + Starting with libcurl 7.8, curl_global_init_() and curl_global_cleanup() were + introduced. They should be called by the application to initialize and clean + up global stuff in the library. As of today, they just do the global SSL + initing if SSL is enabled. libcurl itself has no "global" scope. + All printf()-style functions use the supplied clones in lib/mprintf.c. This makes sure we stay absolutely platform independent. curl_easy_init() allocates an internal struct and makes some initializations. - The returned handle does not reveal internals. + The returned handle does not reveal internals. This is the 'UrlData' struct + which works as a global "anchor" struct. All connections performed will get + connect-specific data allocated that should be used for things related to + particular connections/requests. - curl_easy_setopt() takes a three arguments, where the option stuff must be - passed in pairs, the parameter-ID and the parameter-value. The list of - options is documented in the man page. + curl_easy_setopt() takes three arguments, where the option stuff must be + passed in pairs: the parameter-ID and the parameter-value. The list of + options is documented in the man page. This function mainly sets things in + the 'UrlData' struct. curl_easy_perform() does a whole lot of things: It starts off in the lib/easy.c file by calling Curl_perform() and the main - work then continues lib/url.c. The flow continues with a call to + work then continues in lib/url.c. The flow continues with a call to Curl_connect() to connect to the remote site. o Curl_connect() @@ -94,12 +103,18 @@ Library When Curl_connect is done, we are connected to the remote site. Then it is time to tell the server to get a document/file. Curl_do() arranges this. + This function makes sure there's an allocated and initiated 'connectdata' + struct that is used for this particular connection only (although there may + be several requests performed on the same connect). A bunch of things are + inited/inherited from the UrlData struct. + o Curl_do() Curl_do() makes sure the proper protocol-specific function is called. The functions are named after the protocols they handle. Curl_ftp(), Curl_http(), Curl_dict(), etc. They all reside in their respective files - (ftp.c, http.c and dict.c). + (ftp.c, http.c and dict.c). HTTPS is handled by Curl_http() and FTPS by + Curl_ftp(). The protocol-specific functions of course deal with protocol-specific negotiations and setup. They have access to the Curl_sendf() (from @@ -123,17 +138,18 @@ Library Called after a transfer is done. This function takes care of everything that has to be done after a transfer. This function attempts to leave matters in a state so that Curl_do() should be possible to call again on - the same connection (in a persistent connection case). It may also soon be - closed with Curl_disconnect(). + the same connection (in a persistent connection case). It might also soon + be closed with Curl_disconnect(). o Curl_disconnect() - During normal connection and transfers, no one ever tries to close any + When doing normal connections and transfers, no one ever tries to close any connection so this is not normally called when curl_easy_perform() is used. This function is only used when we are certain that no more transfers - is going to be made on the connection (it can be also closed by - force). This function can also be called at times to make sure that libcurl - doesn't keep too many connections alive at the same time. + is going to be made on the connection. It can be also closed by force, or + it can be called to make sure that libcurl doesn't keep too many + connections alive at the same time (there's a default amount of 5 but that + can be changed with the CURLOPT_MAXCONNECTS option). This function cleans up all resources that are associated with a single connection. @@ -239,26 +255,26 @@ Library Persistent Connections ====================== - With curl 7.7, we added persistent connection support to libcurl which has - introduced a somewhat different treatmeant of things inside of libcurl. + The persistent connection support in libcurl requires some considerations on + how to do things inside of the library. o The 'UrlData' struct returned in the curl_easy_init() call must never hold connection-oriented data. It is meant to hold the root data as well as all the options etc that the library-user may choose. - o The 'UrlData' struct holds the cache array of pointers to 'connectdata' - structs. There's one connectdata struct for each connection that libcurl - knows about. + o The 'UrlData' struct holds the "connection cache" (an array of pointers to + 'connectdata' structs). There's one connectdata struct allocated for each + connection that libcurl knows about. o This also enables the 'curl handle' to be reused on subsequent transfers, - something that was illegal in pre-7.7 versions. + something that was illegal before libcurl 7.7. o When we are about to perform a transfer with curl_easy_perform(), we first check for an already existing connection in the cache that we can use, otherwise we create a new one and add to the cache. If the cache is full already when we add a new connection, we close one of the present ones. We select which one to close dependent on the close policy that may have been previously set. - o When the tranfer operation is complete, we try to leave the connection open. - Particular options may tell us not to, and protocols may signal closure on - connections and then we don't keep it open of course. + o When the transfer operation is complete, we try to leave the connection + open. Particular options may tell us not to, and protocols may signal + closure on connections and then we don't keep it open of course. o When curl_easy_cleanup() is called, we close all still opened connections. You do realize that the curl handle must be re-used in order for the @@ -268,10 +284,9 @@ Library Symbols =============== All symbols used internally in libcurl must use a 'Curl_' prefix if they're - used in more than a single file. Single-file symbols must be made - static. Public (exported) symbols must use a 'curl_' prefix. (There are - exceptions, but they are destined to be changed to follow this pattern in the - future.) + used in more than a single file. Single-file symbols must be made static. + Public ("exported") symbols must use a 'curl_' prefix. (There are exceptions, + but they are to be changed to follow this pattern in future versions.) Return Codes and Informationals =============================== -- cgit v1.2.1 From 2297a7a70c145fefdf546f2c9c0c8ac48239a6f2 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Fri, 2 Nov 2001 13:16:29 +0000 Subject: more accurate --- docs/INTERNALS | 35 +++++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 14 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index e40804197..e2ea59b4c 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,4 +1,4 @@ - Updated for curl 7.8 on May 29, 2001 + Updated for curl 7.9.1 on November 2, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -78,15 +78,15 @@ Library makes sure we stay absolutely platform independent. curl_easy_init() allocates an internal struct and makes some initializations. - The returned handle does not reveal internals. This is the 'UrlData' struct - which works as a global "anchor" struct. All connections performed will get - connect-specific data allocated that should be used for things related to - particular connections/requests. + The returned handle does not reveal internals. This is the 'SessionHandle' + struct which works as an "anchor" struct for all curl_easy functions. All + connections performed will get connect-specific data allocated that should be + used for things related to particular connections/requests. curl_easy_setopt() takes three arguments, where the option stuff must be passed in pairs: the parameter-ID and the parameter-value. The list of options is documented in the man page. This function mainly sets things in - the 'UrlData' struct. + the 'SessionHandle' struct. curl_easy_perform() does a whole lot of things: @@ -106,7 +106,7 @@ Library This function makes sure there's an allocated and initiated 'connectdata' struct that is used for this particular connection only (although there may be several requests performed on the same connect). A bunch of things are - inited/inherited from the UrlData struct. + inited/inherited from the SessionHandle struct. o Curl_do() @@ -123,6 +123,13 @@ Library Curl_Transfer() function (in lib/transfer.c) to setup the transfer and returns. + Starting in 7.9.1, if this DO function fails and the connection is being + re-used, libcurl will then close this connection, setup a new connection + and re-issue the DO request on that. This is because there is no way to be + perfectly sure that we have discovered a dead connection before the DO + function and thus we might wrongly be re-using a connection that was closed + by the remote peer. + o Transfer() Curl_perform() then calls Transfer() in lib/transfer.c that performs @@ -144,7 +151,7 @@ Library o Curl_disconnect() When doing normal connections and transfers, no one ever tries to close any - connection so this is not normally called when curl_easy_perform() is + connections so this is not normally called when curl_easy_perform() is used. This function is only used when we are certain that no more transfers is going to be made on the connection. It can be also closed by force, or it can be called to make sure that libcurl doesn't keep too many @@ -258,12 +265,12 @@ Persistent Connections The persistent connection support in libcurl requires some considerations on how to do things inside of the library. - o The 'UrlData' struct returned in the curl_easy_init() call must never - hold connection-oriented data. It is meant to hold the root data as well - as all the options etc that the library-user may choose. - o The 'UrlData' struct holds the "connection cache" (an array of pointers to - 'connectdata' structs). There's one connectdata struct allocated for each - connection that libcurl knows about. + o The 'SessionHandle' struct returned in the curl_easy_init() call must never + hold connection-oriented data. It is meant to hold the root data as well as + all the options etc that the library-user may choose. + o The 'SessionHandle' struct holds the "connection cache" (an array of + pointers to 'connectdata' structs). There's one connectdata struct + allocated for each connection that libcurl knows about. o This also enables the 'curl handle' to be reused on subsequent transfers, something that was illegal before libcurl 7.7. o When we are about to perform a transfer with curl_easy_perform(), we first -- cgit v1.2.1 From 36e1363e3d85a85c61e838f8ae6168c95faba50f Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Wed, 27 Feb 2002 12:40:01 +0000 Subject: minor edit --- docs/INTERNALS | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index e2ea59b4c..42ac4508b 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -54,7 +54,7 @@ Windows vs Unix Inside the source code, We make an effort to avoid '#ifdef [Your OS]'. All conditionals that deal with features *should* instead be in the format '#ifdef HAVE_THAT_WEIRD_FUNCTION'. Since Windows can't run configure scripts, - we maintain two config-win32.h files (one in / and one in src/) that are + we maintain two config-win32.h files (one in lib/ and one in src/) that are supposed to look exactly as a config.h file would have looked like on a Windows machine! @@ -69,10 +69,10 @@ Library rather small and easy-to-follow. All the ones prefixed with 'curl_easy' are put in the lib/easy.c file. - Starting with libcurl 7.8, curl_global_init_() and curl_global_cleanup() were - introduced. They should be called by the application to initialize and clean - up global stuff in the library. As of today, they just do the global SSL - initing if SSL is enabled. libcurl itself has no "global" scope. + curl_global_init_() and curl_global_cleanup() should be called by the + application to initialize and clean up global stuff in the library. As of + today, it can handle the global SSL initing if SSL is enabled and it can init + the socket layer on windows machines. libcurl itself has no "global" scope. All printf()-style functions use the supplied clones in lib/mprintf.c. This makes sure we stay absolutely platform independent. -- cgit v1.2.1 From 322308e298f5e86ab5261f4289b1636b1130efd1 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 20 Sep 2007 14:05:53 +0000 Subject: the winsock stuff is made by curl_global_init --- docs/INTERNALS | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 42ac4508b..f9532797b 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,4 +1,3 @@ - Updated for curl 7.9.1 on November 2, 2001 _ _ ____ _ ___| | | | _ \| | / __| | | | |_) | | @@ -37,8 +36,8 @@ Windows vs Unix 2. Windows requires a couple of init calls for the socket stuff. - Those must be made by the application that uses libcurl, in curl that means - src/main.c has some code #ifdef'ed to do just that. + That's taken care of by the curl_global_init() call, but if other libs also + do it etc there might be reasons for applications to alter that behaviour. 3. The file descriptors for network communication and file operations are not easily interchangable as in unix. -- cgit v1.2.1 From dc11239ff13951ecc4f178b3986361d1af62ec92 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Mon, 26 Nov 2007 11:02:45 +0000 Subject: slightly less outdated --- docs/INTERNALS | 43 +++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 20 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index f9532797b..f3769b1fd 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -97,7 +97,9 @@ Library ... analyzes the URL, it separates the different components and connects to the remote host. This may involve using a proxy and/or using SSL. The - Curl_gethost() function in lib/hostip.c is used for looking up host names. + Curl_resolv() function in lib/hostip.c is used for looking up host names + (it does then use the proper underlying method, which may vary between + platforms and builds). When Curl_connect is done, we are connected to the remote site. Then it is time to tell the server to get a document/file. Curl_do() arranges this. @@ -122,17 +124,20 @@ Library Curl_Transfer() function (in lib/transfer.c) to setup the transfer and returns. - Starting in 7.9.1, if this DO function fails and the connection is being - re-used, libcurl will then close this connection, setup a new connection - and re-issue the DO request on that. This is because there is no way to be - perfectly sure that we have discovered a dead connection before the DO - function and thus we might wrongly be re-using a connection that was closed - by the remote peer. + If this DO function fails and the connection is being re-used, libcurl will + then close this connection, setup a new connection and re-issue the DO + request on that. This is because there is no way to be perfectly sure that + we have discovered a dead connection before the DO function and thus we + might wrongly be re-using a connection that was closed by the remote peer. + + Some time during the DO function, the Curl_setup_transfer() function must + be called with some basic info about the upcoming transfer: what socket(s) + to read/write and the expected file tranfer sizes (if known). o Transfer() - Curl_perform() then calls Transfer() in lib/transfer.c that performs - the entire file transfer. + Curl_perform() then calls Transfer() in lib/transfer.c that performs the + entire file transfer. During transfer, the progress functions in lib/progress.c are called at a frequent interval (or at the user's choice, a specified callback might get @@ -236,9 +241,8 @@ Library URL encoding and decoding, called escaping and unescaping in the source code, is found in lib/escape.c. - While transfering data in Transfer() a few functions might get - used. curl_getdate() in lib/getdate.c is for HTTP date comparisons (and - more). + While transfering data in Transfer() a few functions might get used. + curl_getdate() in lib/parsedate.c is for HTTP date comparisons (and more). lib/getenv.c offers curl_getenv() which is for reading environment variables in a neat platform independent way. That's used in the client, but also in @@ -254,10 +258,6 @@ Library A function named curl_version() that returns the full curl version string is found in lib/version.c. - If authentication is requested but no password is given, a getpass_r() clone - exists in lib/getpass.c. libcurl offers a custom callback that can be used - instead of this, but it doesn't change much to us. - Persistent Connections ====================== @@ -269,9 +269,11 @@ Persistent Connections all the options etc that the library-user may choose. o The 'SessionHandle' struct holds the "connection cache" (an array of pointers to 'connectdata' structs). There's one connectdata struct - allocated for each connection that libcurl knows about. - o This also enables the 'curl handle' to be reused on subsequent transfers, - something that was illegal before libcurl 7.7. + allocated for each connection that libcurl knows about. Note that when you + use the multi interface, the multi handle will hold the connection cache + and not the particular easy handle. This of course to allow all easy handles + in a multi stack to be able to share and re-use connections. + o This enables the 'curl handle' to be reused on subsequent transfers. o When we are about to perform a transfer with curl_easy_perform(), we first check for an already existing connection in the cache that we can use, otherwise we create a new one and add to the cache. If the cache is full @@ -281,7 +283,8 @@ Persistent Connections o When the transfer operation is complete, we try to leave the connection open. Particular options may tell us not to, and protocols may signal closure on connections and then we don't keep it open of course. - o When curl_easy_cleanup() is called, we close all still opened connections. + o When curl_easy_cleanup() is called, we close all still opened connections, + unless of course the multi interface "owns" the connections. You do realize that the curl handle must be re-used in order for the persistent connections to work. -- cgit v1.2.1 From ebce0a16f6b2587434ba5bcde5b40633bb17c54c Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Mon, 26 Nov 2007 12:26:58 +0000 Subject: more blurb --- docs/INTERNALS | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index f3769b1fd..84538a931 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -289,6 +289,40 @@ Persistent Connections You do realize that the curl handle must be re-used in order for the persistent connections to work. +multi interface/non-blocking +============================ + + We make an effort to provide a non-blocking interface to the library, the + multi interface. To make that interface work as good as possible, no + low-level functions within libcurl must be written to work in a blocking + manner. + + One of the primary reasons we introduced c-ares support was to allow the name + resolve phase to be perfectly non-blocking as well. + + The ultimate goal is to provide the easy interface simply by wrapping the + multi interface functions and thus treat everything internally as the multi + interface is the single interface we have. + + The FTP and the SFTP/SCP protocols are thus perfect examples of how we adapt + and adjust the code to allow non-blocking operations even on multi-stage + protocols. The DICT, TELNET and TFTP are crappy examples and they are subject + for rewrite in the future to better fit the libcurl protocol family. + +SSL libraries +============= + + Originally libcurl supported SSLeay for SSL/TLS transports, but that was then + extended to its successor OpenSSL but has since also been extended to several + other SSL/TLS libraries and we expect and hope to further extend the support + in future libcurl versions. + + To deal with this internally in the best way possible, we have a generic SSL + function API as provided by the sslgen.[ch] system, and they are the only SSL + functions we must use from within libcurl. sslgen is then crafted to use the + appropriate lower-level function calls to whatever SSL library that is in + use. + Library Symbols =============== @@ -312,6 +346,13 @@ Return Codes and Informationals them. They are best used when revealing information that isn't otherwise obvious. +API/ABI +======= + + We make an effort to not export or show internals or how internals work, as + that makes it easier to keep a solid API/ABI over time. See docs/libcurl/ABI + for our promise to users. + Client ====== -- cgit v1.2.1 From 99185417952da30c8ddd82ab962fb58da96260b2 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 12 Jun 2008 21:03:16 +0000 Subject: My first attempt at documenting what we try to support and make curl run with in regard to C standard, third party libraries and operating systems etc. --- docs/INTERNALS | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 84538a931..4209095ab 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -22,6 +22,43 @@ CVS Tagging shall be used extensively, and by the time we release new archives we should tag the sources with a name similar to the released version number. +Portability +=========== + + We write curl and libcurl to compile with C89 compilers. On 32bit and up + machines. Most of libcurl assumes more or less POSIX compliance but that's + not a requirement. + + We write libcurl to build and work with lots of third party tools, and we + want it to remain functional and buildable with these and later versions + (older versions may still work but is not what we work hard to maintain): + + OpenSSL 0.9.6 + GnuTLS 1.2 + zlib 1.1.4 + libssh2 0.16 + c-ares 1.5.0 + libidn 0.4.1 + *yassl 1.4.0 (http://curl.haxx.se/mail/lib-2008-02/0093.html) + openldap 2.0 + MIT krb5 lib 1.2.4 + qsossl V5R2M0 + Heimdal ? + NSS ? + + * = only partly functional, but that's due to bugs in the third party lib, not + because of libcurl code + + On systems where configure runs, we aim at working on them all - if they have + a suitable C compiler. On systems that don't run configure, we strive to keep + curl running fine on: + + Windows 98 + AS/400 V5R2M0 + Symbian 9.1 + Windows CE ? + TPF ? + Windows vs Unix =============== -- cgit v1.2.1 From af779fa57c530575f018048f6d889fcb0cae71e5 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 12 Jun 2008 21:16:00 +0000 Subject: added the versions of a range of build tools that we want to remain to work --- docs/INTERNALS | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 4209095ab..f8927b568 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -59,6 +59,19 @@ Portability Windows CE ? TPF ? + When writing code (mostly for generating stuff included in release tarballs) + we use a few "build tools" and we make sure that we remain functional with + these versions: + + GNU Libtool 1.4.2 + GNU Autoconf 2.57 + GNU Automake 1.7 (we currently avoid 1.10 due to Solaris-related bugs) + GNU M4 1.4 + perl 4 + roffit 0.5 + groff ? (any version that supports "groff -Tps -man [in] [out]") + ps2pdf (gs) ? + Windows vs Unix =============== -- cgit v1.2.1 From 6929d9355f2b5dc039d60300ff8eb7504a1cf2c2 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 26 Jun 2008 07:53:44 +0000 Subject: let's try to maintain compatibility with NSS 3.11.x --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index f8927b568..411e664a9 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -43,8 +43,8 @@ Portability openldap 2.0 MIT krb5 lib 1.2.4 qsossl V5R2M0 + NSS 3.11.x Heimdal ? - NSS ? * = only partly functional, but that's due to bugs in the third party lib, not because of libcurl code -- cgit v1.2.1 From e08296f70c35525003b05ee3e69e4a73fc1721ca Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 30 Apr 2009 08:08:33 +0000 Subject: mention that cvs is needed for releases --- docs/INTERNALS | 3 +++ 1 file changed, 3 insertions(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 411e664a9..5021c82e0 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -472,3 +472,6 @@ Building Releases need to fill in the Makefile.am files properly for which files that should be included in the release archives. + NOTE: you need to have curl checked out from CVS to be able to do a proper + release build. The release tarballs do not have everything setup in order to + do releases properly. -- cgit v1.2.1 From f671d0513ccc9cdcd7516c732c719b2f31a9578e Mon Sep 17 00:00:00 2001 From: Gunter Knauf Date: Tue, 14 Jul 2009 13:25:14 +0000 Subject: renamed generated config.h to curl_config.h in order to avoid clashes when libcurl is used with other projects which also have a config.h. --- docs/INTERNALS | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 5021c82e0..e1b5da5fb 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -103,9 +103,9 @@ Windows vs Unix Inside the source code, We make an effort to avoid '#ifdef [Your OS]'. All conditionals that deal with features *should* instead be in the format '#ifdef HAVE_THAT_WEIRD_FUNCTION'. Since Windows can't run configure scripts, - we maintain two config-win32.h files (one in lib/ and one in src/) that are - supposed to look exactly as a config.h file would have looked like on a - Windows machine! + we maintain two curl_config-win32.h files (one in lib/ and one in src/) that + are supposed to look exactly as a curl_config.h file would have looked like on + a Windows machine! Generally speaking: always remember that this will be compiled on dozens of operating systems. Don't walk on the edge. -- cgit v1.2.1 From 3f8d3e9c50af5b23aa50f2f0336da57023402a78 Mon Sep 17 00:00:00 2001 From: Yang Tse Date: Wed, 21 Oct 2009 18:01:11 +0000 Subject: Update memory tracking/debugging reference --- docs/INTERNALS | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index e1b5da5fb..ced5df51f 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -437,12 +437,23 @@ Memory Debugging after a complete session, memanalyze.pl is the perl script present only present in CVS (not part of the - release archives) that analyzes a log file generated by the memdebug + release archives) that analyzes a log file generated by the memory tracking system. It detects if resources are allocated but never freed and other kinds of errors related to resource management. - Use -DMALLOCDEBUG when compiling to enable memory debugging, this is also - switched on by running configure with --enable-debug. + Internally, definition of preprocessor symbol DEBUGBUILD restricts code which + is only compiled for debug enabled builds. And symbol CURLDEBUG is used to + differentiate code which is _only_ used for memory tracking/debugging. + + Use -DCURLDEBUG when compiling to enable memory debugging, this is also + switched on by running configure with --enable-curldebug. Use -DDEBUGBUILD + when compiling to enable a debug build or run configure with --enable-debug. + + curl --version will list 'Debug' feature for debug enabled builds, and + will list 'TrackMemory' feature for curl debug memory tracking capable + builds. These features are independent and can be controlled when running + the configure script. When --enable-debug is given both features will be + enabled, unless some restriction prevents memory tracking from being used. Test Suite ========== -- cgit v1.2.1 From a07bc79117971b96ebf3188c0a34a73ee0a3609b Mon Sep 17 00:00:00 2001 From: Yang Tse Date: Sun, 14 Feb 2010 19:40:18 +0000 Subject: removed trailing whitespace --- docs/INTERNALS | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index ced5df51f..c9c497140 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -1,7 +1,7 @@ - _ _ ____ _ - ___| | | | _ \| | - / __| | | | |_) | | - | (__| |_| | _ <| |___ + _ _ ____ _ + ___| | | | _ \| | + / __| | | | |_) | | + | (__| |_| | _ <| |___ \___|\___/|_| \_\_____| INTERNALS @@ -304,7 +304,7 @@ Library lib/timeval.c features replacement functions for systems that don't have gettimeofday() and a few support functions for timeval convertions. - + A function named curl_version() that returns the full curl version string is found in lib/version.c. @@ -375,7 +375,7 @@ SSL libraries Library Symbols =============== - + All symbols used internally in libcurl must use a 'Curl_' prefix if they're used in more than a single file. Single-file symbols must be made static. Public ("exported") symbols must use a 'curl_' prefix. (There are exceptions, -- cgit v1.2.1 From 1609685fc2f470600204094d39ea55f63a445abf Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Mon, 22 Mar 2010 00:34:09 +0100 Subject: various changes of CVS to git --- docs/INTERNALS | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index c9c497140..630b72b0f 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -12,9 +12,9 @@ INTERNALS The largest amount of code and complexity is in the library part. -CVS +GIT === - All changes to the sources are committed to the CVS repository as soon as + All changes to the sources are committed to the git repository as soon as they're somewhat verified to work. Changes shall be commited as independently as possible so that individual changes can be easier spotted and tracked afterwards. @@ -436,10 +436,10 @@ Memory Debugging information about what they just did. The logged data can then be analyzed after a complete session, - memanalyze.pl is the perl script present only present in CVS (not part of the - release archives) that analyzes a log file generated by the memory tracking - system. It detects if resources are allocated but never freed and other kinds - of errors related to resource management. + memanalyze.pl is the perl script present in tests/ that analyzes a log file + generated by the memory tracking system. It detects if resources are + allocated but never freed and other kinds of errors related to resource + management. Internally, definition of preprocessor symbol DEBUGBUILD restricts code which is only compiled for debug enabled builds. And symbol CURLDEBUG is used to @@ -483,6 +483,6 @@ Building Releases need to fill in the Makefile.am files properly for which files that should be included in the release archives. - NOTE: you need to have curl checked out from CVS to be able to do a proper + NOTE: you need to have curl checked out from git to be able to do a proper release build. The release tarballs do not have everything setup in order to do releases properly. -- cgit v1.2.1 From 82b55efdbae2a2289608d4b7fdf39b0883729a59 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Thu, 29 Apr 2010 19:19:08 +0200 Subject: INTERNALS: tftp is decent now, ldap is not It's not quite fair to list TFTP is a "crappy" member of the libcurl family so I removed its mentioning. --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 630b72b0f..54119cf79 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -356,7 +356,7 @@ multi interface/non-blocking The FTP and the SFTP/SCP protocols are thus perfect examples of how we adapt and adjust the code to allow non-blocking operations even on multi-stage - protocols. The DICT, TELNET and TFTP are crappy examples and they are subject + protocols. The DICT, LDAP and TELNET are crappy examples and they are subject for rewrite in the future to better fit the libcurl protocol family. SSL libraries -- cgit v1.2.1 From 19f45eaa799a77d5c67cbefe3342f27774e2e6f8 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Tue, 21 Sep 2010 16:41:14 +0200 Subject: duphandle: use ares_dup() curl_easy_duphandle() was not properly duping the ares channel. The ares_dup() function was introduced in c-ares 1.6.0 so by starting to use this function we also raise the bar and require c-ares >= 1.6.0 (released Dec 9, 2008) for such builds. Reported by: Ning Dong Bug: http://curl.haxx.se/mail/lib-2010-08/0318.html --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 54119cf79..9d0bdbaa1 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -37,7 +37,7 @@ Portability GnuTLS 1.2 zlib 1.1.4 libssh2 0.16 - c-ares 1.5.0 + c-ares 1.6.0 libidn 0.4.1 *yassl 1.4.0 (http://curl.haxx.se/mail/lib-2008-02/0093.html) openldap 2.0 -- cgit v1.2.1 From 3427bece89ee00fdf82519e584a971f97c9f2071 Mon Sep 17 00:00:00 2001 From: Dan Fandrich Date: Fri, 21 Jan 2011 14:27:10 -0800 Subject: Mention axTLS in some more documentation --- docs/INTERNALS | 1 + 1 file changed, 1 insertion(+) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 9d0bdbaa1..e667c3307 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -44,6 +44,7 @@ Portability MIT krb5 lib 1.2.4 qsossl V5R2M0 NSS 3.11.x + axTLS 1.2.7 Heimdal ? * = only partly functional, but that's due to bugs in the third party lib, not -- cgit v1.2.1 From 7ba5e8805324250ddf6202a9afed80a33da80bd5 Mon Sep 17 00:00:00 2001 From: Dan Fandrich Date: Thu, 27 Jan 2011 14:37:16 -0800 Subject: Some minor edits including updates to function names --- docs/INTERNALS | 42 +++++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 19 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index e667c3307..5b12639ee 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -15,7 +15,7 @@ INTERNALS GIT === All changes to the sources are committed to the git repository as soon as - they're somewhat verified to work. Changes shall be commited as independently + they're somewhat verified to work. Changes shall be committed as independently as possible so that individual changes can be easier spotted and tracked afterwards. @@ -91,7 +91,7 @@ Windows vs Unix do it etc there might be reasons for applications to alter that behaviour. 3. The file descriptors for network communication and file operations are - not easily interchangable as in unix. + not easily interchangeable as in unix. We avoid this by not trying any funny tricks on file descriptors. @@ -183,7 +183,7 @@ Library Some time during the DO function, the Curl_setup_transfer() function must be called with some basic info about the upcoming transfer: what socket(s) - to read/write and the expected file tranfer sizes (if known). + to read/write and the expected file transfer sizes (if known). o Transfer() @@ -225,15 +225,15 @@ Library A quick roundup on internal function sequences (many of these call protocol-specific function-pointers): - curl_connect - connects to a remote site and does initial connect fluff + Curl_connect - connects to a remote site and does initial connect fluff This also checks for an existing connection to the requested site and uses that one if it is possible. - curl_do - starts a transfer - curl_transfer() - transfers data - curl_done - ends a transfer + Curl_do - starts a transfer + Curl_handler::do_it() - transfers data + Curl_done - ends a transfer - curl_disconnect - disconnects from a remote site. This is called when the + Curl_disconnect - disconnects from a remote site. This is called when the disconnect is really requested, which doesn't necessarily have to be exactly after curl_done in case we want to keep the connection open for a while. @@ -250,13 +250,13 @@ Library HTTPS uses in almost every means the same procedure as HTTP, with only two exceptions: the connect procedure is different and the function used to read or write from the socket is different, although the latter fact is hidden in - the source by the use of curl_read() for reading and curl_write() for writing + the source by the use of Curl_read() for reading and Curl_write() for writing data to the remote server. http_chunks.c contains functions that understands HTTP 1.1 chunked transfer encoding. - An interesting detail with the HTTP(S) request, is the add_buffer() series of + An interesting detail with the HTTP(S) request, is the Curl_add_buffer() series of functions we use. They append data to one single buffer, and when the building is done the entire request is sent off in one single write. This is done this way to overcome problems with flawed firewalls and lame servers. @@ -292,7 +292,7 @@ Library URL encoding and decoding, called escaping and unescaping in the source code, is found in lib/escape.c. - While transfering data in Transfer() a few functions might get used. + While transferring data in Transfer() a few functions might get used. curl_getdate() in lib/parsedate.c is for HTTP date comparisons (and more). lib/getenv.c offers curl_getenv() which is for reading environment variables @@ -304,7 +304,7 @@ Library lib/netrc.c holds the .netrc parser lib/timeval.c features replacement functions for systems that don't have - gettimeofday() and a few support functions for timeval convertions. + gettimeofday() and a few support functions for timeval conversions. A function named curl_version() that returns the full curl version string is found in lib/version.c. @@ -357,8 +357,10 @@ multi interface/non-blocking The FTP and the SFTP/SCP protocols are thus perfect examples of how we adapt and adjust the code to allow non-blocking operations even on multi-stage - protocols. The DICT, LDAP and TELNET are crappy examples and they are subject - for rewrite in the future to better fit the libcurl protocol family. + protocols. They are built around state machines that return when they could + block waiting for data. The DICT, LDAP and TELNET protocols are crappy + examples and they are subject for rewrite in the future to better fit the + libcurl protocol family. SSL libraries ============= @@ -380,7 +382,9 @@ Library Symbols All symbols used internally in libcurl must use a 'Curl_' prefix if they're used in more than a single file. Single-file symbols must be made static. Public ("exported") symbols must use a 'curl_' prefix. (There are exceptions, - but they are to be changed to follow this pattern in future versions.) + but they are to be changed to follow this pattern in future versions.) Public + API functions are marked with CURL_EXTERN in the public header files so that + all others can be hidden on platforms where this is possible. Return Codes and Informationals =============================== @@ -463,15 +467,15 @@ Test Suite subdirectory directly off the root in the curl archive tree, and it contains a bunch of scripts and a lot of test case data. - The main test script is runtests.pl that will invoke the two servers + The main test script is runtests.pl that will invoke test servers like httpserver.pl and ftpserver.pl before all the test cases are performed. The test suite currently only runs on unix-like platforms. - You'll find a complete description of the test case data files in the - tests/README file. + You'll find a description of the test suite in the tests/README file, and the + test case data files in the tests/FILEFORMAT file. The test suite automatically detects if curl was built with the memory - debugging enabled, and if it was it will detect memory leaks too. + debugging enabled, and if it was it will detect memory leaks, too. Building Releases ================= -- cgit v1.2.1 From c91c48723aa3cc9743e4d413ee65fb875deea700 Mon Sep 17 00:00:00 2001 From: Daniel Stenberg Date: Sun, 17 Apr 2011 16:34:25 +0200 Subject: INTERNALS: clean up Clarified the release procedure --- docs/INTERNALS | 46 +++++++++++++++++++++++++++++----------------- 1 file changed, 29 insertions(+), 17 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 5b12639ee..c7308671c 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -39,7 +39,7 @@ Portability libssh2 0.16 c-ares 1.6.0 libidn 0.4.1 - *yassl 1.4.0 (http://curl.haxx.se/mail/lib-2008-02/0093.html) + cyassl 1.4.0 openldap 2.0 MIT krb5 lib 1.2.4 qsossl V5R2M0 @@ -256,10 +256,10 @@ Library http_chunks.c contains functions that understands HTTP 1.1 chunked transfer encoding. - An interesting detail with the HTTP(S) request, is the Curl_add_buffer() series of - functions we use. They append data to one single buffer, and when the - building is done the entire request is sent off in one single write. This is - done this way to overcome problems with flawed firewalls and lame servers. + An interesting detail with the HTTP(S) request, is the Curl_add_buffer() + series of functions we use. They append data to one single buffer, and when + the building is done the entire request is sent off in one single write. This + is done this way to overcome problems with flawed firewalls and lame servers. FTP @@ -285,7 +285,7 @@ Library LDAP - Everything LDAP is in lib/ldap.c. + Everything LDAP is in lib/ldap.c and lib/openldap.c GENERAL @@ -463,9 +463,9 @@ Memory Debugging Test Suite ========== - Since November 2000, a test suite has evolved. It is placed in its own - subdirectory directly off the root in the curl archive tree, and it contains - a bunch of scripts and a lot of test case data. + The test suite is placed in its own subdirectory directly off the root in the + curl archive tree, and it contains a bunch of scripts and a lot of test case + data. The main test script is runtests.pl that will invoke test servers like httpserver.pl and ftpserver.pl before all the test cases are performed. The @@ -481,13 +481,25 @@ Building Releases ================= There's no magic to this. When you consider everything stable enough to be - released, run the 'maketgz' script (using 'make distcheck' will give you a - pretty good view on the status of the current sources). maketgz prompts for - version number of the client and the library before it creates a release - archive. maketgz uses 'make dist' for the actual archive building, why you - need to fill in the Makefile.am files properly for which files that should - be included in the release archives. - - NOTE: you need to have curl checked out from git to be able to do a proper + released, do this: + + 1. Tag the source code accordingly. + + 2. run the 'maketgz' script (using 'make distcheck' will give you a pretty + good view on the status of the current sources). maketgz requires a + version number and creates the release archive. maketgz uses 'make dist' + for the actual archive building, why you need to fill in the Makefile.am + files properly for which files that should be included in the release + archives. + + 3. When that's complete, sign the output files. + + 4. Upload + + 5. Update web site and changelog on site + + 6. Send announcement to the mailing lists + + NOTE: you must have curl checked out from git to be able to do a proper release build. The release tarballs do not have everything setup in order to do releases properly. -- cgit v1.2.1 From 84e7ea2ffcfc22a30a2079cfb9711ed32a9d625c Mon Sep 17 00:00:00 2001 From: Colin Hogben Date: Fri, 23 Dec 2011 10:56:48 +0000 Subject: Require a less ancient version of perl The INTERNALS document suggested that compatibility should be maintained with perl version 4, but this was untrue - scripts such as chksource.pl and runtests.pl use perl5-isms. --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index c7308671c..f965ec516 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -68,7 +68,7 @@ Portability GNU Autoconf 2.57 GNU Automake 1.7 (we currently avoid 1.10 due to Solaris-related bugs) GNU M4 1.4 - perl 4 + perl 5.004 roffit 0.5 groff ? (any version that supports "groff -Tps -man [in] [out]") ps2pdf (gs) ? -- cgit v1.2.1 From ee7e4fc1d1d45713e9e67a955e78f0d9ed1a6059 Mon Sep 17 00:00:00 2001 From: toddouska Date: Tue, 27 Dec 2011 12:17:37 -0800 Subject: cyassl: update to CyaSSL 2.0.x API Modify configure.ac to test for new CyaSSL Init function and remove default install path to system. Change to CyaSSL OpenSSL header and proper Init in code as well. Note that this no longer detects or works with CyaSSL before v2 --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index f965ec516..39c4df720 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -39,7 +39,7 @@ Portability libssh2 0.16 c-ares 1.6.0 libidn 0.4.1 - cyassl 1.4.0 + cyassl 2.0.0 openldap 2.0 MIT krb5 lib 1.2.4 qsossl V5R2M0 -- cgit v1.2.1 From 919c97fa65a5c00f7044e849eeb0095408413505 Mon Sep 17 00:00:00 2001 From: Yang Tse Date: Fri, 6 Apr 2012 23:35:15 +0200 Subject: curl tool: use configuration files from lib directory Configuration files such as curl_config.h and all config-*.h no longer exist nor are generated/copied into 'src' directory, now these only exist in 'lib' directory from where curl tool sources uses them. Additionally old src/setup.h has been refactored into src/tool_setup.h which now pulls lib/setup.h The possibility of a makefile needing an include path adjustment exists. --- docs/INTERNALS | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index 39c4df720..b87e2ce20 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -104,9 +104,9 @@ Windows vs Unix Inside the source code, We make an effort to avoid '#ifdef [Your OS]'. All conditionals that deal with features *should* instead be in the format '#ifdef HAVE_THAT_WEIRD_FUNCTION'. Since Windows can't run configure scripts, - we maintain two curl_config-win32.h files (one in lib/ and one in src/) that - are supposed to look exactly as a curl_config.h file would have looked like on - a Windows machine! + we maintain a curl_config-win32.h file in lib directory that is supposed to + look exactly as a curl_config.h file would have looked like on a Windows + machine! Generally speaking: always remember that this will be compiled on dozens of operating systems. Don't walk on the edge. -- cgit v1.2.1 From 42aa796150a580a0adff714c157d3b38b7672c7f Mon Sep 17 00:00:00 2001 From: Kamil Dudka Date: Fri, 6 Apr 2012 16:05:25 +0200 Subject: nss: unconditionally require PK11_CreateGenericObject() This bumps the minimal supported version of NSS to 3.12.x. --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index b87e2ce20..d2bff0ce5 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -43,7 +43,7 @@ Portability openldap 2.0 MIT krb5 lib 1.2.4 qsossl V5R2M0 - NSS 3.11.x + NSS 3.12.x axTLS 1.2.7 Heimdal ? -- cgit v1.2.1 From cb787b70bf5fa16bc1f83405c16d6c092e6daf46 Mon Sep 17 00:00:00 2001 From: Dan Fandrich Date: Fri, 20 Jul 2012 21:02:58 +0200 Subject: Fixed some typos in documentation --- docs/INTERNALS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'docs/INTERNALS') diff --git a/docs/INTERNALS b/docs/INTERNALS index d2bff0ce5..d06d62209 100644 --- a/docs/INTERNALS +++ b/docs/INTERNALS @@ -220,7 +220,7 @@ Library done" loop. It loops if there's a Location: to follow. When completed, the curl_easy_cleanup() should be called to free up used - resources. It runs Curl_disconnect() on all open connectons. + resources. It runs Curl_disconnect() on all open connections. A quick roundup on internal function sequences (many of these call protocol-specific function-pointers): -- cgit v1.2.1