diff options
Diffstat (limited to 'docs/TheArtOfHttpScripting.md')
| -rw-r--r-- | docs/TheArtOfHttpScripting.md | 54 |
1 files changed, 27 insertions, 27 deletions
diff --git a/docs/TheArtOfHttpScripting.md b/docs/TheArtOfHttpScripting.md index 054c6267b..83b0905dc 100644 --- a/docs/TheArtOfHttpScripting.md +++ b/docs/TheArtOfHttpScripting.md @@ -2,7 +2,7 @@ ## Background - This document assumes that you're familiar with HTML and general networking. + This document assumes that you are familiar with HTML and general networking. The increasing amount of applications moving to the web has made "HTTP Scripting" more frequently requested and wanted. To be able to automatically @@ -59,7 +59,7 @@ want to know the amount of milliseconds between two points in a transfer. For those, and other similar situations, the [`--trace-time`](https://curl.se/docs/manpage.html#--trace-time) option - is what you need. It'll prepend the time to each trace output line: + is what you need. it will prepend the time to each trace output line: curl --trace-ascii d.txt --trace-time http://example.com/ @@ -73,14 +73,14 @@ ## Spec The Uniform Resource Locator format is how you specify the address of a - particular resource on the Internet. You know these, you've seen URLs like + particular resource on the Internet. You know these, you have seen URLs like https://curl.se or https://yourbank.com a million times. RFC 3986 is the canonical spec. And yeah, the formal name is not URL, it is URI. ## Host The host name is usually resolved using DNS or your /etc/hosts file to an IP - address and that's what curl will communicate with. Alternatively you specify + address and that is what curl will communicate with. Alternatively you specify the IP address directly in the URL instead of a name. For development and other trying out situations, you can point to a different @@ -92,7 +92,7 @@ ## Port number Each protocol curl supports operates on a default port number, be it over TCP - or in some cases UDP. Normally you don't have to take that into + or in some cases UDP. Normally you do not have to take that into consideration, but at times you run test servers on other ports or similar. Then you can specify the port number in the URL with a colon and a number immediately following the host name. Like when doing HTTP to port @@ -166,7 +166,7 @@ A single curl command line may involve one or many URLs. The most common case is probably to just use one, but you can specify any amount of URLs. Yes - any. No limits. You'll then get requests repeated over and over for all the + any. No limits. you will then get requests repeated over and over for all the given URLs. Example, send two GETs: @@ -185,13 +185,13 @@ ## Multiple HTTP methods in a single command line Sometimes you need to operate on several URLs in a single command line and do - different HTTP methods on each. For this, you'll enjoy the + different HTTP methods on each. For this, you will enjoy the [`--next`](https://curl.se/docs/manpage.html#-:) option. It is basically a separator that separates a bunch of options from the next. All the URLs before `--next` will get the same method and will get all the POST data merged into one. - When curl reaches the `--next` on the command line, it'll sort of reset the + When curl reaches the `--next` on the command line, it will sort of reset the method and the POST data and allow a new set. Perhaps this is best shown with a few examples. To send first a HEAD and then @@ -236,7 +236,7 @@ previous URL. If the original form was seen on the page `www.example.com/when/birth.html`, - the second page you'll get will become + the second page you will get will become `www.example.com/when/junk.cgi?birthyear=1905&press=OK`. Most search engines work this way. @@ -249,13 +249,13 @@ ## POST The GET method makes all input field names get displayed in the URL field of - your browser. That's generally a good thing when you want to be able to + your browser. That is generally a good thing when you want to be able to bookmark that page with your given data, but it is an obvious disadvantage if you entered secret information in one of the fields or if there are a large amount of fields creating a long and unreadable URL. The HTTP protocol then offers the POST method. This way the client sends the - data separated from the URL and thus you won't see any of it in the URL + data separated from the URL and thus you will not see any of it in the URL address field. The form would look similar to the previous one: @@ -315,7 +315,7 @@ A common way for HTML based applications to pass state information between pages is to add hidden fields to the forms. Hidden fields are already filled - in, they aren't displayed to the user and they get passed along just as all + in, they are not displayed to the user and they get passed along just as all the other fields. A similar example form with one visible field, one hidden field and one @@ -329,15 +329,15 @@ </form> ``` - To POST this with curl, you won't have to think about if the fields are - hidden or not. To curl they're all the same: + To POST this with curl, you will not have to think about if the fields are + hidden or not. To curl they are all the same: curl --data "birthyear=1905&press=OK&person=daniel" [URL] ## Figure Out What A POST Looks Like - When you're about fill in a form and send to a server by using curl instead - of a browser, you're of course interested in sending a POST exactly the way + When you are about fill in a form and send to a server by using curl instead + of a browser, you are of course interested in sending a POST exactly the way your browser does. An easy way to get to see this, is to save the HTML page with the form on @@ -364,7 +364,7 @@ ## Basic Authentication HTTP Authentication is the ability to tell the server your username and - password so that it can verify that you're allowed to do the request you're + password so that it can verify that you are allowed to do the request you are doing. The Basic authentication used in HTTP (which is the type curl uses by default) is **plain text** based, which means it sends username and password only slightly obfuscated, but still fully readable by anyone that sniffs on @@ -419,7 +419,7 @@ A HTTP request may include a 'referer' field (yes it is misspelled), which can be used to tell from which URL the client got to this particular resource. Some programs/scripts check the referer field of requests to verify - that this wasn't arriving from an external site or an unknown page. While + that this was not arriving from an external site or an unknown page. While this is a stupid way to check something so easily forged, many scripts still do it. Using curl, you can put anything you want in the referer-field and thus more easily be able to fool the server into serving your request. @@ -439,14 +439,14 @@ At times, you will see that getting a page with curl will not return the same page that you see when getting the page with your browser. Then you know it - is time to set the User Agent field to fool the server into thinking you're + is time to set the User Agent field to fool the server into thinking you are one of those browsers. To make curl look like Internet Explorer 5 on a Windows 2000 box: curl --user-agent "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" [URL] - Or why not look like you're using Netscape 4.73 on an old Linux box: + Or why not look like you are using Netscape 4.73 on an old Linux box: curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL] @@ -477,7 +477,7 @@ ## Other redirects Browser typically support at least two other ways of redirects that curl - doesn't: first the html may contain a meta refresh tag that asks the browser + does not: first the html may contain a meta refresh tag that asks the browser to load a specific URL after a set number of seconds, or it may use javascript to do it. @@ -529,7 +529,7 @@ Curl's "cookie engine" gets enabled when you use the [`--cookie`](https://curl.se/docs/manpage.html#-b) option. If you only want curl to understand received cookies, use `--cookie` with a file that - doesn't exist. Example, if you want to let curl understand cookies from a + does not exist. Example, if you want to let curl understand cookies from a page and follow a location (and thus possibly send back cookies it received), you can invoke it like: @@ -539,7 +539,7 @@ format that Netscape and Mozilla once used. It is a convenient way to share cookies between scripts or invokes. The `--cookie` (`-b`) switch automatically detects if a given file is such a cookie file and parses it, - and by using the `--cookie-jar` (`-c`) option you'll make curl write a new + and by using the `--cookie-jar` (`-c`) option you will make curl write a new cookie file at the end of an operation: curl --cookie cookies.txt --cookie-jar newcookies.txt \ @@ -580,7 +580,7 @@ verifying the server's certificate against a locally stored CA cert bundle. Failing the verification will cause curl to deny the connection. You must then use [`--insecure`](https://curl.se/docs/manpage.html#-k) - (`-k`) in case you want to tell curl to ignore that the server can't be + (`-k`) in case you want to tell curl to ignore that the server cannot be verified. More about server certificate verification and ca cert bundles can be read in @@ -628,7 +628,7 @@ curl -X POST http://example.org/ - ... but curl will still think and act as if it sent a GET so it won't send + ... but curl will still think and act as if it sent a GET so it will not send any request body etc. # Web Login @@ -651,7 +651,7 @@ Some web-based login systems feature various amounts of javascript, and sometimes they use such code to set or modify cookie contents. Possibly they do that to prevent programmed logins, like this manual describes how to... - Anyway, if reading the code isn't enough to let you repeat the behavior + Anyway, if reading the code is not enough to let you repeat the behavior manually, capturing the HTTP requests done by your browsers and analyzing the sent cookies is usually a working method to work out how to shortcut the javascript need. @@ -666,7 +666,7 @@ ## Some debug tricks - Many times when you run curl on a site, you'll notice that the site doesn't + Many times when you run curl on a site, you will notice that the site does not seem to respond the same way to your curl requests as it does to your browser's. |