Downloads and analysis with HTTPie

Slashdot it! Delicious Share on Facebook Tweet! Digg!
Andrea De Martin, 123RF

Andrea De Martin, 123RF

Web Spy

The ultra-slim web server HTTPie turns out to be an elegant synthesis of curl and wget, making it ideal for use with pipes.

HTTPie has many features of wget and curl , allowing various types of data transfer. It's also suitable for tasks such as detailed website analysis. Output can be redirected to a file, and file inputs are also possible. This makes HTTPie [1] ideal for use with pipes.

On Ubuntu, you can install HTTPie using apt :

sudo apt install httpie

or grabbing the latest version of the source code from the GitHub repository [2].

Call HTTPie using the command:

http <options> <method> <URL>

If you don't specify any options, you'll just see the website's code (Figure 1).

Figure 1: You can call a web page without any additional options.

Important Options

Table 1 lists some of HTTPie's the most useful options, while Table 2 lists important access methods to HTTP servers.

Table 1


Action Option Note
Specify the output scope --print HBhb H (request header), B (request body), h (reply header), B (reply body)
"Complete output" for pipe and output redirection -v
Download -d [URL] Alternatively: --download
Assign name to a download -o [FILENAME] Default: Original name
Resume interrupted download -c
Specify user name -a [USERNAME]

Web Page Output

One pitfall when outputting web pages is that http detects whether output is redirected or given to a pipe. With standard usage, you'll find yourself frustrated at every turn:

http <IP> / <name> <file>

simply does not work.

To resolve this, you need to use the -v or --print options along with the command. You can use the option shown in Listing 1 and Figure 2 for instance. In this example, the web server was also prompted to list available HTTP methods (again refer to Table 2).

Listing 1

# Example 1 Querying a Web Page
http --print Hh $1
echo "-------------------------------------------------"
# Example 2 Query HTTP methods, for the next-
# Processing, the -v option is required:
echo -n "Possible HTTP methods: "
http -v options $1 | grep Allow | cut -d: -f2
echo "-------------------------------------------------"
Figure 2: Output of web pages and further processing.

Table 2

Most Useful HTTP methods

Command Function
get Request contents of a web server (default)
head Request web page headers.
trace Output the request as it arrives at the requested web server
options Lists the HTTP methods allowed on the requested web server
post Send data to the web server (e.g., in the form field = value )
put Send a file to the web server

Use the --print HhBb option to narrow down what HTTPie shows, and see what you send and receive. You can use this for shell scripts. Listing 1 uses this technique. Figure 3 shows the output of two requests sent to the same web server. The first shows the request header (--print H ); the second shows the reply header (--print h ).

Figure 3: Use the option --print to capture different parts of the back and forth with the server.

If you want to check the HTTP status codes, use the following command

http --print h | grep "HTTP" | cut -d" " -f3

in the shell script. Note the blank character used as a field separator for the cut command.

If you look at the first line of the second example's output in Figure 3, you can probably figure out what the line will return: OK .

Buy this article as PDF

Express-Checkout as PDF

Pages: 6

Price $0.99
(incl. VAT)

Buy Ubuntu User

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content