How to cURL the Headers to a Website
Well before we run a cURL we must first know what a cURL is. Simply cURL is a command line tool for sending and receiving information using a URL. In the example below we will cURL davidpolanco.com and see if we can obtain the headers to the website.
Run this in the Command Line Interface (CLI)
curl -IL davidpolanco.com
Here is the output:
HTTP/1.1 301 Moved Permanently Date: Sun, 14 Aug 2016 21:48:25 GMT Connection: keep-alive Set-Cookie: __cfduid=d781a5f85860a7014d73e48e8067f9edb1471211305; expires=Mon, 14-Aug-17 21:48:25 GMT; path=/; domain=.davidpolanco.com; HttpOnly Location: https://davidpolanco.com/ Server: cloudflare-nginx CF-RAY: 2d279c63d5f954c8-ORD HTTP/1.1 200 OK Date: Sun, 14 Aug 2016 21:48:25 GMT Content-Type: text/html; charset=UTF-8 Connection: keep-alive Set-Cookie: __cfduid=d0a5a1bff0caba0f82b2cd8023a17104f1471211305; expires=Mon, 14-Aug-17 21:48:25 GMT; path=/; domain=.davidpolanco.com; HttpOnly Link: ; rel="https://api.w.org/" Expires: Tue, 13 Sep 2016 21:47:28 GMT Vary: Accept-Encoding,Cookie X-Cacheable: YES:2592000.000 Cache-Control: max-age=2592000, must-revalidate X-Cache: HIT: 4 X-Pass-Why: X-Cache-Group: normal X-Type: default X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Server: cloudflare-nginx CF-RAY: 2d279c65479b098e-ORD
As you can see from this information below that a 301 redirect is present from HTTP to HTTPS which then follows to a HTTP/1.1 200 OK. Once that is complete you get full information on the connection, cookie, whether the site is cacheable or not along with some security additions that I added to protect some cross-side scripting attacks and X-Frame injections.
X-Frame-Options: SAMEORIGIN<br> X-Content-Type-Options: nosniff<br> X-XSS-Protection: 1; mode=bl
Interested in in obtaining your external IP Address run this:
- icanhazip.com – returns your IP address
- icanhazptr.com – returns the reverse DNS record (PTR) for your IP
- icanhaztrace.com – returns a traceroute from my servers to your IP address
- icanhaztraceroute.com – returns a traceroute from my servers to your IP address
- icanhazepoch.com – returns the epoch time (also called Unix time)
- icanhazproxy.com – can determine if your traffic is being proxied
For more information on icanhazip visit them at: icanhazip.com
So with that in mind utilizing cURL can be awfully useful for diagnosing a redirect issue, download files, checking cookies, cache expires, and so much more.
You live in