# Basic cURL Commands Every Web Developer Should Know

Whether you‘re a software developer, sysadmin, or tech enthusiast, chances are you‘ve come across **cURL** at some point. Curious what this ubiquitous tool is all about? You‘ve come to the right place!.

By the end of this article, you will have a strong foundation of this powerful command-line tool to start curling like pro !

## Brief History Of cURL

**cURL came into existenance into 1997**, created by **Daniel Stenberg** to facilitate the transferring data over various network protocols. Back then it was named HTTP Get focused solely on HTTP request to fetch data.

**Daniel** was motivated by the need for command line that could makes downloading data without complex code. By ***1998***, the project was renamed to **cURL** to reflect its boarder protocol support.

**Fub Fact :** The name **cURL** is not an acronym. The **c** for **client** while **URL** indicated it does internet transfers.

## What is cURL Used For

cURL is used for transferring data to or from server and some of most used cases are :

1. Transferring data with various protocols like HTTP, FTP, LDAP, MQTT, POP3, SMTP, Telnet etc.
    
2. Making API calls to web APIs — cURL lets you to easily test REST or SOAP APIs.
    
3. Automation : It can be used to script repitative tasks like data transfers.
    
4. web scraping : extract data from website through scripts using cURL.
    
5. Downloading Files on remote sever via SFTP, FTP etc.
    
6. uploading data and Files to web severs and APIs.
    

## Installing cURL

to check cURL is install on your system or not . use `curl —version`

if not install from package manager based on your system:

* **Linux** : install from : apt, yam , zyapper
    
* **Window** : Download binaries from cURL website or use a package manager like Chocolatey
    
* **MacOS** : `brew install curl`
    

## Essential cURL Commands

**cURL** is very easy to use . its basic syntax is `curl [options] [URL]`

### 1 . Basic Webpage Retrieval | GET Request

`curl https://example.com`

This command fetch HTML source code and prints it to terminal.

### 2 . Redirects

website often redirect user to different URLS. To follow redirects use `-L` or `—location` option.

`curl -L` [`https://bit.ly/shortened-url`](https://bit.ly/shortened-url)

to limit the redirect hops to prevent infinite loops use :

`curl -L —max-redirs 5` [`https://bit.ly/shortened-url`](https://bit.ly/shortened-url)

Use case : Testing shortened URLs, following API redirects, or checking redirect chains in your applications.

### 3 . Saving Output to a file

Instead of displaying output to a terminal , you can save into a file using `-O` or `-o` options.

```basic
# Save with original filename
curl -O https://example.com/data.json

# Save with custom filename
curl -o my-data.json https://api.example.com/export

# Save headers and body separately
curl -D headers.txt -o response.json https://api.example.com/data
```

**Use case** : Downloading Files, saving API responses for analysis, or archiving test results.

### 4 . Understanding HTTP Headers

HTTP headers provide valuable information about server’s response. you can view these headers using `-i` or `-I` option.

```basic
# Include headers with response
curl -i https://api.example.com/endpoint

# Headers only (useful for checking status codes) -I or --head
curl -I https://api.example.com/endpoint
```

**Use case** : Checking response codes, debugging caching issues, or examining API headers.

### 5 . Setting User-Agent

The User-Agent header identifies the client making the request . some website may server different content based on the User-Agent. you can customize the header using `-A` or `—user-agent` option:

### 6 . Working With Cookies

Cookies are the small piece of data that website store on the user’s computer to remember information about them.

```basic
# Save cookies to file
curl -c cookies.txt https://example.com/login

# Use saved cookies
curl -b cookies.txt https://example.com/dashboard
```

### 7 . Handling POST Request

**POST** request are commonly used to submit data to the server such as form data or JSON payloads. you can send **POST** request using `-d` or `—data` option

```basic
curl -d "name=John&age=30" https://example.com/submit

# send JSON Data
curl -X POST \
      -H "Content-Type:application/json" \
       -d '{"name":"John Doe", "email":"john@example.com"}'
        https://example.com/submit

# referencing JSON File
curl -X POST \
      -H "Content-Type:application/json" \
       -d @user-data.json \
        https://example.com/submit
```

Use case :Creating resources, submitting forms, or testing POST endpoints with structured data.

### 8 . Authentication

Many API require authentication . **cURL** provides several options for handling authentication including basic authentication and Beare token

```basic
# Basic Authentication
curl -u username:password https://example.com/protected
#Bearer Token
curl -H "Authorization:Bearer TOKEN" \
       https://example.com/api/protected
```

### 9 . Downloading Files

**cURL** is also useful for downloading files from command line. we have already seen `-O` option to save file with some name .

```basic
curl -O https://example.com/large_file.zip

# To resume Interupted downloads
curl -C - -O https://example.com/large_file.zip 
```

### 10 . Uploading Files

**cURL** can also upload files to server.

```basic
# Standard form data
curl -X POST -d "username=john&password=secret" https://example.com/login

# File upload
curl -F "file=@document.pdf" -F "description=Important doc" https://example.com/upload
```

### 11 . Working With Proxy

Route requests through proxies for testing or security:

```basic
curl --proxy http://proxy.company.com:8080 https://api.example.com
```

### 12 . Timeouts

Network operation can sometime take long time or even hang indefinitely. To prevent this you can set timeout using `—connect-timeout` or `—max-time`

* **—connect-timeout** specifies the maximum time in seconds to wait for a connection to be established
    
* **—max-time** specifies the maximum time in seconds that the entire application is allowed to take.
    

```basic
# 10-second connection timeout, 30-second total timeout
curl --connect-timeout 10 --max-time 30 https://slow-api.example.com
```

### 13 . Verbose mode

When requests fail or behave unexpectedly, verbose mode shows exactly what’s happening:

```basic
curl -v https://example.com
```

This displays:

* Request headers sent
    
* Response headers received
    
* SSL/TLS handshake details
    
* Connection information
    

**Use cases** : Debugging failed requests, understanding API behavior, or troubleshooting SSL issues.

## Advanced cURL Technque

### 1 . Using jq for JSON parsing

**first install jq**

```basic
sudo apt update
sudo apt install jq
```

then use to display json data like:

```basic
curl https://api.github.com/users/dipeshxhy | jq
# to get specific fields
curl https://api.github.com/users/hiteshchoudhary | jq ".public_repos"
```

### 2 . SSL Certificate Handling

For development environments with self-signed certificates:

```basic
# Skip certificate verification (development only)
curl -k https://localhost:8443/api

# Use custom CA certificate
curl --cacert custom-ca.pem https://secure-api.example.com
```

## Best Practice for Using cURL

To maximize the effectiveness of cURL and avoid common pitfalls, follow these best practices:

* **Error Handling:** Always check the exit code of cURL to ensure that the command executed successfully. A non-zero exit code indicates an error.
    
* **Security:** Be careful when using cURL with sensitive data, such as passwords or API keys. Avoid storing these credentials directly in scripts. Consider using environment variables or configuration files.
    
* **Rate Limiting:** Respect API rate limits to avoid being blocked. Implement delays in your scripts to prevent excessive requests.
    
* **Documentation:** Consult the cURL documentation for detailed information about options and usage. The `man curl` command provides access to the manual page on Linux systems.
    
* **Testing:** Thoroughly test your cURL commands and scripts to ensure that they work as expected. Use a testing environment before deploying to production.
    

## Summary

**cURL** is a powerful and versatile command-line tool for transferring data with URLs. It supports a wide range of protocols including HTTP, HTTPS, FTP, and more. cURL is widely used by developers and system administrators for testing APIs, automating file downloads and uploads, debugging network issues, and scripting complex data transfers. Its flexibility and extensive options make it an essential tool for web development, DevOps, and cloud workflows.

Mastering these commands will allow you to not only test and debug your own applications, but also to effectively integrate with external services and automate crucial tasks in your development workflow.