- You build or work with API’s on your daily job
- You work with JSON a lot and you need a way to quickly manipulate it, transform, filter, find and/or do any other relevant operations
- You already have CURL on your computer
- You don’t have to install or pay for anything
- It’s light fast, consumes no resources
You’ll feel like a hacker with that green terminal theme🤮
- You’ve been using tools like Postman which were 99% overhead for what you actually needed
- You can combine it with other bash tools you already use and simply pipe to output to whatever you want
- You don’t have a computer
- You are not yet brought to the planet to exist amongst the rest of us apes
- Download and install Jq
- CURL cheatsheet - If you haven’t used curl before, take 5 minutes and familiarize yourself with it
To establish some ground, let’s make sure we understand what CURL and Jq are. (Descriptions taken from the world wide web, links included)
curl is a tool to transfer data from or to a server, using one of the supported protocols (HTTP, HTTPS, FTP, FTPS, SCP, SFTP, TFTP, DICT, TELNET, LDAP or FILE).
jq is a lightweight and flexible command-line JSON processor. If you are a command line addict, you will like the official description. jq is like sed for JSON data – you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
Get ready to rumble
The best way to start is by demonstrating why would one even consider using CURL with Jq. In this specific article, we will mainly focus on fetching JSON data and doing transformations on it.
We will use the Fake JSON Placeholder API to demonstrate how to do transformations using Jq. Before we get to it, let’s first learn about Pipes. If you are already familiar with the concept, feel free to skip it.
In simple words, pipes enable us to pass the output of one program as the input to another one.
someProgramThatReturnsSomething | someOtherProgram
# Whatever 'someProgramThatReturnsSomething' returns will be passed to 'someOtherProgram'
# The | in-between them is called a pipe operator
The reason why pipes are so powerful is that we can keep passing the data to the next program (function) as long as the previous one returns some data. This enables us to pipe the data through as many different tools (functions) as we want, and eventually get the desired ouput. Then we can either store this output as a file or do whatever we want with it. You can think of this like function composition in functional programming.
Transforming JSON with Jq
Now when we know what pipes are and how to use them, let’s finally start with Jq.
Since Jq provides various methods for transforming data, I will showcase the ones I most commonly use. Once you get the feel of how the Jq works, make sure you check the official documentation.
Filters are methods which are gonna be applied to our data. They are the essential part of Jq.
Dot filter prettifies the JSON output by formatting it and adding some colors for enhanced readability. Try the example below in your terminal with and without Jq and you will notice the difference.
curl https://jsonplaceholder.typicode.com/users/1 | jq .
Let’s learn how we can get a single property of an object. Let’s imagine we only want to get the username property
curl https://jsonplaceholder.typicode.com/users/1 | jq ".username"
You can also provide several object keys by separating them by a comma
curl https://jsonplaceholder.typicode.com/users/1 | jq ".id, .username"
curl https://jsonplaceholder.typicode.com/users/1 | jq ".company.name"
curl https://jsonplaceholder.typicode.com/users/1 | jq "keys"
That’s essentially it for this quick demonstration of jq. For many other filters, make sure to check the official documentation. In case you’ve liked what you read, good, I’m happy for you.