The recent mess with the Let’s Encrypt root certificate expiring has meant some weird issues have been popping up.
We found ourselves troubleshooting a potential firewall issue that was affecting only a certain subset of users. There seemed like there was a pattern, but what exactly?
- Was it location-based?
- Was it time-based?
- Internet Provider based?
- Something else?
The only lead we had was that changing to a VPN magically fixed things for the end-users. As a next step, we thought we would examine the metadata associated with the set of IPs that were having problems.
So what do you do when you need to gather lots of info for something you’ve never examined before?
Turn to the terminal and script it!
The basics
There’s probably any number of services we could turn to, but ipinfo.io seemed like a good one. A query to get metadata about an IP address looks like this:
curl http://ipinfo.io/104.57.176.115
{
"ip": "104.57.176.115",
"hostname": "104-57-176-115.lightspeed.austtx.sbcglobal.net",
"city": "Austin",
"region": "Texas",
"country": "US",
"loc": "30.2672,-97.7431",
"org": "AS7018 AT&T Services, Inc.",
"postal": "78701",
"timezone": "America/Chicago",
"readme": "https://ipinfo.io/missingauth"
}
Iterating over many IPs
That’s good for one-off requests, and it also means that we can improve on it to handle a collection of IPs. For example, if a file named ip-list has IP addresses on each line, we can use a BASH while loop to fetch more info about each IP address.
# in a file called ip-list
102.176.65.190
104.219.136.17
104.57.176.115
104.57.29.69
107.190.28.137
109.48.146.95
112.134.164.187
116.97.53.100
12.90.145.30
121.200.4.138
We use a while loop to get more info!
( while read ip;
do curl -s "http://ipinfo.io/${ip}";
done < ips-list ) > ip-info.json
This works and ultimately got me what I needed since I was able to then use jq to investigate various things within the ip-info.json file:
cat ip-info.json | jq -r '[.ip, .timezone, .country, .org] | @tsv'
102.176.65.190 Africa/Accra GH AS29614 Ghana Telecommunications Company Limited
104.219.136.17 America/Chicago US AS14143 Rock Solid Internet & Telephone
104.57.176.115 America/Chicago US AS7018 AT&T Services, Inc.
104.57.29.69 America/New_York US AS7018 AT&T Services, Inc.
107.190.28.137 America/Vancouver CA AS5645 TekSavvy Solutions, Inc.
109.48.146.95 Europe/Lisbon PT AS2860 NOS COMUNICACOES, S.A.
112.134.164.187 Asia/Colombo LK AS9329 Sri Lanka Telecom Internet
116.97.53.100 Asia/Ho_Chi_Minh VN AS7552 Viettel Group
12.90.145.30 America/Chicago US AS7018 AT&T Services, Inc.
121.200.4.138 Australia/Melbourne AU AS4764 Aussie Broadband
123.243.115.186 Australia/Sydney AU AS7545 TPG Telecom Limited
124.186.3.85 Australia/Brisbane AU AS1221 Telstra Corporation Ltd
134.56.52.132 America/New_York US AS23089 Hotwire Communications
136.35.164.218 America/Chicago US AS16591 Google Fiber Inc.
136.49.143.29 America/Chicago US AS16591 Google Fiber Inc.
136.49.143.31 America/Chicago US AS16591 Google Fiber Inc.
136.58.116.229 America/Chicago US AS16591 Google Fiber Inc.
Make it reusable
The above process was good enough for my needs at that moment, but I wondered if there was a way to make something that was reusable. I pictured the ability to run one command followed by any number of IP addresses which then prints out more information about each IP address in JSON format.
Something like this:
ipinfo 102.176.65.190 104.219.136.17 104.57.176.115
This ended up being fairly easy to create!
Since those arguments are interpreted by Bash as an array of values, we can do a for loop with a quick regex validation to make sure it’s valid IPv4 address before making our request to ipinfo.
#!/bin/bash
# Save and name this script something like ipinfo and put it in your $PATH
for arg; do
# Only make requests for valid IPv4 addresses
if [[ $arg =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
curl -s "http://ipinfo.io/${arg}";
fi
done;
The ergonomics of this script mean that it’s pretty composable. We can use something like xargs to mash on any number of IP addresses to be queried!
cat ips-list | xargs ipinfo # print to stdout
cat ips-list | xargs ipinfo \; > ip-info.json # redirect to a file