“If you don't know what's running in your cloud, your wallet does.”
When working in AWS, it's easy to spin up infrastructure and forget about it. Between EC2 instances, Lambda functions, NAT Gateways, and S3 buckets, dormant resources can accumulate quickly—and so can the charges. Over the weekend, I wrote a Go-based utility to automate the discovery of AWS resources across all regions. The goal? To better understand where I was potentially bleeding money and take action before the monthly bill showed up.
In this post, I’ll walk you through the structure of the tool, key implementation choices, and how the data it outputs helps keep your AWS environment lean and cost-efficient.
Why This Script?
AWS provides cost management tools like Cost Explorer and Trusted Advisor, but sometimes you need a clear view of what is running and where. My script uses the AWS SDK for Go (v2) to enumerate a variety of billable AWS resources across every available region and export the results to a single JSON file.
This gives you:
- A cross-regional inventory of EC2 instances, Lambda functions, EBS volumes, RDS databases, etc.
- A record of attached infrastructure such as VPCs, Subnets, Internet/NAT Gateways, and Transit Gateway attachments.
- Visibility into global IAM users and roles.
- S3 bucket listings by region.
Key Components of the Script
1. Initialization and Configuration
The script begins by loading the default AWS configuration using the SDK:
cfg, err := config.LoadDefaultConfig(ctx)
This setup automatically picks up credentials from environment variables or the ~/.aws/credentials file.
2. Global Resource Collection – IAM and S3
IAM is global, not region-specific. The script pulls user and role data using:
client := iam.NewFromConfig(cfg)
For S3, although buckets are globally listed, their physical location affects latency and compliance. Each bucket’s region is queried with GetBucketLocation.
Example output from aws_resources.json:
{
"region": "global",
"iam_users": [
"cloud_user"
],
"iam_roles": [
"admin",
"AWSServiceRoleForAmazonEKS",
"AWSServiceRoleForConfig",
...
]
}
3. Regional Discovery Loop
The script dynamically fetches available AWS regions and loops through each one:
regionResp, _ := ec2Global.DescribeRegions(ctx, &ec2.DescribeRegionsInput{})
Inside each iteration, services like EC2, Lambda, RDS, and EC2 VPC components are queried regionally using the appropriate client.
For example, to get EC2 instances:
resp, err := ec2Client.DescribeInstances(ctx, &ec2.DescribeInstancesInput{})
Each service’s data is collected and stored in a ResourceReport struct, then added to a global slice for later export.
4. Network and Attached Resources
The script takes care to correlate data across services. For instance, when a Transit Gateway is attached to a VPC, the CIDR blocks of the VPC are fetched for context:
vpcResp, err := ec2Client.DescribeVpcs(ctx, &ec2.DescribeVpcsInput{VpcIds: []string{vpcID}})
This helps to understand how routing and connectivity may be impacting overall cost or complexity.
5. Export to JSON
After the full scan, all collected data is written to a single JSON file:
file, err := os.Create("aws_resources.json")
encoder := json.NewEncoder(file)
encoder.Encode(allReports)
This file can be used for human review, cost attribution, or further automation.
Real Data Example
From my own AWS environment, here’s an excerpt from the us-east-1 report:
{
"region": "us-east-1",
"ec2_instances": [
"i-0b63df5471ba418b7 | t2.micro | running"
],
"rds_instances": null,
"lambda_functions": [
"cfst-1449-682ea958117928f3654a3b3c539-InitFunction-QR8fVE7UmejW | python3.12"
],
"vpcs": [
"vpc-02e75185995836a59 | 172.31.0.0/16"
],
"subnets": [
"subnet-03a1448c851943781 | 172.31.80.0/20",
"subnet-0439da184c98a8758 | 172.31.16.0/20",
"subnet-0146e3bc01b82f487 | 172.31.64.0/20",
"subnet-0fa4922118f0ba8c1 | 172.31.0.0/20",
"subnet-07ab5133141f2d04c | 172.31.48.0/20",
"subnet-024e443a6f31a6d25 | 172.31.32.0/20"
],
"internet_gateways": [
"igw-016a0542bcf9d9c84"
],
"nat_gateways": null,
"route_tables": [
"rtb-030cb478d254dcf48"
],
"network_interfaces": [
"eni-086782b921dc7d154 | in-use",
"eni-08cbb0e3872cf8bb7 | in-use",
"eni-037fbda28d4b74e71 | in-use",
"eni-0a66d44c994c8e52c | in-use",
"eni-0cac78a6cfb477bf0 | in-use",
"eni-0f2f1c928f6a3c51e | in-use",
"eni-02ed24ba986c6a7eb | in-use"
],
"security_groups": [
"sg-041c7024a9a8cad80 | default",
"sg-0eb1f3babee948724 | launch-wizard-1"
],
"network_acls": [
"acl-012f69b92f681c107 | vpc-02e75185995836a59"
],
"transit_gateway_attachments": [
{
"region": "us-east-1",
"transit_gateway_id": "tgw-00df4a3fd97ad83f9",
"vpc_id": "vpc-02e75185995836a59",
"cidrs": "172.31.0.0/16",
"state": "available"
}
],
"ebs_volumes": [
"vol-032f7fdc7fb852d96 | 1374391425956 GiB"
],
"s3_buckets": [
"ujkhyiujkhbvjknjhfbjkbd"
]
}From this, I immediately spotted an EBS volume with an absurd size due to misconfiguration—something that would have cost hundreds if left unattended.
Lessons and Takeaways
- Visibility is crucial. You can’t optimize what you don’t know exists.
- Go + AWS SDK v2 is powerful. Native concurrency support and excellent SDK design make Go a great fit for this task.
- Automated cleanup next? The logical next step is to build automated scripts to terminate idle resources based on this JSON report.
Future Improvements
- Add Cost Estimation: Integrate the AWS Pricing API to estimate monthly charges per resource.
- Slack Alerts or Email Reports: Send daily summaries or unusual activity reports.
- Tag Awareness: Filter and group by cost-center or owner using tags.
Final Thoughts
Cloud sprawl is real—and expensive. Building simple tools like this one gives you confidence and control over your environment. With Go, AWS, and a bit of weekend hacking, you can take back that control and stay one step ahead of your AWS bill.
