多线程的 AWS 库存采集工具，重点关注与安全相关的资源和元数据。
A multi-threaded AWS security-focused inventory collection tool written in Ruby.
This tool was created to facilitate efficient collection of a large amount of AWS resource attributes and metadata. It aims to collect nearly everything that is relevant to the security configuration and posture of an AWS environment.
Existing tools (e.g. AWS Config) that do some form of resource collection lack the coverage and specificity to accurately measure security posture (e.g. detailed resource attribute data, fully parsed policy documents, and nested resource relationships).
AWS Recon handles collection from large accounts by taking advantage of automatic retries (either due to network reliability or API throttling), automatic paging of large responses (> 100 resources per API call), and multi-threading parallel requests to speed up collection.
** usage does not imply endorsement
AWS Recon needs an AWS account role or credentials with ReadOnlyAccess. Full AdministratorAccess is over-privileged, but will work as well. The SecurityAudit policy is not sufficient as it omits access to many services.
Use Docker version 19.x or above to run the pre-built image without having to install anything.
If you already have Ruby installed (2.6.x or 2.7.x), you may want to install the Ruby gem.
AWS Recon can be run locally via a Docker container or by installing the Ruby gem.
To run via a Docker a container, pass the necessary AWS credentials into the Docker run command. For example:
$ docker run -t --rm \
-e AWS_REGION \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN \
-v $(pwd)/output.json:/recon/output.json \
darkbitio/aws_recon:latest \
aws_recon -v -s EC2 -r global,us-east-1,us-east-2
To run locally, first install the gem:
$ gem install aws_recon
Fetching aws_recon-0.5.17.gem
Fetching aws-sdk-3.0.1.gem
Fetching parallel-1.20.1.gem
...
Successfully installed aws-sdk-3.0.1
Successfully installed parallel-1.20.1
Successfully installed aws_recon-0.5.17
Or add it to your Gemfile using bundle:
$ bundle add aws_recon
Fetching gem metadata from https://rubygems.org/
Resolving dependencies...
...
Using aws-sdk 3.0.1
Using parallel-1.20.1
Using aws_recon 0.5.17
AWS Recon will leverage any AWS credentials (see requirements) currently available to the environment it runs in. If you are collecting from multiple accounts, you may want to leverage something like aws-vault to manage different credentials.
$ aws-vault exec profile -- aws_recon
Plain environment variables will work fine too.
$ AWS_PROFILE= aws_recon
To run from a Docker container using aws-vault managed credentials (output to stdout):
$ aws-vault exec -- docker run -t --rm \
-e AWS_REGION \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN \
darkbitio/aws_recon:latest \
aws_recon -j -s EC2 -r global,us-east-1,us-east-2
To run from a Docker container using aws-vault managed credentials and output to a file, you will need to satisfy a couple of requirements. First, Docker needs access to bind mount the path you specify (or a parent path above). Second, you need to create an empty file to save the output into (e.g. output.json). This is because only that one file is mounted into the Docker container at run time. For example:
Create an empty file.
$ touch output.json
Run the aws_recon container, specifying the output file.
$ aws-vault exec -- docker run -t --rm \
-e AWS_REGION \
-e AWS_ACCESS_KEY_ID \
-e AWS_SECRET_ACCESS_KEY \
-e AWS_SESSION_TOKEN \
-v $(pwd)/output.json:/recon/output.json \
darkbitio/aws_recon:latest \
aws_recon -s EC2 -v -r global,us-east-1,us-east-2
You may want to use the -v or --verbose flag initially to see status and activity while collection is running.
In verbose mode, the console output will show:
...
The t prefix indicates which thread a particular request is running under. Region, service, and operation indicate which request operation is currently in progress and where.
…
# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2
$ AWS_PROFILE= aws_recon -s S3,EC2 -r global,us-east-1,us-east-2
# collect S3 and EC2 global resources, as well as us-east-1 and us-east-2
$ AWS_PROFILE= aws_recon --services S3,EC2 --regions global,us-east-1,us-east-2
# save output to S3 bucket
$ AWS_PROFILE= aws_recon \
--services S3,EC2 \
--regions global,us-east-1,us-east-2 \
--verbose \
--s3-bucket my-recon-bucket
# save output to S3 bucket with a home region other than us-east-1
$ AWS_PROFILE= aws_recon \
--services S3,EC2 \
--regions global,us-east-1,us-east-2 \
--verbose \
--s3-bucket my-recon-bucket:us-west-2
Example OpenCSPM formatted (NDJSON) output.
$ AWS_PROFILE= aws_recon -l \
-s S3,EC2 \
-r global,us-east-1,us-east-2 \
-f custom
or
$ AWS_PROFILE= aws_recon -j \
-s S3,EC2 \
-r global,us-east-1,us-east-2 \
-f custom > output.json
API exceptions related to permissions are silently ignored in most cases. These errors are usually due to one of these cases:
In verbose mode, you will see exception logs in the output:
…
$ git clone [email protected]:darkbitio/aws-recon.git $ cd aws-recon
Create a sticky gemset if using RVM:
$ rvm use 2.7.2@aws_recon_dev --create --ruby-version
Run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
### TODO
- [ ] Test coverage with AWS SDK stubbed resources
## Kudos
AWS Recon was inspired by the excellent work of the people and teams behind these tools:
- CloudMapper [https://github.com/duo-labs/cloudmapper](https://github.com/duo-labs/cloudmapper)
- Prowler [https://github.com/toniblyx/prowler](https://github.com/toniblyx/prowler)
- CloudSploit [https://github.com/cloudsploit/scans](https://github.com/cloudsploit/scans)
暂无开放 Issues，或尚未同步最近议题。