Contextual Content Discovery Tool
For the longest of times, content discovery has been focused on finding files and folders. While this approach is effective for legacy web servers that host static files or respond with 3xx’s upon a partial path, it is no longer effective for modern web applications, specifically APIs.
Over time, we have seen a lot of time invested in making content discovery tools faster so that larger wordlists can be used, however the art of content discovery has not been innovated upon.
Kiterunner is a tool that is capable of not only performing traditional content discovery at lightning fast speeds, but also bruteforcing routes/endpoints in modern applications.
Modern application frameworks such as Flask, Rails, Express, Django and others follow the paradigm of explicitly defining routes which expect certain HTTP methods, headers, parameters and values.
When using traditional content discovery tooling, such routes are often missed and cannot easily be discovered.
By collating a dataset of Swagger specifications and condensing it into our own schema, Kiterunner can use this dataset to bruteforce API endpoints by sending the correct HTTP method, headers, path, parameters and values for each request it sends.
Swagger files were collected from a number of datasources, including an internet wide scan for the 40+ most common swagger paths. Other datasources included GitHub via BigQuery, and APIs.guru.
You can download a pre-built copy from https://github.com/assetnote/kiterunner/releases.
# build the binary
make build
# symlink your binary
ln -s $(pwd)/dist/kr /usr/local/bin/kr
# compile the wordlist
# kr kb compile <input.json> <output.kite>
kr kb compile routes.json routes.kite
# scan away
kr scan hosts.txt -w routes.kite -x 20 -j 100 --ignore-length=1053
The JSON datasets can be found below:
Alternatively, it is possible to download the compile .kite files from the links below:
Users using a Arch based distro can download the pre-built binary from AUR
You can use a "Aur Helper" like yay to install kiterunner
yay -S kiterunner-bin
kr [scan|brute] <input> [flags]
<input> can be a file, a domain, or URI. we'll figure it out for you. See Input/Host Formatting for more details…
…
bruteforce flags (all the flags above +)
-D, --dirsearch-compat this will replace %EXT% with the extensions provided. backwards compat with dirsearch because shubs loves him some dirsearch
-e, --extensions strings extensions to append while scanning
-w, --wordlist strings normal wordlist to use for scanning
When supplied with an input, kiterunner will attempt to resolve the input in the following order:
If you supply a "domain", but it exists as a file, e.g. google.com but google.com is also a txt file in the current directory,
we'll load google.com the text file, because we found it first.
Domain Parsing
Its preferred that you provide a full URI as the input, however you can provide incomplete URIs and we'll try and guess what you mean. An example list of domains you can supply are:
one.com
two.com:80
three.com:443
four.com:9447
https://five.com:9090
http://six.com:80/api
The above list of domains will expand into the subsequent list of targets
(two targets are created for one.com, since neither port nor protocol was specified)
http://one.com (port 80 implied)
https://one.com (port 443 implied)
http://two.com (port 80 implied)
https://three.com (port 443 implied)
http://four.com:9447 (non-tls port guessed)
https://five.com:9090
http://six.com/api (port 80 implied; basepath API appended)
the rules we apply are:
When you have a single target
# single target
kr scan https://target.com:8443/ -w routes.kite -A=apiroutes-210228:20000 -x 10 --ignore-length=34
# single target, but you want to try http and https
kr scan target.com -w routes.kite -A=apiroutes-210228:20000 -x 10 --ignore-length=34
# a list of targets
kr scan targets.txt -w routes.kite -A=apiroutes-210228:20000 -x 10 --ignore-length=34
kr brute https://target.com -A=raft-large-words -A=apiroutes-210228:20000 -x 10 -d=0 --ignore-length=34 -ejson,txt
For when you have an old-school wordlist that still has %EXT% in the wordlist, you can use -D. this will only substitute the extension where %EXT% is present in the path
kr brute https://target.com -w dirsearch.txt -x 10 -d=0 --ignore-length=34 -ejson,txt -D
A key feature of kiterunner is depth based scanning. This attempts to handle detecting wildcards given virtual application path based routing. The depth defines how many directories deep the baseline checks are performed E.g.
~/kiterunner $ cat wordlist.txt
/api/v1/user/create
/api/v1/user/delete
/api/v2/user/
/api/v2/admin/
/secrets/v1/
/secrets/v2/
/ would have the baseline checks performed for wildcard detection/api and /secrets would have baseline checks performed; and these checks would be used against /api and /secrets correspondingly/api/v1, /api/v2, /secrets/v1 and /secrets/v2 would all have baseline checks performed.By default, kr scan has a depth of 1, since from internal usage, we've often seen this as the most common depth where virtual routing has occured. kr brute has a default depth of 0, as you typically don't want this check to be performed with a static wordlist.
Naturally, increasing the depth will increase the accuracy of your scans, however this also increases the number of requests to the target. (# of baseline checks * # of depth baseline directories). Hence, we recommend against going above 1, and in rare cases going to depth 2.
We provide inbuilt downloading and caching of wordlists from assetnote.io. You can use these with the -A flag which receives a comma delimited list of aliases, or fullnames.
You can get a full list of all the Assetnote wordlists with kr wordlist list.
The wordlists when used, are cached in ~/.cache/kiterunner/wordlists. When used, these are compiled from .txt -> .kite
…
Usage
kr scan targets.txt -A=apiroutes-210228 -x 10 --ignore-length=34
kr brute targets.txt -A=aspx-210228 -x 10 --ignore-length=34 -easp,aspx
When using assetnote provided wordlists, you may not want to use the entire wordlist, so you can opt to use the first N lines in a given wordlist using the head syntax. The format is <wordlist_name>:<N lines> when specifying a wordlist.
Usage
# this will use the first 20000 lines in the api routes wordlist
kr scan targets.txt -A=apiroutes-210228:20000 -x 10 --ignore-length=34
# this will use the first 10 lines in the aspx wordlist
kr brute targets.txt -A=aspx-210228:10 -x 10 --ignore-length=34 -easp,aspx
Kiterunner is made to go fast on a lot of hosts. But, just because you can run kiterunner at 20000 goroutines, doesn't mean its a good idea. Bottlenecks and performance degredation will occur at high thread counts due to more time spent scheduling goroutines that are waiting on network IO and kernel context switching.
There are two main concurrency settings for kiterunner:
-x, --max-connection-per-host - maximum number of open connections we can have on a host. Governed by 1 goroutine each. To avoid DOS'ing a host, we recommend keeping this in a low realm of 5-10. Depending on latency to the target, this will yield on average between 1-5 requests per second per connection (200ms - 1000ms/req) to a host.-j, --max-parallel-hosts - maximum number of hosts to scan at any given time. Governed by 1 goroutine supervisor for eachDepending on the hardware you are scanning from, the "maximum" number of goroutines you can run optimally will vary. On an AWS t3.medium, we saw performance degradation going over 2500 goroutines. Meaning, 500 hosts x 5 conn per host (2500) would yield peak performance.
We recommend against running kiterunner from your macbook. Due to poor kernel optimisations for high IO counts and Epoll syscalls on macOS, we noticed substantially poorer (0.3-0.5x) performance when compared to running kiterunner on a similarly configured linux instance.
To maximise performance when scanning an individual target, or a large attack surface we recommend the following tips:
-x and -j options. We recommend having a typical starting point of around -x 5 -j 100 and moving -j upwards as your CPU usage/network performance permitsKiterunner will also let you convert between the schema JSON, a kite file and a standard txt wordlist.
Usage
The format is decided by the filetype extension supplied by the <input> and <output> fields. We support txt, json and kite
kr kb convert wordlist.txt wordlist.kite
kr kb convert wordlist.kite wordlist.json
kr kb convert wordlist.kite wordlist.txt
…
When you receive a bunch of output from kiterunner, it may be difficult to immediately understand why a request is causing a specific response code/length. Kiterunner offers a method of rebuilding the request from the wordlists used in
No open issues yet, or sync has not completed.