About - v1.12
The idea behind waymore is to find even more links from the Wayback Machine than other existing tools.
Anyone who does bug bounty will have likely used the amazing waybackurls by @TomNomNoms. This tool gets URLs from web.archive.org and additional links (if any) from one of the index collections on index.commoncrawl.org. You would have also likely used the amazing gau by @hacker_ which also finds URL's from wayback archive, Common Crawl, but also from Alien Vault and URLScan. Now waymore gets URL's from ALL of those sources too (with ability to filter more to get what you want):
- Wayback Machine (web.archive.org)
- Common Crawl (index.commoncrawl.org)
- Alien Vault OTX (otx.alienvault.com)
- URLScan (urlscan.io)
Installation
waymore supports Python 3.
$ git clone https://github.com/xnl-h4ck3r/waymore.git
$ cd waymore
$ sudo python setup.py install
if you're having a problem running the setup.py
for whatever reason you can run the following to install the dependencies:
$ sudo pip3 install -r requirements.txt
Run with docker
Install docker
git clone https://github.com/xnl-h4ck3r/waymore.git
cd waymore
Build image:
docker build -t waymore .
Run waymore with this command:
docker run -it --rm -v $PWD/results:/app/results waymore:latest python3 waymore.py -i example.com -mode U
Input and Mode
The input -i
can either be a domain only, e.g. redbull.com
or a specific domain and path, e.g. redbull.com/robots.txt
. You can also pass a file of domains/URLs to process (or pass values in by piping from another program on the command line).
There are different modes that can be run for waymore. The -mode
argument can be 3 different value:
U
- URLs will be retrieved from archive.org (if -xwm
is not passed), commoncrawl.org (if -xcc
is not passed), otx.alienvault.com (if -xvv
is not passed) and urlscan.io (if -xus
is not passed)R
- Responses will be downloaded from archive.orgB
- Both URLs and Responses will be retrieved
If the input was a specific URL, e.g. redbull.com/robots.txt
then the -mode
defaults to R
. Only responses will be downloaded. You cannot change the mode to U
or B
for a domain with path because it isn't necessary to retrieve URLs for a specific URL.
If the input is just a domain, e.g. redbull.com
then the -mode
defaults to B
. It can be changed to U
or R
if required. When a domain only is passed then all URLs/responses are retrieved for that domain (and sub domains unless -n
is passed). If the no sub domain option -n
is passed then the www
sub domain is still included by default.
*Beware click the link!