An incredibly fast proxy checker & IP rotator with ease.
An incredibly fast proxy checker & IP rotator with ease.
Contributing •
What's new •
Documentation •
Report Issues
---
- [Features](#features)
- [Why mubeng?](#why-mubeng)
- [Installation](#installation)
- [Binary](#binary)
- [Docker](#docker)
- [Source](#source)
- [Usage](#usage)
- [Basic](#basic)
- [Options](#options)
- [Notes](#notes)
- [Install SSL Certificate](#install-ssl-certificate)
- [Examples](#examples)
- [Proxy checker](#proxy-checker)
- [Proxy IP rotator](#proxy-ip-rotator)
- [Burp Suite Upstream Proxy](#burp-suite-upstream-proxy)
- [OWASP ZAP Proxy Chain](#owasp-zap-proxy-chain)
- [Proxy format](#proxy-format)
- [Templating](#templating)
- [Amazon API Gateway](#amazon-api-gateway)
- [Limitations](#limitations)
- [Known Bugs](#known-bugs)
- [Contributors](#contributors)
- [Pronunciation](#pronunciation)
- [Changes](#changes)
- [License](#license)
---
# Features
- **Proxy IP rotator**: Rotates your IP address for every specified request.
- **Proxy checker**: Verifies the availability of your proxy IPs.
- **Supports all HTTP/S methods**.
- **Compatible with HTTP, SOCKS v4(A), v5, and Amazon API Gateway** proxy protocols.
- **Passes all parameters and URIs**.
- **User-friendly**: Simply run it against your proxy file and select the desired action.
- **Cross-platform**: Runs seamlessly on Windows, Linux, Mac, and even Raspberry Pi.
# Why mubeng?
It's fairly simple, there is no need for additional configuration.
`mubeng` has 2 core functionality:
### 1. Run proxy server as proxy IP rotation
This is useful to avoid different kinds of IP ban, i.e. bruteforce protection, API rate-limiting or WAF blocking based on IP. We also leave it entirely up to user to use proxy pool resources from anywhere.
### 2. Perform proxy checks
So, you don't need any extra proxy checking tools out there if you want to check your proxy pool.
# Installation
## Binary
Simply, download a pre-built binary from [releases page](https://github.com/mubeng/mubeng/releases) and run!
## Docker
Pull the [Docker](https://docs.docker.com/get-docker/) image by running:
```bash
▶ docker pull ghcr.io/mubeng/mubeng:latest
```
## Source
Using [Go](https://golang.org/doc/install) compiler:
```bash
▶ go install -v github.com/mubeng/mubeng@latest
```
### — or
Manual building executable from source code:
```bash
▶ git clone https://github.com/mubeng/mubeng
▶ cd mubeng
▶ make build
▶ (sudo) install ./bin/mubeng /usr/local/bin
```
# Usage
For usage, it's always required to provide your proxy list, whether it is used to check or as a proxy pool for your proxy IP rotation.
## Basic
```bash
▶ mubeng [-c|-a :8080] -f file.txt [options...]
```
## Options
Here are all the options it supports.
```bash
▶ mubeng -h
```
| **Flag** | **Description** |
|------------------------------- |-------------------------------------------------------------- |
| -f, --file `` | Proxy file. |
| -a, --address `:` | Run proxy server. |
| -A, --auth `:` | Set authorization for proxy server. |
| -d, --daemon | Daemonize proxy server. |
| -c, --check | To perform proxy live check. |
| -g, --goroutine `` | Max. goroutine to use (default: 50). |
| --only-cc `,` | Only show specific country code (comma separated). |
| -t, --timeout | Max. time allowed for proxy server/check (default: 30s). |
| -r, --rotate `` | Rotate proxy IP for every `AFTER` request (default: 1). |
| --rotate-on-error | Rotate proxy IP and retry failed HTTP requests. |
| --remove-on-error | Remove proxy IP from proxy pool on failed HTTP requests. |
| --max-errors `` | Max. errors allowed during rotation (default: 3). |
| | Use this with `--rotate-on-error`. |
| | If value is less than 0 (e.g., -1), rotation will |
| | continue indefinitely. |
| --max-redirs `` | Max. redirects allowed (default: 10). |
| --max-retries `` | Max. retries for failed HTTP requests (default: 0). |
| -m, --method `` | Rotation method (sequent/random) (default: sequent). |
| -s, --sync | Sync will wait for the previous request to complete. |
| -v, --verbose | Dump HTTP request/responses or show died proxy on check. |
| -o, --output `` | Save output from proxy server or live check. |
| -u, --update | Update mubeng to the latest stable version. |
| -w, --watch | Watch proxy file, live-reload from changes. |
| -V, --version | Show current mubeng version. |
NOTES:
- Rotations are counted for all requests, even if the request fails.
- The proxy server runs asynchronously by default, so it doesn't guarantee that your requests after N (which is N+1 and so on) will rotate the proxy IP, instead use the
-s/--sync flag to wait for requests to the previous proxy to complete.
- Daemon mode
(-d/--daemon) will install mubeng as a service on the (Linux/OSX) system/setting up callback (Windows).
- Hence you can control service with
journalctl, service or net (for Windows) command to start/stop proxy server.
- Whenever you activate the daemon mode, it works by forcibly stop and uninstalling the existing mubeng service, then re-install and starting it up in daemon.
- Verbose mode
(-v/--verbose) and timeout (-t/--timeout) apply to both proxy check and proxy IP rotation actions.
- HTTP traffic requests and responses is displayed when verbose mode
(-v/--verbose) is enabled, but
- We DO NOT explicitly display the request/response body, and
- All cookie values in headers will be redacted automatically.
- If you use output option
(-o/--output) to run proxy IP rotator, request/response headers are NOT written to the log file.
- A timeout option
(-t/--timeout) value is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, such as "5s", "300ms", "-1.5h" or "2h45m".
- Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", and "h".
- The max. errors
(--max-errors) ≠ max. retries (--max-retries).
- The max. retries
(--max-retries) apply to retrying a failed HTTP request with the same proxy.
- Meanwhile, max. errors
(--max-errors) refer to the total failed HTTP requests from the proxies used.
- For example, once the max. retries
(--max-retries) are reached (if supplied), the error is counted, and the proxy IP will rotate for the same failed HTTP request until the max. errors (--max-errors) are reached, whereas max. retries (--max-retries) DO NOT trigger a proxy rotation.
- If the value of max. errors
(--max-errors) is less than 0 (e.g. -1), rotation will continue indefinitely.
|
## Install SSL Certificate
mubeng uses built-in certificate authority by [GoProxy](https://github.com/elazarl/goproxy). With mubeng proxy server running, the generated certificate can be exported by visiting `http://mubeng/cert` in a browser.
Installation steps for CA certificate is [similar to other](https://portswigger.net/burp/documentation/desktop/external-browser-config/certificate) proxy tools.
## Examples
For example, you've proxy pool `(proxies.txt)` as:
http://127.0.0.1:8080
https://127.0.0.1:443
socks4://127.0.0.1:4145
socks5://127.0.0.1:2121
...
...
|
> Because we use auto-switch transport, `mubeng` can accept multiple proxy protocol schemes at once.
> Please refer to [documentation](https://pkg.go.dev/github.com/mubeng/mubeng/pkg/mubeng#Transport) for this package.
### Proxy checker
Pass `--check` flag in command to perform proxy checks:
```bash
▶ mubeng -f proxies.txt --check --only-cc AU,US,UK --output live.txt
```
The above case also uses `--output` flag to save a live proxy of specific country code with `--only-cc` flag (`ISO-3166` alpha-2) into file _(live.txt)_ from checking result.
(Figure: Checking proxies mubeng with max. 5s timeout)
#### Custom Output Format
You can customize the output format of the proxy checker using the `--output-format` flag with [fasttemplate](https://github.com/valyala/fasttemplate) syntax:
```bash
▶ mubeng -f proxies.txt --check --output-format "{{proxy}} | {{country}} | {{duration}}"
```
**Available template variables:**
| Variable | Description | Example |
|----------|-------------|---------|
| `{{proxy}}` | Full proxy URL | `http://192.168.1.1:8080` |
| `{{protocol}}` | Proxy protocol scheme | `http`, `https`, `socks5` |
| `{{host}}` | Proxy host/IP address | `192.168.1.1` |
| `{{port}}` | Proxy port | `8080` |
| `{{ip}}` | External IP address | `203.0.113.1` |
| `{{country}}` | Country code | `US`, `UK`, `AU` |
| `{{city}}` | City name | `New York`, `London` |
| `{{org}}` | Organization/ISP | `Google Inc.` |
| `{{region}}` | Region/State | `California`, `England` |
| `{{timezone}}` | Timezone | `America/New_York` |
| `{{loc}}` | Latitude,Longitude | `40.7128,-74.0060` |
| `{{hostname}}` | Hostname | `example.com` |
| `{{duration}}` | Response time | `245ms` |
**Examples:**
```bash
# JSON-like format
▶ mubeng -f proxies.txt --check --output-format '{"proxy":"{{proxy}}","country":"{{country}}","duration":"{{duration}}"}'
# CSV format
▶ mubeng -f proxies.txt --check --output-format "{{proxy}},{{country}},{{city}},{{duration}}"
# Custom detailed format
▶ mubeng -f proxies.txt --check --output-format "[{{country}}] {{proxy}} ({{org}}) - {{duration}}"
```
When using `--output-format`, the formatted output is applied to both console display and file output (when using `-o`/`--output` flag).
### Proxy IP rotator
Furthermore, if you wish to do proxy IP rotator from proxies that are still alive earlier from the results of checking `(live.txt)` _(or if you have your own list)_, you must use `-a` _(--address)_ flag inst