Bug: ThreadingGroup fails with 'Private key file is encrypted' when using SSH agent

Bug Report: ThreadingGroup fails with "Private key file is encrypted" when using SSH agent

Description

When using ThreadingGroup with SSH agent forwarding, only one connection works properly while others fail with "Private key file is encrypted" errors. This happens even though the SSH agent has the key loaded and works fine when using SerialGroup instead.

Environment

• Fabric version: 3.2.2
• Python version: 3.11
• Operating system: macOS (or other relevant OS)

Steps to Reproduce

1. Set up an SSH agent and add a private key that requires a passphrase
2. Ensure the key is loaded in the agent (verify with ssh-add -l)
3. Create a Python script that:
   • Creates a gateway connection to a bastion host
   • Uses ThreadingGroup to connect to multiple hosts through the gateway
   • Enables agent forwarding (forward_agent=True)
   • Sets allow_agent=True in connect_kwargs

gateway = fabric.Connection(
    host="bastion.example.com",
    user="user",
    connect_kwargs={
        "allow_agent": True,
        "look_for_keys": True,
    },
    forward_agent=True,
)

group = fabric.ThreadingGroup(
    *["host1.example.com", "host2.example.com"],
    user="user",
    gateway=gateway,
    connect_kwargs={
        "allow_agent": True,
        "look_for_keys": True,
    },
    forward_agent=True,
)

results = group.run("hostname")

Expected Behavior

All connections should successfully authenticate using the SSH agent, and the command should execute on all hosts.

Actual Behavior

Only one connection succeeds, while others fail with the error:

Exception: Private key file is encrypted

Analysis

The issue appears to be related to how ThreadingGroup handles SSH agent connections in parallel. When multiple threads attempt to use the SSH agent simultaneously, only one succeeds while others fail.

This is likely a race condition in how Paramiko (the underlying SSH library) interacts with the SSH agent when multiple threads are involved. The issue does not occur with SerialGroup because it processes connections one at a time, allowing each connection to properly access the SSH agent.

Workaround

Using SerialGroup instead of ThreadingGroup resolves the issue:

group = fabric.SerialGroup(  # Changed from ThreadingGroup
    *["host1.example.com", "host2.example.com"],
    user="user",
    gateway=gateway,
    connect_kwargs={
        "allow_agent": True,
        "look_for_keys": True,
    },
    forward_agent=True,
)

However, this workaround sacrifices the performance benefits of parallel connections.

Additional Information

This issue might be related to how Paramiko's AgentRequestHandler is implemented or how it interacts with multiple threads. The problem specifically occurs when using a gateway connection with agent forwarding enabled.