Optimize TradingAgents execution complexity by reducing redundant computation and improving scalability

Problem

The current TradingAgents execution pipeline works correctly, but several components have potential scalability issues when running longer analysis sessions, multiple tickers, or repeated backtesting.

The main performance bottlenecks identified:

---

1. Sequential analyst execution increases wall time complexity

Currently, analyst nodes are executed sequentially:

Market Analyst
      ↓
Social/Sentiment Analyst
      ↓
News Analyst
      ↓
Fundamentals Analyst

These analysts are mostly independent and only generate separate reports.

Current complexity:

O(N × T_agent)

Where:

• N = number of analysts
• T_agent = average analyst execution time

For multiple analysts:

Total latency ≈ T_market + T_news + T_social + T_fundamental

Proposed optimization

Execute independent analyst nodes concurrently:

              ┌─ Market Analyst
START ────────┼─ News Analyst
              ├─ Sentiment Analyst
              └─ Fundamentals Analyst
                    ↓
              Research Manager

Expected improvement:

Before:

O(N × T_agent)

After:

O(max(T_agent))

---

2. TradingMemoryLog has O(n) lookup overhead

Current implementation scans the entire history when storing decisions.

Example:

raw = self._log_path.read_text()

for line in raw.splitlines():
    ...

Complexity:

Time: O(n)
Memory: O(n)

Where:

n = total historical log entries

As history grows, every new operation becomes slower.

Proposed optimization

Maintain an indexed metadata structure:

{
  "2026-01-01_AAPL": {
    "offset": 12345
  }
}

Duplicate detection becomes:

O(1)

instead of:

O(n)

---

3. Memory context loading repeatedly parses full history

Current flow:

get_past_context()
        |
        ↓
load_entries()
        |
        ↓
parse entire history file

Complexity:

O(n)

for every agent execution.

Proposed optimization

Introduce:

• Lazy loading
• LRU cache
• Incremental parser
• SQLite backend option

Example:

First request:
Markdown → parsed cache

Later requests:
O(1)/O(log n) lookup

---

4. Debate history growth causes prompt complexity explosion

Current debate states continuously append:

bull_history
bear_history
risk_history

Prompt size increases with debate rounds.

Complexity:

O(R × H)

Where:

• R = number of debate rounds
• H = average history length

Proposed optimization

Implement history compression:

Before:

Round 1:
full response

Round 2:
full response + Round 1

Round 3:
full response + Round 1 + Round 2

After:

Round 3:
compressed summary + latest response

Possible approaches:

• Token-based truncation
• Periodic summarization
• Embedding retrieval instead of full history injection

---

5. Market data caching layer improvement

Multiple agents request overlapping data:

Market Analyst
      ↓
get_stock_data()

Technical Analyst
      ↓
get_indicators()

Fundamental Analyst
      ↓
financial statements

Current network complexity:

O(number_of_agents × API_calls)

Proposed optimization

Introduce request-level cache:

Cache key:

(symbol, data_type, date_range)

Example:

AAPL_price_2026-01-01_2026-09-01

Expected improvement:

Before:

Multiple duplicated API requests

After:

Unique dataset requests only

---

Optimization Priority

Priority Optimization Expected impact

---

High Parallel analyst execution Reduce latency significantly High Memory indexing/cache Improve long-running performance Medium Debate history compression Reduce token cost Medium Shared market data cache Reduce API calls Low Minor loop optimization Limited impact

---

Expected Result

After optimization:

• Lower end-to-end analysis latency
• Better scalability for multi-stock backtesting
• Reduced API usage
• Reduced LLM token consumption
• More predictable runtime complexity

---

Benchmark Proposal

Scenario:

10 tickers × 30 trading days

Measure:

• Total execution time
• API calls
• LLM token usage
• Memory loading time

Compare:

Before optimization vs After optimization