Skip to content

test(mcp_server): add smoke test for tools/call deadlock regression#86

Merged
ykykj merged 1 commit into
HKUDS:mainfrom
Teerapat-Vatpitak:test/mcp-server-smoke-regression
May 10, 2026
Merged

test(mcp_server): add smoke test for tools/call deadlock regression#86
ykykj merged 1 commit into
HKUDS:mainfrom
Teerapat-Vatpitak:test/mcp-server-smoke-regression

Conversation

@Teerapat-Vatpitak

Copy link
Copy Markdown
Contributor

Summary

  • Add a regression test for #85 (the MCP tools/call deadlock fix).
  • Spawns vibe-trading-mcp over stdio and verifies the JSON-RPC happy path: initialize + tools/list + tools/call analyze_options.
  • No network dependency; total runtime ~4s with the fix in place.

Why

#85 fixes a deadlock where tools/call hangs indefinitely if the registry pre-warm in mcp_server.main() is removed. Without a regression test, a future contributor could revert that line during a refactor and reintroduce the bug — initialize and tools/list would still pass, so the breakage would be silent until users hit tools/call.

This test fails deterministically (15s timeout on tools/call) when the pre-warm is removed, with an assertion message that points back at #85.

What it covers

  1. initialize — JSON-RPC handshake completes within 30s (covers cold imports).
  2. tools/list — at least 20 tools registered, including analyze_options, get_market_data, and list_skills. Catches tool-registration regressions and missing package data.
  3. tools/call analyze_options — pure CPU Black-Scholes calculation; must return within 15s. This is the call that deadlocks without the fix.
  4. Result shapeprice > 0, delta ∈ [0, 1]. Catches Black-Scholes formula regressions.

Differential validation

Verified locally on Windows + Python 3.12 + fastmcp from main:

Run mcp_server.main() state Result Wall time
1 with pre-warm (current main) PASS 3.91s
2 pre-warm commented out FAIL — timeout 15s on tools/call 17.45s
3 pre-warm restored PASS 3.81s

tools/list succeeds in all three runs — only tools/call exposes the deadlock, which is the exact symptom users hit (e.g. #42, #32).

Why no network in the test

analyze_options is a Black-Scholes calculation with no external service. Picking it over get_market_data keeps the test:

  • Deterministic (no API rate-limits / outages)
  • CI-friendly (no outbound network needed)
  • Fast (sub-second after init)

Test plan

  • pytest agent/tests/test_mcp_server_smoke.py -v passes locally (~4s)
  • Test fails with a timeout when the pre-warm in mcp_server.main() is removed
  • Restoring the pre-warm makes the test pass again
  • Test only spawns one subprocess and cleans it up in finally; no shared state with other tests

Notes for reviewer

Tested locally on Windows + Python 3.12 with the differential validation table above. CI will be the first run on Linux + Python 3.11. The test uses only portable APIs (subprocess.Popen, pathlib.Path, os.pathsep, sys.executable, threading), so cross-platform issues are unlikely — but I'm happy to follow up with fixes if anything surfaces in CI.

This is also the first subprocess-spawning test in this repo (existing tests use direct imports / FastAPI TestClient). I picked subprocess because the deadlock only manifests across the FastMCP server thread boundary, which is hard to mock faithfully. Happy to refactor toward a fixture-based / mocked approach if that fits the repo's testing philosophy better.

Checklist

  • No changes to protected areas (agent/src/agent/, agent/src/session/, agent/src/providers/) — only adds a test file under agent/tests/.
  • No hardcoded secrets / paths; uses sys.executable and resolves repo root from __file__.
  • Code follows CONTRIBUTING.md (Conventional Commit title, @pytest.mark.integration, Google-style docstrings).
  • Documentation updated — N/A (test file is self-documenting).
Spawns vibe-trading-mcp via subprocess and walks through the JSON-RPC
happy path: initialize + tools/list + tools/call analyze_options. The
critical assertion is that tools/call does not hang — without the
registry pre-warm in mcp_server.main() (PR #85), the lazy registry build
runs inside FastMCP's worker thread and deadlocks on the first call.

The test also verifies (a) the tool catalogue has at least 20 entries
including analyze_options/get_market_data/list_skills, and (b) the
analyze_options result has a positive price and a delta in [0, 1] —
covering tool-registration regressions and Black-Scholes formula breaks
in addition to the deadlock.

analyze_options is chosen over network-dependent tools (get_market_data,
web_search) so the test stays deterministic and runs offline in CI.

Verified locally with differential validation:
- with pre-warm: PASS in 3.91s
- pre-warm commented out: FAIL with 15s timeout on tools/call
- pre-warm restored: PASS in 3.81s
@ykykj ykykj merged commit 318c276 into HKUDS:main May 10, 2026
1 check passed
@warren618

Copy link
Copy Markdown
Collaborator

Thanks for adding this MCP smoke test. I verified the test locally as well, and it is useful for guarding against the tools/call deadlock regression.

One small note for future PRs: since this is an integration-style subprocess test, we may later tune the default pytest selection if the suite gets slower. For now this looks good.

@Teerapat-Vatpitak

Copy link
Copy Markdown
Contributor Author

Thanks for the local verify and the heads-up on integration-style test categorization — noted for future PRs (likely will add a @pytest.mark.integration marker if I introduce more subprocess tests, so default pytest runs stay fast).

@Teerapat-Vatpitak Teerapat-Vatpitak deleted the test/mcp-server-smoke-regression branch May 13, 2026 04:00
12zuev added a commit to 12zuev/Vibe-Trading that referenced this pull request May 25, 2026
…HKUDS#86)

Append-only experiment log per AUTORESEARCH.md protocol. Seeded with
the changes already shipped 2026-05-25:
  - Binance MIN_CONFIDENCE 0.61→0.55
  - TRADE_CRYPTO_ANALYSIS_LIMIT 3→10
  - INLINE_ANALYSIS_LIMIT_MAX 1→3
  - Binance MAX_CONCURRENT_POSITIONS 5→7
  - vol_penalty piecewise + dynamic ceiling (PR HKUDS#144, status=running,
    evaluate at 2026-05-26T17:15Z)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
12zuev added a commit to 12zuev/Vibe-Trading that referenced this pull request May 25, 2026
Karpathy pattern fully wired: AUTORESEARCH.md + experiments.tsv +
new cron job be4888eb that follows the 10-step experiment
protocol. Founder: безлимит на время/токены, цель — постоянный
рост в заработке. Cron now actually learns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants