Skip to content

Fix format_answer() to re-raise OutputParserError for retry logic (Issue #3771) #3773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix format_answer() to re-raise OutputParserError for retry logic (Issue #3771) #3773

wants to merge 1 commit into from

Conversation

@devin-ai-integration
Copy link
Contributor

Fix format_answer() to re-raise OutputParserError for retry logic (Issue #3771)

Summary

This PR fixes a bug where format_answer() was catching all exceptions including OutputParserError and converting them to AgentFinish, which prevented the retry logic in _invoke_loop() from working correctly when LLMs return malformed output.

The Problem:
When an LLM returns malformed output (e.g., missing colons after "Thought", "Action", "Action Input"), the agent should retry with an error message. However, format_answer() was catching the OutputParserError and immediately returning an AgentFinish, bypassing the retry mechanism in crew_agent_executor.py:250-257.

The Fix:
Modified format_answer() to specifically catch and re-raise OutputParserError, allowing it to bubble up to the retry logic in _invoke_loop(). The generic Exception handler is preserved as a fallback for truly unexpected errors.

Changes:

  • Updated lib/crewai/src/crewai/utilities/agent_utils.py to re-raise OutputParserError
  • Added comprehensive unit tests in lib/crewai/tests/utilities/test_agent_utils.py
  • All 51 related tests (parser + agent_utils) pass

Review & Testing Checklist for Human

This is a yellow risk change (small, targeted fix with good unit test coverage, but limited end-to-end testing):

  • End-to-end verification: Test with an actual agent that receives malformed LLM output (missing colons in Thought/Action/Action Input). Verify the agent retries with error messages instead of immediately returning the malformed text
  • Full test suite: Run the complete test suite to ensure no regressions (I ran parser and agent_utils tests successfully, but not the full suite due to an unrelated qdrant import error)
  • Exception handling review: Verify that the generic Exception catch → AgentFinish fallback is the correct behavior for unexpected errors

Test Plan

To manually verify the fix works end-to-end:

  1. Create a simple agent with a task
  2. Mock or inject a malformed LLM response (e.g., "Thought\nThe user wants X.\nAction\nSome Tool\nAction Input:\n{...}")
  3. Verify the agent logs show retry attempts with error messages like "Invalid Format: I missed the ':' after 'Action:'"
  4. Verify the agent doesn't immediately finish with the malformed text

Notes
@devin-ai-integration
Fix format_answer() to re-raise OutputParserError for retry logic
29be99a 
This commit fixes issue #3771 where format_answer() was catching all
exceptions including OutputParserError and converting them to AgentFinish,
which prevented the retry logic in _invoke_loop() from working correctly.

Changes:
- Modified format_answer() in agent_utils.py to specifically catch and
  re-raise OutputParserError, allowing the retry logic to handle malformed
  LLM outputs properly
- Added comprehensive tests in test_agent_utils.py to verify the fix and
  prevent regressions

The fix ensures that when an LLM returns malformed output (e.g., missing
colons in the Action/Action Input format), the agent will retry with an
error message instead of immediately returning an AgentFinish with the
malformed text.

Co-Authored-By: João <joao@crewai.com>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant