Skip to content

Add Groq model support to LLMClient (#1977) #2165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jul 3, 2025
Merged

Add Groq model support to LLMClient (#1977) #2165

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
930099f
feat(llm): add GroqClient support to set any Groq model as default LL…
May 12, 2025
10a4b68
"Address PR feedback: make JSON format support less hardcoded and fix…
May 27, 2025
97ac6c6
Fix: Issues of previous comments
Jun 2, 2025
b9d3f01
Docs: Added Groq Client
Jun 9, 2025
97dc016
Fix: Linting Error
Jun 9, 2025
1659045
Fix: Resolve test failures and ensure all tests pass locally
Jun 11, 2025
3bc5b74
Merge branch 'main' into add-groq-client-support
kunjanshah0811 Jun 13, 2025
4f27293
Add get_config in groq_client.py file
Jun 13, 2025
ef817bf
Fix: code formatting via pre-commit
Jun 18, 2025
3a6362a
Fix: ValueError: Could not load model sentence-transformers/paraphras…
Jun 19, 2025
ff02a9e
Fix: Remove duplicate dependencies and organize LLM-related packages
Jun 20, 2025
8af4202
Apply suggestions from code review
davidberenstein1957 Jun 23, 2025
a70fbb6
fix: removed extra spaces in pyproject.toml
Jun 23, 2025
1fb5174
Merge branch 'add-groq-client-support' of https://github.com/kunjansh…
Jun 23, 2025
7c86562
refactor(llm): Revert making Groq a default LLM client
Jul 1, 2025
ff8d141
docs: Add note about Groq embedding support
Jul 1, 2025
c0f3a5b
Merge branch 'main' into add-groq-client-support
henchaves Jul 2, 2025
9ad79ec
undo changes in llm/client/__init__
henchaves Jul 2, 2025
92981aa
update docs order
henchaves Jul 2, 2025
7c5b277
update pdm.lock
henchaves Jul 2, 2025
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
undo changes in llm/client/__init__
  • Loading branch information
@henchaves
henchaves authored Jul 2, 2025
commit 9ad79ec322b4a82f780435bfdd6f16818ca10fc2
10 changes: 6 additions & 4 deletions giskard/llm/client/__init__.py
View file
Open in desktop
Copy link
Member

@henchaves henchaves Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should set groq as default LLM client.
Could we revert the modifications here?
Copy link
Contributor Author

@kunjanshah0811 kunjanshah0811 Jun 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should set groq as default LLM client. Could we revert the modifications here?

Thank you for the feedback. Before I make any changes, I want to ensure I understand your request correctly:

  1. I'll remove Groq from being automatically selected as a default LLM client by modifying the get_default_llm_api() function in __init__.py

  2. I'll keep the following in place so Groq can still be used when explicitly selected:

    • The Groq client implementation (groq_client.py)
    • The Groq dependency in pyproject.toml
    • The test case for the Groq client
    • "groq" as a valid option in the API list (users can still explicitly set GSK_LLM_API="groq")

Is this understanding correct? If you'd prefer I completely remove all Groq-related changes, please let me know.
Copy link
Member

@henchaves henchaves Jun 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @kunjanshah0811, yes exactly!
Just reverting llm/client/__init__.py should be fine, you can keep the groq_client.py as it is.
Copy link
Contributor Author

@kunjanshah0811 kunjanshah0811 Jul 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @kunjanshah0811, yes exactly! Just reverting llm/client/__init__.py should be fine, you can keep the groq_client.py as it is.

@henchaves I have made suggested changes with the recent commit. Please have a look and let me know. Thanks a lot 🚀.
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,12 @@ def get_default_llm_api() -> str:
global _default_llm_api
if _default_llm_api is None:
_default_llm_api = os.getenv(
"GSK_LLM_API",
"azure" if "AZURE_OPENAI_API_KEY" in os.environ else "openai",
"GSK_LLM_API", "azure" if "AZURE_OPENAI_API_KEY" in os.environ else "openai"
).lower()

if _default_llm_api not in {"azure", "openai", "groq"}:
if _default_llm_api not in {"azure", "openai"}:
logging.warning(
f"LLM-based evaluation is only working with `azure`, `openai`, 'groq'. Found {_default_llm_api} in GSK_LLM_API, falling back to `openai`"
f"LLM-based evaluation is only working with `azure` and `openai`. Found {_default_llm_api} in GSK_LLM_API, falling back to `openai`"
)
_default_llm_api = "openai"

Expand Down Expand Up @@ -91,6 +90,9 @@ def get_default_client() -> LLMClient:
global _default_llm_model
global _disable_structured_output

if _default_client is not None:
return _default_client

try:
from .litellm import LiteLLMClient

Expand Down
Loading