Skip to content

Feature: Llm hub refactoring #1695

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 58 commits into from
Jan 15, 2024
Merged

Feature: Llm hub refactoring #1695

merged 58 commits into from
Jan 15, 2024

Conversation

@kevinmessiaen
Copy link
Member

@kevinmessiaen kevinmessiaen commented Dec 21, 2023

Description

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've updated the code style using make codestyle.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in Google format for all the methods and classes that I used.
kevinmessiaen and others added 20 commits December 7, 2023 11:41
@kevinmessiaen
Added a LLM-as-judge per row evaluator
e0202b0
@kevinmessiaen
Removed wrong @abstractmethod
d3e02fc
@kevinmessiaen
Fixed issue with requirements
17a825f
@kevinmessiaen
Code improvement
d311ce2
@kevinmessiaen
Code improvement
28c0086
@kevinmessiaen
Added test for PerRowRequirementEvaluator
9b30a63
@kevinmessiaen
Added test for PerRowRequirementEvaluator
f00c7c7
@kevinmessiaen
Made output_ds optional for evaluators
bb8893d
@kevinmessiaen
Updated test name
7b854bb
@kevinmessiaen
Merge branch 'main' into feature/gsk-2300-llm-as-a-judge-row-wise-cus…
1f7e22a 
…tom-requirement-test
@kevinmessiaen
Merge branch 'main' into feature/gsk-2300-llm-as-a-judge-row-wise-cus…
12521cc 
…tom-requirement-test
@kevinmessiaen
Merge branch 'main' into feature/gsk-2300-llm-as-a-judge-row-wise-cus…
15d5ae2 
…tom-requirement-test
@kevinmessiaen
Removed legacy debugging for test_llm_single_output_against_requirement
2337c7a
@kevinmessiaen
Merge remote-tracking branch 'origin/feature/gsk-2300-llm-as-a-judge-…
88f930d 
…row-wise-custom-requirement-test' into feature/gsk-2300-llm-as-a-judge-row-wise-custom-requirement-test
@kevinmessiaen
Merge branch 'main' into feature/gsk-2300-llm-as-a-judge-row-wise-cus…
908261e 
…tom-requirement-test
@kevinmessiaen
Fixed tests
3b78afb
@kevinmessiaen
Merge branch 'feature/gsk-2300-llm-as-a-judge-row-wise-custom-require…
6a615f0 
…ment-test' into feature/gsk-2383-add-the-test-metadata-in-the-test-result-page
@kevinmessiaen
Added test result details structure and updated LLM-as-judge evaluators
7070191
@kevinmessiaen
Fixed typing
d99dba9
@henchaves
Merge pull request #1691 from Giskard-AI/feature/gsk-2383-add-the-tes…
d14c6d8 
…t-metadata-in-the-test-result-page

Add the test metadata in the test result page
@kevinmessiaen kevinmessiaen changed the title Feature/llm hub refactoring Dec 21, 2023
kevinmessiaen and others added 9 commits December 21, 2023 17:14
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
2d75f75 
# Conflicts:
#	giskard/datasets/base/__init__.py
#	giskard/llm/evaluators/base.py
#	giskard/testing/tests/llm/output_requirements.py
#	tests/communications/fixtures/with_alias.json
#	tests/testing/test_llm_output_requirement.py
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
66d5090
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
1b4a3d9
@kevinmessiaen
Fixed tests
c2ee9e8
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
6a13dfe
@kevinmessiaen
Updated ground truth tests to have new test details
b621789
@kevinmessiaen
Merge pull request #1711 from Giskard-AI/feature/gsk-2444-update-the-…
0d9bb45 
…similarity-test-to-allow-having-test-details

Updated ground truth tests to have new test details
@andreybavt
Merge branch 'main' into feature/llm-hub-refactoring
fa8a47d
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
a5c4055
@kevinmessiaen kevinmessiaen force-pushed the feature/llm-hub-refactoring branch from 3d73856 to 528ddc7 Compare January 11, 2024 05:33
kevinmessiaen and others added 21 commits January 11, 2024 13:51
@kevinmessiaen
Fixed tests for pass/fail in test_llm_ground_truth and test_llm_groun…
ebb8061 
…d_truth_similarity
@kevinmessiaen
Fixed tests
52f2995
@kevinmessiaen
Fixed tests
c3fe149
@andreybavt
Merge branch 'feature/llm-hub-refactoring' into feature/gsk-2437-get-…
b2058ad 
…logs-from-a-running-job
@kevinmessiaen
Updated test results to enum
aaa92d9
@kevinmessiaen
Fix issue with debugger when text_generation have target
fbb51fd
@kevinmessiaen
Merge branch 'main' into feature/llm-hub-refactoring
9b7e3a8 
# Conflicts:
#	giskard/testing/tests/llm/injections.py
@kevinmessiaen
Fixed tests
e752ada
@andreybavt
GSK-2437 Get logs from a running job: do not save suite logs in the DB
28e2d2f
@andreybavt
GSK-2437 Get logs from a running job: removed old log collector
89ecd01
@andreybavt
Merge branch 'feature/llm-hub-refactoring' into feature/gsk-2437-get-…
33cfae0 
…logs-from-a-running-job
@andreybavt
GSK-2437 fix import
7831419
@andreybavt
GSK-2437 fix import
c7047c3
@andreybavt
GSK-2437 fix tests
1cce4a3
@andreybavt
GSK-2437 fix tests
785fc04
@andreybavt
GSK-2437 fix tests
0dc545f
@andreybavt
GSK-2437 Get logs from a running job: raise exception if log file doe…
e8315f6 
…sn't exist
@andreybavt
Merge branch 'main' into feature/llm-hub-refactoring
a3d18e4
@andreybavt
Merge branch 'feature/llm-hub-refactoring' into feature/gsk-2437-get-…
f7b61e2 
…logs-from-a-running-job

# Conflicts:
#	tests/communications/fixtures/with_alias.json
@andreybavt
GSK-2437 regenerate with_alias.json
694fd1a
@andreybavt
Merge pull request #1727 from Giskard-AI/feature/gsk-2437-get-logs-fr…
88e5831 
…om-a-running-job

GSK-2437 Get logs from a running job
@andreybavt andreybavt added the Lockfile Temporary label to update pdm.lock label Jan 15, 2024
@github-actions github-actions bot removed the Lockfile Temporary label to update pdm.lock label Jan 15, 2024
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jan 15, 2024

Quality Gate Passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
87.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud
@andreybavt andreybavt marked this pull request as ready for review January 15, 2024 15:17
@andreybavt andreybavt merged commit 98b1984 into main Jan 15, 2024
@andreybavt andreybavt deleted the feature/llm-hub-refactoring branch January 15, 2024 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants

@kevinmessiaen @andreybavt @henchaves