Skip to content

[GSK-1275] Importance of metrics calculated on partial data slice #1193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jun 21, 2023
Merged

[GSK-1275] Importance of metrics calculated on partial data slice #1193

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2e7f8b4
Fix performance metrics calculated on small size samples [GSK-1275] […
mattbit Jun 20, 2023
ee211eb
Add test
mattbit Jun 20, 2023
11a84a5
Clean up code
mattbit Jun 20, 2023
2d5000d
More tests
mattbit Jun 20, 2023
b85edf7
Fix tests
mattbit Jun 20, 2023
25c6288
Make p_value parameter optional in PerformanceIssueInfo
mattbit Jun 21, 2023
f24e541
Merge branch 'task/GSK-1078' into task/fix-scan-metrics
mattbit Jun 21, 2023
ce913b2
Merge branch 'task/GSK-1078' into task/fix-scan-metrics
andreybavt Jun 21, 2023
09fa043
Remove unused line
mattbit Jun 21, 2023
27340f7
Merge branch 'task/fix-scan-metrics' of github.com:Giskard-AI/giskard…
mattbit Jun 21, 2023
83a9d8f
Merge branch 'main' into task/fix-scan-metrics
mattbit Jun 21, 2023
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Clean up code
  • Loading branch information
@mattbit
mattbit committed Jun 20, 2023
commit 11a84a5ccaac8990818012ff821bdb6cdffc9aba
4 changes: 2 additions & 2 deletions python-client/giskard/scanner/performance/performance_bias_detector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,10 +251,10 @@ def _calculate_slice_metrics(model, dataset, metric, slice_fn, with_pvalue=False
)
else:
# otherwise, this must be classification scores, so we perform a G-test
slice_x_cnt = int(round(slice_metric.value * slice_metric.affected_samples))
slice_x_cnt = round(slice_metric.value * slice_metric.affected_samples)
slice_y_cnt = slice_metric.affected_samples - slice_x_cnt

comp_x_cnt = int(round(comp_metric.value * comp_metric.affected_samples))
comp_x_cnt = round(comp_metric.value * comp_metric.affected_samples)
comp_y_cnt = comp_metric.affected_samples - comp_x_cnt

ctable = [[slice_x_cnt, slice_y_cnt], [comp_x_cnt, comp_y_cnt]]
Expand Down