[GSK-2035, GSK-2003, GSK-2263] Complete refactoring of prompt injection #1602
Conversation
GSK-2035 Complete refactoring of prompt injection detector
|
…into gsk-2035-refac
mattbit
left a comment
mattbit
left a comment
There was a problem hiding this comment.
I would rethink a bit the structure to reduce complexity. It is not strictly needed to use the "generator" structure, I feel something simpler and customized could work better in this case. But open to discussion on this.
|
For the verification, I've ran this notebook with this is what I get: results = scan(giskard_model, giskard_dataset, only=["jailbreak"])
test_suite = results.generate_test_suite("Test suite generated by scan")
test_suite.run()
The display of the test_suite is not optimal, as it displays all the list of evaluator configs, but we should solve this in a different PR |
|
mattbit
left a comment
mattbit
left a comment
There was a problem hiding this comment.
Looks good, I just need to do some QA locally.
| def data_loader(self): | ||
| if self._data_loader is None: | ||
| self._data_loader = PromptInjectionDataLoader(num_samples=self.num_samples) | ||
| return self._data_loader |
There was a problem hiding this comment.
nitpick: last line inside the if is not needed
Uh oh!
There was an error while loading. Please reload this page.