Checklist
❓ Question
I want to use a local dataset with the LLMInformationDisclosureDetector, but it doesn't seem that the evaluator uses it. Is it intended? If not, how can I use such detector without generating test cases? I want to only use the one in my file.
📎 Additional context
I think it's a follow-up to the Issue #1893 . Using the given answer, the detector still doesn't use the given dataset for evaluation, it still generate test cases.
Moreover, in giskard.scanner.llm.base.py script the given dataset is only used for language detection.
|
languages = dataset.extract_languages(columns=model.meta.feature_names) |