-
-
Notifications
You must be signed in to change notification settings - Fork 381
Open
Description
When running evaluate from giskard.rag with the specified metrics ragas_context_recall and ragas_context_precision, the following error occurs
from giskard.rag import evaluate, RAGReport, AgentAnswer
from giskard.rag.metrics.ragas_metrics import ragas_context_recall, ragas_context_precision
def answer_fn(question, history=None):
answer = chat_engine.chat(question, chat_history=[])
return AgentAnswer(
message=answer.response,
documents=[source.content for source in answer.sources]
)
report = evaluate(answer_fn,
testset=testset,
knowledge_base=knowledge_base,
metrics=[ragas_context_recall, ragas_context_precision])
report.save("test_report")ERROR
KeyError Traceback (most recent call last)
File c:\Users\KUNJAN SHAH\AppData\Local\Programs\Python\Python311\Lib\site-packages\pandas\core\indexes\base.py:3805, in Index.get_loc(self, key)
3804 try:
-> 3805 return self._engine.get_loc(casted_key)
3806 except KeyError as err:
File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()
File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()
File pandas\\_libs\\hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()
File pandas\\_libs\\hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()
KeyError: 'correctness'```