Skip to content

AttributeError when I try to use ContextRelevance, LLMContextPrecisionWithoutReference to evaluate the test dataset #1990

Open
@Haoyuxiaohan

Description

@Haoyuxiaohan

Describe the bug
I got an AttributeError: property' object has no attribute 'get' when I try to use evaluate() function to evaluate a dataset with ContextRelevance, LLMContextPrecisionWithoutReference metrics. It's working when I only use faithfulness. And it's working when I evaluate a SingleTurnSample-

Ragas version: newest version
Python version: newest version

Code to Reproduce
result = evaluate(eval_dataset, metrics=[faithfulness, AnswerRelevancy], llm = evaluator_llm)

Error trace
File , line 27
22 eval_dataset = EvaluationDataset.from_pandas(df_bge_pd)
23 print(eval_dataset)
---> 27 result = evaluate(eval_dataset, metrics=[faithfulness, AnswerRelevancy], llm = evaluator_llm)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-f55819bb-501c-45c7-93b3-dc3c0125c4ca/lib/python3.11/site-packages/ragas/validation.py:60, in validate_required_columns(ds, metrics)
58 metric_type = get_supported_metric_type(ds)
59 for m in metrics:
---> 60 required_columns = set(m.required_columns.get(metric_type, []))
61 available_columns = set(ds.features())
62 if not required_columns.issubset(available_columns):

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmodule-metricsthis is part of metrics module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions