Skip to content

Conversation

@liushz
Copy link
Collaborator

@liushz liushz commented Oct 31, 2025

No description provided.

@liushz liushz requested a review from Copilot October 31, 2025 08:31
@liushz liushz assigned liushz and unassigned tonysy Oct 31, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the SAGE (Scientific Advanced General Evaluation) dataset implementation to ATLAS (A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning), involving a comprehensive rename of classes, functions, variables, and configuration files throughout the codebase.

Key changes:

  • Renamed dataset classes, evaluator classes, and functions from SAGE to ATLAS naming convention
  • Updated dataset loading to use 'opencompass/ATLAS' and changed the answer field from 'standard_answer' to 'refined_standard_answer'
  • Added explicit string conversion for question and answer fields in the dataset loader
  • Reorganized configuration files by removing the old sage configs and adding new atlas configs

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
opencompass/datasets/atlas/prompt.py Newly added prompt templates, but uses incorrect SAGE_* variable names instead of ATLAS_*
opencompass/datasets/atlas/evaluation.py Renamed functions and class from sage_*/SAGE* to atlas_*/ATLAS*
opencompass/datasets/atlas/dataset_loader.py Renamed class to ATLASDataset, updated dataset path and field names with string conversions
opencompass/configs/datasets/sage/sage_gen.py Removed old SAGE configuration entry point
opencompass/configs/datasets/sage/README.md Removed old SAGE documentation
opencompass/configs/datasets/atlas/atlas_val_gen_906a48.py Renamed all references from SAGE to ATLAS, includes duplicate prompt definitions
opencompass/configs/datasets/atlas/atlas_gen.py New ATLAS configuration entry point
opencompass/configs/datasets/atlas/README.md New ATLAS documentation
Comments suppressed due to low confidence (2)

opencompass/configs/datasets/atlas/atlas_val_gen_906a48.py:11

  • The prompt templates ATLAS_INFER_TEMPLATE and ATLAS_EVAL_TEMPLATE are duplicated in this configuration file (lines 10-36 and 38-164). They should be imported from opencompass.datasets.atlas.prompt instead to avoid code duplication and maintain a single source of truth. Note that the prompt.py file has incorrect naming (SAGE_*) that needs to be fixed first.
    opencompass/configs/datasets/atlas/atlas_val_gen_906a48.py:11
  • Extra trailing whitespace after 'Problem:' on this line. Should be 'Problem:' without the trailing space.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 3 to 4
with read_base():
from .atlas_val_gen_906a48 import atlas_datasets # noqa: F401, F403 No newline at end of file
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import of 'atlas_datasets' is not used.

Suggested change
with read_base():
from .atlas_val_gen_906a48 import atlas_datasets # noqa: F401, F403
with read_base():
Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants