Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: open-compass/opencompass
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: open-compass/opencompass
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: revert-1216-patch-7
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.
  • 1 commit
  • 2 files changed
  • 1 contributor

Commits on Jul 26, 2024

  1. Revert "Add en and zh groups to longbench summarizer; Fix longben… 

    …ch overa…"

This reverts commit 9b9855a.
    @bittersweet1999
    bittersweet1999 authored Jul 26, 2024
    Loading
    Loading status checks…
    Copy the full SHA
    638d889 View commit details
Showing with 2 additions and 15 deletions.
  1. +0 −1 configs/summarizers/example.py
  2. +2 −14 configs/summarizers/groups/longbench.py
1 change: 0 additions & 1 deletion configs/summarizers/example.py
View file
Original file line number Diff line number Diff line change
@@ -12,7 +12,6 @@
from .groups.xiezhi import xiezhi_summary_groups
from .groups.scibench import scibench_summary_groups
from .groups.mgsm import mgsm_summary_groups
from .groups.longbench import longbench_summary_groups

summarizer = dict(
summary_groups=sum([v for k, v in locals().items() if k.endswith('_summary_groups')], []),
16 changes: 2 additions & 14 deletions configs/summarizers/groups/longbench.py
View file
Original file line number Diff line number Diff line change
@@ -5,18 +5,6 @@
{'name': 'longbench_few-shot-learning', 'subsets': ['LongBench_trec', 'LongBench_triviaqa', 'LongBench_samsum', 'LongBench_lsht']},
{'name': 'longbench_synthetic-tasks', 'subsets': ['LongBench_passage_count', 'LongBench_passage_retrieval_en', 'LongBench_passage_retrieval_zh']},
{'name': 'longbench_code-completion', 'subsets': ['LongBench_lcc', 'LongBench_repobench-p']},

# code tasks are included in both longbench_zh and longbench_en
{'name': 'longbench_zh', 'subsets': ['LongBench_multifieldqa_zh', 'LongBench_dureader', 'LongBench_vcsum',
'LongBench_lsht', 'LongBench_passage_retrieval_zh',
'LongBench_lcc', 'LongBench_repobench-p']},
{'name': 'longbench_en', 'subsets': [
'LongBench_narrativeqa', 'LongBench_qasper', 'LongBench_multifieldqa_en',
'LongBench_hotpotqa', 'LongBench_2wikimqa', 'LongBench_musique',
'LongBench_gov_report', 'LongBench_qmsum', 'LongBench_multi_news',
'LongBench_trec', 'LongBench_triviaqa', 'LongBench_samsum',
'LongBench_passage_count', 'LongBench_passage_retrieval_en',
'LongBench_lcc', 'LongBench_repobench-p'
]},
{'name': 'longbench', 'subsets': ['longbench_single-document-qa', 'longbench_multi-document-qa', 'longbench_summarization', 'longbench_few-shot-learning', 'longbench_synthetic-tasks', 'longbench_code-completion']},
{'name': 'longbench_code-completion', 'subsets': ['LongBench_lcc', 'LongBench_repobench-p']},
{'name': 'longbench', 'subsets': ['longbench_single-document-qa', 'longbench_multi-document-qa', 'longbench_summarization', 'longbench_few-shot-learning', 'longbench_synthetic-tasks', 'longbench_code-completion', 'longbench_code-completion']},
]