HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
suzgunmirac
BIG-Bench-Hard
OPEN SOURCE2023–2023
suzgunmirac / BIG-Bench-Hard

BIG-Bench-Hard

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

3Distinct people
3Mentions
0Explicit recommenders
577GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month3 dated mentions

Observed mentions: 2023-03-22 06:01:06 UTC to 2023-11-23 07:04:50 UTC.

012023-03-01 00:00:00 UTC to 2023-04-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMar 20232023-04-01 00:00:00 UTC to 2023-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-01 00:00:00 UTC to 2023-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-06-01 00:00:00 UTC to 2023-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-01 00:00:00 UTC to 2023-08-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJul 20232023-08-01 00:00:00 UTC to 2023-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-09-01 00:00:00 UTC to 2023-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-10-01 00:00:00 UTC to 2023-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-11-01 00:00:00 UTC to 2023-12-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsNov 2023

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Mar 20232023-03-01 00:00:00 UTC2023-04-01 00:00:00 UTC101
Apr 20232023-04-01 00:00:00 UTC2023-05-01 00:00:00 UTC000
May 20232023-05-01 00:00:00 UTC2023-06-01 00:00:00 UTC000
Jun 20232023-06-01 00:00:00 UTC2023-07-01 00:00:00 UTC000
Jul 20232023-07-01 00:00:00 UTC2023-08-01 00:00:00 UTC101
Aug 20232023-08-01 00:00:00 UTC2023-09-01 00:00:00 UTC000
Sep 20232023-09-01 00:00:00 UTC2023-10-01 00:00:00 UTC000
Oct 20232023-10-01 00:00:00 UTC2023-11-01 00:00:00 UTC000
Nov 20232023-11-01 00:00:00 UTC2023-12-01 00:00:00 UTC101

What Hacker News says

3 sources
It depends on the eval, but I think it's fair to say that it's close. Here is the AGI Eval results organized into a table w/ averages (also I put in the new Hermes LLama2 13B model as well: https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYp... It beats out ChatGPT in every category except SAT-Math. We definitely need harder benchmarks. So far, there's BIG-Bench Hard https://github.com/suzgunmirac/BIG-Bench-Hard and just published, Advanced Reasoning Benchmark https://arb.duckai.org/
In LLaMA2 Chat 70B outperformed ChatGPT

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.