marcus-claims-dataset

I'm sorry, but we can tell absolutely nothing about Gary Marcus from this. People should have a look at the final data: https://github.com/davegoldblatt/marcus-claims-dataset/blob/... Many of the "supported" claims here are vague, banal, obvious, or just opinion. E.g. "the general public hasn't quite realized what's not possible yet" "loads of things scale, but not at all" "To be sentient is to be aware of yourself in the world; LaMDA simply isn't." "To date, nobody, ever, has given a convincing and thorough account of how human children (and human children alone) learn language." "A cat holding a remote control shouldn't have a human hand." "What I didn't see last night was vision" (about Tesla Optimus)

Gotcha', but I'm just trying to see the audit for how claim X was rated and based on what sources. If we're looking at the Claude logs, we have huge files that have things like this[1]: {"id": "claim_0081", "date": "2023-02-11", "claim": "Current Level 2 self-driving operates under easy conditions and is nowhere close to handling real-world complexity.", "type": "descriptive", "target": "Level 2 self-driving", "status": "supported", "horizon": null} Why is this supported? How is this supported? Waymo would probably disagree, etc. Here's another one: {"id": "claim_0083", "date": "2023-02-11", "claim": "Tesla's product naming ('Autopilot', 'Full Self Driving') misleads customers into thinking the cars are more capable than they are, potentially causing accidents and deaths.", "type": "causal", "target": "Tesla marketing", "status":

I'm still not sure I fully understand the methodology. For example if Marcus makes the claim: "OpenAI sucks!" why would OpenAI's blog ever corroborate that? The sources used are all AI company blogs (Anthropic, Google, OpenAI) filled with inoffensive corpo-speak likely written to be as middle-of-the-ground as possible. In fact, I'd need an A/B test to make sure the LLM itself can properly rate various claims (positive, negative, and neutral) against such corporate sludge. Small aside: I'm only bringing this up because last year I worked on a game where you had to solve various moral dilemmas in a 1v1 situation (think trolley experiment and one player says "flip the switch" and the other says "don't flip the switch")—the idea was to get an LLM to rate the arguments in a fun turn-based online game. I built it out, but I kind of gave up when