mle-bench

Autoresearch is nothing new, big players are already in the game with more sophisticated solutions: - https://arxiv.org/abs/2602.02660 (MARS) - https://arxiv.org/abs/2601.14525 (Execution-grounded automated AI research) - https://arxiv.org/abs/2601.10402 (ML-Master 2.0) The mostly used benchmark for automated AI engineering/ research is: https://github.com/openai/mle-bench

Hi HN, I’m the creator of AIBuildAI. We built this because building AI models still requires a lot of manual work. AIBuildAI is an AI agent that automatically designs AI models, writes code to implement them, trains them, tunes hyperparameters, and iteratively improves the models. It ranked #1 on OpenAI’s MLE-Bench benchmark: https://github.com/openai/mle-bench/pull/126 GitHub: https://github.com/aibuildai/AI-Build-AI Curious to hear what the community thinks, and happy to answer questions.

This is amazing. Porting library from one language to one language are easy for LLMs, LLMs are tired-less and aware of coding syntax very well. What I like in machine learning benchmarks is that agents develop and test many solutions, and this search process is very human-alike. Yesterday, I was looking into MLE-Bench for benchamrking coding Agents on machine learning tasks from Kaggle https://github.com/openai/mle-bench There are many projects that provide agents which performance is simply incredible, they can solve several Kaggle competitions under 24 hours and be on medal place. I think this is already above human level. I was reading ML-Master article and they describe AI4AI where AI is used to create AI systems: https://arxiv.org/abs/2506.16499

Wow..Vaporware? how? Did you even check the leaderboard on openAI's official github? If not, first check that before calling it whatever. Here:https://github.com/openai/mle-bench They are not just distributing trophies to anybody. It was built over 10 months with a small team and some GPUs.