AlphaCodium

My $0.02: it's too hard to build and iterate on complex workflows. Every agent uses a meta-workflow (eg. ReAct is plan->act->observe, with some added steps to check for completion etc.). The teams that have been successful with agents do so by building better but more complex workflows. Most notably, AlphaCodium's "From Prompt Engineering to Flow Engineering" https://github.com/Codium-ai/AlphaCodium Our current tools don't do a great job of making it simple to build and iterate on these workflows. For example, here's a HN post from yesterday where a user created their own workflow management platform because of their frustration with the leading tooling providers: https://news.ycombinator.com/item?id=42299098 I think once we get this tooling right and start to build more expertise in the process of flow engineering, we'll start to faster

Hey, co-creator here, I agree with the sentiment that code coverage may be a proxy and even sometimes a vanity metric but at the same time, IMO unit regression tests are necessary for a maintainable production codebase. I personally don’t feel confident making changes to production code that isn’t tested. Specifically for generating unit regression tests the Cover-Agent tool already works quite well in the wild for some projects, especially isolated projects (as opposed to complex enterprise-level code). You can see in the few (somewhat cherry-picked) examples we posted [0] that it generates working tests that increase coverage (they were cherry-picked in the sense that these are examples we like to work with often internally at CodiumAI). I believe that it’s possible to generate additional meaningful tests including end-to-end tests by c

AlphaCodium is the newest kid on the block that's SoTA pass@5 on coding tasks (authors claim at least 2x better than GPT4): https://github.com/Codium-ai/AlphaCodium As for small models, Microsoft has been making noise with the unreleased WaveCoder-Ultra-6.7b (https://arxiv.org/abs/2312.14187).

Paper: https://arxiv.org/abs/2401.08500 Blog: https://codium.ai/blog/alphacodium-state-of-the-art-code-gen... Code: https://github.com/Codium-ai/AlphaCodium