advanced-context-engineering-for-coding-agents

I think this is where we may have to agree to disagree, because this not something that I'd do outside of toy projects. Not necessarily because I'm worried about the correctness of a single MR (in fact, it may very well be more likely to be correct than a human-authored and -reviewed one), but because I'm concerned about the accrual of technical debt. Even if each individual MR is correct, the trajectory for code quality can trend downwards. The recent update to SlopCodeBench that includes Astra and Opus 5.5 (including the insight that higher reasoning levels may not improve the situation) seems to match my intuition. [1] [1] https://github.com/humanlayer/advanced-context-engineering-f...

Quick update here since a lot of folks asked about what slopcodebench is / where it fits in Slop code bench is unique among benchmarks in that it makes a model work against a progressively disclosed list of features (checkpoints) that it doesn’t know it advance, inheriting the same codebase through the lifecycle of a single multi-checkpoint challenge I like this because it much more mirrors how real software dev happens, rather than most benchmarks with one off tasks where slop code doesn’t hinder future progress The original post[1] and the SOL/fable follow up[2] went into a bit more detail on the benchmark and its challenges 1. https://news.ycombinator.com/item?id=49076391 2. https://github.com/humanlayer/advanced-context-engineering-f...

Sol is closer to Fable than Opus - I like SlopCodeBench the most as a test - https://github.com/humanlayer/advanced-context-engineering-f... You add requirements and make previous tests invisible to see how pigeon brained the model is - Sol and Fable seem to rank the same as Opus tends to fall behind

read. https://github.com/humanlayer/advanced-context-engineering-f...

Did you even read further on? The section right after that addresses the step changes in current frontier models https://github.com/humanlayer/advanced-context-engineering-f...

I like reading these types of breakdowns. Really gives you ideas and insight into how others are approaching development with agents. I'm surprised the author hasn't broken down the developer agent persona into smaller subagents. There is a lot of context used when your agent needs to write in a larger breadth of code areas (i.e. database queries, tests, business logic, infrastructure, the general code skeleton). I've also read[1] that having a researcher and then a planner helps with context management in the pre-dev stage as well. I like his use of multiple reviewers, and am similarly surprised that they aren't refined into specialized roles. I'll admit to being a "one prompt to rule them all" developer, and will not let a chat go longer than the first input I give. If mistakes are made, I fix the system prompt or the input prompt and t

I'm kind of curious why language specializations matter much to all but the most hardcore of hardcore performance firms. People have demonstrated now that you can pick up even Rust with an LLM and be productive with it in a few days if you know what you are doing: https://github.com/humanlayer/advanced-context-engineering-f...

This is effectively what I'm doing, inspired by HumanLayer's Advanced Context Engineering guidelines: https://github.com/humanlayer/advanced-context-engineering-f... We've taken those prompts, tweaked them to be more relevant to us and our stack, and have pulled them in as custom commands that can be executed in Claude Code, i.e. `/research_codebase`, `/create_plan`, and `/implement_plan`. It's working exceptionally well for me, it helps that I'm very meticulous about reviewing the output and correcting it during the research and planning phase. Aside from a few use cases with mixed results, it hasn't really taken off throughout our team unfortunately.