circuit_training

The article you linked is not an example of this happening. Google open-sourced the chip design method, and uses it in production for TPU and other chips. https://github.com/google-research/circuit_training https://deepmind.google/discover/blog/how-alphachip-transfor...

It is open: https://github.com/google-research/circuit_training

> "scientific publication" These air quotes suggests the commenter above doesn't think the paper qualifies a scientific publication. Such a characterization is unfair. When I read the Nature article titled "Addendum: A graph placement methodology for fast chip design" [1], I see writing that more than meets the bar for a scientific publication. For example: > Since publication, we have open-sourced a software repository [21] to fully reproduce the methods described in our paper. External researchers can use this repository to pre-train on a variety of chip blocks and then apply the pre-trained model to new blocks, as was done in our original paper. As part of this addendum, we are also releasing a model checkpoint pre-trained on 20 TPU blocks [22]. For best results, however, we continue to recommend that developers pre-train on their own

In the tweet Jeff Dean says that Cheng at al. failed to follow the steps required to replicate the work of the Google researchers. Specifically: > In particular the authors did no pre-training (despite pre-training being mentioned 37 times in our Nature article), robbing our learning-based method of its ability to learn from other chip designs But in the Circuit Training Google repo[1] they specifically say: > Our results training from scratch are comparable or better than the reported results in the paper (on page 22) which used fine-tuning from a pre-trained model. I may be misunderstanding something here, but which one is it? Did they mess up when they did not pre-train or they followed the "steps" described in the original repo and tried to get a fair reproduction? Also, the UCSD group had to reverse-engineer several steps to reproduc

As someone who has no skin in the game and is only loosely following this, there is a tool: https://github.com/google-research/circuit_training, the detractors claim to not be able to reproduce Google's results (what Dean is commenting on) with it, Google and 1-2 other companies claim to be using it internally to success (e.g. see the end of this article: https://deepmind.google/discover/blog/how-alphachip-transfor...).

One is basically an evidence-free ad hominem attack. > [2] https://twitter.com/JacobSteinhardt/status/15215993404137881... The other two sources make a concrete claim that in mid-2002 there was an independent, open-source, replication of the Nature paper: > [1] https://twitter.com/ZoubinGhahrama1/status/15122035096467415... >> Google stands by this work published in Nature on ML for Chip Design, which has been independently replicated, open-sourced, and used in production at Google. > [3] https://twitter.com/sguada/status/1521587406385807361 >> The results in the Nature paper were independently replicated and validated by my team, the results were used in actual chips and Sat and his collaborators know it. >> Furthermore, the code was open-sourced. >> It is sad that you are providing a platform for someone's resentments. The claims about

Igor Markov, along with Sat Chatterjee, seem to be pursuing a bizarre vendetta (after Sat failed to take over their project) against the lead authors of the chip placement work, not some sort of intellectually honest critique. This was covered previously in the press and on social media, with statements from a variety of prominent researchers (e.g. [1][2][3]). The code is even available for the Nature paper's method, along with an FAQ: https://github.com/google-research/circuit_training#FAQ [1] https://twitter.com/ZoubinGhahrama1/status/15122035096467415... [2] https://twitter.com/JacobSteinhardt/status/15215993404137881... [3] https://twitter.com/sguada/status/1521587406385807361

The TF-Agents team replicated the RL training (with the corresponding teams' very deep collaboration) and open-sourced it here: https://github.com/google-research/circuit_training It pretty much gets the same results as found in the Nature paper. The original codebase was heavily research-focused, used TF1, was impossible to run distributed training outside of Google's infra, and made it hard to try algorithms other than PPO. So it was reimplemented on top of TF2 and using some distributed training and collection technologies developed by the TF-Agents team at Google Brain and infra teams at DeepMind. Everyone is welcome to poke at the training code and the model, and convince themselves that it does what it says on the box :)