aloha

The UMI gripper project is working on this. they have a handheld gripper device full of sensors that they use to record doing things in the field, like picking Starbucks, which they then use as training data. https://umi-gripper.github.io/ The other thing to note is part of the aloha project isn't just to record people folding laundry and loading the dishwasher, but to take that data and plug it into a simulator with a physics engine, and use a digital twin to get 10x the amount of data to be used in training the model than if they'd just used real world data. So yes we need that data, but not as much as we would otherwise. https://mobile-aloha.github.io/ https://github.com/tonyzhaozh/aloha

It's already open-source; most of it anyway: 1. https://github.com/tonyzhaozh/aloha 2. https://aloha-2.github.io/ 3. https://github.com/tonyzhaozh/aloha/tree/main/aloha2