vimGPT

What I don't like about LLMs is that people keep re-inventing the wheel over and over. For example, we've been able to control browsers using GPT for about 2 years now: - https://github.com/mayt/BrowserGPT - https://github.com/TaxyAI/browser-extension - https://github.com/browser-use/browser-use - https://github.com/Skyvern-AI/skyvern - https://github.com/m1guelpf/browser-agent - https://github.com/richardyc/Chrome-GPT - https://github.com/handrew/browserpilot - https://github.com/ishan0102/vimGPT - https://github.com/Jiayi-Pan/GPT-V-on-Web

How does the performance compare to VimGPT[0]? I assume the screenshot-based approach is similar, whereas the text approach should be improved? Very cool either way! [0] https://github.com/ishan0102/vimGPT

Scrapers are one of the main use cases we're seeing for Magic Loops[0]. ...and you've hit the nail on the head in terms of our design philosophy: use LLMs to generate useful logic, then run that logic without needing to call an LLM/Agent. With that said, we don't support browser automation. Skyvern is very neat, it reminds me of VimGPT[1], but with a more robust planning implementation. [0] https://magicloops.dev [1] https://github.com/ishan0102/vimGPT

It's insane that this is now possible: https://github.com/ishan0102/vimGPT/blob/682b5e539541cd6d710... > "You need to choose which action to take to help a user do this task: {objective}. Your options are navigate, type, click, and done. Navigate should take you to the specified URL. Type and click take strings where if you want to click on an object, return the string with the yellow character sequence you want to click on, and to type just a string with the message you want to type. For clicks, please only respond with the 1-2 letter sequence in the yellow box, and if there are multiple valid options choose the one you think a user would select. For typing, please return a click to click on the box along with a type with the message to write. When the page seems satisfactory, return done as a key with no value. You must respond in JSON