skyvern

tps://github.com/Skyvern-AI/skyvern/tree/main/integrations... We built this mostly for fun, but can see this being integrated into AI agents to give them custom access to browsers and execute complex tasks like booking appointments, downloading your electricity statements, looking up freight shipment information, etc

opic.com/en/docs/build-with-claude/compute...) Achieving this SOTA result required expanding Skyvern’s original architecture. Skyvern 1.0 involved a single prompt operating in a loop both making decisions and taking actions on a website. This approach was a good starting point, but scored ~45% on the WebVoyager benchmark because it had insufficient memory of previous actions and could not do complex reasoning. We re-built this all using a Planner-Actor-Validator agent architecture: 1. Planner - Decides that goals to accomplish on a website, and maintains a working memory of the overall goal and progress towards it 2. Actor - Given a narrowly scoped goal, executes the goal on the website, reporting back 3. Validator - Asserts whether the goal was successfully achieved and passes feedback back to the Actor + Planner We ran the benchmark on Skyvern cloud to test Skyvern 2.0 in a real-world environment – autonomously navigating the web in a remotely hosted browser without any human involvement. To keep with our open source mission, we decided to publish benchmark, modifications, and final results for anyone to review. This is important because we’re seeing an increasing trend of companies publishing their benchmarks with no way to access the results, so we’ve decided to make everything public. [1] Eval Dataset: https://github.com/Skyvern-AI/skyvern/tree/main/evaluation/d... [2] Modifications: https://github.com/Skyvern-AI/skyvern/pull/1576/commits/60dc... [3] Each run (incl prompts + responses) can be inspected here: https://eval.skyvern.com/ The full report (incl an architecture diagram) can be found here: https://blog.skyvern.com/skyvern-2-0-state-of-the-art-web-na... If you’d like to give Skyvern a try, you can grab the open source version (https://github.com/Skyvern-AI/Skyvern) or the cloud version (https://app.skyvern.com/) and give it a go and share any feedback with us. We look forward to any and all of your comments!

problem: they’re brittle (wow this website added a new pop-up dialog and my script broke), and fail to achieve the same objective across multiple websites (how can I fill out a contact-us form on hundreds of different websites?) We did a Show HN a few months ago (https://news.ycombinator.com/item?id=39706004), and since then, we’ve onboarded customers for a wide variety of use cases: generating insurance quotes on websites like Geico.com; applying to jobs on websites like lever.co; automating filing permits in local government portals; registering new corporations for employment identification; fetching invoices from hundreds of different portals such as hydroone.com; automating purchasing on a handful of e-commerce websites like zooplus.com; and filling out contact us forms on a bunch of random smb websites (such as HVAC websites). To be able to service all of these, we’ve built and open-sourced quite a few interesting features: (1) a fully-featured React application allowing you to see every action Skyvern is taking in real-time; (2) livestreaming browser instances to allow our users to see what Skyvern is doing when running inside of a docker container; (3) authenticated sessions, integrating with Bitwarden and allowing users to specify Email + Phone + QR-code based 2FAs; (4) “workflows” allowing users to chain multiple goal-based prompts together, which can handle tasks like invoice downloading, or automating purchasing pipelines; (5) processing HTML Elements (ex. identifying + summarizing SVGs) and performing website interactions (ex. Iterating over dynamic autocompletes to fill in address information correctly) (6) “cached workflows”, allowing Skyvern to memorize previous interactions (ie text inputs) and re-use them in future runs. We’ve also been blessed with a few model advancements to solve some of the cost concerns the community brought up. Skyvern’s token costs went down 80% from $15 / 1M tokens (GPT-4V) to $2.50 / 1M tokens (GPT-4O) Despite the model costs going down 80%, Skyvern is still quite expensive to run, so we give every new user $5 of credits to try it out and see if it can be useful for you. We would be honored if you could give it a try at https://app.skyvern.com and share some feedback with us, and we look forward to any and all of your comments!