windmill

Our open-source project [1] was built with nothing but Rust + Postgres on the backend side of things and it allows us to iterate quickly and have very simple deployments for self-hosting. The queue which is the core bottleneck of our stack will eventually get replaced by redis but for a scale of less than 10k/s jobs, it scale incredibly well. [1]: https://github.com/windmill-labs/windmill

I am working on something adjacent to this problem. We focus much less on data pipelines but on automation, but in the end also have an abstraction for flows that one can use to build data pipeline. The locking-in issue was something I thought a lot about and ended up deciding that our generic steps should just be plain code in typescript/python/go/bash, the only requirement is that those snippets code have a main function and return a result. We built the https://hub.windmill.dev where users can share their scripts directly and we have a team of moderators to approve the one to integrate directly into the main product. The goal with those snippets is that they are generic enough to be reusable outside of Windmill and they might be able to work straight out of the box for orchest for the python ones. nb: author of https://github.com/windm

You missed an "l" with 1. https://github.com/windmill-labs/windmill/ Even for the simplest and my own sites, I usually typed it out in the browser and copy-paste. :-)

Correct URL to windmill on GitHub https://github.com/windmill-labs/windmill (Or @rubenfiszel could edit their post)

I'm an ex-palantir (left 3y ago, things might have changed) that worked on Foundry and building Windmill [1] which is an open-source framework that would actually be closer to Foundry except we do workflows and not data pipelines. I do not think the comparison stands between Retool, or this tool and Foundry. There is indeed a sub-product in Foundry called 'Slate' which is an UI builder but it's a small part of Foundry. Foundry is mostly about data pipelines, to do spark transforms on large ETL, and then having lots of product on top of it to make it easy to make Spark work in an enterprise environment such as a UI builder (the slate mentioned above), a graph viewer of the ETL (monocle), a report builder, RBAC, a timeseries processor, data lineage, versioning of the code, a webeditor and so on. [1]: https://github.com/windmill-labs/windmil

Congrats on the launch! The app builder looks amazing and seems indeed very inspired by Retool. We are building a tool in the same space with Windmill, also open-source [1]. The goal is not to hijack this thread since I do not believe we are that competitive. We focus on workflows and more backendy/complex logic by converting automatically Python, Typescript, Go, Bash into endpoints and workflow modules that you can run at scale on workers that are deployable on one's own infra. We are currently also finishing an UI builder [2] but because it is not our prime focus, it will always be a much simpler alternative to what you and Retool are building. Hence, I see a lot of potential for integrations/collaboration for users with need of UIs similar to retool while needing more complex logic than REST or raw database queries. In any case, congra

We bet big on deno with Windmill (OSS platform to turn scripts into internal UI and workflows) [1][2] and are extremely pleased. The language design choices and the performances/caching strategies are a perfect match for building a distributed workflow runtime at scale. However, the compatibility with npm was the most requested feature. Before the compatibility mode with npm was released, we were about to build a completely separate mode of execution based on node. This solves it very elegantly. Also super excited about the lsp improvements. I believe we are the only project to have a webeditor that features lsp integration with deno in the web (to test, go on [3], New Script -> Next, it works really well). It's very pleasant to work with and make the webeditor experience match closely a local dev setup. We are super excited about the imp

Congrats on the series B and very interesting approach with the Views. Developer tools, especially ones around code should be open-source. If you like the idea of productionizing one-off scripts, building flows, and managing secrets and schedules in an integrated platform, we have made the same observation as the Airplane team but in a fully open-source manner: [1]. Being open-source in this space offers a lot of synergies, you can for instance build a community around sharing the modules that compose your flows: [2] In addition, if your primary languages are Typescript, Python or Go, we offer a more integrated experience by parsing the script parameters directly to generate the input spec and the frontend. We are developing our own UI builder as well, more to be announced in the next month. We just released our helm-charts for easier se

This is great, thank you for open-sourcing. One limitation is see with this model which is clearly stated in the post is that it is javascript/wasm and HTTP servers only. The deployable workers model is a great one and I had hoped to provide an adapter for the open-source windmill[1] project by transforming the normal scripts that are our units of execution into http servers runnable in workderd. We support typescript (deno), go and python. Short of those languages becoming 100% transpilable to wasm, we will have to fallback to use firecracker or keep our current system implemented from scratch using nsjail for sandboxing and a fork for each process. [1]: https://github.com/windmill-labs/windmill

At windmill (https://github.com/windmill-labs/windmill), our whole frontend is in svelte, we love it, the performance are second to none and the dev experience is great. But finding svelte dev is proving itself harder than I thought. So there is that issue as well.

Windmill (YC S22) https://windmill.dev | Founding SWE/DevOps | Rust/Svelte, K8S | Full-Time | Remote/Paris | $90-130k + $100k stock-options Windmill is the sweet-spot between Retool and Temporal, it is fully open-source (https://github.com/windmill-labs/windmill) and has a hub (https://hub.windmill.dev) for sharing scripts and flows. Apply here: https://www.ycombinator.com/companies/windmill/jobs

Self-host: https://github.com/windmill-labs/windmill#how-to-self-host Custom domain: not yet but for sure we will add it as an option for the teams plan

Yes. If you want to build from source, replace `image: ...` by `context: .` in https://github.com/windmill-labs/windmill/blob/main/docker-c... and docker-compose will rebuild from source. If you just want to compile backend and frontend separately: `cd backend && cargo build` `cd frontend && npm install && run build`

They are tightly coupled at the moment but easy to extract out. I did not think there would be interest for that but I could carve it out as a separate dependency project for everyone to reuse! Here is the magic part to implement the queue in sql: https://github.com/windmill-labs/windmill/blob/main/backend/...

Thank you for giving me the opportunity to clarify this! https://github.com/windmill-labs/windmill#how-to-self-host You just need to docker-compose up https://github.com/windmill-labs/windmill/blob/main/docker-c... and you are all set :)