airbyte

That's sort of true, although in reality Airbyte was only truly "open source" for a very small period[0]. In reality, since about 1 year into the project, it's operated with a mix of open and "less open" licenses for different parts of the codebase, in a way that would make it difficult to just use the MIT licensed bit. I think that kinda proves the point you were going for. [0] https://github.com/airbytehq/airbyte/commits/master/LICENSE

Just an advisory so folks don't get caught out, as far as I can tell the offering advertised as the "open-source version" defaults to [1] (and heavily makes use of) the Elastic license. More context here [2]. [1]: https://github.com/airbytehq/airbyte/blob/v1.0.0/LICENSE [2]: https://github.com/ssddanbrown/Open-Source-Confusion-Cases/b...

We've been working on this for a long time - Reliability, new connectors, error handling and more! Check out the technical release notes to learn more: https://github.com/airbytehq/airbyte/releases/tag/v1.0.0

Four years to the date since 0.1.0! Nice work! (https://github.com/airbytehq/airbyte/releases/tag/v0.1.0-alp...)

Checkout Airbyte and singer.io for data movement. They're tailored for moving offsite data local, but also have support for moving data between different database vendors. I'd recommend starting with Airbyte. For transformations I'm a sucker for dbt-core, which executes transformations as SQL against your data warehouse vs. a separate process. This keeps everything simple and (generally) snappy. It uses the Python Jinja2 template library to compose SQL templates, and the process that parses templates to SQL also implicitly creates an execution DAG so you could have hundreds of transformations running in the exact order they need to. There are also many commercial offerings (Fivetran, Dataform) that don't tick self-hosted, but could be easier to deploy if getting internal hardware support is an issue. https://github.com/airbytehq/airbyte h

When it says "available in open source", is that under the main airbyte repo's licensing [1], hence primarily licensed under the Elastic License v2 and therefore not typically considered open source by many? Airbyte has previous of advertising their offering as open source while not really being as per the OSD[2]. This has been raised with them previously but without response [3][4]. They've also been extending their use of ELv2, recently moving many of their existing MIT licensed connectors to be ELv2 [5]. I don't personally take issue with the choice of license here, I respect the right to protect your work and choose a license that works for you, I'm just against the misuse of open source wording. [1] https://github.com/airbytehq/airbyte/blob/master/LICENSE [2] https://opensource.org/osd/ [3] https://github.com/airbytehq/airbyte/issues

Have you check Airbyte's license[0]? Much of it is Elastic License 2.0, which I don't think allows what you're doing. 0 - https://github.com/airbytehq/airbyte/blob/master/LICENSE

(i work at airbyte) this was announced last month at our conf to a very positive response (https://movedata.airbyte.com/, keynote) The context is that Airbyte is now (after pivoting 3x during YC https://airbyte.com/blog/how-we-pivoted-3-times-in-the-1st-m...) the largest/fastest growing open source community (see our github https://github.com/airbytehq/airbyte) of data pipeline connectors[0], so in a sense they have always been free if you are self hosting. But now using them on Airbyte Cloud is going to be free as well aka "we will do your ELT for free no matter the volume as long as our connectors are not GA yet". This is a massive commitment to improve the quality of our connectors, which is also something we have been pushing the industry on: https://airbyte.com/blog/connector-release-stages : Alpha: new, basic docs, works, passes acc

Oh, declarative doesn't necessarily mean no-code. Airbyte data integration connectors are built with an SDK in Python, Java, and a low-code SDK that was just released... You can then build custom connectors on top of these and many users actually need to modify an existing connector, but would rather start from a template than from scratch. Airbyte also provides a CLI and YAML configuration language that you can use to declare sources, destinations and connections without the UI: https://github.com/airbytehq/airbyte/blob/master/octavia-cli... I agree with you that code is here to stay and power users need to see the code and modify it. That's why Airbyte code is open-source.

we have a big tagging problem where i work and yesterday I tried using gpt3 to assist. worked well! code and context: https://github.com/airbytehq/airbyte/issues/17893

congrats on launching Kyle and Jacob (hi Kyle!) wondering what kinds of projects are/are not suitable for this. the only context i have is from working at open source devtool companies that provide docker builds for people to pull down. might speed up the release process slightly. i suspect my company https://github.com/airbytehq/airbyte/ could benefit. but is it also useful for internal usage?

Over at Airbyte, we had a project this quarter to update all of our build & publish processes over to building multi-arch (AMD and ARM) docker images. As Airbyte runs entirely within docker, getting a smooth local experience for folks on M1/(2?) Macs was important. We had a long lived support thread (1) where you can see us grow through all the phases - from "nothing works", to "our deps don't work", to "the platform works" and finally to "the connectors work"! Assuming your base images are themselves already multi-arch, most of the tooling we needed was already built into the `dockerx` build tool, which is awesome - check it out if you haven't (2). Docker has bundled all the tooling and emulation packages (qemu) needed into a single docker image that can publish multi-arch docker images for you! You run docker to emulate docker to publ