May 25, 2022

Airbyte, an open source data integration platform for building ELT data pipelines, today announced the general availability of its cloud service. The company also details its new reward plan for open source developers who write and maintain data connectors for the platform, and plans to launch its new CLI later this week.

Airbyte promises that this will allow companies to easily create flexible ELT (Extract, Load, and Transform) pipelines using the company’s off-the-shelf connectors or building their own. The fast-growing company, which recently raised a $150 million Series B round of funding, first launched a limited beta version of its cloud service last October and has since improved many aspects of the user experience, including the onboarding process. supposed. To learn more about how the needs of cloud users differ from those of open source users.

image credit: air bite

“I don’t think you can just move open source to the cloud. There are some adjustments that need to be made to the product,” explains Michel Trico, co-founder and CEO of Airbyte. He said open source users tend to be more technical and willing to get their hands dirty to make things work. No wonder cloud users want everything to work.

Earlier this year, the company announced that deployments increased 6x in 2021, from 400 per month in the first quarter to 2,500 by the end of the year. It currently offers over 165 data connectors and is expected to expand to 500 by the end of the year.

Compensating open source contributors is a hot issue right now. Airbyte picks an interesting model here, choosing to compensate developers who write, maintain, and repair connectors for their platform.

image credit: air bite

“We’ve always had members and the number has grown every week,” said Airbyte co-founder and COO John LaFleur. “One way to make sure that some part of the community is focused on what has the most impact on the user community is to encourage them. We’ve been beta testing for the last few weeks and it’s all about giving some kind of reward, some kind of reward for solving some of the issues in the priority. This means moving from alpha to beta and from beta to production, as well as taking the necessary steps to achieve public certification or fix bugs.”

In practice, this means that a developer can earn up to $300, for example, by adding API flows to the connector, and additional incremental updates can earn up to $200.

Regarding the new command line access, Tricott noted that AirBytes has a lot of users. As more data engineers want to have programmatic access to AirByte rather than data analysts who want a UI based service. Airbyte has always provided programmatic access, but Trico noted that the community was looking for a way to programmatically access the service that would include all GitOps best practices for service access. This new command line allows users to store their configuration and connection status in Git and include them in their existing CI pipelines.

Looking ahead, the Airbyte team noted that they are working to ensure that cloud users can manage both data and control layers. “This is forever part of our common northern star – omnipresence and reliability. Its omnipresence means we need to access data from anywhere, be it in our cloud, be it AWS, Azure, GCP, or across regions,” says Tricolor.

He also said that now that Airbyte has focused on centralizing data in the data warehouse, the next leg of the journey will have more focus on moving that data to the API (essentially the other side of the coin compared to data pipelines). The first step for many datasets is building the foundation that Airbyte is currently focused on, but the natural next step is to discover new uses for that data.

“From air bite With the cloud, we remove the headache of building and maintaining a custom data infrastructure by providing businesses with an easy, cost-effective way to move data,” Tricott said. “It is just as important that we can guarantee that air bite As data connectors become more available, they serve quality goals, in part by compensating administrators for a long tail of otherwise orphaned data connectors. We also think it’s important, given the Linux Foundation’s recent findings, that contributors not only be recognized, but also rewarded. ,

Leave a Reply

Your email address will not be published.