In this special edition of Data Planet, Onebridge data engineer and Fivetran expert Bradley Nielsen provides his unique perspective on this SaaS data integration product.
The Real Scoop on Fivetran
Since Onebridge recently became a Fivetran partner, we thought we’d ask Data Engineer Bradley Nielsen to give you an assessment of the service. Nielsen has deep experience with Fivetran and helping our clients use the product.
Note that being a partner means Fivetran has vetted Onebridge as a company with the expertise to help organizations use their service effectively and efficiently. Nielsen’s thoughts are below.
What is Fivetran?
Fivetran develops a Software as a Service (SaaS) data integration product. Simply put, Fivetran copies data from source systems (ERPs, CRMs, SaaS Apps, Files, etc.) and puts the data into a data warehouse (BigQuery, Snowflake, etc.). Fivetran’s mission statement is to “make access to data as simple and reliable as electricity.”
Strengths
Gartner’s 2022 reviews of data integration tools give Fivetran a rating of 4.4 out of 5. Simplicity and ease of use are common themes, so Fivetran does deliver on those fronts.
Fivetran’s value proposition is that they save you money by:
- Developing connectors for you. To pull data from a source system often requires some form of custom code. This code is referred to as a connector. Connector complexity can range from trivial to highly advanced, depending on the source. Developing and maintaining these connectors requires time, money, and talent. Fivetran offers 150 prebuilt connectors and is constantly adding new ones.
- Pipeline management. Fivetran takes care of pipeline management tasks such as routing, error recovery, incremental loading, and scheduling. This reduces the workload on your data engineers.
- Infrastructure management. Fivetran is SaaS, so there is no software to install, nor servers to manage.
Fivetran is partnered with dbt labs to provide data transformation capabilities via dbt. Fivetran is also expanding capabilities in the areas of data quality and observability.
How much does it cost?
Fivetran’s costs are consumption-based, depending on the number of rows loaded or updated. They call it Monthly Active Rows. The important things to remember are that Fivetran:
- Doesn’t charge extra for updating a row you’ve already loaded. If you load a row and then update it 10 times (within a month), you are only charged for it only once.
- Offers progressive discounts for higher data volumes.
- Doesn’t charge for the initial load (syncs and resync, to use their language), only the data loaded afterwards.
Considerations
For some organizations with appropriate in-house talent, it may be more cost effective to develop your own pipelines. This is especially true for simple high-volume pipelines like database-to-database.
Estimating the monthly active rows can be tricky, making forecasting costs difficult.
Connecting to on-premises resources requires setting up an SSH tunnel, which is not trivial. Even with 150 connectors, there are still hundreds of sources that Fivetran does not support. Custom connectors are created by wrapping the source in an Azure Function. Doing this takes effort and technical expertise and can negate part of the value proposition.
Conclusion
At Onebridge, we pride ourselves on being technology agnostic. This means we recommend solutions to organizations based on what best meets their needs, not just who our “favorite” is. We are a Fivetran partner because we found this to be a great tool for specific needs.
Depending on the situation, Fivetran is a great way to get up and running with a variety of connected sources of data quickly. This dramatically cuts down the time-to-value and can help move an organization’s data strategy forward by providing quick results.
However, there are potential long-term cost considerations, depending on the amount of data that your organization requires. Based on this, we would recommend Fivetran in situations where a lot of data sources need to be connected quickly, and the total volume of data over a long period of time would make the cost justifiable.
It could also be a good temporary solution to prove results and get buy in for building your own pipeline later down the road.
Further Reading
ETL Platform Alternatives: Fivetran vs. Informatica and More
