Which features are required for data exports?

When building or buying data exports, you must go beyond basic data transfer. The product must connect to multiple platforms, enforce security and compliance, deliver a smooth customer experience, and support automation for scale. This guide breaks down the essential capabilities to help you make the right choice.

Data exports aren’t ETLs

A common misconception is that building a data export is similar to setting up an ETL (Extract, Transform, Load) process. But while both involve data movement, they serve different purposes. ETLs move data between one provider and one consumer, often within a single organization’s systems. Data exports, on the other hand, are designed for external sharing—pushing data from one provider to multiple consumers. This requires handling multi-tenancy, permissions, and complex customer support that ETL tools aren't equipped for.

Enterprises are the core customer

Each enterprise has unique infrastructure, security, and compliance needs. Data exports need to be able to support a wide variety of data platforms, connection methods, and transfer types while ensuring security, reliability, and speed.

Capabilities overview

At a high level, data exports need to be able to:

• Connect to your data platform
• Connect to your customers’ data platforms
• Read from your data platform and write to your customers’ data platforms
• Provide data security and comply with regulatory requirements
• Deliver an excellent customer experience from onboarding to support
• Monetize your data export product
• Use tools, including APIs, to automate workflows

The following sections break down each of these capabilities into features or frames the questions businesses should ask themselves to define their own requirements.

‍

Connect to your data platform

Obviously, vendors will need to be able to connect to your platform, and you’ll know best what’s required to make that happen. We recommend checking the vendor’s developer documentation to understand the feasibility and complexity of that connection.

‍

Connect to your customers’ data platforms

For planning purposes, we’ve found that most software companies need to support a minimum of seven data platforms to cover the majority of their customers. However, we recommend finding a single product that will support every major data platform to minimize the cost and complexity of support.

Destinations

The most popular databases, data warehouses, and object storage services include:

Data warehouses

• Snowflake
• Google BigQuery
• Amazon Redshift
• Databricks
• Amazon Athena
• ClickHouse

Databases

• PostgreSQL
• Amazon Aurora
• MariaDB
• MySQL

Object storage

• Google Cloud Storage
• Amazon S3
• Microsoft Azure Blob Storage
• SFTP

Connection modalities

Each enterprise has its own setup and security requirements. We recommend planning to support the following connection modalities:

• Username and password authentication
• Role-based authentication
• IP whitelisting
• SSH tunneling

‍

Read from your data platform and write to your customers’ data platforms

Products will also need to be able to read from the source and write to destinations.

Column & schema-based tenancy

How does your company organize user information? Does it keep all customer information in the same table and use a column, like customer_id, to identify which customer the data belongs to? Or does each customer have its own schema? You’ll need to confirm that your data export product is compatible with your method of choice.

Max throughput per destination

How much data do your customers need to sync? How often? That will tell you how much volume and frequency you need to support.

Version-controlled schemas

Schemas should be version-controlled and managed using a solution like GitHub so that changes over time can be tracked and managed along with other code.

Schema migration

What happens downstream when schemas change? Products should have a way to cascade changes from the source to each destination in a way that doesn’t break downstream pipelines. Products should support the addition and removal of both columns and tables.

Types of transfers

Products should support:

1. Full transfers

Products should push all relevant data in the initial sync. Then, users should be able to trigger a full refresh when needed.

2. Incremental transfers

After the initial sync, products should be able to push new data to each destination.

3. Windowed transfers

When transfers are very large, products should be able to split that transfer into smaller chunks so that customers don’t have to wait for full transfers to complete to start working. 

Eventual consistency

Products should have a method to prevent data loss during the transfer process due to data arriving out of order.

Data integrity checks

Products should include metrics that help both the sender and receiver understand transfer health and verify that data transfers are complete.

‍

Provide data security and comply with regulatory requirements

Your data export product you use should be at least as careful with customer data as the rest of your systems. 

Deployment

Can the product be deployed privately behind your company’s firewall, or do you need to evaluate its system security? 

Data retention

Does the system retain any of the data it transfers? Or does it use an ephemeral server to read from one system and write to the other?

Data residency

Where is the data processed? Does the product help prevent you from sending data to locations where it cannot be sent?

Certifications & testing

Does the product have all of the certifications your business needs for a vendor that handles customer data, such as SOC2, GDPR, CCPA, and HIPAA? 

‍

Deliver an excellent customer experience from onboarding to support

The data export product should offer a customer experience that matches existing experience standards.

Embedding

Where do customers interact with the product? Can it be embedded in your website or app, or do customers need to go somewhere else for help with setup and support?

White-labeling

Does the vendor offer a white-labeled product, or will you need to introduce your customers to a third party?

Magic Links

Can you send your customers a unique link to begin the onboarding process?

Data and Frequency Selection

Customers should be able to choose which data they want to receive and how often they want it synched.

Documentation

Does the product provide all the documentation customers need to set up and use your data export feature? 

Direct support

Will you have direct access to the vendor for customer support? Does the SLA support your existing SLAs with customers?

‍

Use tools, including APIs, to automate workflows

Products should deliver an excellent developer experience, including access to all the tools needed to automate and streamline workflows.

Webhooks & alerts

Products should be capable of sending HTTP callbacks. Ideally, they should also integrate with commonly used platforms like PagerDuty and Slack for easy monitoring.

API

Products should expose a RESTful API to handle most common actions, like configuring sources and destinations, initiating data transfers, monitoring activity, and more.

Developer Documentation

Developers should have access to clear, comprehensive documentation to build and manage data export products confidently.

‍

Monetize your data product

If you’re going to charge for data export, you’ll want all of the features you’ll need to charge customers for service.

Affordable

The product should offer a cost model that will sustain a business. That likely means you’ll need to hit a cost target that will allow you to maintain existing profit margins. 

Client cost model

The product should offer a cost model that allows you to allocate costs to clients.

‍

Building an enterprise-class product

Getting the enterprise experience right takes careful planning. Now that you have a comprehensive list of the features your customers may need, explore our data export market guide to find the best tools for building your product.