Skip to main content

Managed Syncs

Managed Syncs allows you to configure Supaglue to sync data from your customers' third-party providers into a destination database hosted in either your cloud, or Supaglue's cloud.

It's the fastest way to read data into your application and allows you to offload third-party provider authentication, APIs, rate limits, retries, and more to Supaglue.

integration patterns syncsintegration patterns syncs

How it works

  1. Use the Management Portal to create a sync configuration (SyncConfig) that defines "what" and "how" to sync from a third-party Provider to a Destination.
  2. Your customer connects via our Managed Authentication feature.
  3. Supaglue starts fetching data from your customers’ third-party Providers and landing them in a Destination database (e.g. Postgres), that you can then query from your application.

Core concepts

managed syncs conceptsmanaged syncs concepts


A Provider is a third-party SaaS tool that Supaglue can connect to, to sync data.

Use the Connectors -> Providers page in the Management Portal to configure your provider. For providers who support OAuth-based authentication, you'll be asked to provide your OAuth app credentials.
provider configprovider config


A Destination is a database that Supaglue syncs third-party provider data to. You can choose one in your infrastructure or Supaglue's infrastructure.

Use the Connectors --> Destinations page in the Management Portal to set your destination credentials:
destination configdestination config

Before syncing data, Supaglue generates the destination tables and columns in your database if they do not already exist.

See data models for more detail on destination schemas.


A SyncConfig ties a Provider (source) to a Destination. It defines what objects to sync and how to sync them (frequency, strategy, etc).
sync configsync config

Sync strategies

Supaglue offers two sync strategies that you can configure in your sync configuration:

  • full only: Every sync fetches all records from the Provider.
  • full then incremental: The initial sync performs a full sync, then incremental after using a persistent high watermark.
    • full_sync_every_n_incrementals: Setting this configuration allows you to perform incremental syncs and a full refresh after every n incremental syncs. This strategy is helpful to ensure the eventual consistency of records deleted on the provider side that some incremental syncs do not support.

full then incremental is used where we can, but for specific objects and Providers, we may only be able to offer full only sync strategies. We list these in the Providers documentation.

Query patterns

Direct query

The quickest way to query data is to query the raw JSONB data that Supaglue lands: this is well-suited if you can easily do transforms upon reading the raw data in your application using code. You may optionally add indexes to help query performance.

Logical SQL view

You can create Postgres views if you need to rename fields or run transformations easily expressed using SQL.

Postgres Generated Columns

Building on Pattern 2, if your SQL transformations are expensive to run at query time, you can use Postgres Generated Columns to speed up queries by pushing the transformation work to be maintained at write time.

Transformation pipeline

If you have well-structured tables that you wish to write Supaglue-synced data into, you can paginate over the Supaglue-synced tables, run your transformations, and write the transformed data into your tables.

transformation pipelinetransformation pipeline

Read more about transformation pipelines and patterns in our transformation tutorial section.

Destination options

You can decide where to host the database that Supaglue syncs to:

  1. Self-host your database: Supaglue writes raw data directly into your database.
  2. Use a Supaglue-hosted database + API: Supaglue writes to a Supaglue-hosted database and provides an API to paginate, transform, and write data into your database schema.

Option 1: Your database

Supaglue writes raw data directly into your database.

managed syncsmanaged syncs

Option 2: Supaglue-hosted database + Data Listing API

Supaglue writes raw data into a Supaglue-hosted database and provides a convenient API to paginate over records, which is helpful if you don't have an existing transformation pipeline.

data listing apidata listing api

Choosing between the destination options

There are several differences between the two methods of hosting Destinations:

DimensionYour application databaseSupaglue-hosted database + Data Listing API
TransformationsIdeal if you have existing transformation pipelinesIdeal if you don't have an existing transformation pipeline
Compliance (data residency)Your customer data is never stored in Supaglue at-restYour customer data is stored in Supaglue at-rest
SecurityYour database needs to be accessible from the public internetYour final target database can be behind a VPC
OperationalYou need to ensure your database is upSupaglue will ensure the uptime and sync reliability
Performance/LoadSupaglue determines how fast to sync to your databaseYou want to control how fast to sync to your database