Skip to content

Built-in replicate

When to pick this recipe: source is a service the plugin already speaks (OData V2 / V4, REST, or a CQN-native service), target is the local DB or a remote OData service, and you want row-preserving copy — one source row produces one target row, possibly filtered, projected, renamed. No custom code.

This is the "replicate" use-case label — Concepts → Inference rules gives the formal rules. The pipeline is entity-shape because source.entity (or rest.path) is set and source.query is absent.

To the local DB

Consumption-view target

The idiomatic CAP pattern is to declare the target with a consumption view — a projection on <remote.Entity> annotated with @cds.persistence.table. The projection defines the target schema, column restriction, renames, and optional static where. At addPipeline time, viewMapping is inferred from that projection when you omit it (same extraction idea as cds-data-federation). See Concepts → Consumption views and the capire CAP-level Data Federation guide.

cds
using { S4 } from '../srv/external/API_BUSINESS_PARTNER';

@cds.persistence.table
entity Customers as projection on S4.A_BusinessPartner {
    BusinessPartner as ID,
    PersonFullName  as Name,
    LastChangeDate  as modifiedAt,
} where BusinessPartnerCategory = '1';

Pipeline registration

javascript
const cds = require('@sap/cds');

module.exports = async () => {
    const pipelines = await cds.connect.to('data-pipeline');

    await pipelines.addPipeline({
        name: 'Customers',
        source: { service: 'API_BUSINESS_PARTNER', entity: 'A_BusinessPartner' },
        target: { entity: 'db.Customers' },
        // viewMapping omitted — inferred from the consumption view on db.Customers
        delta: { field: 'LastChangeDate', mode: 'timestamp' },
        schedule: 600000, // every 10 minutes
    });
};

delta.field is the remote watermark field. To duplicate the mapping in code (optional):

javascript
await pipelines.addPipeline({
    name: 'Customers',
    source: { service: 'API_BUSINESS_PARTNER', entity: 'A_BusinessPartner' },
    target: { entity: 'db.Customers' },
    viewMapping: {
        isWildcard: false,
        projectedColumns: ['BusinessPartner', 'PersonFullName', 'LastChangeDate'],
        remoteToLocal: {
            BusinessPartner: 'ID',
            PersonFullName:  'Name',
            LastChangeDate:  'modifiedAt',
        },
    },
    delta: { field: 'LastChangeDate', mode: 'timestamp' },
    schedule: 600000,
});

The source adapter is selected from the connected source service's kind (or an explicit source.kind):

Source transportAdapterNotes
kind: 'odata' | 'odata-v2' | 'hcql'RemoteCqnAdapterAll three delta modes supported. See Remote CQN adapter.
kind: 'rest'RestAdapterCursor / offset / page pagination, timestamp delta via URL params. See REST adapter.
kind: 'cqn' | 'postgres' | 'hana' | 'sqlite' | …CqnAdapterIn-process CAP services and CQN-native DB bindings. See CQN adapter.

Target adapter selection: with target.service unset (or 'db'), the default DbTargetAdapter is used. No target.adapter required.

To a remote OData target

If the destination is an OData service — you are "moving" data from one CAP-visible system to another — no custom adapter is needed either. Point target.service at a CAP service registered with kind: 'odata' (or 'odata-v2') and ODataTargetAdapter is used automatically.

javascript
await pipelines.addPipeline({
    name: 'CustomersToCrm',
    source: { service: 'OrdersOData', entity: 'Orders' },
    target: {
        service: 'CrmOData',
        entity: 'Customers',
        // kind: 'odata',       // optional; auto-detected from the connected service
        // batchSize: 500,      // optional; page size for key scans + write chunks
        // keyColumns: ['ID'],  // optional; defaults to CDS model keys
    },
    mode: 'delta',
});

The OData target adapter routes writes through CAP's remote runtime (POST / PUT / PATCH / DELETE, with $batch change sets where supported) and supports mode: 'delta', mode: 'full', and source.query snapshots. OData has no bulk DELETE, so truncate and deleteSlice are O(n) round-trips on large targets — prefer mode: 'delta' for high-volume pipelines. See Targets → OData for the full tuning table and known limitations.

What happens at runtime

  1. Schedule fires (or a manual run is dispatched via the management service).
  2. PIPELINE.READ runs; the source adapter returns an async iterable of source batches, filtered by the delta watermark when the mode is delta.
  3. For each batch: PIPELINE.MAP applies view-mapping renames (via viewMapping.remoteToLocal) or any user MAP hooks.
  4. PIPELINE.WRITE upserts the mapped rows into the target entity through the resolved target adapter. UPSERT is idempotent across re-runs.
  5. The tracker row is updated with new lastSync / lastKey values.

Constraints

  • source.entity (or rest.path for REST sources) is required.
  • source.query is not allowed — that signal switches into query-shape (materialize) mode. See Built-in materialize.
  • mode defaults to 'full'; pass mode: 'delta' with a delta block for incremental sync.

See also

Released under the MIT License.