What this is
Pub/Sub Batch is a Zeotap CDP source: it brings data into Zeotap CDP from a Google Cloud Pub/Sub (Publish/Subscribe) topic that you own and publish to. It belongs to the Message Queue source category and runs as a batch integration, so messages published to your topic are collected and ingested in batches rather than one at a time. The source accepts both Customer Data (CE) and Non-Customer Entity (NCE) data. If you need messages to reach Zeotap CDP as soon as they are published, use Pub/Sub Stream instead — the real-time source in the same category. To send data the other way, out of Zeotap CDP and into your own Pub/Sub topic from a journey, see the Google Pub/Sub destination.Prerequisites
Arrange the storage access before you start. The Pub/Sub service account does not have the permissions it needs on the storage bucket by default, so batch subscriptions cannot push files to cloud storage until those permissions are granted. Contact your Zeotap POC before you set up the Pub/Sub Batch source.
How the batch source works
- You publish messages to a Pub/Sub topic in your own Google Cloud project. Each message must be minified (compact) JSON.
- When you create the source in Zeotap CDP, you supply the fully qualified Topic ID of that topic, and Zeotap CDP displays the service account it will use.
- You grant that service account the permission to attach a subscription to your topic (
pubsub.topics.attachSubscription) in your Pub/Sub instance. - Zeotap CDP attaches a subscription to the topic. A batch subscription pushes the collected messages to cloud storage as files, which Zeotap CDP then ingests.
Use a separate topic for batch and stream
Use a different Topic ID for a Pub/Sub Batch source and a Pub/Sub Stream source. If both sources point at the same Topic ID, the same data arrives in both.Next steps
- Create a Pub/Sub Batch source — prerequisites, Topic ID, service account, data format, and the step-by-step setup.
- Pub/Sub Stream — the real-time source for the same Pub/Sub topic model.
- Message Queue — every Message Queue source integration Zeotap CDP supports.