Skip to main content

Audience Engine Architecture

The DriveMetaData (DMD) Audience Platform separates definition ownership and persistence from live query evaluation and analytical execution. This decoupled architecture enables no-code, interactive audience segmentation across millions of customer records with sub-second estimation and preview times.

1. Dual-Engine Architecture​

Audience Management operates across two distinct backend services:

ComponentResponsibilityTechnologyStorage
Audience Control Plane (dmd-api-frontend-go)Manages saved audience definitions, versions, lifecycle status (Draft, Saved), user ownership, and access control.Go (Gin)PostgreSQL
Analytical Query Engine (analytics-query-engine)Receives executable audience JSON payloads, validates rules, resolves field catalog metadata, and evaluates criteria against raw events and customer profiles.Go (Gin)ClickHouse

Why This Separation Matters​

  1. Stateless Query Execution: The execution plane does not persist matched user rows. Each estimate and preview is evaluated dynamically against real-time data, ensuring reach calculations always reflect the latest event streams.
  2. Immutable Traceability: When an audience definition is updated, a new definition version and cryptographic hash (definitionHash) are generated by the control plane. This hash is passed to the analytical engine to trace query performance across iterations.
  3. Storage Tiering: Postgres provides ACID transactions for audience metadata and campaign workflows, while ClickHouse delivers columnar, vectorized scans across billions of behavioral touchpoints.

2. Tenant Context & Isolation Boundaries​

DriveMetaData enforces strict multi-tenant isolation across all audience operations. Every request from the frontend must include tenant identification headers:

Authorization: Bearer <JWT_TOKEN>
Client-ID: <CLIENT_ID>
Workspace-ID: <WORKSPACE_ID>

Scope Fingerprinting & Frontend Protection​

To prevent accidental cross-tenant data leakage or corrupted local state, ui-audience-builder computes a continuous Scope Fingerprint:

Scope Fingerprint = Client-ID + Workspace-ID + App-ID

When the active workspace or tenant context changes:

  1. The frontend runtime detects the fingerprint mismatch via shouldResetAudienceRuntimeForScopeChange.
  2. All active audience queries, cached field catalogs, and unsaved condition states are immediately canceled and purged.
  3. The visual builder resets to a clean state, preventing rules from one tenant from being saved or evaluated in another.

3. The PII Protection & Data Privacy Model​

Audience Builder balances operational marketing needs with enterprise privacy and data governance regulations (GDPR, CCPA/CPRA, HIPAA).

PII Classification​

Field catalog members are categorized into distinct privacy classes:

ClassDescriptionExamplesLookup Policy
noneNon-sensitive behavioral or categorical attributesCustomer.country, touchpoint.source, Orders.purchaseCountAllowed in autocomplete and dropdown searches (/field-values).
direct_piiDirect identifiers that pinpoint individual identitiesCustomer.email, Customer.mobile, Customer.firstNameBlocked from dropdown value lookup. Requires explicit authorization to preview.

The X-Audience-PII-Access Authorization Header​

By default, preview queries mask sensitive fields (e.g., returning j***@example.com or empty rows). To preview raw, unmasked PII rows:

  1. The user's role must possess explicit PII read clearance in the tenant's RBAC matrix.
  2. The client must supply the header: 
    X-Audience-PII-Access: true
  3. If this header is omitted or unauthorized, the analytics engine automatically redacts direct identifiers before returning records to the preview table.

4. Query Execution Lifecycle​

When a user interacts with the Audience Builder, the system progresses through four execution phases:

  1. Catalog Resolution: On load, the UI fetches /api/v2/analytics/audiences/audience-fields to discover available entities, fields, data types, and supported operators.
  2. Payload Validation: Before evaluating or saving, the UI sends the executable query.filters payload to /validate. The backend confirms syntax, date constraints, and field eligibility.
  3. Reach Estimation: When requested, /preview executes with includeRows: false to calculate matching user counts, breakdown steps, and exclusion impacts without data transfer overhead.
  4. Sample Inspection: With attributes selected, /preview executes with includeRows: true to return row-level sample data under strict pagination limits.
  5. Persistence: The validated definition is submitted to dmd-api-frontend-go for long-term storage in Postgres, where downstream modules (Activation, Campaigns, Reporting) can ingest it.