Docs/Data & Analytics/Data Export

Data Export

Export your Ilara data to BigQuery, S3, and other data warehouses for advanced analysis.

10 min read

Overview

Ilara supports exporting your raw event data to popular data warehouses and storage systems. Use exported data for custom analysis, ML training, or regulatory compliance.

Supported Destinations

DestinationTypeSync Frequency
BigQueryData WarehouseReal-time, Hourly, Daily
SnowflakeData WarehouseReal-time, Hourly, Daily
Amazon RedshiftData WarehouseHourly, Daily
Amazon S3Object StorageReal-time, Hourly, Daily
Google Cloud StorageObject StorageReal-time, Hourly, Daily
Azure Blob StorageObject StorageHourly, Daily

BigQuery Setup

Configuration

  1. Go to Settings → Integrations → Data Export
  2. Click "Add Destination" → BigQuery
  3. Upload your service account JSON key
  4. Select project and dataset
  5. Choose sync frequency
  6. Save and test connection
BigQuery Configuration
json
{
  class="code-string">"destination": class="code-string">"bigquery",
  class="code-string">"credentials": {
    class="code-string">"project_id": class="code-string">"your-gcp-project",
    class="code-string">"dataset_id": class="code-string">"ilara_data",
    class="code-string">"service_account_key": class="code-string">"..."
  },
  class="code-string">"sync": {
    class="code-string">"frequency": class="code-string">"hourly",
    class="code-string">"include_historical": true,
    class="code-string">"start_date": class="code-string">"2025-01-01"
  },
  class="code-string">"tables": {
    class="code-string">"events": class="code-string">"ilara_events",
    class="code-string">"players": class="code-string">"ilara_players",
    class="code-string">"sessions": class="code-string">"ilara_sessions"
  }
}

Schema

Ilara creates these tables in your dataset:

TableDescription
ilara_eventsAll tracked events with properties
ilara_playersPlayer profiles and attributes
ilara_sessionsSession records
ilara_purchasesPurchase transactions

Snowflake Setup

Snowflake Configuration
json
{
  class="code-string">"destination": class="code-string">"snowflake",
  class="code-string">"credentials": {
    class="code-string">"account": class="code-string">"your-account.us-east-1",
    class="code-string">"user": class="code-string">"Ilara_SERVICE_USER",
    class="code-string">"password": class="code-string">"...",
    class="code-string">"warehouse": class="code-string">"Ilara_WH",
    class="code-string">"database": class="code-string">"Ilara_DB",
    class="code-string">"schema": class="code-string">"RAW"
  },
  class="code-string">"sync": {
    class="code-string">"frequency": class="code-string">"real_time",
    class="code-string">"batch_size": 10000
  }
}

Amazon S3 Setup

S3 Configuration
json
{
  class="code-string">"destination": class="code-string">"s3",
  class="code-string">"credentials": {
    class="code-string">"access_key_id": class="code-string">"AKIA...",
    class="code-string">"secret_access_key": class="code-string">"...",
    class="code-string">"region": class="code-string">"us-east-1"
  },
  class="code-string">"bucket": class="code-string">"your-ilara-exports",
  class="code-string">"prefix": class="code-string">"raw/events/",
  class="code-string">"format": class="code-string">"parquet",
  class="code-string">"compression": class="code-string">"snappy",
  class="code-string">"sync": {
    class="code-string">"frequency": class="code-string">"hourly",
    class="code-string">"partition_by": [class="code-string">"date", class="code-string">"event_name"]
  }
}

File Formats

FormatBest ForCompression
ParquetAnalytics queries, Athena/SparkSnappy, Gzip
JSON LinesFlexibility, debuggingGzip
CSVSpreadsheet toolsGzip

Export Schema

Events Table

Events Schema
sql
CREATE TABLE ilara_events(
  event_id STRING,
  player_id STRING,
  event_name STRING,
  timestamp TIMESTAMP,
  session_id STRING,
  properties JSON,
  tenant_id STRING,
  received_at TIMESTAMP,
  -- Partitioned by date for query performance
  partition_date DATE
);

Players Table

Players Schema
sql
CREATE TABLE ilara_players(
  player_id STRING,
  external_id STRING,
  device_id STRING,
  email STRING,
  lifecycle_stage STRING,
  session_count INT,
  total_revenue DECIMAL,
  is_payer BOOLEAN,
  first_seen_at TIMESTAMP,
  last_seen_at TIMESTAMP,
  attributes JSON,
  tenant_id STRING
);

Export Filtering

Filter which data is exported:

Filter Configuration
json
{
  class="code-string">"filters": {
    class="code-string">"event_names": {
      class="code-string">"mode": class="code-string">"include",
      class="code-string">"values": [class="code-string">"session_start", class="code-string">"session_end", class="code-string">"purchase_complete", class="code-string">"level_complete"]
    },
    class="code-string">"segments": {
      class="code-string">"mode": class="code-string">"include",
      class="code-string">"values": [class="code-string">"paying-users", class="code-string">"high-value"]
    },
    class="code-string">"date_range": {
      class="code-string">"start": class="code-string">"2025-01-01",
      class="code-string">"end": null
    }
  }
}
Cost Optimization
Filter to only the events you need for analysis. This reduces storage costs and query times.

Historical Backfill

When setting up a new export, you can backfill historical data:

  • Full backfill: Export all historical data (may take time)
  • Date range: Specify start/end dates for backfill
  • Forward only: Only export new data from setup time
Backfill Costs
Large historical backfills may incur additional data transfer costs. Contact support for enterprise backfill options.

Monitoring Exports

Monitor export health in Settings → Data Export → Status:

  • Last Sync: When data was last exported
  • Records Exported: Count of events/players synced
  • Errors: Any sync failures with details
  • Latency: Time from event to export

Next Steps

Previous
Custom Reports
Next
Webhooks