Data Connectors

Data Connectors are the integration points that bring external data into your Fusion datalake. DTACT supports multiple connection types to accommodate different data architectures and use cases.

Connection Types

Virtual Connections (Schema Providers)

Virtual connections use schema providers to query external systems directly without storing data in DTACT. This "mesh" approach keeps data in place while enabling unified querying.

How it works:

• DTACT connects to the external system's API
• Queries are translated and executed at the source
• Results are returned in real-time
• No data is copied or stored in DTACT

Best for:

• Large datasets where duplication is impractical
• Compliance requirements that mandate data residency
• Real-time access to source systems
• Reducing storage costs

Examples:

• Cloud provider APIs (AWS, Azure, GCP)
• SaaS security tools (CrowdStrike, SentinelOne)
• Identity providers (Okta, Azure AD)
• SIEM systems

-- Query external data directly through schema provider
SELECT * FROM crowdstrike.detections 
WHERE severity = 'Critical'
AND timestamp > NOW() - INTERVAL '24 hours'

---

Individual Tables

Create standalone tables within your datalake for custom data that doesn't fit other connector types.

Use cases:

• Reference data (IP allowlists, asset inventories)
• Lookup tables for enrichment
• Manual data imports
• Custom datasets

---

Ingest Endpoints

Ingest endpoints provide HTTP and Filebeat-compatible receivers for external systems to push data into DTACT.

HTTP Endpoints

Each ingest endpoint provides a unique URL where external systems can POST data in any format.

How it works:

1. Create an ingest endpoint in Fusion
2. Configure your external system to send data to the endpoint URL
3. Data is automatically parsed and stored as a table
4. Query the data immediately via Query Analyzer

Supported formats:

• JSON (single objects or arrays)
• NDJSON (newline-delimited JSON)
• Custom formats with parsing rules

Example workflow:

External System → HTTP POST → Ingest Endpoint → Table → Query Analyzer

Filebeat Endpoints

For systems using Elastic Beats, DTACT provides Filebeat-compatible endpoints.

Configuration example:

output.elasticsearch:
  hosts: ["your-dtact-ingest-endpoint.dtact.com:443"]
  protocol: "https"

Best for:

• Custom applications sending telemetry
• Legacy systems with webhook capabilities
• IoT devices and sensors
• Third-party integrations without native DTACT support

---

File Uploads

Upload files directly to create queryable tables in your datalake.

Supported formats:

• JSON — Arrays of objects or NDJSON
• CSV — Comma-separated values with headers

How it works:

1. Navigate to Fusion → Data Connectors
2. Select "Upload File"
3. Choose your JSON or CSV file
4. Configure table name and schema options
5. File contents become a queryable table

Use cases:

• One-time data imports
• Historical data migration
• Analyst-provided reference data
• Threat intelligence feeds (STIX, CSV indicators)

-- Query uploaded threat intel file
SELECT * FROM uploaded_files.ioc_list
WHERE indicator_type = 'ip'
AND last_seen > NOW() - INTERVAL '30 days'

---

Choosing the Right Connector

Connector Type	Data Storage	Best For	Update Frequency
Virtual (Schema Provider)	At source	Large datasets, compliance	Real-time
Individual Tables	In DTACT	Reference data, lookups	Manual
Ingest Endpoints	In DTACT	Streaming data, webhooks	Continuous
File Uploads	In DTACT	One-time imports, historical	On upload

Data Availability

Once connected, all data sources appear as schemas in your datalake and can be:

• Queried via the Query Analyzer
• Joined with other data sources
• Used in Detections for alerting
• Visualized in Dashboards