# DomainDrift -- Continuous Internet Telemetry Index
> Built by DRM3 Labs. Monitors ~1,006,000 domains across 70+ categories. Every scan is cryptographically signed with Ed25519 provenance receipts.

DomainDrift is an internet telemetry index that continuously scans domains for DNS records (A, AAAA, MX, NS, TXT, CNAME, SOA, CAA, SRV), MCP TXT records, TLS certificates (CN, issuer, SANs, validity), WHOIS/RDAP, HTTPS probes with verdict classification, DNSSEC, robots.txt analysis, and ASN data. It stores time-series snapshots with content-hash change detection and cryptographic provenance chains.

## Access

Get a DomainDrift API key by enabling DomainDrift in your DRM3 account (https://drm3.network/account) or requesting access at https://interest.drm3.network. Pass it via the X-DomainDrift-Key header (keys are prefixed dd_).

## Quick Start

curl -H "X-DomainDrift-Key: YOUR_KEY" https://domaindrift.io/v1/domains/google.com
curl -H "X-DomainDrift-Key: YOUR_KEY" https://domaindrift.io/v1/domains?category=big_tech&limit=10

## Provenance

Every scan is signed with Ed25519. Each domain has a provenance chain -- receipts linked by parent_receipt_id. Verify any receipt by checking the Ed25519 signature against the public key in the receipt JSON. Provenance proves this data was observed by a specific scanner at a specific time.

Multi-plane provenance: the fast scan (DNS + TLS + HTTPS) gets its own receipt, and each enrichment plane (whois, asn, robots, dnssec) gets a separate receipt. A composite receipt wraps all planes for a given scan cycle.

---

## API Endpoints

All /v1/ endpoints require X-DomainDrift-Key header. Rate limits apply per key (demo: 100/min, service: 10,000/min).

### Public (no auth required)
- GET /health -- service health (domains tracked, snapshots, scan rate)
- GET /llms.txt -- LLM-friendly summary (this file's short version)
- GET /llms-full.txt -- LLM-friendly full reference (this file)
- GET /robots.txt -- crawler directives
- GET /sitemap.xml -- sitemap
- GET /v1/openapi.json -- OpenAPI 3.1 specification
- GET /probe?domain=example.com -- diagnostic DNS probe (rate limited, 10/min)

### Domain Queries
- GET /v1/domains -- list tracked domains
  Query params: category, source, active (0|1), page, limit
- GET /v1/domains/{domain} -- latest full signal snapshot
  Returns: DNS records, WHOIS, certificates, subdomains, HTTPS probe, robots.txt analysis, ASN, DNSSEC, enrichment data
- GET /v1/domains/{domain}/history -- time-series snapshots
  Query params: limit (default 20)
- GET /v1/domains/{domain}/provenance -- provenance receipt chain
  Query params: limit (default 20)

### Per-Plane Observations
- GET /v1/domains/{domain}/planes -- latest observation per plane + completeness summary
  Planes: fast, whois, asn, robots, dnssec, composite
- GET /v1/domains/{domain}/planes/{plane}/history -- paginated per-plane history

### Change Feed
- GET /v1/changes -- global delta feed of DNS changes
  Query params: limit, domain
- GET /v1/stream -- SSE event stream (changes, scans, provenance)
- GET /v1/stream/json -- JSON fallback for event stream

### Write (scanner service key required)
- POST /v1/domains -- add domain(s) to catalog
  Body: { domain, domains[], category }
- POST /v1/scan/{domain} -- push complete scan result
  Body: { signal, receipt } -- both required, unsigned pushes rejected

### Enrichment
- POST /v1/enrich-batch?batch=N -- trigger enrichment (RDAP, DNSSEC, robots.txt, ASN)
- POST /v1/enqueue -- enqueue domains for async enrichment
- GET /v1/queue/depth -- enrichment queue depth
- GET /v1/queue/errors -- enrichment errors
- GET /v1/queue/done -- completed enrichment jobs
- GET /v1/queue/enriched -- enriched domain count

### BigQuery
- POST /v1/bq/backfill -- backfill BigQuery from D1 (500 rows/batch)
- GET /v1/bq/status -- BigQuery connection status

---

## Data Per Domain

Each scan signal contains:
- DNS records: A, AAAA, CNAME, MX, TXT, NS, SOA, CAA, SRV
- MCP TXT records: _mcp._tcp.{domain} for AI service discovery
- TLS certificates: CN, issuer org/country/locality, SANs, validity dates (via crt.sh CT logs)
- WHOIS/RDAP: registrar, creation/expiration dates, org, country, name servers
- HTTPS probe with verdict: ok, redirect, bot_blocked, timeout, tls_error, connection_refused, server_error, not_found, rate_limited, cf_blocked, network_error, dns_failed
- Subdomains: discovered from certificate SAN names
- robots.txt: raw content, parsed rules, AI crawler blocking detection
- ASN: autonomous system number and organization
- DNSSEC: validation status
- DNS conflicts: CNAME+A coexistence, CNAME+MX, orphaned delegations
- Content hash: SHA-256 of canonicalized stable signal
- Provenance receipt: Ed25519 signed, chained to previous scan

---

## Domain Categories

70+ categories (sample): your_domains, big_tech, dev_tools_platforms, cloud_hosting, programming_languages_ecosystems, ai_ml, crypto_web3, fintech, cybersecurity, devops_observability, email_comms, cms_content, media_publishing, ecommerce, data_analytics, design_creative, search_discovery, testing, api_docs, infrastructure, gaming_metaverse, health_biotech, education, legal_compliance, hr_recruiting, real_estate_proptech, travel, food_agriculture, logistics, misc_notable, saas_tools, blockchain_infra, developer, open_source, fintech_extras, enterprise, deep_tech, banks, fortune_500, government, aerospace, automotive, user_added

---

## Scan Signal JSON Structure

```json
{
  "domain": "example.com",
  "scanned_at": "2026-03-24T12:00:00.000Z",
  "public_key": "connor-fly-v0.3.1",
  "probe_status": {
    "dns": { "status": "ok", "duration_ms": 45 },
    "whois": { "status": "ok", "duration_ms": 1200 },
    "certificates": { "status": "ok", "duration_ms": 800 },
    "https_probe": { "status": "ok", "duration_ms": 150 }
  },
  "dns": {
    "a": ["93.184.216.34"],
    "aaaa": ["2606:2800:220:1:248:1893:25c8:1946"],
    "mx": [{ "priority": 10, "exchange": "mail.example.com" }],
    "txt": ["v=spf1 -all"],
    "ns": ["ns1.example.com", "ns2.example.com"],
    "cname": [],
    "soa": { "mname": "ns1.example.com", "rname": "admin.example.com", "serial": 2024031201 },
    "caa": [{ "flags": 0, "tag": "issue", "value": "letsencrypt.org" }],
    "srv": []
  },
  "whois": {
    "registrar": "Example Registrar",
    "creation_date": "1995-08-14",
    "expiration_date": "2027-08-13",
    "registrant_org": "Example Inc.",
    "registrant_country": "US"
  },
  "certificates": [
    {
      "common_name": "example.com",
      "issuer": "R3",
      "subject_org": "Example Inc.",
      "not_after": "2025-01-01T00:00:00Z",
      "san_names": ["example.com", "www.example.com"]
    }
  ],
  "output_hash": "sha256:abc123..."
}
```

---

## Delta (Change Detection) Structure

```json
{
  "domain": "example.com",
  "detected_at": "2026-03-24T18:00:00.000Z",
  "changes": [
    { "field": "dns.a", "action": "changed", "old_value": "93.184.216.34", "new_value": "93.184.216.35" },
    { "field": "dns.txt", "action": "added", "old_value": null, "new_value": "v=DMARC1; p=reject" }
  ],
  "change_count": 2,
  "change_summary": "A record changed, TXT record added"
}
```

---

## Provenance Receipt Structure

```json
{
  "id": "receipt-uuid",
  "domain": "example.com",
  "scanned_at": "2026-03-24T12:00:00.000Z",
  "output_hash": "sha256:abc123...",
  "public_key": "ed25519:<hex>",
  "signature": "ed25519:<hex>",
  "parent_receipt_id": "previous-receipt-uuid-or-null"
}
```

Verify: check that Ed25519 signature over output_hash matches the public_key. Chain integrity: follow parent_receipt_id links back to the genesis receipt (parent_receipt_id = null).

---

## Links
- DomainDrift: https://domaindrift.io
- OpenAPI spec: https://domaindrift.io/v1/openapi.json
- Who it is for: https://domaindrift.io/for/domain-investors, /for/webmasters, /for/sales-teams, /for/compliance, /for/security-teams, /for/business-owners
- DRM3: https://drm3.network