HTTP webhook ingest

A webhook pipeline exposes a stable URL that accepts NDJSON over POST. Every line of the body becomes one record in the pipeline's target_table. It's the simplest end-to-end path from "your service has events" to "they're queryable in the Explore Query tab".

When to use

• Application events from a backend you control (POST from your /login, /checkout, /error handlers)
• Output of a log shipper — Vector, Fluent Bit, OpenTelemetry Collector's HTTP exporter, Loki Promtail
• Third-party webhooks (GitHub, Slack, Stripe, Linear, …) — Agentcy receives the event payload directly
• Anywhere a curl would work

If you're already running a log shipper for observability, point its HTTP output at an Agentcy webhook pipeline and you have a unified telemetry layer next to your knowledge graph.

1. Create the pipeline

Open Explore → Pipelines and click Add pipeline. Pick HTTP webhook.

Field	Value	Why
Name	something memorable, e.g. app-events	Shows up everywhere
Target table	context_events for events; context_nodes for entities	Determines the physical table
Realm	default or your own	Scopes the records for queries and policy
Require signature	off (start) → on later	Forces HMAC-SHA256 on every request

Or via API:

curl -X POST https://your.agentcy.dev/api/v1/context/pipelines \
  -H "authorization: Bearer $JWT" \
  -H "content-type: application/json" \
  -d '{
    "name": "app-events",
    "kind": "webhook",
    "target_table": "context_events",
    "realm": "default",
    "require_signature": false
  }'

2. Issue a token

Pipelines never accept un-authenticated traffic. Each token is a separate credential — you can issue many per pipeline (one per shipper, one per environment, etc.) and revoke individually. Tokens look like ctx_<32 random alphanumeric>.

In the UI: Issue token → leave Signing secret empty for now → Generate. The token, full URL, and a copy-pasteable curl example appear inline.

Or via API:

curl -X POST https://your.agentcy.dev/api/v1/context/pipelines/$PIPELINE_ID/tokens \
  -H "authorization: Bearer $JWT" -H 'content-type: application/json' \
  -d '{}'
# => { "token": "ctx_3a9b…" }

The token is shown once — copy it.

3. Send your first event

The endpoint accepts NDJSON. One JSON object per line, separated by \n.

curl -X POST 'https://your.agentcy.dev/api/v1/context/ingest/webhook/ctx_3a9b…' \
  -H 'content-type: application/x-ndjson' \
  --data-binary $'{"label":"Event","name":"login","user_email":"alice@…","at":"2026-04-29T12:00:00Z"}\n{"label":"Event","name":"checkout","amount_usd":29.99,"at":"2026-04-29T12:00:01Z"}\n'

Schema lands as you go

You don't pre-create the table or its columns. On the first request the target table is auto-created with the default schema (at, label, body, props). Every additional top-level property in your records becomes a new Utf8 column, capped at 32 new columns per request. Reserved names (at, label, body, props) are not promoted — at is the timestamp, props is a Binary catch-all that holds anything we couldn't promote. You can always drop the receiver strictness by sending X-Auto-Create: false and X-Schema-Evolve: false once your shape stabilises.

The server returns:

{ "rows_in": 2, "rows_out": 2, "run_id": 4711 }

Within a second the records are visible in Explore → Search (filter by label = Event) and in Explore → Query:

context_events
| where realm == "default" and name == "checkout"
| summarize total_usd = sum(amount_usd) by bin(at, 1h)

4. Wire your log shipper

Vector

[sources.app_logs]
type = "file"
include = ["/var/log/myapp/*.log"]

[transforms.json]
type = "remap"
inputs = ["app_logs"]
source = '''
. = parse_json!(.message)
.label = "AppLog"
'''

[sinks.agentcy]
type = "http"
inputs = ["json"]
uri = "https://your.agentcy.dev/api/v1/context/ingest/webhook/ctx_3a9b…"
method = "post"
encoding.codec = "ndjson"
framing.method = "newline_delimited"
batch.timeout_secs = 1
batch.max_events = 500

Fluent Bit

[OUTPUT]
    Name        http
    Match       *
    URI         /api/v1/context/ingest/webhook/ctx_3a9b…
    Host        your.agentcy.dev
    Port        443
    tls         On
    Format      json_lines
    json_date_format  iso8601
    json_date_key     at

OpenTelemetry Collector

Use the OTLP/HTTP exporter pointed at the webhook URL with a JSON marshaler. See the telemetry guide.

5. Sign every request (recommended for production)

Toggle Require signature on the pipeline (or require_signature: true in the create body). Then issue a token with a signing_secret:

curl -X POST .../tokens -H 'content-type: application/json' \
  -d '{"signing_secret": "openssl-rand-hex-32-or-similar"}'

Every request must now carry an X-Agentcy-Signature header:

X-Agentcy-Signature: sha256=<hmac of the body, hex-encoded>

In bash:

SECRET="openssl-rand-hex-32-or-similar"
BODY=$'{"label":"Event","name":"login"}\n'
SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$SECRET" -binary | xxd -p -c 256)

curl -X POST '...webhook/ctx_3a9b…' \
  -H "X-Agentcy-Signature: sha256=$SIG" \
  -H 'content-type: application/x-ndjson' \
  --data-binary "$BODY"

Most log shippers (Vector, Fluent Bit) have built-in HMAC sign-on-send; consult their HTTP-output docs.

6. Observability

Each POST becomes one run with rows_in, rows_out, status, and error populated. Open the pipeline in Explore and click Show runs for the live history. Failed runs surface their error message inline.

Errors you'll see

Status	Meaning	Fix
400 Bad NDJSON	One line of the body wasn't valid JSON	Validate before send; most shippers do this
401 Unknown token	Token revoked or never existed	Issue a fresh one
401 Bad signature	require_signature is on and HMAC didn't match	Re-check secret, binary mode, header format
413 Payload too large	Body exceeds limit (default 8 MB)	Batch smaller
503 Engine unavailable	The Context Engine refused (maintenance, kyma down on Advanced)	Retry with exponential backoff

Limits & tuning

• Body size: 8 MB default (configurable via CONTEXT_INGEST_MAX_BYTES)
• Records per request: no hard cap; recommend 1–5k for low latency
• Idempotency: send Idempotency-Key: <uuid> to dedupe identical retries on Advanced (context_pipeline_runs.source_ref records the key)
• Concurrency: pipelines run requests in parallel; ordering across requests is not guaranteed

Next

• Filedrop ingest — for batch ingestion via S3 / MinIO
• Telemetry pipelines — sending OTel-shaped data via HTTP today
• Context Engine concept — what queries can target this data