DuckVizCustom Log Formats
Save your own log regex + columns so detection is local-only next time a similar file drops.
DuckViz ships a 47-format log catalog (syslog, Apache/Nginx, Windows Event XML, systemd, CloudTrail, many more) that handles most real-world files. When the catalog misses, an LLM proposes a regex and columns — and that result is auto-saved to your account, so the next file in the same shape parses locally with zero server contact.
You can also define formats by hand at /settings/log-formats — useful when you already know the regex for an in-house log format.
Format shape
A saved format has four fields you fill in on the form:
| Field | What it is |
|---|---|
| Name | Human-readable display name (e.g. "Orders API app log") |
| Slug | URL-safe identifier, unique per user (e.g. orders-api) |
| Regex | A named-capture regex matching one log line. Use (?P<ts>...), (?P<level>...), etc. |
| Columns | Column names in capture order — must match the named captures in your regex |
| Hints (optional) | Freeform notes the AI uses when working with this format — date format, units, categorical hints |
If your named captures don't line up with the columns you list, the save is rejected with a descriptive error.
Writing a format
Example — an in-house app log line:
2026-04-22T10:33:14.221Z [WARN] [orders-api] req=1f9a user=alice path=/checkout status=503 ms=812A format for this:
{
"slug": "orders-api",
"name": "Orders API app log",
"regex": "^(?P<ts>\\S+)\\s+\\[(?P<level>\\w+)\\]\\s+\\[(?P<service>[^\\]]+)\\]\\s+req=(?P<req>\\S+)\\s+user=(?P<user>\\S+)\\s+path=(?P<path>\\S+)\\s+status=(?P<status>\\d+)\\s+ms=(?P<ms>\\d+)",
"columns": ["ts", "level", "service", "req", "user", "path", "status", "ms"],
"hints": "Orders API timestamps are ISO-8601 UTC (%Y-%m-%dT%H:%M:%S.%fZ)."
}The hints field is passed to the AI when it's working with this format — use it to nudge date-format inference, flag that ms is latency in milliseconds (not microseconds), or declare that a numeric column is categorical.
CRUD UI
/settings/log-formats is the CRUD interface. Actions:
| Action | What it does |
|---|---|
| New | Empty form; paste regex + columns, save |
| Try | Paste 3–5 lines of sample log; the UI runs your regex locally and shows parsed + unparsed rows |
| Edit | Modify in place; saves bump the format's updated_at |
| Delete | Removes the row; files currently parsing with it will fail gracefully (fallback to LLM detect) |
Formats are scoped to your account — there's no team-shared catalog today.
Detection order
When a log file drops, DuckViz runs detection in this order:
- Built-in catalog — the 47-format catalog
- Your saved formats — anything you've saved on your account
- AI detection — paid, 1 credit
- Auto-save the AI result — so next time it's a step-2 hit
Step 4 is the killer feature. Drop the same format a second time and you pay zero credits.
Export / share
No team-share today — every user maintains their own format library. If you want to share a format with a colleague, send them the regex + columns and they paste it into their own /settings/log-formats.
Debugging a format
If your regex parses some lines but not all:
- Go to the Unparsed tab in the explorer after ingest
- Select a couple of representative lines
- Click Generate Pattern & Ingest → this runs the LLM on the selection, which often produces a regex that subsumes both cases
- Save the new pattern at
/settings/log-formatsand delete your old one
See Log Analysis → Re-parse the leftovers.
See also
- Log Analysis — the end-to-end detection funnel
- Supported formats — the built-in 47-format catalog