Transform Blocks#

Transform blocks process data as it flows through your workflow, filtering unwanted content and extracting valuable information.

Keyword Filter#

Filters messages based on keyword matching.

Configuration#

Field	Description
Whitelist	Keywords that must be present
Blacklist	Keywords that must not be present
Match Mode	ANY (or) / ALL (and)
Case Sensitive	Enable case-sensitive matching

Example#

yaml

whitelist: ["malware", "vulnerability", "CVE"]
blacklist: ["false positive", "patched"]
match_mode: "ANY"
case_sensitive: false

Regex Extract#

Extracts data from messages using regular expressions.

Configuration#

Field	Description
Pattern	Regular expression pattern
Field	Which field to search (content, title)
Output Field	Name for extracted data
Match Mode	First match or all matches

Example#

yaml

pattern: "CVE-\\d{4}-\\d{4,}"
field: "content"
output_field: "cve_ids"
match_mode: "all"

IP Address Extract#

Automatically extracts IPv4 and IPv6 addresses.

Configuration#

Field	Description
IPv4	Extract IPv4 addresses
IPv6	Extract IPv6 addresses
Validate	Validate extracted addresses
Private	Include private addresses

Email Extract#

Extracts email addresses from content.

Configuration#

Field	Description
Domain Filter	Only extract from specific domains
Exclude Domains	Exclude specific domains

Crypto Address Extract#

Extracts cryptocurrency wallet addresses.

Supported Currencies#

Bitcoin (BTC)
Ethereum (ETH)
Solana (SOL)
Monero (XMR)

Tip: Chain multiple transforms to build comprehensive extraction pipelines.

Regex Filter#

Filters messages using regular expression pattern matching. Messages that match (or don't match) the pattern are passed through or rejected.

Configuration#

Field	Description
Regex Pattern	Regular expression to match against
Filter Mode	Whitelist (keep matches) or Blacklist (remove matches)
Case Sensitive	Enable case-sensitive matching
Search In	Which fields to search (content, title, or both)

Example#

yaml

pattern: "CVE-\\d{4}-\\d{4,}"
filter_mode: "whitelist"
case_sensitive: false
source_fields: ["content", "title"]

Tip: Use the Regex Filter for pattern-based filtering (e.g., keeping only posts mentioning CVE identifiers). Use the Keyword Filter for simpler word-based filtering.

Translate#

Translates message content to a target language. Supports auto-detection of source language.

Configuration#

Field	Description
Target Language	Language to translate into
Source Language	Source language (or Auto-detect)
Translate Fields	Which fields to translate (content, title)
Reject Failed	Remove messages that fail to translate

Supported Languages#

English, Spanish, French, German, Chinese, Arabic, Russian.

How It Works#

Messages pass through the node and their content/title fields are sent to the translation service. The translated text is stored in message.enrichments.translation:

json

{
  "enrichments": {
    "translation": {
      "target_language": "en",
      "source_language": "auto",
      "status": "success",
      "content": "Translated content here..."
    }
  }
}

Translation Status#

Status	Meaning
success	Translation completed and validated
unverified	Translation returned but could not be verified
partial	Some fields translated, others failed
failed	Translation failed entirely

Note: If "Reject Failed Translations" is enabled, messages with a "failed" status are removed from the pipeline entirely.

Chaining Transforms#

Transforms can be connected in sequence to create powerful processing pipelines:

Next Steps#

Output Blocks