Workflow Concepts#

Understanding the core concepts behind HubFeed workflows will help you build more effective intelligence pipelines.

What is a Workflow?#

A workflow is a visual representation of a data processing pipeline. It defines:

• Where data comes from (Sources)
• How data is processed (Transforms)
• Where data goes (Outputs)
• Which credentials are used (Configuration)

Data Flow#

Data flows through your workflow in a directed graph:

Multi-Source Workflows#

You can aggregate data from multiple sources into a single pipeline:

Messages#

The fundamental unit of data in HubFeed is a Message. Each message contains:

• id - Unique identifier
• content - The main text content
• title - Optional title
• url - Source URL
• author - Content author
• created_at - When the content was created
• metadata - Platform-specific data

Enrichments#

As messages pass through transforms, they accumulate enrichments:

{
  "enrichments": {
    "extracted_ips": ["192.168.1.1"],
    "extracted_emails": ["test@example.com"],
    "matched_keywords": ["security", "threat"]
  }
}

Workflow States#

Workflows can be in one of several states:

Validation#

Before a workflow can be activated, it must pass validation:

• All required fields must be configured
• Connections must be valid
• No circular dependencies

Important: Invalid workflows cannot be saved or activated.

Next Steps#

• Source Blocks
• Transform Blocks
• Output Blocks
• Configuration Blocks