Using Repeating Entities with the Incoming Webhook Step

The Incoming Webhooks step allows builders to receive external data into a workflow. With Repeating Entities, this step can accept hierarchical and nested JSON payloads containing repeating records (such as parent-child or multi-level data structures).

When a webhook is received, Streamline:

• Detects collections (arrays of data) in the payload
• Normalizes the data structure
• Stores collections externally via the File Upload Service
• Makes the data available in the workflow context for downstream steps

This enables workflows to process complex datasets such as:

• Invoices with multiple line items
• Orders with nested product details
• Records with parent-child relationships

📖 Learn more: [How to create an Inbound Webhook Step]

Who Can Use This Feature

• Builders configure inbound webhook steps, view collection fields, and map data to downstream steps
• Admins have full access to webhook configuration and account-level management
• Operators can view workflow execution results, including received collection data
• Managers can review workflow configurations and execution history

How It Works

When a webhook payload is received:

1. The system parses the incoming JSON payload
2. Arrays are detected and converted into collections
3. Collections are normalized into a structured format
4. Collection data is stored in the File Upload Service
5. References to those collections are added to the workflow data context

Downstream steps (such as Document Generation or Forms) can then access and iterate over this data.

Example: Invoice Payload with Line Items

Incoming JSON payload:

{
  "invoice_id": "INV-1001",
  "customer": "Acme Corp",
  "line_items": [
    { "name": "Product A", "price": 10 },
    { "name": "Product B", "price": 25 },
    { "name": "Product C", "price": 15 }
  ]
}

Result in workflow:

• invoice_id and customer are stored as standard fields
• line_items is detected as a collection
• Each object in the array becomes a record in that collection

This collection can then be used in downstream steps like:

• Document Generation (for tables or lists)
• Data processing or integrations

What You Can Do

Receive and Parse Collection Data

Inbound Webhooks support JSON payloads containing arrays of objects.

These arrays are automatically detected and converted into structured collections within the workflow data context.

Support Multiple Data Structures

The ingestion layer supports:

• Standard arrays of objects
• Object-based lists (keys preserved as an id field)
• Arrays of primitive values

This allows flexibility in how external systems send data.

Work with Nested Collections

Nested collections (parent-child relationships) are supported during ingestion.

The system can process multiple levels of nested data, enabling complex hierarchical structures to be passed into workflows.

Store Large Collection Data Efficiently

All collection data is stored externally using the File Upload Service, rather than within the workflow engine payload.

This:

• Avoids payload size constraints within the workflow engine
• Enables downstream steps to retrieve data on demand

View Collection Data in the Builder

Collection fields are displayed in the builder UI with:

• Hierarchical structure (parent → child relationships)
• Field names
• Repeating indicators
• Item counts

This makes it easier to map and understand incoming data.

Use Native Data Types Without Transformation

Collection records support the same field types as standard fields, including:

• Text
• Numbers
• Dates and date-time values
• Booleans
• File references

Type inference automatically detects formats such as ISO date/time strings.

Handle File Data in Payloads

Webhook payloads can include FILE fields using base64-encoded content.

These files are:

• Extracted during ingestion
• Stored in the File Upload Service
• Made available for downstream steps

Authenticate Incoming Webhooks

Inbound Webhooks support multiple authentication methods:

• No authentication (public endpoint)
• PAT (Personal Access Token)
• HMAC (SHA-256 signature verification) using the Streamline-Signature header

Maintain Backward Compatibility

Existing webhook configurations continue to work unchanged.

The system stores both:

• Flattened (legacy) field formats
• Collection-based (new) formats

This ensures compatibility with existing workflows while enabling new functionality.

Feature Considerations

Payload Size Limits

Incoming webhook payloads have a maximum size of 5 MB.

• Payloads exceeding this limit are rejected
• A warning is logged when payload size exceeds 2 MB

Large datasets should be split across multiple webhook calls if needed.

Nesting Depth and Downstream Usage

The ingestion layer supports deep nesting (up to 15 levels).

However, downstream steps may have practical limits:

• Document Generation supports up to 2 levels of nesting per loop path

Plan your data structure based on how it will be used later in the workflow.

Collection Volume

The system supports a large number of collections per payload (up to 200 configurable).

A recommended guideline is:

• ~50 collections per payload for optimal performance

Collection Size and Field Count

There are no strict limits on:

• Number of records per collection
• Number of fields per record

However, large or complex datasets may impact performance and usability.

Schema Detection Behavior

Collection fields are inferred from incoming payloads.

If fields are missing or inconsistent:

• They may not appear in the builder
• Schema may vary between payloads

Consistent payload structure is recommended.

Collection Key Mapping

Object-based lists preserve keys as an id field by default.

Custom key naming is not currently configurable.

Downstream Dependency

Repeating Entities require downstream steps (such as Document Generation or Forms) to fully utilize collection data.

This feature enables ingestion, but full workflows depend on integration with other steps.

Troubleshooting Common Issues

Compliance

This feature follows existing platform security and data handling practices:

• No new data storage systems are introduced
• Collection data is stored using the File Upload Service
• Existing authentication and access controls apply
• Data retention follows existing workflow execution policies
• No additional PII processing is introduced