Skip to content

Files

Latest commit

2d5beee · Dec 3, 2021

History

History
141 lines (121 loc) · 5.61 KB

control-flow-filter-activity.md

File metadata and controls

141 lines (121 loc) · 5.61 KB
title titleSuffix description author ms.author ms.reviewer ms.service ms.subservice ms.custom ms.topic ms.date
Filter activity
Azure Data Factory & Azure Synapse
The Filter activity filters the inputs to Azure Data Factory and Synapse Analytics pipelines.
chez-charlie
chez
jburchel
data-factory
orchestration
synapse
conceptual
09/09/2021

Filter activity in Azure Data Factory and Synapse Analytics pipelines

You can use a Filter activity in a pipeline to apply a filter expression to an input array. [!INCLUDEappliesto-adf-asa-md]

Syntax

{
    "name": "MyFilterActivity",
    "type": "filter",
    "typeProperties": {
        "condition": "<condition>",
        "items": "<input array>"
    }
}

Create a Filter activity with UI

To use a Filter activity in a pipeline, complete the following steps:

  1. You can use any array type variable or outputs from other activities as the input for your filter condition. To create an array variable, select the background of the pipeline canvas and then select the Variables tab to add an array type variable as shown below.

    :::image type="content" source="media/control-flow-activities-common/pipeline-array-variable.png" alt-text="Shows an empty pipeline canvas with an array type variable added to the pipeline.":::

  2. Search for Filter in the pipeline Activities pane, and drag a Filter activity to the pipeline canvas.

  3. Select the new Filter activity on the canvas if it is not already selected, and its Settings tab, to edit its details.

    :::image type="content" source="media/control-flow-filter-activity/filter-activity.png" alt-text="Shows the UI for a Filter activity.":::

  4. Select the Items field and then select the Add dynamic content link to open the dynamic content editor pane.

    :::image type="content" source="media/control-flow-filter-activity/add-dynamic-content-link.png" alt-text="Shows the  Add dynamic content  link for the Items property.":::

  5. Select your input array to be filtered in the dynamic content editor. In this example, we select the variable created in the first step.

    :::image type="content" source="media/control-flow-activities-common/add-dynamic-content-pane.png" alt-text="Shows the dynamic content editor with the variable created in the first step selected":::

  6. Use the dynamic content editor again to specify a filter condition for the Condition property, as shown above.

  7. You can use the output from the Filter activity as an input to other activities like the ForEach activity.

Type properties

Property Description Allowed values Required
name Name of the Filter activity. String Yes
type Must be set to filter. String Yes
condition Condition to be used for filtering the input. Expression Yes
items Input array on which filter should be applied. Expression Yes

Example

In this example, the pipeline has two activities: Filter and ForEach. The Filter activity is configured to filter the input array for items with a value greater than 3. The ForEach activity then iterates over the filtered values and sets the variable test to the current value.

{
    "name": "PipelineName",
    "properties": {
        "activities": [{
                "name": "MyFilterActivity",
                "type": "filter",
                "typeProperties": {
                    "condition": "@greater(item(),3)",
                    "items": "@pipeline().parameters.inputs"
                }
            },
            {
            "name": "MyForEach",
            "type": "ForEach",
            "dependsOn": [
                {
                    "activity": "MyFilterActivity",
                    "dependencyConditions": [
                        "Succeeded"
                    ]
                }
            ],
            "userProperties": [],
            "typeProperties": {
                "items": {
                    "value": "@activity('MyFilterActivity').output.value",
                    "type": "Expression"
                },
                "isSequential": "false",
                "batchCount": 1,
                "activities": [
                    {
                        "name": "Set Variable1",
                        "type": "SetVariable",
                        "dependsOn": [],
                        "userProperties": [],
                        "typeProperties": {
                            "variableName": "test",
                            "value": {
                                "value": "@string(item())",
                                "type": "Expression"
                            }
                        }
                    }
                ]
            }
        }],
        "parameters": {
            "inputs": {
                "type": "Array",
                "defaultValue": [1, 2, 3, 4, 5, 6]
            }
        },
        "variables": {
            "test": {
                "type": "String"
            }
        },
        "annotations": []
    }
}

Next steps

See other supported control flow activities: