title | titleSuffix | description | services | ms.service | ms.subservice | ms.author | author | ms.date | ms.topic | ms.custom |
---|---|---|---|---|---|---|---|---|---|---|
Use pipeline parameters to build versatile pipelines |
Azure Machine Learning |
How to use pipeline parameters in Azure Machine Learning designer. |
machine-learning |
machine-learning |
mlops |
keli19 |
likebupt |
10/21/2021 |
how-to |
designer |
Use pipeline parameters to build flexible pipelines in the designer. Pipeline parameters let you dynamically set values at runtime to encapsulate pipeline logic and reuse assets.
Pipeline parameters are especially useful when resubmitting a pipeline run, retraining models, or performing batch predictions.
In this article, you learn how to do the following:
[!div class="checklist"]
- Create pipeline parameters
- Delete and manage pipeline parameters
- Trigger pipeline runs while adjusting pipeline parameters
-
An Azure Machine Learning workspace. See Create an Azure Machine Learning workspace.
-
For a guided introduction to the designer, complete the designer tutorial.
[!INCLUDE machine-learning-missing-ui]
There are three ways to create a pipeline parameter in the designer:
- Create a pipeline parameter in the settings panel, and bind it to a component.
- Promote a component parameter to a pipeline parameter.
- Promote a dataset to a pipeline parameter
Note
Pipeline parameters only support basic data types like int
, float
, and string
.
In this section, you create a pipeline parameter in the settings panel.
In this example, you create a pipeline parameter that defines how a pipeline fills in missing data using the Clean missing data component.
-
Next to the name of your pipeline draft, select the gear icon to open the Settings panel.
-
In the Pipeline parameters section, select the + icon.
-
Enter a name for the parameter and a default value.
For example, enter
replace-missing-value
as parameter name and0
as default value.
After you create a pipeline parameter, you must attach it to the component parameter that you want to dynamically set.
The simplest way to create a pipeline parameter for a component value is to promote a component parameter. Use the following steps to promote a component parameter to a pipeline parameter:
-
Select the component you want to attach a pipeline parameter to.
-
In the component detail pane, mouseover the parameter you want to specify.
-
Select the ellipses (...) that appear.
-
Select Add to pipeline parameter.
-
Enter a parameter name and default value.
-
Select Save
You can now specify new values for this parameter anytime you submit this pipeline.
If you want to submit your pipeline with variable datasets, you must promote your dataset to a pipeline parameter:
-
Select the dataset you want to turn into a pipeline parameter.
-
In the detail panel of dataset, check Set as pipeline parameter.
You can now specify a different dataset by using the pipeline parameter the next time you run the pipeline.
In this section, you will learn how to attach and detach component parameter to pipeline parameter.
You can attach the same component parameters of duplicated components to the same pipeline parameter if you want to alter the value at one time when triggering the pipeline run.
The following example has duplicated Clean Missing Data component. For each Clean Missing Data component, attach Replacement value to pipeline parameter replace-missing-value:
-
Select the Clean Missing Data component.
-
In the component detail pane, to the right of the canvas, set the Cleaning mode to "Custom substitution value".
-
Mouseover the Replacement value field.
-
Select the ellipses (...) that appear.
-
Select the pipeline parameter
replace-missing-value
.
You have successfully attached the Replacement value field to your pipeline parameter.
After you attach Replacement value to pipeline parameter, it is non-actionable.
You can detach component parameter to pipeline parameter by clicking the ellipses (...) next to the component parameter, and select Detach from pipeline parameter.
In this section, you learn how to update and delete pipeline parameters.
Use the following steps to update a component pipeline parameter:
- At the top of the canvas, select the gear icon.
- In the Pipeline parameters section, you can view and update the name and default value for all of your pipeline parameter.
Use the following steps to delete a dataset pipeline parameter:
- Select the dataset component.
- Uncheck the option Set as pipeline parameter.
Use the following steps to delete a component pipeline parameter:
-
At the top of the canvas, select the gear icon.
-
Select the ellipses (...) next to the pipeline parameter.
This view shows you which components the pipeline parameter is attached to.
-
Select Delete parameter to delete the pipeline parameter.
[!NOTE] Deleting a pipeline parameter will cause all attached component parameters to be detached and the value of detached component parameters will keep current pipeline parameter value.
In this section, you learn how to submit a pipeline run while setting pipeline parameters.
After submitting a pipeline with pipeline parameters, you can resubmit a pipeline run with different parameters:
-
Go to pipeline detail page. In the Pipeline run overview window, you can check current pipeline parameters and values.
-
Select Resubmit.
-
In the Setup pipeline run, specify your new pipeline parameters.
You can also publish a pipeline to use its pipeline parameters. A published pipeline is a pipeline that has been deployed to a compute resource, which client applications can invoke via a REST endpoint.
Published endpoints are especially useful for retraining and batch prediction scenarios. For more information, see How to retrain models in the designer or Run batch predictions in the designer.
In this article, you learned how to create pipeline parameters in the designer. Next, see how you can use pipeline parameters to retrain models or perform batch predictions.
You can also learn how to use pipelines programmatically with the SDK.