| title | titleSuffix | description | ms.author | author | ms.service | ms.subservice | ms.topic | ms.custom | ms.date |
|---|---|---|---|---|---|---|---|---|---|
Memory optimized compute type for Data Flows |
Azure Data Factory & Azure Synapse |
Learn about the memory optimized compute type setting in Azure Data Factory and Azure Synapse. |
jburchel |
jonburchel |
data-factory |
data-flows |
conceptual |
synapse |
11/12/2021 |
[!INCLUDEappliesto-adf-asa-md]
Data flow activities in Azure Data Factory and Azure Synapse support the Compute type setting to help optimize the cluster configuration for cost and performance of the workload. The default selection for the setting is General and will be sufficient for most data flow workloads. General purpose clusters typically provide the best balance of performance and cost. However, the Memory optimized setting can significantly improve performance in some scenarios by maximizing the memory available per core for the cluster.
If your data flow has many joins and lookups, you may want to use a memory optimized cluster. These more memory intensive operations will benefit particularly by additional memory, and any out-of-memory errors encountered with the default compute type will be minimized. Memory optimized clusters do incur the highest cost per core, but may avoid pipeline failures for memory intensive operations. If you experience any out of memory errors when executing data flows, switch to a memory optimized Azure IR configuration.