Azure Data Factory vs AWS Glue: Which Cloud ETL Service? (2026)

Quick verdict

Short answer

Choose Azure Data Factory when your destination is Azure (Synapse, Fabric, Azure SQL, Data Lake Storage), when you need to pull data from on-premises systems through a self-hosted integration runtime, when you want to lift existing SSIS packages to the cloud, or when most of your pipelines are copy-and-orchestrate work rather than heavy transformation. Choose AWS Glue when your data lake is on Amazon S3, when you want a shared Data Catalog that Athena, EMR and Redshift Spectrum also use, and when your team is comfortable writing PySpark or Scala. Note that Microsoft now tells new users to start with Data Factory in Microsoft Fabric, so check whether Fabric is the better Microsoft option for a new project.

How we know: This comparison is research-based: services, components, versions, connectors, integration runtimes and prices were checked against Microsoft Learn, the Azure Data Factory pricing page and Microsoft's Azure Retail Prices API, and against the AWS Glue Developer Guide and the AWS Glue pricing page, on 7 November 2026. We have not run pipelines on either service, so no performance or cost results are measured.

Azure Data Factory (ADF) is, in Microsoft's words, a managed cloud service built for complex hybrid ETL, ELT and data integration projects. You build pipelines of activities (data movement, data transformation and control flow), connect to stores through linked services and datasets, and run the work on an integration runtime: the Azure-hosted runtime, a managed virtual network runtime, a self-hosted runtime you install inside your own network, or an Azure-SSIS runtime for SQL Server Integration Services packages. Visual mapping data flows run on Spark clusters that ADF starts and stops for you. ADF has no version numbers; it is a continuously updated service. Microsoft's documentation now carries a banner calling Data Factory in Microsoft Fabric "the next generation of Azure Data Factory" and suggesting that new users start there.

AWS Glue is described by AWS as a serverless data integration service for discovering, preparing, moving and integrating data. Its core is a serverless Apache Spark engine for ETL jobs (batch and streaming), the Glue Data Catalog of databases and tables, crawlers that infer schemas into that catalog, the visual Glue Studio editor and interactive notebook sessions. AWS says Glue can connect to more than 70 data sources. Jobs pin a Glue version: AWS Glue 5.1 (Spark 3.5.6, Python 3.11) is the default for new jobs, and AWS Glue 6.0 (Spark 4.1.1, Python 3.13) is the newest listed. AWS Glue for Ray closed to new customers on 30 April 2026.

Same category, different centre of gravity. ADF is orchestration first: it moves data and calls other compute (its own data flows, Databricks, HDInsight, SQL). Glue is Spark first: you write or generate Spark code and Glue runs it serverlessly. For the concepts, see ETL vs ELT and ETL vs data pipeline.

Side by side

Aspect	Azure Data Factory	AWS Glue
Cloud	Microsoft Azure	Amazon Web Services
Core model	Pipelines of activities run on integration runtimes; orchestration plus copy plus visual data flows	Serverless Spark ETL jobs plus a central Data Catalog and crawlers
Transformation engine	Mapping data flows (visual, executed on managed Spark), or external activities on Databricks, HDInsight, SQL and others	Spark scripts in Python or Scala, generated in Glue Studio or written by hand; Python shell jobs for small tasks
Streaming	Batch and event-triggered pipelines; no streaming job type of its own	Spark Streaming job type for Kinesis and Kafka sources
Metadata catalog	No built-in table catalog; datasets describe structures per factory	Glue Data Catalog, shared with Athena, EMR and Redshift Spectrum
On-premises access	Self-hosted integration runtime installed inside your network	Connections attached to your VPC; on-premises sources need network routing into AWS
Connectors (vendor documented)	Long built-in list across Azure, databases, NoSQL, files, generic protocols and SaaS, several marked Preview	More than 70 sources per AWS; native connection types include JDBC databases, Kafka, Kinesis, DynamoDB, MongoDB, Snowflake, BigQuery, Azure SQL and Azure Cosmos DB
Legacy package support	Azure-SSIS integration runtime runs existing SSIS packages	No SSIS equivalent
Billing unit	Per 1,000 activity runs, per DIU-hour of data movement, per vCore-hour of data flows, plus small operations charges	Per DPU-hour (1 DPU = 4 vCPUs, 16 GB), billed per second with a 1-minute minimum; Flex class at a lower rate
Source control	Git integration and CI/CD with Azure DevOps and GitHub	Job scripts in Amazon S3; managed with the console, CLI, API or infrastructure as code
Product direction	Microsoft points new users to Data Factory in Microsoft Fabric	Actively versioned (5.1 default, 6.0 newest); Glue for Ray closed to new customers

Key differences

Orchestration service versus serverless Spark

The most useful way to see the two services is by what they are built around. ADF is a workflow engine. A pipeline chains activities with branching, loops and parameters; the Copy activity moves data between stores; and transformation is either a mapping data flow or a call out to another engine such as Azure Databricks, HDInsight or a SQL stored procedure. Many ADF estates contain little Spark at all, just scheduled copies and stored procedure calls.

Glue is an execution environment for Spark. A job is a PySpark or Scala script (Glue Studio can generate it from a visual canvas), you choose a worker type and count, and Glue starts the cluster, runs the script and bills per second. Orchestration exists (triggers, workflows, event-based starts), but it is lighter than ADF's control flow, and many AWS teams schedule Glue jobs from Step Functions or Apache Airflow instead. In our view, ADF suits teams whose work is mostly moving and orchestrating; Glue suits teams whose work is mostly transforming large files in a lake.

# AWS Glue (PySpark): read a catalogued table and write Parquet back to S3
from awsglue.context import GlueContext
from pyspark.context import SparkContext

glue = GlueContext(SparkContext.getOrCreate())
orders = glue.create_dynamic_frame.from_catalog(database="sales", table_name="raw_orders")
glue.write_dynamic_frame.from_options(
    frame=orders, connection_type="s3", format="parquet",
    connection_options={"path": "s3://example-lake/curated/orders/"})

Transformations: visual data flows or Spark code

ADF's mapping data flows let you design joins, aggregations, derived columns and lookups on a canvas; Microsoft says they execute on Spark "without needing to understand Spark clusters or Spark programming". The pricing page sets a minimum data flow cluster of 8 vCores, billed per minute and rounded up, which matters for many short runs. Teams that prefer to hand-code send the work to Databricks or another engine through external activities.

Glue's transformations are Spark itself, plus Glue extensions such as DynamicFrames, job bookmarks for incremental processing, FindMatches for fuzzy deduplication and sensitive data detection. Glue 5.x and 6.0 also bring open table format support (Iceberg, Hudi and Delta Lake versions are listed per Glue version). If your team already writes Spark, Glue removes cluster management without hiding the code; if your team does not, ADF's data flows are the gentler start.

Whatever the engine, most transformations end up as SQL somewhere. In Glue you can run Spark SQL inside a job:

-- Spark SQL inside a Glue job (after registering a temp view named orders)
SELECT customer_id, date_trunc('month', order_ts) AS order_month, SUM(amount) AS revenue
FROM orders
GROUP BY customer_id, date_trunc('month', order_ts);

Hybrid and on-premises connectivity

ADF's self-hosted integration runtime is a component you install on a machine inside your own network. It reaches on-premises SQL Server, Oracle, file shares and similar sources and moves data out, so the source does not need to be exposed to the internet. The Azure-SSIS integration runtime runs existing SSIS packages in Azure, which is often the deciding factor for SQL Server shops. Self-hosted runtime activity is priced separately (and lower) than Azure-hosted data movement.

Glue has no installable agent. Jobs reach databases through connections that place the job in your VPC subnets, so reaching an on-premises system means routing your network into that VPC first. Glue also lists cross-cloud connection types, including Azure SQL, Azure Cosmos DB for NoSQL and Google BigQuery. ADF, for its part, lists Amazon S3, Amazon Redshift and Google BigQuery among its connectors. Both can read from the other cloud; neither is the natural home for a pipeline whose destination is the other cloud.

Catalog and data lake integration

The Glue Data Catalog is one of the main reasons to use Glue. Crawlers infer schemas and partitions from files in S3 or tables in databases and register them as catalog tables, which AWS says you can query from Amazon Athena, Amazon EMR and Amazon Redshift Spectrum. AWS Lake Formation adds fine-grained permissions on top. Many AWS teams use the Data Catalog even when they do not run Glue ETL jobs at all.

ADF has no equivalent shared table catalog. Its datasets describe the structure of data for its own activities, and the Azure lakehouse story has moved to Microsoft Fabric and OneLake. If a central catalog for many engines is what you need on Azure, plan for it separately from ADF.

Pricing models and the Fabric question

ADF charges for several small meters: activity runs (per 1,000), data movement (per DIU-hour on Azure-hosted runtimes, per hour on self-hosted runtimes), pipeline and external activity execution hours, data flow vCore-hours, and read/write and monitoring operations. Costs therefore follow the number of runs and copies as much as data volume. Glue charges DPU-hours for jobs, crawlers and interactive sessions, so costs follow worker count times run time; the Data Catalog is free up to one million objects and one million requests a month.

On the Microsoft side there is also a strategic question. Microsoft calls Data Factory in Microsoft Fabric the next generation of ADF, bills it through Fabric capacity, and publishes an upgrade guide. At the same time, Microsoft's own feature comparison has listed Azure-SSIS runtimes and managed virtual networks with private endpoints as still to be determined in Fabric (see our ETL tools guide), so ADF remains the option for those needs. On AWS there is no equivalent shift: Glue keeps adding versions, and only the Ray engine has been closed to new customers.

Pricing and licensing

Azure Data Factory. The pricing page loads its numbers dynamically, so these East US list prices (USD, pay as you go) come from Microsoft's Azure Retail Prices API on 7 November 2026: orchestration USD 1.00 per 1,000 activity runs on the Azure integration runtime (USD 1.50 on a self-hosted runtime); data movement USD 0.25 per DIU-hour on the Azure runtime and USD 0.10 per hour on a self-hosted runtime; pipeline activity USD 0.005 per hour and external activity USD 0.00025 per hour on the Azure runtime; mapping data flows USD 0.27325 per vCore-hour (General Purpose) or USD 0.34275 (Memory Optimized), with a minimum cluster of 8 vCores, prorated by the minute and rounded up; read/write operations USD 0.50 per 50,000 entities and monitoring USD 0.25 per 50,000 run records; and USD 0.80 per inactive pipeline per month. Azure-SSIS runtimes are billed per node-hour, with lower rates when you bring your own SQL Server licence. As arithmetic on these prices, one General Purpose data flow on the 8-vCore minimum for 15 minutes is 8 x 0.25 x 0.27325 = about USD 0.55, before activity and movement charges. Prices vary by region.

AWS Glue. Listed on aws.amazon.com/glue/pricing on 7 November 2026: ETL jobs and interactive sessions USD 0.44 per DPU-hour, billed per second with a 1-minute minimum per run; the Flex execution class (for non-urgent jobs on G.1X or G.2X workers) at USD 0.29 per DPU-hour; crawlers at USD 0.44 per DPU-hour; Data Catalog storage free for the first million objects, then USD 1.00 per 100,000 objects a month, and the first million requests free; DataBrew jobs USD 0.48 per node-hour. AWS's own example is a Spark job using 6 DPUs for 15 minutes: 6 x 0.25 x 0.44 = USD 0.66. An interactive session defaults to 5 DPUs. AWS notes that pricing can vary by region.

The units are not directly comparable (an ADF vCore is not a Glue DPU, and ADF also charges per run), so model your own pipeline counts, durations and cluster sizes in each vendor's calculator.

Pricing checked on the vendors' official pages on 7 November 2026. Prices change; confirm before buying.

Where each one leads

Azure Data Factory strengths

Rich control flow (branching, loops, parameters, triggers) for orchestrating many steps and services
Self-hosted integration runtime for on-premises sources without exposing them to the internet
Azure-SSIS integration runtime runs existing SSIS packages in the cloud
Visual mapping data flows on managed Spark for teams that do not write Spark code
Git integration and CI/CD with Azure DevOps and GitHub

AWS Glue strengths

Serverless Spark with per-second billing and a choice of worker sizes, including memory-optimized workers
Glue Data Catalog shared with Athena, EMR and Redshift Spectrum
Crawlers that infer schemas and partitions from data in S3 and databases
Spark Streaming jobs for Kinesis and Kafka sources
Clear version line (Glue 5.1 default, Glue 6.0 with Spark 4.1.1) and a cheaper Flex class for non-urgent jobs

Limitations

Azure Data Factory limitations

Microsoft now steers new data integration work to Data Factory in Microsoft Fabric
Many small meters (runs, movement hours, operations) make costs harder to predict
Data flows have an 8-vCore minimum cluster, billed by the minute
No built-in shared table catalog and no streaming job type
Several SaaS connectors are still marked Preview and not recommended for production

AWS Glue limitations

Getting the most out of Glue assumes Spark skills in Python or Scala
No installable on-premises agent; private sources need network routing into your VPC
Orchestration is lighter than ADF's control flow, so many teams add Step Functions or Airflow
Glue for Ray is closed to new customers, and Glue 0.9, 1.0 and 2.0 reached end of life on 1 April 2026
Tied to AWS: the Data Catalog and job runtime have no equivalent outside AWS

When to choose each

Choose Azure Data Factory if

Your warehouse or lake is on Azure, such as Azure SQL, Synapse or Data Lake Storage
You must pull data from on-premises databases through an agent inside your network
You have SSIS packages to move to the cloud with minimal rework
Most pipelines are copy, orchestrate and call stored procedures rather than heavy Spark transformations
You need managed virtual networks with private endpoints that Fabric Data Factory does not yet cover

Choose AWS Glue if

Your data lake is on Amazon S3 and you want one catalog for Athena, EMR and Redshift Spectrum
Your team writes PySpark or Scala and wants Spark without managing clusters
You need streaming ETL from Kinesis or Kafka in the same service as batch
You want per-second billing for bursty, short jobs, or a cheaper Flex rate for non-urgent loads
You use open table formats such as Iceberg on S3

When neither is right

You are starting fresh on Microsoft's platform: evaluate Data Factory in Microsoft Fabric first, and see Databricks vs Microsoft Fabric.
You mostly need SaaS and database connectors maintained for you across clouds: see Airbyte vs Fivetran.
You need a cloud-neutral orchestrator for jobs on several platforms: see Apache Airflow vs Airbyte and Apache Airflow vs Dagster.
You need low-latency streaming flows with routing and delivery guarantees: see Apache NiFi vs Airbyte and Kafka vs RabbitMQ.

Final recommendation

Bottom line

For most teams the cloud decides: Azure Data Factory on Azure, AWS Glue on AWS. Beyond that, ADF is the better fit when the job is orchestration, hybrid copying from on-premises systems or running SSIS packages, and when people prefer visual design to Spark code. Glue is the better fit when the job is transforming large data in an S3 lake with Spark, and when a shared Data Catalog for several query engines matters. Microsoft users starting something new should also price Data Factory in Microsoft Fabric before committing to ADF.

Frequently asked questions

Is Azure Data Factory being replaced by Fabric?

Microsoft's ADF documentation calls Data Factory in Microsoft Fabric "the next generation of Azure Data Factory", suggests that new users start with Fabric, and links to an upgrade guide for existing ADF workloads. ADF is still sold and documented, and some features, such as Azure-SSIS integration runtimes, have no confirmed Fabric equivalent yet.

What is a DPU in AWS Glue?

A Data Processing Unit is AWS Glue's billing and capacity unit: 4 vCPUs and 16 GB of memory. A G.1X worker is 1 DPU and a G.2X worker is 2 DPUs. Jobs are billed per DPU-hour, per second, with a 1-minute minimum per run; AWS lists USD 0.44 per DPU-hour (USD 0.29 on the Flex class).

Can AWS Glue connect to Azure, or ADF to AWS?

Yes, in both directions. AWS documents Glue connection types for Azure SQL, Azure Cosmos DB for NoSQL and Google BigQuery. Microsoft lists Amazon S3, Amazon Redshift and Google BigQuery among ADF connectors. Data transfer charges from the source cloud still apply.

Does Azure Data Factory use Spark?

Mapping data flows do: Microsoft says ADF runs the data flow logic on a Spark cluster that it starts and stops as needed, so you never manage the cluster. Copy activities and control flow do not use Spark.

Which AWS Glue version should I use?

AWS Glue 5.1 (Spark 3.5.6, Python 3.11) is the default for jobs created without a version, and AWS Glue 6.0 (Spark 4.1.1, Python 3.13) is the newest listed. Glue 0.9, 1.0 and 2.0 reached end of life on 1 April 2026, so move older jobs to a current version.

Sources

Checked November 2026.

How we research comparisons: our editorial method.