Skip to content

Latest commit

 

History

History
194 lines (136 loc) · 6.54 KB

File metadata and controls

194 lines (136 loc) · 6.54 KB

Documentation Index

Welcome to the azure.datafactory.tools documentation. This guide covers everything from installation to advanced deployment scenarios.

🚀 Getting Started

New to the module? Start here:

📖 Core Guides

Learn how the deployment process works step-by-step, from loading files to restarting triggers.

Control which objects get deployed using includes/excludes patterns and templates.

Replace environment-specific values (connection strings, URLs, etc.) using CSV or JSON config files.

Advanced deployment scenarios with complex trigger management and safety considerations.

Speed up CI/CD by deploying only changed objects (60-90% faster).

🔧 Features

Validate ADF code before deployment with Test-AdfCode.

Auto-generate Mermaid diagrams showing object relationships and dependencies.

Automate testing of linked service connectivity with Test-AdfLinkedServiceConnection.

Convert ADF code to ARM templates for infrastructure-as-code deployments.

💻 Advanced Topics

Build multi-stage CI/CD pipelines with approval gates and environment promotions.

Complete reference of all PowerShell cmdlets and configuration options.


Quick Navigation by Task

"I want to..."

Goal Documentation
Deploy ADF for the first time Getting StartedPublishing Workflow
Deploy only specific objects Publish Options
Use different values per environment Configuration & Stages
Speed up deployments Incremental Deployment
Validate code before deploy Build & Test
Understand what will deploy Generate Dependencies
Test connections automatically Test Linked Services
Automate in Azure DevOps DevOps Integration
View all functions/options Cmdlet Reference

Common Workflows

Scenario 1: Deploy to Production

Import-Module azure.datafactory.tools

# 1. Validate code
$errors = Test-AdfCode -RootFolder 'c:\MyADF'
if ($errors -gt 0) { exit 1 }

# 2. Deploy with prod configuration
Publish-AdfV2FromJson `
    -RootFolder 'c:\MyADF' `
    -ResourceGroupName 'rg-prod' `
    -DataFactoryName 'adf-prod' `
    -Location 'NorthEurope' `
    -Stage 'PROD'

See: Configuration & Stages

Scenario 2: Deploy Only Pipelines

$opt = New-AdfPublishOption
$opt.Includes.Add('pipeline.*', '')  # Include only pipelines
$opt.DeleteNotInSource = $false

Publish-AdfV2FromJson -Option $opt ...

See: Publish Options

Scenario 3: Speed Up With Incremental Deployment

$opt = New-AdfPublishOption
$opt.IncrementalDeployment = $true
$opt.IncrementalDeploymentStorageUri = 'https://storage.../path'

Publish-AdfV2FromJson -Option $opt ...

See: Incremental Deployment

Scenario 4: CI/CD Pipeline

# See: Azure DevOps Integration
stages:
  - stage: Validate
  - stage: DeployDev
  - stage: DeployProd
    environment: Production  # Approval gate

See: Azure DevOps Integration


Folder Structure

docs/
  README.md                               # This file
  GETTING_STARTED.md                     # Installation & quick start
  GUIDE/
    PUBLISHING.md                         # Core workflow
    PUBLISH_OPTIONS.md                    # Filtering & includes/excludes
    CONFIGURATION.md                      # Environment values, stages, config files
    SELECTIVE_DEPLOYMENT.md               # Advanced trigger logic, safety
    INCREMENTAL_DEPLOYMENT.md             # Change detection, speed optimization
  FEATURES/
    BUILD_AND_TEST.md                     # Code validation
    DEPENDENCIES_DIAGRAM.md               # Mermaid diagram generation
    LINKED_SERVICE_TESTING.md             # Connection testing
    ARM_TEMPLATE.md                       # ARM template export/deploy
  ADVANCED/
    DEVOPS_INTEGRATION.md                 # CI/CD pipelines, Azure DevOps
    API_REFERENCE.md                      # Complete cmdlet reference

Support & Resources


Key Concepts

Objects & Types

ADF contains different object types: pipelines, datasets, linked services, triggers, data flows, integration runtimes, etc.

Deployment Order

The module automatically determines the correct deployment order based on object dependencies.

Configuration Files

Use CSV or JSON files to replace environment-specific values without modifying source code.

Publish Options

Control deployment behavior: what gets deployed, whether triggers are managed, whether objects are deleted.

Incremental Mode

Speeds up deployments by tracking object hashes and deploying only changes.


Need Help?


Last Updated: March 2026
Version: 1.0 (Restructured Documentation)