Skip to content

can't deploy "For each" task #887

@sstrong99

Description

@sstrong99

When deploying a for_each_task, dbx applies the whl library to the outer task rather than the inner task, which causes the databricks API. See the following query payload created by dbx:

{
    "job_clusters": [
        ...
    ],
    "access_control_list": [
        ...
    ],
    "name": "Test For Each",
    "tasks": [
        {
            "task_key": "for_each",
            "libraries": [
                {
                    "whl": "dbfs:/dbx/...whl"
                }
            ],
            "for_each_task": {
                "inputs": [1, 2, 3],
                "concurrency": 100,
                "task": {
                    "task_key": "build_one",
                    "job_cluster_key": "default",
                    "python_wheel_task": {
                        "package_name": "fake",
                        "entry_point": "build-one",
                        "named_parameters": {
                            "input": "{{input}}"
                        }
                    }
                }
            }
        }
    ],
    "workflow_type": "job-v2.1"
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions