Ensure Nuke host get installed in terminal mode.

[rdelillo]

Changelog Description

Fixes: #188
Changes:

• Ensure Nuke host get installed in non-GUI mode
• Rework the PySide related import to ensure our python code is compatible in headless mode

Testing notes:

1. Ensure Nuke works as before when launched through AYON
2. Run this python script to ensure Nuke host is installed in terminal mode

import subprocess
import tempfile

from ayon_applications import ApplicationManager, LaunchTypes


def _get_python_script() -> str:
    with tempfile.NamedTemporaryFile(suffix=".py", delete=False, mode="w") as _file:
        _file.write(
            """
from ayon_core.pipeline import get_current_host_name

host_name = get_current_host_name()
print("current host name: %s" % host_name)
            """
        )
        return _file.name


def _run_nuke_terminal_mode(python_script: str):
    app_manager = ApplicationManager()
    app_group = app_manager.app_groups["nuke"]
    app = app_manager.find_latest_available_variant_for_group(
        app_group.name
    )
    app_args = ["-t", python_script]
    context_kwargs = {
        "project_name": "Dummy",  # TODO: change this
        "folder_path": "/all_shots/edit_sequence/sh010",   # TODO: change this
        "task_name": "Compositing",   # TODO: change this
        "task_type": "Compositing",   # TODO: change this
        "app_args": app_args or [],
        "launch_type": LaunchTypes.automated,
    }

    launch_context = app_manager.create_launch_context(
        app.full_name,
        **context_kwargs,
    )
    launch_context.run_prelaunch_hooks()
    launch_args = launch_context.launch_args
    kwargs = launch_context.kwargs
    kwargs.update({
        "stdout": subprocess.PIPE,
        "stderr": subprocess.STDOUT,
        "text": True,
        "encoding": "utf-8",
        "errors": "replace",
    })

    process = subprocess.Popen(launch_args, **kwargs)
    process.wait()
    out = process.stdout.read()

    print(out)
    assert "current host name: nuke" in out, "Nuke host is not initialized."


# Run Nuke in terminal mode.
pscript_path = _get_python_script()
_run_nuke_terminal_mode(pscript_path)

[BigRoy]

Overall the changes look good.

But @LiborBatek (and @rdelillo or @jakubjezek001) can you please test this on the farm rendering. The way I understand this will now trigger the installation of AYON on the farm I suspect. And with that, it'll now possibly trigger any of the registered callbacks or startup logic - and we'll want to ensure that e.g. none of the "check inventory versions" or "on_script_load" callbacks get in the way and suddenly start altering the scene.

E.g. set_context_settings may now trigger and actually end up triggering reset_resolution or reset_frame_range_handles etc.

With that now all being present this may affect nuke render speeds on the farm slightly because it may now start performing more AYON logic in the headless modes?

[moonyuet]

I tested in farm rendering but I hit some monitored process error for _baking.mov job

Job_2026-03-03_15-38-46_69a690064e744912f2c43577.txt

[rdelillo]

Overall the changes look good.

But @LiborBatek (and @rdelillo or @jakubjezek001) can you please test this on the farm rendering. The way I understand this will now trigger the installation of AYON on the farm I suspect. And with that, it'll now possibly trigger any of the registered callbacks or startup logic - and we'll want to ensure that e.g. none of the "check inventory versions" or "on_script_load" callbacks get in the way and suddenly start altering the scene.

E.g. set_context_settings may now trigger and actually end up triggering reset_resolution or reset_frame_range_handles etc.

With that now all being present this may affect nuke render speeds on the farm slightly because it may now start performing more AYON logic in the headless modes?

I looked into it some more today total cost for all callbacks is about 4-5 seconds maximum.
Still I've came up with those improvements:

• split our Nuke callbacks between none-GUI (installed by init.py) and GUI ones (installed by menu.py)
• allow Nuke host register to fail, log the message as a warning but do not prevent Nuke from starting in none-GUI mode.
  This one is for backward compatibility: some renderfarms might have been setup wrongly but still functional as AYON nuke host is not mandatory for some renders.

I've tested those changes locally and in both Deadline and RoyalRender.

[BigRoy]

I still have my concerns about the callbacks triggering in e.g. terminal mode and likely on renders as well.

See e.g.

Launching with file

C:\projects\test\apps\nuke\work\comp>Nuke16.0 -t "C:\projects\test\apps\nuke\work\comp\tes_nuke_comp_v028.nk"
Nuke 16.0v4, 64 bit, built Jun 13 2025.
Copyright (c) 2025 The Foundry Visionmongers Ltd.  All Rights Reserved.
Licence expires on: 2026/7/23

>>> [ Added Nuke callbacks ... ]
>>> [ Resetting resolution ]
>>> [ Format is set. ]
>>> [ Setting colorspace to workfile... ]
>>> [ Colorspace management is disabled globally. ]
>>> [ Setting colorspace to viewers... ]
>>> [ Setting colorspace to write nodes... ]
>>> [ Setting colorspace to read nodes... ]
>>> [ Resetting resolution ]
>>> [ Format is set. ]
>>> [ Setting colorspace to workfile... ]
>>> [ Colorspace management is disabled globally. ]
>>> [ Setting colorspace to viewers... ]
>>> [ Setting colorspace to write nodes... ]
>>> [ Setting colorspace to read nodes... ]

Launching without file

C:\projects\test\apps\nuke\work\comp>Nuke16.0 -t
Nuke 16.0v4, 64 bit, built Jun 13 2025.
Copyright (c) 2025 The Foundry Visionmongers Ltd.  All Rights Reserved.
Licence expires on: 2026/7/23

>>> [ Added Nuke callbacks ... ]
>>> [ Resetting resolution ]
>>> [ Creating new format: 1920 1080 1.00 test ]
>>> [ Format is set. ]
>>> [ Setting colorspace to workfile... ]
>>> [ Colorspace management is disabled globally. ]
>>> [ Setting colorspace to viewers... ]
[22:05.40] ERROR: Bad value for monitorOutDevice : 0
[22:05.40] ERROR: Bad value for monitorOutDisplayMode : 4294967295
[22:05.40] ERROR: Bad value for monitorOutPixelFormat : 4294967295
*** WRN: >>> { ayon_nuke.api.lib }: [ Attention! Viewer nodes ['Viewer1'] were erased. It had wrong color profile ]
>>> [ Setting colorspace to write nodes... ]
>>> [ Setting colorspace to read nodes... ]

Notice how launching with file does a lot of the logic twice, which hints to me that the second run of logic runs inside the opened workfile. And if the workers now start opening the scenes and start adjusting format, colorspaces, etc. then suddenly it'd be changing the workfile on the farm before it initiates the render. That sounds like a no-go really. It shouldn't be touching the scene bits at all?

[rdelillo]

Notice how launching with file does a lot of the logic twice, which hints to me that the second run of logic runs inside the opened workfile. And if the workers now start opening the scenes and start adjusting format, colorspaces, etc. then suddenly it'd be changing the workfile on the farm before it initiates the render. That sounds like a no-go really. It shouldn't be touching the scene bits at all?

@BigRoy just to be sure I understand you correctly, are you suggesting we should just drop this and not have any of the callbacks called by default in terminal mode ?

The initial issue was flagged here: https://github.com/ynput/ayon-sn4/pull/57
This was when Nuke templates on the farm didn't resolve the OCIO file properly hence colorspace issues.

ayon-nuke/client/ayon_nuke/api/lib.py

Line 1595 in 2d5b203

def set_root_colorspace(self, imageio_host):

And the fix was to ensure we got colorspace related callbacks called.
But I get your point, so maybe we should treat it as a special case and enable callbacks through an environment variable or something ?

[BigRoy]

@BigRoy just to be sure I understand you correctly, are you suggesting we should just drop this and not have any of the callbacks called by default in terminal mode ?

The initial issue was flagged here: ynput/ayon-sn4#57 This was when Nuke templates on the farm didn't resolve the OCIO file properly hence colorspace issues.

I'd say there's a difference between setting up Nuke for AYON, and changing a workfile that you open.
We should set any global state of Nuke and for new workfiles as we usually should. But admittedly, I'm not keen on having workfiles being altered as you open them without any user intervention.

What we should have is that:

• a new scene gets initialized according to AYON task context
• opening a workfile opens the scene as is (of course global OCIO would still be set, etc because that is an env var?) but I feel we shouldn't really be messing with the scene state itself.
• (optionally) we can log if state is differing from what we expect for the context, and in UI mode prompt the user whether they want to fix it. kind of like the intent of this PR: Added setting for disabling forcing context settings #193

The question regarding "Otherwise OCIO environment might not be properly intialized." is. Why wouldn't it? Isnt' that all OCIO env var?

[jakubjezek001]

The question regarding "Otherwise OCIO environment might not be properly intialized." is. Why wouldn't it? Isnt' that all OCIO env var?

Yes one thing is to set OCIO via callback, but then another callback is setting up correct settings for colormanagement set in workfile (ayon+settings://nuke/imageio/workfile) settings. Because the default nuke settings on colorspace might not be exactly the same as a project settings.

[jakubjezek001]

works well

[BigRoy]

Yes one thing is to set OCIO via callback, but then another callback is setting up correct settings for colormanagement set in workfile (ayon+settings://nuke/imageio/workfile) settings. Because the default nuke settings on colorspace might not be exactly the same as a project settings.

We're not discussing the 'defaults' of Nuke. I'm fine with adjusting Nuke's defaults - we just shouldn't automatically change an opened workfile. That'd mean you couldn't open files without them being 'broken' (converted) without your knowledge. Especially on the farm that's not something we'd want.

[BigRoy]

Let's default now to setting up none of the scene altering callbacks in headless mode - that should match the previous behavior where we did nothing on the farm. Then make an issue to track this remainder discussion. That way we can at least patch up the terminal side of things here.

[rdelillo]

Let's default now to setting up none of the scene altering callbacks in headless mode - that should match the previous behavior where we did nothing on the farm. Then make an issue to track this remainder discussion. That way we can at least patch up the terminal side of things here.

I've rework this PR so it only setup callbacks in GUI mode.
I've also create this issue to track further discussions: #211

[BigRoy]

Looks good to me - @jakubjezek001 can you confirm and merge if ok please?

[jakubjezek001]

This is working but without the callback my batchdelivery workfow is failing.

[rdelillo]

This is working but without the callback my batchdelivery workfow is failing.

Just to be sure, this is not new right ? I should fail the same way with current develop branch, am I right ?

[jakubjezek001]

Just to be sure, this is not new right ? I should fail the same way with current develop branch, am I right ?

at the end all is fine. I will need to fix some issues in batch delivery, but this is unrelated.