Failure to import TransC_Sampler under Windows

[Cuttlefish-Joe]

Hello pyTransC team,

I've been attempting to run the Gaussian examples from this project, however the TransC_Sampler does not import for me in any of these documents. The installation of pyTransC did not raise any issues itself, but importing TransC_Sampler from pyTransC raises the error: "ValueError: cannot find context for 'fork'":

Any ideas for troubleshooting this?

[JuergHauser]

This could be related to the multiprocessing module and how it behaves on different operating systems. Are you able to run the following test program in your python environment?

import multiprocessing
if __name__ == '__main__':
    multiprocessing.set_start_method("fork")
    def worker():
        print("Worker process")
    p = multiprocessing.Process(target=worker)
    p.start()
    p.join()

This should work on Linux and Mac OSX but I would expect it to fail on Windows. From the file path in your error message I get the impression you might be on Windows. If that is the case can you try to run the following test program and let us know if it works.

import multiprocessing
if __name__ == '__main__':
    multiprocessing.set_start_method("spawn")
    def worker():
        print("Worker process")
    p = multiprocessing.Process(target=worker)
    p.start()
    p.join()

This would allow us to understand if we need to update pyTransC so that it runs on Windows...

[Cuttlefish-Joe]

Yep, I'm running windows.

The first code block fails, the second one succeeds (though it throws a RuntimeError if it is run multiple times).

Talking to Malcolm, he suggested it might be due to the version of python I'm using (3.11.0) - hopefully we can work this out this afternoon!

[JuergHauser]

The problem is described here. I think pyTransC would need to be updated for Windows as it uses fork to setup multiple processes. It might be possible to use the Linux subsystem on Windows which may allow to use fork. https://learn.microsoft.com/en-us/windows/wsl/

[msambridge]

I added

if name == 'main':
multiprocessing.set_start_method("spawn")
def worker():
print("Worker process")
p = multiprocessing.Process(target=worker)
p.start()
p.join()

in the pyTransc.py and it ran OK on both windows and MacOSX. Should we just use this as default?

[auggiemarignier]

What was the rationale for setting the start method to "fork" in the first place? The default on both windows and mac is "spawn"

[JuergHauser]

We have created a branch that uses spawn explicitly, which might address the issue encountered on Windows. @Cuttlefish-Joe you can install it directly from this repository by running.

pip install git+https://github.com/inlab-geo/pyTransC@spawn

[msambridge]

@auggiemarignier I used fork originally because that was what was in the emcee docs example.

[auggiemarignier]

I think it's best if we just use the default, let the OS decide the appropriate startup method

[JuergHauser]

Well according to this spawn will be the default in Python 3.14 so I think there is no harm in explicitly setting it on POSIX where it currently isn't the default. So it should only affect POSIX excluding MacOS X i.e. Linux...

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

[JuergHauser]

Can someone on windows test the current version? Our latest commit should work on windows as we have changed the approach around achieving parallelism...

1da0151