Failed to spawn new process - UnderlyingError(24)

[btoms20]

Describe the bug

After running many small processes (one after another, not concurrently) subprocesses throws the following error and hangs until manually killed.

Failed to spawn the new process with underlying error: UnderlyingError(rawValue: 24)

To Reproduce
Steps to reproduce the behavior:

Call the run method exposed by Subprocess a few hundred (maybe thousands of) times and it'll eventually throw the error.
I'm specifically calling the run method within an In-Process Vapor Queue / Job once a minute to monitor a running service

Expected behavior

For the subprocess command to not fail and the program to not hang

Environment (please complete the following information):

• OS:

Linux version 6.12.34+rpt-rpi-v8 (aarch64-linux-gnu-gcc-12 (Debian 12.2.0-14+deb12u1) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40)

• Swift version:

Swift version 6.1.2 (swift-6.1.2-RELEASE)
Target: aarch64-unknown-linux-gnu

and

Swift version 6.1.3 (swift-6.1.3-RELEASE)
Target: aarch64-unknown-linux-gnu

Additional context

I did some googling and I think error 24 might be related to exceeding the maximum number of open file descriptors in linux. Is it possible subprocess isn't properly closing the process?

Note

All of the tests pass except for SubprocessUnixTests.testConcurrentRun(). This test results in a hang every time it's ran. I think we hit the 1024 file descriptor limit (on linux as noted in the comments) and instead of the thrown error (.spawnError) canceling the task, it hangs indefinitely instead.

Tested against both 0.1.0 and the main branch

[btoms20]

I added a couple of tests to try and hone in on the problem and I'm pretty sure it is related to not properly closing file descriptors on linux.

This test fails with the same UnderlyingError(rawValue: 24)

@Test("Execute an `ls` command many times using swift-subprocess")
func execLSCommandMultiple() async throws {
    for _ in 0 ..< 2048 {
        let res = try await run(.name("ls"), arguments: ["-a"], output: .string(limit: .max, encoding: UTF8.self))
        
        #expect(res.terminationStatus == .exited(0))
        #expect(res.standardOutput != nil)
    }
}

This test passes without issue

@Test("Execute an `ls` command many times using Process")
func execLSCommandMultipleUsingProcess() async throws {
    for _ in 0 ..< 2048 {
        let task = Process()
        let pipe = Pipe()
        
        task.standardOutput = pipe
        task.standardError = pipe
        task.arguments = ["-c", "ls"] + ["-a"]
        task.executableURL = URL(fileURLWithPath: "/usr/bin/sh")
        task.standardInput = nil
        
        try task.run()
        
        guard let data = try pipe.fileHandleForReading.readToEnd() else {
            try pipe.fileHandleForReading.close()
            return
        }

        let output = String(data: data, encoding: .utf8)!
        try pipe.fileHandleForReading.close()

        #expect(output.isEmpty == false)
    }
}

[iCharlesHu]

This one was fixed and regression testing was added in #309