When trying to start a plugin, it sometimes ends up in a broken state and the only way to unblock is to restart the host.
Main function trying to start the plugin
t.client[pluginName] = goPlugin.NewClient(&goPlugin.ClientConfig{
HandshakeConfig: handshakeConfig,
Plugins: pluginMap,
Cmd: exec.Command(pluginPath, args...),
Managed: true,
})
rpcClient, err := t.client[pluginName].Client()
if err != nil {
// getting log: waiting for RPC address
// then error: timeout while waiting for plugin to start
return nil, fmt.Errorf("unable to get plugin client (%s): %w", pluginName, err)
}Function trying to get the plugin if it exist, and somewhat recover on error
client, err := t.client[pluginName].Client()
if err != nil {
// Getting error: exec: Stdout already set
// cannot run kill() because it requires a valid client
return nil, fmt.Errorf("unable to get plugin client (%s) for ping: %w", pluginName, err)
}
if err := client.Ping(); err != nil {
t.client[pluginName].Kill()
t.client[pluginName] = nil
return nil, fmt.Errorf("could not ping plugin will cleanup process so we can restart it next reconcile (%w)", err)
}Everytime we create a new client (restart the program), there is a new process that is created in the following state
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
agaudreault 99885 0.0 0.0 410089056 48 ?? UE 1:58pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 98102 0.0 0.0 410089056 48 ?? UE 1:56pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 96380 0.0 0.0 410089056 48 ?? UE 1:55pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 94682 0.0 0.0 410089056 48 ?? UE 1:54pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 92807 0.0 0.0 410089056 48 ?? UE 1:52pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 91001 0.0 0.0 410089056 48 ?? UE 1:51pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 89169 0.0 0.0 410089056 48 ?? UE 1:50pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
agaudreault 87411 0.0 0.0 410089056 48 ?? UE 1:48pm 0:00.00 /Users/agaudreault/plugin-bin/my-plugin
The process is running locally on Mac
System Software Overview:
System Version: macOS 14.4.1 (23E224)
Kernel Version: Darwin 23.4.0
Chip: Apple M1 Max
Running on version
github.com/hashicorp/go-plugin v1.6.0
When trying to start a plugin, it sometimes ends up in a broken state and the only way to unblock is to restart the host.
Main function trying to start the plugin
Function trying to get the plugin if it exist, and somewhat recover on error
Everytime we create a new client (restart the program), there is a new process that is created in the following state
The process is running locally on Mac
Running on version