WARN Exec socket appeared but heartbeat failed, exec will not be available

[colorfulshark]

Hi guys,

I would like to use this project as a light weight sandbox, but had some trouble running the example.

I installed the binaries with

cp a3s* lib /usr/local/bin
ln -s /usr/local/bin/lib/libkrunfw.so* /usr/local/lib/

And when I try to launch the very start box, It outputted some error log

root@ai45lab:~# a3s-box run --name hello alpine:latest -- echo "Hello from MicroVM"
Creating box hello (a143eb40fcda)...
2026-04-21T07:34:20.157417Z  WARN Exec socket appeared but heartbeat failed, exec will not be available
Box hello (a143eb40fcda) started. Press Ctrl-C to stop.
init.krun: entered main argc=1
init.krun: mount_filesystems ok
init.krun: after cmdline env import KRUN_INIT=/sbin/init KRUN_INIT_PID1=<null> BOX_EXEC_EXEC=echo
init.krun: root propagation ok
init.krun: tty/session configured
init.krun: config parsed
init.krun: selected exec=/sbin/init init_pid1=0
init.krun: setup_redirects ok
init.krun: execvp(/sbin/init) starting
init.krun: execvp(/sbin/init) failed errno=2
Couldn't execute '/sbin/init' inside the vm: No such file or directory

Stopping box hello...
Box hello stopped.

I also tried running a box with interactive shell, still got errors

root@ai45lab:~# a3s-box run -it --name dev alpine:latest -- /bin/sh
Creating box dev (3a4989f02274)...
2026-04-21T07:47:58.636724Z  WARN Exec socket appeared but heartbeat failed, exec will not be available
Error: Exec error: Failed to connect to PTY server at /root/.a3s/boxes/3a4989f0-2274-4ef8-bef1-26a645a81af2/sockets/pty.sock: Connection refused (os error 111)

Appreciate for your help.

[ZhiXiao-Lin]

Hi, thanks for reporting this issue. To help diagnose the root cause, could you please provide the following information:

System Information

1. Machine type — What CPU architecture? (e.g., x86_64, aarch64)
2. Host OS — What Linux distribution and version are you running on? (e.g., Ubuntu 22.04, Debian 12)
3. a3s-box version — What version of a3s-box are you using?

   a3s-box --version

Installation Details

4. How did you install a3s-box? (from source, binary release, homebrew, etc.)

5. **What's the full contents of your /usr/local/bin and /usr/local/lib directories?

   ls -la /usr/local/bin/a3s* /usr/local/lib/libkrun* 2>/dev/null

6. **Is the guest init binary present?

   find /usr/local/bin -name \"*guest-init*\" 2>/dev/null
   # Also check in lib dir
   find /usr/local/lib -name \"*guest-init*\" 2>/dev/null

Debugging

7. Can you run with verbose debug logging? Try:

   RUST_LOG=debug a3s-box run --name hello alpine:latest -- echo \"Hello\"

8. Check if the rootfs was built correctly — look for the box directory that was created:

   ls -la ~/.a3s/boxes/*/rootfs/sbin/init 2>/dev/null
   file ~/.a3s/boxes/*/rootfs/sbin/init 2>/dev/null

9. What does `strace` show? If available:

   strace -f a3s-box run --name hello alpine:latest -- echo \"Hello\" 2>&1 | grep -E \"(execve|sbin/init|BOX_EXEC)\" | head -30

This will help determine whether:

• The guest init binary is missing from your installation
• The guest init is not being installed into the rootfs
• The `BOX_EXEC_EXEC` environment variable is not being passed correctly to the VM

[colorfulshark]

Hi, thanks for reporting this issue. To help diagnose the root cause, could you please provide the following information:

System Information

1. Machine type — What CPU architecture? (e.g., x86_64, aarch64)
2. Host OS — What Linux distribution and version are you running on? (e.g., Ubuntu 22.04, Debian 12)
3. a3s-box version — What version of a3s-box are you using?
   a3s-box --version

CPU: AMD Ryzen H255
Host OS: Debian 13
a3s-box version: 2.0.0

Installation Details

4. How did you install a3s-box? (from source, binary release, homebrew, etc.)
   Download binary from github release and install manually.
5. What's the full contents of your /usr/local/bin and /usr/local/lib directories?
   ls -la /usr/local/bin/a3s /usr/local/lib/libkrun 2>/dev/null

root@ai45lab:~#  ls -la /usr/local/bin/a3s* /usr/local/lib/libkrun* 2>/dev/null
-rwxr-xr-x 1 root root 23345633 Apr 21 07:32 /usr/local/bin/a3s-box
-rwxr-xr-x 1 root root 15669697 Apr 21 07:32 /usr/local/bin/a3s-box-cri
-rwxr-xr-x 1 root root  5165632 Apr 21 07:32 /usr/local/bin/a3s-box-guest-init
-rwxr-xr-x 1 root root  4457417 Apr 21 07:32 /usr/local/bin/a3s-box-shim
-rwxr-xr-x 1 root root  5538225 Apr 21 06:35 /usr/local/lib/libkrun.so
-rwxr-xr-x 1 root root  5538225 Apr 21 06:35 /usr/local/lib/libkrun.so.1
-rwxr-xr-x 1 root root  5538225 Apr 21 06:35 /usr/local/lib/libkrun.so.1.17.0
lrwxrwxrwx 1 root root       31 Apr 21 07:33 /usr/local/lib/libkrunfw.so -> /usr/local/bin/lib/libkrunfw.so
lrwxrwxrwx 1 root root       33 Apr 21 07:33 /usr/local/lib/libkrunfw.so.5 -> /usr/local/bin/lib/libkrunfw.so.5

6. **Is the guest init binary present?
   find /usr/local/bin -name "guest-init" 2>/dev/null

root@ai45lab:~# find /usr/local/bin -name "*guest-init*" 2>/dev/null
/usr/local/bin/a3s-box-guest-init

Also check in lib dir

find /usr/local/lib -name "guest-init" 2>/dev/null

root@ai45lab:~# find /usr/local/bin -name "*guest-init*" 2>/dev/null
/usr/local/bin/a3s-box-guest-init

root@ai45lab:~# find /usr/local/lib -name "guest-init" 2>/dev/null
root@ai45lab:~# 

Debugging

7. Can you run with verbose debug logging? Try:
   RUST_LOG=debug a3s-box run --name hello alpine:latest -- echo "Hello"

Too many logs but end with
[2026-04-21T09:28:55Z DEBUG vmm] using vmm exit code: 127
[2026-04-21T09:28:55Z INFO  vmm] Vmm is stopping.
2026-04-21T09:28:56.105809Z DEBUG vm_boot{box_id=781ec7fe-754c-46eb-ace0-6cb62781777f}:wait_for_ready: VM process is running
2026-04-21T09:28:56.105851Z DEBUG vm_boot{box_id=781ec7fe-754c-46eb-ace0-6cb62781777f}:wait_for_ready: Waiting for exec server socket socket_path=/root/.a3s/boxes/781ec7fe-754c-46eb-ace0-6cb62781777f/sockets/exec.sock
2026-04-21T09:28:56.105887Z DEBUG vm_boot{box_id=781ec7fe-754c-46eb-ace0-6cb62781777f}:wait_for_ready: Exec socket file detected
2026-04-21T09:28:56.105963Z DEBUG vm_boot{box_id=781ec7fe-754c-46eb-ace0-6cb62781777f}:wait_for_ready: Exec connect failed, retrying error=Exec error: Failed to connect to exec server at /root/.a3s/boxes/781ec7fe-754c-46eb-ace0-6cb62781777f/sockets/exec.sock: Connection refused (os error 111)

8. Check if the rootfs was built correctly — look for the box directory that was created:
   ls -la ~/.a3s/boxes/*/rootfs/sbin/init 2>/dev/null

root@ai45lab:~# ls -la ~/.a3s/boxes/*/rootfs/sbin/init 2>/dev/null
-rwxr-xr-x 1 root root 5338136 Apr 21 06:26 /root/.a3s/boxes/02245875-3cfe-481e-930c-d9c6b7a8ac61/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 06:35 /root/.a3s/boxes/157b339a-fac1-42b7-b136-445e1ac08ea2/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 07:47 /root/.a3s/boxes/3a4989f0-2274-4ef8-bef1-26a645a81af2/rootfs/sbin/init
-rwxr-xr-x 1 root root 5165632 Apr 21 06:30 /root/.a3s/boxes/50d12f99-f2d1-4b83-99ee-965d63614c55/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 09:28 /root/.a3s/boxes/77f493d4-9c1a-4526-9b31-67232e6185a7/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 09:28 /root/.a3s/boxes/781ec7fe-754c-46eb-ace0-6cb62781777f/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 06:29 /root/.a3s/boxes/a5c81bda-b5fa-4981-a588-12d412a42f49/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 09:17 /root/.a3s/boxes/c20851dd-8164-41c4-af7f-232f9b591ad3/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 09:17 /root/.a3s/boxes/e9637925-cbc4-40ff-91d6-bb0b163686bc/rootfs/sbin/init
-rwxr-xr-x 1 root root 5338136 Apr 21 09:28 /root/.a3s/boxes/f99ea77d-3ee5-4836-90ac-31be5980a983/rootfs/sbin/init

file ~/.a3s/boxes/*/rootfs/sbin/init 2>/dev/null

/root/.a3s/boxes/02245875-3cfe-481e-930c-d9c6b7a8ac61/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/157b339a-fac1-42b7-b136-445e1ac08ea2/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/3a4989f0-2274-4ef8-bef1-26a645a81af2/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/50d12f99-f2d1-4b83-99ee-965d63614c55/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), static-pie linked, BuildID[sha1]=fbc9fdc0af3584321e0f60e54899e6255e2d59db, not stripped
/root/.a3s/boxes/77f493d4-9c1a-4526-9b31-67232e6185a7/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/781ec7fe-754c-46eb-ace0-6cb62781777f/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/a5c81bda-b5fa-4981-a588-12d412a42f49/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/c20851dd-8164-41c4-af7f-232f9b591ad3/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/e9637925-cbc4-40ff-91d6-bb0b163686bc/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped
/root/.a3s/boxes/f99ea77d-3ee5-4836-90ac-31be5980a983/rootfs/sbin/init: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=843eb736ca1b3963d8fc1c112e144341b6860d96, not stripped

9. What does strace show? If available:
   strace -f a3s-box run --name hello alpine:latest -- echo "Hello" 2>&1 | grep -E "(execve|sbin/init|BOX_EXEC)" | head -30

no output

This will help determine whether:

• The guest init binary is missing from your installation
• The guest init is not being installed into the rootfs
• The BOX_EXEC_EXEC environment variable is not being passed correctly to the VM

[ZhiXiao-Lin]

Root Cause Analysis

This issue had two distinct root causes:

1. wait_for_children() exits too early (causes both errors)

In guest-init/src/main.rs, the wait_for_children() function called process::exit() immediately when the container process exited. However, the vsock servers (exec, PTY, attestation) run as background threads started AFTER the container process spawns. When the container process exited quickly (e.g. echo "Hello" completes in milliseconds), the entire VM was terminated before the vsock servers had a chance to start - hence the WARN Exec socket appeared but heartbeat failed and PTY connection refused errors.

2. Fallback Entrypoint /sbin/init does not exist in Alpine

When neither ENTRYPOINT nor CMD is set in the OCI image, the code fell back to /sbin/init. Alpine Linux does not have /sbin/init - it uses busybox init invoked differently. This caused the execvp(/sbin/init) failed errno=2 error.

Fixes Applied (v2.0.1)

Fix 1: VM stays alive until SIGTERM

The wait_for_children() function now loops indefinitely after the container process exits, until it receives a SIGTERM shutdown signal. This gives the vsock server threads time to start before the VM exits.

Before:

if pid == container_pid {
    process::exit(status);
}

After:

if pid == container_pid {
    loop {
        if SHUTDOWN_REQUESTED.load(Ordering::SeqCst) {
            graceful_shutdown(CHILD_SHUTDOWN_TIMEOUT_MS);
            return Ok(());
        }
        std::thread::sleep(std::time::Duration::from_secs(1));
    }
}

Fix 2: Fallback entrypoint changed from /sbin/init to /bin/sh

/bin/sh is universally available across all Linux distributions including Alpine.

Testing

All 787 runtime unit tests pass. The original failing command was verified:

a3s-box run alpine:latest -- echo "Hello from MicroVM"

Pull Request: #4
Release: v2.0.1 - https://github.com/AI45Lab/Box/releases/tag/v2.0.1