Windows PowerShell Compatibility Gaps in Dev Environment Documentation

[jaiswal-naman]

Summary

While setting up the OLake development environment on Windows 11 using default Windows PowerShell (5.1), I encountered multiple reproducible command-level incompatibilities caused by Bash-specific assumptions in the documentation.

After installing all documented prerequisites and verifying the runtime environment, OLake containers and services run successfully on Windows. The issues described below are strictly related to:

• Shell assumptions (Bash vs PowerShell)
• Unix-specific utilities
• Command chaining syntax
• Script execution behavior

This issue aims to provide:

• Exact commands executed
• Exact PowerShell error outputs
• Root cause analysis
• Confirmed working alternatives
• Concrete documentation improvement suggestions

This is not a runtime bug report — OLake itself works correctly once commands are adapted for Windows.

---

Test Environment

Operating System

• Windows 11 (x64)

Shell

• Windows PowerShell 5.1 (default Windows shell)

Installed Prerequisites

• Docker (latest stable)
• Docker Compose v2
• Go
• Node
• Maven
• Java (installed per documentation requirement)

All prerequisites were installed and verified before testing.

---

✅ Verified: OLake Runtime Works on Windows

After adapting shell-specific commands, the Docker-based environment starts successfully.

Example:

docker compose -f docker-compose.source.yml --profile postgres -f docker-compose.destination.yml up -d

Result:

✔ Container iceberg-postgres    Healthy
✔ Container primary_postgres    Healthy
✔ Container spark-iceberg       Started

---

❗ Identified Windows Compatibility Gaps

---

1️⃣ curl -sSL ... | docker compose -f - up -d

Documented Command

curl -sSL https://raw.githubusercontent.com/... | docker compose -f - up -d

Observed Behavior in PowerShell

Invoke-WebRequest : A parameter cannot be found that matches parameter name 'sSL'.

📸 Screenshot – curl alias error

Root Cause

In Windows PowerShell:

• curl is an alias for Invoke-WebRequest
• -sSL are Unix curl flags
• PowerShell does not interpret them correctly
• The pipeline behavior differs from Bash

Verified Working Alternative

curl.exe -sSL https://raw.githubusercontent.com/... | docker compose -f - up -d

Using curl.exe (explicit binary) works successfully.

Documentation Impact

The current docs assume Unix curl behavior, but Windows users using default PowerShell will encounter immediate failure.

Suggested Improvement

Either:

• Use curl.exe explicitly in documentation
  OR
• Provide a Windows tab with:

Invoke-WebRequest -Uri "https://raw.githubusercontent.com/..." -OutFile "docker-compose.yml"
docker compose -f docker-compose.yml up -d

---

2️⃣ sh -c Usage in Dev Environment Setup

Documented Command

sh -c 'curl ... && curl ... && docker compose ...'

PowerShell Output

sh : The term 'sh' is not recognized as the name of a cmdlet

📸 Screenshot – sh not recognized

Root Cause

• Windows does not include a POSIX shell by default
• sh is unavailable unless using Git Bash or WSL

Documentation Impact

The current step cannot be executed in default Windows PowerShell.

Suggested Improvement

Option A:
Split commands into separate PowerShell steps.

Option B:
Add explicit note:

This command requires a Bash-compatible shell (Git Bash or WSL).

---

3️⃣ ./build.sh Execution in PowerShell

Documented Command

./build.sh driver-postgres discover --config $(pwd)/source.json

Observed Behavior

./build.sh : The term './build.sh' is not recognized

📸 Screenshot – build.sh not executable

Root Cause

• .sh scripts require Bash
• PowerShell does not execute shebang scripts
• Script association differs on Windows

Impact

Windows contributors cannot run CLI commands as documented.

Suggested Improvements

Option A – Provide direct Go execution alternative:

go run ./drivers/postgres/main.go discover --config "$PWD\source.json"

Option B – Provide build.ps1 PowerShell equivalent.

Option C – Explicitly document that Bash shell is required for CLI development.

---

4️⃣ git clone ... && cd ... Syntax

Documented Command

git clone git@github.com:datazip-inc/olake.git && cd olake

PowerShell Output

The token '&&' is not a valid statement separator

📸 Screenshot – && not supported

Additionally:

git@github.com: Permission denied (publickey)

Root Cause

• PowerShell 5.1 does not support &&
• SSH cloning assumes SSH key configuration

Suggested Improvement

Provide Windows-friendly alternative:

git clone https://github.com/datazip-inc/olake.git
Set-Location olake

---

5️⃣ lsof in Troubleshooting Section

Documented Command

lsof -i :8000

PowerShell Output

lsof : The term 'lsof' is not recognized

📸 Screenshot – lsof not found

Root Cause

lsof is a Unix utility and not available on Windows.

Suggested Windows Alternative

netstat -ano | findstr :8000

---

🔎 Broader Documentation Observation

The current documentation implicitly assumes:

• Bash shell environment
• POSIX utilities
• Unix-style command chaining
• SSH key setup
• Unix debugging tools

However:

• Windows is listed as a supported OS
• No explicit mention is made that Git Bash or WSL is required

This creates confusion for Windows users using default PowerShell.

---

💡 Proposed Improvements

Option 1 (Best Long-Term)

Introduce OS-specific tabs:

• macOS / Linux
• Windows (PowerShell)

For:

• Docker setup
• Dev environment setup
• CLI commands
• Troubleshooting steps

---

Option 2 (Minimal Change)

Add a clear note at top of Dev Environment page:

Development commands assume a Bash-compatible shell. Windows users should use Git Bash or WSL.

---

Conclusion

OLake runs successfully on Windows once commands are adapted appropriately.

The friction points are:

• Shell assumptions
• Unix-specific commands
• Lack of PowerShell alternatives

Happy to contribute a PR adding PowerShell-specific instructions if maintainers agree this would improve onboarding for Windows contributors.

Thank you for the great work on OLake 🚀