Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 77 additions & 28 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,27 @@ The library requires a running SQL Server instance for its integration tests.

```
lib/
ex_sql_client.ex # Public API – the main entry point
ex_sql_client.ex # Public API – start_link/query/prepare/transaction
ex_sql_client/
connection.ex # DBConnection behaviour implementation
protocol.ex # Netler RPC call helpers
query.ex # Query struct
result.ex # Result struct
protocol.ex # DBConnection behaviour implementation (Netler RPC)
query.ex # Query struct (statement + statement_id)
ecto.ex # Ecto 3 adapter entry point (use Ecto.Adapters.SQL)
ecto/
connection.ex # Ecto.Adapters.SQL.Connection: SQL generation + result normalisation
dotnet/dotnet_sql_client/
DotnetSqlClient.csproj # .NET 8 project file
Program.cs # Netler.NET server bootstrap
SqlAdapter.cs # SQL Server operations via Microsoft.Data.SqlClient
test/
query_test.exs
transaction_test.exs
prepared_statement_test.exs
data_type_test.exs
test_helper.exs
query_test.exs # Integration: raw query tests
transaction_test.exs # Integration: transaction tests
prepared_statement_test.exs # Integration: prepared statement tests
data_type_test.exs # Integration: type mapping tests
test_helper.exs # Testcontainers setup and connection string injection
ecto/
query_test.exs # Unit: SQL generation tests (no DB required)
ecto_adapter_test.exs # Integration: end-to-end Ecto adapter tests
mix.exs # Build config and project metadata
docker-compose.yml # Local SQL Server for integration tests
.github/workflows/ # GitHub Actions CI (build + test + publish)
```

Expand All @@ -60,18 +63,30 @@ in `dotnet/dotnet_sql_client/` and places the binary in `priv/`.

## Testing

Integration tests spin up a SQL Server container automatically via
[Testcontainers](https://hex.pm/packages/testcontainers). Docker (or a
compatible runtime) must be available on the machine.
There are two categories of tests:

**Unit tests** (no database required) — SQL generation tests for the Ecto adapter:

```bash
mix test test/ecto/query_test.exs
```

**Integration tests** spin up a SQL Server container automatically via
[Testcontainers](https://hex.pm/packages/testcontainers). Docker or a
compatible rootless runtime (e.g. Podman) must be available.
Comment on lines +66 to +76
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section says the Ecto adapter SQL-generation tests require no database, but test/test_helper.exs currently starts Testcontainers and a SQL Server container unconditionally (even when running only test/ecto/query_test.exs). Either update test/test_helper.exs to only start containers when integration tests are enabled, or adjust this doc to reflect the current behavior.

Copilot uses AI. Check for mistakes.

```bash
mix test --only integration
# Core driver integration tests
mix test --include integration

# All Ecto adapter tests (unit + integration)
mix test test/ecto/ --include integration
```

The container is started once in `test/test_helper.exs` and the connection
string is shared with all test modules via `Application.put_env`. All tests are
tagged with `@tag :integration`. There are no unit tests that run without a
live database.
string is shared with all test modules via `Application.put_env`. Integration
tests are tagged with `@tag :integration` and are excluded by default; pass
`--include integration` to run them.

---

Expand Down Expand Up @@ -101,25 +116,43 @@ live database.

## Architecture notes

The call chain for a query is:
The call chain for a raw (`ExSqlClient`) query is:

```
Elixir caller
→ ExSqlClient (DBConnection behaviour)
→ Netler RPC (TCP socket, port process)
→ Program.cs (Netler.NET server)
→ SqlAdapter.cs
→ Microsoft.Data.SqlClient
→ SQL Server
→ ExSqlClient (public API)
→ ExSqlClient.Protocol (DBConnection behaviour)
→ Netler RPC (TCP socket, port process)
→ Program.cs (Netler.NET server)
→ SqlAdapter.cs
→ Microsoft.Data.SqlClient
→ SQL Server
```

When using the Ecto adapter, an additional layer sits in front:

```
Ecto / Repo
→ ExSqlClient.Ecto (Ecto.Adapters.SQL)
→ ExSqlClient.Ecto.Connection (SQL generation, result normalisation)
→ ExSqlClient.Protocol (DBConnection behaviour)
→ … (same chain as above)
```

Key invariants:
- The .NET process is started by Netler as a port. It listens on a TCP port
passed as `args[0]`; the Elixir PID is passed as `args[1]`.
- All route names in `Program.cs` must be registered and match what the
Elixir layer calls via Netler.
Elixir layer calls via `Netler.Client.invoke/3`.
- Connection and transaction state is held in `SqlAdapter` — one instance per
connection process.
- The Ecto adapter generates MSSQL-dialect SQL: bracket identifiers `[name]`,
`TOP(n)` for limits without offset, `OFFSET … FETCH NEXT … ROWS ONLY` for
pagination, and `OUTPUT INSERTED/DELETED` for `RETURNING`.
- Netler/MessagePack deserialises result rows as Elixir `Map`, which sorts
string keys alphabetically. `ExSqlClient.Ecto.Connection` recovers the
correct column order by parsing the SELECT projection or OUTPUT clause from
the SQL string before returning results to Ecto.

---

Expand All @@ -130,21 +163,31 @@ Good first contributions:
- Adding `@spec` / `@type` annotations to the Elixir modules.
- Improving error propagation from the .NET layer to Elixir.
- Updating dependencies as new versions are released.
- Expanding `test/ecto/query_test.exs` with additional SQL generation cases.

Areas requiring extra care:
- Anything touching `Program.cs` or `SqlAdapter.cs` — changes must compile
with the pinned .NET version and be verified against a live SQL Server.
- `SqlAdapter.cs` DML result handling — `ExecuteReader` is used for all
statements so that `OUTPUT` clauses are supported; `RecordsAffected` is
captured inside the `using` block and injected as a synthetic
`__rows_affected__` row for DML without an `OUTPUT` clause.
- Netler version upgrades — the RPC protocol may change between major versions;
always check `Program.cs` against the new Netler.NET API.
- `DBConnection` behaviour callbacks — maintain compatibility with the
`db_connection` contract.
- `ExSqlClient.Ecto.Connection` — column-order recovery relies on regex parsing
of the generated SQL; changes to the SQL generator must keep
`column_order_from_sql/1` in sync.

---

## Pull request checklist

- [ ] `mix compile --warnings-as-errors` passes.
- [ ] `mix test --only integration` passes against a local SQL Server.
- [ ] `mix test test/ecto/query_test.exs` passes (no DB needed).
- [ ] `mix test --include integration` passes against a local SQL Server.
- [ ] `mix test test/ecto/ --include integration` passes for Ecto adapter changes.
- [ ] `mix format --check-formatted` passes.
- [ ] `mix credo` passes.
- [ ] New public functions have `@doc` and `@spec`.
Expand All @@ -157,5 +200,11 @@ Areas requiring extra care:
- Windows CI — the SQL Server Docker container is Linux-only in GitHub Actions;
the library itself is cross-platform but CI runs on Linux.
- Supporting databases other than Microsoft SQL Server.
- Ecto migrations / DDL — `execute_ddl/1` raises intentionally; use
`ExSqlClient.query/3` directly for schema changes.
- `Repo.stream/2` and cursor-based fetching — the Netler/C# layer does not
implement server-side cursors.
- Multiple result sets via the Ecto adapter (`query_many/4` raises); use the
raw `ExSqlClient` API if you need multiple result sets.
- Changing the `DBConnection` protocol — maintain compatibility with standard
Elixir database tooling (Ecto, etc.).
Elixir database tooling.
Loading