feat: add native code coverage support (#348)

This PR introduces native code coverage support for Ruby targets using Bazel's coverage command. It integrates SimpleCov and simplecov-lcov to generate LCOV-formatted reports.

Key implementation details:
- Coverage Helper: A specialized coverage.rb script is automatically required via RUBYOPT when coverage is enabled.
- Workspace-Relative Paths: To ensure Bazel can aggregate coverage from multiple targets, we ensure all SF: paths in the LCOV report are relative to the workspace root.
  - MRI: MRI preserves loaded paths as symlinks within the Bazel sandbox. By setting the SimpleCov root to Dir.pwd (the sandbox root), we naturally get workspace-relative paths.
  - JRuby: JRuby resolves all files to their absolute realpaths on the physical disk, bypassing the sandbox symlinks. To generate relative paths, we dynamically calculate the realpath of the workspace root relative to the coverage.rb helper and set it as the SimpleCov root.
- Bazel Integration:
  - COVERAGE_OUTPUT_FILE: SimpleCov is configured to write directly to the path provided by Bazel.
  - BAZEL_TARGET: Used as the command_name to provide unique identification for merged results and avoid warnings.
- JRuby Support: Automatically enables --debug via JRUBY_OPTS when coverage is requested, as required for JRuby coverage collection.
- Documentation: Updated the main README and rb_test rule documentation.
- Examples: Added a test_coverage.sh script to the gem example to verify coverage across different engines.
- CI Verification: Added a coverage verification step to the CI matrix in examples/gem to ensure it works across all supported MRI and JRuby versions and platforms.

Fixes #347

Author: noahkawasakigoogle

.github/workflows/ci.yml

@@ -81,6 +81,23 @@ jobs:
       - run: bazel run "@bundle//bin:rake" "--" --version
       - run: bazel run "@bundle//bin:rspec" "--" --help
       - run: bazel test ...
+      - name: Run coverage
+        if: matrix.os != 'windows-latest'
+        shell: bash
+        run: |
+          bazel coverage //spec:add
+          COVERAGE_FILE=bazel-testlogs/spec/add/coverage.dat
+          if ! grep -q "SF:.*lib/gem/add.rb" "$COVERAGE_FILE"; then
+            echo "Error: Coverage report does not contain expected file coverage"
+            cat "$COVERAGE_FILE"
+            exit 1
+          fi
+      - name: Upload coverage report
+        if: matrix.os != 'windows-latest'
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-${{ matrix.os }}-${{ matrix.ruby }}-${{ matrix.mode }}
+          path: examples/gem/bazel-testlogs/spec/add/coverage.dat
       - if: failure() && runner.debug == '1'
         uses: mxschmitt/action-tmate@v3
 

README.md

@@ -214,6 +214,37 @@ However, some are known not to work or work only partially (e.g. mRuby has no bu
   To work it around, use [`--experimental_inprocess_symlink_creation`][16] Bazel flag.
   See [`bazelbuild/bazel#4327`][17] for more details.
 
+## Code Coverage
+
+Ruby rules support Bazel's native coverage collection using [SimpleCov][21] and [simplecov-lcov][22].
+
+> [!NOTE]
+> Code coverage is currently not supported on Windows.
+
+To enable coverage:
+
+1. Add `simplecov` and `simplecov-lcov` gems to your `Gemfile`.
+2. Run your tests with the `coverage` command:
+
+```bash
+bazel coverage //...
+```
+
+The rules automatically configure SimpleCov to use the LCOV formatter and output reports to the location expected by Bazel. Coverage reports will have workspace-relative paths, allowing Bazel to aggregate results from multiple targets.
+
+For JRuby, coverage requires `--debug` mode, which is automatically enabled by the rules when coverage is requested.
+
+You can add additional coverage filters using `coverage_filters` attribute in `rb_test` or `rb_binary`:
+
+```python
+rb_test(
+    name = "my_test",
+    ...
+    coverage_filters = ["/vendor/", "/custom/"],
+)
+```
+
+
 [1]: https://www.ruby-lang.org
 [2]: https://bazel.build
 [3]: docs/repository_rules.md
@@ -234,3 +265,5 @@ However, some are known not to work or work only partially (e.g. mRuby has no bu
 [18]: docs/rails.md
 [19]: https://github.com/jdx/ruby
 [20]: ruby/private/portable_ruby_checksums.bzl
+[21]: https://github.com/simplecov-ruby/simplecov
+[22]: https://github.com/fortissimo1997/simplecov-lcov

docs/rules.md

@@ -7,4 +7,6 @@ gem 'debug', '>= 1.0.0', platforms: %i[mri mswin64]
 gem 'psych', '~> 5.3' # native extensions
 gem 'rspec', '~> 3.0'
 gem 'rubocop', '~> 1.64'
+gem 'simplecov', '~> 0.22'
+gem 'simplecov-lcov', '~> 0.8'
 gem 'testcontainers', '~> 0.2' # empty gem

examples/gem/Gemfile

@@ -16,6 +16,7 @@ GEM
       irb (~> 1.10)
       reline (>= 0.3.8)
     diff-lcs (1.5.1)
+    docile (1.4.1)
     docker-api (2.4.0)
       excon (>= 0.64.0)
       multi_json
@@ -83,6 +84,13 @@ GEM
     rubocop-ast (1.37.0)
       parser (>= 3.3.1.0)
     ruby-progressbar (1.13.0)
+    simplecov (0.22.0)
+      docile (~> 1.1)
+      simplecov-html (~> 0.11)
+      simplecov_json_formatter (~> 0.1)
+    simplecov-html (0.13.2)
+    simplecov-lcov (0.9.0)
+    simplecov_json_formatter (0.1.4)
     stringio (3.2.0)
     testcontainers (0.2.0)
       testcontainers-core (= 0.2.0)
@@ -106,6 +114,8 @@ DEPENDENCIES
   psych (~> 5.3)
   rspec (~> 3.0)
   rubocop (~> 1.64)
+  simplecov (~> 0.22)
+  simplecov-lcov (~> 0.8)
   testcontainers (~> 0.2)
 
 BUNDLED WITH

examples/gem/Gemfile.lock

@@ -34,6 +34,7 @@ ruby.bundle_fetch(
         "date-3.5.1-java": "12e09477dc932afe45bf768cd362bf73026804e0db1e6c314186d6cd0bee3344",
         "debug-1.10.0": "11e28ca74875979e612444104f3972bd5ffb9e79179907d7ad46dba44bd2e7a4",
         "diff-lcs-1.5.1": "273223dfb40685548436d32b4733aa67351769c7dea621da7d9dd4813e63ddfe",
+        "docile-1.4.1": "96159be799bfa73cdb721b840e9802126e4e03dfc26863db73647204c727f21e",
         "docker-api-2.4.0": "824be734f4cc8718189be9c8e795b6414acbbf7e8b082a06f959a27dd8dd63e6",
         "excon-1.3.2": "a089babe98638e58042a7d542b2bbd183304527e33d612b6dde22fa491a544a5",
         "i18n-1.14.7": "ceba573f8138ff2c0915427f1fc5bdf4aa3ab8ae88c8ce255eb3ecf0a11a5d0f",
@@ -67,6 +68,10 @@ ruby.bundle_fetch(
         "rubocop-1.71.0": "e19679efd447346ac476122313d3788ae23c38214790bcf660e984c747608bf0",
         "rubocop-ast-1.37.0": "9513ac88aaf113d04b52912533ffe46475de1362d4aa41141b51b2455827c080",
         "ruby-progressbar-1.13.0": "80fc9c47a9b640d6834e0dc7b3c94c9df37f08cb072b7761e4a71e22cff29b33",
+        "simplecov-0.22.0": "fe2622c7834ff23b98066bb0a854284b2729a569ac659f82621fc22ef36213a5",
+        "simplecov-html-0.13.2": "bd0b8e54e7c2d7685927e8d6286466359b6f16b18cb0df47b508e8d73c777246",
+        "simplecov-lcov-0.9.0": "7a77a31e200a595ed4b0249493056efd0c920601f53d2ef135ca34ee796346cd",
+        "simplecov_json_formatter-0.1.4": "529418fbe8de1713ac2b2d612aa3daa56d316975d307244399fa4838c601b428",
         "stringio-3.2.0": "c37cb2e58b4ffbd33fe5cd948c05934af997b36e0b6ca6fdf43afa234cf222e1",
         "testcontainers-0.2.0": "b8fb466cfbba02eef9120fa0f4e1ebce6c7220b65c5b025ce003d8c6a6b68c67",
         "testcontainers-core-0.2.0": "94f316f08f388a8afb75a4fec6ff9ed4289cf60ca7a77522d4ca26688f4c4da4",

examples/gem/MODULE.bazel

@@ -1,5 +1,7 @@
 load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
 
+exports_files(["coverage.rb"])
+
 bzl_library(
     name = "binary",
     srcs = ["binary.bzl"],

ruby/private/BUILD

@@ -61,10 +61,17 @@ Supports `$(location)` expansion for targets from `srcs`, `data` and `deps`.
     "_windows_constraint": attr.label(
         default = "@platforms//os:windows",
     ),
+    "_coverage_helper": attr.label(
+        allow_single_file = True,
+        default = "@rules_ruby//ruby/private:coverage.rb",
+    ),
+    "coverage_filters": attr.string_list(
+        doc = "Additional coverage filters to add to SimpleCov. Only applied during 'bazel coverage'.",
+    ),
 }
 
 # buildifier: disable=function-docstring
-def generate_rb_binary_script(ctx, binary, bundler = False, args = [], env = {}, java_bin = "", jars_home_strip_suffix = ""):
+def generate_rb_binary_script(ctx, binary, bundler = False, args = [], env = {}, java_bin = "", jars_home_strip_suffix = "", coverage_helper = "", is_jruby = False):
     toolchain = ctx.toolchains["@rules_ruby//ruby:toolchain_type"]
     if ctx.attr.ruby != None:
         toolchain = ctx.attr.ruby[platform_common.ToolchainInfo]
@@ -79,7 +86,7 @@ def generate_rb_binary_script(ctx, binary, bundler = False, args = [], env = {},
         # running binary scripts directly and should not affect normal `bazel run`.
         # See BATCH_RLOCATION_FUNCTION comments for more details.
         if binary_path.startswith("../") and not _is_windows(ctx):
-            binary_path = _to_rlocation_path(binary)
+            binary_path = _to_rlocation_path(ctx, binary)
             locate_binary_in_runfiles = "true"
         else:
             binary_path = _normalize_path(ctx, binary_path)
@@ -116,11 +123,13 @@ def generate_rb_binary_script(ctx, binary, bundler = False, args = [], env = {},
             "{env}": _convert_env_to_script(ctx, environment),
             "{bundler_command}": bundler_command,
             "{jars_home_strip_suffix}": jars_home_strip_suffix,
-            "{ruby}": _to_rlocation_path(toolchain.ruby),
+            "{ruby}": _to_rlocation_path(ctx, toolchain.ruby),
             "{ruby_binary_name}": toolchain.ruby.basename,
             "{java_bin}": java_bin,
             "{rlocation_function}": rlocation_function,
             "{locate_binary_in_runfiles}": locate_binary_in_runfiles,
+            "{coverage_helper}": coverage_helper,
+            "{is_jruby}": "true" if is_jruby else "",
         },
     )
 
@@ -143,6 +152,7 @@ def rb_binary_impl(ctx):
     if ctx.attr.ruby != None:
         ruby_toolchain = ctx.attr.ruby[platform_common.ToolchainInfo]
     tools = list(ruby_toolchain.files)
+    tools.append(ctx.file._coverage_helper)
 
     if ruby_toolchain.version.startswith("jruby"):
         java_toolchain = ctx.toolchains["@bazel_tools//tools/jdk:runtime_toolchain_type"]
@@ -163,8 +173,8 @@ def rb_binary_impl(ctx):
 
             # See https://bundler.io/v2.5/man/bundle-config.1.html for confiugration keys.
             env.update({
-                "BUNDLE_GEMFILE": _to_rlocation_path(info.gemfile),
-                "BUNDLE_PATH": _to_rlocation_path(info.path),
+                "BUNDLE_GEMFILE": _to_rlocation_path(ctx, info.gemfile),
+                "BUNDLE_PATH": _to_rlocation_path(ctx, info.path),
             })
     if len(bundler_srcs) > 0:
         transitive_srcs = depset(bundler_srcs, transitive = [transitive_srcs])
@@ -174,6 +184,9 @@ def rb_binary_impl(ctx):
     env.update(ruby_toolchain.env)
     env.update(ctx.attr.env)
 
+    if ctx.configuration.coverage_enabled and ctx.attr.coverage_filters:
+        env["COVERAGE_FILTERS"] = ",".join(ctx.attr.coverage_filters)
+
     runfiles = ctx.runfiles(tools, transitive_files = depset(transitive = [transitive_srcs, transitive_data]))
     runfiles = get_transitive_runfiles(runfiles, ctx.attr.srcs, ctx.attr.deps, ctx.attr.data)
     runfiles = runfiles.merge(ctx.attr._runfiles_library[DefaultInfo].default_runfiles)
@@ -190,6 +203,8 @@ def rb_binary_impl(ctx):
         env = env,
         java_bin = java_bin,
         jars_home_strip_suffix = jars_home_strip_suffix,
+        coverage_helper = _to_rlocation_path(ctx, ctx.file._coverage_helper),
+        is_jruby = ruby_toolchain.version.startswith("jruby"),
     )
 
     return [
@@ -214,6 +229,7 @@ def rb_binary_impl(ctx):
 rb_binary = rule(
     implementation = rb_binary_impl,
     executable = True,
+    fragments = ["coverage"],
     attrs = dict(
         ATTRS,
         srcs = LIBRARY_ATTRS["srcs"],

ruby/private/binary.bzl

@@ -40,6 +40,11 @@ if "{bundler_command}" neq "" (
   )
 )
 
+if "%COVERAGE%" == "1" (
+  echo>&2 ERROR: Code coverage is currently not supported on Windows.
+  exit 1
+)
+
 {bundler_command} {ruby_binary_name} {binary} {args} %*
 
 :: vim: ft=dosbatch

ruby/private/binary/binary.cmd.tpl

@@ -81,6 +81,13 @@ else
   binary="{binary}"
 fi
 
+if [[ "${COVERAGE:-}" == "1" ]]; then
+  export RUBYOPT="-r$(rlocation {coverage_helper}) ${RUBYOPT:-}"
+  if [ -n "{is_jruby}" ]; then
+    export JRUBY_OPTS="--debug ${JRUBY_OPTS:-}"
+  fi
+fi
+
 exec {bundler_command} {ruby_binary_name} $binary {args} "$@"
 
 # vim: ft=bash