feat: add category health score to project insights

[gaspergrom]

Note

Medium Risk
Adds new columns to the project_insights_copy_ds schema and propagates them through the copy/serving pipes, which can impact downstream consumers and Tinybird copy jobs if not coordinated. Logic changes are straightforward SELECT/field-mapping with minimal behavioral complexity.

Overview
Project insights now returns health score category breakdowns in addition to the overall healthScore.

This extends project_insights_copy_ds with four new nullable fields (contributorHealthScore, popularityHealthScore, developmentHealthScore, securityHealthScore), populates them in project_insights_copy.pipe from health_score_copy_ds percentage columns, and selects them through project_insights.pipe so they are available to the endpoint response.

^{Written by Cursor Bugbot for commit f2d4f65. This will update automatically on new commits. Configure here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[Copilot]

Pull request overview

This PR expands the Tinybird “project insights” dataset/API to include health score category breakdowns (contributors, popularity, development, security) alongside the existing overall health score.

Changes:

• Expose four new health score category fields from the health score dataset into the materialized project_insights_copy_ds.
• Update the project_insights endpoint pipe to return the new category health score fields.
• Document the new fields in the project_insights_copy_ds datasource description and schema.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
services/libs/tinybird/pipes/project_insights.pipe	Adds the new health score category fields to the API-facing SELECT and updates the pipe description.
services/libs/tinybird/pipes/project_insights_copy.pipe	Pulls category score percentages from health_score_copy_ds and maps them into the materialized project insights dataset.
services/libs/tinybird/datasources/project_insights_copy_ds.datasource	Extends the materialized datasource schema/docs with the new category health score columns.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The new category score fields are pulled directly from health_score_copy_ds without normalizing invalid float values. Elsewhere (e.g. pipes/health_score_sink.pipe) these score/percentage columns are guarded with if(isNaN(...), null, ...), which suggests NaNs can occur. Consider applying the same NaN-to-null (and optionally COALESCE) handling here so project_insights_copy_ds doesn’t materialize NaNs into the API-facing dataset.

Suggested change

	dep.contributorPercentage AS contributorHealthScore,
	dep.popularityPercentage AS popularityHealthScore,
	dep.developmentPercentage AS developmentHealthScore,
	dep.securityPercentage AS securityHealthScore,
	if(isNaN(dep.contributorPercentage), null, dep.contributorPercentage) AS contributorHealthScore,
	if(isNaN(dep.popularityPercentage), null, dep.popularityPercentage) AS popularityHealthScore,
	if(isNaN(dep.developmentPercentage), null, dep.developmentPercentage) AS developmentHealthScore,
	if(isNaN(dep.securityPercentage), null, dep.securityPercentage) AS securityHealthScore,

[Copilot]

The description states these category scores are in the 0-100 range, but the schema marks them as Nullable(Float64) and the upstream join can yield NULL (or even NaN if not sanitized). Consider updating the column docs to mention when NULL is expected (e.g., missing health score data), or alternatively make the pipe coalesce/sanitize and change the schema to non-nullable for clearer API semantics.

Suggested change

	- `healthScore` column is the overall health score for the project (0-100).
	- `contributorHealthScore` column is the health score percentage for the contributors category (0-100).
	- `popularityHealthScore` column is the health score percentage for the popularity category (0-100).
	- `developmentHealthScore` column is the health score percentage for the development category (0-100).
	- `securityHealthScore` column is the health score percentage for the security category (0-100).
	- `healthScore` column is the overall health score for the project; when present, it is a value in the 0-100 range, and it is NULL when health score data is unavailable.
	- `contributorHealthScore` column is the health score percentage for the contributors category; when present, it is a value in the 0-100 range, and it is NULL when the contributors health score is unavailable.
	- `popularityHealthScore` column is the health score percentage for the popularity category; when present, it is a value in the 0-100 range, and it is NULL when the popularity health score is unavailable.
	- `developmentHealthScore` column is the health score percentage for the development category; when present, it is a value in the 0-100 range, and it is NULL when the development health score is unavailable.
	- `securityHealthScore` column is the health score percentage for the security category; when present, it is a value in the 0-100 range, and it is NULL when the security health score is unavailable.