docs: Migrate from sphinx to mkdocs

.github/workflows/doc-build.yml

@@ -7,21 +7,27 @@ on:
     branches:
       - '**'
     tags:
-      - 'v*.*.*'
+      - 'docs-v*.*.*'
   release:
     types: [published]
+
+defaults:
+  run:
+    working-directory: ./docs
 jobs:
   build:
     runs-on: ubuntu-latest
-    env:
-      working-directory: ./docs
     steps:
       - uses: actions/checkout@v3
-      - name: Check documentation build
+      - name: Install uv for Python 3.13
+        uses: astral-sh/setup-uv@v6
+        with:
+          python-version: 3.13
+          enable-cache: true
+          cache-dependency-glob: "docs/uv.lock"
+      - name: Install the project
+        run: |
+          uv sync --all-extras
+      - name: Build documentation
         run: |
-          cd docs
-          pip3 install -r requirements.txt
-          make clean
-          # Fail buiild on any docs warning
-          # make html O=-W # Removed whilst migrating existing docs
-          make html
+          uv run mkdocs build

.readthedocs.yml

@@ -5,14 +5,19 @@
 version: 2
 
 build:
-  os: ubuntu-20.04
+  os: "ubuntu-24.04"
   tools:
-    python: "3.9"
+    python: "3.13"
+  jobs:
+    pre_create_environment:
+        - cd docs
+        - asdf plugin add uv
+        - asdf install uv latest
+        - asdf global uv latest
+    create_environment:
+        - uv venv "${READTHEDOCS_VIRTUALENV_PATH}"
+    install:
+        - cd docs && UV_PROJECT_ENVIRONMENT="${READTHEDOCS_VIRTUALENV_PATH}" uv sync --frozen
 
-sphinx:
-  configuration: docs/conf.py
-  fail_on_warning: false # Removed warnings to migrate existing docs
-
-python:
-  install:
-    - requirements: docs/requirements.txt
+mkdocs:
+  configuration: docs/mkdocs.yml

docs/README.md

@@ -0,0 +1,40 @@
+# CogStack Documentation
+
+The central documentation for cogstack, hosted on docs.cogstack.org
+
+## Setup
+
+### Prerequisites
+
+- Python 3.10 or higher
+- [uv](https://github.com/astral-sh/uv) package manager
+
+### Installation
+
+```bash
+uv venv --python 3.12 --allow-existing 
+source .venv/bin/activate
+uv sync --dev
+```
+
+## Usage
+
+### Serve documentation locally (Development)
+
+To preview the documentation locally with live reload:
+
+```bash
+uv run mkdocs serve
+```
+
+The documentation will be available at `http://127.0.0.1:8000`
+
+### Build documentation
+
+To build the static site:
+
+```bash
+uv run mkdocs build
+```
+
+The built site will be in the `site/` directory.

docs/docs/cogstack-ai/index.md

@@ -0,0 +1,11 @@
+# CogStack AI
+
+CogStack AI provides powerful natural language processing tools for healthcare and biomedical text analysis.
+
+## Available Tools
+
+- **[MedCAT](https://docs.cogstack.org/projects/nlp)** - Medical Concept Annotation Tool for Named Entity Recognition and Linking (NER+L) from clinical free-text documents.
+
+- **[MedCAT Trainer](https://docs.cogstack.org/projects/medcat-trainer/)** - Interface for building, improving, and customizing Named Entity Recognition and Linking models for biomedical domain text.
+
+Click on the links above to access the full documentation for each tool.

docs/docs/data-engineering/index.md

@@ -0,0 +1,9 @@
+# Data Engineering
+
+CogStack Data Engineering provides tools and frameworks for building scalable, configurable data processing pipelines.
+
+## Available Tools
+
+- **[NiFi](https://docs.cogstack.org/projects/nifi/)** - Apache NiFi integration for CogStack, providing fully configurable and scalable data flows with built-in monitoring, data provenance, and security features.
+
+Click on the link above to access the full documentation.

docs/overview/cogstack-documentation.md → docs/docs/index.md

@@ -1,21 +1,30 @@
-# CogStack Documentation
+Welcome to the CogStack Documentation site.
 
 ## What is CogStack?
+![CogStack Architecture](overview/attachments/architecture.png)
 
 CogStack is a lightweight distributed, fault tolerant database processing architecture and ecosystem, intended to make NLP processing and preprocessing easier in resource constrained environments. It comprises of multiple components, and has been designed to provide configurable data processing pipelines for working with EHR data.
 
 CogStack uses databases and files as primary sources of EHR data, with support for custom data connectors. The platform leverages [Apache NiFi](https://nifi.apache.org/) to provide fully configurable data processing pipelines with the goal of generating annotated JSON standardised schema files that can be readily indexed into [ElasticSearch](https://www.elastic.co/), stored as files or pushed back to a database.
 
-![](./attachments/architecture.png)
 
 CogStack is a commercial open-source product, with the code available on GitHub: [https://github.com/CogStack/](https://github.com/CogStack/) . For enterprise deployments, full platform setup, and advanced features, please [contact us](https://docs.cogstack.org/en/latest/).
 
-:::{tip}
-CogStack is designed as a microservices-based ecosystem. The recommended deployment method is on **Kubernetes using Helm charts**, which provides cloud-native support, scalability, and reliability. Ready-to-use CogStack images are available from the official Docker Hub under the [cogstacksystems](https://hub.docker.com/u/cogstacksystems/) organisation. Docker Compose is still supported for development and smaller deployments, but Kubernetes is recommended for production environments.
-:::
+!!! tip
 
-## Why does this project exist?
+    CogStack is designed as a microservices-based ecosystem. The recommended deployment method is on **Kubernetes using Helm charts**, which provides cloud-native support, scalability, and reliability. Ready-to-use CogStack images are available from the official Docker Hub under the [cogstacksystems](https://hub.docker.com/u/cogstacksystems/) organisation. Docker Compose is still supported for development and smaller deployments, but Kubernetes is recommended for production environments.
+
+## What is CogStack For?
 
 CogStack consists of a range of technologies designed to support modern, open source healthcare analytics, and is chiefly comprised of the Elastic stack ([ElasticSearch](https://www.elastic.co/products/elasticsearch), [Kibana](https://www.elastic.co/products/kibana), etc.), [MedCAT](https://github.com/CogStack/MedCAT) (clinical natural language processing for named entity extraction and linking, contextualization, and realtion extraction), clinical text [OCR](https://github.com/CogStack/ocr-service), and clinical text de-identification. Since the processed EHR data can be represented and stored in databases or ElasticSearch, CogStack can be perfectly utilised as one of the solutions for integrating EHR data with other types of biomedical, -omics, wearables data, etc.
 
----
+---
+
+## Community and support
+
+- **Questions?** Reach out in the [CogStack community forum](https://discourse.cogstack.org/).
+- **Code and projects:** [CogStack on GitHub](https://github.com/orgs/CogStack/repositories).
+
+## Next Steps
+
+[Get Started ](overview/getting-started.md){ .md-button .md-button--primary  }