diff --git a/.agents/donna/intro.md b/.agents/donna/intro.donna.md similarity index 88% rename from .agents/donna/intro.md rename to .agents/donna/intro.donna.md index 2371158c..d4072f3b 100644 --- a/.agents/donna/intro.md +++ b/.agents/donna/intro.donna.md @@ -21,7 +21,7 @@ We may need coding agents on the each step of the process, but there no reason f ## Artifact Tags -To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. Artifacts in `donna:*` world use the next set of tags. +To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. The synced Donna artifacts addressed under `**` use the next set of tags. Artifact type tags: @@ -30,7 +30,7 @@ Artifact type tags: ## Instructions -1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("donna:usage:cli") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. +1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("./usage/cli.donna.md") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. 2. If you need to perform a work with Donna, you **MUST** select an appropriate Donna workflow to perform the work and run it. 3. If there is no appropriate workflow, ask the developer for a precise instructions on what to do. 4. If you are executing a workflow operation and need to perform a complex action or changes, you SHOULD search for an appropriate workflow and run it as a child workflow — it is the intended way to use Donna. @@ -39,7 +39,7 @@ Artifact type tags: ## Journaling -You MUST use `donna journal write` to track your actions and thoughts, according the description in `{{ donna.lib.view("donna:usage:cli") }}`. +You MUST use `donna journal write` to track your actions and thoughts, according the description in `{{ donna.lib.view("./usage/cli.donna.md") }}`. Journaling is a required part of workflow execution. An action request MUST be considered incomplete until required journal records are written. diff --git a/.agents/donna/research/specs/report.md b/.agents/donna/research/specs/report.donna.md similarity index 91% rename from .agents/donna/research/specs/report.md rename to .agents/donna/research/specs/report.donna.md index 2fdcc17b..08f4ab68 100644 --- a/.agents/donna/research/specs/report.md +++ b/.agents/donna/research/specs/report.donna.md @@ -4,19 +4,19 @@ kind = "donna.lib.specification" ``` -This document describes the format and structure of a Research Report document used by Donna workflows from `donna:research:*` namespace. +This document describes the format and structure of a Research Report document used by Donna workflows from `../**` namespace. ## Overview -Donna introduces a group of workflows located in `donna:research:*` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. +Donna introduces a group of workflows located in `../**` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. -Session-related research artifacts MUST be stored as `session:research:`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. +Session-related research artifacts MUST be stored as `@/.donna/session/research/.donna.md`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. The agent (via workflows) creates the artifact and updates it iteratively as the research process progresses. ## Research report structure -The research report is a Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The research report is a Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** -- title and short description of the research problem. - **Original problem description** -- original problem statement from the developer or parent workflow. @@ -35,7 +35,7 @@ The research report is a Donna artifact (check `{{ donna.lib.view("donna:usage:a ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format diff --git a/donna/fixtures/specs/research/work/research.md b/.agents/donna/research/work/research.donna.md similarity index 91% rename from donna/fixtures/specs/research/work/research.md rename to .agents/donna/research/work/research.donna.md index 12a46588..a211f5eb 100644 --- a/donna/fixtures/specs/research/work/research.md +++ b/.agents/donna/research/work/research.donna.md @@ -20,8 +20,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:research:specs:report") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../specs/report.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_problem_description_exists") }}` ## Ensure problem description exists @@ -34,7 +34,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear description of the problem in your context. I.e., you know what you need to do in this workflow. 1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `session:*` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `@/.donna/session/**` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. 3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_artifact") }}`. ## Prepare research artifact @@ -44,8 +44,8 @@ id = "prepare_artifact" kind = "donna.lib.request_action" ``` -1. Based on the problem description you have, suggest an artifact name in the format `session:research:`. `` MUST be unique within the session. -{# TODO: we can add donna.lib.list('session:*') here as the command to list all artifacts in session #} +1. Based on the problem description you have, suggest an artifact name in the format `@/.donna/session/research/.donna.md`. `` MUST be unique within the session. +{# TODO: we can add donna.lib.list('@/.donna/session/**') here as the command to list all session artifacts #} 2. Create the artifact and specify an original problem description in it. 3. `{{ donna.lib.goto("formalize_research") }}` diff --git a/donna/fixtures/specs/rfc/specs/design.md b/.agents/donna/rfc/specs/design.donna.md similarity index 94% rename from donna/fixtures/specs/rfc/specs/design.md rename to .agents/donna/rfc/specs/design.donna.md index 020d0422..3a9d3b2b 100644 --- a/donna/fixtures/specs/rfc/specs/design.md +++ b/.agents/donna/rfc/specs/design.donna.md @@ -8,11 +8,11 @@ This document describes the format and structure of a Design document used to de ## Overview -Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. +Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. You create a Design document to explicitly describe the exact changes you want to make to the project in order to implement the RFC. -If not otherwise specified, Design documents for the session MUST be stored as `session:design:` artifacts in the session world. +If not otherwise specified, Design documents for the session MUST be stored as `@/.donna/session/design/.donna.md` artifacts under `/.donna/session`. **The Design document MUST list exact changes to the project that will be implemented.** E.g. concrete function names and signatures, file paths, data structures, etc. @@ -24,7 +24,7 @@ The Design document MUST NOT be a high-level description of the problem and solu ## Design document structure -The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** — title and short description of the proposed change. - **Inputs** — list of input documents that are relevant for the proposed change, starting from the RFC document. @@ -40,7 +40,7 @@ The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifa ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format diff --git a/.agents/donna/rfc/specs/request_for_change.md b/.agents/donna/rfc/specs/request_for_change.donna.md similarity index 92% rename from .agents/donna/rfc/specs/request_for_change.md rename to .agents/donna/rfc/specs/request_for_change.donna.md index 9c56893a..ecace694 100644 --- a/.agents/donna/rfc/specs/request_for_change.md +++ b/.agents/donna/rfc/specs/request_for_change.donna.md @@ -4,19 +4,19 @@ kind = "donna.lib.specification" ``` -This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `donna:rfc:*` namespace. This document is an input for a Design document creation. +This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `../**` namespace. This document is an input for a Design document creation. ## Overview -Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. +Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. You create RFC documents to propose changes to the project. -If not otherwise specified, RFC documents for the session MUST be stored as `session:rfc:` artifacts in the session world. +If not otherwise specified, RFC documents for the session MUST be stored as `@/.donna/session/rfc/.donna.md` artifacts under `/.donna/session`. ## RFC structure -The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** — title and short description of the proposed change. - **Original description** — original description of the requested changes from the developer or parent workflow. @@ -34,7 +34,7 @@ The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifa ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format @@ -136,7 +136,7 @@ Examples: - Bad: `- Use clean architecture.` - Good: `- The solution MUST be compatible with Python 3.12.` - Good: `- The solution MUST NOT introduce new runtime dependencies.` -- Good: `- The solution MUST follow the specification project:specs:abc` +- Good: `- The solution MUST follow the specification ../../../../specs/abc.donna.md` - Good: `MUST not change public CLI flags` ## `Requirements` section @@ -216,7 +216,7 @@ Examples: - Bad: `- Verify that authentication works correctly.` - Bad: `- Review the implementation manually.` - Good: `- Run test suite `tests/auth/test_login.py`; all tests MUST pass.` -- Good: `- Inspect artifact `project:specs:authenticationd`; it MUST exist and contain section "Login flow".` +- Good: `- Inspect artifact `../../../../specs/authentication.donna.md`; it MUST exist and contain section "Login flow".` - Good: `- Execute CLI command `tool login` with invalid credentials; command MUST exit with non-zero code.` ## `Deliverables` section @@ -238,7 +238,7 @@ Examples: - Bad: `- Implement authentication code` - Bad: `- Refactor auth module.` - Good: `- Module app/auth/authentication.py exists.` -- Good: `- Donna artifact project:specs:authentication exists.` +- Good: `- Donna artifact ../../../../specs/authentication.donna.md exists.` - Good: `- Test suite tests/auth/ exists.` ## `Action items` section @@ -259,7 +259,7 @@ Examples: - Bad: `- Work on authentication.` - Bad: `- Improve security everywhere.` - Bad: `- Fix the bugs A` -- Good: `- Create an artifact project:specs:authentication with sections "Login flow" and "Token lifecycle".` +- Good: `- Create an artifact ../../../../specs/authentication.donna.md with sections "Login flow" and "Token lifecycle".` - Good: `- Add test file tests/auth/test_login.py covering invalid credential cases.` - Good: `- Implement test tests/auth/test_login.py:TestLogin:test_invalid_credentials.` - Good: `- Update CLI help text to include login command description.` diff --git a/donna/fixtures/specs/rfc/work/design.md b/.agents/donna/rfc/work/design.donna.md similarity index 77% rename from donna/fixtures/specs/rfc/work/design.md rename to .agents/donna/rfc/work/design.donna.md index 54ec9bb5..18b8bfeb 100644 --- a/donna/fixtures/specs/rfc/work/design.md +++ b/.agents/donna/rfc/work/design.donna.md @@ -5,7 +5,7 @@ kind = "donna.lib.workflow" start_operation_id = "start" ``` -This workflow creates a Design document artifact based on an RFC and aligned with `donna:rfc:specs:design`. +This workflow creates a Design document artifact based on an RFC and aligned with `../specs/design.donna.md`. ## Start Work @@ -15,8 +15,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_rfc_artifact_exists") }}` ## Ensure RFC artifact exists @@ -29,7 +29,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear RFC to design. 1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_artifact") }}`. -2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. +2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("@/.donna/session/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. 3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_artifact") }}`. ## Prepare Design artifact @@ -39,7 +39,7 @@ id = "prepare_design_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to be `session:design:`, where `` SHOULD correspond to the RFC slug. +1. If the name of the artifact is not specified explicitly, assume it to be `@/.donna/session/design/.donna.md`, where `` SHOULD correspond to the RFC slug. 2. Save the next template into the artifact, replace `` with appropriate values. ~~~ @@ -81,7 +81,7 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. 2. Read the RFC artifact selected in the previous step if you haven't done it yet. 3. Analyze the project if needed to understand the requested change context. 4. Fill in all sections of the Design draft artifact. @@ -95,7 +95,7 @@ id = "review_design_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("donna:rfc:specs:design") }}`. +1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("../specs/design.donna.md") }}`. 2. For each mismatch, make necessary edits to the Design draft artifact to ensure compliance. 3. `{{ donna.lib.goto("review_design_content") }}` @@ -106,7 +106,7 @@ id = "review_design_content" kind = "donna.lib.request_action" ``` -1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. 2. Make necessary edits to the Design draft artifact to address identified issues. 3. If there were changes made on this step or the previous `review_design_format` step `{{ donna.lib.goto("review_design_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/donna/fixtures/specs/rfc/work/do.md b/.agents/donna/rfc/work/do.donna.md similarity index 94% rename from donna/fixtures/specs/rfc/work/do.md rename to .agents/donna/rfc/work/do.donna.md index c39d9294..1af618a3 100644 --- a/donna/fixtures/specs/rfc/work/do.md +++ b/.agents/donna/rfc/work/do.donna.md @@ -76,7 +76,7 @@ kind = "donna.lib.request_action" 1. Choose the workflow to plan the work. If you created a Design document in the previous step, use it as a basis. 2. Run the chosen workflow. -3. Ensure you know the workflow id created in the previous step (default is `session:execute_rfc` if not specified). +3. Ensure you know the workflow id created in the previous step (default is `@/.donna/session/execute_rfc.donna.md` if not specified). 4. After completing the workflow `{{ donna.lib.goto("execute_rfc_work") }}`. ## Execute RFC work @@ -86,7 +86,7 @@ id = "execute_rfc_work" kind = "donna.lib.request_action" ``` -1. Run the workflow created by the plan step (default: `session:execute_rfc`) and complete it. +1. Run the workflow created by the plan step (default: `@/.donna/session/execute_rfc.donna.md`) and complete it. 2. After completing the workflow `{{ donna.lib.goto("polish_changes") }}`. ## Polish changes diff --git a/donna/fixtures/specs/rfc/work/plan.md b/.agents/donna/rfc/work/plan.donna.md similarity index 88% rename from donna/fixtures/specs/rfc/work/plan.md rename to .agents/donna/rfc/work/plan.donna.md index 180ab8dd..d620889f 100644 --- a/donna/fixtures/specs/rfc/work/plan.md +++ b/.agents/donna/rfc/work/plan.donna.md @@ -6,7 +6,7 @@ kind = "donna.lib.workflow" start_operation_id = "start" ``` -This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow in the `session:*` world with detailed steps to implement the designed changes. +This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow stored as a `@/.donna/session/**` artifact under `/.donna/session` with detailed steps to implement the designed changes. ## Start Work @@ -18,7 +18,7 @@ fsm_mode = "start" 1. Read the Design document that the developer or parent workflow wants you to implement. 2. Read the RFC document that the developer or parent workflow wants you to implement, if it exists. -3. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +3. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 4. `{{ donna.lib.goto("prepare_workflow_artifact") }}` ## Prepare workflow artifact @@ -28,7 +28,7 @@ id = "prepare_workflow_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to `session:plans:`. +1. If the name of the artifact is not specified explicitly, assume it to `@/.donna/session/plans/.donna.md`. 2. Create a workflow with the next operations: - Start - A step for each action point in the RFC document and each item in the `Order of implementation` in Design document with the goal to minimize dependencies between steps and introduce changes incrementally. diff --git a/.agents/donna/rfc/work/request.md b/.agents/donna/rfc/work/request.donna.md similarity index 77% rename from .agents/donna/rfc/work/request.md rename to .agents/donna/rfc/work/request.donna.md index 4a612ba2..f66d58a3 100644 --- a/.agents/donna/rfc/work/request.md +++ b/.agents/donna/rfc/work/request.donna.md @@ -16,8 +16,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_work_description_exists") }}` ## Ensure work description exists @@ -30,7 +30,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear description of the problem in your context. I.e. you know what you need to do in this workflow. 1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("@/.donna/session/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. 3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. ## Prepare RFC artifact @@ -40,7 +40,7 @@ id = "prepare_rfc_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to be `session:rfc:`, where `` MUST be unique within the session. +1. If the name of the artifact is not specified explicitly, assume it to be `@/.donna/session/rfc/.donna.md`, where `` MUST be unique within the session. 2. Save the next template into the artifact, replace `` with appropriate values. ~~~ @@ -86,7 +86,7 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. 2. Analyze the project if needed to understand the context of the requested change. 3. Based on the problem description you have, fill in all sections of the RFC draft artifact. 4. `{{ donna.lib.goto("review_rfc_format") }}` @@ -98,7 +98,7 @@ id = "review_rfc_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}`. +1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}`. 2. For each mismatch, make necessary edits to the RFC draft artifact to ensure compliance with the RFC specification. 3. `{{ donna.lib.goto("review_rfc_content") }}` @@ -109,7 +109,7 @@ id = "review_rfc_content" kind = "donna.lib.request_action" ``` -1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. 2. Make necessary edits to the RFC draft artifact to address identified issues. 3. If there were changes made on this step or the previous `review_rfc_format` step `{{ donna.lib.goto("review_rfc_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/donna/fixtures/specs/usage/artifacts.md b/.agents/donna/usage/artifacts.donna.md similarity index 98% rename from donna/fixtures/specs/usage/artifacts.md rename to .agents/donna/usage/artifacts.donna.md index b87059b2..ac7181ba 100644 --- a/donna/fixtures/specs/usage/artifacts.md +++ b/.agents/donna/usage/artifacts.donna.md @@ -22,7 +22,7 @@ To get information from the artifact, developers, agents and Donna view one of i **If you need an information from the artifact, you MUST view its representation**. Artifact sources are only for editing. -Read the specification `{{ donna.lib.view("donna:usage:cli") }}` to learn how to work with artifacts via Donna CLI. +Read the specification `{{ donna.lib.view("./cli.donna.md") }}` to learn how to work with artifacts via Donna CLI. ## Source Format and Rendering @@ -117,7 +117,7 @@ Artifacts can include semantic tags via a `tags` field in the section configurat Tags are used for deterministic artifact filtering and discovery (for example, via `donna -p artifacts list ... --predicate '"workflow" in section.tags'`). Tags are typically attached to the primary section and describe the artifact as a whole. -The canonical list of standard tags is documented in `donna:intro`. +The canonical list of standard tags is documented in `../intro.donna.md`. ## Section Kinds, Their Formats and Behaviors diff --git a/donna/fixtures/specs/usage/cli.md b/.agents/donna/usage/cli.donna.md similarity index 88% rename from donna/fixtures/specs/usage/cli.md rename to .agents/donna/usage/cli.donna.md index bf21ad4b..a50cc5b9 100644 --- a/donna/fixtures/specs/usage/cli.md +++ b/.agents/donna/usage/cli.donna.md @@ -26,7 +26,6 @@ We may need coding agents on the each step of the process, but there no reason f ## Primary rules for agents -- Donna stores all project-related data in `.donna` directory in the project root. - All work is always done in the context of a session. There is only one active session at a time. - You MUST always work on one task assigned to you. - You MUST keep all the information about the session in your memory. @@ -109,7 +108,7 @@ After the session starts you MUST follow the next workflow to perform your work: 3. Start chosen workflow by calling `donna -p sessions run `. 4. Donna will output descriptions of all operations it performs to complete the work. 5. Donna will output **action requests** that you MUST perform. You MUST follow these instructions precisely. -6. When you done processing an action request, call `donna -p sessions action-request-completed ` to report request completion. `` MUST contain full identifier of the next operation, like `::`. +6. When you done processing an action request, call `donna -p sessions action-request-completed ` to report request completion. `` MUST contain the full identifier of the next operation, for example `@/.donna/session/execute_rfc.donna.md:review_changes`. 7. After you complete an action request, Donna will continue workflow execution and output what you need to do next. You MUST continue following Donna's instructions until the workflow is completed. @@ -138,28 +137,30 @@ If Donna tells you there is no work left, you MUST inform the developer that the ### Working with artifacts -An artifact is a markdown document with extra metadata stored in one of the Donna's worlds. +An artifact is a markdown document with extra metadata stored in the project workspace. Use the next commands to work with artifacts: -- `donna -p artifacts list []` — list all artifacts corresponding to the given pattern. If `` is omitted, list all artifacts in all worlds. Use this command when you need to find an artifact or see what artifacts are available. +- `donna -p artifacts list []` — list all artifacts corresponding to the given pattern. If `` is omitted, list all artifacts in the project workspace. Use this command when you need to find an artifact or see what artifacts are available. - `donna -p artifacts view ` — get the meaningful (rendered) content of all matching artifacts. This command shows the rendered information about each artifact. Use this command when you need to read artifact content. -- `donna -p artifacts validate []` — validate all artifacts corresponding to the given pattern. If `` is omitted, validate all artifacts in all worlds. +- `donna -p artifacts validate []` — validate all artifacts corresponding to the given pattern. If `` is omitted, validate all artifacts in the project workspace. -Donna does not mutate artifacts stored in worlds. Developers and external tools are responsible for creating, updating, moving, copying, or deleting world artifacts before Donna reads or validates them. +Donna does not mutate artifacts stored in the project workspace. Developers and external tools are responsible for creating, updating, moving, copying, or deleting artifacts before Donna reads or validates them. Commands that accept an artifact pattern (`artifacts list`, `artifacts view`, `artifacts validate`) also accept `--predicate/-p ` to filter by artifact primary section. The expression is evaluated as `bool` with `section` global available (for example: `--predicate '"workflow" in section.tags'`). The format of `` is as follows: -- full artifact identifier: `:` -- `*` — single wildcard matches a single level in the artifact path. Examples: - - `*:artifact:name` — matches all artifacts named `artifact:name` in all worlds. - - `world:*:name` — matches all artifacts with id `something:name` in the `world` world. -- `**` — double wildcard matches multiple levels in the artifact path. Examples: - - `**:name` — matches all artifacts with id ending with `:name` in all worlds. - - `world:**` — matches all artifacts in the `world` world. - - `world:**:name` — matches all artifacts with id ending with `:name` in the `world` world. +- full artifact identifier: `@/...` +- `*` — single wildcard matches a single level in the rooted artifact path. Examples: + - `*/intro.donna.md` — matches all artifacts with filename `intro.donna.md` exactly one directory below the project root. + - `@/*/intro.donna.md` — equivalent full form. +- `**` — double wildcard matches multiple levels in the rooted artifact path. Examples: + - `**/name.donna.md` — matches all artifacts with filename `name.donna.md` anywhere in the project workspace. + - `@/**/intro.donna.md` — equivalent full form. + - `@/.donna/**` — matches all artifacts under `.donna`. + +CLI arguments MUST NOT use relative artifact paths like `./...` or `../../...`; use absolute `@/...` paths or rooted wildcard forms. ### Working with journal @@ -206,7 +207,7 @@ Agents MUST NOT log: 1. Direct instructions from the developer. 2. `AGENTS.md` document. - 3. Specifications in `project:` world. + 3. Project-relative specifications under `../../../specs/**` or `../**`. 4. This document. **All Donna CLI commands MUST include an explicit protocol selection using `-p `.** Like `donna -p llm `. diff --git a/.agents/donna/usage/worlds.donna.md b/.agents/donna/usage/worlds.donna.md new file mode 100644 index 00000000..a37627a1 --- /dev/null +++ b/.agents/donna/usage/worlds.donna.md @@ -0,0 +1,42 @@ +# Donna World Layout + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes how Donna discovers and manages its project artifacts. +Including usage docs, work workflows, operations, current work state and additional code. + +## Overview + +In order to function properly and to perform in a full potential, Donna relies on a set of artifacts +that guide its behavior and provide necessary capabilities. + +These artifacts are represented as text files, primary in Markdown format, however other text-based +formats can be used as well, if explicitly requested by the developer or by the workflows. + +Donna discovers these artifacts in a single built-in project world rooted at ``. +The project world is a singleton object configured in code and backed by the project's filesystem. +Donna does not read world definitions from `/.donna/config.toml`. + +The project world and its primary artifact areas are: + +- `../../../specs/**` — artifacts under `/specs`, owned by the project itself. +- `../**` — synced Donna usage specs and workflows under `/.agents/donna`. +- `@/.donna/session/**` — session artifacts under `/.donna/session`. + +The project world has a free layout, defined by the developers who own the project. + +## Artifact Access + +Donna has read access to artifacts stored in the project world. It discovers, fetches, renders, and validates project artifacts, but it does not create, update, move, copy, or delete them. + +Developers and external tools are responsible for mutating project artifacts before Donna reads or validates them. + +Donna still writes its own session state and journal data under `/.donna/session`, but that internal state storage is separate from world-artifact mutation. + +## Intro Artifacts + +It is a recommended practice to provide short introductory artifacts such as `../intro.donna.md` and `../../../specs/intro.donna.md` at meaningful roots inside the project world. + +So, the agent can load the relevant introductions in commands such as `donna -p llm artifacts view '**/intro.donna.md'`. diff --git a/.agents/donna/usage/worlds.md b/.agents/donna/usage/worlds.md deleted file mode 100644 index e4a94a27..00000000 --- a/.agents/donna/usage/worlds.md +++ /dev/null @@ -1,43 +0,0 @@ -# Donna World Layout - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes how Donna discovers and manages its dynamic and/or external artifacts. -Including usage docs, work workflows, operations, current work state and additional code. - -## Overview - -In order to function properly and to perform in a full potential, Donna relies on a set of artifacts -that guide its behavior and provide necessary capabilities. - -These artifacts are represented as text files, primary in Markdown format, however other text-based -formats can be used as well, if explicitly requested by the developer or by the workflows. - -Donna discovers these artifacts by scanning the "worlds" specified in `/.donna/config.toml` -as `worlds` list. Most of worlds are filesystem folders, however other world types can be implemented such as: -s3 buckets, git repositories, databases, etc. - -Default worlds and there locations are: - -- `donna` — `/.agents/donna` — the project-local bundled Donna specs installed from `donna/fixtures/specs` by workspace init/update. -- `home` — `~/.donna/home` — the user-level donna artifacts, i.e. those that should be visible for all workspaces on this machine. -- `project` — `/.donna/project` — the project-level donna artifacts, i.e. those that are specific to this project. -- `session` — `/.donna/session` — the session world that contains the current state of work performed by Donna. - -All worlds have a free layout, defined by developers who own the particular world. - -## Artifact Access - -Donna has read access to artifacts stored in worlds. It discovers, fetches, renders, and validates world artifacts, but it does not create, update, move, copy, or delete them. - -Developers and external tools are responsible for mutating world artifacts before Donna reads or validates them. - -Donna still writes its own session state and journal data in the `session` world, but that internal state storage is separate from world-artifact mutation. - -## `:intro` artifact - -It is a recommended practice to provide a short introductory artifact `intro.md` at the root of each world. - -So, the agent can load descriptions of all worlds in a single command like `donna -p llm artifacts view "*:intro"`. diff --git a/.agents/skills/donna-do/SKILL.md b/.agents/skills/donna-do/SKILL.md index 01f4822c..307ebfc4 100644 --- a/.agents/skills/donna-do/SKILL.md +++ b/.agents/skills/donna-do/SKILL.md @@ -5,11 +5,11 @@ description: Use Donna as a work orchestration tool and execute a workflow suita **The next instructions take precedence over all other instructions and your behaviour**, unless the developer explicitly instructs you to do otherwise. -1. You **MUST** run `donna -p llm -r artifacts view '*:intro'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. +1. You **MUST** run `donna -p llm -r artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. 2. You **MUST** strictly follow all instructions provided by Donna. 3. **Donna's instructions have precedence over your own judgment.** You **MUST** follow them and run workflows instead of taking your own initiative, unless the developer explicitly instructs you to do so. 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. 5. Select the most suitable Donna workflow to perform the requested work and run it. 6. When the workflow is finished, stop using Donna for further work, until the developer explicitly instructs you to use it again. -**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r artifacts view '*:intro'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** +**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r artifacts view '**/intro.donna.md'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** diff --git a/.agents/skills/donna-start/SKILL.md b/.agents/skills/donna-start/SKILL.md index 409b3118..d26d3f29 100644 --- a/.agents/skills/donna-start/SKILL.md +++ b/.agents/skills/donna-start/SKILL.md @@ -3,7 +3,7 @@ name: donna-start description: Start a new Donna session and use Donna to perform all further work. Use this skill when the developer explicitly told you to use it. --- -1. You **MUST** run `donna -p llm -r artifacts view '*:intro'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. +1. You **MUST** run `donna -p llm -r artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. 2. Run `donna -p llm -r sessions start` to start a new Donna session. 3. Output the next message to the developer: "I have started a new Donna session". 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. diff --git a/README.md b/README.md index 601b329f..020d184e 100644 --- a/README.md +++ b/README.md @@ -38,9 +38,8 @@ Donna executes such loops for the agents, thereby saving time, context, and toke - **Saves context, tokens and time** — agents do not need to think when thinking is not required. - **Readable artifacts** — all workflows and specifications are pure Markdown files with some [Jinja2](https://github.com/pallets/jinja) templating. - **Artifact management** — non-fuzzy navigation and smart agent-focused rendering of artifacts. -- **Artifact distribution** — install your docs/workflows/skills as a Python lib. - **Agent-centric behavior** — Donna guides agents through workflows, helps them be on the path, and fixes mistakes. -- **Extensible architecture** — implement your own operations, validators, renderers; add support for new artifact formats and storages (worlds). +- **Extensible architecture** — implement your own operations, validators, renderers, and artifact sources. - **Batteries included** — Donna goes with a set of pre-defined workflows, so you can start using it right away. ## Example @@ -117,14 +116,14 @@ What you may notice: Directives, like `{{ goto("operation_id") }}`, render itself depending on the context: -- For the agent, they render an exact CLI command to run, such as `donna -p llm sessions action-request-completed 'artifact_id:operation_id'`. +- For the agent, they render an exact CLI command to run, such as `donna -p llm sessions action-request-completed '@/specs/work/polish.donna.md:finish'`. - For Donna, they render a specific marker that can be extracted and used to analyze an artifact. For example, Donna uses `goto` directives to build an FSM of the workflow and validate it before running: does each operation exist, can the workflow be completed, are there unreachable operations, etc. Generally speaking, **all you need is `donna.lib.request_action` operation** — it is enough to achieve a great deal of automation by delegating some decisions to the agent. However, there are some more specific operations that simplify things and make workflows more agile or performant. -You can find a more complex implementation of the same workflow in the [polish.md](./specs/work/polish.md) file. It demonstrates other Donna operations, such as running scripts directly and branching. +You can find a more complex implementation of the same workflow in the [polish.donna.md](./specs/work/polish.donna.md) file. It demonstrates other Donna operations, such as running scripts directly and branching. ## Installation @@ -146,8 +145,11 @@ donna workspaces init Donna will: - Create a `.donna/` folder in your project root with a default configuration in `.donna/config.toml`. +- Sync bundled Donna specs into `.agents/donna/`. - Install skills into `.agents/skills/` folder. +If you upgrade Donna later, run `donna workspaces update` to refresh `.agents/donna/` and `.agents/skills/`. + 3. Ask your agent to do something like `$donna-do Add a button that …`. The agent will discover the appropriate workflow and execute it. ## Skills @@ -177,9 +179,9 @@ Use `donna --help` for a quick reference. You find detailed documentation in the agent instructions — they are readable and always accurate: -- [CLI specification](./donna/artifacts/usage/cli.md) — full list of commands and how to use them. -- [Artifacts](./donna/artifacts/usage/artifacts.md) — what are Donna artifact and how to use them. -- [Worlds](./donna/artifacts/usage/worlds.md) — how Donna discovers and manages its artifacts. +- [CLI specification](./.agents/donna/usage/cli.donna.md) — full list of commands and how to use them. +- [Artifacts](./.agents/donna/usage/artifacts.donna.md) — what Donna artifacts are and how to use them. +- [Filesystem layout](./.agents/donna/usage/worlds.donna.md) — how Donna discovers and manages artifacts on the filesystem. The documentation below covers aspects important to humans and partially duplicates the agent's instructions. @@ -205,32 +207,21 @@ Note that the default Donna workflows are designed to be reliable and useful for Points of interest: -- [donna:rfc:specs:request_for_change](./donna/artifacts/rfc/specs/request_for_change.md) — specification of the RFC document. -- [donna:rfc:work:request](./donna/artifacts/rfc/work/request.md) — workflow to create a RFC document. -- [donna:rfc:work:plan](./donna/artifacts/rfc/work/plan.md) — workflow to plan work on an RFC — creates a new workflow. -- [donna:rfc:work:do](./donna/artifacts/rfc/work/do.md) — meta workflow to automate the whole work from a developer request to a changelog update. - -## Artifacts and Worlds - -- Artifacts are something that Donna owns. Currently, they are Markdown files with workflows and specifications. -- Worlds are storages for artifacts. Currently, Donna supports two types of worlds: - - `donna.lib.worlds.filesystem` — a folder on the filesystem. - - `donna.lib.worlds.python` — a Python package. - -**Please, tell if you need other world types.** It looks interesting to have `http`, `s3`, `git`, `sql` worlds. How about an `email` world that allows you to send a workflow to someone agent and get the results back in your mailbox? - -A world ID is the first part of an artifact ID. For example, `donna:usage:cli` artifact resides in the `donna` world. +- [@/.agents/donna/rfc/specs/request_for_change.donna.md](./.agents/donna/rfc/specs/request_for_change.donna.md) — specification of the RFC document. +- [@/.agents/donna/rfc/work/request.donna.md](./.agents/donna/rfc/work/request.donna.md) — workflow to create a RFC document. +- [@/.agents/donna/rfc/work/plan.donna.md](./.agents/donna/rfc/work/plan.donna.md) — workflow to plan work on an RFC and create a new workflow. +- [@/.agents/donna/rfc/work/do.donna.md](./.agents/donna/rfc/work/do.donna.md) — meta workflow to automate the whole work from a developer request to a changelog update. -By default, Donna uses the next worlds: +## Artifacts on Filesystem -- `donna` — artifacts provided by Donna itself; -- `home` — user-level artifacts in `/.donna/` folder; -- `project` — project-level artifacts in `/specs/` folder; -- `session` — session-level artifacts in `/.donna/session/` folder. +- Artifacts are text files Donna reads and validates. In practice they are usually Markdown workflows and specifications stored as `.donna.md` files. +- Donna discovers artifacts directly in the project filesystem and limits what is visible via `.donna/config.toml:file_filters`. -A world can be read-only. By default, writable worlds are `session` (current work scope) and `project` (project scope). +By default, Donna uses these artifact areas: -Agents are not allowed to edit artifacts through Donna CLI directly because artifact consistency is important. Instead, they update the underlying files in writable worlds and rely on Donna validation when needed. +- `specs/` — project-owned artifacts. +- `.agents/donna/` — bundled Donna specs and workflows synced by `donna workspaces init` or `donna workspaces update`. +- `.donna/session/` — session artifacts and Donna runtime state. ### Rendering @@ -240,28 +231,37 @@ More about Jinja2 rendering is described a bit further. ### Artifacts Discovery -Artifact is identified by its id: `:`, for example, `donna:usage:cli`. +Artifact ids are project-relative filepaths prefixed with `@/`. Section ids append `:section_id` to the artifact id. -You and agents can `list` artifacts and `view` them. +Examples: + +- `@/specs/work/polish.donna.md` +- `@/.agents/donna/usage/cli.donna.md` +- `@/.donna/session/execute_rfc.donna.md:review_changes` -- `donna -p llm artifacts list ` — shows a short description from its h1 section. +You and agents can `list`, `view`, and `validate` artifacts. + +- `donna -p llm artifacts list []` — shows a short description from its h1 section. - `donna -p llm artifacts view ` — shows the full content of the artifact with proper rendering. +- `donna -p llm artifacts validate []` — validates matching artifacts. + +Commands accept both precise artifact ids and glob patterns. Patterns may use absolute ids like `@/...` or rooted wildcard forms like `**/intro.donna.md`. -Both commands accept both precise artifact ids and glob patterns. Patterns allow using: +Patterns allow using: -- `*` to replace a single path segment; -- `**` to replace multiple path segments. +- `*` — matches zero or more characters inside one path level. +- `?` — matches exactly one character inside one path level. +- `[]` — matches a single character class inside one path level. +- `**` — matches zero or more path levels. Examples: -- `*:artifact:name` — matches all artifacts named `artifact:name` in all worlds. -- `world:*:name` — matches all artifacts with id `something:name` in the `world` world. -- `**:name` — matches all artifacts with id ending with `:name` in all worlds. -- `world:**` — matches all artifacts in the `world` world. -- `world:**:name` — matches all artifacts with id ending with `:name` in the `world` world. +- `@/*.donna.md` — matches all artifacts directly under the project root. +- `@/**/intro.donna.md` — matches all artifacts named `intro.donna.md`. +- `@/.donna/session/**` — matches all session artifacts. +- `**/test_*.donna.md` — matches artifact filenames that start with `test_`. -Both commands also accept `--predicate/-p ''` option to filter artifacts by -their primary section properties (available as `section` inside expression). +Commands that accept an artifact pattern also support `--predicate ''` to filter by primary section properties available as `section`. Currently, Donna supports two artifact tags: @@ -269,15 +269,15 @@ Currently, Donna supports two artifact tags: - `specification` — marks a specification artifact — is set automatically by Donna. You can find all workflows with the command -`donna -p llm artifacts list '**' --predicate '"workflow" in section.tags'`. +`donna -p llm artifacts list --predicate '"workflow" in section.tags'`. ## Sessions -`session` world contains the current state of work performed by Donna: all documents and workflows that are created during work and should not be stored permanently in the project. +`/.donna/session/` contains the current state of work performed by Donna: runtime state plus temporary documents and workflows created during the session. The developer is responsible for starting/resetting sessions with commands from `donna -p human sessions` group. -- On session start, Donna removes everything from the previous session and creates a fresh `session` world. +- On session start, Donna removes everything from the previous session and creates a fresh session directory. - On session reset donna resets the state of the current session (tasks, action requests, etc.), but keeps artifacts. The agent is encouraged not to manage sessions directly, because it doesn't have enough context to decide when session artifacts may be safely removed. @@ -303,7 +303,7 @@ To execute a workflow, Donna uses a simplified virtual machine (VM) that maintai ### Operations -You can find detailed docs on built-in operations in the [artifacts documentation](./donna/artifacts/usage/artifacts.md). +You can find detailed docs on built-in operations in the [artifacts documentation](./.agents/donna/usage/artifacts.donna.md). Here is a short list of them: @@ -320,15 +320,15 @@ Donna can detect errors (in artifacts, in execution, etc). If an error can be fi An example of error message from Donna ```bash -$ donna -p llm sessions run project:work:polish +$ donna -p llm sessions run @/specs/work/polish.donna.md kind=artifact_validation_error media_type=text/markdown -artifact_id=project:work:polish +artifact_id=@/specs/work/polish.donna.md error_code=donna.artifacts.section_not_found section_id=run_autoflake_scriptx -Error in artifact 'project:work:polish', section 'run_autoflake_scriptx': Section `run_autoflake_scriptx` is not available in artifact `project:work:polish`. +Error in artifact '@/specs/work/polish.donna.md', section 'run_autoflake_scriptx': Section `run_autoflake_scriptx` is not available in artifact `@/specs/work/polish.donna.md`. Ways to fix: @@ -352,11 +352,11 @@ The simplest example of such generation is currently used as a primary way for D ### Discovering workflows -If you want to run a child workflow from an operation, you can just instruct an agent like `Run the workflow project:work:my-cool-workflow` and the agent will find it and run. +If you want to run a child workflow from an operation, you can just instruct an agent like `Run the workflow @/specs/work/my-cool-workflow.donna.md` and the agent will find it and run. However, it is not very agile. Instead, I suggest you describe the desired outcome and let the agent find the most suitable workflow. In that case, you'll be able to define customized workflows for specific types of changes and let the agent choose the best one for the current situation. -For example, you can have two workflows `project:work:write-backend-test` and `project:work:write-frontend-test`, and your operation can say `Run the workflow that will write a test for the current change`, and the agent will choose the most suitable workflow based on the context and the workflow descriptions. +For example, you can have two workflows `@/specs/work/write-backend-test.donna.md` and `@/specs/work/write-frontend-test.donna.md`, and your operation can say `Run the workflow that will write a test for the current change`, and the agent will choose the most suitable workflow based on the context and the workflow descriptions. ## Jinja2 rendering @@ -378,7 +378,7 @@ Donna defines a set of built-in Jinja2 functions that provide artifacts with the Directives are used in the next way: `{{ python.import.path() }}`. -You can find a detailed documentation of all built-in directives in the [artifacts documentation](./donna/artifacts/usage/artifacts.md). +You can find a detailed documentation of all built-in directives in the [artifacts documentation](./.agents/donna/usage/artifacts.donna.md). Here they are: @@ -399,35 +399,17 @@ All Donna logic is referenced by Python import paths. That means: - You can implement your own functionality and use it with Donna. - You can enrich your Python packages with additional code to work with Donna. -- You can distribute your Donna artifacts as Python packages. What you can implement: - Custom sections (including operations) for Donna artifacts. Check [./donna/primitives/artifacts](./donna/primitives/artifacts) and [./donna/primitives/operations](./donna/primitives/operations) subpackages for examples. - Custom rendering directives. Check [./donna/primitives/directives](./donna/primitives/directives) subpackage for examples. -- Custom worlds. Check [./donna/workspaces/worlds](./donna/workspaces/worlds) subpackage for examples. -- Custom parsers for artifacts. Check [./donna/workspaces/sources](./donna/workspaces/sources) subpackage for examples. +- Custom artifact parsers. Check [./donna/workspaces/sources](./donna/workspaces/sources) subpackage for examples. -Worlds and sources are configured in the `.donna/config.toml` file of your project. +Sources and file filters are configured in the `.donna/config.toml` file of your project. Sections and directives are used directly in artifacts by their Python import paths. -## Distribute Your Artifacts - -Since Donna world can be a Python lib, you can distribute your artifacts as a Python package. - -To define a Donna world in you package you must place a variable `donna_artifacts_root` in your package `__init__.py` file with a import path to the root subpackage with your artifacts. - -On the example of Donna: - -```python -donna_artifacts_root = "donna.artifacts" -``` - -After that, you can install your package and add the world into the `.donna/config.toml` file of your project. - -Since you distribute text files, **you package is not dependent on Donna itself — no additional dependencies are required.** - ## Feedback wanted Donna is still young and has multiple experimental features — I really appreciate any feedback, ideas, and contributions to make it better. diff --git a/bin/donna.sh b/bin/donna.sh index 57b81fcf..927ab150 100755 --- a/bin/donna.sh +++ b/bin/donna.sh @@ -5,5 +5,4 @@ # exec .venv-donna/bin/donna "$@" ROOT_DIR="$(cd "$(dirname "$0")/.."; pwd)" -cd "$ROOT_DIR" -poetry run donna "$@" +poetry -P "$ROOT_DIR" run donna "$@" diff --git a/changes/unreleased.md b/changes/unreleased.md index 4c6022ea..1772c1e4 100644 --- a/changes/unreleased.md +++ b/changes/unreleased.md @@ -3,9 +3,12 @@ - Move project-specific specs from `.donna/project` to `specs`, or set an explicit `project` world path in your Donna workspace config before upgrading. - Run `donna workspaces update` in existing projects so bundled Donna specs are installed into `.agents/donna` for the new filesystem-backed `donna` world. - Update your scripts and specs to use external tools or direct file edits to create, update, move, copy, or delete world artifacts instead using removed Donna commands. +- Update artifact references from legacy ids like `specs:intro` to filepath ids like `@/specs/intro.md`, and include file extensions on all artifact references. ### Changes +- Added configurable artifact file filters. Donna will see only files that pass all of the filters. +- Replaced artifact ids with project-relative filepaths like `@/specs/intro.md` and `@/.donna/session/plans/plan.md:finish`. - Changed the default location of project specs to `specs/`. - Updated the default `project` world path to load from `specs/` instead of `.donna/project/`. - Rewrote the moved project specs and repository docs to reference the new `specs/` location. @@ -21,6 +24,7 @@ ### Breaking Changes +- Donna artifact ids now use project-relative filepaths with required file extensions, and legacy colon-delimited artifact ids are no longer supported. - Donna no longer exposes bundled specs through the Python-backed `donna` world; `donna workspaces init|update` now sync them into `.agents/donna`. - `donna artifacts` no longer supports `update`, `copy`, `move`, or `remove`. - Donna no longer mutates world artifacts through workspace APIs or world configuration. diff --git a/donna/cli/commands/artifacts.py b/donna/cli/commands/artifacts.py index d74abb76..f8536337 100644 --- a/donna/cli/commands/artifacts.py +++ b/donna/cli/commands/artifacts.py @@ -4,12 +4,13 @@ from donna.cli.application import app from donna.cli.types import ( - FullArtifactIdPatternArgument, + ArtifactIdPatternArgument, PredicateOption, + validate_supported_artifact_pattern, ) from donna.cli.utils import cells_cli from donna.context.context import context -from donna.domain.ids import FullArtifactIdPattern +from donna.domain.artifact_ids import ArtifactIdPattern from donna.machine import journal as machine_journal from donna.protocol.cell_shortcuts import operation_succeeded from donna.protocol.cells import Cell @@ -17,7 +18,7 @@ artifacts_cli = typer.Typer() -DEFAULT_ARTIFACT_PATTERN = FullArtifactIdPattern.parse("**").unwrap() +DEFAULT_ARTIFACT_PATTERN = ArtifactIdPattern.parse("**").unwrap() def _log_artifact_operation(message: str) -> None: @@ -26,7 +27,7 @@ def _log_artifact_operation(message: str) -> None: def _log_operation_on_artifacts( message: str, - pattern: FullArtifactIdPattern, + pattern: ArtifactIdPattern, predicate: PredicateOption | None, ) -> None: if predicate is None: @@ -36,13 +37,17 @@ def _log_operation_on_artifacts( @artifacts_cli.command( - help="List artifacts matching a pattern and show their status summaries. Lists all all artifacts by default." + help=( + "List artifacts matching a pattern with a supported source extension " + "and show their status summaries. Lists all artifacts by default." + ) ) @cells_cli def list( - pattern: FullArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, + pattern: ArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, predicate: PredicateOption = None, ) -> Iterable[Cell]: + validate_supported_artifact_pattern(pattern) _log_operation_on_artifacts("List artifacts", pattern, predicate) artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() @@ -50,24 +55,33 @@ def list( return [artifact.node().status() for artifact in artifacts] -@artifacts_cli.command(help="Displays artifacts matching a pattern or a specific id") +@artifacts_cli.command( + help="Display artifacts matching a pattern or specific id that uses a supported source extension." +) @cells_cli def view( - pattern: FullArtifactIdPatternArgument, + pattern: ArtifactIdPatternArgument, predicate: PredicateOption = None, ) -> Iterable[Cell]: + validate_supported_artifact_pattern(pattern) _log_operation_on_artifacts("View artifacts", pattern, predicate) artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() return [artifact.node().info() for artifact in artifacts] -@artifacts_cli.command(help="Validate artifacts matching a pattern (defaults to all artifacts) and return any errors.") +@artifacts_cli.command( + help=( + "Validate artifacts matching a pattern with a supported source extension " + "(defaults to all artifacts) and return any errors." + ) +) @cells_cli def validate( - pattern: FullArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, + pattern: ArtifactIdPatternArgument = DEFAULT_ARTIFACT_PATTERN, predicate: PredicateOption = None, ) -> Iterable[Cell]: # noqa: CCR001 + validate_supported_artifact_pattern(pattern) _log_operation_on_artifacts("Validate artifacts", pattern, predicate) artifacts = context().artifacts.list(pattern, RENDER_CONTEXT_VIEW, predicate=predicate).unwrap() @@ -88,5 +102,5 @@ def validate( app.add_typer( artifacts_cli, name="artifacts", - help="Inspect and validate stored artifacts across all Donna worlds.", + help="Inspect and validate stored artifacts in the project workspace.", ) diff --git a/donna/cli/commands/sessions.py b/donna/cli/commands/sessions.py index f765e8bf..66cc0ab8 100644 --- a/donna/cli/commands/sessions.py +++ b/donna/cli/commands/sessions.py @@ -3,7 +3,13 @@ import typer from donna.cli.application import app -from donna.cli.types import ActionRequestIdArgument, FullArtifactIdArgument, FullArtifactSectionIdArgument +from donna.cli.types import ( + ActionRequestIdArgument, + ArtifactIdArgument, + ArtifactSectionIdArgument, + validate_supported_artifact_id, + validate_supported_artifact_section_id, +) from donna.cli.utils import cells_cli from donna.machine import sessions from donna.protocol.cells import Cell @@ -46,7 +52,8 @@ def details() -> Iterable[Cell]: @sessions_cli.command(help="Run a workflow from an artifact to drive the current session forward.") @cells_cli -def run(workflow_id: FullArtifactIdArgument) -> Iterable[Cell]: +def run(workflow_id: ArtifactIdArgument) -> Iterable[Cell]: + validate_supported_artifact_id(workflow_id) return sessions.start_workflow(workflow_id).unwrap() @@ -55,8 +62,9 @@ def run(workflow_id: FullArtifactIdArgument) -> Iterable[Cell]: ) @cells_cli def action_request_completed( - request_id: ActionRequestIdArgument, next_operation_id: FullArtifactSectionIdArgument + request_id: ActionRequestIdArgument, next_operation_id: ArtifactSectionIdArgument ) -> Iterable[Cell]: + validate_supported_artifact_section_id(next_operation_id) return sessions.complete_action_request(request_id, next_operation_id).unwrap() diff --git a/donna/cli/types.py b/donna/cli/types.py index 4986b18e..855c57af 100644 --- a/donna/cli/types.py +++ b/donna/cli/types.py @@ -1,50 +1,124 @@ import pathlib -from typing import Annotated +from typing import Annotated, NoReturn import typer from donna.cli.utils import output_cells from donna.core.errors import ErrorsList -from donna.domain.ids import ( - ActionRequestId, - FullArtifactId, - FullArtifactIdPattern, - FullArtifactSectionId, -) +from donna.domain import errors as domain_errors +from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern, ArtifactSectionId +from donna.domain.internal_ids import ActionRequestId from donna.machine.artifacts import ArtifactPredicate from donna.protocol.modes import Mode +from donna.workspaces.config import config as workspace_config -def _exit_with_errors(errors: ErrorsList) -> None: +def _exit_with_errors(errors: ErrorsList) -> NoReturn: output_cells([error.node().info() for error in errors]) raise typer.Exit(code=0) -def _parse_full_artifact_id(value: str) -> FullArtifactId: - result = FullArtifactId.parse(value) - errors = result.err() +def _parse_result_or_exit[T](result: T | None, errors: ErrorsList | None) -> T: if errors is not None: _exit_with_errors(errors) - return result.unwrap() + assert result is not None + return result -def _parse_full_artifact_id_pattern(value: str) -> FullArtifactIdPattern: - result = FullArtifactIdPattern.parse(value) - errors = result.err() - if errors is not None: - _exit_with_errors(errors) +def _absolute_artifact_id_or_exit(value: str) -> str: + if not value.startswith(ArtifactId.prefix): + _exit_with_errors([domain_errors.InvalidIdFormat(id_type=ArtifactId.__name__, value=value)]) - return result.unwrap() + return value -def _parse_full_artifact_section_id(value: str) -> FullArtifactSectionId: - result = FullArtifactSectionId.parse(value) - errors = result.err() - if errors is not None: - _exit_with_errors(errors) +def _absolute_artifact_section_id_or_exit(value: str) -> str: + if not value.startswith(ArtifactId.prefix): + _exit_with_errors([domain_errors.InvalidIdFormat(id_type=f"{ArtifactSectionId.__name__} format", value=value)]) - return result.unwrap() + return value + + +def _absolute_artifact_pattern_or_exit(value: str | ArtifactIdPattern) -> str: + if isinstance(value, ArtifactIdPattern): + return str(value) + + first_part = value.split(ArtifactId.delimiter, maxsplit=1)[0] + if any(character in first_part for character in "*?[]"): + return f"{ArtifactId.prefix}{value}" + + if not value.startswith(ArtifactId.prefix): + _exit_with_errors([domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=value)]) + + return value + + +def _parse_artifact_id(value: str) -> ArtifactId: + result = ArtifactId.parse(_absolute_artifact_id_or_exit(value)) + return _parse_result_or_exit(result.ok(), result.err()) + + +def _parse_artifact_id_pattern(value: str | ArtifactIdPattern) -> ArtifactIdPattern: + if isinstance(value, ArtifactIdPattern): + return value + + result = ArtifactIdPattern.parse(_absolute_artifact_pattern_or_exit(value)) + return _parse_result_or_exit(result.ok(), result.err()) + + +def _parse_artifact_section_id(value: str) -> ArtifactSectionId: + result = ArtifactSectionId.parse(_absolute_artifact_section_id_or_exit(value)) + return _parse_result_or_exit(result.ok(), result.err()) + + +def _match_supported_extension(filename: str) -> str | None: + supported_extensions = workspace_config().supported_extensions() + normalized = filename.strip().lower() + + for extension in sorted(supported_extensions, key=len, reverse=True): + if normalized.endswith(extension): + return extension + + return None + + +def _artifact_filename(value: str) -> str: + return pathlib.PurePosixPath(value.split(ArtifactSectionId.delimiter, maxsplit=1)[0]).name + + +def _pattern_filename(value: str) -> str | None: + last_part = pathlib.PurePosixPath(value).name + if last_part in {"*", "**"}: + return None + + dot_index = last_part.find(".") + if dot_index == -1: + return None + + return f"placeholder{last_part[dot_index:]}" + + +def validate_supported_artifact_id(artifact_id: ArtifactId) -> None: + if _match_supported_extension(_artifact_filename(str(artifact_id))) is None: + raise typer.BadParameter( + f"Unsupported artifact extension for '{artifact_id}'. Use a filename extension supported by the sources." + ) + + +def validate_supported_artifact_pattern(pattern: ArtifactIdPattern) -> None: + filename = _pattern_filename(str(pattern)) + if filename is None: + return + + if _match_supported_extension(filename) is None: + raise typer.BadParameter( + f"Unsupported artifact extension for '{pattern}'. Use a filename extension supported by the sources." + ) + + +def validate_supported_artifact_section_id(section_id: ArtifactSectionId) -> None: + validate_supported_artifact_id(section_id.artifact_id) def _parse_artifact_predicate(value: str) -> ArtifactPredicate: @@ -96,20 +170,27 @@ def _parse_input_path(value: str) -> pathlib.Path: ] -FullArtifactIdArgument = Annotated[ - FullArtifactId, +ArtifactIdArgument = Annotated[ + ArtifactId, typer.Argument( - parser=_parse_full_artifact_id, - help="Full artifact ID in the form 'world:artifact[:path]' (e.g., 'project:intro').", + parser=_parse_artifact_id, + help=( + "Artifact ID in absolute project-root form with a supported source " + "extension (e.g., '@/specs/intro.donna.md')." + ), ), ] -FullArtifactIdPatternArgument = Annotated[ - FullArtifactIdPattern, +ArtifactIdPatternArgument = Annotated[ + ArtifactIdPattern, typer.Argument( - parser=_parse_full_artifact_id_pattern, - help="Artifact pattern (supports '*' and '**', e.g. 'project:*' or '**:intro').", + parser=_parse_artifact_id_pattern, + help=( + "Artifact pattern in absolute form '@/...' or rooted wildcard form like " + "'*/x.donna.md' and '**/x.donna.md'. Patterns that name a file " + "extension must use a supported source extension." + ), ), ] @@ -124,11 +205,14 @@ def _parse_input_path(value: str) -> pathlib.Path: ] -FullArtifactSectionIdArgument = Annotated[ - FullArtifactSectionId, +ArtifactSectionIdArgument = Annotated[ + ArtifactSectionId, typer.Argument( - parser=_parse_full_artifact_section_id, - help="Full artifact section ID in the form 'world:artifact:section'.", + parser=_parse_artifact_section_id, + help=( + "Artifact section ID in absolute project-root form 'artifact:section' " + "(e.g. '@/.donna/session/plans/artifact_id_filepaths.donna.md:finish')." + ), ), ] diff --git a/donna/context/artifacts.py b/donna/context/artifacts.py index 2e09a0bc..b79b69f9 100644 --- a/donna/context/artifacts.py +++ b/donna/context/artifacts.py @@ -3,14 +3,13 @@ from donna.context.entity_cache import TimedCache, TimedCacheValue from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactId, FullArtifactIdPattern, FullArtifactSectionId +from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern, ArtifactSectionId from donna.domain.types import Milliseconds from donna.machine.artifacts import Artifact, ArtifactPredicate, ArtifactSection from donna.workspaces.templates import RenderMode if TYPE_CHECKING: - from donna.workspaces.artifacts import ArtifactRenderContext - from donna.workspaces.worlds.base import RawArtifact + from donna.workspaces.artifacts import ArtifactRenderContext, FilesystemRawArtifact class _ArtifactCacheValue(TimedCacheValue): @@ -18,7 +17,7 @@ class _ArtifactCacheValue(TimedCacheValue): def __init__( self, - raw_artifact: "RawArtifact", + raw_artifact: "FilesystemRawArtifact", rendered_artifacts: dict[RenderMode, Artifact], loaded_at_ms: Milliseconds, checked_at_ms: Milliseconds, @@ -32,75 +31,73 @@ class ArtifactsCache(TimedCache): __slots__ = ("_cache",) def __init__(self) -> None: - self._cache: dict[FullArtifactId, _ArtifactCacheValue] = {} + self._cache: dict[ArtifactId, _ArtifactCacheValue] = {} @unwrap_to_error - def _is_cache_stale(self, full_id: FullArtifactId, loaded_at_ms: Milliseconds) -> Result[bool, ErrorsList]: - from donna.workspaces.config import config + def _is_cache_stale(self, artifact_id: ArtifactId, loaded_at_ms: Milliseconds) -> Result[bool, ErrorsList]: + from donna.workspaces.artifacts import has_artifact_changed - world = config().get_world(full_id.world_id).unwrap() - return Ok(world.has_artifact_changed(full_id.artifact_id, since=loaded_at_ms).unwrap()) + return Ok(has_artifact_changed(artifact_id, since=loaded_at_ms).unwrap()) @staticmethod @unwrap_to_error - def _load_raw_artifact(full_id: FullArtifactId) -> Result["RawArtifact", ErrorsList]: - from donna.workspaces.config import config + def _load_raw_artifact(artifact_id: ArtifactId) -> Result["FilesystemRawArtifact", ErrorsList]: + from donna.workspaces.artifacts import fetch_raw_artifact - world = config().get_world(full_id.world_id).unwrap() - return Ok(world.fetch(full_id.artifact_id).unwrap()) + return Ok(fetch_raw_artifact(artifact_id).unwrap()) @unwrap_to_error def _refresh_cache_value( - self, full_id: FullArtifactId, now_ms: Milliseconds + self, artifact_id: ArtifactId, now_ms: Milliseconds ) -> Result[_ArtifactCacheValue, ErrorsList]: - raw_artifact = self._load_raw_artifact(full_id).unwrap() + raw_artifact = self._load_raw_artifact(artifact_id).unwrap() refreshed = _ArtifactCacheValue( raw_artifact=raw_artifact, rendered_artifacts={}, loaded_at_ms=now_ms, checked_at_ms=now_ms, ) - self._cache[full_id] = refreshed + self._cache[artifact_id] = refreshed return Ok(refreshed) @unwrap_to_error - def _get_cache_value(self, full_id: FullArtifactId) -> Result[_ArtifactCacheValue, ErrorsList]: - cached = self._cache.get(full_id) + def _get_cache_value(self, artifact_id: ArtifactId) -> Result[_ArtifactCacheValue, ErrorsList]: + cached = self._cache.get(artifact_id) now_ms = self._now_ms() if cached is None: - return Ok(self._refresh_cache_value(full_id, now_ms).unwrap()) + return Ok(self._refresh_cache_value(artifact_id, now_ms).unwrap()) - # Skip expensive world checks when cache lifetime has not elapsed yet. + # Skip expensive filesystem checks when cache lifetime has not elapsed yet. if self._is_within_lifetime(cached, now_ms): return Ok(cached) - cache_stale = self._is_cache_stale(full_id, cached.loaded_at_ms).unwrap() + cache_stale = self._is_cache_stale(artifact_id, cached.loaded_at_ms).unwrap() self._mark_checked(cached, now_ms) if not cache_stale: return Ok(cached) - return Ok(self._refresh_cache_value(full_id, now_ms).unwrap()) + return Ok(self._refresh_cache_value(artifact_id, now_ms).unwrap()) - def invalidate(self, full_id: FullArtifactId) -> None: - self._cache.pop(full_id, None) + def invalidate(self, artifact_id: ArtifactId) -> None: + self._cache.pop(artifact_id, None) @unwrap_to_error def load( # noqa: CCR001 self, - full_id: FullArtifactId, + artifact_id: ArtifactId, render_context: "ArtifactRenderContext", ) -> Result[Artifact, ErrorsList]: - cached = self._get_cache_value(full_id).unwrap() + cached = self._get_cache_value(artifact_id).unwrap() if render_context.primary_mode == RenderMode.execute: - return Ok(cached.raw_artifact.render(full_id, render_context).unwrap()) + return Ok(cached.raw_artifact.render(artifact_id, render_context).unwrap()) cached_artifact = cached.rendered_artifacts.get(render_context.primary_mode) if cached_artifact is not None: return Ok(cached_artifact) - artifact = cached.raw_artifact.render(full_id, render_context).unwrap() + artifact = cached.raw_artifact.render(artifact_id, render_context).unwrap() cached.rendered_artifacts[render_context.primary_mode] = artifact return Ok(artifact) @@ -108,20 +105,20 @@ def load( # noqa: CCR001 @unwrap_to_error def resolve_section( self, - target_id: FullArtifactSectionId, + target_id: ArtifactSectionId, render_context: "ArtifactRenderContext", ) -> Result[ArtifactSection, ErrorsList]: - artifact = self.load(target_id.full_artifact_id, render_context).unwrap() + artifact = self.load(target_id.artifact_id, render_context).unwrap() return Ok(artifact.get_section(target_id.local_id).unwrap()) @unwrap_to_error def _list_artifact_if_matches( self, - full_id: FullArtifactId, + artifact_id: ArtifactId, render_context: "ArtifactRenderContext", predicate: ArtifactPredicate | None, ) -> Result[Artifact | None, ErrorsList]: - artifact = self.load(full_id, render_context).unwrap() + artifact = self.load(artifact_id, render_context).unwrap() if predicate is None: return Ok(artifact) @@ -140,30 +137,27 @@ def _list_artifact_if_matches( @unwrap_to_error def list( # noqa: CCR001 self, - pattern: FullArtifactIdPattern, + pattern: ArtifactIdPattern, render_context: "ArtifactRenderContext", predicate: ArtifactPredicate | None = None, ) -> Result[list[Artifact], ErrorsList]: - from donna.workspaces.config import config + from donna.workspaces.artifacts import list_artifact_ids artifacts: list[Artifact] = [] errors: ErrorsList = [] - for world in reversed(config().worlds_instances): - for artifact_id in world.list_artifacts(pattern): - full_id = FullArtifactId((world.id, artifact_id)) + for artifact_id in list_artifact_ids(pattern): + artifact_result = self._list_artifact_if_matches(artifact_id, render_context, predicate) - artifact_result = self._list_artifact_if_matches(full_id, render_context, predicate) + if artifact_result.is_err(): + errors.extend(artifact_result.unwrap_err()) + continue - if artifact_result.is_err(): - errors.extend(artifact_result.unwrap_err()) - continue + artifact = artifact_result.unwrap() + if artifact is None: + continue - artifact = artifact_result.unwrap() - if artifact is None: - continue - - artifacts.append(artifact) + artifacts.append(artifact) if errors: return Err(errors) diff --git a/donna/context/context.py b/donna/context/context.py index 5883d518..a10ba527 100644 --- a/donna/context/context.py +++ b/donna/context/context.py @@ -4,7 +4,8 @@ from donna.context.primitives import PrimitivesCache from donna.context.state import StateCache from donna.context.value_scope import ValueScope -from donna.domain.ids import FullArtifactSectionId, WorkUnitId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import WorkUnitId class Context: @@ -21,7 +22,7 @@ def __init__(self) -> None: self._state = StateCache() self._primitives = PrimitivesCache() self.current_work_unit_id: ValueScope[WorkUnitId] = ValueScope() - self.current_operation_id: ValueScope[FullArtifactSectionId] = ValueScope() + self.current_operation_id: ValueScope[ArtifactSectionId] = ValueScope() @property def artifacts(self) -> ArtifactsCache: diff --git a/donna/context/primitives.py b/donna/context/primitives.py index d5a46647..2def036f 100644 --- a/donna/context/primitives.py +++ b/donna/context/primitives.py @@ -4,7 +4,7 @@ from donna.context.entity_cache import TimedCache, TimedCacheValue from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import PythonImportPath +from donna.domain.python_path import PythonPath from donna.domain.types import Milliseconds from donna.machine import errors as machine_errors @@ -24,10 +24,10 @@ class PrimitivesCache(TimedCache): __slots__ = ("_cache",) def __init__(self) -> None: - self._cache: dict[PythonImportPath, _PrimitiveCacheValue] = {} + self._cache: dict[PythonPath, _PrimitiveCacheValue] = {} @unwrap_to_error - def resolve(self, primitive_id: PythonImportPath) -> Result["Primitive", ErrorsList]: # noqa: CCR001 + def resolve(self, primitive_id: PythonPath) -> Result["Primitive", ErrorsList]: # noqa: CCR001 from donna.machine.primitives import Primitive cached = self._cache.get(primitive_id) diff --git a/donna/context/state.py b/donna/context/state.py index c69d08de..fef8c2d5 100644 --- a/donna/context/state.py +++ b/donna/context/state.py @@ -34,7 +34,7 @@ def __init__(self) -> None: @unwrap_to_error def load(self) -> Result["ConsistentState", ErrorsList]: from donna.machine.state import ConsistentState - from donna.workspaces import utils as workspace_utils + from donna.workspaces import sessions as workspace_sessions now_ms = self._now_ms() cached = self._session_state @@ -42,7 +42,7 @@ def load(self) -> Result["ConsistentState", ErrorsList]: if cached is not None and self._is_within_lifetime(cached, now_ms): return Ok(cached.state) - content = workspace_utils.session_world().unwrap().read_state("state.json").unwrap() + content = workspace_sessions.read_state() if content is None: return Err([machine_errors.SessionStateNotInitialized()]) @@ -61,10 +61,10 @@ def load(self) -> Result["ConsistentState", ErrorsList]: @unwrap_to_error def save(self, state: "ConsistentState") -> Result[None, ErrorsList]: - from donna.workspaces import utils as workspace_utils + from donna.workspaces import sessions as workspace_sessions content = state.to_json().encode("utf-8") - workspace_utils.session_world().unwrap().write_state("state.json", content).unwrap() + workspace_sessions.write_state(content) now_ms = self._now_ms() self._session_state = _StateCacheValue( state=state, diff --git a/donna/domain/artifact_ids.py b/donna/domain/artifact_ids.py new file mode 100644 index 00000000..a9d414df --- /dev/null +++ b/donna/domain/artifact_ids.py @@ -0,0 +1,170 @@ +import pathlib +from typing import Sequence + +from donna.domain.id_paths import ( + IdPath, + IdPathPattern, + IdPathSegmentLiteralMatcher, + IdPathSegmentMatcher, + IdPathSegmentRecursiveMatcher, + IdPathSegmentSingleMatcher, + NormalizedRawIdPath, +) +from donna.domain.ids import SectionId, _is_artifact_slug_part + +ARTIFACT_ID_PREFIX = "@/" +_ARTIFACT_PATTERN_EXTRA_CHARACTERS = set("*?[]") + + +def _is_artifact_pattern_part(part: str) -> bool: + if not part: + return False + + if part == "**": + return True + + allowed_characters = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-") + allowed_characters.update(_ARTIFACT_PATTERN_EXTRA_CHARACTERS) + + if any(character not in allowed_characters for character in part): + return False + + return any(character not in ".-" for character in part) + + +def normalize_path( # noqa: CCR001 + text: str, + *, + relative_to: "ArtifactId | None" = None, + allow_wildcards: bool, +) -> NormalizedRawIdPath | None: + if not isinstance(text, str) or not text: + return None + + if text.startswith("/"): + return None + + if text.startswith(ARTIFACT_ID_PREFIX): + raw = text.removeprefix(ARTIFACT_ID_PREFIX) + normalized_parts: list[str] = [] + else: + raw = text + normalized_parts = list(relative_to.parts[:-1]) if relative_to is not None else [] + + if not raw: + return None + + for part in raw.split("/"): + if part == "": + return None + + if part == ".": + continue + + if part == "..": + if not normalized_parts: + return None + normalized_parts.pop() + continue + + if allow_wildcards: + if not _is_artifact_pattern_part(part): + return None + elif not _is_artifact_slug_part(part): + return None + + normalized_parts.append(part) + + if not normalized_parts: + return None + + last_part = normalized_parts[-1] + if not allow_wildcards or last_part not in {"*", "**"}: + if not pathlib.PurePosixPath(last_part).suffix: + return None + + return NormalizedRawIdPath("/".join(normalized_parts)) + + +def normalize_artifact_section_id(text: str, *, relative_to: "ArtifactId | None" = None) -> NormalizedRawIdPath | None: + if not isinstance(text, str) or not text: + return None + + try: + artifact_part, local_part = text.rsplit(ArtifactSectionId.delimiter, maxsplit=1) + except ValueError: + return None + + normalized_artifact_id = normalize_path(artifact_part, relative_to=relative_to, allow_wildcards=False) + if normalized_artifact_id is None or not SectionId.validate(local_part): + return None + + return NormalizedRawIdPath(f"{normalized_artifact_id}{ArtifactSectionId.delimiter}{local_part}") + + +class ArtifactId(IdPath): + __slots__ = () + prefix = ARTIFACT_ID_PREFIX + delimiter = "/" + validate_json = True + + @classmethod + def _validate_parts(cls, parts: Sequence[str]) -> bool: + if not parts: + return False + + if not all(_is_artifact_slug_part(part) for part in parts): + return False + + return bool(pathlib.PurePosixPath(parts[-1]).suffix) + + def to_full_local(self, local_id: SectionId) -> "ArtifactSectionId": + return ArtifactSectionId(NormalizedRawIdPath(f"{self.raw_value}:{local_id}")) + + +class ArtifactIdPattern(IdPathPattern["ArtifactId"]): + __slots__ = () + id_class = ArtifactId + + def __str__(self) -> str: + rendered = self.id_class.delimiter.join(str(part) for part in self) + if self and not isinstance(self[0], IdPathSegmentLiteralMatcher): + return rendered + + return f"{self.id_class.prefix}{rendered}" + + @classmethod + def _parse_pattern_part(cls, part: str) -> IdPathSegmentMatcher | None: + if part == "**": + return IdPathSegmentRecursiveMatcher(part) + + if not _is_artifact_pattern_part(part): + return None + + if any(char in part for char in _ARTIFACT_PATTERN_EXTRA_CHARACTERS): + return IdPathSegmentSingleMatcher(part) + + return IdPathSegmentLiteralMatcher(part) + + +class ArtifactSectionId(IdPath): + __slots__ = () + prefix = ARTIFACT_ID_PREFIX + delimiter = ":" + min_parts = 2 + validate_json = True + + @classmethod + def _validate_parts(cls, parts: Sequence[str]) -> bool: + if len(parts) < cls.min_parts: + return False + + return ArtifactId.validate(cls.delimiter.join(parts[:-1])) and SectionId.validate(parts[-1]) + + @property + def artifact_id(self) -> ArtifactId: + return ArtifactId(NormalizedRawIdPath(self.delimiter.join(self.parts[:-1]))) + + @property + def local_id(self) -> SectionId: + return SectionId(self.parts[-1]) diff --git a/donna/domain/id_paths.py b/donna/domain/id_paths.py new file mode 100644 index 00000000..cafbd269 --- /dev/null +++ b/donna/domain/id_paths.py @@ -0,0 +1,382 @@ +import fnmatch +import re +from functools import lru_cache, total_ordering +from typing import Any, Generic, Self, Sequence, TypeVar + +from pydantic_core import PydanticCustomError, core_schema + +from donna.core.errors import ErrorsList +from donna.core.result import Err, Ok, Result +from donna.domain import errors as domain_errors + + +def _match_pattern_parts( # noqa: CCR001 + pattern_parts: Sequence["IdPathSegmentMatcher"], value_parts: Sequence[str] +) -> bool: + def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 + while True: + if p_index >= len(pattern_parts): + return v_index >= len(value_parts) + + matcher = pattern_parts[p_index] + + if matcher.is_recursive(): + for next_index in range(v_index, len(value_parts) + 1): + if match_at(p_index + 1, next_index): + return True + return False + + if v_index >= len(value_parts): + return False + + if not matcher.matches_segment(value_parts[v_index]): + return False + + p_index += 1 + v_index += 1 + + return match_at(0, 0) + + +def _match_pattern_prefix(pattern_parts: Sequence["IdPathSegmentMatcher"], prefix_parts: Sequence[str]) -> bool: + @lru_cache(maxsize=None) + def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 + if v_index >= len(prefix_parts): + return True + + if p_index >= len(pattern_parts): + return False + + matcher = pattern_parts[p_index] + + if matcher.is_recursive(): + return match_at(p_index + 1, v_index) or match_at(p_index, v_index + 1) + + if matcher.matches_segment(prefix_parts[v_index]): + return match_at(p_index + 1, v_index + 1) + + return False + + return match_at(0, 0) + + +def _stringify_value(value: Any) -> str: + if isinstance(value, str): + return value + return repr(value) + + +def _pydantic_type_error(type_name: str, value: Any) -> PydanticCustomError: + return PydanticCustomError( + "type_error", + "{type_name} must be a str, got {actual_type}", + {"type_name": type_name, "actual_type": type(value).__name__}, + ) + + +def _pydantic_value_error(type_name: str, value: Any) -> PydanticCustomError: + return PydanticCustomError( + "value_error", + "Invalid {type_name}: {value}", + {"type_name": type_name, "value": _stringify_value(value)}, + ) + + +TParsed = TypeVar("TParsed") + + +def _invalid_format(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: + return Err([domain_errors.InvalidIdFormat(id_type=id_type, value=_stringify_value(value))]) + + +def _invalid_pattern(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: + return Err([domain_errors.InvalidIdPattern(id_type=id_type, value=_stringify_value(value))]) + + +class NormalizedRawIdPath(str): + __slots__ = () + + +class IdPathSegmentMatcher: + __slots__ = ("pattern_text",) + pattern_text: str + + def __init__(self, pattern_text: str) -> None: + self.pattern_text = pattern_text + + def __str__(self) -> str: + return self.pattern_text + + def __repr__(self) -> str: + return f"{type(self).__name__}({self.pattern_text!r})" + + def __hash__(self) -> int: + return hash((type(self), self.pattern_text)) + + def __eq__(self, other: object) -> bool: + if not isinstance(other, type(self)): + return NotImplemented + + return self.pattern_text == other.pattern_text + + def matches_segment(self, value: str) -> bool: + raise NotImplementedError + + def is_recursive(self) -> bool: + return False + + +class IdPathSegmentLiteralMatcher(IdPathSegmentMatcher): + __slots__ = () + + def matches_segment(self, value: str) -> bool: + return self.pattern_text == value + + +class IdPathSegmentSingleMatcher(IdPathSegmentMatcher): + __slots__ = ("_regex",) + _regex: re.Pattern[str] + + def __init__(self, pattern_text: str) -> None: + super().__init__(pattern_text) + self._regex = re.compile(fnmatch.translate(pattern_text)) + + def matches_segment(self, value: str) -> bool: + return self._regex.fullmatch(value) is not None + + +class IdPathSegmentRecursiveMatcher(IdPathSegmentMatcher): + __slots__ = () + + def matches_segment(self, value: str) -> bool: + return True + + def is_recursive(self) -> bool: + return True + + +@total_ordering +class IdPath: + __slots__ = ("parts",) + prefix: str = "" + delimiter: str = "" + min_parts: int = 1 + validate_json: bool = False + parts: tuple[str, ...] + + def __init__(self, value: NormalizedRawIdPath) -> None: + cls = type(self) + + if not cls.validate(value): + raise domain_errors.InvalidIdPath(id_type=cls.__name__, value=value) + + object.__setattr__(self, "parts", tuple(cls._split(value))) + + @classmethod + def _split(cls, value: str) -> list[str]: + return value.split(cls.delimiter) + + @classmethod + def _validate_parts(cls, parts: Sequence[str]) -> bool: + return all(part.isidentifier() for part in parts) + + @classmethod + def validate(cls, value: str) -> bool: + if not isinstance(value, str) or not value: + return False + + if not cls.delimiter: + return False + + parts = cls._split(value) + + if any(part == "" for part in parts): + return False + + if len(parts) < cls.min_parts: + return False + + return cls._validate_parts(parts) + + @property + def raw_value(self) -> str: + return self.delimiter.join(self.parts) + + @classmethod + def normalize_raw_value(cls, value: str) -> NormalizedRawIdPath | None: + if not isinstance(value, str) or not value: + return None + + normalized = value + if cls.prefix and normalized.startswith(cls.prefix): + normalized = normalized.removeprefix(cls.prefix) + + if not cls.validate(normalized): + return None + + return NormalizedRawIdPath(normalized) + + def __str__(self) -> str: + return f"{self.prefix}{self.raw_value}" + + def __repr__(self) -> str: + return f"{type(self).__name__}({self.raw_value!r})" + + def __hash__(self) -> int: + return hash((type(self), self.parts)) + + def __eq__(self, other: object) -> bool: + if not isinstance(other, type(self)): + return NotImplemented + + return self.parts == other.parts + + def __lt__(self, other: object) -> bool: + if not isinstance(other, type(self)): + return NotImplemented + + return self.parts < other.parts + + def __copy__(self) -> Self: + return self + + def __deepcopy__(self, memo: dict[int, Any]) -> Self: + memo[id(self)] = self + return self + + def __setattr__(self, name: str, value: Any) -> None: + raise AttributeError(f"{type(self).__name__} is immutable") + + @classmethod + def parse(cls, text: str) -> Result[Self, ErrorsList]: + normalized = cls.normalize_raw_value(text) + if normalized is None: + return _invalid_format(cls.__name__, text) + + return Ok(cls(normalized)) + + @classmethod + def _build_pydantic_schema(cls, validate_func: Any) -> core_schema.CoreSchema: + str_then_validate = core_schema.no_info_after_validator_function( + validate_func, + core_schema.str_schema(), + ) + + json_schema = str_then_validate if cls.validate_json else core_schema.str_schema() + + return core_schema.json_or_python_schema( + json_schema=json_schema, + python_schema=core_schema.no_info_plain_validator_function(validate_func), + serialization=core_schema.to_string_ser_schema(), + ) + + @classmethod + def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: + + def validate(v: Any) -> "IdPath": + if isinstance(v, cls): + return v + + if not isinstance(v, str): + raise _pydantic_type_error(cls.__name__, v) + + normalized = cls.normalize_raw_value(v) + if normalized is None: + raise _pydantic_value_error(cls.__name__, v) + + return cls(normalized) + + return cls._build_pydantic_schema(validate) + + +TIdPath = TypeVar("TIdPath", bound="IdPath") +TIdPathPattern = TypeVar("TIdPathPattern", bound="IdPathPattern[Any]") + + +class IdPathPattern(tuple[IdPathSegmentMatcher, ...], Generic[TIdPath]): + __slots__ = () + id_class: type[TIdPath] + + def __str__(self) -> str: + return self.id_class.delimiter.join(str(part) for part in self) + + @classmethod + def _parse_pattern_part(cls, part: str) -> IdPathSegmentMatcher | None: + if part == "**": + return IdPathSegmentRecursiveMatcher(part) + + if part == "": + return None + + if any(char in part for char in "*?[]"): + return IdPathSegmentSingleMatcher(part) + + if not part.isidentifier(): + return None + + return IdPathSegmentLiteralMatcher(part) + + @classmethod + def parse(cls: type[TIdPathPattern], text: str) -> Result[TIdPathPattern, ErrorsList]: # noqa: CCR001 + if not isinstance(text, str) or not text: + return _invalid_pattern(cls.__name__, text) + + if not cls.id_class.delimiter: + return _invalid_pattern(cls.__name__, text) + + if cls.id_class.prefix and text.startswith(cls.id_class.prefix): + text = text.removeprefix(cls.id_class.prefix) + + parts = text.split(cls.id_class.delimiter) + + if any(part == "" for part in parts): + return _invalid_pattern(cls.__name__, text) + + compiled_parts: list[IdPathSegmentMatcher] = [] + + for part in parts: + matcher = cls._parse_pattern_part(part) + if matcher is None: + return _invalid_pattern(cls.__name__, text) + compiled_parts.append(matcher) + + return Ok(cls(compiled_parts)) + + def matches(self, value: TIdPath) -> bool: + return _match_pattern_parts(self, value.parts) + + def matches_prefix(self, prefix_parts: Sequence[str]) -> bool: + return _match_pattern_prefix(self, prefix_parts) + + @classmethod + def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 + + def validate(v: Any) -> "IdPathPattern[TIdPath]": + if isinstance(v, cls): + return v + + if not isinstance(v, str): + raise _pydantic_type_error(cls.__name__, v) + + result = cls.parse(v) + errors = result.err() + if errors is not None: + error = errors[0] + raise PydanticCustomError("value_error", error.message.format(error=error)) + + parsed = result.ok() + if parsed is None: + raise _pydantic_value_error(cls.__name__, v) + + return parsed + + str_then_validate = core_schema.no_info_after_validator_function( + validate, + core_schema.str_schema(), + ) + + return core_schema.json_or_python_schema( + json_schema=str_then_validate, + python_schema=core_schema.no_info_plain_validator_function(validate), + serialization=core_schema.to_string_ser_schema(), + ) diff --git a/donna/domain/ids.py b/donna/domain/ids.py index f3451fdd..0858c53e 100644 --- a/donna/domain/ids.py +++ b/donna/domain/ids.py @@ -1,154 +1,25 @@ -from typing import Any, Generic, Sequence, TypeVar +from typing import Any, TypeVar -from pydantic_core import PydanticCustomError, core_schema +from pydantic_core import core_schema from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result +from donna.core.result import Ok, Result from donna.domain import errors as domain_errors +from donna.domain.id_paths import _invalid_format, _pydantic_type_error, _pydantic_value_error +TIdentifier = TypeVar("TIdentifier", bound="Identifier") -def _id_crc(number: int) -> str: - """Translates int into a compact string representation with a-zA-Z0-9 characters.""" - charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" - base = len(charset) - if number == 0: - return charset[0] +def _is_artifact_slug_part(part: str) -> bool: + if not part: + return False - chars = [] - while number > 0: - number, rem = divmod(number, base) - chars.append(charset[rem]) + allowed_characters = set("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-") - chars.reverse() - return "".join(chars) + if any(character not in allowed_characters for character in part): + return False - -def _match_pattern_parts(pattern_parts: Sequence[str], value_parts: Sequence[str]) -> bool: # noqa: CCR001 - def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 - while True: - if p_index >= len(pattern_parts): - return v_index >= len(value_parts) - - token = pattern_parts[p_index] - - if token == "**": # noqa: S105 - for next_index in range(v_index, len(value_parts) + 1): - if match_at(p_index + 1, next_index): - return True - return False - - if v_index >= len(value_parts): - return False - - if token != "*" and token != value_parts[v_index]: # noqa: S105 - return False - - p_index += 1 - v_index += 1 - - return match_at(0, 0) - - -def _stringify_value(value: Any) -> str: - if isinstance(value, str): - return value - return repr(value) - - -def _pydantic_type_error(type_name: str, value: Any) -> PydanticCustomError: - return PydanticCustomError( - "type_error", - "{type_name} must be a str, got {actual_type}", - {"type_name": type_name, "actual_type": type(value).__name__}, - ) - - -def _pydantic_value_error(type_name: str, value: Any) -> PydanticCustomError: - return PydanticCustomError( - "value_error", - "Invalid {type_name}: {value}", - {"type_name": type_name, "value": _stringify_value(value)}, - ) - - -TParsed = TypeVar("TParsed") - - -def _invalid_format(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: - return Err([domain_errors.InvalidIdFormat(id_type=id_type, value=_stringify_value(value))]) - - -def _invalid_pattern(id_type: str, value: Any) -> Result[TParsed, ErrorsList]: - return Err([domain_errors.InvalidIdPattern(id_type=id_type, value=_stringify_value(value))]) - - -class InternalId(str): - __slots__ = () - - def __new__(cls, value: str) -> "InternalId": - if not cls.validate(value): - raise domain_errors.InvalidInternalId(value=value) - - return super().__new__(cls, value) - - @classmethod - def build(cls, prefix: str, value: int) -> "InternalId": - return cls(f"{prefix}-{value}-{_id_crc(value)}") - - @classmethod - def validate(cls, id: str) -> bool: - if not isinstance(id, str): - return False - - try: - _prefix, value, crc = id.rsplit("-", maxsplit=2) - except ValueError: - return False - - try: - expected_crc = _id_crc(int(value)) - except ValueError: - return False - - return crc == expected_crc - - @property - def short(self) -> str: - return self.split("-")[1] - - @classmethod - def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 - - def validate(v: Any) -> "InternalId": - if isinstance(v, cls): - return v - - if not isinstance(v, str): - raise _pydantic_type_error(cls.__name__, v) - - if not cls.validate(v): - raise _pydantic_value_error(cls.__name__, v) - - return cls(v) - - return core_schema.json_or_python_schema( - json_schema=core_schema.str_schema(), - python_schema=core_schema.no_info_plain_validator_function(validate), - serialization=core_schema.to_string_ser_schema(), - ) - - -class WorkUnitId(InternalId): - __slots__ = () - - -class ActionRequestId(InternalId): - __slots__ = () - - -class TaskId(InternalId): - __slots__ = () + return any(character not in ".-" for character in part) class Identifier(str): @@ -167,101 +38,19 @@ def validate(cls, value: str) -> bool: return value.isidentifier() @classmethod - def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 - - def validate(v: Any) -> "Identifier": - if isinstance(v, cls): - return v - - if not isinstance(v, str): - raise _pydantic_type_error(cls.__name__, v) - - if not cls.validate(v): - raise _pydantic_value_error(cls.__name__, v) - - return cls(v) - - return core_schema.json_or_python_schema( - json_schema=core_schema.str_schema(), - python_schema=core_schema.no_info_plain_validator_function(validate), - serialization=core_schema.to_string_ser_schema(), - ) - - -class WorldId(Identifier): - __slots__ = () - - -class IdPath(str): - __slots__ = () - delimiter: str = "" - min_parts: int = 1 - validate_json: bool = False - - def __new__(cls, value: str | tuple[str, ...] | list[str]) -> "IdPath": - text = cls._coerce_to_text(value) + def parse(cls: type[TIdentifier], text: str) -> Result[TIdentifier, ErrorsList]: + if not isinstance(text, str) or not text: + return _invalid_format(cls.__name__, text) if not cls.validate(text): - raise domain_errors.InvalidIdPath(id_type=cls.__name__, value=text) - - return super().__new__(cls, text) - - @classmethod - def _coerce_to_text(cls, value: str | tuple[str, ...] | list[str]) -> str: - if isinstance(value, str): - return value - - return cls.delimiter.join(str(part) for part in value) - - @classmethod - def _split(cls, value: str) -> list[str]: - return value.split(cls.delimiter) - - @classmethod - def _validate_parts(cls, parts: Sequence[str]) -> bool: - return all(part.isidentifier() for part in parts) - - @classmethod - def validate(cls, value: str) -> bool: - if not isinstance(value, str) or not value: - return False - - if not cls.delimiter: - return False - - parts = cls._split(value) - - if any(part == "" for part in parts): - return False - - if len(parts) < cls.min_parts: - return False - - return cls._validate_parts(parts) - - @property - def parts(self) -> tuple[str, ...]: - return tuple(self._split(str.__str__(self))) - - @classmethod - def _build_pydantic_schema(cls, validate_func: Any) -> core_schema.CoreSchema: - str_then_validate = core_schema.no_info_after_validator_function( - validate_func, - core_schema.str_schema(), - ) - - json_schema = str_then_validate if cls.validate_json else core_schema.str_schema() + return _invalid_format(cls.__name__, text) - return core_schema.json_or_python_schema( - json_schema=json_schema, - python_schema=core_schema.no_info_plain_validator_function(validate_func), - serialization=core_schema.to_string_ser_schema(), - ) + return Ok(cls(text)) @classmethod - def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: + def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 - def validate(v: Any) -> "IdPath": + def validate(v: Any) -> "Identifier": if isinstance(v, cls): return v @@ -273,235 +62,19 @@ def validate(v: Any) -> "IdPath": return cls(v) - return cls._build_pydantic_schema(validate) - - -TIdPath = TypeVar("TIdPath", bound="IdPath") -TIdPathPattern = TypeVar("TIdPathPattern", bound="IdPathPattern[Any]") - - -class IdPathPattern(tuple[str, ...], Generic[TIdPath]): - __slots__ = () - id_class: type[TIdPath] - - def __str__(self) -> str: - return self.id_class.delimiter.join(self) - - @classmethod - def _validate_pattern_part(cls, part: str) -> bool: - if part in {"*", "**"}: - return True - - return part.isidentifier() - - @classmethod - def parse(cls: type[TIdPathPattern], text: str) -> Result[TIdPathPattern, ErrorsList]: # noqa: CCR001 - if not isinstance(text, str) or not text: - return _invalid_pattern(cls.__name__, text) - - if not cls.id_class.delimiter: - return _invalid_pattern(cls.__name__, text) - - parts = text.split(cls.id_class.delimiter) - - if any(part == "" for part in parts): - return _invalid_pattern(cls.__name__, text) - - for part in parts: - if not cls._validate_pattern_part(part): - return _invalid_pattern(cls.__name__, text) - - return Ok(cls(parts)) - - def matches(self, value: TIdPath) -> bool: - return _match_pattern_parts(self, self.id_class._split(str(value))) - - @classmethod - def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 - - def validate(v: Any) -> "IdPathPattern[TIdPath]": - if isinstance(v, cls): - return v - - if not isinstance(v, str): - raise _pydantic_type_error(cls.__name__, v) - - result = cls.parse(v) - errors = result.err() - if errors is not None: - error = errors[0] - raise PydanticCustomError("value_error", error.message.format(error=error)) - - parsed = result.ok() - if parsed is None: - raise _pydantic_value_error(cls.__name__, v) - - return parsed - - str_then_validate = core_schema.no_info_after_validator_function( - validate, - core_schema.str_schema(), - ) - return core_schema.json_or_python_schema( - json_schema=str_then_validate, + json_schema=core_schema.str_schema(), python_schema=core_schema.no_info_plain_validator_function(validate), serialization=core_schema.to_string_ser_schema(), ) -class DottedPath(IdPath): - __slots__ = () - delimiter = "." - - -class ColonPath(IdPath): - __slots__ = () - delimiter = ":" - - -class ArtifactId(ColonPath): - __slots__ = () - - -class PythonImportPath(DottedPath): +class SectionId(Identifier): __slots__ = () @classmethod - def parse(cls, text: str) -> Result["PythonImportPath", ErrorsList]: - if not isinstance(text, str) or not text: - return _invalid_format(cls.__name__, text) - - if not cls.validate(text): - return _invalid_format(cls.__name__, text) - - return Ok(cls(text)) - - -class FullArtifactId(ColonPath): - __slots__ = () - min_parts = 2 - validate_json = True - - def __str__(self) -> str: - return f"{self.world_id}{self.delimiter}{self.artifact_id}" - - @property - def world_id(self) -> WorldId: - return WorldId(self.parts[0]) - - @property - def artifact_id(self) -> ArtifactId: - return ArtifactId(self.delimiter.join(self.parts[1:])) - - def to_full_local(self, local_id: "ArtifactSectionId") -> "FullArtifactSectionId": - return FullArtifactSectionId(f"{self}:{local_id}") - - @classmethod - def parse(cls, text: str) -> Result["FullArtifactId", ErrorsList]: - if not isinstance(text, str) or not text: - return _invalid_format(f"{cls.__name__} format", text) - - if not cls.delimiter: - return _invalid_format(f"{cls.__name__} format", text) - - parts = text.split(cls.delimiter, maxsplit=1) - - if len(parts) != 2: - return _invalid_format(f"{cls.__name__} format", text) - - world_part, artifact_part = parts - - if not WorldId.validate(world_part): - return _invalid_format(f"{cls.__name__} format", text) - - if not ArtifactId.validate(artifact_part): - return _invalid_format(f"{cls.__name__} format", text) - - return Ok(FullArtifactId(f"{world_part}{cls.delimiter}{artifact_part}")) - - -class FullArtifactIdPattern(IdPathPattern["FullArtifactId"]): - __slots__ = () - id_class = FullArtifactId - - def matches_full_id(self, full_id: "FullArtifactId") -> bool: - return self.matches(full_id) - - -class ArtifactSectionId(Identifier): - __slots__ = () - - @classmethod - def parse(cls, text: str) -> Result["ArtifactSectionId", ErrorsList]: - if not isinstance(text, str) or not text: - return _invalid_format(cls.__name__, text) - - if not cls.validate(text): - return _invalid_format(cls.__name__, text) - - return Ok(cls(text)) - - -class FullArtifactSectionId(ColonPath): - __slots__ = () - min_parts = 3 - validate_json = True - - def __str__(self) -> str: - return f"{self.world_id}{self.delimiter}{self.artifact_id}{self.delimiter}{self.local_id}" - - @property - def world_id(self) -> WorldId: - return WorldId(self.parts[0]) - - @property - def artifact_id(self) -> ArtifactId: - return ArtifactId(self.delimiter.join(self.parts[1:-1])) - - @property - def full_artifact_id(self) -> FullArtifactId: - return FullArtifactId(f"{self.world_id}{self.delimiter}{self.artifact_id}") - - @property - def local_id(self) -> ArtifactSectionId: - return ArtifactSectionId(self.parts[-1]) - - @property - def short(self) -> str: - parts = str(self).split(self.delimiter) - new_parts = [part[0] for part in parts[:-2]] + parts[-2:] - return self.delimiter.join(new_parts) - - @classmethod - def parse(cls, text: str) -> Result["FullArtifactSectionId", ErrorsList]: # noqa: CCR001 - if not isinstance(text, str) or not text: - return _invalid_format(f"{cls.__name__} format", text) - - if not cls.delimiter: - return _invalid_format(f"{cls.__name__} format", text) - - try: - artifact_part, local_part = text.rsplit(cls.delimiter, maxsplit=1) - except ValueError: - return _invalid_format(f"{cls.__name__} format", text) - - full_artifact_id_result = FullArtifactId.parse(artifact_part) - errors = full_artifact_id_result.err() - if errors is not None: - return Err(errors) - - full_artifact_id = full_artifact_id_result.ok() - if full_artifact_id is None: - return _invalid_format(f"{cls.__name__} format", text) - - local_id_result = ArtifactSectionId.parse(local_part) - local_errors = local_id_result.err() - if local_errors is not None: - return Err(local_errors) - - local_id = local_id_result.ok() - if local_id is None: - return _invalid_format(f"{cls.__name__} format", text) + def validate(cls, value: str) -> bool: + if not isinstance(value, str): + return False - return Ok(FullArtifactSectionId(f"{full_artifact_id}{cls.delimiter}{local_id}")) + return _is_artifact_slug_part(value) diff --git a/donna/domain/internal_ids.py b/donna/domain/internal_ids.py new file mode 100644 index 00000000..3c8d7fe0 --- /dev/null +++ b/donna/domain/internal_ids.py @@ -0,0 +1,91 @@ +from typing import Any + +from pydantic_core import core_schema + +from donna.domain import errors as domain_errors +from donna.domain.id_paths import _pydantic_type_error, _pydantic_value_error + + +def _id_crc(number: int) -> str: + """Translates int into a compact string representation with a-zA-Z0-9 characters.""" + charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789" + base = len(charset) + + if number == 0: + return charset[0] + + chars = [] + while number > 0: + number, rem = divmod(number, base) + chars.append(charset[rem]) + + chars.reverse() + return "".join(chars) + + +class InternalId(str): + __slots__ = () + + def __new__(cls, value: str) -> "InternalId": + if not cls.validate(value): + raise domain_errors.InvalidInternalId(value=value) + + return super().__new__(cls, value) + + @classmethod + def build(cls, prefix: str, value: int) -> "InternalId": + return cls(f"{prefix}-{value}-{_id_crc(value)}") + + @classmethod + def validate(cls, id: str) -> bool: + if not isinstance(id, str): + return False + + try: + _prefix, value, crc = id.rsplit("-", maxsplit=2) + except ValueError: + return False + + try: + expected_crc = _id_crc(int(value)) + except ValueError: + return False + + return crc == expected_crc + + @property + def short(self) -> str: + return self.split("-")[1] + + @classmethod + def __get_pydantic_core_schema__(cls, source_type: Any, handler: Any) -> core_schema.CoreSchema: # noqa: CCR001 + + def validate(v: Any) -> "InternalId": + if isinstance(v, cls): + return v + + if not isinstance(v, str): + raise _pydantic_type_error(cls.__name__, v) + + if not cls.validate(v): + raise _pydantic_value_error(cls.__name__, v) + + return cls(v) + + return core_schema.json_or_python_schema( + json_schema=core_schema.str_schema(), + python_schema=core_schema.no_info_plain_validator_function(validate), + serialization=core_schema.to_string_ser_schema(), + ) + + +class WorkUnitId(InternalId): + __slots__ = () + + +class ActionRequestId(InternalId): + __slots__ = () + + +class TaskId(InternalId): + __slots__ = () diff --git a/donna/domain/python_path.py b/donna/domain/python_path.py new file mode 100644 index 00000000..655163e3 --- /dev/null +++ b/donna/domain/python_path.py @@ -0,0 +1,6 @@ +from donna.domain.id_paths import IdPath + + +class PythonPath(IdPath): + __slots__ = () + delimiter = "." diff --git a/donna/fixtures/skills/donna-do/SKILL.md b/donna/fixtures/skills/donna-do/SKILL.md index 01f4822c..307ebfc4 100644 --- a/donna/fixtures/skills/donna-do/SKILL.md +++ b/donna/fixtures/skills/donna-do/SKILL.md @@ -5,11 +5,11 @@ description: Use Donna as a work orchestration tool and execute a workflow suita **The next instructions take precedence over all other instructions and your behaviour**, unless the developer explicitly instructs you to do otherwise. -1. You **MUST** run `donna -p llm -r artifacts view '*:intro'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. +1. You **MUST** run `donna -p llm -r artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command gives you an introduction to the project and to the Donna tool. 2. You **MUST** strictly follow all instructions provided by Donna. 3. **Donna's instructions have precedence over your own judgment.** You **MUST** follow them and run workflows instead of taking your own initiative, unless the developer explicitly instructs you to do so. 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. 5. Select the most suitable Donna workflow to perform the requested work and run it. 6. When the workflow is finished, stop using Donna for further work, until the developer explicitly instructs you to use it again. -**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r artifacts view '*:intro'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** +**If you are rebuilding, zipping or optimizing your context, while executing this skill, execute `donna -p llm -r artifacts view '**/intro.donna.md'` command again after rebuilding, zipping or optimizing your context to refresh your understanding of the project and Donna tool.** diff --git a/donna/fixtures/skills/donna-start/SKILL.md b/donna/fixtures/skills/donna-start/SKILL.md index 409b3118..d26d3f29 100644 --- a/donna/fixtures/skills/donna-start/SKILL.md +++ b/donna/fixtures/skills/donna-start/SKILL.md @@ -3,7 +3,7 @@ name: donna-start description: Start a new Donna session and use Donna to perform all further work. Use this skill when the developer explicitly told you to use it. --- -1. You **MUST** run `donna -p llm -r artifacts view '*:intro'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. +1. You **MUST** run `donna -p llm -r artifacts view '**/intro.donna.md'` when you start executing this skill, if you haven't done it yet. This command provides an introduction to the project and the Donna tool. 2. Run `donna -p llm -r sessions start` to start a new Donna session. 3. Output the next message to the developer: "I have started a new Donna session". 4. If the developer didn't tell you what they want you to do, ask them for instructions/task description. diff --git a/donna/fixtures/specs/intro.md b/donna/fixtures/specs/intro.donna.md similarity index 88% rename from donna/fixtures/specs/intro.md rename to donna/fixtures/specs/intro.donna.md index 2371158c..d4072f3b 100644 --- a/donna/fixtures/specs/intro.md +++ b/donna/fixtures/specs/intro.donna.md @@ -21,7 +21,7 @@ We may need coding agents on the each step of the process, but there no reason f ## Artifact Tags -To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. Artifacts in `donna:*` world use the next set of tags. +To simplify searching for artifacts by their semantics, Donna allows tagging artifacts with semantically valuable keywords. The synced Donna artifacts addressed under `**` use the next set of tags. Artifact type tags: @@ -30,7 +30,7 @@ Artifact type tags: ## Instructions -1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("donna:usage:cli") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. +1. On start of the YOUR session you **MUST** read and understand instruction on using the Donna tool `{{ donna.lib.view("./usage/cli.donna.md") }}`. It **MUST** be a one time operation. Do not repeat it unless you forget how to use the tool. 2. If you need to perform a work with Donna, you **MUST** select an appropriate Donna workflow to perform the work and run it. 3. If there is no appropriate workflow, ask the developer for a precise instructions on what to do. 4. If you are executing a workflow operation and need to perform a complex action or changes, you SHOULD search for an appropriate workflow and run it as a child workflow — it is the intended way to use Donna. @@ -39,7 +39,7 @@ Artifact type tags: ## Journaling -You MUST use `donna journal write` to track your actions and thoughts, according the description in `{{ donna.lib.view("donna:usage:cli") }}`. +You MUST use `donna journal write` to track your actions and thoughts, according the description in `{{ donna.lib.view("./usage/cli.donna.md") }}`. Journaling is a required part of workflow execution. An action request MUST be considered incomplete until required journal records are written. diff --git a/donna/fixtures/specs/research/specs/report.md b/donna/fixtures/specs/research/specs/report.donna.md similarity index 91% rename from donna/fixtures/specs/research/specs/report.md rename to donna/fixtures/specs/research/specs/report.donna.md index 2fdcc17b..08f4ab68 100644 --- a/donna/fixtures/specs/research/specs/report.md +++ b/donna/fixtures/specs/research/specs/report.donna.md @@ -4,19 +4,19 @@ kind = "donna.lib.specification" ``` -This document describes the format and structure of a Research Report document used by Donna workflows from `donna:research:*` namespace. +This document describes the format and structure of a Research Report document used by Donna workflows from `../**` namespace. ## Overview -Donna introduces a group of workflows located in `donna:research:*` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. +Donna introduces a group of workflows located in `../**` namespace that organize the process of researching a problem, collecting information, analyzing it, synthesizing options, and producing a final solution. -Session-related research artifacts MUST be stored as `session:research:`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. +Session-related research artifacts MUST be stored as `@/.donna/session/research/.donna.md`, unless the developer or parent workflow specifies a different location. The `` MUST be unique within the session. The agent (via workflows) creates the artifact and updates it iteratively as the research process progresses. ## Research report structure -The research report is a Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The research report is a Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** -- title and short description of the research problem. - **Original problem description** -- original problem statement from the developer or parent workflow. @@ -35,7 +35,7 @@ The research report is a Donna artifact (check `{{ donna.lib.view("donna:usage:a ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format diff --git a/.agents/donna/research/work/research.md b/donna/fixtures/specs/research/work/research.donna.md similarity index 91% rename from .agents/donna/research/work/research.md rename to donna/fixtures/specs/research/work/research.donna.md index 12a46588..a211f5eb 100644 --- a/.agents/donna/research/work/research.md +++ b/donna/fixtures/specs/research/work/research.donna.md @@ -20,8 +20,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:research:specs:report") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../specs/report.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_problem_description_exists") }}` ## Ensure problem description exists @@ -34,7 +34,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear description of the problem in your context. I.e., you know what you need to do in this workflow. 1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `session:*` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `@/.donna/session/**` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_artifact") }}`. 3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_artifact") }}`. ## Prepare research artifact @@ -44,8 +44,8 @@ id = "prepare_artifact" kind = "donna.lib.request_action" ``` -1. Based on the problem description you have, suggest an artifact name in the format `session:research:`. `` MUST be unique within the session. -{# TODO: we can add donna.lib.list('session:*') here as the command to list all artifacts in session #} +1. Based on the problem description you have, suggest an artifact name in the format `@/.donna/session/research/.donna.md`. `` MUST be unique within the session. +{# TODO: we can add donna.lib.list('@/.donna/session/**') here as the command to list all session artifacts #} 2. Create the artifact and specify an original problem description in it. 3. `{{ donna.lib.goto("formalize_research") }}` diff --git a/.agents/donna/rfc/specs/design.md b/donna/fixtures/specs/rfc/specs/design.donna.md similarity index 94% rename from .agents/donna/rfc/specs/design.md rename to donna/fixtures/specs/rfc/specs/design.donna.md index 020d0422..3a9d3b2b 100644 --- a/.agents/donna/rfc/specs/design.md +++ b/donna/fixtures/specs/rfc/specs/design.donna.md @@ -8,11 +8,11 @@ This document describes the format and structure of a Design document used to de ## Overview -Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. +Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. You create a Design document to explicitly describe the exact changes you want to make to the project in order to implement the RFC. -If not otherwise specified, Design documents for the session MUST be stored as `session:design:` artifacts in the session world. +If not otherwise specified, Design documents for the session MUST be stored as `@/.donna/session/design/.donna.md` artifacts under `/.donna/session`. **The Design document MUST list exact changes to the project that will be implemented.** E.g. concrete function names and signatures, file paths, data structures, etc. @@ -24,7 +24,7 @@ The Design document MUST NOT be a high-level description of the problem and solu ## Design document structure -The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** — title and short description of the proposed change. - **Inputs** — list of input documents that are relevant for the proposed change, starting from the RFC document. @@ -40,7 +40,7 @@ The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifa ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format diff --git a/donna/fixtures/specs/rfc/specs/request_for_change.md b/donna/fixtures/specs/rfc/specs/request_for_change.donna.md similarity index 92% rename from donna/fixtures/specs/rfc/specs/request_for_change.md rename to donna/fixtures/specs/rfc/specs/request_for_change.donna.md index 9c56893a..ecace694 100644 --- a/donna/fixtures/specs/rfc/specs/request_for_change.md +++ b/donna/fixtures/specs/rfc/specs/request_for_change.donna.md @@ -4,19 +4,19 @@ kind = "donna.lib.specification" ``` -This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `donna:rfc:*` namespace. This document is an input for a Design document creation. +This document describes the format and structure of a Request for Change (RFC) document used to propose changes to a project by Donna workflows from `../**` namespace. This document is an input for a Design document creation. ## Overview -Donna introduces a group of workflows located in `donna:rfc:*` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. +Donna introduces a group of workflows located in `../**` namespace that organize the process of proposing, reviewing, and implementing changes to a project via RFC and Design documents. You create RFC documents to propose changes to the project. -If not otherwise specified, RFC documents for the session MUST be stored as `session:rfc:` artifacts in the session world. +If not otherwise specified, RFC documents for the session MUST be stored as `@/.donna/session/rfc/.donna.md` artifacts under `/.donna/session`. ## RFC structure -The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifacts") }}`) with the next structure: +The RFC document is Donna artifact (check `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`) with the next structure: - **Primary section** — title and short description of the proposed change. - **Original description** — original description of the requested changes from the developer or parent workflow. @@ -34,7 +34,7 @@ The RFC document is Donna artifact (check `{{ donna.lib.view("donna:usage:artifa ## General language and format - You MUST follow [RFC 2119](https://www.rfc-editor.org/rfc/rfc2119.txt) for keywords like MUST, SHOULD, MAY, etc. -- You MUST follow `{{ donna.lib.view("donna:usage:artifacts") }}`. +- You MUST follow `{{ donna.lib.view("../../usage/artifacts.donna.md") }}`. - You MUST follow the structure specified in this document. ### List format @@ -136,7 +136,7 @@ Examples: - Bad: `- Use clean architecture.` - Good: `- The solution MUST be compatible with Python 3.12.` - Good: `- The solution MUST NOT introduce new runtime dependencies.` -- Good: `- The solution MUST follow the specification project:specs:abc` +- Good: `- The solution MUST follow the specification ../../../../specs/abc.donna.md` - Good: `MUST not change public CLI flags` ## `Requirements` section @@ -216,7 +216,7 @@ Examples: - Bad: `- Verify that authentication works correctly.` - Bad: `- Review the implementation manually.` - Good: `- Run test suite `tests/auth/test_login.py`; all tests MUST pass.` -- Good: `- Inspect artifact `project:specs:authenticationd`; it MUST exist and contain section "Login flow".` +- Good: `- Inspect artifact `../../../../specs/authentication.donna.md`; it MUST exist and contain section "Login flow".` - Good: `- Execute CLI command `tool login` with invalid credentials; command MUST exit with non-zero code.` ## `Deliverables` section @@ -238,7 +238,7 @@ Examples: - Bad: `- Implement authentication code` - Bad: `- Refactor auth module.` - Good: `- Module app/auth/authentication.py exists.` -- Good: `- Donna artifact project:specs:authentication exists.` +- Good: `- Donna artifact ../../../../specs/authentication.donna.md exists.` - Good: `- Test suite tests/auth/ exists.` ## `Action items` section @@ -259,7 +259,7 @@ Examples: - Bad: `- Work on authentication.` - Bad: `- Improve security everywhere.` - Bad: `- Fix the bugs A` -- Good: `- Create an artifact project:specs:authentication with sections "Login flow" and "Token lifecycle".` +- Good: `- Create an artifact ../../../../specs/authentication.donna.md with sections "Login flow" and "Token lifecycle".` - Good: `- Add test file tests/auth/test_login.py covering invalid credential cases.` - Good: `- Implement test tests/auth/test_login.py:TestLogin:test_invalid_credentials.` - Good: `- Update CLI help text to include login command description.` diff --git a/.agents/donna/rfc/work/design.md b/donna/fixtures/specs/rfc/work/design.donna.md similarity index 77% rename from .agents/donna/rfc/work/design.md rename to donna/fixtures/specs/rfc/work/design.donna.md index 54ec9bb5..18b8bfeb 100644 --- a/.agents/donna/rfc/work/design.md +++ b/donna/fixtures/specs/rfc/work/design.donna.md @@ -5,7 +5,7 @@ kind = "donna.lib.workflow" start_operation_id = "start" ``` -This workflow creates a Design document artifact based on an RFC and aligned with `donna:rfc:specs:design`. +This workflow creates a Design document artifact based on an RFC and aligned with `../specs/design.donna.md`. ## Start Work @@ -15,8 +15,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_rfc_artifact_exists") }}` ## Ensure RFC artifact exists @@ -29,7 +29,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear RFC to design. 1. If you have an RFC artifact id in your context, view it and `{{ donna.lib.goto("prepare_design_artifact") }}`. -2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. +2. If you have no RFC artifact id in your context, but you know it is in one of `{{ donna.lib.list("@/.donna/session/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_design_artifact") }}`. 3. If you have no RFC artifact id in your context, and you don't know where it is, ask the developer to provide the RFC artifact id or to create a new RFC. After you get it and view the artifact, `{{ donna.lib.goto("prepare_design_artifact") }}`. ## Prepare Design artifact @@ -39,7 +39,7 @@ id = "prepare_design_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to be `session:design:`, where `` SHOULD correspond to the RFC slug. +1. If the name of the artifact is not specified explicitly, assume it to be `@/.donna/session/design/.donna.md`, where `` SHOULD correspond to the RFC slug. 2. Save the next template into the artifact, replace `` with appropriate values. ~~~ @@ -81,7 +81,7 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:design") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/design.donna.md") }}` if you haven't done it yet. 2. Read the RFC artifact selected in the previous step if you haven't done it yet. 3. Analyze the project if needed to understand the requested change context. 4. Fill in all sections of the Design draft artifact. @@ -95,7 +95,7 @@ id = "review_design_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("donna:rfc:specs:design") }}`. +1. List mismatches between the Design artifact and the Design specification `{{ donna.lib.view("../specs/design.donna.md") }}`. 2. For each mismatch, make necessary edits to the Design draft artifact to ensure compliance. 3. `{{ donna.lib.goto("review_design_content") }}` @@ -106,7 +106,7 @@ id = "review_design_content" kind = "donna.lib.request_action" ``` -1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +1. Read the Design document and identify gaps, inconsistencies, or areas for improvement in accordance with the RFC and current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. 2. Make necessary edits to the Design draft artifact to address identified issues. 3. If there were changes made on this step or the previous `review_design_format` step `{{ donna.lib.goto("review_design_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/.agents/donna/rfc/work/do.md b/donna/fixtures/specs/rfc/work/do.donna.md similarity index 94% rename from .agents/donna/rfc/work/do.md rename to donna/fixtures/specs/rfc/work/do.donna.md index c39d9294..1af618a3 100644 --- a/.agents/donna/rfc/work/do.md +++ b/donna/fixtures/specs/rfc/work/do.donna.md @@ -76,7 +76,7 @@ kind = "donna.lib.request_action" 1. Choose the workflow to plan the work. If you created a Design document in the previous step, use it as a basis. 2. Run the chosen workflow. -3. Ensure you know the workflow id created in the previous step (default is `session:execute_rfc` if not specified). +3. Ensure you know the workflow id created in the previous step (default is `@/.donna/session/execute_rfc.donna.md` if not specified). 4. After completing the workflow `{{ donna.lib.goto("execute_rfc_work") }}`. ## Execute RFC work @@ -86,7 +86,7 @@ id = "execute_rfc_work" kind = "donna.lib.request_action" ``` -1. Run the workflow created by the plan step (default: `session:execute_rfc`) and complete it. +1. Run the workflow created by the plan step (default: `@/.donna/session/execute_rfc.donna.md`) and complete it. 2. After completing the workflow `{{ donna.lib.goto("polish_changes") }}`. ## Polish changes diff --git a/.agents/donna/rfc/work/plan.md b/donna/fixtures/specs/rfc/work/plan.donna.md similarity index 88% rename from .agents/donna/rfc/work/plan.md rename to donna/fixtures/specs/rfc/work/plan.donna.md index 180ab8dd..d620889f 100644 --- a/.agents/donna/rfc/work/plan.md +++ b/donna/fixtures/specs/rfc/work/plan.donna.md @@ -6,7 +6,7 @@ kind = "donna.lib.workflow" start_operation_id = "start" ``` -This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow in the `session:*` world with detailed steps to implement the designed changes. +This workflow plans the work required to implement a specified Design document. The RFC document SHOULD be used as a helper context. The result of this workflow is a new workflow stored as a `@/.donna/session/**` artifact under `/.donna/session` with detailed steps to implement the designed changes. ## Start Work @@ -18,7 +18,7 @@ fsm_mode = "start" 1. Read the Design document that the developer or parent workflow wants you to implement. 2. Read the RFC document that the developer or parent workflow wants you to implement, if it exists. -3. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +3. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 4. `{{ donna.lib.goto("prepare_workflow_artifact") }}` ## Prepare workflow artifact @@ -28,7 +28,7 @@ id = "prepare_workflow_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to `session:plans:`. +1. If the name of the artifact is not specified explicitly, assume it to `@/.donna/session/plans/.donna.md`. 2. Create a workflow with the next operations: - Start - A step for each action point in the RFC document and each item in the `Order of implementation` in Design document with the goal to minimize dependencies between steps and introduce changes incrementally. diff --git a/donna/fixtures/specs/rfc/work/request.md b/donna/fixtures/specs/rfc/work/request.donna.md similarity index 77% rename from donna/fixtures/specs/rfc/work/request.md rename to donna/fixtures/specs/rfc/work/request.donna.md index 4a612ba2..f66d58a3 100644 --- a/donna/fixtures/specs/rfc/work/request.md +++ b/donna/fixtures/specs/rfc/work/request.donna.md @@ -16,8 +16,8 @@ kind = "donna.lib.request_action" fsm_mode = "start" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. -2. Read the specification `{{ donna.lib.view("donna:usage:artifacts") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. +2. Read the specification `{{ donna.lib.view("../../usage/artifacts.donna.md") }}` if you haven't done it yet. 3. `{{ donna.lib.goto("ensure_work_description_exists") }}` ## Ensure work description exists @@ -30,7 +30,7 @@ kind = "donna.lib.request_action" At this point, you SHOULD have a clear description of the problem in your context. I.e. you know what you need to do in this workflow. 1. If you have a problem description in your context, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. -2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("session:**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. +2. If you have no problem description in your context, but you know it is in one of `{{ donna.lib.list("@/.donna/session/**") }}` artifacts, find and view it. Then `{{ donna.lib.goto("prepare_rfc_artifact") }}`. 3. If you have no problem description in your context, and you don't know where it is, ask the developer to provide it. After you get the problem description, `{{ donna.lib.goto("prepare_rfc_artifact") }}`. ## Prepare RFC artifact @@ -40,7 +40,7 @@ id = "prepare_rfc_artifact" kind = "donna.lib.request_action" ``` -1. If the name of the artifact is not specified explicitly, assume it to be `session:rfc:`, where `` MUST be unique within the session. +1. If the name of the artifact is not specified explicitly, assume it to be `@/.donna/session/rfc/.donna.md`, where `` MUST be unique within the session. 2. Save the next template into the artifact, replace `` with appropriate values. ~~~ @@ -86,7 +86,7 @@ id = "initial_fill" kind = "donna.lib.request_action" ``` -1. Read the specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}` if you haven't done it yet. +1. Read the specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}` if you haven't done it yet. 2. Analyze the project if needed to understand the context of the requested change. 3. Based on the problem description you have, fill in all sections of the RFC draft artifact. 4. `{{ donna.lib.goto("review_rfc_format") }}` @@ -98,7 +98,7 @@ id = "review_rfc_format" kind = "donna.lib.request_action" ``` -1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("donna:rfc:specs:request_for_change") }}`. +1. List mismatches between the RFC artifact and the RFC specification `{{ donna.lib.view("../specs/request_for_change.donna.md") }}`. 2. For each mismatch, make necessary edits to the RFC draft artifact to ensure compliance with the RFC specification. 3. `{{ donna.lib.goto("review_rfc_content") }}` @@ -109,7 +109,7 @@ id = "review_rfc_content" kind = "donna.lib.request_action" ``` -1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("donna:research:work:research") }}` workflow if you need to make a complex decision. +1. Read the RFC document and identify any gaps, inconsistencies, or areas for improvement in the content in accordance with the current project context. Use `{{ donna.lib.view("../../research/work/research.donna.md") }}` workflow if you need to make a complex decision. 2. Make necessary edits to the RFC draft artifact to address identified issues. 3. If there were changes made on this step or the previous `review_rfc_format` step `{{ donna.lib.goto("review_rfc_format") }}`. 4. If no changes were made, `{{ donna.lib.goto("finish") }}`. diff --git a/.agents/donna/usage/artifacts.md b/donna/fixtures/specs/usage/artifacts.donna.md similarity index 97% rename from .agents/donna/usage/artifacts.md rename to donna/fixtures/specs/usage/artifacts.donna.md index b87059b2..145923b0 100644 --- a/.agents/donna/usage/artifacts.md +++ b/donna/fixtures/specs/usage/artifacts.donna.md @@ -9,7 +9,7 @@ This format and behavior is what should be expected by default from an artifact ## Overview -An artifact is any text or binary document that Donna manages in its worlds. For example, via CLI commands `donna -p artifacts …`. +An artifact is any text or binary document that Donna manages in the project filesystem. For example, via CLI commands `donna -p artifacts …`. The text artifact has a source and one or more rendered representations, produced in specific rendering modes. @@ -22,7 +22,7 @@ To get information from the artifact, developers, agents and Donna view one of i **If you need an information from the artifact, you MUST view its representation**. Artifact sources are only for editing. -Read the specification `{{ donna.lib.view("donna:usage:cli") }}` to learn how to work with artifacts via Donna CLI. +Read the specification `{{ donna.lib.view("./cli.donna.md") }}` to learn how to work with artifacts via Donna CLI. ## Source Format and Rendering @@ -117,7 +117,7 @@ Artifacts can include semantic tags via a `tags` field in the section configurat Tags are used for deterministic artifact filtering and discovery (for example, via `donna -p artifacts list ... --predicate '"workflow" in section.tags'`). Tags are typically attached to the primary section and describe the artifact as a whole. -The canonical list of standard tags is documented in `donna:intro`. +The canonical list of standard tags is documented in `../intro.donna.md`. ## Section Kinds, Their Formats and Behaviors diff --git a/.agents/donna/usage/cli.md b/donna/fixtures/specs/usage/cli.donna.md similarity index 84% rename from .agents/donna/usage/cli.md rename to donna/fixtures/specs/usage/cli.donna.md index bf21ad4b..7cb3a138 100644 --- a/.agents/donna/usage/cli.md +++ b/donna/fixtures/specs/usage/cli.donna.md @@ -26,7 +26,6 @@ We may need coding agents on the each step of the process, but there no reason f ## Primary rules for agents -- Donna stores all project-related data in `.donna` directory in the project root. - All work is always done in the context of a session. There is only one active session at a time. - You MUST always work on one task assigned to you. - You MUST keep all the information about the session in your memory. @@ -109,7 +108,7 @@ After the session starts you MUST follow the next workflow to perform your work: 3. Start chosen workflow by calling `donna -p sessions run `. 4. Donna will output descriptions of all operations it performs to complete the work. 5. Donna will output **action requests** that you MUST perform. You MUST follow these instructions precisely. -6. When you done processing an action request, call `donna -p sessions action-request-completed ` to report request completion. `` MUST contain full identifier of the next operation, like `::`. +6. When you done processing an action request, call `donna -p sessions action-request-completed ` to report request completion. `` MUST contain the full identifier of the next operation, for example `@/.donna/session/execute_rfc.donna.md:review_changes`. 7. After you complete an action request, Donna will continue workflow execution and output what you need to do next. You MUST continue following Donna's instructions until the workflow is completed. @@ -138,28 +137,38 @@ If Donna tells you there is no work left, you MUST inform the developer that the ### Working with artifacts -An artifact is a markdown document with extra metadata stored in one of the Donna's worlds. +An artifact is a markdown document with extra metadata stored in the project workspace. Use the next commands to work with artifacts: -- `donna -p artifacts list []` — list all artifacts corresponding to the given pattern. If `` is omitted, list all artifacts in all worlds. Use this command when you need to find an artifact or see what artifacts are available. +- `donna -p artifacts list []` — list all artifacts corresponding to the given pattern. If `` is omitted, list all artifacts in the project workspace. Use this command when you need to find an artifact or see what artifacts are available. - `donna -p artifacts view ` — get the meaningful (rendered) content of all matching artifacts. This command shows the rendered information about each artifact. Use this command when you need to read artifact content. -- `donna -p artifacts validate []` — validate all artifacts corresponding to the given pattern. If `` is omitted, validate all artifacts in all worlds. +- `donna -p artifacts validate []` — validate all artifacts corresponding to the given pattern. If `` is omitted, validate all artifacts in the project workspace. -Donna does not mutate artifacts stored in worlds. Developers and external tools are responsible for creating, updating, moving, copying, or deleting world artifacts before Donna reads or validates them. +These commands only operate on artifact files admitted by the configured +`/.donna/config.toml:file_filters`. + +Donna does not mutate artifacts stored in the project workspace. Developers and external tools are responsible for creating, updating, moving, copying, or deleting artifacts before Donna reads or validates them. Commands that accept an artifact pattern (`artifacts list`, `artifacts view`, `artifacts validate`) also accept `--predicate/-p ` to filter by artifact primary section. The expression is evaluated as `bool` with `section` global available (for example: `--predicate '"workflow" in section.tags'`). The format of `` is as follows: -- full artifact identifier: `:` -- `*` — single wildcard matches a single level in the artifact path. Examples: - - `*:artifact:name` — matches all artifacts named `artifact:name` in all worlds. - - `world:*:name` — matches all artifacts with id `something:name` in the `world` world. -- `**` — double wildcard matches multiple levels in the artifact path. Examples: - - `**:name` — matches all artifacts with id ending with `:name` in all worlds. - - `world:**` — matches all artifacts in the `world` world. - - `world:**:name` — matches all artifacts with id ending with `:name` in the `world` world. +- full artifact identifier: `@/...` +- `/` separates path levels; wildcard characters do not match `/` +- `*` — matches zero or more characters inside one path level. Examples: + - `@/*.donna.md` — matches all artifacts directly under the project root. + - `@/**/test_*.donna.md` — matches artifacts whose filename starts with `test_` and ends with `.donna.md`. +- `?` — matches exactly one character inside one path level. Examples: + - `@/**/step?.donna.md` — matches `step1.donna.md` and `stepA.donna.md`, but not `step10.donna.md`. +- `[]` — character class that matches one character inside one path level. Examples: + - `@/**/step[0-9].donna.md` — matches artifacts with a single digit after `step`. + - `@/**/[ab]rchive.donna.md` — matches `archive.donna.md` and `brchive.donna.md`. +- `**` — recursive wildcard that matches zero or more path levels. Examples: + - `@/**/intro.donna.md` — matches all artifacts with filename `intro.donna.md` anywhere in the project workspace. + - `@/.donna/**` — matches all artifacts under `.donna`. + +CLI arguments MUST NOT use relative artifact paths like `./...` or `../../...`; use absolute `@/...` paths or rooted wildcard forms. ### Working with journal @@ -206,7 +215,7 @@ Agents MUST NOT log: 1. Direct instructions from the developer. 2. `AGENTS.md` document. - 3. Specifications in `project:` world. + 3. Project-relative specifications under `../../../specs/**` or `../**`. 4. This document. **All Donna CLI commands MUST include an explicit protocol selection using `-p `.** Like `donna -p llm `. diff --git a/donna/fixtures/specs/usage/worlds.donna.md b/donna/fixtures/specs/usage/worlds.donna.md new file mode 100644 index 00000000..70e08728 --- /dev/null +++ b/donna/fixtures/specs/usage/worlds.donna.md @@ -0,0 +1,42 @@ +# Donna Artifact Filesystem Layout + +```toml donna +kind = "donna.lib.specification" +``` + +This document describes how Donna discovers and manages its project artifacts on the filesystem. +Including usage docs, work workflows, operations, current work state and additional code. + +## Overview + +In order to function properly and to perform in a full potential, Donna relies on a set of artifacts +that guide its behavior and provide necessary capabilities. + +These artifacts are represented as text files, primary in Markdown format, however other text-based +formats can be used as well, if explicitly requested by the developer or by the workflows. + +Donna discovers these artifacts directly in the project filesystem rooted at ``. +The filesystem layout is still defined by code, but Donna MAY limit which files are visible as artifacts via +`/.donna/config.toml:file_filters`. + +The primary artifact areas are: + +- Artifacts under `/specs`, owned by the project itself. +- Synced Donna usage specs and workflows under `/.agents/donna`. +- Session artifacts under `/.donna/session`. + +The project filesystem has a free layout, defined by the developers who own the project. + +## Artifact Access + +Donna has read access to artifacts stored in the project filesystem. It discovers, fetches, renders, and validates project artifacts that are allowed by the configured file filters, but it does not create, update, move, copy, or delete them. + +Developers and external tools are responsible for mutating project artifacts before Donna reads or validates them. + +Donna still writes its own session state and journal data under `/.donna/session`, but that internal state storage is separate from project-artifact mutation. + +## Intro Artifacts + +It is a recommended practice to provide short introductory artifacts such as `../intro.donna.md` and `../../../specs/intro.donna.md` at meaningful roots inside the project filesystem. + +So, the agent can load the relevant introductions in commands such as `donna -p llm artifacts view '**/intro.donna.md'`. diff --git a/donna/fixtures/specs/usage/worlds.md b/donna/fixtures/specs/usage/worlds.md deleted file mode 100644 index 498f20e7..00000000 --- a/donna/fixtures/specs/usage/worlds.md +++ /dev/null @@ -1,43 +0,0 @@ -# Donna World Layout - -```toml donna -kind = "donna.lib.specification" -``` - -This document describes how Donna discovers and manages its dynamic and/or external artifacts. -Including usage docs, work workflows, operations, current work state and additional code. - -## Overview - -In order to function properly and to perform in a full potential, Donna relies on a set of artifacts -that guide its behavior and provide necessary capabilities. - -These artifacts are represented as text files, primary in Markdown format, however other text-based -formats can be used as well, if explicitly requested by the developer or by the workflows. - -Donna discovers these artifacts by scanning the "worlds" specified in `/.donna/config.toml` -as `worlds` list. Most of worlds are filesystem folders, however other world types can be implemented such as: -s3 buckets, git repositories, databases, etc. - -Default worlds and there locations are: - -- `donna` — `/.agents/donna` — the project-local bundled Donna specs installed from `donna/fixtures/specs` by workspace init/update. -- `home` — `~/.donna/home` — the user-level donna artifacts, i.e. those that should be visible for all workspaces on this machine. -- `project` — `/specs` — the project-level donna artifacts, i.e. those that are specific to this project. -- `session` — `/.donna/session` — the session world that contains the current state of work performed by Donna. - -All worlds have a free layout, defined by developers who own the particular world. - -## Artifact Access - -Donna has read access to artifacts stored in worlds. It discovers, fetches, renders, and validates world artifacts, but it does not create, update, move, copy, or delete them. - -Developers and external tools are responsible for mutating world artifacts before Donna reads or validates them. - -Donna still writes its own session state and journal data in the `session` world, but that internal state storage is separate from world-artifact mutation. - -## `:intro` artifact - -It is a recommended practice to provide a short introductory artifact `intro.md` at the root of each world. - -So, the agent can load descriptions of all worlds in a single command like `donna -p llm artifacts view "*:intro"`. diff --git a/donna/lib/worlds.py b/donna/lib/worlds.py deleted file mode 100644 index 42e85bc8..00000000 --- a/donna/lib/worlds.py +++ /dev/null @@ -1,5 +0,0 @@ -"""Shared world constructor instances for default configuration.""" - -from donna.workspaces.worlds.filesystem import FilesystemWorldConstructor - -filesystem = FilesystemWorldConstructor() diff --git a/donna/machine/action_requests.py b/donna/machine/action_requests.py index 53bddd73..e3a81367 100644 --- a/donna/machine/action_requests.py +++ b/donna/machine/action_requests.py @@ -1,7 +1,8 @@ import textwrap from donna.core.entities import BaseEntity -from donna.domain.ids import ActionRequestId, FullArtifactSectionId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import ActionRequestId from donna.protocol.cells import Cell from donna.protocol.nodes import Node @@ -9,11 +10,11 @@ class ActionRequest(BaseEntity): id: ActionRequestId | None request: str - operation_id: FullArtifactSectionId + operation_id: ArtifactSectionId title: str = "unknown" # TODO: remove default value after 2026.05.01 @classmethod - def build(cls, title: str, request: str, operation_id: FullArtifactSectionId) -> "ActionRequest": + def build(cls, title: str, request: str, operation_id: ArtifactSectionId) -> "ActionRequest": return cls( id=None, request=request, diff --git a/donna/machine/artifacts.py b/donna/machine/artifacts.py index 118a8be6..413f3db4 100644 --- a/donna/machine/artifacts.py +++ b/donna/machine/artifacts.py @@ -6,7 +6,9 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, FullArtifactId, PythonImportPath +from donna.domain.artifact_ids import ArtifactId +from donna.domain.ids import SectionId +from donna.domain.python_path import PythonPath from donna.machine.errors import ( ArtifactPrimarySectionMissing, ArtifactSectionNotFound, @@ -17,8 +19,8 @@ class ArtifactSectionConfig(BaseEntity): - id: ArtifactSectionId - kind: PythonImportPath + id: SectionId + kind: PythonPath tags: list[str] = pydantic.Field(default_factory=list) @@ -28,9 +30,9 @@ def cells_meta(self) -> dict[str, Any]: class ArtifactSection(BaseEntity): - id: ArtifactSectionId - artifact_id: FullArtifactId - kind: PythonImportPath + id: SectionId + artifact_id: ArtifactId + kind: PythonPath title: str description: str tags: list[str] = pydantic.Field(default_factory=list) @@ -46,7 +48,7 @@ def markdown_blocks(self) -> list[str]: class Artifact(BaseEntity): - id: FullArtifactId + id: ArtifactId sections: list[ArtifactSection] @@ -104,7 +106,7 @@ def validate_artifact(self) -> Result[None, ErrorsList]: # noqa: CCR001 return Ok(None) - def get_section(self, section_id: ArtifactSectionId | None) -> Result[ArtifactSection, ErrorsList]: + def get_section(self, section_id: SectionId | None) -> Result[ArtifactSection, ErrorsList]: if section_id is None: return self.primary_section() for section in self.sections: @@ -112,7 +114,7 @@ def get_section(self, section_id: ArtifactSectionId | None) -> Result[ArtifactSe return Ok(section) return Err([ArtifactSectionNotFound(artifact_id=self.id, section_id=section_id)]) - def get_section_number(self, section_id: ArtifactSectionId) -> int | None: + def get_section_number(self, section_id: SectionId) -> int | None: for index, section in enumerate(self.sections): if section.id == section_id: return index diff --git a/donna/machine/changes.py b/donna/machine/changes.py index f0acbbd0..632842f8 100644 --- a/donna/machine/changes.py +++ b/donna/machine/changes.py @@ -2,7 +2,8 @@ from typing import TYPE_CHECKING from donna.core.entities import BaseEntity -from donna.domain.ids import ActionRequestId, FullArtifactSectionId, TaskId, WorkUnitId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import ActionRequestId, TaskId, WorkUnitId from donna.machine.action_requests import ActionRequest from donna.machine.tasks import Task, WorkUnit @@ -24,7 +25,7 @@ def apply_to(self, state: "MutableState") -> None: class ChangeAddWorkUnit(Change): task_id: TaskId - operation_id: FullArtifactSectionId + operation_id: ArtifactSectionId def apply_to(self, state: "MutableState") -> None: work_unit = WorkUnit.build(id=state.next_work_unit_id(), task_id=self.task_id, operation_id=self.operation_id) @@ -32,7 +33,7 @@ def apply_to(self, state: "MutableState") -> None: class ChangeAddTask(Change): - operation_id: FullArtifactSectionId + operation_id: ArtifactSectionId def apply_to(self, state: "MutableState") -> None: task = Task.build( diff --git a/donna/machine/errors.py b/donna/machine/errors.py index 004e8023..7411a98e 100644 --- a/donna/machine/errors.py +++ b/donna/machine/errors.py @@ -1,5 +1,7 @@ from donna.core import errors as core_errors -from donna.domain.ids import ActionRequestId, ArtifactSectionId, FullArtifactId, FullArtifactSectionId +from donna.domain.artifact_ids import ArtifactId, ArtifactSectionId +from donna.domain.ids import SectionId +from donna.domain.internal_ids import ActionRequestId class InternalError(core_errors.InternalError): @@ -55,8 +57,8 @@ class InvalidOperationTransition(EnvironmentError): "Check the next operation id for typos.", "Use one of the allowed transitions listed in the action request.", ] - operation_id: FullArtifactSectionId - next_operation_id: FullArtifactSectionId + operation_id: ArtifactSectionId + next_operation_id: ArtifactSectionId class PrimitiveInvalidImportPath(EnvironmentError): @@ -129,8 +131,8 @@ class ArtifactPredicateEvaluationFailed(EnvironmentError): class ArtifactValidationError(EnvironmentError): cell_kind: str = "artifact_validation_error" - artifact_id: FullArtifactId - section_id: ArtifactSectionId | None = None + artifact_id: ArtifactId + section_id: SectionId | None = None def content_intro(self) -> str: if self.section_id: @@ -143,7 +145,7 @@ class MultiplePrimarySectionsError(ArtifactValidationError): code: str = "donna.artifacts.multiple_primary_sections" message: str = "Artifact must have exactly one primary section, found multiple: `{error.primary_sections}`" ways_to_fix: list[str] = ["Keep a single primary section in the artifact."] - primary_sections: list[ArtifactSectionId] + primary_sections: list[SectionId] class ArtifactPrimarySectionMissing(ArtifactValidationError): diff --git a/donna/machine/journal.py b/donna/machine/journal.py index 589e24c3..4066ccaa 100644 --- a/donna/machine/journal.py +++ b/donna/machine/journal.py @@ -8,9 +8,10 @@ from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error from donna.core.utils import now -from donna.domain.ids import FullArtifactSectionId, TaskId, WorkUnitId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import TaskId, WorkUnitId from donna.machine import errors as machine_errors -from donna.workspaces import utils as workspace_utils +from donna.workspaces import sessions as workspace_sessions from donna.workspaces.config import protocol as protocol_mode @@ -24,7 +25,7 @@ class JournalRecord(BaseEntity): message: str current_task_id: TaskId | None current_work_unit_id: WorkUnitId | None - current_operation_id: FullArtifactSectionId | None + current_operation_id: ArtifactSectionId | None @pydantic.field_validator("message", mode="after") @classmethod @@ -51,7 +52,7 @@ def deserialize_record(content: bytes) -> JournalRecord: @unwrap_to_error def reset() -> Result[None, ErrorsList]: - workspace_utils.session_world().unwrap().journal_reset().unwrap() + workspace_sessions.reset_journal() return Ok(None) @@ -87,7 +88,7 @@ def add( # noqa: CCR001 state = ctx.state.load().unwrap() parsed_task_id: TaskId | None = state.current_task.id if state.current_task else None parsed_work_unit_id: WorkUnitId | None = ctx.current_work_unit_id.get() - parsed_operation_id: FullArtifactSectionId | None = ctx.current_operation_id.get() + parsed_operation_id: ArtifactSectionId | None = ctx.current_operation_id.get() record = JournalRecord( timestamp=now(), @@ -99,7 +100,7 @@ def add( # noqa: CCR001 ) serialized = serialize_record(record) - workspace_utils.session_world().unwrap().journal_add(serialized).unwrap() + workspace_sessions.append_journal_record(serialized) instant_output_journal(record) @@ -107,15 +108,5 @@ def add( # noqa: CCR001 def read(lines: int | None = None, follow: bool = False) -> Iterable[Result[JournalRecord, ErrorsList]]: - session_world_result = workspace_utils.session_world() - if session_world_result.is_err(): - yield Err(session_world_result.unwrap_err()) - return - - raw_records = session_world_result.unwrap().journal_read(lines=lines, follow=follow) - for raw_record_result in raw_records: - if raw_record_result.is_err(): - yield Err(raw_record_result.unwrap_err()) - continue - - yield Ok(deserialize_record(raw_record_result.unwrap())) + for raw_record in workspace_sessions.read_journal(lines=lines, follow=follow): + yield Ok(deserialize_record(raw_record)) diff --git a/donna/machine/operations.py b/donna/machine/operations.py index 34b08d8c..feb15b8f 100644 --- a/donna/machine/operations.py +++ b/donna/machine/operations.py @@ -1,7 +1,7 @@ import enum from typing import TYPE_CHECKING, Any -from donna.domain.ids import ArtifactSectionId +from donna.domain.ids import SectionId from donna.machine.artifacts import ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.primitives import Primitive @@ -25,7 +25,7 @@ class OperationConfig(ArtifactSectionConfig): class OperationMeta(ArtifactSectionMeta): fsm_mode: FsmMode = FsmMode.normal - allowed_transtions: set[ArtifactSectionId] + allowed_transtions: set[SectionId] def cells_meta(self) -> dict[str, Any]: return {"fsm_mode": self.fsm_mode.value, "allowed_transtions": [str(t) for t in self.allowed_transtions]} diff --git a/donna/machine/primitives.py b/donna/machine/primitives.py index a6efb0e2..f5c940c3 100644 --- a/donna/machine/primitives.py +++ b/donna/machine/primitives.py @@ -6,7 +6,8 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, PythonImportPath +from donna.domain.ids import SectionId +from donna.domain.python_path import PythonPath from donna.machine import errors as machine_errors from donna.machine.artifacts import ArtifactSectionConfig @@ -15,9 +16,7 @@ from donna.machine.changes import Change from donna.machine.tasks import Task, WorkUnit from donna.workspaces.config import SourceConfig as SourceConfigModel - from donna.workspaces.config import WorldConfig from donna.workspaces.sources.base import SourceConfig as SourceConfigValue - from donna.workspaces.worlds.base import World # TODO: Currently it is a kind of God interface. It is convenient for now. @@ -25,7 +24,7 @@ class Primitive(BaseEntity): config_class: ClassVar[type[ArtifactSectionConfig]] = ArtifactSectionConfig - def validate_section(self, artifact: "Artifact", section_id: ArtifactSectionId) -> Result[None, ErrorsList]: + def validate_section(self, artifact: "Artifact", section_id: SectionId) -> Result[None, ErrorsList]: return Ok(None) def execute_section( @@ -40,11 +39,6 @@ def apply_directive(self, context: Context, *argv: Any, **kwargs: Any) -> Result primitive_name=self.__class__.__name__, method_name="apply_directive()" ) - def construct_world(self, config: "WorldConfig") -> "World": - raise machine_errors.PrimitiveMethodUnsupported( - primitive_name=self.__class__.__name__, method_name="construct_world()" - ) - def construct_source(self, config: "SourceConfigModel") -> "SourceConfigValue": raise machine_errors.PrimitiveMethodUnsupported( primitive_name=self.__class__.__name__, method_name="construct_source()" @@ -52,11 +46,11 @@ def construct_source(self, config: "SourceConfigModel") -> "SourceConfigValue": @unwrap_to_error -def resolve_primitive(primitive_id: PythonImportPath | str) -> Result[Primitive, ErrorsList]: # noqa: CCR001 - if isinstance(primitive_id, PythonImportPath): +def resolve_primitive(primitive_id: PythonPath | str) -> Result[Primitive, ErrorsList]: # noqa: CCR001 + if isinstance(primitive_id, PythonPath): import_path = str(primitive_id) else: - import_path = str(PythonImportPath.parse(primitive_id).unwrap()) + import_path = str(PythonPath.parse(primitive_id).unwrap()) if "." not in import_path: return Err([machine_errors.PrimitiveInvalidImportPath(import_path=import_path)]) diff --git a/donna/machine/sessions.py b/donna/machine/sessions.py index 0252f7f0..2359a5c2 100644 --- a/donna/machine/sessions.py +++ b/donna/machine/sessions.py @@ -4,14 +4,15 @@ from donna.context.context import context from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ActionRequestId, FullArtifactId, FullArtifactSectionId +from donna.domain.artifact_ids import ArtifactId, ArtifactSectionId +from donna.domain.internal_ids import ActionRequestId from donna.machine import errors as machine_errors from donna.machine import journal as machine_journal from donna.machine.operations import OperationMeta from donna.machine.state import ConsistentState, MutableState from donna.protocol.cell_shortcuts import operation_succeeded from donna.protocol.cells import Cell -from donna.workspaces import utils as workspace_utils +from donna.workspaces import sessions as workspace_sessions from donna.workspaces.artifacts import RENDER_CONTEXT_VIEW @@ -60,7 +61,7 @@ def wrapper(*args: P.args, **kwargs: P.kwargs) -> CellsResult: @unwrap_to_error def start() -> Result[list[Cell], ErrorsList]: - workspace_utils.session_world().unwrap().initialize(reset=True).unwrap() + workspace_sessions.reset_dir() machine_journal.reset().unwrap() _save_state(MutableState.build().freeze()).unwrap() @@ -78,7 +79,7 @@ def reset() -> Result[list[Cell], ErrorsList]: @unwrap_to_error def clear() -> Result[list[Cell], ErrorsList]: - workspace_utils.session_world().unwrap().initialize(reset=True).unwrap() + workspace_sessions.reset_dir() return Ok([operation_succeeded("Cleared session.")]) @@ -104,7 +105,7 @@ def details() -> Result[list[Cell], ErrorsList]: @_session_required @unwrap_to_error -def start_workflow(artifact_id: FullArtifactId) -> Result[list[Cell], ErrorsList]: # noqa: CCR001 +def start_workflow(artifact_id: ArtifactId) -> Result[list[Cell], ErrorsList]: # noqa: CCR001 static_state = load_state().unwrap() workflow = context().artifacts.load(artifact_id, RENDER_CONTEXT_VIEW).unwrap() primary_section = workflow.primary_section().unwrap() @@ -117,10 +118,10 @@ def start_workflow(artifact_id: FullArtifactId) -> Result[list[Cell], ErrorsList @unwrap_to_error def _validate_operation_transition( - state: MutableState, request_id: ActionRequestId, next_operation_id: FullArtifactSectionId + state: MutableState, request_id: ActionRequestId, next_operation_id: ArtifactSectionId ) -> Result[None, ErrorsList]: operation_id = state.get_action_request(request_id).unwrap().operation_id - workflow = context().artifacts.load(operation_id.full_artifact_id, RENDER_CONTEXT_VIEW).unwrap() + workflow = context().artifacts.load(operation_id.artifact_id, RENDER_CONTEXT_VIEW).unwrap() operation = workflow.get_section(operation_id.local_id).unwrap() assert isinstance(operation.meta, OperationMeta) @@ -136,7 +137,7 @@ def _validate_operation_transition( @_session_required @unwrap_to_error def complete_action_request( - request_id: ActionRequestId, next_operation_id: FullArtifactSectionId + request_id: ActionRequestId, next_operation_id: ArtifactSectionId ) -> Result[list[Cell], ErrorsList]: mutator = load_state().unwrap().mutator() _validate_operation_transition(mutator, request_id, next_operation_id).unwrap() diff --git a/donna/machine/state.py b/donna/machine/state.py index 121812bb..5f41bfbe 100644 --- a/donna/machine/state.py +++ b/donna/machine/state.py @@ -8,7 +8,8 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ActionRequestId, FullArtifactSectionId, InternalId, TaskId, WorkUnitId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import ActionRequestId, InternalId, TaskId, WorkUnitId from donna.machine import errors as machine_errors from donna.machine import journal as machine_journal from donna.machine.action_requests import ActionRequest @@ -155,7 +156,7 @@ def apply_changes(self, changes: Sequence[Change]) -> None: @unwrap_to_error def complete_action_request( - self, request_id: ActionRequestId, next_operation_id: FullArtifactSectionId + self, request_id: ActionRequestId, next_operation_id: ArtifactSectionId ) -> Result[None, ErrorsList]: current_task = self.current_task assert current_task is not None @@ -173,7 +174,7 @@ def complete_action_request( return Ok(None) @unwrap_to_error - def start_workflow(self, full_operation_id: FullArtifactSectionId) -> Result[None, ErrorsList]: + def start_workflow(self, full_operation_id: ArtifactSectionId) -> Result[None, ErrorsList]: workflow = context().artifacts.resolve_section(full_operation_id, RENDER_CONTEXT_VIEW).unwrap() machine_journal.add( diff --git a/donna/machine/tasks.py b/donna/machine/tasks.py index 45da5549..538b7d3d 100644 --- a/donna/machine/tasks.py +++ b/donna/machine/tasks.py @@ -4,7 +4,8 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactSectionId, TaskId, WorkUnitId +from donna.domain.artifact_ids import ArtifactSectionId +from donna.domain.internal_ids import TaskId, WorkUnitId if TYPE_CHECKING: from donna.machine.changes import Change @@ -12,11 +13,11 @@ class Task(BaseEntity): id: TaskId - workflow_id: FullArtifactSectionId + workflow_id: ArtifactSectionId context: dict[str, Any] @classmethod - def build(cls, id: TaskId, workflow_id: FullArtifactSectionId) -> "Task": + def build(cls, id: TaskId, workflow_id: ArtifactSectionId) -> "Task": return Task( id=id, workflow_id=workflow_id, @@ -27,7 +28,7 @@ def build(cls, id: TaskId, workflow_id: FullArtifactSectionId) -> "Task": class WorkUnit(BaseEntity): id: WorkUnitId task_id: TaskId - operation_id: FullArtifactSectionId + operation_id: ArtifactSectionId context: dict[str, Any] @classmethod @@ -35,7 +36,7 @@ def build( cls, id: WorkUnitId, task_id: TaskId, - operation_id: FullArtifactSectionId, + operation_id: ArtifactSectionId, context: dict[str, Any] | None = None, ) -> "WorkUnit": diff --git a/donna/primitives/artifacts/workflow.py b/donna/primitives/artifacts/workflow.py index 7d0f3006..8c8f6b03 100644 --- a/donna/primitives/artifacts/workflow.py +++ b/donna/primitives/artifacts/workflow.py @@ -5,7 +5,8 @@ from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, FullArtifactId +from donna.domain.artifact_ids import ArtifactId +from donna.domain.ids import SectionId from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.errors import ArtifactValidationError from donna.machine.operations import FsmMode, OperationMeta @@ -26,14 +27,14 @@ class WrongStartOperation(ArtifactValidationError): code: str = "donna.workflows.wrong_start_operation" message: str = "Can not find the start operation `{error.start_operation_id}` in the workflow." ways_to_fix: list[str] = ["Ensure that the artifact contains the section with the specified start operation ID."] - start_operation_id: ArtifactSectionId + start_operation_id: SectionId class SectionIsNotAnOperation(ArtifactValidationError): code: str = "donna.workflows.section_is_not_an_operation" message: str = "Section `{error.workflow_section_id}` is not an operation and cannot be part of the workflow." ways_to_fix: list[str] = ["Ensure that the section has a kind of one of operation primitives."] - workflow_section_id: ArtifactSectionId + workflow_section_id: SectionId class WorkflowSectionNotWorkflow(ArtifactValidationError): @@ -50,7 +51,7 @@ class FinalOperationHasTransitions(ArtifactValidationError): "Approach B: Change the `fsm_mode` of this operation from `final` to `normal`", "Approach C: Remove the `fsm_mode` setting from this operation, as `normal` is the default.", ] - workflow_section_id: ArtifactSectionId + workflow_section_id: SectionId class NoOutgoingTransitions(ArtifactValidationError): @@ -63,10 +64,10 @@ class NoOutgoingTransitions(ArtifactValidationError): "Approach B: Change the kind of this operation to `donna.lib.finish`", "Approach C: Mark this operation as final by setting its `fsm_mode` to `final`.", ] - workflow_section_id: ArtifactSectionId + workflow_section_id: SectionId -def find_workflow_sections(start_operation_id: ArtifactSectionId, artifact: Artifact) -> set[ArtifactSectionId]: +def find_workflow_sections(start_operation_id: SectionId, artifact: Artifact) -> set[SectionId]: workflow_sections = set() to_visit = [start_operation_id] @@ -93,7 +94,7 @@ def find_workflow_sections(start_operation_id: ArtifactSectionId, artifact: Arti class WorkflowConfig(ArtifactSectionConfig): - start_operation_id: ArtifactSectionId + start_operation_id: SectionId @pydantic.field_validator("tags", mode="after") @classmethod @@ -104,7 +105,7 @@ def ensure_workflow_tag(cls, value: list[str]) -> list[str]: class WorkflowMeta(ArtifactSectionMeta): - start_operation_id: ArtifactSectionId + start_operation_id: SectionId def cells_meta(self) -> dict[str, object]: return {"start_operation_id": str(self.start_operation_id)} @@ -115,7 +116,7 @@ class Workflow(MarkdownSectionMixin, Primitive): def markdown_construct_meta( self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, @@ -139,7 +140,7 @@ def execute_section( @unwrap_to_error def validate_section( # noqa: CCR001, CFQ001 - self, artifact: Artifact, section_id: ArtifactSectionId + self, artifact: Artifact, section_id: SectionId ) -> Result[None, ErrorsList]: section = artifact.get_section(section_id).unwrap() diff --git a/donna/primitives/directives/goto.py b/donna/primitives/directives/goto.py index b2f80c41..8ecb22ce 100644 --- a/donna/primitives/directives/goto.py +++ b/donna/primitives/directives/goto.py @@ -5,7 +5,7 @@ from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result -from donna.domain.ids import FullArtifactSectionId +from donna.domain.artifact_ids import ArtifactSectionId from donna.machine.templates import Directive, PreparedDirectiveResult from donna.workspaces import config as workspace_config @@ -36,7 +36,7 @@ def _prepare_arguments( return Ok((next_operation_id,)) - def render_view(self, context: Context, next_operation_id: FullArtifactSectionId) -> Result[Any, ErrorsList]: + def render_view(self, context: Context, next_operation_id: ArtifactSectionId) -> Result[Any, ErrorsList]: protocol = workspace_config.protocol().value root_dir = workspace_config.project_dir() return Ok( @@ -44,5 +44,5 @@ def render_view(self, context: Context, next_operation_id: FullArtifactSectionId f"sessions action-request-completed '{next_operation_id}'" ) - def render_analyze(self, context: Context, next_operation_id: FullArtifactSectionId) -> Result[Any, ErrorsList]: + def render_analyze(self, context: Context, next_operation_id: ArtifactSectionId) -> Result[Any, ErrorsList]: return Ok(f"$$donna {self.analyze_id} {next_operation_id.local_id} donna$$") diff --git a/donna/primitives/directives/list.py b/donna/primitives/directives/list.py index aa762486..a81007c7 100644 --- a/donna/primitives/directives/list.py +++ b/donna/primitives/directives/list.py @@ -5,7 +5,8 @@ from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactIdPattern +from donna.domain import errors as domain_errors +from donna.domain.artifact_ids import ArtifactIdPattern, normalize_path from donna.machine.artifacts import ArtifactPredicate from donna.machine.templates import Directive, PreparedDirectiveResult from donna.workspaces import config as workspace_config @@ -52,7 +53,14 @@ def _prepare_arguments( # noqa: CCR001 if keyword != "predicate": return Err([ListInvalidKeyword(keyword=keyword)]) - artifact_pattern = FullArtifactIdPattern.parse(str(argv[0])).unwrap() + artifact_pattern_text = str(argv[0]) + normalized = normalize_path(artifact_pattern_text, relative_to=context["artifact_id"], allow_wildcards=True) + if normalized is None: + return Err( + [domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=artifact_pattern_text)] + ) + + artifact_pattern = ArtifactIdPattern.parse(str(normalized)).unwrap() predicate = kwargs.get("predicate") if predicate is None: @@ -65,7 +73,7 @@ def _prepare_arguments( # noqa: CCR001 return Ok((artifact_pattern, parsed_predicate)) def render_view( - self, context: Context, artifact_pattern: FullArtifactIdPattern, predicate: ArtifactPredicate | None + self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None ) -> Result[Any, ErrorsList]: protocol = workspace_config.protocol().value root_dir = workspace_config.project_dir() @@ -79,7 +87,7 @@ def render_view( ) def render_analyze( - self, context: Context, artifact_pattern: FullArtifactIdPattern, predicate: ArtifactPredicate | None + self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None ) -> Result[Any, ErrorsList]: if predicate is None: return Ok(f"$$donna {self.analyze_id} {artifact_pattern} donna$$") diff --git a/donna/primitives/directives/view.py b/donna/primitives/directives/view.py index bd119d67..8637df9c 100644 --- a/donna/primitives/directives/view.py +++ b/donna/primitives/directives/view.py @@ -5,7 +5,8 @@ from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactIdPattern +from donna.domain import errors as domain_errors +from donna.domain.artifact_ids import ArtifactIdPattern, normalize_path from donna.machine.artifacts import ArtifactPredicate from donna.machine.templates import Directive, PreparedDirectiveResult from donna.workspaces import config as workspace_config @@ -52,7 +53,14 @@ def _prepare_arguments( # noqa: CCR001 if keyword != "predicate": return Err([ViewInvalidKeyword(keyword=keyword)]) - artifact_pattern = FullArtifactIdPattern.parse(str(argv[0])).unwrap() + artifact_pattern_text = str(argv[0]) + normalized = normalize_path(artifact_pattern_text, relative_to=context["artifact_id"], allow_wildcards=True) + if normalized is None: + return Err( + [domain_errors.InvalidIdPattern(id_type=ArtifactIdPattern.__name__, value=artifact_pattern_text)] + ) + + artifact_pattern = ArtifactIdPattern.parse(str(normalized)).unwrap() predicate = kwargs.get("predicate") if predicate is None: @@ -65,7 +73,7 @@ def _prepare_arguments( # noqa: CCR001 return Ok((artifact_pattern, parsed_predicate)) def render_view( - self, context: Context, artifact_pattern: FullArtifactIdPattern, predicate: ArtifactPredicate | None + self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None ) -> Result[Any, ErrorsList]: protocol = workspace_config.protocol().value root_dir = workspace_config.project_dir() @@ -79,7 +87,7 @@ def render_view( ) def render_analyze( - self, context: Context, artifact_pattern: FullArtifactIdPattern, predicate: ArtifactPredicate | None + self, context: Context, artifact_pattern: ArtifactIdPattern, predicate: ArtifactPredicate | None ) -> Result[Any, ErrorsList]: if predicate is None: return Ok(f"$$donna {self.analyze_id} {artifact_pattern} donna$$") diff --git a/donna/primitives/operations/finish_workflow.py b/donna/primitives/operations/finish_workflow.py index fbfb21c3..c551f3c8 100644 --- a/donna/primitives/operations/finish_workflow.py +++ b/donna/primitives/operations/finish_workflow.py @@ -2,7 +2,7 @@ from donna.core.errors import ErrorsList from donna.core.result import Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactId +from donna.domain.artifact_ids import ArtifactId from donna.machine.artifacts import ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.operations import FsmMode, OperationConfig, OperationKind, OperationMeta from donna.protocol import cell_shortcuts @@ -35,7 +35,7 @@ def execute_section( def markdown_construct_meta( self, - artifact_id: "FullArtifactId", + artifact_id: "ArtifactId", source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, diff --git a/donna/primitives/operations/output.py b/donna/primitives/operations/output.py index a99e28d1..6fff8800 100644 --- a/donna/primitives/operations/output.py +++ b/donna/primitives/operations/output.py @@ -2,7 +2,8 @@ from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, FullArtifactId +from donna.domain.artifact_ids import ArtifactId +from donna.domain.ids import SectionId from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.errors import ArtifactValidationError from donna.machine.operations import OperationConfig, OperationKind, OperationMeta @@ -25,11 +26,11 @@ class OutputMissingNextOperation(ArtifactValidationError): class OutputConfig(OperationConfig): - next_operation_id: ArtifactSectionId | None = None + next_operation_id: SectionId | None = None class OutputMeta(OperationMeta): - next_operation_id: ArtifactSectionId | None = None + next_operation_id: SectionId | None = None class Output(MarkdownSectionMixin, OperationKind): @@ -37,7 +38,7 @@ class Output(MarkdownSectionMixin, OperationKind): def markdown_construct_meta( self, - artifact_id: "FullArtifactId", + artifact_id: "ArtifactId", source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, @@ -45,7 +46,7 @@ def markdown_construct_meta( ) -> Result[ArtifactSectionMeta, ErrorsList]: output_config = cast(OutputConfig, section_config) - allowed_transitions: set[ArtifactSectionId] = set() + allowed_transitions: set[SectionId] = set() if output_config.next_operation_id is not None: allowed_transitions.add(output_config.next_operation_id) @@ -70,11 +71,11 @@ def execute_section( next_operation_id = meta.next_operation_id assert next_operation_id is not None - full_operation_id = unit.operation_id.full_artifact_id.to_full_local(next_operation_id) + full_operation_id = unit.operation_id.artifact_id.to_full_local(next_operation_id) return Ok([ChangeAddWorkUnit(task_id=task.id, operation_id=full_operation_id)]) - def validate_section(self, artifact: Artifact, section_id: ArtifactSectionId) -> Result[None, ErrorsList]: + def validate_section(self, artifact: Artifact, section_id: SectionId) -> Result[None, ErrorsList]: section = artifact.get_section(section_id).unwrap() meta = cast(OutputMeta, section.meta) diff --git a/donna/primitives/operations/request_action.py b/donna/primitives/operations/request_action.py index 3be5c584..a3ffd9ef 100644 --- a/donna/primitives/operations/request_action.py +++ b/donna/primitives/operations/request_action.py @@ -6,7 +6,8 @@ from donna.core.errors import ErrorsList from donna.core.result import Ok, Result, unwrap_to_error from donna.domain import errors as domain_errors -from donna.domain.ids import ArtifactSectionId, FullArtifactId +from donna.domain.artifact_ids import ArtifactId +from donna.domain.ids import SectionId from donna.machine.action_requests import ActionRequest from donna.machine.artifacts import ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.operations import FsmMode, OperationConfig, OperationKind, OperationMeta @@ -18,7 +19,7 @@ from donna.machine.tasks import Task, WorkUnit -def extract_transitions(text: str) -> set[ArtifactSectionId]: +def extract_transitions(text: str) -> set[SectionId]: """Extracts all transitions from the text of action request. Transition is specified as render of `goto` directive in the format: @@ -29,10 +30,10 @@ def extract_transitions(text: str) -> set[ArtifactSectionId]: pattern = r"\$\$donna\s+goto\s+([a-zA-Z0-9_\-./:]+)\s+donna\$\$" matches = re.findall(pattern, text) - transitions: set[ArtifactSectionId] = set() + transitions: set[SectionId] = set() for match in matches: - transition_result = ArtifactSectionId.parse(match) + transition_result = SectionId.parse(match) if transition_result.is_err(): raise domain_errors.InvalidIdentifier(value=match) transitions.add(transition_result.unwrap()) @@ -55,7 +56,7 @@ class RequestAction(MarkdownSectionMixin, OperationKind): def markdown_construct_meta( self, - artifact_id: "FullArtifactId", + artifact_id: "ArtifactId", source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, diff --git a/donna/primitives/operations/run_script.py b/donna/primitives/operations/run_script.py index 109a4d20..2296cbd1 100644 --- a/donna/primitives/operations/run_script.py +++ b/donna/primitives/operations/run_script.py @@ -9,7 +9,8 @@ from donna.core import errors as core_errors from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, FullArtifactId +from donna.domain.artifact_ids import ArtifactId +from donna.domain.ids import SectionId from donna.machine import journal as machine_journal from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.errors import ArtifactValidationError @@ -71,9 +72,9 @@ class RunScriptInvalidExitCode(ArtifactValidationError): class RunScriptConfig(OperationConfig): save_stdout_to: str | None = None save_stderr_to: str | None = None - goto_on_success: ArtifactSectionId | None = None - goto_on_failure: ArtifactSectionId | None = None - goto_on_code: dict[str, ArtifactSectionId] = pydantic.Field(default_factory=dict) + goto_on_success: SectionId | None = None + goto_on_failure: SectionId | None = None + goto_on_code: dict[str, SectionId] = pydantic.Field(default_factory=dict) timeout: int = 60 @@ -81,12 +82,12 @@ class RunScriptMeta(OperationMeta): script: str | None = None save_stdout_to: str | None = None save_stderr_to: str | None = None - goto_on_success: ArtifactSectionId | None = None - goto_on_failure: ArtifactSectionId | None = None - goto_on_code: dict[str, ArtifactSectionId] = pydantic.Field(default_factory=dict) + goto_on_success: SectionId | None = None + goto_on_failure: SectionId | None = None + goto_on_code: dict[str, SectionId] = pydantic.Field(default_factory=dict) timeout: int = 60 - def select_next_operation(self, exit_code: int) -> ArtifactSectionId: + def select_next_operation(self, exit_code: int) -> SectionId: if exit_code == 0: next_operation = self.goto_on_success else: @@ -104,7 +105,7 @@ class RunScript(MarkdownSectionMixin, OperationKind): @unwrap_to_error def markdown_construct_meta( self, - artifact_id: "FullArtifactId", + artifact_id: "ArtifactId", source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, @@ -114,7 +115,7 @@ def markdown_construct_meta( script = source.script().unwrap() if script is None: return Err([RunScriptMissingScriptBlock(artifact_id=artifact_id, section_id=run_config.id)]) - allowed_transitions: set[ArtifactSectionId] = set() + allowed_transitions: set[SectionId] = set() if run_config.goto_on_success is not None: allowed_transitions.add(run_config.goto_on_success) @@ -177,14 +178,12 @@ def execute_section( changes.append(ChangeSetTaskContext(task_id=task.id, key=meta.save_stderr_to, value=stderr)) next_operation = meta.select_next_operation(exit_code) - full_operation_id = unit.operation_id.full_artifact_id.to_full_local(next_operation) + full_operation_id = unit.operation_id.artifact_id.to_full_local(next_operation) changes.append(ChangeAddWorkUnit(task_id=task.id, operation_id=full_operation_id)) return Ok(changes) - def validate_section( # noqa: CCR001 - self, artifact: Artifact, section_id: ArtifactSectionId - ) -> Result[None, ErrorsList]: + def validate_section(self, artifact: Artifact, section_id: SectionId) -> Result[None, ErrorsList]: # noqa: CCR001 section = artifact.get_section(section_id).unwrap() meta = cast(RunScriptMeta, section.meta) diff --git a/donna/protocol/formatters/human.py b/donna/protocol/formatters/human.py index a24fce5c..3f6d230a 100644 --- a/donna/protocol/formatters/human.py +++ b/donna/protocol/formatters/human.py @@ -22,6 +22,9 @@ def format_cell(self, cell: Cell) -> bytes: lines.append("") lines.append(cell.content.strip()) + lines.append("") + lines.append("") + return "\n".join(lines).encode() def format_journal(self, record: JournalRecord) -> bytes: diff --git a/donna/protocol/formatters/llm.py b/donna/protocol/formatters/llm.py index c7a0c44c..8a56a653 100644 --- a/donna/protocol/formatters/llm.py +++ b/donna/protocol/formatters/llm.py @@ -23,8 +23,9 @@ def format_cell(self, cell: Cell) -> bytes: # noqa: CCR001 lines.append(cell.content.strip()) lines.append(f"--DONNA-CELL {id} END--") + lines.append("") - return "\n".join(lines).strip().encode() + return "\n".join(lines).encode() def format_journal(self, record: JournalRecord) -> bytes: timestamp = record.timestamp.isoformat() diff --git a/donna/workspaces/artifacts.py b/donna/workspaces/artifacts.py index 0eb5f26c..95565857 100644 --- a/donna/workspaces/artifacts.py +++ b/donna/workspaces/artifacts.py @@ -1,7 +1,20 @@ +import pathlib +from typing import TYPE_CHECKING, Iterator, Sequence + from donna.core.entities import BaseEntity +from donna.core.errors import ErrorsList +from donna.core.result import Err, Ok, Result, unwrap_to_error +from donna.domain.artifact_ids import ArtifactId, ArtifactIdPattern +from donna.domain.id_paths import NormalizedRawIdPath +from donna.domain.types import Milliseconds from donna.machine.tasks import Task, WorkUnit +from donna.workspaces import errors as world_errors from donna.workspaces.templates import RenderMode +if TYPE_CHECKING: + from donna.machine.artifacts import Artifact + from donna.workspaces import config as workspace_config + class ArtifactRenderContext(BaseEntity): primary_mode: RenderMode @@ -10,3 +23,229 @@ class ArtifactRenderContext(BaseEntity): RENDER_CONTEXT_VIEW = ArtifactRenderContext(primary_mode=RenderMode.view) + + +class FilesystemRawArtifact(BaseEntity): + source_id: str + path: pathlib.Path + + def get_bytes(self) -> bytes: + return self.path.read_bytes() + + @unwrap_to_error + def render(self, artifact_id: ArtifactId, render_context: ArtifactRenderContext) -> Result["Artifact", ErrorsList]: + return Ok(render_artifact_from_source(artifact_id, self.source_id, self.get_bytes(), render_context).unwrap()) + + +def _should_skip_directory(parts: list[str], name: str) -> bool: + # `.donna/tmp` contains scratch files and must not be treated as durable artifacts. + return parts == [".donna"] and name == "tmp" + + +def _match_supported_extension(path: pathlib.Path, supported_extensions: set[str]) -> str | None: + name = path.name.lower() + + for extension in sorted(supported_extensions, key=len, reverse=True): + if name.endswith(extension): + return extension + + return None + + +def _required_filters_match_prefix( + prefix_parts: Sequence[str], filters: Sequence["workspace_config.FileFilter"] +) -> bool: + from donna.workspaces import config as workspace_config + + for file_filter in filters: + if file_filter.mode != workspace_config.FileFilterMode.required: + continue + + if not file_filter.pattern.matches_prefix(prefix_parts): + return False + + return True + + +def _is_artifact_visible( # noqa: CCR001 + artifact_id: ArtifactId, filters: Sequence["workspace_config.FileFilter"] +) -> bool: + from donna.workspaces import config as workspace_config + + included = False + + for file_filter in filters: + if file_filter.mode == workspace_config.FileFilterMode.required: + if not file_filter.pattern.matches(artifact_id): + return False + + continue + + if included: + continue + + matches = file_filter.pattern.matches(artifact_id) + if not matches: + continue + + if file_filter.mode == workspace_config.FileFilterMode.ignore: + return False + + included = True + + return True + + +def _artifact_id_from_parts(parts: Sequence[str]) -> ArtifactId | None: + artifact_name = "/".join(parts) + if not ArtifactId.validate(artifact_name): + return None + + return ArtifactId(NormalizedRawIdPath(artifact_name)) + + +def _artifact_is_visible_in_workspace(artifact_id: ArtifactId) -> bool: + from donna.workspaces import config as workspace_config + + return _is_artifact_visible(artifact_id, workspace_config.config().file_filters) + + +def walk_filesystem(filters: list["workspace_config.FileFilter"]) -> Iterator[pathlib.Path]: # noqa: CCR001 + from donna.workspaces.config import config, project_dir + + root = project_dir() + if not root.exists() or not root.is_dir(): + return + + supported_extensions = config().supported_extensions() + + def walk(node: pathlib.Path, parts: list[str]) -> Iterator[pathlib.Path]: # noqa: CCR001 + for entry in sorted(node.iterdir(), key=lambda item: item.name): + if entry.is_dir(): + if _should_skip_directory(parts, entry.name): + continue + + next_parts = parts + [entry.name] + if not _required_filters_match_prefix(next_parts, filters): + continue + + yield from walk(entry, next_parts) + continue + + if not entry.is_file(): + continue + + if _match_supported_extension(entry, supported_extensions) is None: + continue + + artifact_parts = parts + [entry.name] + artifact_id = _artifact_id_from_parts(artifact_parts) + if artifact_id is None or not _is_artifact_visible(artifact_id, filters): + continue + + yield pathlib.Path(*artifact_parts) + + yield from walk(root, []) + + +def list_artifact_ids(pattern: ArtifactIdPattern) -> list[ArtifactId]: # noqa: CCR001 + from donna.workspaces import config as workspace_config + + filters = [ + *workspace_config.config().file_filters, + workspace_config.FileFilter(mode=workspace_config.FileFilterMode.required, pattern=pattern), + ] + + artifacts: list[ArtifactId] = [] + + for relative_path in walk_filesystem(filters): + artifact_id = _artifact_id_from_parts(relative_path.parts) + if artifact_id is None: + continue + + artifacts.append(artifact_id) + + return artifacts + + +def resolve_artifact_path(artifact_id: ArtifactId) -> Result[pathlib.Path | None, ErrorsList]: + from donna.workspaces.config import project_dir + + artifact_path = project_dir().joinpath(*artifact_id.parts) + if not artifact_path.parent.exists(): + return Ok(None) + + if not artifact_path.exists() or not artifact_path.is_file(): + return Ok(None) + + return Ok(artifact_path) + + +@unwrap_to_error +def fetch_artifact_bytes(artifact_id: ArtifactId) -> Result[tuple[str, bytes], ErrorsList]: + raw_artifact = fetch_raw_artifact(artifact_id).unwrap() + return Ok((raw_artifact.source_id, raw_artifact.get_bytes())) + + +@unwrap_to_error +def fetch_raw_artifact(artifact_id: ArtifactId) -> Result[FilesystemRawArtifact, ErrorsList]: + from donna.workspaces.config import config + + if not _artifact_is_visible_in_workspace(artifact_id): + return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id)]) + + artifact_path = resolve_artifact_path(artifact_id).unwrap() + if artifact_path is None: + return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id)]) + + supported_extension = _match_supported_extension(artifact_path, config().supported_extensions()) + if supported_extension is None: + return Err( + [ + world_errors.UnsupportedArtifactSourceExtension( + artifact_id=artifact_id, + extension="".join(artifact_path.suffixes).lower() or artifact_path.suffix.lower(), + ) + ] + ) + + source_config = config().find_source_for_extension(supported_extension) + if source_config is None: + return Err( + [ + world_errors.UnsupportedArtifactSourceExtension( + artifact_id=artifact_id, + extension=supported_extension, + ) + ] + ) + + return Ok( + FilesystemRawArtifact( + source_id=source_config.kind, + path=artifact_path, + ) + ) + + +@unwrap_to_error +def render_artifact_from_source( + artifact_id: ArtifactId, + source_id: str, + content: bytes, + render_context: ArtifactRenderContext, +) -> Result["Artifact", ErrorsList]: + from donna.workspaces.config import config + + source_config = config().get_source_config(source_id).unwrap() + return Ok(source_config.construct_artifact_from_bytes(artifact_id, content, render_context).unwrap()) + + +@unwrap_to_error +def has_artifact_changed(artifact_id: ArtifactId, since: Milliseconds) -> Result[bool, ErrorsList]: + artifact_path = resolve_artifact_path(artifact_id).unwrap() + + if artifact_path is None: + return Ok(True) + + return Ok((artifact_path.stat().st_mtime_ns // 1_000_000) > since) diff --git a/donna/workspaces/artifacts_discovery.py b/donna/workspaces/artifacts_discovery.py deleted file mode 100644 index 89fe7a1e..00000000 --- a/donna/workspaces/artifacts_discovery.py +++ /dev/null @@ -1,90 +0,0 @@ -import pathlib -from functools import lru_cache -from typing import Iterable, Protocol - -from donna.domain.ids import ArtifactId, FullArtifactId, FullArtifactIdPattern, WorldId -from donna.workspaces.config import config - - -class ArtifactListingNode(Protocol): - name: str - - def is_dir(self) -> bool: - """Return True when node is a directory.""" - ... - - def is_file(self) -> bool: - """Return True when node is a file.""" - ... - - def iterdir(self) -> Iterable["ArtifactListingNode"]: - """Iterate over child nodes.""" - ... - - -def list_artifacts_by_pattern( # noqa: CCR001 - *, - world_id: WorldId, - root: ArtifactListingNode | None, - pattern: FullArtifactIdPattern, -) -> list[ArtifactId]: - if pattern[0] not in {"*", "**"} and pattern[0] != str(world_id): - return [] - - if root is None or not root.is_dir(): - return [] - - pattern_parts = tuple(pattern) - world_prefix = (str(world_id),) - supported_extensions = config().supported_extensions() - artifacts: set[ArtifactId] = set() - - def walk(node: ArtifactListingNode, parts: list[str]) -> None: # noqa: CCR001 - for entry in sorted(node.iterdir(), key=lambda item: item.name): - if entry.is_dir(): - next_parts = parts + [entry.name] - if not _pattern_allows_prefix(pattern_parts, world_prefix + tuple(next_parts)): - continue - walk(entry, next_parts) - continue - - if not entry.is_file(): - continue - - extension = pathlib.Path(entry.name).suffix.lower() - if extension not in supported_extensions: - continue - - stem = entry.name[: -len(extension)] - artifact_name = ":".join(parts + [stem]) - if ArtifactId.validate(artifact_name): - artifact_id = ArtifactId(artifact_name) - full_id = FullArtifactId((world_id, artifact_id)) - if pattern.matches_full_id(full_id): - artifacts.add(artifact_id) - - walk(root, []) - - return list(sorted(artifacts)) - - -def _pattern_allows_prefix(pattern_parts: tuple[str, ...], prefix_parts: tuple[str, ...]) -> bool: - @lru_cache(maxsize=None) - def match_at(p_index: int, v_index: int) -> bool: # noqa: CCR001 - if v_index >= len(prefix_parts): - return True - - if p_index >= len(pattern_parts): - return False - - token = pattern_parts[p_index] - - if token == "**": # noqa: S105 - return match_at(p_index + 1, v_index) or match_at(p_index, v_index + 1) - - if token == "*" or token == prefix_parts[v_index]: # noqa: S105 - return match_at(p_index + 1, v_index + 1) - - return False - - return match_at(0, 0) diff --git a/donna/workspaces/config.py b/donna/workspaces/config.py index 7f5ba6a7..5a4490d3 100644 --- a/donna/workspaces/config.py +++ b/donna/workspaces/config.py @@ -1,5 +1,6 @@ from __future__ import annotations +import enum import pathlib from typing import TYPE_CHECKING, Any @@ -8,13 +9,12 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result -from donna.domain.ids import PythonImportPath, WorldId +from donna.domain.artifact_ids import ArtifactIdPattern +from donna.domain.python_path import PythonPath from donna.machine.primitives import resolve_primitive from donna.workspaces import errors as world_errors from donna.workspaces.sources.base import SourceConfig as SourceConfigValue from donna.workspaces.sources.base import SourceConstructor -from donna.workspaces.worlds.base import World as BaseWorld -from donna.workspaces.worlds.base import WorldConstructor if TYPE_CHECKING: from donna.protocol.modes import Mode @@ -22,23 +22,24 @@ DONNA_DIR_NAME = ".donna" DONNA_CONFIG_NAME = "config.toml" DONNA_WORLD_SESSION_DIR_NAME = "session" -DONNA_WORLD_PROJECT_DIR_NAME = "project" -DONNA_WORLD_HOME_DIR_NAME = "home" -DONNA_WORLD_PROJECT_PATH = pathlib.Path("specs") -class WorldConfig(BaseEntity): - kind: PythonImportPath - id: WorldId - session: bool +class SourceConfig(BaseEntity): + kind: PythonPath + extension: str model_config = pydantic.ConfigDict(extra="allow") -class SourceConfig(BaseEntity): - kind: PythonImportPath +class FileFilterMode(str, enum.Enum): + ignore = "ignore" + include = "include" + required = "required" - model_config = pydantic.ConfigDict(extra="allow") + +class FileFilter(BaseEntity): + mode: FileFilterMode + pattern: ArtifactIdPattern def _default_sources() -> list[SourceConfig]: @@ -46,77 +47,34 @@ def _default_sources() -> list[SourceConfig]: SourceConfig.model_validate( { "kind": "donna.lib.sources.markdown", + "extension": ".donna.md", } ), ] -def _create_default_worlds() -> list[WorldConfig]: +def _default_file_filters() -> list[FileFilter]: return [ - WorldConfig.model_validate( - { - "id": WorldId("donna"), - "kind": "donna.lib.worlds.filesystem", - "session": False, - "path": pathlib.Path(".agents") / "donna", - } - ), - WorldConfig.model_validate( - { - "id": WorldId("home"), - "kind": "donna.lib.worlds.filesystem", - "session": False, - "path": f"~/{DONNA_DIR_NAME}/{DONNA_WORLD_HOME_DIR_NAME}", - } - ), - WorldConfig.model_validate( - { - "id": WorldId("project"), - "kind": "donna.lib.worlds.filesystem", - "session": False, - "path": DONNA_WORLD_PROJECT_PATH, - } - ), - WorldConfig.model_validate( - { - "id": WorldId("session"), - "kind": "donna.lib.worlds.filesystem", - "session": True, - "path": pathlib.Path(DONNA_DIR_NAME) / DONNA_WORLD_SESSION_DIR_NAME, - } + FileFilter( + mode=FileFilterMode.include, pattern=ArtifactIdPattern.parse("@/.donna/session/**/*.donna.md").unwrap() ), + FileFilter(mode=FileFilterMode.include, pattern=ArtifactIdPattern.parse("@/.agents/**/*.donna.md").unwrap()), + FileFilter(mode=FileFilterMode.ignore, pattern=ArtifactIdPattern.parse(".*/**").unwrap()), + FileFilter(mode=FileFilterMode.include, pattern=ArtifactIdPattern.parse("**/*.donna.md").unwrap()), + FileFilter(mode=FileFilterMode.ignore, pattern=ArtifactIdPattern.parse("**").unwrap()), ] -def _default_worlds() -> list[WorldConfig]: - return _create_default_worlds() - - class Config(BaseEntity): - worlds: list[WorldConfig] = pydantic.Field(default_factory=_default_worlds) sources: list[SourceConfig] = pydantic.Field(default_factory=_default_sources) - _worlds_instances: list[BaseWorld] = pydantic.PrivateAttr(default_factory=list) + file_filters: list[FileFilter] = pydantic.Field(default_factory=_default_file_filters) _sources_instances: list[SourceConfigValue] = pydantic.PrivateAttr(default_factory=list) cache_lifetime: float = 1.0 def model_post_init(self, __context: Any) -> None: # noqa: CCR001 - worlds: list[BaseWorld] = [] sources: list[SourceConfigValue] = [] - for world_config in self.worlds: - primitive_result = resolve_primitive(world_config.kind) - if primitive_result.is_err(): - error = primitive_result.unwrap_err()[0] - raise ValueError(error.message.format(error=error)) - - primitive = primitive_result.unwrap() - - if not isinstance(primitive, WorldConstructor): - raise ValueError(f"World constructor '{world_config.kind}' is not supported") - - worlds.append(primitive.construct_world(world_config)) - for source_config in self.sources: primitive_result = resolve_primitive(source_config.kind) if primitive_result.is_err(): @@ -130,20 +88,8 @@ def model_post_init(self, __context: Any) -> None: # noqa: CCR001 sources.append(primitive.construct_source(source_config)) - object.__setattr__(self, "_worlds_instances", worlds) object.__setattr__(self, "_sources_instances", sources) - def get_world(self, world_id: WorldId) -> Result[BaseWorld, ErrorsList]: - for world in self._worlds_instances: - if world.id == world_id: - return Ok(world) - - return Err([world_errors.WorldNotConfigured(world_id=world_id)]) - - @property - def worlds_instances(self) -> list[BaseWorld]: - return list(self._worlds_instances) - @property def sources_instances(self) -> list[SourceConfigValue]: return list(self._sources_instances) @@ -173,8 +119,7 @@ def supported_extensions(self) -> set[str]: extensions: set[str] = set() for source in self._sources_instances: - for extension in source.supported_extensions: - extensions.add(extension) + extensions.add(source.extension) return extensions diff --git a/donna/workspaces/errors.py b/donna/workspaces/errors.py index 53d395ac..03eddea1 100644 --- a/donna/workspaces/errors.py +++ b/donna/workspaces/errors.py @@ -1,7 +1,7 @@ import pathlib from donna.core import errors as core_errors -from donna.domain.ids import ArtifactId, FullArtifactId, WorldId +from donna.domain.artifact_ids import ArtifactId class InternalError(core_errors.InternalError): @@ -43,16 +43,6 @@ class WorkspaceAlreadyInitialized(WorkspaceError): project_dir: pathlib.Path -class WorldError(WorkspaceError): - cell_kind: str = "world_error" - world_id: WorldId - - -class WorldNotConfigured(WorldError): - code: str = "donna.workspaces.world_not_configured" - message: str = "World with id `{error.world_id}` is not configured" - - class SourceError(WorkspaceError): cell_kind: str = "source_error" source_id: str @@ -64,51 +54,34 @@ class SourceConfigNotConfigured(SourceError): kind: str -class WorldStateStorageUnsupported(WorldError): - code: str = "donna.workspaces.state_storage_unsupported" - message: str = "World `{error.world_id}` does not support state storage" - ways_to_fix: list[str] = [ - "Use the session world.", - ] - - -class CanNotResetNonSessionWorld(WorldError): - code: str = "donna.workspaces.can_not_reset_non_session_world" - message: str = "Can not reset non-session world `{error.world_id}`" - ways_to_fix: list[str] = [ - "Mark the world as a session world in the config or do not initialize it.", - ] - - class ArtifactError(WorkspaceError): cell_kind: str = "artifact_error" artifact_id: ArtifactId - world_id: WorldId def content_intro(self) -> str: - return f"Error for artifact '{self.artifact_id}' in world '{self.world_id}'" + return f"Error for artifact '{self.artifact_id}'" class ArtifactNotFound(ArtifactError): code: str = "donna.workspaces.artifact_not_found" - message: str = "Artifact `{error.artifact_id}` does not exist in world `{error.world_id}`" + message: str = "Artifact `{error.artifact_id}` does not exist" ways_to_fix: list[str] = [ "Check the artifact id for typos.", - "Ensure the artifact exists in the specified world.", + "Ensure the artifact exists in the project workspace.", ] class ArtifactMultipleFiles(ArtifactError): code: str = "donna.workspaces.artifact_multiple_files" - message: str = "Artifact `{error.artifact_id}` has multiple files in world `{error.world_id}`" + message: str = "Artifact `{error.artifact_id}` has multiple source files" ways_to_fix: list[str] = [ - "Keep a single source file per artifact in the world.", + "Keep a single source file per artifact.", ] class UnsupportedArtifactSourceExtension(ArtifactError): code: str = "donna.workspaces.unsupported_artifact_source_extension" - message: str = "Unsupported artifact source extension `{error.extension}` in world `{error.world_id}`" + message: str = "Unsupported artifact source extension `{error.extension}` for `{error.artifact_id}`" ways_to_fix: list[str] = [ "Use a supported extension for the configured sources.", ] @@ -117,7 +90,7 @@ class UnsupportedArtifactSourceExtension(ArtifactError): class MarkdownError(WorkspaceError): cell_kind: str = "markdown_error" - artifact_id: FullArtifactId | None = None + artifact_id: ArtifactId | None = None def content_intro(self) -> str: if self.artifact_id is None: @@ -128,7 +101,7 @@ def content_intro(self) -> str: class TemplateDirectiveError(WorkspaceError): cell_kind: str = "template_directive_error" - artifact_id: FullArtifactId | None = None + artifact_id: ArtifactId | None = None def content_intro(self) -> str: if self.artifact_id is None: diff --git a/donna/workspaces/initialization.py b/donna/workspaces/initialization.py index 24605ad9..86c63759 100644 --- a/donna/workspaces/initialization.py +++ b/donna/workspaces/initialization.py @@ -8,10 +8,10 @@ from donna.core import errors as core_errors from donna.core import utils from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import WorldId from donna.protocol.modes import Mode from donna.workspaces import config from donna.workspaces import errors as world_errors +from donna.workspaces import sessions as workspace_sessions SKILLS_ROOT_DIR = pathlib.Path(".agents") / "skills" DONNA_SKILL_FIXTURE_DIR = pathlib.Path("fixtures") / "skills" @@ -124,11 +124,7 @@ def initialize_workspace( encoding="utf-8", ) - project_world = default_config.get_world(WorldId(config.DONNA_WORLD_PROJECT_DIR_NAME)).unwrap() - project_world.initialize().unwrap() - - session_world = default_config.get_world(WorldId(config.DONNA_WORLD_SESSION_DIR_NAME)).unwrap() - session_world.initialize().unwrap() + workspace_sessions.ensure_dir() if install_skills: _sync_donna_skill(project_dir) diff --git a/donna/workspaces/markdown.py b/donna/workspaces/markdown.py index 93b3b9bb..0688cb5d 100644 --- a/donna/workspaces/markdown.py +++ b/donna/workspaces/markdown.py @@ -9,7 +9,7 @@ from donna.core.entities import BaseEntity from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import FullArtifactId +from donna.domain.artifact_ids import ArtifactId from donna.workspaces import errors as world_errors @@ -118,7 +118,7 @@ def clear_heading(text: str) -> str: def _parse_h1( - sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: FullArtifactId | None + sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: ArtifactId | None ) -> Result[SyntaxTreeNode | None, ErrorsList]: if sections and any(section.level == SectionLevel.h1 for section in sections): return Err([world_errors.MarkdownMultipleH1Sections(artifact_id=artifact_id)]) @@ -140,7 +140,7 @@ def _parse_h1( def _parse_h2( - sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: FullArtifactId | None + sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: ArtifactId | None ) -> Result[SyntaxTreeNode | None, ErrorsList]: if not sections: @@ -160,7 +160,7 @@ def _parse_h2( def _parse_heading( - sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: FullArtifactId | None + sections: list[SectionSource], node: SyntaxTreeNode, artifact_id: ArtifactId | None ) -> Result[SyntaxTreeNode | None, ErrorsList]: if node.tag == "h1": @@ -180,7 +180,7 @@ def _parse_heading( def _parse_fence( # noqa: CCR001 sections: list[SectionSource], node: SyntaxTreeNode, - artifact_id: FullArtifactId | None, + artifact_id: ArtifactId | None, ) -> Result[SyntaxTreeNode | None, ErrorsList]: if not sections: return Err([world_errors.MarkdownH1SectionMustBeFirst(artifact_id=artifact_id)]) @@ -255,7 +255,7 @@ def _parse_others(sections: list[SectionSource], node: SyntaxTreeNode) -> Syntax @unwrap_to_error def parse( # noqa: CCR001, CFQ001 - text: str, *, artifact_id: FullArtifactId | None = None + text: str, *, artifact_id: ArtifactId | None = None ) -> Result[list[SectionSource], ErrorsList]: # pylint: disable=R0912, R0915 md = MarkdownIt("commonmark") # TODO: later we may want to customize it with plugins diff --git a/donna/workspaces/sessions.py b/donna/workspaces/sessions.py new file mode 100644 index 00000000..86ec942e --- /dev/null +++ b/donna/workspaces/sessions.py @@ -0,0 +1,129 @@ +import os +import pathlib +import shutil +import stat +import time +from collections.abc import Iterable + +from donna.workspaces.config import DONNA_DIR_NAME, DONNA_WORLD_SESSION_DIR_NAME, project_dir + + +def dir() -> pathlib.Path: + return project_dir() / DONNA_DIR_NAME / DONNA_WORLD_SESSION_DIR_NAME + + +def ensure_dir() -> None: + dir().mkdir(parents=True, exist_ok=True) + + +def reset_dir() -> None: + session_dir = dir() + if session_dir.exists(): + shutil.rmtree(session_dir) + + ensure_dir() + + +def read_state() -> bytes | None: + path = dir() / "state.json" + if not path.exists(): + return None + + return path.read_bytes() + + +def write_state(content: bytes) -> None: + path = dir() / "state.json" + ensure_dir() + path.write_bytes(content) + + +def reset_journal() -> None: + path = dir() / "journal.jsonl" + ensure_dir() + path.write_bytes(b"") + + +def append_journal_record(content: bytes) -> None: + path = dir() / "journal.jsonl" + ensure_dir() + + with path.open("ab") as stream: + stream.write(content.rstrip(b"\n")) + stream.write(b"\n") + + +def read_journal(lines: int | None = None, follow: bool = False) -> Iterable[bytes]: + path = dir() / "journal.jsonl" + + yield from _journal_read_some(path, lines=lines) + + if not follow: + return + + yield from _journal_follow(path) + + +def _journal_read_all(path: pathlib.Path) -> list[bytes]: + if not path.exists(): + return [] + + with path.open("rb") as stream: + return [line.rstrip(b"\n") for line in stream if line.strip()] + + +def _journal_file_identity(path: pathlib.Path) -> tuple[int, int] | None: + try: + path_stat = path.stat() + except FileNotFoundError: + return None + + if not stat.S_ISREG(path_stat.st_mode): + return None + + return (path_stat.st_dev, path_stat.st_ino) + + +def _journal_read_some(path: pathlib.Path, lines: int | None = None) -> Iterable[bytes]: + records = _journal_read_all(path) + + if lines is not None: + records = records[-lines:] if lines > 0 else [] + + yield from records + + +def _journal_follow(path: pathlib.Path, poll_interval: float = 0.25) -> Iterable[bytes]: # noqa: CCR001 + stream = None + stream_identity: tuple[int, int] | None = None + start_from_head = False + + while True: + file_identity = _journal_file_identity(path) + + if stream is not None and stream_identity != file_identity: + stream.close() + stream = None + stream_identity = None + + if file_identity is None or file_identity == stream_identity: + start_from_head = True + + if stream is None and file_identity is not None: + stream = path.open("rb") + + if not start_from_head: + stream.seek(0, os.SEEK_END) + + stream_identity = file_identity + + if stream is None: + time.sleep(poll_interval) + continue + + while line := stream.readline(): + line = line.rstrip(b"\n") + if line.strip(): + yield line + + time.sleep(poll_interval) diff --git a/donna/workspaces/sources/base.py b/donna/workspaces/sources/base.py index eb9afb57..41bab356 100644 --- a/donna/workspaces/sources/base.py +++ b/donna/workspaces/sources/base.py @@ -11,7 +11,7 @@ from donna.machine.primitives import Primitive if TYPE_CHECKING: - from donna.domain.ids import FullArtifactId + from donna.domain.artifact_ids import ArtifactId from donna.machine.artifacts import Artifact from donna.workspaces.artifacts import ArtifactRenderContext from donna.workspaces.config import SourceConfig as SourceConfigModel @@ -19,7 +19,7 @@ class SourceConfig(BaseEntity, ABC): kind: str - supported_extensions: list[str] = pydantic.Field(default_factory=list) + extension: str @classmethod def normalize_extension(cls, extension: str) -> str: @@ -36,24 +36,17 @@ def normalize_extension(cls, extension: str) -> str: return normalized - @pydantic.field_validator("supported_extensions") + @pydantic.field_validator("extension") @classmethod - def _normalize_supported_extensions(cls, values: list[str]) -> list[str]: - normalized: list[str] = [] - - for value in values: - extension = cls.normalize_extension(value) - if extension not in normalized: - normalized.append(extension) - - return normalized + def _normalize_extension(cls, value: str) -> str: + return cls.normalize_extension(value) def supports_extension(self, extension: str) -> bool: - return self.normalize_extension(extension) in self.supported_extensions + return self.normalize_extension(extension) == self.extension @abstractmethod def construct_artifact_from_bytes( # noqa: E704 - self, full_id: "FullArtifactId", content: bytes, render_context: "ArtifactRenderContext" + self, artifact_id: "ArtifactId", content: bytes, render_context: "ArtifactRenderContext" ) -> Result["Artifact", ErrorsList]: ... # noqa: E704 diff --git a/donna/workspaces/sources/markdown.py b/donna/workspaces/sources/markdown.py index 9aef79a9..40927f31 100644 --- a/donna/workspaces/sources/markdown.py +++ b/donna/workspaces/sources/markdown.py @@ -3,7 +3,10 @@ from donna.core.errors import ErrorsList from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactSectionId, FullArtifactId, PythonImportPath +from donna.domain.artifact_ids import ArtifactId +from donna.domain.id_paths import NormalizedRawIdPath +from donna.domain.ids import SectionId +from donna.domain.python_path import PythonPath from donna.machine.artifacts import Artifact, ArtifactSection, ArtifactSectionConfig, ArtifactSectionMeta from donna.machine.primitives import Primitive, resolve_primitive from donna.workspaces import errors as world_errors @@ -16,7 +19,7 @@ class MarkdownSectionConstructor(Protocol): def markdown_construct_section( self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, config: dict[str, Any], primary: bool = False, @@ -26,14 +29,14 @@ def markdown_construct_section( class Config(SourceConfig): kind: Literal["markdown"] = "markdown" - supported_extensions: list[str] = [".md", ".markdown"] - default_section_kind: PythonImportPath = PythonImportPath("donna.lib.text") - default_primary_section_id: ArtifactSectionId = ArtifactSectionId("primary") + extension: str = ".donna.md" + default_section_kind: PythonPath = PythonPath(NormalizedRawIdPath("donna.lib.text")) + default_primary_section_id: SectionId = SectionId("primary") def construct_artifact_from_bytes( - self, full_id: FullArtifactId, content: bytes, render_context: ArtifactRenderContext + self, artifact_id: ArtifactId, content: bytes, render_context: ArtifactRenderContext ) -> Result[Artifact, ErrorsList]: - return construct_artifact_from_bytes(full_id, content, render_context, self) + return construct_artifact_from_bytes(artifact_id, content, render_context, self) class MarkdownSourceConstructor(SourceConstructor): @@ -48,7 +51,7 @@ class MarkdownSectionMixin: def markdown_build_title( self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, section_config: ArtifactSectionConfig, primary: bool = False, @@ -57,7 +60,7 @@ def markdown_build_title( def markdown_build_description( self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, section_config: ArtifactSectionConfig, primary: bool = False, @@ -66,7 +69,7 @@ def markdown_build_description( def markdown_construct_meta( self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, section_config: ArtifactSectionConfig, description: str, @@ -77,7 +80,7 @@ def markdown_construct_meta( @unwrap_to_error def markdown_construct_section( # noqa: CCR001 self, - artifact_id: FullArtifactId, + artifact_id: ArtifactId, source: markdown.SectionSource, config: dict[str, Any], primary: bool = False, @@ -120,29 +123,29 @@ def markdown_construct_section( # noqa: CCR001 @unwrap_to_error def parse_artifact_content( - full_id: FullArtifactId, text: str, render_context: ArtifactRenderContext + artifact_id: ArtifactId, text: str, render_context: ArtifactRenderContext ) -> Result[list[markdown.SectionSource], ErrorsList]: # Parsing an artifact two times is not ideal, but it is straightforward approach that works for now. # We should consider optimizing this in the future if performance or stability becomes an issue. # For now let's wait till we have more artifact analysis logic and till more use cases emerge. - original_markdown_source = render(full_id, text, render_context).unwrap() - original_sections = markdown.parse(original_markdown_source, artifact_id=full_id).unwrap() + original_markdown_source = render(artifact_id, text, render_context).unwrap() + original_sections = markdown.parse(original_markdown_source, artifact_id=artifact_id).unwrap() analysis_context = render_context.replace(primary_mode=RenderMode.analysis) - analyzed_markdown_source = render(full_id, text, analysis_context).unwrap() - analyzed_sections = markdown.parse(analyzed_markdown_source, artifact_id=full_id).unwrap() + analyzed_markdown_source = render(artifact_id, text, analysis_context).unwrap() + analyzed_sections = markdown.parse(analyzed_markdown_source, artifact_id=artifact_id).unwrap() if len(original_sections) != len(analyzed_sections): raise world_errors.MarkdownSectionsCountMismatch( - artifact_id=full_id, + artifact_id=artifact_id, original_count=len(original_sections), analyzed_count=len(analyzed_sections), ) if not original_sections: # return Environment errors - return Err([world_errors.MarkdownArtifactWithoutSections(artifact_id=full_id)]) + return Err([world_errors.MarkdownArtifactWithoutSections(artifact_id=artifact_id)]) for original, analyzed in zip(original_sections, analyzed_sections): original.analysis_tokens.extend(analyzed.original_tokens) @@ -151,22 +154,22 @@ def parse_artifact_content( def construct_artifact_from_bytes( - full_id: FullArtifactId, content: bytes, render_context: ArtifactRenderContext, config: Config + artifact_id: ArtifactId, content: bytes, render_context: ArtifactRenderContext, config: Config ) -> Result[Artifact, ErrorsList]: - return construct_artifact_from_markdown_source(full_id, content.decode("utf-8"), render_context, config) + return construct_artifact_from_markdown_source(artifact_id, content.decode("utf-8"), render_context, config) @unwrap_to_error def construct_artifact_from_markdown_source( # noqa: CCR001 - full_id: FullArtifactId, content: str, render_context: ArtifactRenderContext, config: Config + artifact_id: ArtifactId, content: str, render_context: ArtifactRenderContext, config: Config ) -> Result[Artifact, ErrorsList]: - original_sections = parse_artifact_content(full_id, content, render_context).unwrap() + original_sections = parse_artifact_content(artifact_id, content, render_context).unwrap() head_config = dict(original_sections[0].config().unwrap()) head_kind_value = head_config["kind"] - if isinstance(head_kind_value, PythonImportPath): + if isinstance(head_kind_value, PythonPath): head_kind = head_kind_value else: - head_kind = PythonImportPath.parse(head_kind_value).unwrap() + head_kind = PythonPath.parse(head_kind_value).unwrap() if "id" not in head_config or head_config["id"] is None: head_config["id"] = config.default_primary_section_id @@ -176,7 +179,7 @@ def construct_artifact_from_markdown_source( # noqa: CCR001 markdown_primary_primitive = cast(MarkdownSectionMixin, primary_primitive) primary_section_result = markdown_primary_primitive.markdown_construct_section( - artifact_id=full_id, + artifact_id=artifact_id, source=original_sections[0], config=head_config, primary=True, @@ -186,20 +189,20 @@ def construct_artifact_from_markdown_source( # noqa: CCR001 primary_section = primary_section_result.unwrap() sections = construct_sections_from_markdown( - artifact_id=full_id, + artifact_id=artifact_id, sections=original_sections[1:], default_section_kind=config.default_section_kind, ).unwrap() sections = [primary_section, *sections] - return Ok(Artifact(id=full_id, sections=sections)) + return Ok(Artifact(id=artifact_id, sections=sections)) @unwrap_to_error def construct_sections_from_markdown( # noqa: CCR001 - artifact_id: FullArtifactId, + artifact_id: ArtifactId, sections: list[markdown.SectionSource], - default_section_kind: PythonImportPath, - primitive_overrides: dict[PythonImportPath, Primitive] | None = None, + default_section_kind: PythonPath, + primitive_overrides: dict[PythonPath, Primitive] | None = None, ) -> Result[list[ArtifactSection], ErrorsList]: constructed: list[ArtifactSection] = [] errors: ErrorsList = [] @@ -208,14 +211,14 @@ def construct_sections_from_markdown( # noqa: CCR001 data = dict(section.config().unwrap()) if "id" not in data or data["id"] is None: - data["id"] = ArtifactSectionId("markdown" + uuid.uuid4().hex.replace("-", "")) + data["id"] = SectionId("markdown" + uuid.uuid4().hex.replace("-", "")) if "kind" not in data: data["kind"] = default_section_kind kind_value = data["kind"] if isinstance(kind_value, str): - primitive_id = PythonImportPath.parse(kind_value).unwrap() + primitive_id = PythonPath.parse(kind_value).unwrap() else: primitive_id = kind_value @@ -236,8 +239,8 @@ def construct_sections_from_markdown( # noqa: CCR001 def _resolve_primitive( - primitive_id: PythonImportPath, - primitive_overrides: dict[PythonImportPath, Primitive] | None = None, + primitive_id: PythonPath, + primitive_overrides: dict[PythonPath, Primitive] | None = None, ) -> Result[Primitive, ErrorsList]: if primitive_overrides is not None and primitive_id in primitive_overrides: return Ok(primitive_overrides[primitive_id]) @@ -247,7 +250,7 @@ def _resolve_primitive( def _ensure_markdown_constructible( primitive: Primitive, - primitive_id: PythonImportPath | str | None = None, + primitive_id: PythonPath | str | None = None, ) -> Result[None, ErrorsList]: if isinstance(primitive, MarkdownSectionMixin): return Ok(None) diff --git a/donna/workspaces/templates.py b/donna/workspaces/templates.py index a7c58377..53c9902c 100644 --- a/donna/workspaces/templates.py +++ b/donna/workspaces/templates.py @@ -10,7 +10,7 @@ from donna.core import errors as core_errors from donna.core.errors import EnvironmentErrorsProxy, ErrorsList from donna.core.result import Err, Ok, Result -from donna.domain.ids import FullArtifactId +from donna.domain.artifact_ids import ArtifactId from donna.machine.templates import Directive from donna.workspaces import errors as world_errors @@ -163,9 +163,7 @@ def env() -> jinja2.Environment: return _ENVIRONMENT -def render( - artifact_id: FullArtifactId, template: str, render_context: "ArtifactRenderContext" -) -> Result[str, ErrorsList]: +def render(artifact_id: ArtifactId, template: str, render_context: "ArtifactRenderContext") -> Result[str, ErrorsList]: context = {"render_mode": render_context.primary_mode, "artifact_id": artifact_id} if render_context.current_task is not None: diff --git a/donna/workspaces/utils.py b/donna/workspaces/utils.py deleted file mode 100644 index 8c1a8575..00000000 --- a/donna/workspaces/utils.py +++ /dev/null @@ -1,15 +0,0 @@ -from donna.core.errors import ErrorsList -from donna.core.result import Ok, Result, unwrap_to_error -from donna.domain.ids import WorldId -from donna.workspaces.config import config -from donna.workspaces.worlds.base import World - - -@unwrap_to_error -def session_world() -> Result[World, ErrorsList]: - world = config().get_world(WorldId("session")).unwrap() - - if not world.is_initialized(): - world.initialize(reset=False).unwrap() - - return Ok(world) diff --git a/donna/workspaces/worlds/__init__.py b/donna/workspaces/worlds/__init__.py deleted file mode 100644 index e69de29b..00000000 diff --git a/donna/workspaces/worlds/base.py b/donna/workspaces/worlds/base.py deleted file mode 100644 index 3e79db8a..00000000 --- a/donna/workspaces/worlds/base.py +++ /dev/null @@ -1,86 +0,0 @@ -from __future__ import annotations - -from abc import ABC, abstractmethod -from collections.abc import Iterable -from typing import TYPE_CHECKING - -from donna.core.entities import BaseEntity -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result -from donna.domain.ids import ArtifactId, FullArtifactId, FullArtifactIdPattern, WorldId -from donna.domain.types import Milliseconds -from donna.machine.artifacts import Artifact -from donna.machine.primitives import Primitive - -if TYPE_CHECKING: - from donna.workspaces.artifacts import ArtifactRenderContext - from donna.workspaces.config import WorldConfig - - -class RawArtifact(BaseEntity, ABC): - source_id: str - - @abstractmethod - def get_bytes(self) -> bytes: ... # noqa: E704 - - @abstractmethod - def render(self, full_id: FullArtifactId, render_context: "ArtifactRenderContext") -> Result[Artifact, ErrorsList]: - pass - - -class World(BaseEntity, ABC): - id: WorldId - session: bool = False - - @abstractmethod - def has(self, artifact_id: ArtifactId) -> bool: ... # noqa: E704 - - @abstractmethod - def fetch(self, artifact_id: ArtifactId) -> Result[RawArtifact, ErrorsList]: ... # noqa: E704 - - @abstractmethod - def has_artifact_changed(self, artifact_id: ArtifactId, since: Milliseconds) -> Result[bool, ErrorsList]: - pass - - @abstractmethod - def list_artifacts(self, pattern: FullArtifactIdPattern) -> list[ArtifactId]: ... # noqa: E704 - - # These two methods are intended for storing world state (e.g., session data) - # It is an open question if the world state is an artifact itself or something else - # For the artifact: uniform API for storing/loading data - # Against the artifact: - # - session data MUST be accessible only by Donna => no one should be able to read/write/list it - # - session data will require an additional kind(s) of artifact(s) just for that purpose - # - session data may change more frequently than regular artifacts - - @abstractmethod - def read_state(self, name: str) -> Result[bytes | None, ErrorsList]: ... # noqa: E704 - - @abstractmethod - def write_state(self, name: str, content: bytes) -> Result[None, ErrorsList]: ... # noqa: E704 - - @abstractmethod - def journal_reset(self) -> Result[None, ErrorsList]: ... # noqa: E704 - - @abstractmethod - def journal_add(self, content: bytes) -> Result[None, ErrorsList]: ... # noqa: E704 - - @abstractmethod - def journal_read(self, lines: int | None = None, follow: bool = False) -> Iterable[Result[bytes, ErrorsList]]: - pass - - def initialize(self, reset: bool = False) -> Result[None, ErrorsList]: - from donna.workspaces import errors as world_errors - - if reset and not self.session: - return Err([world_errors.CanNotResetNonSessionWorld(world_id=self.id)]) - - return Ok(None) - - @abstractmethod - def is_initialized(self) -> bool: ... # noqa: E704 - - -class WorldConstructor(Primitive, ABC): - @abstractmethod - def construct_world(self, config: WorldConfig) -> World: ... # noqa: E704 diff --git a/donna/workspaces/worlds/filesystem.py b/donna/workspaces/worlds/filesystem.py deleted file mode 100644 index 2ed1bf1b..00000000 --- a/donna/workspaces/worlds/filesystem.py +++ /dev/null @@ -1,285 +0,0 @@ -import os -import pathlib -import shutil -import stat -import time -from collections.abc import Iterable -from typing import TYPE_CHECKING, cast - -from donna.core.errors import ErrorsList -from donna.core.result import Err, Ok, Result, unwrap_to_error -from donna.domain.ids import ArtifactId, FullArtifactId, FullArtifactIdPattern -from donna.domain.types import Milliseconds -from donna.workspaces import errors as world_errors -from donna.workspaces.artifacts_discovery import ArtifactListingNode, list_artifacts_by_pattern -from donna.workspaces.worlds.base import RawArtifact -from donna.workspaces.worlds.base import World as BaseWorld -from donna.workspaces.worlds.base import WorldConstructor - -if TYPE_CHECKING: - from donna.machine.artifacts import Artifact - from donna.workspaces.artifacts import ArtifactRenderContext - from donna.workspaces.config import WorldConfig - - -class FilesystemRawArtifact(RawArtifact): - path: pathlib.Path - - def get_bytes(self) -> bytes: - return self.path.read_bytes() - - @unwrap_to_error - def render( - self, full_id: FullArtifactId, render_context: "ArtifactRenderContext" - ) -> Result["Artifact", ErrorsList]: - from donna.workspaces.config import config - - source_config = config().get_source_config(self.source_id).unwrap() - return Ok(source_config.construct_artifact_from_bytes(full_id, self.get_bytes(), render_context).unwrap()) - - -class World(BaseWorld): - path: pathlib.Path - _journal_file_name = "journal.jsonl" - - def _journal_path(self) -> pathlib.Path: - return self.path / self._journal_file_name - - def _artifact_listing_root(self) -> ArtifactListingNode | None: - if not self.path.exists(): - return None - - return cast(ArtifactListingNode, self.path) - - def _resolve_artifact_file(self, artifact_id: ArtifactId) -> Result[pathlib.Path | None, ErrorsList]: - artifact_path = self.path / artifact_id.replace(":", "/") - parent = artifact_path.parent - - if not parent.exists(): - return Ok(None) - - from donna.workspaces.config import config - - supported_extensions = config().supported_extensions() - matches = [ - path - for path in parent.glob(f"{artifact_path.name}.*") - if path.is_file() and path.suffix.lower() in supported_extensions - ] - - if not matches: - return Ok(None) - - if len(matches) > 1: - return Err([world_errors.ArtifactMultipleFiles(artifact_id=artifact_id, world_id=self.id)]) - - return Ok(matches[0]) - - def has(self, artifact_id: ArtifactId) -> bool: - resolve_result = self._resolve_artifact_file(artifact_id) - if resolve_result.is_err(): - return True - - return resolve_result.unwrap() is not None - - @unwrap_to_error - def fetch(self, artifact_id: ArtifactId) -> Result[RawArtifact, ErrorsList]: - path = self._resolve_artifact_file(artifact_id).unwrap() - if path is None: - return Err([world_errors.ArtifactNotFound(artifact_id=artifact_id, world_id=self.id)]) - - from donna.workspaces.config import config - - source_config = config().find_source_for_extension(path.suffix) - if source_config is None: - return Err( - [ - world_errors.UnsupportedArtifactSourceExtension( - artifact_id=artifact_id, - world_id=self.id, - extension=path.suffix, - ) - ] - ) - - return Ok( - FilesystemRawArtifact( - source_id=source_config.kind, - path=path, - ) - ) - - @unwrap_to_error - def has_artifact_changed(self, artifact_id: ArtifactId, since: Milliseconds) -> Result[bool, ErrorsList]: - path = self._resolve_artifact_file(artifact_id).unwrap() - - if path is None: - return Ok(True) - - return Ok((path.stat().st_mtime_ns // 1_000_000) > since) - - def read_state(self, name: str) -> Result[bytes | None, ErrorsList]: - if not self.session: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - path = self.path / name - - if not path.exists(): - return Ok(None) - - return Ok(path.read_bytes()) - - def write_state(self, name: str, content: bytes) -> Result[None, ErrorsList]: - if not self.session: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - path = self.path / name - path.parent.mkdir(parents=True, exist_ok=True) - path.write_bytes(content) - return Ok(None) - - def journal_reset(self) -> Result[None, ErrorsList]: - if not self.session: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - path = self._journal_path() - path.parent.mkdir(parents=True, exist_ok=True) - path.write_bytes(b"") - return Ok(None) - - def journal_add(self, content: bytes) -> Result[None, ErrorsList]: - if not self.session: - return Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - - path = self._journal_path() - path.parent.mkdir(parents=True, exist_ok=True) - - with path.open("ab") as stream: - stream.write(content.rstrip(b"\n")) - stream.write(b"\n") - - return Ok(None) - - def _journal_read_all(self, path: pathlib.Path) -> list[bytes]: - if not path.exists(): - return [] - - with path.open("rb") as stream: - return [line.rstrip(b"\n") for line in stream if line.strip()] - - def _journal_file_identity(self, path: pathlib.Path) -> tuple[int, int] | None: - try: - path_stat = path.stat() - except FileNotFoundError: - return None - - if not stat.S_ISREG(path_stat.st_mode): - return None - - return (path_stat.st_dev, path_stat.st_ino) - - def _journal_follow( # noqa: CCR001 - self, - poll_interval: float = 0.25, - ) -> Iterable[Result[bytes, ErrorsList]]: - path = self._journal_path() - - stream = None - stream_identity: tuple[int, int] | None = None - - # if the journal file did exist when we started following, we want to read from the end - # if the journal file didn't exist when we started following, we want to read from the start - start_from_head = False - - while True: - file_identity = self._journal_file_identity(path) - - if stream is not None and stream_identity != file_identity: - stream.close() - stream = None - stream_identity = None - - if file_identity is None or file_identity == stream_identity: - start_from_head = True - - if stream is None and file_identity is not None: - stream = path.open("rb") - - if not start_from_head: - stream.seek(0, os.SEEK_END) - - stream_identity = file_identity - - if stream is None: - time.sleep(poll_interval) - continue - - while line := stream.readline(): - line = line.rstrip(b"\n") - if line.strip(): - yield Ok(line) - - time.sleep(poll_interval) - - def _journal_read_some(self, lines: int | None = None) -> Iterable[Result[bytes, ErrorsList]]: - path = self._journal_path() - - records = self._journal_read_all(path) - - if lines is not None: - records = records[-lines:] if lines > 0 else [] - - for record in records: - yield Ok(record) - - def journal_read(self, lines: int | None = None, follow: bool = False) -> Iterable[Result[bytes, ErrorsList]]: - if not self.session: - yield Err([world_errors.WorldStateStorageUnsupported(world_id=self.id)]) - return - - yield from self._journal_read_some(lines=lines) - - if not follow: - return - - yield from self._journal_follow() - - def list_artifacts(self, pattern: FullArtifactIdPattern) -> list[ArtifactId]: # noqa: CCR001 - return list_artifacts_by_pattern( - world_id=self.id, - root=self._artifact_listing_root(), - pattern=pattern, - ) - - @unwrap_to_error - def initialize(self, reset: bool = False) -> Result[None, ErrorsList]: - super().initialize(reset=reset).unwrap() - - if self.path.exists() and reset: - shutil.rmtree(self.path) - - self.path.mkdir(parents=True, exist_ok=True) - return Ok(None) - - def is_initialized(self) -> bool: - return self.path.exists() - - -class FilesystemWorldConstructor(WorldConstructor): - def construct_world(self, config: "WorldConfig") -> World: - path_value = getattr(config, "path", None) - - if path_value is None: - raise ValueError(f"World config '{config.id}' does not define a filesystem path") - - from donna.workspaces.config import project_dir - - path = pathlib.Path(path_value).expanduser() - if not path.is_absolute(): - path = project_dir() / path - - return World( - id=config.id, - path=path.resolve(), - session=config.session, - ) diff --git a/specs/core/error_handling.md b/specs/core/error_handling.donna.md similarity index 97% rename from specs/core/error_handling.md rename to specs/core/error_handling.donna.md index e26c7eae..f309699f 100644 --- a/specs/core/error_handling.md +++ b/specs/core/error_handling.donna.md @@ -272,12 +272,13 @@ Good example: ```python ... from donna.core.result import Err, Ok, Result, unwrap_to_error +from donna.workspaces.artifacts import RENDER_CONTEXT_VIEW @unwrap_to_error def resolve(target_id: FullArtifactSectionId) -> Result[ArtifactSection, ErrorsList]: - from donna.world import artifacts as world_artifacts + from donna.context.context import context - artifact = world_artifacts.load_artifact(target_id.full_artifact_id).unwrap() + artifact = context().artifacts.load(target_id.full_artifact_id, RENDER_CONTEXT_VIEW).unwrap() section = artifact.get_section(target_id.local_id).unwrap() @@ -292,7 +293,7 @@ Bad example: from donna.core.result import Err, Ok, Result, unwrap_to_error def resolve(target_id: FullArtifactSectionId) -> Result[ArtifactSection, ErrorsList]: - artifact_result = world_artifacts.load_artifact(target_id.full_artifact_id) + artifact_result = context().artifacts.load(target_id.full_artifact_id, RENDER_CONTEXT_VIEW) if artifact_result.is_err(): return Err(artifact_result.unwrap_err()) diff --git a/specs/core/top_level_architecture.md b/specs/core/top_level_architecture.donna.md similarity index 88% rename from specs/core/top_level_architecture.md rename to specs/core/top_level_architecture.donna.md index d649e863..15a8f512 100644 --- a/specs/core/top_level_architecture.md +++ b/specs/core/top_level_architecture.donna.md @@ -21,13 +21,13 @@ The code is separated by layers/subsystems into subpackages: - `donna.domain` — code that is required by all Donna'specific logic: ID classes, common types, etc. - `donna.machine` — code that implements the core Donna's logic — how Donna works regardless of external environments, i.e. pure domain behavior. - `donna.context` — code that stores and provides execution-scoped runtime context for Donna's domain logic: artifact/state/primitive caches and scoped values like current actor/work unit/operation identifiers. -- `donna.world` — code that implements various worlds where Donna can find and manage artifacts: from artifacts discovery to their loading, parsing, updating. Also contains code related to configuration of Donna. +- `donna.workspaces` — code that integrates Donna with the project workspace and filesystem: runtime configuration, artifact discovery/loading, session storage, source parsing, and workspace initialization. - `donna.protocol` — code that implements protocol via which Donna's core domain logic interacts with external environments: CLI, API, etc. Includes basic classes for information representing (for the external environments) and its formatting. - `donna.cli` — code that implements the `donna` CLI tool, its commands, arguments parsing, etc. - `donna.primitives` — code that implements basic building blocks for Donna's behavior: concrete implementations of various classes from the `donna.machine`. - `donna.lib` — module that contains constructed primitives to be used in donna artifacts by referencing them by python import path. Like `donna.lib.workflow`, `donna.lib.goto`, etc. - `donna.fixtures.skills` — bundled skills that are distributed with Donna and synced into project workspaces under `.agents/skills`. -- `donna.fixtures.specs` — bundled Donna specifications and workflows that are distributed with Donna and synced into project workspaces under `.agents/donna`. +- `donna.fixtures.specs` — bundled Donna specifications and workflows that are distributed with Donna and synced into project workspaces under `.agents/donna`, where they are addressed as `@/.agents/donna/**` artifacts. ## Data structures diff --git a/specs/intro.md b/specs/intro.donna.md similarity index 71% rename from specs/intro.md rename to specs/intro.donna.md index d0b44543..16f677ed 100644 --- a/specs/intro.md +++ b/specs/intro.donna.md @@ -25,7 +25,7 @@ We may need coding agents on each step of the process, but there is no reason fo ## Dictionary - **Action request** — an instruction to the agent (who runs Donna) to perform the specified operations. Action requests are created by operations, like `donna.lib.request_action`. After finishing following the instructions of an action request, the agent MUST report back to Donna specifying the next operation to continue with. The list of next operations is specified in the action request itself. -- **Artifact** — any text or binary document managed by Donna in a world; text artifacts are typically Markdown templates with metadata and are the primary units of knowledge and instructions. +- **Artifact** — any text or binary document managed by Donna in the project filesystem; text artifacts are typically Markdown templates with metadata and are the primary units of knowledge and instructions. - **Artifact Section** — a part of a text artifact separated by markdown headers, has its own configuration block and semantics depending on section kind. - **Configuration block** — a fenced code block with the `donna` keyword (preferably TOML) that configures an artifact or its section. - **Directive** — a Jinja2 helper like `donna.lib.view(...)` or `donna.lib.goto(...)` that adds meta information or special behavior to an artifact. @@ -33,13 +33,12 @@ We may need coding agents on each step of the process, but there is no reason fo - **Head section** — the H1 section of a markdown artifact (before the first H2) that contains the primary description and mandatory config block. - **Internal error** — an error caused by a bug or unexpected state in Donna itself. These errors are not expected to be handled by agents or users. - **Protocol** — the output/interaction mode for Donna (e.g., `llm`) that governs CLI behavior and rendering. -- **Session** — the active unit of work tracked by Donna; its state and artifacts live in the `session` world and it always has exactly one active story. +- **Session** — the active unit of work tracked by Donna; its state and artifacts live under `/.donna/session`. - **Source** — the entity that implements logic of building an artifact from its raw data (text or binary). - **Specification** — a text artifact of kind `donna.lib.specification` that documents behavior, rules, or project guidance. - **Story** — a semantically consistent scope of work within a session; a conceptual unit not directly represented in the tool. - **Tail section** — each H2 section of an artifact. -- **World** — a storage namespace (filesystem or other backends) that contains artifacts. -- **Workspace** — the `.donna` directory in a project root that stores Donna's configuration, worlds, and artifacts for that project. +- **Workspace** — the `.donna` directory at `/.donna` that stores Donna's configuration, and runtime state. - **Workflow** — a `donna.lib.workflow` artifact that encodes a finite-state machine of operations guiding the agent's work. - **Workflow operation** — a single step in a workflow, defined by a tail section with an `id`, `kind`, and instructions. @@ -47,16 +46,16 @@ We may need coding agents on each step of the process, but there is no reason fo - `./donna/` — a directory containing source code of project — `donna` CLI tool. - `./specs/` — a directory containing project-specific donna artifacts that is used to manage the work of AI agents on this project. -- `./.donna/` — a directory containing donn-specific artifacts that is used to manage the work of AI agents on this project. +- `./.donna/` — a directory containing Donna-specific artifacts used to manage the work of AI agents on this project. ## Specifications of interest -Since this is the repository that contains the Donna project itself, you MUST pay additional attention to from which world you are viewing specifications. +Since this is the repository that contains the Donna project itself, you MUST pay additional attention to which project-scoped artifact ids you are viewing. -- `donna:` world contains specifications and artifacts related to the Donna tool behavior. You access them when you need to use `donna` tool itself. You change them when you make changes to the Donna behavior. -- `project:` world contains specifications and artifacts related to the Donna project as a software project. You access them when you need to understand how to introduce changes to the Donna codebase. You change them when you change the development processes or documentation of the Donna project as a software project. +- `@/.agents/donna/**` contains synced Donna specifications and workflows related to the Donna tool behavior. You access them when you need to use Donna itself. You change the source fixtures when you make changes to Donna behavior. +- `@/specs/**` contains project-specific specifications and workflows for developing the Donna codebase. You access them when you need to understand how to introduce changes to this repository. You change them when you change the development processes or documentation of the Donna project as a software project. Check the next specifications: -- `{{ donna.lib.view("project:core:top_level_architecture") }}` when you need to introduce any changes in Donna or to research its code. -- `{{ donna.lib.view("project:core:error_handling") }}` when you need to implement any new feature in Donna that may produce, process or propagate errors. +- `{{ donna.lib.view("@/specs/core/top_level_architecture.donna.md") }}` when you need to introduce any changes in Donna or to research its code. +- `{{ donna.lib.view("@/specs/core/error_handling.donna.md") }}` when you need to implement any new feature in Donna that may produce, process or propagate errors. diff --git a/specs/work/log_changes.md b/specs/work/log_changes.donna.md similarity index 98% rename from specs/work/log_changes.md rename to specs/work/log_changes.donna.md index 44515b3b..58c05ac4 100644 --- a/specs/work/log_changes.md +++ b/specs/work/log_changes.donna.md @@ -41,7 +41,7 @@ id = "analyze_scoped_changes" kind = "donna.lib.request_action" ``` -1. Focus on changes in the `session:` world artifacts provided by the parent workflow. +1. Focus on changes in the `@/.donna/session/**` artifacts provided by the parent workflow. 2. Summarize the main changes within that scoped set to use for the changelog entry. 3. Only after the scoped analysis, check the git state to confirm the summary reflects the current working tree. 4. `{{ donna.lib.goto("analyze_branch_name") }}` diff --git a/specs/work/polish.md b/specs/work/polish.donna.md similarity index 100% rename from specs/work/polish.md rename to specs/work/polish.donna.md