From 2f3f17373414c3e42c58669bb6fc7db059e87ddc Mon Sep 17 00:00:00 2001 From: "claude[bot]" <41898282+claude[bot]@users.noreply.github.com> Date: Fri, 29 May 2026 17:29:52 +0000 Subject: [PATCH 01/22] fix(thm-deriv-lincom): clarify dot-product derivative theorem per issue 840 - Add explicit condition that vx is constant w.r.t. vb (dx/dbeta = 0) - Wrap dot products in parentheses for unambiguous operator precedence - Fix proof LHS to use dot-product notation (was transpose-product) - Extract note about transposing the coefficient into a new corollary (cor-deriv-lincom-tp) for the transpose-product form x'b Co-Authored-By: Claude Sonnet 4.6 --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 31 +++++++++++++++++---- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 919c26fafa..fea2bc90e2 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -65,13 +65,14 @@ $$\deriv{ \vb} f(\vb) = \pt{\deriv{ \vb\'} f(\vb)}$$ #### Derivative of a dot product +If $\vx$ is constant with respect to $\vb$ +(i.e., $\deriv{\vb} \vx = \mathbf{0}$), +then: + $$ -\deriv{\vb} \vx \cdot \vb = \deriv{\vb} \vb \cdot \vx = \vx +\deriv{\vb} (\vx \cdot \vb) = \deriv{\vb} (\vb \cdot \vx) = \vx $$ -:::: notes -This looks a lot like non-vector calculus, except that you have to transpose the coefficient. -:::: ::: --- @@ -80,7 +81,7 @@ This looks a lot like non-vector calculus, except that you have to transpose the $$ \ba -\deriv{ \beta} (x\'\beta) +\deriv{\vb} (\vx \cdot \vb) &= \begin{bmatrix} \deriv{\beta_1}(x_1\beta_1+x_2\beta_2 +...+x_p \beta_p ) \\ @@ -103,6 +104,26 @@ $$ --- +:::{#cor-deriv-lincom-tp} + +#### Derivative of a dot product, transpose-product form + +If $\vx$ is constant with respect to $\vb$, then: + +$$ +\deriv{\vb} (\tp{\vx} \vb) = \vx +$$ + +:::: notes +This looks a lot like non-vector calculus, except that you have to transpose the coefficient: +in scalar calculus $\deriv{x}(cx) = c$, but here the coefficient $\tp{\vx}$ (a row vector) +becomes $\vx$ (a column vector) in the result. +:::: + +::: + +--- + :::{#thm-quadratic-form} #### Derivative of a quadratic form From e564216dd758d6f6c61e92c42e0109645f966b41 Mon Sep 17 00:00:00 2001 From: "claude[bot]" Date: Fri, 29 May 2026 18:06:03 +0000 Subject: [PATCH 02/22] fix matrix dimension annotations and update CLAUDE.md Addresses review comment: annotate zero matrix dimensions explicitly in the condition for thm-deriv-lincom; add underbrace dimension annotations to theorem and corollary display math. Also adds matrix dimension checking rules to CLAUDE.md: - always verify dimension compatibility in matrix expressions - annotate matrix dimensions with underbraces in display math - subscript zero matrix dimensions: mathbf{0}_{m x n} - Jacobian deriv{vb} vx is a p x p matrix when both are p-vectors --- CLAUDE.md | 4 ++++ _subfiles/math-prereqs/_sec_vector_calc.qmd | 9 ++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 275ce9b1f4..79f4ad670e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,6 +50,10 @@ Before committing any `.qmd`, `.R`, or config file change: - Key macros: `\E{Y|X=x}`, `\ba`/`\ea`, `\tp{v}`, `\b`, `\g`, `\a`, `\devn(...)`, `\erf{...}` - Include every intermediate step in derivations — do not skip steps - Color coding: `\red{...}` for focal/extra terms, `\blue{...}` for shared terms +- **Matrix dimensions**: always verify dimension compatibility for every matrix expression -- dimensions of each operand must be consistent with the operation +- **Annotate matrix dimensions with underbraces** in display math: use `\underbrace{M}_{m \times n}` for each matrix or vector +- **Zero matrices**: never write bare `\mathbf{0}` in a matrix equation -- subscript dimensions: `\mathbf{0}_{m \times n}` +- **Jacobian**: `\deriv{\vb} \vx` where both are p-vectors produces a p x p Jacobian matrix (not a vector) ### Citations - Always use BibTeX keys with `@citekey` Pandoc syntax — never plaintext author-date diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index fea2bc90e2..35dcc0fb28 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -66,11 +66,13 @@ $$\deriv{ \vb} f(\vb) = \pt{\deriv{ \vb\'} f(\vb)}$$ #### Derivative of a dot product If $\vx$ is constant with respect to $\vb$ -(i.e., $\deriv{\vb} \vx = \mathbf{0}$), +(i.e., $\underbrace{\deriv{\vb} \vx}_{p \times p} = \underbrace{\mathbf{0}}_{p \times p}$), then: $$ -\deriv{\vb} (\vx \cdot \vb) = \deriv{\vb} (\vb \cdot \vx) = \vx +\underbrace{\deriv{\vb} (\vx \cdot \vb)}_{p \times 1} = +\underbrace{\deriv{\vb} (\vb \cdot \vx)}_{p \times 1} = +\underbrace{\vx}_{p \times 1} $$ ::: @@ -111,7 +113,8 @@ $$ If $\vx$ is constant with respect to $\vb$, then: $$ -\deriv{\vb} (\tp{\vx} \vb) = \vx +\underbrace{\deriv{\vb} (\underbrace{\tp{\vx}}_{1 \times p} \underbrace{\vb}_{p \times 1})}_{p \times 1} = +\underbrace{\vx}_{p \times 1} $$ :::: notes From f7d6df4cbd3f6c055b874e8c958a0ac461373c28 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 29 May 2026 23:50:53 +0000 Subject: [PATCH 03/22] Address review comments: add Jacobian note, product rule theorem, and two proofs --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 75 +++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 35dcc0fb28..f2775e4a8e 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -73,6 +73,12 @@ $$ \underbrace{\deriv{\vb} (\vx \cdot \vb)}_{p \times 1} = \underbrace{\deriv{\vb} (\vb \cdot \vx)}_{p \times 1} = \underbrace{\vx}_{p \times 1} + +:::: notes +Here $\deriv{\vb} \vx$ is the $p \times p$ Jacobian matrix in denominator layout: +entry $(i,j)$ equals $\partial x_j / \partial \beta_i$. +No transpose on $\vb$ or $\vx$ is needed. +:::: $$ ::: @@ -103,6 +109,41 @@ x_{p} $$ ::: +--- + +:::{#thm-deriv-linear-map} + +#### Derivative of a linear map + +If $A$ is a constant $m \times p$ matrix +(constant with respect to $\vb$), +then: + +$$ +\underbrace{\deriv{\vb} (A\vb)}_{p \times m} = +\underbrace{\tp{A}}_{p \times m} +$$ + +::: + +::: proof + +For entry $(i,j)$, where row $i$ indexes the denominator $\vb$ +and column $j$ indexes the numerator $A\vb$: + +$$ +\ba +\left[\deriv{\vb} (A\vb)\right]_{ij} +&= \deriv{\beta_i} (A\vb)_j \\ +&= \deriv{\beta_i} \sum_{k=1}^{p} a_{jk} \beta_k \\ +&= a_{ji} \\ +&= \left[\tp{A}\right]_{ij} +\ea +$$ + +::: + + --- @@ -124,6 +165,40 @@ becomes $\vx$ (a column vector) in the result. :::: ::: +::: proof + +**Using @thm-deriv-lincom:** + +Since $\tp{\vx}\vb = \vx \cdot \vb$ (@def-dot-product), +and $\vx$ is constant with respect to $\vb$: + +$$ +\deriv{\vb}(\tp{\vx}\vb) += \deriv{\vb}(\vx \cdot \vb) += \vx +$$ + +by @thm-deriv-lincom. + +::: + +::: proof + +**Using @thm-deriv-linear-map:** + +Since $\vx$ is constant with respect to $\vb$, +$A = \tp{\vx}$ is a constant $1 \times p$ matrix. +Applying @thm-deriv-linear-map: + +$$ +\deriv{\vb}(\tp{\vx}\vb) += \tp{(\tp{\vx})} += \vx +$$ + +::: + + --- From 763b552338ad0433e4ece3b9b8cebc72367148a2 Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 10:29:58 -0700 Subject: [PATCH 04/22] Refine explanations for constant vectors and derivatives Clarified the condition for constant vectors and improved the explanation of the derivative of a dot product. --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 27 ++++++++++++++------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index f2775e4a8e..e0ecd55bd2 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -61,24 +61,33 @@ $$\deriv{ \vb} f(\vb) = \pt{\deriv{ \vb\'} f(\vb)}$$ --- +:::{#def-constant-wrt-vector} +#### Constant + +$\vx$ is constant with respect to $\vb$ if + +$$ +\underbrace{ +\underbrace{\deriv{\vb}}_{p \times 1} +\underbrace{\tp{\vx}}_{1 \times p} +}_{p \times p} += \underbrace{\mathbf{0}}_{p \times p} +$$ + +::: + +--- + :::{#thm-deriv-lincom} #### Derivative of a dot product -If $\vx$ is constant with respect to $\vb$ -(i.e., $\underbrace{\deriv{\vb} \vx}_{p \times p} = \underbrace{\mathbf{0}}_{p \times p}$), -then: +If $\vx$ is constant with respect to $\vb$, then: $$ \underbrace{\deriv{\vb} (\vx \cdot \vb)}_{p \times 1} = \underbrace{\deriv{\vb} (\vb \cdot \vx)}_{p \times 1} = \underbrace{\vx}_{p \times 1} - -:::: notes -Here $\deriv{\vb} \vx$ is the $p \times p$ Jacobian matrix in denominator layout: -entry $(i,j)$ equals $\partial x_j / \partial \beta_i$. -No transpose on $\vb$ or $\vx$ is needed. -:::: $$ ::: From cb679f5c0e36348f90bf7b6c7785f43e073082ca Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 13:51:25 -0700 Subject: [PATCH 05/22] Update _sec_vector_calc.qmd --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 42 +++++++++++++++++++-- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index e0ecd55bd2..eefa464645 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -118,14 +118,50 @@ x_{p} $$ ::: ---- + +{{< slidebreak >}} + + +:::{#thm-deriv-dot-product} + +#### Product rule for dot-products + +$$ +\ba +\deriv{\ubf{\vx}{p \times 1}} \dpf{\ubf{a}{p \times 1}}{\ubf{b}{\p \times 1}} +&= +\paren{ + \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{a}}{1 \times p} + } +\ubf{b}{p \times 1} ++ +\paren{ + \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{b}}{1 \times p} +} +\ubf{a}{p \times 1} +\\ +&= +\paren{ + \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{a}}{1 \times p} + } +\ubf{b}{p \times 1} ++ +\ubf{\tp{a}}{1 \times p} +\paren{ + \ubf{b}{p \times 1} \deriv{\ubf{\tp{\vx}}{1 \times p}} +} +$$ + +::: + +{{< slidebreak >}} :::{#thm-deriv-linear-map} #### Derivative of a linear map -If $A$ is a constant $m \times p$ matrix -(constant with respect to $\vb$), +If $A$ is a $m \times p$ matrix +that is with respect to $\vb$, then: $$ From 8f967501063747f2be41758b4b6c73ea14b95e75 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Mon, 1 Jun 2026 21:22:23 +0000 Subject: [PATCH 06/22] fix typo, missing word, blank line, and add proof for thm-deriv-dot-product --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 23 ++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index eefa464645..4e67b3b133 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -128,7 +128,7 @@ $$ $$ \ba -\deriv{\ubf{\vx}{p \times 1}} \dpf{\ubf{a}{p \times 1}}{\ubf{b}{\p \times 1}} +\deriv{\ubf{\vx}{p \times 1}} \dpf{\ubf{a}{p \times 1}}{\ubf{b}{p \times 1}} &= \paren{ \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{a}}{1 \times p} @@ -154,14 +154,30 @@ $$ ::: +::: proof + +Entry-wise, for $i = 1, \ldots, p$: + +$$ +\ba +\left[\deriv{\vx} (\mathbf{a} \cdot \mathbf{b})\right]_i +&= \deriv{x_i} \sum_{k=1}^p a_k b_k \\ +&= \sum_{k=1}^p \paren{b_k \deriv{x_i} a_k + a_k \deriv{x_i} b_k} \\ +&= \left[\paren{\deriv{\vx} \tp{\mathbf{a}}}\mathbf{b}\right]_i + + \left[\paren{\deriv{\vx} \tp{\mathbf{b}}}\mathbf{a}\right]_i +\ea +$$ + +::: + {{< slidebreak >}} :::{#thm-deriv-linear-map} #### Derivative of a linear map -If $A$ is a $m \times p$ matrix -that is with respect to $\vb$, +If $A$ is an $m \times p$ matrix +that is constant with respect to $\vb$, then: $$ @@ -210,6 +226,7 @@ becomes $\vx$ (a column vector) in the result. :::: ::: + ::: proof **Using @thm-deriv-lincom:** From 8bb96a85e45c3d1577bf9fafb1d0c2b651f58b49 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Mon, 1 Jun 2026 21:23:08 +0000 Subject: [PATCH 07/22] fix p x p -> p x p in CLAUDE.md --- CLAUDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 79f4ad670e..7f3433e6fe 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,7 +53,7 @@ Before committing any `.qmd`, `.R`, or config file change: - **Matrix dimensions**: always verify dimension compatibility for every matrix expression -- dimensions of each operand must be consistent with the operation - **Annotate matrix dimensions with underbraces** in display math: use `\underbrace{M}_{m \times n}` for each matrix or vector - **Zero matrices**: never write bare `\mathbf{0}` in a matrix equation -- subscript dimensions: `\mathbf{0}_{m \times n}` -- **Jacobian**: `\deriv{\vb} \vx` where both are p-vectors produces a p x p Jacobian matrix (not a vector) +- **Jacobian**: `\deriv{\vb} \vx` where both are p-vectors produces a p × p Jacobian matrix (not a vector) ### Citations - Always use BibTeX keys with `@citekey` Pandoc syntax — never plaintext author-date From 2b0dfe384fcab77d40f84e2a1932b187d1e469f7 Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 16:43:19 -0700 Subject: [PATCH 08/22] Fix non-rendering thm-deriv-dot-product equation The "Product rule for dot-products" display equation opened an aligned environment with \ba but was missing the closing \ea, so MathJax failed to render it. Add the \ea. Co-Authored-By: Claude Opus 4.8 (1M context) Co-authored-by: d-morrison <2474437+d-morrison@users.noreply.github.com> --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 4e67b3b133..2133f5d9b3 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -148,8 +148,9 @@ $$ + \ubf{\tp{a}}{1 \times p} \paren{ - \ubf{b}{p \times 1} \deriv{\ubf{\tp{\vx}}{1 \times p}} + \ubf{b}{p \times 1} \deriv{\ubf{\tp{\vx}}{1 \times p}} } +\ea $$ ::: From 6e864caa67cadda1a5e48a5ea44b74fc25955d35 Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 16:58:24 -0700 Subject: [PATCH 09/22] Add vector-derivative product-rule theorems (matrix-vector, matrix product) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add two theorems to the vector-calculus section, in the section's denominator- layout ∂/∂ convention with constant matrices: - thm-deriv-matrix-vector: ∂/∂β (A v) = (∂v/∂β) Aᵀ, for constant A and a vector v = v(β). Generalizes thm-deriv-linear-map (the v = β special case). - thm-deriv-matrix-product-vector: ∂/∂β (A B v) = (∂v/∂β) Bᵀ Aᵀ, for constant A, B. Both include entrywise/derived proofs and underbrace dimension annotations, matching the existing theorems. Co-Authored-By: Claude Opus 4.8 (1M context) Co-authored-by: d-morrison <2474437+d-morrison@users.noreply.github.com> --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 80 +++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 2133f5d9b3..45b28bd90f 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -205,6 +205,86 @@ $$ ::: +--- + +:::{#thm-deriv-matrix-vector} + +#### Vector-derivative of a matrix-vector product + +If $A$ is an $m \times q$ matrix that is constant with respect to $\vb$, +and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector +that depends on the $p \times 1$ vector $\vb$, +then: + +$$ +\underbrace{\deriv{\vb} (A\vecf{v})}_{p \times m} = +\underbrace{\paren{\deriv{\vb} \vecf{v}}}_{p \times q} +\underbrace{\tp{A}}_{q \times m} +$$ + +::: + +::: notes +This generalizes @thm-deriv-linear-map, +which is the special case $\vecf{v} = \vb$ +(so that $\deriv{\vb} \vb = \matr{I}$ +and $\deriv{\vb} (A\vb) = \tp{A}$). +::: + +::: proof + +For entry $(i,j)$, where row $i$ indexes the denominator $\vb$ +and column $j$ indexes the numerator $A\vecf{v}$: + +$$ +\ba +\left[\deriv{\vb} (A\vecf{v})\right]_{ij} +&= \deriv{\beta_i} (A\vecf{v})_j \\ +&= \deriv{\beta_i} \sum_{k=1}^{q} a_{jk} v_k \\ +&= \sum_{k=1}^{q} a_{jk} \deriv{\beta_i} v_k \\ +&= \sum_{k=1}^{q} \left[\deriv{\vb} \vecf{v}\right]_{ik} \left[\tp{A}\right]_{kj} \\ +&= \left[\paren{\deriv{\vb} \vecf{v}} \tp{A}\right]_{ij} +\ea +$$ + +::: + +--- + +:::{#thm-deriv-matrix-product-vector} + +#### Vector-derivative of a product of matrices + +If $A$ ($\ell \times m$) and $B$ ($m \times q$) +are constant with respect to $\vb$, +and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector, +then: + +$$ +\underbrace{\deriv{\vb} (A B \vecf{v})}_{p \times \ell} = +\underbrace{\paren{\deriv{\vb} \vecf{v}}}_{p \times q} +\underbrace{\tp{B}}_{q \times m} +\underbrace{\tp{A}}_{m \times \ell} +$$ + +::: + +::: proof + +Apply @thm-deriv-matrix-vector +with the constant $\ell \times q$ matrix $AB$, +then use $\tp{(AB)} = \tp{B}\,\tp{A}$: + +$$ +\ba +\deriv{\vb} (A B \vecf{v}) +&= \paren{\deriv{\vb} \vecf{v}} \tp{(AB)} \\ +&= \paren{\deriv{\vb} \vecf{v}} \tp{B}\,\tp{A} +\ea +$$ + +::: + --- From 5653ebcf375663aea7addfc01aaba0e85b99422d Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 17:12:27 -0700 Subject: [PATCH 10/22] Park the matrix-product vector-derivative theorem in a non-included subfile Remove thm-deriv-matrix-product-vector ("Vector-derivative of a product of matrices") from the rendered vector-calculus section and move it verbatim to _subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd, which is not included anywhere (parked for later). The matrix-vector product theorem (thm-deriv-matrix-vector) stays in the section. Co-Authored-By: Claude Opus 4.8 (1M context) Co-authored-by: d-morrison <2474437+d-morrison@users.noreply.github.com> --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 38 ------------------- .../_thm-deriv-matrix-product-vector.qmd | 33 ++++++++++++++++ 2 files changed, 33 insertions(+), 38 deletions(-) create mode 100644 _subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 45b28bd90f..5af4d6b930 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -249,44 +249,6 @@ $$ ::: ---- - -:::{#thm-deriv-matrix-product-vector} - -#### Vector-derivative of a product of matrices - -If $A$ ($\ell \times m$) and $B$ ($m \times q$) -are constant with respect to $\vb$, -and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector, -then: - -$$ -\underbrace{\deriv{\vb} (A B \vecf{v})}_{p \times \ell} = -\underbrace{\paren{\deriv{\vb} \vecf{v}}}_{p \times q} -\underbrace{\tp{B}}_{q \times m} -\underbrace{\tp{A}}_{m \times \ell} -$$ - -::: - -::: proof - -Apply @thm-deriv-matrix-vector -with the constant $\ell \times q$ matrix $AB$, -then use $\tp{(AB)} = \tp{B}\,\tp{A}$: - -$$ -\ba -\deriv{\vb} (A B \vecf{v}) -&= \paren{\deriv{\vb} \vecf{v}} \tp{(AB)} \\ -&= \paren{\deriv{\vb} \vecf{v}} \tp{B}\,\tp{A} -\ea -$$ - -::: - - - --- :::{#cor-deriv-lincom-tp} diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd new file mode 100644 index 0000000000..017c1a0020 --- /dev/null +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd @@ -0,0 +1,33 @@ +:::{#thm-deriv-matrix-product-vector} + +#### Vector-derivative of a product of matrices + +If $A$ ($\ell \times m$) and $B$ ($m \times q$) +are constant with respect to $\vb$, +and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector, +then: + +$$ +\underbrace{\deriv{\vb} (A B \vecf{v})}_{p \times \ell} = +\underbrace{\paren{\deriv{\vb} \vecf{v}}}_{p \times q} +\underbrace{\tp{B}}_{q \times m} +\underbrace{\tp{A}}_{m \times \ell} +$$ + +::: + +::: proof + +Apply @thm-deriv-matrix-vector +with the constant $\ell \times q$ matrix $AB$, +then use $\tp{(AB)} = \tp{B}\,\tp{A}$: + +$$ +\ba +\deriv{\vb} (A B \vecf{v}) +&= \paren{\deriv{\vb} \vecf{v}} \tp{(AB)} \\ +&= \paren{\deriv{\vb} \vecf{v}} \tp{B}\,\tp{A} +\ea +$$ + +::: From 2c34315c3147e2f3df91f23dbeea1f843878c323 Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Mon, 1 Jun 2026 17:28:10 -0700 Subject: [PATCH 11/22] Add parked matrix-derivative product theorem (non-included subfile) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add _subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd (not included anywhere) holding the matrix-derivative case: a short matrix-derivative definition plus thm-deriv-matrix-product-matrix, ∂/∂X tr(A X B) = Aᵀ Bᵀ for constant A, B, with an entrywise proof. The trace is needed to keep the result a matrix (the matrix derivative of the bare product A X B is a fourth-order tensor). Parked for later, like the vector-derivative product theorem. Co-Authored-By: Claude Opus 4.8 (1M context) Co-authored-by: d-morrison <2474437+d-morrison@users.noreply.github.com> --- .../_thm-deriv-matrix-product-matrix.qmd | 58 +++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 _subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd new file mode 100644 index 0000000000..3240ff4392 --- /dev/null +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd @@ -0,0 +1,58 @@ +:::{#def-matrix-derivative} + +#### Matrix derivative + +For a scalar-valued function $f(\matr{X})$ +of an $m \times n$ matrix $\matr{X}$, +the **matrix derivative** is the $m \times n$ matrix +whose $(i,j)$ entry is the partial derivative of $f$ +with respect to the $(i,j)$ entry of $\matr{X}$: + +$$ +\left[\deriv{\matr{X}} f\right]_{ij} = \deriv{X_{ij}} f +$$ + +::: + +--- + +:::{#thm-deriv-matrix-product-matrix} + +#### Matrix-derivative of a product of matrices + +If $A$ ($r \times m$) and $B$ ($n \times r$) +are constant with respect to the $m \times n$ matrix $\matr{X}$, +then: + +$$ +\underbrace{\deriv{\matr{X}} \operatorname{tr}(A \matr{X} B)}_{m \times n} = +\underbrace{\tp{A}}_{m \times r} +\underbrace{\tp{B}}_{r \times n} +$$ + +::: + +::: notes +The trace makes $\operatorname{tr}(A \matr{X} B)$ a scalar, +so its matrix derivative is again an $m \times n$ matrix. +The derivative of the matrix product $A \matr{X} B$ itself +(without the trace) is a fourth-order tensor, +which is why this result is stated for the scalar $\operatorname{tr}(A \matr{X} B)$. +::: + +::: proof + +Write $\operatorname{tr}(A \matr{X} B) = \sum_{a} \sum_{b} \sum_{c} A_{ab} X_{bc} B_{ca}$. +For entry $(i,j)$: + +$$ +\ba +\left[\deriv{\matr{X}} \operatorname{tr}(A \matr{X} B)\right]_{ij} +&= \deriv{X_{ij}} \sum_{a} \sum_{b} \sum_{c} A_{ab} X_{bc} B_{ca} \\ +&= \sum_{a} A_{ai} B_{ja} \\ +&= \sum_{a} \left[\tp{A}\right]_{ia} \left[\tp{B}\right]_{aj} \\ +&= \left[\tp{A}\,\tp{B}\right]_{ij} +\ea +$$ + +::: From 7eaffb2e8750d6c2609031b2d02f8d60bd8d2974 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 17:44:58 +0000 Subject: [PATCH 12/22] address review comments: fix def-constant notation, use thm-deriv-matrix-vector in proof 2, add layout ref, fix spacing --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 5af4d6b930..cd15f86878 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -67,10 +67,7 @@ $$\deriv{ \vb} f(\vb) = \pt{\deriv{ \vb\'} f(\vb)}$$ $\vx$ is constant with respect to $\vb$ if $$ -\underbrace{ -\underbrace{\deriv{\vb}}_{p \times 1} -\underbrace{\tp{\vx}}_{1 \times p} -}_{p \times p} +\underbrace{\deriv{\vb} \tp{\vx}}_{p \times p} = \underbrace{\mathbf{0}}_{p \times p} $$ @@ -191,6 +188,7 @@ $$ ::: proof For entry $(i,j)$, where row $i$ indexes the denominator $\vb$ +(see @def-vector-derivative) and column $j$ indexes the numerator $A\vb$: $$ @@ -289,22 +287,24 @@ by @thm-deriv-lincom. ::: proof -**Using @thm-deriv-linear-map:** +**Using @thm-deriv-matrix-vector:** Since $\vx$ is constant with respect to $\vb$, $A = \tp{\vx}$ is a constant $1 \times p$ matrix. -Applying @thm-deriv-linear-map: +Applying @thm-deriv-matrix-vector with $\vecf{v} = \vb$ +(so $\deriv{\vb}\vb = \matr{I}$): $$ +\ba \deriv{\vb}(\tp{\vx}\vb) -= \tp{(\tp{\vx})} -= \vx +&= \paren{\deriv{\vb}\vb} \tp{(\tp{\vx})} \\ +&= \matr{I} \cdot \vx \\ +&= \vx +\ea $$ ::: - - --- :::{#thm-quadratic-form} From 5488bd180199b3e05257214e1d3a7b08f9b2c601 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 18:02:11 +0000 Subject: [PATCH 13/22] add numerical examples for new definitions and theorems --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 62 +++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index cd15f86878..b823766bc5 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -73,6 +73,16 @@ $$ ::: +:::{#exm-constant-wrt-vector} +#### Example: constant vector + +Let $\vb = \tp{(\beta_1, \beta_2)}$ and $\vx = \tp{(3, 5)}$. +Since $x_1 = 3$ and $x_2 = 5$ do not depend on $\vb$, +$\vx$ is constant with respect to $\vb$, +so $\underbrace{\deriv{\vb} \tp{\vx}}_{2 \times 2} = \underbrace{\mathbf{0}}_{2 \times 2}$. + +::: + --- :::{#thm-deriv-lincom} @@ -203,6 +213,21 @@ $$ ::: +:::{#exm-deriv-linear-map} +#### Example: derivative of linear map + +Let $A = \begin{pmatrix} 2 & 3 \end{pmatrix}$ ($1 \times 2$) and $\vb = \tp{(\beta_1, \beta_2)}$. +Then $A\vb = 2\beta_1 + 3\beta_2$, and by @thm-deriv-linear-map: + +$$ +\underbrace{\deriv{\vb}(A\vb)}_{2 \times 1} += \underbrace{\tp{A}}_{2 \times 1} += \begin{pmatrix} 2 \\ +3 \end{pmatrix} +$$ + +::: + --- :::{#thm-deriv-matrix-vector} @@ -247,6 +272,28 @@ $$ ::: +:::{#exm-deriv-matrix-vector} +#### Example: vector-derivative of matrix-vector product + +Let $A = \begin{pmatrix} 2 & 3 \end{pmatrix}$ ($1 \times 2$, constant) +and $\vecf{v}(\vb) = \tp{(\beta_1^2, \beta_2^2)}$. +Then $A\vecf{v} = 2\beta_1^2 + 3\beta_2^2$. +By @thm-deriv-matrix-vector: + +$$ +\ba +\underbrace{\deriv{\vb}(A\vecf{v})}_{2 \times 1} +&= \begin{pmatrix} 2\beta_1 & 0 \\ +0 & 2\beta_2 \end{pmatrix} +\begin{pmatrix} 2 \\ +3 \end{pmatrix} \\\\ +&= \begin{pmatrix} 4\beta_1 \\ +6\beta_2 \end{pmatrix} +\ea +$$ + +::: + --- :::{#cor-deriv-lincom-tp} @@ -305,6 +352,21 @@ $$ ::: +:::{#exm-deriv-lincom-tp} +#### Example: derivative of transpose product + +Let $\vx = \tp{(3, 5)}$ and $\vb = \tp{(\beta_1, \beta_2)}$. +Then $\tp{\vx}\vb = 3\beta_1 + 5\beta_2$, and by @cor-deriv-lincom-tp: + +$$ +\underbrace{\deriv{\vb}\left(\underbrace{\tp{\vx}}_{1 \times 2}\underbrace{\vb}_{2 \times 1}\right)}_{2 \times 1} += \underbrace{\vx}_{2 \times 1} += \begin{pmatrix} 3 \\ +5 \end{pmatrix} +$$ + +::: + --- :::{#thm-quadratic-form} From b65db0b29583f44ad2cca29160f37190ef889c9b Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 18:09:45 +0000 Subject: [PATCH 14/22] add examples for def-matrix-derivative and thm-deriv-matrix-product-matrix --- .../_thm-deriv-matrix-product-matrix.qmd | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd index 3240ff4392..d8febd499f 100644 --- a/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd @@ -14,6 +14,17 @@ $$ ::: +:::{#exm-matrix-derivative} + +Let $\matr{X}$ be a $2 \times 2$ matrix and $f(\matr{X}) = \operatorname{tr}(\matr{X}) = X_{11} + X_{22}$. +Then $\deriv{X_{ij}} f = 1$ if $i = j$ and $0$ otherwise, so: + +$$ +\deriv{\matr{X}} f = \matr{I}_2 +$$ + +::: + --- :::{#thm-deriv-matrix-product-matrix} @@ -56,3 +67,20 @@ $$ $$ ::: + +:::{#exm-deriv-matrix-product-matrix} + +Let $A = \matr{I}_2$ ($2 \times 2$) and $B = \begin{pmatrix}2 & 0 \\ +0 & 3\end{pmatrix}$ ($2 \times 2$). +Then $\operatorname{tr}(A \matr{X} B) = 2X_{11} + 3X_{22}$, and: + +$$ +\underbrace{\deriv{\matr{X}} \operatorname{tr}(A \matr{X} B)}_{2 \times 2} += \underbrace{\tp{A}}_{2 \times 2} \underbrace{\tp{B}}_{2 \times 2} += \matr{I}_2 \begin{pmatrix}2 & 0 \\ +0 & 3\end{pmatrix} += \begin{pmatrix}2 & 0 \\ +0 & 3\end{pmatrix} +$$ + +::: From a58403421a9ae8f8eb1a63903f74ad1d0564871f Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 18:09:57 +0000 Subject: [PATCH 15/22] add example for thm-deriv-matrix-product-vector --- .../_thm-deriv-matrix-product-vector.qmd | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd index 017c1a0020..efeba1913c 100644 --- a/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd @@ -31,3 +31,27 @@ $$ $$ ::: + +:::{#exm-deriv-matrix-product-vector} + +Let $A = \begin{pmatrix}1 & 0\end{pmatrix}$ ($1 \times 2$), +$B = \begin{pmatrix}2 & 0 \\ +0 & 3\end{pmatrix}$ ($2 \times 2$), +and $\vecf{v}(\vb) = \vb$ where $\vb = \tp{(\beta_1, \beta_2)}$. +Then $AB\vecf{v} = 2\beta_1$, and: + +$$ +\underbrace{\deriv{\vb}(AB\vecf{v})}_{2 \times 1} += \underbrace{\paren{\deriv{\vb}\vb}}_{2 \times 2} +\underbrace{\tp{B}}_{2 \times 2} +\underbrace{\tp{A}}_{2 \times 1} += \matr{I}_2 +\begin{pmatrix}2 & 0 \\ +0 & 3\end{pmatrix} +\begin{pmatrix}1 \\ +0\end{pmatrix} += \begin{pmatrix}2 \\ +0\end{pmatrix} +$$ + +::: From 00ffc81fdef170631ffe3e80afd90e37fb001c6f Mon Sep 17 00:00:00 2001 From: Douglas Ezra Morrison Date: Fri, 5 Jun 2026 16:07:20 -0700 Subject: [PATCH 16/22] exm-constant-wrt-vector: show worked steps; drop redundant "Example" label MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Expand d/db x' into its 2x2 matrix of scalar partial derivatives (per @def-vector-derivative) and evaluate each entry to 0, instead of asserting the zero matrix in one line — so the example demonstrates the mechanism behind @def-constant-wrt-vector. Also rename the heading "Example: constant vector" -> "A constant vector": the "Example" label is supplied by the exm environment, so typing it into the heading duplicates it. Co-Authored-By: Claude Opus 4.8 (1M context) Co-authored-by: d-morrison <2474437+d-morrison@users.noreply.github.com> --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 32 +++++++++++++++++---- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index b823766bc5..95c9d40a35 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -74,12 +74,34 @@ $$ ::: :::{#exm-constant-wrt-vector} -#### Example: constant vector +#### A constant vector -Let $\vb = \tp{(\beta_1, \beta_2)}$ and $\vx = \tp{(3, 5)}$. -Since $x_1 = 3$ and $x_2 = 5$ do not depend on $\vb$, -$\vx$ is constant with respect to $\vb$, -so $\underbrace{\deriv{\vb} \tp{\vx}}_{2 \times 2} = \underbrace{\mathbf{0}}_{2 \times 2}$. +Let $\vb = \tp{(\beta_1, \beta_2)}$ and $\vx = \tp{(3, 5)}$, +so $x_1 = 3$ and $x_2 = 5$ do not depend on $\vb$. +Expanding $\deriv{\vb} \tp{\vx}$ into its matrix of scalar partial derivatives +(@def-vector-derivative, applied to each component of the row $\tp{\vx}$) +and evaluating each entry: + +$$ +\underbrace{\deriv{\vb} \tp{\vx}}_{2 \times 2} += \deriv{\vb} \sbmat{x_1 & x_2} += \sbmat{ +\deriv{\beta_1} x_1 & \deriv{\beta_1} x_2 \\ +\deriv{\beta_2} x_1 & \deriv{\beta_2} x_2 +} += \sbmat{ +\deriv{\beta_1} 3 & \deriv{\beta_1} 5 \\ +\deriv{\beta_2} 3 & \deriv{\beta_2} 5 +} += \sbmat{ +0 & 0 \\ +0 & 0 +} += \underbrace{\mathbf{0}}_{2 \times 2} +$$ + +Every entry is the derivative of a constant, so $\deriv{\vb} \tp{\vx} = \mathbf{0}$ +and $\vx$ is constant with respect to $\vb$ (@def-constant-wrt-vector). ::: From 3401891a36ab36cbc7ffd25a0b1464c3dde20bc2 Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Sat, 6 Jun 2026 00:57:13 +0000 Subject: [PATCH 17/22] fix: address review issues in vector calc section - Delete dimensionally inconsistent second equality in thm-deriv-dot-product - Add preamble introducing a and b in thm-deriv-dot-product - Add example exm-deriv-dot-product after proof - Replace three --- separators with {{< slidebreak >}} - Move thm-deriv-matrix-vector notes inside theorem div - Remove redundant 'Example:' prefix from three example headings --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 92 +++++++++++++++------ 1 file changed, 69 insertions(+), 23 deletions(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 95c9d40a35..87ff3ec68e 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -105,7 +105,7 @@ and $\vx$ is constant with respect to $\vb$ (@def-constant-wrt-vector). ::: ---- +{{< slidebreak >}} :::{#thm-deriv-lincom} @@ -155,30 +155,22 @@ $$ #### Product rule for dot-products +If $\mathbf{a} = \mathbf{a}(\vx)$ and $\mathbf{b} = \mathbf{b}(\vx)$ +are differentiable $p \times 1$ vector functions of $\vx$, then: + $$ \ba -\deriv{\ubf{\vx}{p \times 1}} \dpf{\ubf{a}{p \times 1}}{\ubf{b}{p \times 1}} -&= -\paren{ - \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{a}}{1 \times p} - } -\ubf{b}{p \times 1} -+ -\paren{ - \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{b}}{1 \times p} -} -\ubf{a}{p \times 1} -\\ +\deriv{\ubf{\vx}{p \times 1}} \dpf{\ubf{a}{p \times 1}}{\ubf{b}{p \times 1}} &= \paren{ \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{a}}{1 \times p} } \ubf{b}{p \times 1} + -\ubf{\tp{a}}{1 \times p} \paren{ - \ubf{b}{p \times 1} \deriv{\ubf{\tp{\vx}}{1 \times p}} + \deriv{\ubf{\vx}{p \times 1}} \ubf{\tp{b}}{1 \times p} } +\ubf{a}{p \times 1} \ea $$ @@ -200,6 +192,60 @@ $$ ::: +:::{#exm-deriv-dot-product} +#### Example of the dot-product rule + +Let $\vx = \tp{(\beta_1, \beta_2)}$, +$\mathbf{a}(\vx) = \tp{(\beta_1, \beta_1\beta_2)}$, +and $\mathbf{b}(\vx) = \tp{(\beta_2, \beta_1)}$. +Then: + +$$ +\mathbf{a} \cdot \mathbf{b} += \beta_1 \cdot \beta_2 + \beta_1\beta_2 \cdot \beta_1 += \beta_1\beta_2 + \beta_1^2\beta_2 +$$ + +By direct calculation: + +$$ +\underbrace{\deriv{\vx}(\mathbf{a} \cdot \mathbf{b})}_{2 \times 1} += \deriv{\vx}(\beta_1\beta_2 + \beta_1^2\beta_2) += \begin{pmatrix} \beta_2 + 2\beta_1\beta_2 \\ \beta_1 + \beta_1^2 \end{pmatrix} +$$ + +By the product rule (@thm-deriv-dot-product), +using +$\underbrace{\deriv{\vx}\tp{\mathbf{a}}}_{2 \times 2} += \begin{pmatrix}1 & \beta_2 \\ 0 & \beta_1\end{pmatrix}$ +and +$\underbrace{\deriv{\vx}\tp{\mathbf{b}}}_{2 \times 2} += \begin{pmatrix}0 & 1 \\ 1 & 0\end{pmatrix}$: + +$$ +\ba +\underbrace{\deriv{\vx}(\mathbf{a} \cdot \mathbf{b})}_{2 \times 1} +&= +\underbrace{\begin{pmatrix}1 & \beta_2 \\ 0 & \beta_1\end{pmatrix}}_{2 \times 2} +\underbrace{\begin{pmatrix}\beta_2 \\ \beta_1\end{pmatrix}}_{2 \times 1} ++ +\underbrace{\begin{pmatrix}0 & 1 \\ 1 & 0\end{pmatrix}}_{2 \times 2} +\underbrace{\begin{pmatrix}\beta_1 \\ \beta_1\beta_2\end{pmatrix}}_{2 \times 1} +\\ +&= +\begin{pmatrix}\beta_2 + \beta_1\beta_2 \\ \beta_1^2\end{pmatrix} ++ +\begin{pmatrix}\beta_1\beta_2 \\ \beta_1\end{pmatrix} +\\ +&= +\begin{pmatrix}\beta_2 + 2\beta_1\beta_2 \\ \beta_1^2 + \beta_1\end{pmatrix} +\ea +$$ + +Both methods agree. ✓ + +::: + {{< slidebreak >}} :::{#thm-deriv-linear-map} @@ -236,7 +282,7 @@ $$ ::: :::{#exm-deriv-linear-map} -#### Example: derivative of linear map +#### Derivative of a linear map Let $A = \begin{pmatrix} 2 & 3 \end{pmatrix}$ ($1 \times 2$) and $\vb = \tp{(\beta_1, \beta_2)}$. Then $A\vb = 2\beta_1 + 3\beta_2$, and by @thm-deriv-linear-map: @@ -250,7 +296,7 @@ $$ ::: ---- +{{< slidebreak >}} :::{#thm-deriv-matrix-vector} @@ -267,13 +313,13 @@ $$ \underbrace{\tp{A}}_{q \times m} $$ -::: - -::: notes +:::: notes This generalizes @thm-deriv-linear-map, which is the special case $\vecf{v} = \vb$ (so that $\deriv{\vb} \vb = \matr{I}$ and $\deriv{\vb} (A\vb) = \tp{A}$). +:::: + ::: ::: proof @@ -295,7 +341,7 @@ $$ ::: :::{#exm-deriv-matrix-vector} -#### Example: vector-derivative of matrix-vector product +#### Vector-derivative of a matrix-vector product Let $A = \begin{pmatrix} 2 & 3 \end{pmatrix}$ ($1 \times 2$, constant) and $\vecf{v}(\vb) = \tp{(\beta_1^2, \beta_2^2)}$. @@ -316,7 +362,7 @@ $$ ::: ---- +{{< slidebreak >}} :::{#cor-deriv-lincom-tp} @@ -375,7 +421,7 @@ $$ ::: :::{#exm-deriv-lincom-tp} -#### Example: derivative of transpose product +#### Derivative of a transpose product Let $\vx = \tp{(3, 5)}$ and $\vb = \tp{(\beta_1, \beta_2)}$. Then $\tp{\vx}\vb = 3\beta_1 + 5\beta_2$, and by @cor-deriv-lincom-tp: From a46d7d05943b653bac60311041705e3b2ab8a53a Mon Sep 17 00:00:00 2001 From: "claude[bot]" <209825114+claude[bot]@users.noreply.github.com> Date: Sat, 6 Jun 2026 00:57:14 +0000 Subject: [PATCH 18/22] fix: introduce p in thm-deriv-matrix-product-vector Add 'that depends on the p x 1 vector vb' so that p is defined before use in the result's dimension annotation. --- _subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd index efeba1913c..913267a914 100644 --- a/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-vector.qmd @@ -4,7 +4,8 @@ If $A$ ($\ell \times m$) and $B$ ($m \times q$) are constant with respect to $\vb$, -and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector, +and $\vecf{v} = \vecf{v}(\vb)$ is a $q \times 1$ vector +that depends on the $p \times 1$ vector $\vb$, then: $$ From d1b37199cefb0418c30b321cc96702134a88f1fb Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 6 Jun 2026 01:53:38 +0000 Subject: [PATCH 19/22] Add example after thm-deriv-lincom MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses d-morrison's request to follow each new definition and theorem with a relevant example. `thm-deriv-lincom` was the only new theorem in this section without an immediately-following `{#exm-...}` div — all the others (thm-deriv-dot-product, thm-deriv-linear-map, thm-deriv-matrix-vector, cor-deriv-lincom-tp, and the parked thm-deriv-matrix-product-* files) already had one. The new exm-deriv-lincom uses the same numerical setup as exm-constant-wrt-vector ($\vx = (3,5)^T$, $\vb = (\beta_1, \beta_2)^T$) so the worked entry-wise check connects directly to the constancy example two divs earlier. --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 30 +++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 87ff3ec68e..7278a61d07 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -150,6 +150,36 @@ $$ {{< slidebreak >}} +:::{#exm-deriv-lincom} +#### Derivative of a dot product + +Let $\vx = \tp{(3, 5)}$ (constant with respect to $\vb$; +see @exm-constant-wrt-vector) and $\vb = \tp{(\beta_1, \beta_2)}$. +Then $\vx \cdot \vb = 3\beta_1 + 5\beta_2$, and by @thm-deriv-lincom: + +$$ +\underbrace{\deriv{\vb}(\vx \cdot \vb)}_{2 \times 1} += \underbrace{\vx}_{2 \times 1} += \begin{pmatrix} 3 \\ +5 \end{pmatrix} +$$ + +Verifying entry-wise: + +$$ +\deriv{\vb}(3\beta_1 + 5\beta_2) += \begin{pmatrix} \deriv{\beta_1}(3\beta_1 + 5\beta_2) \\ +\deriv{\beta_2}(3\beta_1 + 5\beta_2) \end{pmatrix} += \begin{pmatrix} 3 \\ +5 \end{pmatrix} +$$ + +Both methods agree. ✓ + +::: + +{{< slidebreak >}} + :::{#thm-deriv-dot-product} From b01f57d81eccf6e7fdb1cb5b58ea409c596ef4a6 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 9 Jun 2026 07:54:46 +0000 Subject: [PATCH 20/22] =?UTF-8?q?Fix=20parked=20thm-deriv-matrix-product-m?= =?UTF-8?q?atrix:=20---=20=E2=86=92=20slidebreak,=20nest=20notes=20inside?= =?UTF-8?q?=20theorem=20div?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../math-prereqs/_thm-deriv-matrix-product-matrix.qmd | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd index d8febd499f..6cfa6ebd07 100644 --- a/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd +++ b/_subfiles/math-prereqs/_thm-deriv-matrix-product-matrix.qmd @@ -25,7 +25,7 @@ $$ ::: ---- +{{< slidebreak >}} :::{#thm-deriv-matrix-product-matrix} @@ -41,14 +41,14 @@ $$ \underbrace{\tp{B}}_{r \times n} $$ -::: - -::: notes +:::: notes The trace makes $\operatorname{tr}(A \matr{X} B)$ a scalar, so its matrix derivative is again an $m \times n$ matrix. The derivative of the matrix product $A \matr{X} B$ itself (without the trace) is a fourth-order tensor, which is why this result is stated for the scalar $\operatorname{tr}(A \matr{X} B)$. +:::: + ::: ::: proof From a99c4aefad08662e866b66358c7f24585d41e1cf Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 15 Jun 2026 07:48:12 +0000 Subject: [PATCH 21/22] ci: re-trigger checks after transient claude-review API error The claude-review job failed on a transient socket-close API error on 2026-06-09; build-deploy, lint, and spellcheck all passed. This empty commit re-triggers CI. https://claude.ai/code/session_01NQSLJmjMN22RpwpgJ6Lw6u From 1b2a7e7f5a00922c83cfd5c6ede61bf18bb54a7b Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 19 Jun 2026 07:54:02 +0000 Subject: [PATCH 22/22] Fix doubled line break in matrix-vector derivative example Replace \\\\ with \\ in the aligned block of exm-deriv-matrix-vector so the step does not insert an unintended blank row. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_011CqGbHzLqKVGnbEpLEdadG --- _subfiles/math-prereqs/_sec_vector_calc.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_subfiles/math-prereqs/_sec_vector_calc.qmd b/_subfiles/math-prereqs/_sec_vector_calc.qmd index 7278a61d07..8197f3d8ca 100644 --- a/_subfiles/math-prereqs/_sec_vector_calc.qmd +++ b/_subfiles/math-prereqs/_sec_vector_calc.qmd @@ -384,7 +384,7 @@ $$ &= \begin{pmatrix} 2\beta_1 & 0 \\ 0 & 2\beta_2 \end{pmatrix} \begin{pmatrix} 2 \\ -3 \end{pmatrix} \\\\ +3 \end{pmatrix} \\ &= \begin{pmatrix} 4\beta_1 \\ 6\beta_2 \end{pmatrix} \ea