Merge main into qwen tilelet branch and resolve opcode conflicts

Author: HecreReed

.github/workflows/ci.yml

@@ -293,7 +293,7 @@ jobs:
           # suite (RUN_ONLY_CASES is empty), skip the non-matching variant based
           # on SOC_VERSION to keep the remote validation portable.
           A3_ONLY_CASES="partition5d,partition5d_dynamic,mrgsort,tmatmulk_autosync"
-          A5_ONLY_CASES="partition5d_a5,partition5d_dynamic_a5,mrgsort_a5,tmatmulk_autosync_a5"
+          A5_ONLY_CASES="partition5d_a5,partition5d_dynamic_a5,mrgsort_a5,tmatmulk_autosync_a5,tpack"
 
           sv_lc="$(printf '%s' "${SOC_VERSION}" | tr '[:upper:]' '[:lower:]')"
           is_a5=0

docs/PTO_IR_manual.md

@@ -707,6 +707,103 @@ pto.tload ins(%pv : !pto.partition_tensor_view<16x16xf16>)
 
 ---
 
+##### `pto.tprefetch` - Prefetch Partition View into Tile
+
+**Summary:** Prefetches a GM-backed partition view into a temporary local tile buffer. This maps to PTO-ISA `TPREFETCH(dst, src)` and, unlike most PTO intrinsics, does not add implicit wait-event synchronization in the C++ wrapper.
+
+**Semantics:**
+
+```
+TPREFETCH(dst, src)
+```
+
+The detailed caching / hint behavior is target-defined by PTO-ISA. In PTOAS the
+op is modeled as writing the prefetched data into `dst`.
+
+**Arguments:**
+
+| Name | Type | Description |
+|------|------|-------------|
+| `src` | `pto.partition_tensor_view` or lowered GM memref | Source global view |
+| `dst` | `pto.tile_buf` or lowered local memref | Destination local tile |
+
+**Results:** None. Writes into `dst` via DPS pattern.
+
+**Constraints & Verification:**
+
+- `src` must be a partition view before lowering, or the corresponding lowered ranked memref form after `PTOViewToMemref`.
+- `dst` must be a tile buffer before lowering, or the corresponding lowered ranked memref form after `PTOViewToMemref`.
+- `dst` must use `loc=vec` or `loc=mat`.
+- Static source extents and static destination valid extents must be positive when known.
+- `src` and `dst` element types must have the same element size in bytes.
+
+**Hardware Mapping:**
+
+- Executes on the **DMA pipeline** (`PIPE_MTE2`, GM -> local tile)
+
+**Basic Example:**
+
+```mlir
+pto.tprefetch ins(%pv : !pto.partition_tensor_view<16x16xf16>)
+              outs(%tb : !pto.tile_buf<loc=vec, dtype=f16, rows=16, cols=16,
+                    v_row=16, v_col=16, blayout=row_major, slayout=none_box,
+                    fractal=512, pad=0>)
+```
+
+---
+
+##### `pto.tpack` - Pack a Wider Vec Tile into a Narrower Vec Tile
+
+**Summary:** A5-only vector packing operation that narrows a source VEC tile into
+a destination VEC tile of the same valid shape.
+
+**Semantics:**
+
+```
+TPACK(dst, src)
+```
+
+Supported packing directions follow PTO-ISA:
+- `b32 -> b16`
+- `b32 -> b8`
+- `b16 -> b8`
+
+**Arguments:**
+
+| Name | Type | Description |
+|------|------|-------------|
+| `src` | `pto.tile_buf` | Source VEC tile with wider element type |
+| `dst` | `pto.tile_buf` | Destination VEC tile with narrower element type |
+
+**Results:** None. Writes into `dst` via DPS pattern.
+
+**Constraints & Verification:**
+
+- `pto.tpack` is only supported on **A5** targets.
+- `src` and `dst` must both be VEC tiles (`loc=vec`) with row-major layout.
+- `src` and `dst` must have the same `valid_shape`.
+- Supported element-size pairs are exactly:
+  - `4 -> 2` bytes
+  - `4 -> 1` bytes
+  - `2 -> 1` bytes
+
+**Hardware Mapping:**
+
+- Executes on the **Vector pipeline** (`PIPE_V`)
+
+**Basic Example:**
+
+```mlir
+pto.tpack ins(%src : !pto.tile_buf<loc=vec, dtype=i32, rows=128, cols=128,
+              v_row=128, v_col=128, blayout=row_major, slayout=none_box,
+              fractal=512, pad=0>)
+          outs(%dst : !pto.tile_buf<loc=vec, dtype=i16, rows=128, cols=128,
+               v_row=128, v_col=128, blayout=row_major, slayout=none_box,
+               fractal=512, pad=0>)
+```
+
+---
+
 ##### `pto.tstore` - Store Tile to Partition View
 
 **Summary:** Stores a 2-D tile buffer back to a 2-D partition view. Supports phase/atomic/relu/pre-quant controls that lower to the corresponding `TSTORE` template overload family.
@@ -1801,6 +1898,7 @@ Division-by-zero behavior is target-defined.
 |------|------|-------------|
 | `src0` | `pto.tile_buf` | Dividend tile buffer |
 | `src1` | `pto.tile_buf` | Divisor tile buffer |
+| `tmp` | `pto.tile_buf` | Temporary tile buffer required by the ISA API |
 | `dst` | `pto.tile_buf` | Destination tile buffer |
 
 **Results:** None. Writes into `dst` via DPS pattern.
@@ -1996,23 +2094,25 @@ For each element (i, j):
 **Assembly Format:**
 
 ```
-pto.trem ins(<src0>, <src1> : <src0_type>, <src1_type>)
+pto.trem ins(<src0>, <src1>, <tmp> : <src0_type>, <src1_type>, <tmp_type>)
          outs(<dst> : <dst_type>)
 ```
 
 **Constraints & Verification:**
 
 - The implementation uses `dst valid row` / `dst valid column` as the iteration domain.
 - **Implementation checks (A2A3)**
-  - Tile element type must be one of: `i32`, `i16`, `f16`, `f32`.
-  - Tile must use row-major layout (`blayout=row_major`).
-  - Valid bounds: `valid row <= rows` and `valid column <= cols`.
-  - Runtime: `src0`, `src1` and `dst` tiles should have the same `validRow/validCol`.
+  - `src0/src1/dst` element type must match, and must be `i32` or `f32`.
+  - `tmp` element type must match `dst`.
+  - `src0/src1/tmp/dst` must use row-major layout (`blayout=row_major`).
+  - `src0/src1/dst` must have the same `validRow/validCol`.
+  - `tmp` must provide at least `1` valid row and `tmp.validCol >= dst.validCol`.
 - **Implementation checks (A5)**
-  - Tile element type must be one of: `i32`, `i16`, `f32`, `f16`.
-  - Tile must use row-major layout (`blayout=row_major`).
-  - Valid bounds: `valid row <= rows` and `valid column <= cols`.
-  - Runtime: `src0`, `src1` and `dst` tiles should have the same `validRow/validCol`.
+  - `src0/src1/dst` element type must match, and must be one of: `i32`, `i16`, `f16`, `f32`.
+  - `tmp` element type must match `dst`.
+  - `src0/src1/tmp/dst` must use row-major layout (`blayout=row_major`).
+  - `src0/src1/dst` must have the same `validRow/validCol`.
+  - `tmp` must provide at least `1` valid row and `tmp.validCol >= dst.validCol`.
 
 **Hardware Mapping:**
 
@@ -2022,11 +2122,14 @@ pto.trem ins(<src0>, <src1> : <src0_type>, <src1_type>)
 **Basic Example:**
 
 ```mlir
-pto.trem ins(%a, %b : !pto.tile_buf<loc=vec, dtype=f32, rows=16, cols=16,
+pto.trem ins(%a, %b, %tmp : !pto.tile_buf<loc=vec, dtype=f32, rows=16, cols=16,
              v_row=16, v_col=16, blayout=row_major, slayout=none_box,
              fractal=512, pad=0>,
              !pto.tile_buf<loc=vec, dtype=f32, rows=16, cols=16,
              v_row=16, v_col=16, blayout=row_major, slayout=none_box,
+             fractal=512, pad=0>,
+             !pto.tile_buf<loc=vec, dtype=f32, rows=1, cols=16,
+             v_row=1, v_col=16, blayout=row_major, slayout=none_box,
              fractal=512, pad=0>)
          outs(%c : !pto.tile_buf<loc=vec, dtype=f32, rows=16, cols=16,
              v_row=16, v_col=16, blayout=row_major, slayout=none_box,
@@ -2809,30 +2912,35 @@ For each element (i, j):
 |------|------|-------------|
 | `src` | `pto.tile_buf` | Source tile buffer |
 | `scalar` | `ScalarType` (signless integer / float) | Scalar divisor |
+| `tmp` | `pto.tile_buf` | Temporary tile buffer required by the ISA API |
 | `dst` | `pto.tile_buf` | Destination tile buffer |
 
 **Results:** None. Writes into `dst` via DPS pattern.
 
 **Assembly Format:**
 
 ```
-pto.trems ins(<src>, <scalar> : <src_type>, <scalar_type>)
+pto.trems ins(<src>, <scalar>, <tmp> : <src_type>, <scalar_type>, <tmp_type>)
           outs(<dst> : <dst_type>)
 ```
 
 **Constraints & Verification:**
 
 - Division-by-zero behavior is target-defined; the CPU simulator asserts in debug builds.
 - **Implementation checks (A2A3)**
-  - Tile element type must be one of: `i32`, `int`, `i16`, `f16`, `f32`.
-  - Tile must use `loc=vec`.
-  - Valid bounds: `valid row <= rows` and `valid column <= cols`.
-  - Runtime: `src0 valid row == dst valid row` and `src0 valid column == dst valid column`.
+  - `src/dst` element type must match, and must be `i32` or `f32`.
+  - `scalar` type must match the tile element type.
+  - `tmp` element type must match `dst`.
+  - `src/tmp/dst` must use row-major layout (`blayout=row_major`).
+  - `src` and `dst` must have the same `validRow/validCol`.
+  - `tmp` must provide at least `1` valid row and `tmp.validCol >= dst.validCol`.
 - **Implementation checks (A5)**
-  - Tile element type must be one of: `i8`, `i16`, `i32`, `f16`, `f32`, `bf16`.
-  - Tile must use `loc=vec`.
-  - Valid bounds: `valid row <= rows` and `valid column <= cols`.
-  - Runtime: `src0 valid row == dst valid row` and `src0 valid column == dst valid column`.
+  - `src/dst` element type must match, and must be one of: `i32`, `i16`, `f16`, `f32`.
+  - `scalar` type must match the tile element type.
+  - `tmp` element type must match `dst`.
+  - `src/tmp/dst` must use row-major layout (`blayout=row_major`).
+  - `src` and `dst` must have the same `validRow/validCol`.
+  - `tmp` must provide at least `1` valid row and `tmp.validCol >= dst.validCol`.
 
 **Hardware Mapping:**
 
@@ -2842,9 +2950,12 @@ pto.trems ins(<src>, <scalar> : <src_type>, <scalar_type>)
 **Basic Example:**
 
 ```mlir
-pto.trems ins(%a, %s : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=32,
+pto.trems ins(%a, %s, %tmp : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=32,
               v_row=32, v_col=32, blayout=row_major, slayout=none_box,
-              fractal=512, pad=0>, f32)
+              fractal=512, pad=0>, f32,
+              !pto.tile_buf<loc=vec, dtype=f32, rows=1, cols=32,
+              v_row=1, v_col=32, blayout=row_major, slayout=none_box,
+              fractal=512, pad=0>)
           outs(%c : !pto.tile_buf<loc=vec, dtype=f32, rows=32, cols=32,
               v_row=32, v_col=32, blayout=row_major, slayout=none_box,
               fractal=512, pad=0>)
@@ -3377,6 +3488,72 @@ pto.tsqrt ins(%a : !pto.tile_buf<loc=vec, dtype=f16, rows=16, cols=16,
 
 ---
 
+##### `pto.ttri` - Fill Triangular Tile Region
+
+**Summary:** Fills a VEC tile using the PTO-ISA `TTRI` triangular pattern.
+
+**Semantics:**
+
+```
+TTRI(dst, diagonal)
+```
+
+`upperOrLower=0` selects the lower-triangular form and `upperOrLower=1`
+selects the upper-triangular form. The exact per-element fill pattern follows
+the target PTO-ISA implementation.
+
+**Arguments:**
+
+| Name | Type | Description |
+|------|------|-------------|
+| `diagonal` | integer SSA value | Runtime diagonal selector |
+| `dst` | `pto.tile_buf` | Destination vector tile |
+
+**Attributes:**
+
+| Name | Type | Description |
+|------|------|-------------|
+| `upperOrLower` | `I32Attr` (default: `0`) | `0` for lower triangular, `1` for upper triangular |
+
+**Results:** None. Writes into `dst` via DPS pattern.
+
+**Assembly Format:**
+
+```mlir
+pto.ttri ins(%diag {upperOrLower = 1 : i32} : i32)
+         outs(%dst : !pto.tile_buf<...>)
+```
+
+**Constraints & Verification:**
+
+- `dst` must be a VEC tile (`loc=vec`) whose valid shape stays within the static tile shape.
+- `dst` must use `blayout=row_major`.
+- `diagonal` must have an integer type.
+- `upperOrLower` must be either `0` or `1`.
+- Supported element types:
+  - A2/A3: `f16`, `f32`, `i16`, `i32`, `u16`, `u32`
+  - A5: `f16`, `f32`, `bf16`, `i8`, `i16`, `i32`, `u8`, `u16`, `u32`
+
+**Hardware Mapping:**
+
+- Executes on the **Vector pipeline** (`PIPE_V`)
+
+**Basic Example:**
+
+```mlir
+pto.ttri ins(%diag : i32)
+         outs(%lower : !pto.tile_buf<loc=vec, dtype=i32, rows=32, cols=32,
+               v_row=32, v_col=32, blayout=row_major, slayout=none_box,
+               fractal=512, pad=0>)
+
+pto.ttri ins(%diag {upperOrLower = 1 : i32} : i32)
+         outs(%upper : !pto.tile_buf<loc=vec, dtype=i32, rows=32, cols=32,
+               v_row=32, v_col=32, blayout=row_major, slayout=none_box,
+               fractal=512, pad=0>)
+```
+
+---
+
 ##### `pto.trsqrt` - Elementwise Reciprocal Square Root
 
 **Summary:** Computes the reciprocal square root for every element.