Skip to content

Add processing_status fields and error recovery docs #95

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mintlify wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add processing_status fields and error recovery docs #95

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion api-reference/comments-list.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,10 @@ Retrieve classified comments with filtering and pagination. This endpoint is **f
| `sentiment` | string | No | Filter by sentiment: `positive`, `negative`, `neutral`, `mixed` |
| `dialect` | string | No | Filter by dialect: `gulf`, `egyptian`, `levantine`, `msa` |
| `toxicity` | string | No | Filter by toxicity: `none`, `mild`, `moderate`, `severe` |
| `processing_status` | string | No | Filter by processing state: `pending`, `processing`, `completed`, `failed`, `auto_executed`, `retry_pending` |
| `from` | string | No | Start date (ISO 8601): `2025-01-01T00:00:00Z` |
| `to` | string | No | End date (ISO 8601): `2025-01-31T23:59:59Z` |
| `limit` | integer | No | Results per page (1–100). Default: 25. |
| `limit` | integer | No | Results per page (1-100). Default: 25. |
| `cursor` | string | No | Pagination cursor from previous response. |

### Example request
Expand Down Expand Up @@ -77,6 +78,8 @@ for comment in nawa.comments.list(platform="youtube"):
"sentiment": "neutral",
"dialect": "gulf",
"toxicity": "none",
"processing_status": "completed",
"error_message": null,
"created_at": "2025-01-15T10:30:00Z",
"channel_id": "ch_123"
}
Expand All @@ -91,3 +94,34 @@ for comment in nawa.comments.list(platform="youtube"):
"request_id": "req_lst_abc123"
}
```

### Result fields

| Field | Type | Description |
|-------|------|-------------|
| `id` | string | Unique comment identifier |
| `text` | string | The comment text |
| `platform` | string | Source platform: `youtube`, `instagram`, `twitter`, `facebook` |
| `intent` | string | Classified intent label |
| `sentiment` | string | Detected sentiment |
| `dialect` | string | Detected Arabic dialect, if applicable |
| `toxicity` | string | Toxicity severity level |
| `processing_status` | string | Current processing state (see table below) |
| `error_message` | string \| null | Error description when processing failed. `null` on success. |
| `created_at` | string | ISO 8601 timestamp of comment creation |
| `channel_id` | string | Channel or account the comment belongs to |

### Processing status values

| Status | Description |
|--------|-------------|
| `pending` | Comment is queued for classification |
| `processing` | Classification is actively running |
| `completed` | Successfully classified and ready for review |
| `failed` | Classification failed. Check `error_message` for details. |
| `auto_executed` | Classified and automatically actioned (reply, heart, or skip) |
| `retry_pending` | Failed with a transient error (e.g. provider timeout) and queued for automatic retry |

<Tip>
Filter for `processing_status=failed` to find comments that need reprocessing. Failed comments always include an `error_message` with the failure reason, making it straightforward to diagnose and retry.
</Tip>
42 changes: 42 additions & 0 deletions errors.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,11 +267,53 @@ An unexpected error occurred on the NAWA side.
"suggested_action": "Retry after a few seconds. Check status.trynawa.com for updates."
}
```

When a provider failure occurs during comment processing, the comment's `processing_status` is set to `failed` and the `error_message` field contains the failure reason. This ensures the comment is never stuck in a `processing` state and can be retried. Transient failures (e.g. rate limits from the upstream provider) set `processing_status` to `retry_pending` and are automatically retried.
</Accordion>
</AccordionGroup>

---

## Comment processing error recovery

When NAWA processes a comment (via classification, reply generation, or batch processing), the comment's `processing_status` field tracks its lifecycle. If an error occurs at any point during processing, the status is automatically set to `failed` with a descriptive `error_message`.

| Processing status | Meaning | Action |
|-------------------|---------|--------|
| `failed` | A permanent error occurred (invalid input, access denied) | Investigate `error_message`, fix the issue, and reclassify |
| `retry_pending` | A transient error occurred (provider timeout, upstream rate limit) | NAWA automatically retries these. No action needed. |

You can query failed comments and retry them:

<CodeGroup>

```bash cURL
curl "https://api.trynawa.com/v1/comments?processing_status=failed&limit=50" \
-H "Authorization: Bearer nawa_live_sk_xxx"
```

```typescript TypeScript
const { data } = await nawa.comments.list({
processing_status: 'failed',
limit: 50
})

for (const comment of data.comments) {
console.log(`${comment.id}: ${comment.error_message}`)
}
```

```python Python
result = nawa.comments.list(processing_status="failed", limit=50)

for comment in result.data["comments"]:
print(f"{comment['id']}: {comment['error_message']}")
```

</CodeGroup>

---

## Handling errors in code

<CodeGroup>
Expand Down
56 changes: 56 additions & 0 deletions guides/batch-processing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,59 @@ Estimate batch cost before processing:
<Warning>
Always respect the `X-RateLimit-Remaining` header. If it reaches 0, wait until `X-RateLimit-Reset` before sending more requests.
</Warning>

## Error recovery

After a batch run, some comments may end up with `processing_status` set to `failed`. NAWA automatically resets stuck comments so they never stay in `processing` state. You can query for failed comments and retry them.

### Fetching failed comments

<CodeGroup>

```bash cURL
curl "https://api.trynawa.com/v1/comments?processing_status=failed&limit=100" \
-H "Authorization: Bearer nawa_live_sk_xxx"
```

```typescript TypeScript
const { data } = await nawa.comments.list({
processing_status: 'failed',
limit: 100
})

console.log(`${data.comments.length} comments failed`)
for (const c of data.comments) {
console.log(` ${c.id}: ${c.error_message}`)
}
```

```python Python
result = nawa.comments.list(processing_status="failed", limit=100)

print(f"{len(result.data['comments'])} comments failed")
for c in result.data["comments"]:
print(f" {c['id']}: {c['error_message']}")
```

</CodeGroup>

### Retrying failed comments

Re-classify failed comments by passing them through the same batch pipeline. Comments with `retry_pending` status are automatically retried by NAWA and do not need manual intervention.

```typescript
const failed = await nawa.comments.list({ processing_status: 'failed', limit: 100 })

const retryBatch = failed.data.comments.map(c => ({
id: c.id,
text: c.text,
platform: c.platform
}))

const retryResults = await processBatch(retryBatch, 3) // lower concurrency for retries
console.log(`Retried ${retryResults.length} comments`)
```

<Tip>
Use the `classification.failed` [webhook event](/webhooks) to get real-time notifications when comments fail processing, instead of polling.
</Tip>
24 changes: 24 additions & 0 deletions openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -977,6 +977,13 @@ paths:
type: string
enum: [none, mild, moderate, severe]
description: Filter by toxicity
- name: processing_status
in: query
required: false
schema:
type: string
enum: [pending, processing, completed, failed, auto_executed, retry_pending]
description: Filter by processing state
- name: from
in: query
required: false
Expand Down Expand Up @@ -1030,6 +1037,8 @@ paths:
sentiment: neutral
dialect: gulf
toxicity: none
processing_status: completed
error_message: null
created_at: "2025-01-15T10:30:00Z"
channel_id: ch_123
pagination:
Expand Down Expand Up @@ -2076,6 +2085,21 @@ components:
toxicity:
type: string
enum: [none, mild, moderate, severe]
processing_status:
type: string
enum: [pending, processing, completed, failed, auto_executed, retry_pending]
description: |
Current processing state. `pending` = awaiting classification,
`processing` = classification in progress, `completed` = successfully
classified, `failed` = classification failed (see `error_message`),
`auto_executed` = classified and auto-actioned, `retry_pending` =
failed with a transient error and queued for retry.
error_message:
type: string
nullable: true
description: |
Human-readable error description when `processing_status` is `failed`
or `retry_pending`. Null when processing succeeded.
created_at:
type: string
format: date-time
Expand Down
24 changes: 23 additions & 1 deletion webhooks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,20 @@ Receive real-time notifications when events occur in your NAWA account. Webhooks
| Event | Description | Trigger |
|-------|-------------|---------|
| `classification.completed` | A classification request succeeded | After `/v1/classify` completes |
| `classification.failed` | A classification request failed | On provider or internal errors |
| `classification.failed` | A classification request failed | On any provider, internal, or transient error |
| `comment.new` | A new comment was ingested | When a connected platform receives a comment |
| `comment.replied` | A reply was posted | After `/v1/comments/:id/reply` succeeds |
| `credits.low` | Credit balance below threshold | Balance drops below $5 |
| `credits.exhausted` | Credit balance is $0 | Balance reaches $0 |

<Note>
`classification.failed` fires on every failure path, including provider timeouts, rate limit errors from upstream AI providers, and unexpected internal errors. The comment's `processing_status` is always reset to `failed` so it can be retried. There is no scenario where a comment stays stuck in `processing` state.
</Note>

## Webhook payload

### classification.completed

```json
{
"id": "evt_abc123",
Expand All @@ -34,6 +40,22 @@ Receive real-time notifications when events occur in your NAWA account. Webhooks
}
```

### classification.failed

```json
{
"id": "evt_fail789",
"type": "classification.failed",
"created_at": "2025-01-15T12:01:00Z",
"data": {
"request_id": "req_ghi012",
"comment_id": "cmt_abc123",
"error_message": "AI provider temporarily unavailable.",
"processing_status": "failed"
}
}
```

## Webhook signing secret

Your webhook signing secret starts with `nawa_wh_` and is generated when you register a webhook endpoint. Store it securely - it's shown only once.
Expand Down