Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
188 changes: 188 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
# bus-ui (Leo Botmon) — Agent Guide

## Overview

bus-ui is a React 15 monitoring dashboard for the Leo Platform's event streaming bus. It provides real-time visibility into bot execution, queue health, event lag, and system metrics. Deployed as a CloudFormation-managed Lambda microservice with a compiled React frontend and multiple API handlers.

## Tech Stack

- **Frontend**: React 15.6.2, Redux 4.2.0, Redux Thunk, MobX 3.3.0, Material-UI 0.19.4
- **State Management**: Redux (global UI state) + MobX (DataStore for bot/queue/system data)
- **Build**: Babel with presets (react, es2015, stage-0) + transform-decorators-legacy
- **Backend**: Node.js 16.x Lambda handlers, leo-sdk 6.0.17, leo-auth 3.0.2
- **Infrastructure**: CloudFormation, API Gateway, DynamoDB, Cognito, S3, SNS, Kinesis
- **Testing**: Mocha 3.0.2
- **CLI Tooling**: leo-cli (installed globally, not a project dependency)

## Architecture

```
bus-ui/
├── api/ # Lambda API handlers (one folder per endpoint)
│ ├── dashboard/ # Bot/queue/system metrics (complex time-bucket aggregation)
│ ├── cron/ # Bot CRUD (get/, save/, delete/, saveOverrides/)
│ ├── stats/ # Time-series statistics queries
│ ├── logs/ # Event/execution log retrieval
│ ├── settings/ # Lambda templates, botmon settings
│ ├── eventTrace/ # Event lineage tracking
│ ├── eventSettings/ # Event configuration (get/, save/)
│ ├── searchQueue/ # Queue search/filtering
│ ├── queueSchema/ # Queue metadata
│ ├── sns/ # SNS topic management
│ └── system/ # External system config (get/, save/, proxy/)
├── bots/ # Background processing bots
│ ├── healthSNS/ # Monitors bot health, triggers SNS alerts
│ └── stats-processor/ # Aggregates events into time-bucketed stats
├── ui/
│ ├── js/
│ │ ├── index.js # Redux store, MobX DataStore provider, Cognito auth bootstrap
│ │ ├── actions.js # Redux actions
│ │ ├── reducers.js # Redux reducers
│ │ └── components/
│ │ ├── main.jsx # Root App — URL hash state, workflows, search, dialogs
│ │ ├── main/ # Layout: Header, LeftNav, Content, DetailPane, Footer
│ │ ├── tabs/ # Detail panels: botDashboard, queueDashboard, eventViewer, logs, etc.
│ │ ├── elements/ # Widgets: nodeChart, timeSlider, dynamicForm, toggleSwitch, etc.
│ │ ├── dialogs/ # Modals: EventTrace, DataSourceConnect
│ │ └── pages/ # Page views: dashboard, node, list
│ └── static/ # Compiled bundles (generated by leo-cli, not committed)
├── stores/
│ └── dataStore.js # MobX store: 30+ @observable properties for all app data
├── lib/
│ ├── stats-buckets.js # Time bucket definitions (minute_1 through week)
│ ├── stats.js # Stats query builder and aggregation
│ └── humanize.js # Human-readable duration formatting
├── views/
│ └── index # HTML template (${leo.*} variable interpolation)
├── cloudformation/ # AWS resource definitions (DynamoDB, roles, etc.)
├── leo_config.js # Runtime config (leosdk, leoauth, Cognito, region)
├── leo_cli_config.js # Build/deploy config (publish targets, test personas)
└── publish.js # CloudFormation hooks (arm64, compression, tagging)
```

### API Handler Pattern

Each handler lives in `api/{resource}/{action}/` with:
- `index.js` — Handler code, wrapped with leo-sdk/wrappers and leo-auth for RBAC
- `package.json` — Metadata: URI path, HTTP method, memory, timeout

All handlers use callback-style `(err, data)` or Promise patterns. Authorization uses `request.authorize()` with LRN `lrn:leo:botmon:::`.

### State Management (Dual Pattern)

- **Redux**: Global settings, auth state, display state. Actions in `actions.js`, reducers in `reducers.js`.
- **MobX**: DataStore (`stores/dataStore.js`) holds bot/queue/system data with `@observable`, `@action`, `@computed`. Exposed globally as `window.dataStore`.
- **URL hash**: Entire navigation state is JSON-stringified into the URL hash. Reloading preserves state.

## Coding Conventions

- **Components**: Class components (no hooks — React 15). PascalCase filenames for components.
- **Actions**: camelCase verbs (`saveSettings`, `setIsAuthenticated`)
- **Redux action types**: UPPER_SNAKE_CASE (`SET_DISPLAY_STATE`, `SAVE_SETTINGS`)
- **Indentation**: Tabs (enforced by ESLint)
- **Line endings**: Unix LF
- **Semicolons**: Required
- **MobX decorators**: `@observable`, `@action`, `@computed` — requires babel transform-decorators-legacy

## Build & Test

### Local Development Setup

1. **Install leo-cli globally** (not a project dependency):
```bash
npm install -g leo-cli
```

2. **Install project dependencies**:
```bash
npm install
```

3. **Create `test/process.js`** with environment config pointing to a deployed Bus stack:
```javascript
module.exports = {
env: {
leoauthsdk: {
LeoAuth: "<LeoAuth table ARN>",
LeoAuthUser: "<LeoAuthUser table ARN>",
Region: "us-west-2"
},
leosdk: {
LeoStream: "<stream ARN>",
LeoCron: "<cron table ARN>",
LeoEvent: "<event table ARN>",
LeoSettings: "<settings table ARN>",
LeoSystem: "<system table ARN>",
LeoS3: "<S3 bucket>",
LeoKinesisStream: "<Kinesis stream ARN>",
LeoFirehoseStream: "<Firehose stream ARN>",
Region: "us-west-2"
},
Resources: {
LeoStats: "<DynamoDB stats table ARN>",
CognitoId: "<Cognito identity pool ID>"
},
StackName: "<stack name, e.g. DevBotmon>"
}
}
```
These values come from the CloudFormation stack outputs of a deployed Bus environment.

4. **Run the dev server**:
```bash
npm start # Runs "leo-cli test ." — bundles UI and serves locally
bus=prod npm start # Point at a different environment
```

5. **Node version note**: React 15.6.2 + Babel ES2015 may require Node 14 or 16. Node 23+ may have compatibility issues.

### Commands

| Command | Description |
|---------|-------------|
| `npm start` | Local dev server (`leo-cli test .`) |
| `npm test` | Mocha tests in watch mode |
| `npm run build` | Compile + create deployment package |
| `npm run publish` | Deploy to configured regions (S3 + CloudFormation) |

### Deployment (Manual — No CI/CD)

1. `npm run publish` — generates CloudFormation template, uploads to S3
2. AWS Console → CloudFormation → update stack with new S3 template URL
3. Stack parameters: CognitoId, leoauth, leosdk
4. Only `dev` (DevBotmon) has a deploy config in `leo_cli_config.js`; prod/staging require manual console updates

## Key Patterns

1. **URL Hash as State**: Navigation state is JSON in the URL hash (`#{"view":"dashboard","node":"bot_id"}`). All state changes go through hash updates. localStorage stores saved views/searches.

2. **Time Bucket Aggregation**: Stats are bucketed (minute_1, minute_5, minute_15, hour, day, week) by the stats-processor bot. Dashboard API reconstructs metrics from DynamoDB using bucket ranges.

3. **Ref/RefId Utilities**: `leo-sdk/lib/reference.js` provides `ref()`, `refId()`, `botRef()`, `botRefId()` for standardized bot/queue/system ID formatting.

4. **Parallel DynamoDB Queries**: Dashboard uses `async.parallel()` and `async.parallelLimit()` to fan out stats queries for bots, queues, and systems concurrently.

## Domain Context

- **Bot**: Lambda-based event processor (e.g., "MyBot_Lambda")
- **Queue**: Named event stream (e.g., "system_table_events")
- **System**: External data system (Elasticsearch, MongoDB, CSV, etc.)
- **Checkpoint**: Latest event processed by a bot for a queue
- **Lag**: Delay between event source timestamp and processing timestamp
- **Health**: Execution count, error count, duration metrics over time buckets

## Gotchas

1. **React 15 auto-escapes JSX text**: jQuery HTML encoding (`.text().html()`) before React rendering causes double-encoding. Don't pre-encode text content that goes into `{expressions}`.

2. **Decorator plugin required**: Missing `transform-decorators-legacy` in Babel config breaks all MobX `@observable`/`@action`/`@computed` usage.

3. **arm64 Lambda**: `publish.js` forces arm64 architecture on all Lambdas. Third-party native binaries may not support arm64.

4. **DynamoDB query limits**: Dashboard queries cap at `mb: 100`. Large time ranges may return incomplete data.

5. **Lag computation edge case**: If read checkpoint < write checkpoint with no recent reads, dashboard synthesizes growing lags for missing data points.

6. **No test directory in repo**: `test/process.js` must be created locally — it contains environment-specific AWS ARNs and is not committed.

7. **React 15 is EOL**: No hooks, context API, or suspense. Upgrade is non-trivial due to MobX 3 + redux-react 5 coupling.
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@AGENTS.MD
2 changes: 1 addition & 1 deletion api/cron/delete/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
],
"name": "Leo_Botmon_api_cron_delete",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/cron/get/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
],
"name": "Leo_Botmon_api_cron_get",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/cron/save/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
],
"name": "Leo_Botmon_api_cron_save",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/cron/saveOverrides/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
],
"name": "Leo_Botmon_api_setting_saveOverrides",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/eventSettings/get/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
],
"name": "Leo_Botmon_api_event_settings_get",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/eventSettings/save/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
],
"name": "Leo_Botmon_api_event_settings_save",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"build": {}
}
Expand Down
2 changes: 1 addition & 1 deletion api/logs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"name": "Leo_Botmon_api_logs",
"handler": "handler",
"role": null,
"memory": 256,
"memory": 512,
"timeout": 30,
"env": {
"Resources": {
Expand Down
2 changes: 1 addition & 1 deletion api/settings/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
],
"name": "Leo_Botmon_api_settings",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 10,
"env": {
"Resources": {
Expand Down
2 changes: 1 addition & 1 deletion api/sns/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
],
"name": "Leo_Botmon_api_sns",
"handler": "handler",
"memory": 256,
"memory": 512,
"timeout": 30,
"build": {},
"env": {
Expand Down
2 changes: 1 addition & 1 deletion api/system/get/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"name": "Leo_Botmon_api_system_get",
"handler": "handler",
"role": null,
"memory": 256,
"memory": 512,
"timeout": 10
}
},
Expand Down
2 changes: 1 addition & 1 deletion api/system/proxy/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"name": "Leo_Botmon_api_proxy",
"handler": "handler",
"role": null,
"memory": 256,
"memory": 512,
"timeout": 10
}
},
Expand Down
2 changes: 1 addition & 1 deletion api/system/save/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"name": "Leo_Botmon_api_system_save",
"handler": "handler",
"role": null,
"memory": 256,
"memory": 512,
"timeout": 10
}
},
Expand Down
2 changes: 1 addition & 1 deletion ui/js/components/tabs/eventViewer.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,7 @@ class EventViewer extends React.Component {
// let [diffElementThingy, setDiffElementThingy] = useState(old_new && old_obj && new_obj ? getOldNewDiff(old_obj, new_obj) : '');


detail = $('<div/>').text(JSON.stringify(detail, null, 4)).html()
detail = JSON.stringify(detail, null, 4)

var detailSearch = detail
var detailString = detail
Expand Down
2 changes: 1 addition & 1 deletion views/index
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
window.leoDocsLink = 'http://docs.leoplatform.io/docs/'

window.leo = Object.assign(${leo}, window.leo);
window.leoAws = {cognitoId: leo.logins || leo.CognitoId, region: "${leo.Region}", cognito_region: '${leo.Region}'}
window.leoAws = {cognitoId: leo.logins || leo.CognitoId || (leo.cognito && leo.cognito.id), region: "${leo.Region}" || leo.region, cognito_region: '${leo.Region}' || leo.region}

</script>
<script src="https://d3js.org/d3.v3.min.js"></script>
Expand Down