SecondStream

SecondStream is the vertical operating core for regulated industrial streams. It owns the canonical data, evidence, workflow state, assessments, offers, pricing, outcomes, permissions, and audit trail that AI agents and industrial teams need to safely operate waste and secondary-stream workflows.

The web app is one interface. Chat and agents operate the product. Deterministic software owns the source of truth.

Status: active MVP / portfolio-grade product prototype in development. The core discovery, stream, client, and offer workflows are implemented; marketplace, CRM sync, vendor network, and advanced compliance automation are downstream roadmap layers.

Product thesis

Do not build the waste agent.
Build the operating system the waste agent must use.

SecondStream is not chat-first and not marketplace-first. The defensible product is the vertical core underneath:

clients + locations + streams + evidence + assessments + offers + vendors + pricing + outcomes

AI is embedded where it creates leverage: extracting messy source material, proposing structured drafts, surfacing missing information, assisting assessment, and helping move qualified streams toward offer readiness. Humans approve canonical truth; the database owns workflow state.

Product preview

Discovery and AI-assisted intake

SecondStream starts before a stream formally exists. Users can create a stream manually or upload source material so an AI agent can identify candidate streams, companies, and locations.

Discovery entry	File upload	AI analysis

AI results	Draft editing

Stream operations

Confirmed and draft streams move into an operational workbench where field teams can monitor opportunities, review AI-created drafts, and continue qualification.

Streams workbench	Draft streams

Organization, clients, and field-agent oversight

The product is built around organization-scoped work. Admins can monitor field-agent activity, manage leads before they become clients, and inspect client/location records tied to stream activity.

Login	Organization context	Admin dashboard

Leads	Clients	Client detail

Client activity	Client streams	Client locations

Offer follow-up

Qualified streams move into an offer pipeline so the team can track commercial status, ownership, and audit what happened to each opportunity.

The problem

Industrial waste and byproduct opportunities are often buried in PDFs, lab reports, spreadsheets, voice notes, emails, and site conversations. Teams need to identify what each stream is, whether it is commercially viable, what information is missing, and how to turn it into a concrete offer.

SecondStream is designed as an operating layer for that workflow: capture messy inputs, extract candidate secondary streams, guide qualification, and move qualified opportunities toward commercial proposals.

Core workflow

flowchart LR
    A[Files, audio, text, chat] --> B[AI Discovery]
    B --> C[Human-reviewed drafts]
    C --> D[Canonical stream workspace]
    D --> E[Assessment and offer readiness]
    E --> F[Offer pipeline]
    F --> G[Vendor / logistics / compliance follow-up]
    G --> H[Outcome capture]
    H --> I[Learning loop]
    I --> D

Loading

In the current backend, the product concept of a stream/opportunity is represented primarily by the Project model. Product-facing copy uses “secondary stream” or “stream”; some older code and docs still use Project, waste, or water-treatment naming from earlier iterations.

What SecondStream must own

Layer	Why it matters
Canonical stream records	AI agents and users need a durable source of truth.
Evidence and provenance	Important claims need files, notes, transcripts, or human-approved sources.
Pending review	AI output becomes controlled, human-approved truth instead of silent mutation.
Assessments and offer readiness	Discovery only matters if it moves toward qualified commercial action.
Vendors, pricing, and outcomes	This is the data moat: what worked, failed, blocked, or converted.
Compliance state	Regulated stream workflows need explicit risk and approval boundaries.
Permissions and audit	Enterprise users need to know who changed what, when, and why.
Tool/API layer	Future Claude, ChatGPT, MCP, portals, and internal agents need governed access to the core.

Key features

• Multi-source discovery — create discovery sessions from files, audio, and free-text notes.
• AI extraction pipeline — convert unstructured source material into draft stream, company, and location candidates for review.
• Manual stream creation — support direct stream entry when the user already knows the opportunity.
• Draft review and editing — inspect, amend, confirm, or reject AI-created stream drafts before they enter the main workflow.
• Stream workspace — qualify streams with base fields, custom fields, evidence, questionnaire progress, and ownership.
• Lead-to-client lifecycle — companies begin as leads and become clients after real stream activity is attached.
• Admin oversight — monitor field-agent activity and operational progress inside an organization.
• AI chat operation layer — persistent organization-scoped threads, attachments, streaming responses, and tool/status events without making chat the source of truth.
• Offer pipeline — track each stream through commercial follow-up, audit what happened, and see whether it sold or where it stalled.
• Proposal/report surfaces — internal/client report modes, decision support, ratings, checklists, and PDF-oriented workflows.

Architecture

flowchart TB
    Core[SecondStream Core]
    Core --> Data[Clients / locations / streams]
    Core --> Evidence[Evidence / provenance / review]
    Core --> Workflow[Assessments / offers / outcomes]
    Core --> Governed[Permissions / audit / governed actions]

    Web[Next.js web app] --> Core
    Chat[Chat and agents] --> Core
    Future[Future MCP / portals / integrations] --> Core

    Core --> API[FastAPI backend]
    API --> DB[(PostgreSQL)]
    API --> Redis[(Redis)]
    API --> S3[(S3 / local file storage)]
    API --> AI[PydanticAI agents]
    AI --> Bedrock[AWS Bedrock / Claude]

    Infra[Terraform] --> ECS[ECS Fargate services]
    Infra --> RDS[RDS PostgreSQL]
    Infra --> Cache[ElastiCache Redis]
    Infra --> Bucket[S3]
    Infra --> ALB[Application Load Balancer]

Loading

Backend

• FastAPI application with versioned API routes.
• SQLAlchemy models and Alembic migrations.
• Organization-scoped multi-tenant data model.
• PostgreSQL JSONB workspace data for flexible stream facts.
• Redis-backed job status, caching, and rate limiting.
• PydanticAI agents for extraction, chat, proposal, and insight workflows.
• Background job flow for long-running proposal generation.

Frontend

• Next.js App Router with React 19 and TypeScript.
• Role-aware authenticated app shell.
• React Query for server state and Zustand for client/workspace state.
• Centralized API clients with auth and organization headers.
• Tailwind CSS v4 design system with custom OKLCH tokens.
• AI chat UI with streaming transport, attachments, and persisted history.

Infrastructure

• Terraform-managed AWS production stack.
• ECS Fargate backend service plus worker services.
• RDS PostgreSQL, ElastiCache Redis, S3, ALB, CloudWatch, ECR, and Secrets Manager.
• Amplify build configuration for the frontend.
• Production AI runtime is oriented around AWS Bedrock access for backend/worker agents.

Engineering highlights

• Vertical operating core strategy: the product owns regulated stream data and workflow state instead of competing as a generic AI chat wrapper.
• AI-native workflow design: discovery, workspace qualification, chat, and offer/proposal flows are connected rather than isolated demos.
• Typed LLM boundaries: agents use structured dependencies and outputs through Pydantic/PydanticAI patterns.
• Multi-tenant SaaS foundation: organizations, roles, companies, locations, streams, proposals, files, and timeline-like workflow data are modeled explicitly.
• Flexible domain modeling: Project.project_data uses PostgreSQL JSONB for evolving stream/workspace facts while the product model is still changing.
• Operational deployment story: Terraform defines private ECS services, managed database/cache/storage, logs, secrets, and load balancing.
• Testing discipline: backend and frontend checks cover auth, multi-tenancy, discovery sessions, offers, chat streaming, and UI/store structure.

Tech stack

Area	Stack
Frontend	Next.js, React, TypeScript, Tailwind CSS, React Query, Zustand
Backend	FastAPI, SQLAlchemy, Alembic, Pydantic, PydanticAI
AI	AWS Bedrock / Claude-oriented agents, structured extraction, streaming chat
Data	PostgreSQL, JSONB, Redis, S3/local storage
Infrastructure	Terraform, AWS ECS Fargate, RDS, ElastiCache, S3, ALB, CloudWatch, ECR, Secrets Manager, Amplify
Tooling	Bun, uv, Docker Compose, Ruff, Pytest

Repository structure

.
├── backend/          # FastAPI API, services, models, agents, migrations, tests
├── frontend/         # Next.js app, UI components, stores, API clients, tests
├── infrastructure/   # Terraform production infrastructure and deployment scripts
├── docs/             # Product, architecture, design, and agent-facing documentation
├── scripts/          # Project support scripts
└── amplify.yml       # Frontend Amplify build configuration

Local development

Backend

cd backend
uv sync
cp .env.example .env
make check

Common backend commands:

cd backend && docker-compose up
cd backend && docker-compose exec app alembic upgrade head
cd backend && make test

Frontend

cd frontend
bun install
bun dev

Quality check:

cd frontend && bun run check:ci

See also:

• backend/README.md
• frontend/README.md
• infrastructure/README.md
• docs/agents/development-commands.md

Testing and quality

Canonical checks:

cd backend && make check
cd frontend && bun run check:ci

The test suite includes coverage around:

• authentication and authorization
• multi-tenant isolation
• discovery sessions and source intake
• bulk import / draft workflows
• chat threads, attachments, and streaming protocol behavior
• offers and proposal-related flows
• frontend routes, stores, and UI structure

Deployment notes

The infrastructure directory contains a Terraform production stack for AWS:

• public ALB in front of private ECS Fargate services
• backend service plus intake/bulk worker tasks
• private RDS PostgreSQL
• ElastiCache Redis
• S3 document/PDF storage
• CloudWatch logging and deployment health checks
• Secrets Manager-backed runtime secrets

Start with infrastructure/README.md and infrastructure/terraform/prod/README.md for deployment details.

Current status and roadmap

Implemented or actively represented in the app:

• AI chat surface
• discovery sessions from file/audio/text sources
• streams workbench, draft review, and stream workspace
• lead/client/location management
• offer pipeline and offer detail pages for stream follow-up/audit
• backend APIs for organizations, companies, locations, streams/projects, offers, chat, files, discovery, and proposal flows
• AWS Terraform deployment foundation

Roadmap / integration direction:

• richer assessment and offer-readiness workflows
• vendor, pricing, and outcome capture for the learning loop
• deeper compliance and logistics gates
• stable tool/API layer for internal agents and future MCP/portal interfaces
• deeper CRM and marketplace synchronization after verified stream data exists
• production observability and security hardening
• stronger public demo flow and hosted portfolio walkthrough
• expanded knowledge-base/retrieval layer for accumulated stream intelligence

Notes for reviewers

This repository is in active development. Some internal names still reflect earlier prototypes or implementation terms:

• Product name: SecondStream / Second Stream
• Product entity: secondary stream or stream
• Current backend anchor model: Project
• Older/stale references may mention waste, DSR Waste Platform, H2O Allegiant, or water-treatment concepts

The public README intentionally uses the current product narrative while preserving implementation accuracy.