Skip to content

Commit a68e121

Browse files
committed
Simplify workflow review flow and refresh docs
1 parent c7480b3 commit a68e121

27 files changed

Lines changed: 633 additions & 588 deletions

DEVLOG.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,6 +258,48 @@ Persistent per-user usage storage, saved artifact history, and quotas are intent
258258
- Added `src/usage_store.py` to persist assisted usage events in Supabase Postgres for authenticated users.
259259
- Extended `src/openai_service.py` with an optional usage-event callback so persistence stays transport-agnostic.
260260
- Wired authenticated usage-event recording from `src/ui/workflow.py` without leaking Streamlit concerns into the service layer.
261+
262+
## Day 18: Single-Pass Review-Correction Workflow
263+
264+
- Removed the live `ProfileAgent` and `JobAgent` stages from the supervised workflow because they were mostly restating deterministic inputs without adding enough value for the latency cost.
265+
- Simplified the active orchestrator path to:
266+
- fit
267+
- tailoring
268+
- strategy
269+
- review
270+
- resume generation
271+
- Removed the bounded rerun loop that previously sent tailoring and strategy back through another full pass after review.
272+
- Changed Review so it can return direct corrections for tailoring and strategy, and the orchestrator now feeds those corrected outputs straight into final resume generation.
273+
- Removed interview-theme style outputs that were adding contract size without being core to the current product output.
274+
- Updated the UI, payload layer, and report rendering so they reflect the smaller workflow and the direct-correction review model.
275+
- Verified the redesign with focused workflow, prompt, builder, and UI test coverage.
276+
277+
## Day 19: Model Routing And Output Budget Tuning
278+
279+
- Rebalanced reasoning effort by task based on real runtime logs instead of keeping one default posture for every agent.
280+
- Changed the active routing defaults to:
281+
- `fit`: `gpt-5-mini-2025-08-07` with `low` reasoning
282+
- `tailoring`: `gpt-5-mini-2025-08-07` with `medium` reasoning
283+
- `strategy`: `gpt-5-mini-2025-08-07` with `low` reasoning
284+
- `review`: `gpt-5.4` with `medium` reasoning
285+
- `resume_generation`: `gpt-5.4` with `medium` reasoning
286+
- Increased the Review output budget to start at 4000 tokens so the stage does not immediately fall into retry-on-truncation for corrected JSON payloads.
287+
- Reduced oversized output caps where observed usage made the previous limits unnecessary:
288+
- `fit`: 1600
289+
- `strategy`: 1500
290+
- `resume_generation`: 3000
291+
- Kept `tailoring` at 3200 and `review` at 4000 because they still carry the heaviest grounded payloads in the current flow.
292+
- Verified the new routing and cap defaults with targeted orchestration and OpenAI-service tests.
293+
294+
## Day 20: Review Approval Semantics And Backward Compatibility
295+
296+
- Clarified Review semantics so `approved` now means the final corrected output is safe to use, not that the incoming tailoring or strategy draft was perfect before correction.
297+
- Added `unresolved_issues` to the review contract so the app can distinguish between:
298+
- issues found in the incoming draft
299+
- blockers that still remain after correction
300+
- Updated UI and report labels to show `Approved After Corrections` when Review repaired the output successfully.
301+
- Added backward-compatible access patterns so older saved or in-memory `ReviewAgentOutput` objects without `unresolved_issues` do not crash the app.
302+
- Logged PDF-output quality as a follow-up documentation item because export aesthetics still need a dedicated pass even though workflow runtime is now much healthier.
261303
- Added the `usage_events` SQL schema and RLS policies in `docs/supabase-usage-events.sql`.
262304
- Kept assisted requests resilient: usage persistence failures are logged but do not break the user-facing AI response.
263305

README.md

Lines changed: 19 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -25,17 +25,15 @@ The repository now follows the same product-style structure as the GitHub Portfo
2525
- Generate a deterministic fit snapshot against the active job description
2626
- Produce first-pass resume-tailoring guidance from grounded profile and JD signals
2727
- Run a supervised specialist-agent workflow on demand:
28-
- profile
29-
- job
3028
- fit
3129
- tailoring
3230
- strategy
3331
- review
3432
- resume generation
35-
- Run a bounded review-revision loop before the final resume artifact is generated
33+
- Let the review stage directly correct tailoring and strategy outputs before final resume generation instead of rerunning the full workflow loop
3634
- Use OpenAI when configured, with deterministic fallback when it is not
3735
- Route different assisted tasks to different model tiers instead of relying on one global model
38-
- Route GPT-5 reasoning effort by task, with medium effort for normal tasks and high effort for review, resume generation, and grounded application Q&A
36+
- Route GPT-5 reasoning effort by task, with low effort on fit and strategy, medium effort on tailoring, review, and final resume generation, and higher-trust routing kept for the final grounding stages
3937
- Use the OpenAI Responses API for assisted JSON generation and usage tracking
4038
- Build a tailored resume artifact from the current workflow state
4139
- Preview the tailored resume directly in the app before export
@@ -55,7 +53,7 @@ The repository now follows the same product-style structure as the GitHub Portfo
5553

5654
## Current Status
5755

58-
The app is still an MVP, but it is now a coherent authenticated workflow product rather than only a deterministic prototype. Resume parsing, JD structuring, deterministic fit analysis, supervised specialist-agent orchestration, bounded review-driven revision, tailored resume generation, report generation, preview-before-download flows, export packaging, model-aware assisted routing, grounded in-app assistance, Google sign-in, persisted usage tracking, daily quotas, and 24-hour saved workspace reloads are all working.
56+
The app is still an MVP, but it is now a coherent authenticated workflow product rather than only a deterministic prototype. Resume parsing, JD structuring, deterministic fit analysis, supervised specialist-agent orchestration, direct review-driven correction, tailored resume generation, report generation, preview-before-download flows, export packaging, model-aware assisted routing, grounded in-app assistance, Google sign-in, persisted usage tracking, daily quotas, and 24-hour saved workspace reloads are all working.
5957

6058
The active product scope is intentionally focused:
6159

@@ -212,6 +210,22 @@ The current OpenAI Responses API integration also includes runtime safeguards fo
212210
- one retry with a higher output-token budget when a response is incomplete because the original output budget was exhausted
213211
- longer client timeouts plus SDK retries to reduce transient read-timeout failures
214212

213+
Current default assisted routing is intentionally asymmetric:
214+
215+
- `fit`: GPT-5 Mini with `low` reasoning
216+
- `tailoring`: GPT-5 Mini with `medium` reasoning
217+
- `strategy`: GPT-5 Mini with `low` reasoning
218+
- `review`: GPT-5.4 with `medium` reasoning
219+
- `resume_generation`: GPT-5.4 with `medium` reasoning
220+
221+
Current default output-token caps are also tuned by task rather than kept uniform:
222+
223+
- `fit`: 1600
224+
- `tailoring`: 3200
225+
- `strategy`: 1500
226+
- `review`: 4000
227+
- `resume_generation`: 3000
228+
215229
The current app also does not require Supabase for a first hosted deploy. If Supabase is not configured yet, the app can still run the non-authenticated product shell and deterministic workflow. Google sign-in, saved-workspace reloads, and account-level quotas remain inactive until Supabase is configured.
216230

217231
If you plan to deploy before creating Supabase, set `AUTH_REQUIRED_FOR_ASSISTED_WORKFLOW=false` so the AI-assisted workflow button is not blocked by the missing login layer. Once Supabase exists, turn it back on if you want assisted usage tied to authenticated accounts.
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# ADR-010: Single-Pass Review Corrections and Task-Tuned Model Budgets
2+
3+
## Status
4+
5+
Accepted
6+
7+
## Context
8+
9+
The earlier supervised workflow had grown into a more expensive sequence than the product needed:
10+
11+
- `ProfileAgent` and `JobAgent` were generating summaries that mostly restated deterministic candidate and JD structure already available elsewhere in the system
12+
- the orchestrator ran a bounded revision loop that resent tailoring, strategy, and review through another full pass when review rejected the first draft
13+
- Review was functioning as both a gate and a revision trigger, but the product goal had shifted toward direct grounded correction rather than repeated full-pipeline iteration
14+
- real runtime logs showed that the second tailoring + strategy + review pass was one of the largest contributors to total latency
15+
- the final high-trust stages were worth more model budget than the early summarization stages, but the routing defaults had not yet been tightened around that reality
16+
17+
At the same time, the product still needed:
18+
19+
- deterministic fit analysis as the grounding backbone
20+
- a strong Tailoring step because that stage carries the most content-heavy rewrite work
21+
- a strong Review step that can reject or repair unsupported wording
22+
- a final Resume Generation step that turns the reviewed output into the export-ready artifact
23+
24+
## Decision
25+
26+
Adopt a single-pass supervised workflow with direct review corrections and task-tuned reasoning / output budgets.
27+
28+
The accepted workflow is:
29+
30+
1. `fit`
31+
2. `tailoring`
32+
3. `strategy`
33+
4. `review`
34+
5. `resume_generation`
35+
36+
Implementation details:
37+
38+
1. remove live `ProfileAgent` and `JobAgent` execution from the active orchestrator path
39+
2. keep deterministic `CandidateProfile`, `JobDescription`, `FitAnalysis`, and `TailoredResumeDraft` as the source-of-truth inputs
40+
3. make Review return direct corrected tailoring and strategy outputs when repairs are straightforward
41+
4. stop rerunning the entire tailoring / strategy / review chain after review feedback
42+
5. define review approval in terms of the final corrected state, not only the cleanliness of the incoming draft
43+
6. route earlier cheaper stages to cheaper reasoning levels than the final grounding stages
44+
7. tune output-token caps by observed usage instead of using one oversized default for every task
45+
46+
The current routing defaults following this decision are:
47+
48+
- `fit`: `gpt-5-mini-2025-08-07` with `low` reasoning and a 1600-token output cap
49+
- `tailoring`: `gpt-5-mini-2025-08-07` with `medium` reasoning and a 3200-token output cap
50+
- `strategy`: `gpt-5-mini-2025-08-07` with `low` reasoning and a 1500-token output cap
51+
- `review`: `gpt-5.4` with `medium` reasoning and a 4000-token output cap
52+
- `resume_generation`: `gpt-5.4` with `medium` reasoning and a 3000-token output cap
53+
54+
## Alternatives Considered
55+
56+
### 1. Keep the full Profile + Job + Fit + Tailoring + Strategy + Review + Resume Generation stack
57+
58+
Rejected because Profile and Job were not adding enough unique value relative to the deterministic data they were summarizing, while still costing additional sequential model latency.
59+
60+
### 2. Keep the revision loop but reduce model size only
61+
62+
Rejected because the largest avoidable cost was architectural: repeated full-pipeline passes. Model tuning alone would not remove that structural latency.
63+
64+
### 3. Remove Review entirely and trust Tailoring / Strategy output directly
65+
66+
Rejected because Review is still the main grounding defense against unsupported claims, inferred tenure, and overstated tooling experience.
67+
68+
### 4. Lower every stage to the same cheapest reasoning tier
69+
70+
Rejected because the stages do not have the same risk profile. Review and final resume generation justify more careful reasoning than early fit and strategy summarization.
71+
72+
## Consequences
73+
74+
- the workflow becomes materially faster because it removes redundant live stages and the second-pass loop
75+
- deterministic inputs remain the grounding backbone even though the live agent count is smaller
76+
- Review becomes a direct correcting editor rather than only a rejection gate
77+
- the meaning of `approved` must reflect the final corrected output state, which required updates to UI and report wording
78+
- model routing becomes easier to reason about because costlier reasoning is reserved for the stages that materially affect grounding and final export quality
79+
- PDF output quality remains a separate follow-up concern; the workflow and routing changes improve runtime and correctness, but they do not solve visual export polish by themselves

docs/adr/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ This directory tracks the architectural decisions that shape the AI Job Applicat
1111
- [ADR-007: Remove LinkedIn import from active product scope](ADR-007-remove-linkedin-import-from-active-product-scope.md)
1212
- [ADR-008: Two-mode grounded assistant panel](ADR-008-two-mode-grounded-assistant-panel.md)
1313
- [ADR-009: Google sign-in via Supabase for persistent identity](ADR-009-google-sign-in-via-supabase-for-persistent-identity.md)
14+
- [ADR-010: Single-pass review corrections and task-tuned model budgets](ADR-010-single-pass-review-corrections-and-task-tuned-model-budgets.md)
1415

1516
## Superseded
1617

docs/recommended-changes-status-2026-03-16.md

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -249,18 +249,26 @@ Checkpoint:
249249

250250
## 18. Revision Loop
251251

252-
Status: `Already implemented earlier`
252+
Status: `Superseded by later workflow simplification`
253253

254-
Changes present in the current codebase:
254+
Earlier state:
255+
256+
- `ApplicationOrchestrator` previously reran tailoring, strategy, and review in a bounded revision loop
257+
- review feedback was injected back into `TailoringAgent.run(...)` as `revision_requests`
258+
- revision pass history was preserved on `review_history`
259+
- the loop was capped by `max_revision_passes`
260+
261+
Current state:
255262

256-
- `ApplicationOrchestrator` reruns tailoring, strategy, and review in a bounded revision loop
257-
- review feedback is injected back into `TailoringAgent.run(...)` as `revision_requests`
258-
- revision pass history is preserved on `review_history`
259-
- the loop is capped by `max_revision_passes`
263+
- the bounded rerun loop was removed in favor of one single-pass workflow
264+
- Review now applies direct corrections to tailoring and strategy outputs instead of sending the whole flow through another pass
265+
- `review_history` remains only as a compatibility field, not as an active revision-loop record for the current live flow
260266

261267
Current evidence:
262268

263269
- `src/agents/orchestrator.py`
270+
- `src/agents/review_agent.py`
271+
- `docs/adr/ADR-010-single-pass-review-corrections-and-task-tuned-model-budgets.md`
264272

265273
## 19. Application Strategy Agent
266274

@@ -269,7 +277,7 @@ Status: `Already implemented earlier`
269277
Changes present in the current codebase:
270278

271279
- `StrategyAgent` exists as a first-class agent under `src/agents/strategy_agent.py`
272-
- the orchestrator runs it on each revision pass
280+
- the orchestrator runs it once in the active single-pass workflow
273281
- its output is included in workflow payloads, UI rendering, report generation, and resume generation context
274282

275283
Current evidence:
@@ -484,6 +492,7 @@ Current state:
484492

485493
These are the next practical checks to run in the app after the Supabase bootstrap update.
486494

495+
0. Review the generated PDF outputs themselves and improve the visual/layout quality, because the current exported documents still look off even when the workflow data and runtime are behaving correctly.
487496
1. Sign in with a normal non-internal account and confirm the daily quota panel renders without warnings or silent fallback.
488497
2. Verify the saved workspace flow still works normally after the updated bootstrap SQL, including reload and download regeneration.
489498
3. Do one final spot-check with a normal non-internal account so the persisted quota panel, assisted run, and post-run quota refresh all behave correctly end to end.

src/agents/fit_agent.py

Lines changed: 5 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,7 @@
44
CandidateProfile,
55
FitAnalysis,
66
FitAgentOutput,
7-
JobAgentOutput,
87
JobDescription,
9-
ProfileAgentOutput,
108
)
119

1210
from .common import coerce_string, coerce_string_list, unique_strings
@@ -21,16 +19,12 @@ def run(
2119
candidate_profile: CandidateProfile,
2220
job_description: JobDescription,
2321
fit_analysis: FitAnalysis,
24-
profile_output: ProfileAgentOutput,
25-
job_output: JobAgentOutput,
2622
) -> FitAgentOutput:
2723
if self._openai_service and self._openai_service.is_available():
2824
prompt = build_fit_agent_prompt(
2925
candidate_profile,
3026
job_description,
3127
fit_analysis,
32-
profile_output,
33-
job_output,
3428
)
3529
payload = self._openai_service.run_json_prompt(
3630
prompt["system"],
@@ -44,15 +38,14 @@ def run(
4438
fit_summary=coerce_string(payload.get("fit_summary")),
4539
top_matches=coerce_string_list(payload.get("top_matches"), limit=4),
4640
key_gaps=coerce_string_list(payload.get("key_gaps"), limit=4),
47-
interview_themes=coerce_string_list(payload.get("interview_themes"), limit=4),
4841
)
49-
return self._fallback(fit_analysis, profile_output, job_output)
42+
return self._fallback(fit_analysis, candidate_profile, job_description)
5043

5144
@staticmethod
5245
def _fallback(
5346
fit_analysis: FitAnalysis,
54-
profile_output: ProfileAgentOutput,
55-
job_output: JobAgentOutput,
47+
candidate_profile: CandidateProfile,
48+
job_description: JobDescription,
5649
) -> FitAgentOutput:
5750
fit_summary = (
5851
"{label} for {role} with a score of {score}/100. {experience}".format(
@@ -63,28 +56,16 @@ def _fallback(
6356
)
6457
)
6558
top_matches = unique_strings(
66-
fit_analysis.strengths + profile_output.evidence_highlights + job_output.priority_skills,
59+
fit_analysis.strengths + fit_analysis.matched_hard_skills + candidate_profile.skills,
6760
limit=4,
6861
)
6962
key_gaps = unique_strings(
70-
fit_analysis.gaps + fit_analysis.missing_hard_skills + profile_output.cautions,
63+
fit_analysis.gaps + fit_analysis.missing_hard_skills + fit_analysis.missing_soft_skills,
7164
limit=4,
7265
)
73-
interview_themes = []
74-
if fit_analysis.matched_hard_skills:
75-
interview_themes.append(
76-
"Prepare concrete stories around " + ", ".join(fit_analysis.matched_hard_skills[:3]) + "."
77-
)
78-
if fit_analysis.missing_hard_skills:
79-
interview_themes.append(
80-
"Frame a credible upskilling plan for " + ", ".join(fit_analysis.missing_hard_skills[:3]) + "."
81-
)
82-
if not interview_themes:
83-
interview_themes.append("Prepare outcome-focused examples from your strongest recent work.")
8466

8567
return FitAgentOutput(
8668
fit_summary=fit_summary,
8769
top_matches=top_matches,
8870
key_gaps=key_gaps,
89-
interview_themes=unique_strings(interview_themes, limit=4),
9071
)

0 commit comments

Comments
 (0)