Skip to content

rh: add REST endpoint that triggers DHCR form submission [sc-14556]#2510

Draft
aidencullo wants to merge 9 commits into
JustFixNYC:masterfrom
aidencullo:rh-rest-endpoints
Draft

rh: add REST endpoint that triggers DHCR form submission [sc-14556]#2510
aidencullo wants to merge 9 commits into
JustFixNYC:masterfrom
aidencullo:rh-rest-endpoints

Conversation

@aidencullo

@aidencullo aidencullo commented May 6, 2026

Copy link
Copy Markdown
Collaborator

Shortcut: sc-14556

Summary

  • Adds POST /rh/submit — validates RhForm, creates a RentalHistoryRequest, returns the row id.
  • Wires path("rh/", include("rh.urls")) into project/urls.py.
  • Persists fields that ModelForm.save() was silently dropping (phone, address, borough, zipcode, address_verified) because RhForm.Meta.fields only declares three.
  • Adds an opt-in Playwright submitter (rh/dhcr_portal_submit.py) for the eventual DHCR portal channel referenced in sc-14556. Dry-run by default (DHCR_PORTAL_DRY_RUN=1); no live submission happens without HCR sign-off.
  • Extracts the HCR reference number (#YYMMDD-NNNNNN) from the post-submit success page and returns it on SubmissionResult.reference_number for the caller to persist.
  • Adds a manage.py test_dhcr_portal_empty smoke command that opens portal.hcr.ny.gov/app/ask, clicks Submit on an empty form, and prints the server-side validation banner — confirms reachability without creating a record.

How it works at a high level

  1. External caller POSTs tenant info as JSON to /rh/submit → Django saves a RentalHistoryRequest row, returns its id.
  2. (Separate, gated, not yet wired into the view) The Playwright bot launches headless Chromium, fills HCR's form at portal.hcr.ny.gov/app/ask, clicks Submit, scrapes the reference number off the success page.

The end-to-end Playwright pipeline (fill → submit → success-banner wait → body scrape → regex match r"#(\\d{6}-\\d{6})") has been validated against a real HCR submission and successfully extracted a live reference.

Test plan

  • curl -X POST :8000/rh/submit with no body → 400 with {"errors": {...}}.
  • curl -X POST :8000/rh/submit -H 'Content-Type: application/json' -d '{...full form...}' → 201 with {"id": <pk>}, row appears in rh_rentalhistoryrequest with all fields persisted.
  • (manual) pipenv run python manage.py test_dhcr_portal_empty after installing playwright + chromium → reports validation banner, no record created at HCR.

Not in this PR

  • Bearer-token auth on /rh/submit — the sibling REST endpoints (/gce/upload, /efnyc/upload, /mailchimp/subscribe) gate access with authorize_with_token(...) + a CORS allowlist. /rh/submit should follow that pattern before going public.
  • playwright is not added to Pipfile; the management command + portal submitter require an ad-hoc install. Production use needs Chromium baked into the Dockerfile.
  • No dhcr_reference_number / portal_submitted_at columns on RentalHistoryRequest yet — needed before wiring submit_via_portal into the /rh/submit view so the reference can be persisted.
  • No HCR authorization to switch the production channel from email to portal submission. DHCR_PORTAL_DRY_RUN=0 must not be flipped until that's in writing.

🤖 Generated with Claude Code

aidencullo and others added 3 commits May 6, 2026 09:55
Adds /rh/submit and /rh/send-email POST endpoints so the website and
TextIt flow can post rent history requests over HTTP, alongside the
existing GraphQL mutations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
views.py:
  - submit() now copies phone_number, address, borough, zipcode, and
    address_verified from RhForm.cleaned_data onto the model. RhForm
    only listed first_name/last_name/apartment_number in Meta.fields,
    so ModelForm.save() was silently dropping the rest.
  - 404 -> 400 for "id not found" in send-email to dodge Django's
    LocaleMiddleware redirect-on-404 turning the response into a 302.

dhcr_portal_submit.py (new):
  - Stub Playwright submission to portal.hcr.ny.gov/app/ask, the HCR
    public inquiry portal.
  - DRY-RUN BY DEFAULT (DHCR_PORTAL_DRY_RUN=1). Fills the form but
    does not click Submit. Flip to "0" for live submission once HCR
    has confirmed third-party programmatic access.
  - Returns a structured SubmissionResult so callers can record
    success/error on the row.

Notes for review:
  - Selectors are best-guess against a Power Pages portal; verify
    against live DOM before flipping DRY_RUN.
  - Not yet wired into any view. Caller integration deferred until
    we add a submission-status column on RentalHistoryRequest.
  - Adds a runtime dependency on playwright (not yet in Pipfile).
`pipenv run python manage.py test_dhcr_portal_empty` drives the
portal.hcr.ny.gov/app/ask form in headless Chromium, clicks SUBMIT
without filling any fields, and prints the server-side validation
banner that comes back. Confirms reachability + selector hygiene
without ever creating a real DHCR record.

Requires playwright + chromium in the image; cleanly errors with a
guidance message when missing so it doesn't break existing runs.
@aidencullo aidencullo changed the title rh: add REST endpoints for submit and send-email rh: add REST endpoints for submit and send-email [sc-14556] May 11, 2026
aidencullo and others added 2 commits May 17, 2026 19:40
sc-14556 is a single endpoint to trigger DHCR form submission. The
parallel send-email REST endpoint duplicates the existing GraphQL
RhSendEmail mutation and isn't part of the new portal path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After a successful portal submission, HCR's success page shows
"Use this reference number for follow up: #YYMMDD-NNNNNN."
SubmissionResult now exposes that as `reference_number` so the
caller can persist it on the RentalHistoryRequest row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@aidencullo aidencullo changed the title rh: add REST endpoints for submit and send-email [sc-14556] rh: add REST endpoint that triggers DHCR form submission [sc-14556] May 18, 2026
@aidencullo aidencullo force-pushed the rh-rest-endpoints branch from df20f9e to e19ced7 Compare May 18, 2026 14:00
aidencullo and others added 2 commits May 18, 2026 10:04
Sequence: validate form -> save row -> trigger Playwright submission.
Portal call is dry-run by default (DHCR_PORTAL_DRY_RUN=1), so no
traffic to HCR until that flag is explicitly flipped. Response now
includes portal.{success, dry_run, reference_number, error} so the
caller can record the HCR reference.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
submit_via_portal() promised "never raises on portal failures" but
pw.chromium.launch() sat before the try block, so a launch failure
(e.g. missing Chromium binary) escaped the function and turned
POST /rh/submit into a 500 -- after a row was already saved. Move
launch/context/page setup inside the try and guard the finally so the
failure is captured as portal.error and the endpoint returns 201.

Add rh/tests/test_views.py covering the view (400 on missing field,
201 + saved row on valid input, portal called with the saved request)
and the regression (launch failure returns a SubmissionResult, never
raises) plus reference-number extraction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@aidencullo

Copy link
Copy Markdown
Collaborator Author

stale

@aidencullo aidencullo closed this May 25, 2026
@aidencullo aidencullo reopened this May 26, 2026
@aidencullo

Copy link
Copy Markdown
Collaborator Author

Update since the original summary:

The PR now wires submit_via_portal() into POST /rh/submit. The endpoint creates a RentalHistoryRequest, calls the portal submitter, returns portal status info, and persists portal_result.reference_number to a new dhcr_reference_number column when present.

Also added:

  • RentalHistoryRequest.dhcr_reference_number + migration
  • admin list display for dhcr_reference_number
  • local/debug GET /rh/requests endpoint, guarded by DEBUG
  • tests for reference-number persistence and local request listing

Still not included:

  • bearer-token auth / CORS allowlist for public use
  • Playwright/Chromium baked into project dependencies or production image on this branch
  • HCR sign-off to disable dry-run in production

Important local behavior:
POST /rh/submit can return 201 even when the portal step fails, because the DB row was created. Check the portal object in the JSON response for success, dry_run, reference_number, and error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant