feat: replace dataclasses-json with Pydantic v2 models by layertwo · Pull Request #277 · layertwo/ffsync

layertwo · 2026-03-05T05:23:17Z

Summary

Migrate all route responses to Pydantic v2 output models (BSOOutput, ModifiedOutput, BatchResultOutput, AccountCreateOutput, ProfileOutput, TokenOutput, etc.) across 21 routes
Replace dataclasses-json DataClassJsonMixin with Pydantic BaseModel for DynamoDB models (BasicStorageObject, CollectionData, UserRecord) via a DynamoModel base class that handles Decimal/datetime coercion
Add to_item() methods on DynamoDB models for complete item serialization (replaces _encode_* methods in StorageManager/UserManager)
Remove dataclasses-json dependency and all dead helpers (DecimalEncoder, json_dumps, datetime_encoder/decoder, float_to_decimal, validate_sortindex, validate_ttl)
Enable Powertools enable_validation=True on all API routers with RequestValidationError/ResponseValidationError handlers

Test plan

All 993 tests pass with 100% coverage
black, isort, flake8, mypy all pass
dataclasses-json fully removed from dependencies (+ transitive marshmallow, typing-inspect)
Deploy to dev and verify Firefox Sync end-to-end (bookmarks, tabs, history)

Add pydantic dependency and enable_validation parameter on ApiRouter. All 4 routers now have enable_validation=True with RequestValidationError and ResponseValidationError exception handlers.

…s-json Replace dataclasses-json with Pydantic v2 across the entire codebase: - Convert BasicStorageObject, CollectionData, BatchResult to Pydantic BaseModel with DynamoModel base class handling Decimal/datetime coercion - Add to_item() methods on DynamoDB models for complete item serialization - Migrate all 21 route outputs to Pydantic models (BSOOutput, ModifiedOutput, BatchResultOutput, AccountCreateOutput, ProfileOutput, TokenOutput, etc.) - Add BSOOutput.from_bso() factory for BSO→output conversion - Remove dataclasses-json dependency, DecimalEncoder, json_dumps, and all datetime_encoder/decoder/float_to_decimal helpers - Convert OIDCTokenClaims, OIDCProviderConfig, ErrorDetail, TokenResponse to plain dataclasses (no serialization mixin needed) - Convert UserRecord to DynamoModel with to_item() for DynamoDB writes

github-actions · 2026-03-05T05:26:32Z

Diff for stage: DefaultStage

Warning

3 Destructive Changes

Diff for stack: GitHubOidcStack - 0 to add, 1 to update, 0 to destroy 🟡

Details

Resources
[~] Custom::AWSCDKOpenIdConnectProvider GitHubOidcProvider7EBF861F
 ├─ [~] CodeHash
 │   ├─ [-] 62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb
 │   └─ [+] d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50
 └─ [~] RejectUnauthorized
[~] AWS::Lambda::Function CustomAWSCDKOpenIdConnectProviderCustomResourceProviderHandlerF2C543E0
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb.zip
 │       └─ [+] d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.62fa02efcaa700e1c247e1d3cc2aa0cd07a0808a9a3e3d2230e51f57a02233fb
         └─ [+] asset.d75c48c9f82cde63e9bf414df335e84e8ac24f11eb34889be255b702aec71e50

Diff for stack: Service-prod - 3 to add, 3 to update, 3 to destroy ❌

Details

[!WARNING]
Destructive Changes ‼️
Stack: Service-prod - Resource: AuthApiDeploymentB62B2E46e6df84b588a887363a13ab29adc3623c - Impact: WILL_DESTROY

Stack: Service-prod - Resource: TokenApiDeploymentB896C219b16a73c2bacf13dff150a851137e236c - Impact: WILL_DESTROY

Stack: Service-prod - Resource: ProfileApiDeployment84A54415e0158c62574d24c31b8dbd21e82873c4 - Impact: WILL_DESTROY

Resources
[-] AWS::ApiGateway::Deployment AuthApiDeploymentB62B2E46e6df84b588a887363a13ab29adc3623c destroy
[-] AWS::ApiGateway::Deployment TokenApiDeploymentB896C219b16a73c2bacf13dff150a851137e236c destroy
[-] AWS::ApiGateway::Deployment ProfileApiDeployment84A54415e0158c62574d24c31b8dbd21e82873c4 destroy
[+] AWS::ApiGateway::Deployment AuthApiDeploymentB62B2E46b5c04cfbd7bd117a1ee4f8acc7bb74f8
[+] AWS::ApiGateway::Deployment TokenApiDeploymentB896C2197f4fd8f40928d8ff6b65325e6f60c797
[+] AWS::ApiGateway::Deployment ProfileApiDeployment84A54415afa95090416ea0bbd4b1d2aac5a9241c
[~] AWS::Lambda::Function AuthApiHandlerED50ACFA
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147.zip
 │       └─ [+] 97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147
         └─ [+] asset.97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a
[~] AWS::Lambda::Function TokenApiHandler2E66DB25
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147.zip
 │       └─ [+] 97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147
         └─ [+] asset.97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a
[~] AWS::Lambda::Function ProfileApiHandler9B65A298
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147.zip
 │       └─ [+] 97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147
         └─ [+] asset.97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a
[~] AWS::Lambda::Function ApiHandler5E7490E8
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] 1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147.zip
 │       └─ [+] 97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.1527c534dc3ec24d621d57e297d669e22dc2ace57cbcae6d6da540d42832a147
         └─ [+] asset.97638fb87dc714ebe303cdc70246ff3fa200e74abd5b8cee8e2b4f6261ba707a
[~] AWS::ApiGateway::Stage AuthApiDeploymentStageprodB0E4172A
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] AuthApiDeploymentB62B2E46e6df84b588a887363a13ab29adc3623c
         └─ [+] AuthApiDeploymentB62B2E46b5c04cfbd7bd117a1ee4f8acc7bb74f8
[~] AWS::ApiGateway::Stage TokenApiDeploymentStageprod11035AE4
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] TokenApiDeploymentB896C219b16a73c2bacf13dff150a851137e236c
         └─ [+] TokenApiDeploymentB896C2197f4fd8f40928d8ff6b65325e6f60c797
[~] AWS::ApiGateway::Stage ProfileApiDeploymentStageprodF609D968
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] ProfileApiDeployment84A54415e0158c62574d24c31b8dbd21e82873c4
         └─ [+] ProfileApiDeployment84A54415afa95090416ea0bbd4b1d2aac5a9241c
[~] AWS::Lambda::Function ChannelApiHandler02759D57
 ├─ [~] Code
 │   └─ [~] .S3Key:
 │       ├─ [-] ca37232fdfd500adbaa06e972c23a68046efd788f7781db21a059050cda4a508.zip
 │       └─ [+] ff9d24ca18cd246f688865f3ee12d7134d22cea4e80a7e842eea35b0ba7ce347.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
         ├─ [-] asset.ca37232fdfd500adbaa06e972c23a68046efd788f7781db21a059050cda4a508
         └─ [+] asset.ff9d24ca18cd246f688865f3ee12d7134d22cea4e80a7e842eea35b0ba7ce347

No Changes for stack: Frontend-prod ✅
No Changes for stack: Monitoring-prod ✅

Generated for commit b1d465f at 2026-03-05T06:12:27.528Z

- Use Field(default=None, ...) for BSOInput fields so mypy recognizes defaults - Use model_validate() for dict/Decimal input tests instead of constructor - Add type: ignore for model_validator descriptor proxy call - Add BSOOutput.from_bso() test coverage

github-actions · 2026-03-05T05:41:28Z

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
lambda/src/environment
service_provider.py
lambda/src/routes/auth
account_attached_clients.py
account_create.py
account_devices.py
account_login.py
oauth_token.py
scoped_key_data.py
session_status.py
lambda/src/routes/bso
delete.py
read.py
update.py
lambda/src/routes/collections
create.py
delete.py
list.py
read.py
update.py
lambda/src/routes/info
read_collections.py
read_counts.py
read_quota.py
lambda/src/routes/profile
get_profile.py
lambda/src/routes/storage
delete_all.py
delete_root.py
lambda/src/services
storage_manager.py
user_manager.py
lambda/src/shared
models.py
Project Total

The report is truncated to 25 files out of 38. To see the full report, please visit the workflow summary page.

_{This report was generated by python-coverage-comment-action}

Replace json.dumps([m.model_dump() ...]) with Pydantic TypeAdapter for single-pass list serialization in device, client, and collection routes. Use a _CollectionsResponse wrapper model instead of string concatenation.

Move BSOListAdapter, DeviceListAdapter, ClientListAdapter, and CollectionsResponse from route modules into shared/models.py alongside the models they serialize.

layertwo added 2 commits March 4, 2026 17:42

feat: add pydantic, enable powertools validation on ApiRouter

78623ce

Add pydantic dependency and enable_validation parameter on ApiRouter. All 4 routers now have enable_validation=True with RequestValidationError and ResponseValidationError exception handlers.

layertwo added 3 commits March 4, 2026 21:57

refactor: use TypeAdapter and model_dump_json for list serialization

cc68806

Replace json.dumps([m.model_dump() ...]) with Pydantic TypeAdapter for single-pass list serialization in device, client, and collection routes. Use a _CollectionsResponse wrapper model instead of string concatenation.

refactor: colocate TypeAdapters and CollectionsResponse in models.py

f5a3cfc

Move BSOListAdapter, DeviceListAdapter, ClientListAdapter, and CollectionsResponse from route modules into shared/models.py alongside the models they serialize.

feat: add ScopedKeyDataEntry model for scoped-key-data response

b1d465f

layertwo force-pushed the feat/pydantic-models branch from cdd893a to b1d465f Compare March 5, 2026 06:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: replace dataclasses-json with Pydantic v2 models#277

feat: replace dataclasses-json with Pydantic v2 models#277
layertwo wants to merge 6 commits intomainlinefrom
feat/pydantic-models

layertwo commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

layertwo commented Mar 5, 2026

Summary

Test plan

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diff for stage: DefaultStage

Diff for stack: GitHubOidcStack - 0 to add, 1 to update, 0 to destroy 🟡

Diff for stack: Service-prod - 3 to add, 3 to update, 3 to destroy ❌

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot commented Mar 5, 2026 •

edited

Loading

github-actions bot commented Mar 5, 2026 •

edited

Loading