Skip to content

Fix unit test timeouts and other failures.#2903

Merged
brandonpage merged 1 commit into
forcedotcom:devfrom
brandonpage:fix-mockk-test-timeouts
May 26, 2026
Merged

Fix unit test timeouts and other failures.#2903
brandonpage merged 1 commit into
forcedotcom:devfrom
brandonpage:fix-mockk-test-timeouts

Conversation

@brandonpage
Copy link
Copy Markdown
Contributor

Summary

Fix screenLockActivity_onAuthError_sendsAccessibilityEvent and several other instrumentation tests that were timing out or failing on Firebase Test Lab.

Root cause

mockk's Android dispatcher uses dexmaker to bytecode-instrument any class whose instances are mocked, including the entire ancestor chain. Once a test calls mockk<LoginActivity>(relaxed = true) (or any other Activity-derived type) in a given JVM process, every subsequent ContextThemeWrapper.getResources() call in that process is routed through io.mockk.proxy.android.advice.Advice.findMethod — even after unmockkAll().

Compose calls getResources() thousands of times per frame. With mockk's reflection on the hot path, single Compose frames take 8+ seconds. Tests like screenLockActivity_onAuthError_sendsAccessibilityEvent that launch a Compose Activity then ANR on AuthenticatorService bind, while others (tokenMigrationView_*, screenLockView_*) silently slow from ~2 s to ~30 s per test until the shard hits its timeout.

Confirmed via JVM main-thread stack from a timed-out run:

io.mockk.proxy.android.advice.Advice$Companion.findMethod(Advice.kt:171)
io.mockk.proxy.android.AndroidMockKDispatcher.getOrigin(...)
android.view.ContextThemeWrapper.getResources(Unknown Source:14)
androidx.core.content.ContextCompat.getColor(...)
com.salesforce.androidsdk.ui.theme.SFColors.primaryColor(SFColors.kt:72)
androidx.compose.ui.platform.ComposeView.Content(...)

The pollution is irreversible within a JVM lifetime, so the fix has to keep Activity-mocking tests and Compose-rendering tests in separate test processes (separate Firebase shards).

Changes

Test isolation

  • BiometricAuthenticationCallbackTest.kt (new) — extracted three mockk<ScreenLockActivity>(relaxed = true) tests out of ScreenLockActivityScenarioTest. They tested an inner class via the outer Activity mock and were poisoning the rest of the class once they ran.
  • ScreenLockActivityScenarioTest.kt — removed those three tests and unused imports. The remaining 29 Compose-driven cases now run unaffected.
  • .github/test-shards/SalesforceSDK.json:
    • Compose-rendering classes (ScreenLockActivityScenarioTest, ScreenLockViewTest, TokenMigrationViewActivityTest) added to the ui shard.
    • TokenMigrationWebViewTest moved to a new webview shard (slow on the emulator for unrelated WebView reasons; isolating it keeps it from eating budget elsewhere).
    • The mockk-of-Activity polluters (LoginActivityTest, NativeLoginManagerTest, SalesforceSDKManagerTests, BiometricAuthenticationCallbackTest) stay in remaining.
    • Comment in the file documents the rule so future test authors know not to mix the two categories.

Test fixes

  • TokenMigrationWebViewTest.kt — deleted shouldOverrideUrlLoading_returnsTrueForCallbackUrl_userAgentFlow. PR Fix Token Migration silent failure. #2892 collapsed the useWebServerFlow branch in the production code; the test was verifying a path that no longer exists.
  • AuthConfigUtilTest.java — broadcast tests (testBroadcastSucceeds, testBroadcastFails) now register their BroadcastReceiver on a HandlerThread instead of the implicit main thread. Other tests in the same shard launch Activities, and the cumulative main-looper backlog could prevent the receiver's onReceive from running before the test timeout, even though the broadcast had already been dispatched. Also widened the JUnit @Test(timeout=...) from 5 s to 30 s and added an explicit Future.get(20, SECONDS) for a clearer failure mode.

Test plan

  • screenLockActivity_onAuthError_sendsAccessibilityEvent passes
  • All 29 ScreenLockActivityScenarioTest cases pass (~2 s each)
  • All 4 ScreenLockViewTest cases pass (~2.5 s each, down from 27 s)
  • All 3 extracted BiometricAuthenticationCallbackTest cases pass
  • testBroadcastSucceeds / testBroadcastFails pass (~2.5 s each)
  • Shard config validator (CLASSES == NOT_CLASSES) passes

Couple of usage notes for the PR description:

  • Untracked: run_firebase_tests.sh is a local helper — not part of the PR. Either gitignore or delete before pushing.
  • external/shared: pre-existing dirty submodule pointer, unrelated.

{
"name": "ui",
"comment": "Tests that exercise Activities/UI",
"comment": "Compose-rendering tests must NOT share a shard with classes that call mockk<...>(relaxed = true) on Activity-derived types (LoginActivity, FragmentActivity, etc.). mockk's Android dispatcher bytecode-instruments those class hierarchies for the JVM process lifetime, which slows ContextThemeWrapper.getResources() to seconds per call — hanging Compose rendering. The current polluters are LoginActivityTest, NativeLoginManagerTest, and SalesforceSDKManagerTests, all kept in the `remaining` shard.",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will admit that using shards to separate tests that mock activities and compose tests feels more like a workaround than a real solution, but I think it is the best we can do today. Completely re-writing many tests to not mock LoginActivity seems like it will be an enormous amount of work and will likely reduce our overall coverage.

This leaves us with the (existing) issue that there isn't a trivial way to run all tests locally. I have a script I created to test these changes that I could add to the PR. This would let us easily run tests like CI does. Let me know how you feel about that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your analysis short and long term. This looks like a practical way to improve our current test inventory and document future needs as well.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.45%. Comparing base (b9411ff) to head (097edfd).
⚠️ Report is 10 commits behind head on dev.

Additional details and impacted files
@@              Coverage Diff              @@
##                dev    #2903       +/-   ##
=============================================
+ Coverage     55.17%   66.45%   +11.27%     
- Complexity     2509     3100      +591     
=============================================
  Files           226      226               
  Lines         17781    17774        -7     
  Branches       2328     2327        -1     
=============================================
+ Hits           9810    11811     +2001     
+ Misses         6966     4886     -2080     
- Partials       1005     1077       +72     
Components Coverage Δ
Analytics 48.71% <ø> (ø)
SalesforceSDK 62.60% <100.00%> (+20.69%) ⬆️
Hybrid 59.30% <ø> (ø)
SmartStore 78.22% <ø> (ø)
MobileSync 82.12% <ø> (ø)
React 51.50% <ø> (ø)
see 60 files with indirect coverage changes
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@brandonpage
Copy link
Copy Markdown
Contributor Author

All SalesforceSDK lib test pass! (which was the goal)

Copy link
Copy Markdown
Contributor

@JohnsonEricAtSalesforce JohnsonEricAtSalesforce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 Great research on this one, @brandonpage ⭐️

@brandonpage brandonpage merged commit b2af85e into forcedotcom:dev May 26, 2026
33 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants