Cache-bust dynamic configuration files and theme CSS #3993
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
DSpace Angular's production server can modify the configuration it serves to CSR-only clients through YAML or environment variables. However, these files can remain cached in the browser and leave users with out-of-date configuration until the TTL runs out or the user does a "hard refresh". On build time this sort of problem is solved by saving the content hash as part of the path of each file (JS, CSS, ...) We introduce HashedFileMapping to bridge the same gap for dynamic content generated _after_ the server is built: - Files added to this mapping on the server are copied to a hashed path - A copy is injected into index.html, where clients can read it out and resolve the hashed paths - If a given path is not found in the mapping, the client will fall back to the original version (to prevent errors in development mode) With this mechanism we can ensure updates to config.json and similar files take immediate effect without losing the performance benefit of client-side caching.
ybnd
added
bug
performance / caching
|
Hi @ybnd, |
The JSON mapping needs to be declared as a data block, otherwise browers may complain (nothing seems to really break though, except for Cypress) See https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script#any_other_value
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
6e6570e to
921ef18
Compare
ybnd
changed the title
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
921ef18 to
aa7ef08
Compare
tdonohue
added
port to dspace-7_x
ybnd
added
port to main
Member
Author
|
@tdonohue adjusted the labels since this PR was opened against 7.x |
This was referenced Feb 24, 2025
Member
Author
|
Note: when running multiple DSpace Angular instances in PM2 cluster mode this would cause the same filesystem operations in parallel, could cause problems. Could be addressed with Redis similar to #3110 (comment) |
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
2831efd to
4176ac9
Compare
Member
Author
|
Update: while debugging #4081 I realised this PR also gives us the opportunity to prefetch assets. Angular does similar for global CSS, but doesn't know about our themes or JSON configuration. Before: After: |
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
cbdaa7a to
4a276d5
Compare
Mainly for 8.x and beyond, where ThemeService is injected more often ~ standalone components
Both the configuration and theme CSS are known to be needed for every render. We can save some time by retrieving them as soon as the HTML is read.
This didn't work well in one specific case: requests without a path (e.g. https://demo.dspace.org) Such requests would result in a 304 redirect directly to index.html, losing the hashed file mapping and causing it to get cached. Then it could remain in the cache across rebuilds. Solutions: - Don't try to hash index.html, but modify it in place - Introduce configuration to disable caching for specific static files and apply this to index.html to prevent similar problems - Don't let browsers cache index.html when it's served for CSR under another path
…mic-configuration-7.6
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
533c71c to
600b468
Compare
pcg-kk
reviewed
Apr 15, 2025
Contributor
pcg-kk
left a comment
pcg-kk
left a comment
There was a problem hiding this comment.
Work for me fine, tested with SSR & without SSR
….6_CLEAN' into cache-bust-dynamic-configuration-7.6
ybnd
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
12b51de to
bc9a33e
Compare
Member
|
@ybnd could you rebase + repush to force the build tasks to re-run ? the unit test was so long ago that the logs aren't available so it's not clear why it failed |
- Track head links in a Map, because Set doesn't deduplicate them properly - Make ServerHashedFileMapping independent of ThemeConfig - Make theme pre-fetching configurable (e.g. only prefetch the main theme CSS, but leave route-based theme CSS lazy)
….6_CLEAN' into cache-bust-dynamic-configuration-7.6
Member
Author
|
@kshepherd if possible I'd like avoid rebasing to keep the history consistent with the "clean 7.6" version of the branch (already using that elsewhere) Looking back now I did find a few small improvements to make (mainly with the theme prefetching part), so that will revive the CI just as well. |
|
Hi @ybnd, |
alexandrevryghem
removed
the
port to dspace-8_x
12 tasks
…mic-configuration-7.6
alexandrevryghem
force-pushed
the
cache-bust-dynamic-configuration-7.6
branch
from
aa47ce7 to
5f27668
Compare
alexandrevryghem
removed
claimed: Atmire
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
References
Description
DSpace Angular's production server can modify the configuration it serves to CSR-only clients through YAML or environment variables. However, these files can remain cached in the browser and leave users with out-of-date configuration until the TTL runs out or the user does a "hard refresh". On build time this sort of problem is solved by saving the content hash as part of the path of each file (JS, CSS, ...)
We introduce HashedFileMapping to bridge the same gap for dynamic content generated after the server is built:
With this mechanism we can ensure updates to config.json and similar files take immediate effect without losing the performance benefit of client-side caching.
Compression support is included in anticipation of a pending PR for #3961, where the initial size of
config.jsoncan become large enough where compressing it would be a plus.Discussion
Instructions for Reviewers
To test:
[dist]/assets/config.<hash>.json→ hashed configuration file[dist]/dspace-theme.<hash>.css→ hashed theme CSS file[dist]/index.<hash>.html→ index file with an extrascript#hashed-file-mappingelement containing the mapping of plain paths to hashed pathsconfig.<hash>.jsonanddspace-theme.<hash>.cssare correctly served to the browser, and that the app runs/looks as before the PRChecklist
This checklist provides a reminder of what we are going to look for when reviewing your PR. You do not need to complete this checklist prior creating your PR (draft PRs are always welcome).
However, reviewers may request that you complete any actions in this list if you have not done so. If you are unsure about an item in the checklist, don't hesitate to ask. We're here to help!
mainbranch of code (unless it is a backport or is fixing an issue specific to an older branch).npm run lintnpm run check-circ-deps)package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.