-
Notifications
You must be signed in to change notification settings - Fork 35
feat: add translation script and GitHub Action for mobile i18n autofill #108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Changes from all commits
fd65a26
63eb15b
e6093fc
e6e8a00
2c484d7
eab5c7c
4dee8cc
a33d7f1
dca68b2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| name: Mobile i18n Autofill (bot PR) | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| locales: | ||
| description: 'Comma-separated locale codes to translate' | ||
| required: false | ||
| default: 'ar,bn,de,en,es,fa,fr,gu,hi,hu,id,km,kn,ml,mr,ms,my,pl,pt,ru,si,sw,te,ur' | ||
| type: string | ||
| model: | ||
| description: 'Gemini model to use for translation' | ||
| required: false | ||
| default: 'gemma-3-27b-it' | ||
| type: string | ||
| batch-size: | ||
| description: 'Number of strings per translation batch' | ||
| required: false | ||
| default: '15' | ||
| type: string | ||
| repo-root: | ||
| description: 'Module root to scope translation (e.g. ./feature/settings)' | ||
| required: false | ||
| default: '.' | ||
| type: string | ||
|
|
||
| pull_request: | ||
| types: [labeled] | ||
| branches: [dev] | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| i18n-autofill: | ||
| if: >- | ||
| github.event_name == 'workflow_dispatch' || | ||
| github.event.label.name == 'needs-translation' | ||
| runs-on: ubuntu-latest | ||
|
|
||
| env: | ||
| LOCALES: ${{ inputs.locales || 'ar,bn,de,en,es,fa,fr,gu,hi,hu,id,km,kn,ml,mr,ms,my,pl,pt,ru,si,sw,te,ur' }} | ||
| MODEL: ${{ inputs.model || 'gemma-3-27b-it' }} | ||
| BATCH_SIZE: ${{ inputs.batch-size || '15' }} | ||
| REPO_ROOT: ${{ inputs.repo-root || '.' }} | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| repository: ${{ github.event.pull_request.head.repo.full_name || github.repository }} | ||
| ref: ${{ github.event.pull_request.head.ref || github.ref }} | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Set up JDK 21 | ||
| uses: actions/setup-java@v4 | ||
| with: | ||
| java-version: '21' | ||
| distribution: 'temurin' | ||
|
|
||
| - name: Set up Python 3.11 | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install Python dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install google-genai lxml | ||
|
|
||
| - name: Run translation autofill | ||
| env: | ||
| GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} | ||
| run: | | ||
| python translate.py \ | ||
| --mode apply \ | ||
| --repo-root "$REPO_ROOT" \ | ||
| --locales "$LOCALES" \ | ||
| --model "$MODEL" \ | ||
| --batch-size $BATCH_SIZE | ||
|
|
||
| - name: Validate Android resources compile | ||
| run: | | ||
| ./gradlew :cmp-android:processDemoDebugResources | ||
|
|
||
| - name: Commit and Push changes | ||
| run: | | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| git add cmp-android/ cmp-navigation/ feature/ | ||
|
|
||
| if ! git diff --cached --quiet; then | ||
| git commit -m "chore: auto-generate mobile i18n translations" | ||
| git push origin HEAD:${{ github.event.pull_request.head.ref || github.ref_name }} | ||
| else | ||
| echo "No changes to commit." | ||
| fi | ||
|
Comment on lines
+35
to
+100
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Workflow violates repository standard for reusable workflows. This job is implemented inline, but workflow files in this repo must delegate to reusable workflows from As per coding guidelines 🧰 Tools🪛 actionlint (1.7.11)[error] 89-89: "github.event.pull_request.head.ref" is potentially untrusted. avoid using it directly in inline scripts. instead, pass it through an environment variable. see https://docs.github.com/en/actions/reference/security/secure-use#good-practices-for-mitigating-script-injection-attacks for more details (expression) 🤖 Prompt for AI Agents |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| # Android String Resource Translator (`translate.py`) | ||
|
|
||
| A production-ready Python script for translating Android string resources (`strings.xml` and `arrays.xml`) using the Google Gemini API. | ||
|
|
||
| ## Features | ||
|
|
||
| - **Format Preservation**: Ensures comments, spacing (blank lines), and structure match the source file exactly. | ||
| - **Placeholder & Markup Safety**: Freezes placeholders (e.g., `%s`, `%1$d`) and markup tags (e.g., `<b>`, `<xliff:g>`) before translating to guarantee they are preserved and kept in the correct order. | ||
| - **Source Attribute Propagation**: Copies attributes like `formatted`, `product`, and `tools:*` to the translated strings. | ||
| - **Robust Error Handling**: Includes batch translation with individual string fallback on failure, and automatic retry mechanisms for rate limits (429) or model overloads (503). | ||
| - **Change Detection**: Tracks source strings through a simple hash-based snapshot mechanism (`.translation_snapshots/`). Only new strings and strings whose source text modified are re-translated, saving time and tokens. | ||
| - **Advanced Resource Support**: Translates single `<string>`, ordered `<string-array>`, and `<plurals>` resources out-of-the-box. | ||
| - **Character Compatibility**: Manages HTML entity conversions and robust Android special character escaping. | ||
| - **AAPT2 Compatibility**: Implements proper `xliff` namespace handling to prevent build errors. | ||
|
|
||
| ## Requirements | ||
|
|
||
| 1. Python 3.8+ | ||
| 2. Required packages: | ||
| ```bash | ||
| pip install google-genai lxml | ||
| ``` | ||
| 3. A Google Gemini API Key | ||
|
|
||
| ## Usage | ||
|
|
||
| Set your Google Gemini API key as an environment variable: | ||
| ```bash | ||
| export GEMINI_API_KEY=your_api_key_here | ||
| ``` | ||
| *(You can customize the environment variable name via the `--api-key-env` flag).* | ||
|
|
||
| ### Applying Translations | ||
|
|
||
| Run the script in `apply` mode to fetch missing strings and write translated files directly to their respective `values-{locale}` folders. | ||
|
|
||
| ```bash | ||
| # Basic usage | ||
| python translate.py --mode apply --locales es,de,fr | ||
|
|
||
| # Using a specific model and fine-tuning batch parameters | ||
| python translate.py \ | ||
| --mode apply \ | ||
| --repo-root . \ | ||
| --locales ar \ | ||
| --model gemma-3-27b-it \ | ||
| --batch-size 15 \ | ||
| --request-delay 4.0 | ||
| ``` | ||
|
|
||
| ### Checking for Missing Translations | ||
|
|
||
| Run the script in `check` mode inside CI/CD workflows to simply verify whether all strings are translated without making any actual API calls or file modifications. | ||
|
|
||
| ```bash | ||
| python translate.py --mode check --locales es,de,fr | ||
| ``` | ||
| *In `check` mode, the script exits with code `2` if translations are missing.* | ||
|
|
||
| ## Available Command-Line Arguments | ||
|
|
||
| - `--mode` (Required): `apply` (to translate and write xml) or `check` (to only check for missing keys). | ||
| - `--locales`: A comma-separated list of target Android language/region codes (e.g. `es,fr,de,ar`). Default is `es,de`. | ||
| - `--repo-root`: The path to the root of the Android project (where to search for `src/*/res/values/strings.xml` or Compose Multiplatform equivalent). Default is `.`. | ||
| - `--model`: The Gemini API model to use. Default is `gemini-2.0-flash`. | ||
| - `--batch-size`: Number of strings to send in a single Gemini API request. Default is `20` (capped at `15` for Gemma models). | ||
| - `--request-delay`: Delay in seconds between API requests to prevent immediate rate-limiting. Default is `2.0` (forced to `4.0` for Gemma models). | ||
| - `--api-key-env`: Name of the environment variable used to retrieve the API key. Default is `GEMINI_API_KEY`. | ||
| - `--no-validate`: Disable automatic malformed XML checks after writing translations. | ||
| - `--verbose` / `-v`: Enable debug-level logging. | ||
|
|
||
| ## Under The Hood | ||
|
|
||
| ### 1. Snapshot Tracking | ||
| When you successfully translate strings, the script saves a JSON file in `.translation_snapshots/` within the source module. Subsequent runs will compare current source text against these hashes, allowing `translate.py` to seamlessly fix previously translated strings if you tweak the original English wording. | ||
|
|
||
| ### 2. Orphaned Translations cleanup | ||
| In `apply` mode, if a developer deletes a string or an array item from the english source, the script reliably detects and strips the orphaned translation from all localized strings files to avoid accumulation of unused strings. | ||
|
|
||
| ### 3. Rate Limit Handling | ||
| If the Google Gemini backend responds with `429 Rate limited` or `503 Service Unavailable`, `translate.py` will automatically backoff and retry according to `--max-retries` and the wait times embedded in API responses. | ||
markrizkalla marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Uh oh!
There was an error while loading. Please reload this page.