feat: add external submissions onboarding CLI#217
Conversation
| return re.sub(r"^anonymous #(\d+)$", r"anonymous \1", normalized) | ||
|
|
||
|
|
||
| def get_next_forecast_due_date(today: Optional[date] = None) -> str: |
There was a problem hiding this comment.
move this to src/helpers/question_curation.py
| r"""Admin CLI to manage external submission teams. | ||
|
|
||
| Run locally by an admin with gcloud application-default credentials; not deployed. See | ||
| `src/external-submissions/DESIGN.md` for the architecture and the requirements this implements. |
There was a problem hiding this comment.
the comment is not correct, e.g. no file named src/external-submissions/DESIGN.md
| subject = "ForecastBench — your team has been registered" | ||
| anonymous_note = ( | ||
| f"\nYou are registered anonymously. Your public name is '{organization}': use it for" | ||
| " both 'organization' and 'model_organization' in your forecast files.\n" |
There was a problem hiding this comment.
just for 'organization'. They can choose whether or not to use it for 'model_organization'
| {next_due_date}.{organization}.1.json | ||
| 4. Upload it with either command before 23:59:59 UTC on the forecast due date: | ||
| gsutil cp your-file.json gs://{bucket}/{team_id}/ | ||
| gcloud storage cp your-file.json gs://{bucket}/{team_id}/ |
There was a problem hiding this comment.
Delete 1-4 above. Let's just leave the link to the wiki, that way if anything changes we only change it there
There was a problem hiding this comment.
remove "You may submit up to 3 forecast sets per round (one per model). "
| register_parser.add_argument("--service-accounts", nargs="*", default=[]) | ||
| register_parser.add_argument("--anonymous", action="store_true") | ||
|
|
||
| remove_parser = subparsers.add_parser("remove", help="Deactivate a team.") |
There was a problem hiding this comment.
perhaps call this deactivate instead of remove
| remove_parser = subparsers.add_parser("remove", help="Deactivate a team.") | ||
| remove_parser.add_argument("--team-id", required=True) | ||
|
|
||
| init_parser = subparsers.add_parser("init-counters", help="One-time counter setup.") |
There was a problem hiding this comment.
If this is run once, then 2 months later run again, will that break things? If so, then remove it as an option and move the initialization to a setup script that will only be run once
| RUNNING_LOCALLY = bool(int(os.environ.get("RUNNING_LOCALLY", False))) | ||
| BUCKET_MOUNT_POINT = os.environ.get("BUCKET_MOUNT_POINT", "") | ||
| WORKSPACE_BUCKET = os.environ.get("WORKSPACE_BUCKET") | ||
| BUILD_ENV = os.environ.get("BUILD_ENV", "dev") |
There was a problem hiding this comment.
Let's remove this. Prefer RunMode.
The rule is that BUILD_ENV is used to choose the deployment environment (e.g., the website builds the dev version or the prod version). RunMode controls Python behavior.
| register_parser.add_argument("--emails", nargs="*", default=[]) | ||
| register_parser.add_argument("--service-accounts", nargs="*", default=[]) | ||
| register_parser.add_argument("--anonymous", action="store_true") | ||
|
|
There was a problem hiding this comment.
Following the BUILD_ENV change above, please add this functionality to the CLI:
--mode TEST: do not attempt SMTP, return welcome_email_sent: false, and add a warning like "Welcome email skipped in TEST mode. email would have been sent to <<email address(es)>>. pass--send-email-in-testto send a rerouted test email."--mode TEST --send-email-in-test: email is sent with[TEST]prefix in subject--mode PROD: Email is sent normally without[TEST]prefix.--send-email-in-testhas no effect.
We'll have a dev version of the firebase in the dev enivonnment that will have our email addresses in it.
There was a problem hiding this comment.
if --mode is not passed, default to TEST
| @: | ||
|
|
||
| clean: | ||
| rm -rf __pycache__ |
There was a problem hiding this comment.
Add a file called onboard.example.sh that looks likething like:
EMAILS=[]
ORGANIZATION=""
...
python main.py --emails=$EMAILS --organization=$ORGANIZATION --run-mode=TEST ...add onboard.sh to .gitignore
Then we'll run from the script such that fields are easier to modify and commands don't stay in local history (so aren't unintentionally run).
| def test_returns_false_without_password(self, monkeypatch): | ||
| monkeypatch.setattr(env, "SMTP_USER", "sender@example.org") | ||
| monkeypatch.delenv("SMTP_PASSWORD", raising=False) | ||
| assert email.send_email(["a@x.com"], "subject", "body") is False |
There was a problem hiding this comment.
prefer a different domain than x.com since it actually means something. just a random hash will do: dummy-domain-x92ah8.com.
RaidedCluster
force-pushed
the
johan/external-submissions-v3
branch
from
550bcdb to
3c1ad45
Compare
2 participants
No description provided.