BaseSource.__init_subclass__ validates only a fixed global set, _REQUIRED_METADATA_KEYS = {source_type, source_intro, resolution_criteria}, then injects all SOURCE_METADATA[name] keys onto the class. Source-specific keys a source actually depends on are neither required nor defaulted, so a missing/typo'd key fails late or silently:
ticker_renames (yfinance) is read in sources/yfinance.py and func_yfinance_update/main.py, but is not in the required set and has no base-class default → omitting it raises AttributeError at update() runtime instead of at import.nullified_questions (fred/polymarket/wikipedia/yfinance) is saved by the base-class default = [] → omitting it silently behaves as "nothing nullified" rather than failing.
infer/manifold/metaculus use only the three required keys, so they're unaffected today; this is forward-looking as sources add source-specific metadata.
Proposed fix: add an opt-in required_metadata_keys: ClassVar[set[str]] = set() so a source can declare what is necessary for it to even run. Then we check on BaseSource; in __init_subclass__ check _REQUIRED_METADATA_KEYS | cls.required_metadata_keys against meta.keys() so a source (e.g. yfinance → {"ticker_renames"}) fails loudly at class-definition time. Backward-compatible (default empty set for all other sources).
I'd also suggest moving away from a default nullified_questions so that nullification is more explicit by source - if a source wants to have/apply nullification at any point, it is welcome to add an empty list in _metadata.py so as to say "I can have and should consider nullified questions in the proecssing, but right now there are none" - which can happen either when a source is newly added or when at some point, for whatever reason, a nullification no longer applies.
BaseSource.__init_subclass__validates only a fixed global set,_REQUIRED_METADATA_KEYS = {source_type, source_intro, resolution_criteria}, then injects allSOURCE_METADATA[name]keys onto the class. Source-specific keys a source actually depends on are neither required nor defaulted, so a missing/typo'd key fails late or silently:ticker_renames(yfinance) is read insources/yfinance.pyandfunc_yfinance_update/main.py, but is not in the required set and has no base-class default → omitting it raisesAttributeErroratupdate()runtime instead of at import.nullified_questions(fred/polymarket/wikipedia/yfinance) is saved by the base-class default= []→ omitting it silently behaves as "nothing nullified" rather than failing.infer/manifold/metaculus use only the three required keys, so they're unaffected today; this is forward-looking as sources add source-specific metadata.
Proposed fix: add an opt-in
required_metadata_keys: ClassVar[set[str]] = set()so a source can declare what is necessary for it to even run. Then we check onBaseSource; in__init_subclass__check_REQUIRED_METADATA_KEYS | cls.required_metadata_keysagainstmeta.keys()so a source (e.g. yfinance → {"ticker_renames"}) fails loudly at class-definition time. Backward-compatible (default empty set for all other sources).I'd also suggest moving away from a default
nullified_questionsso that nullification is more explicit by source - if a source wants to have/apply nullification at any point, it is welcome to add an empty list in_metadata.pyso as to say "I can have and should consider nullified questions in the proecssing, but right now there are none" - which can happen either when a source is newly added or when at some point, for whatever reason, a nullification no longer applies.