Staging branch for LLM module #363
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* import llm code from robotl * make optional deps optional * replace template with Template.define * bring in tests * typing bullshit * disable incremental type checking to avoid crash * format * remove anthropic and cache modules * fix * remove dupe * rename * fix not handled * reorganize * wip * wip * wip * wip * restrict python version * specify python version * rename * rename
* import llm code from robotl * make optional deps optional * replace template with Template.define * bring in tests * typing bullshit * disable incremental type checking to avoid crash * format * remove anthropic and cache modules * fix * remove dupe * rename * fix not handled * reorganize * wip * wip * wip * wip * restrict python version * specify python version * rename * work on a tool call interface * wip * wip * format * wip * revert whitespace changes * fix decorator calls * lint * lint * allow no arguments to decorator * work
)" This reverts commit 02c4378.
* Adding a lower-level event and a logger example * Bring back the note * Linting * Add a logging handler that expose python logging * Lint * Miscs * Update LoggingHandler interface * Linting and fixing signatures
* added support for images * updated conversion to be delegated through a dynamic pydantic model instead * Revert "updated conversion to be delegated through a dynamic pydantic model instead" This reverts commit d41e810. * updated implementation to use OpenAI types and unify conversion logic under Tool.
* Adding caching handler * Adding caching handler * Linting and cleaner implementation of hashing keys * Minor linting * Slightly more precise type for cacher * Slightly more precise type for cacher * Addressing comments * Attempt to fix issue with pypandoc * Fixing CI * Linting
* refactor OpenAIAPIProvider Template.__call__ now desugars into more primitive operations * add an inheritable class that provides stateful behavior to templates * remove unnecessary unset
* add litellm to the dependencies for effectful llm * switched internals to use litellm & fixed bug with dicts * updated schemas and imports * fixed encoding-decoding issue with shim * switched to completions and litellm * removed old compatibility shim * replaced openai type with litellm type * s/LLMProvider/LiteLLMProvider/ * modified LiteLLMProvider to inject params into llm_request * fixed ipynb imports * updated s/llm_request/completion and functools.wraps * s/llm_request/completion * dropped try-catch on litellm import * updated providers and dropped openai dep * updated completion to be more compositional * minor fix for single positional argument for completion * updated init to default to gpt-4o to preserve previous behaviour
* updates ProgramSynthesis to use linecache and integrate with inspect.getsource
* refactored encoding to be type directed * updated handling of str and bytes
* Release v0.2.3 (#374) * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * lint --------- Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com> Co-authored-by: Eli <eli@basis.ai>
Contributor
Author
|
Seems like something went wrong with #423? Why does it think |
This was referenced Dec 4, 2025
* Release v0.2.3 (#374) * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * Try pulling in pyproject.toml from staging-llm to master (#425) --------- Co-authored-by: Jack Feser <jack.feser@gmail.com> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com>
* Logging retries * Remove redundant assignment * Linting * Adding mypy check and test * SynthesizedFunction for constrained decoding, adapt the tests * Linting * Let LLM generate param names * Pydantic field annotation * Adding validation error example * Linting * Fix minor formatting for type context * Update llm dependencies to include mypy * More comprehensive error message * linting * More comprehensive import * Fix `staging-llm` diff against `master` (#426) * Release v0.2.3 (#374) * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * Try pulling in pyproject.toml from staging-llm to master (#425) --------- Co-authored-by: Jack Feser <jack.feser@gmail.com> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com> * Logging retries * Make linter happy with the exception_cls * More linting * Revert the changes to synthesis, it belongs to a different PR * Add test for retry handler * More linting * Removing mypy for llm --------- Co-authored-by: eb8680 <eb8680@users.noreply.github.com> Co-authored-by: Jack Feser <jack.feser@gmail.com> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com>
* Release v0.2.3 (#374) * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * Try pulling in pyproject.toml from staging-llm to master (#425) * Generate instance-level `Operation`s for bound methods (#351) * generalize __get__ * nits * coverage of methoddescriptor api * methodtype * simplify * simplify * simplify * format * revert * restore * simplify * simplify * retain instance op on term construction * Simplify apply inheritance * assign * put call next to init_subclass * add explanatory comment * Operation.apply -> Operation.__apply__ * add test based on issue description * fix doctest * Fix dataclass @defops and added dataclass metaclass (#439) * fixed dataclass ordering and added metaclass for simplifying construction of dataclass terms * ensure term fields are not being overriden * added decorator and dataclass * updated to make defdata registration automatic * simplified dataclass loop * updated to give property op an appropriate name * added failing tests * fixed failing test * fixed numpyro/pyro/torch interfaces * minor fix + test for deffn kwargs --------- Co-authored-by: Jack Feser <jack.feser@gmail.com> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com> Co-authored-by: Kiran Gopinathan <23038502+kiranandcode@users.noreply.github.com>
* implemented unified encoding type * implemented decoding * unified __init__ * added tests for basemodels * s/@property/@functools.cached_property/ * type for encode and decode * removed handling for numbers.Number and explicit tests for complex * fixed is_dataclass checks * updated to check parameter annotations in Tool.of_operation constructor * updated serializer to be more selective in what is an image * reducing number of #type: ignores, and switching to typing.Any * removed comment * dropped dataclass support * dropped tests for dataclass with image * updated dataclass tests to stop assuming pydantic models * test for tool that returns list of images * made serialization a parameter of encodable and thus type-directed * dropped test for tool that returns list of images * dropped registration of encodable types * dropped unused typevar * s/_Encodable/EncodableAs/
…Template (#434) * Adding lexical context collection * Allow model input to refer to anything within the lexical context * Different handling between representable object and other types when referred to in Template * More edge case handling * More edge case handling * More edge case handling * Register more instead of _get_source_for_object * Tune down the scope to only collect relevant tools templates * Constructing Tool from Template, allowing higher-order Tempalte * Linting * Fix exception type * Fix exception type to NotHandled * Fix stringinified annotation * Fix tool.define signature * Trim Tool.define * Trip format_value.register * Add warning on no doc case * Removing default for __name__ * More specific lexical context type annotation * Additional tools for pydandic-compatible conversion and recursion depth limiting * Linting * Linting * Linting * Linting * Minor fixes * Merge encoding/decoding * Update a lot of prompt with "do not use any tools" * Update a lot of prompt with "do not use any tools" * More tool limiting prompts * Lint * Lint * Factoring out book and poem tools after multiple failed prompt modification attempts * Revert semantics * Lint * Update name of lexical_context to __context__ * Lint * Minor * More prompt modification * Make sure notebook run
* Release v0.2.3 (#374) * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * Try pulling in pyproject.toml from staging-llm to master (#425) * Generate instance-level `Operation`s for bound methods (#351) * generalize __get__ * nits * coverage of methoddescriptor api * methodtype * simplify * simplify * simplify * format * revert * restore * simplify * simplify * retain instance op on term construction * Simplify apply inheritance * assign * put call next to init_subclass * add explanatory comment * Operation.apply -> Operation.__apply__ * add test based on issue description * fix doctest * Fix dataclass @defops and added dataclass metaclass (#439) * fixed dataclass ordering and added metaclass for simplifying construction of dataclass terms * ensure term fields are not being overriden * added decorator and dataclass * updated to make defdata registration automatic * simplified dataclass loop * updated to give property op an appropriate name * added failing tests * fixed failing test * fixed numpyro/pyro/torch interfaces * minor fix + test for deffn kwargs * Type check and lint example code (#449) * format example code * type check examples * Add beam search example using thermometer continuations (#431) * add beam search example using thermometer continuations * address comments * add docstring * lint * Fix for jax 0.8.2 (#455) * fix for jax 0.8.2 * add more register * format --------- Co-authored-by: Jack Feser <jack.feser@gmail.com> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com> Co-authored-by: Kiran Gopinathan <23038502+kiranandcode@users.noreply.github.com>
* Release v0.2.3 (#374) * stash * Install prettyprinter for term when library is available (#386) * install prettyprinter for term when library is available * lint * move code into types.py * fixes * fix pypandoc issue (#397) * Convert evaluate to a singledispatch (#398) * convert evaluate to a singledispatch * lint * add jnp.pi and ArrayTerm.T (#394) * Deprecate defterm (#399) * deprecate defterm * remove defterm case * remove defterm * lint * evaluate distribution arguments * lint * remove interpreter * Revert "remove interpreter" This reverts commit 3044277. * wip * lint * Rework numpyro distribution handling to enable symbolic distributions and handling of distribution methods (#311) * refactor distribution operations * add a test for typeof of distributions * add tests for symbolic dists/arguments * introduce operations for distribution methods * comment * fix tests * work around #310 * replace hack with new hack * tweak repr for _BaseOperation * lint * work around #312 * clean up access to dist ops * wip * wip * add type annotations to get correct term conversion * lint * include distribution arguments as properties * fix distribution calls * try again * fixes * format * Box the output of `__type_rule__` (#387) * box the output of __type_rule__ * fix tests * fix tests * require callers of __type_rule__ to box arguments * fix * move Box out of ops.types * lint * fix test * fix syntactic_eq implementation for jax arrays (#405) * Fix recursion error in sizesof (#406) * fix recursion error in sizesof * format * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * wip * Allow `_BaseOperation` subclasses to have an overrideable `apply` method (#414) * stash * fixes * initial * wip * lint * ensure each subclass has a fresh operation * wip * wip * lint * wip * wip * lint * refactor class method support * move defops * fix test * remove singledispatch case and add test * move definition * cleanup * simplify * cleanup * lint * fix failing test * fix classmethod * __isabstractmethod__ * revert --------- Co-authored-by: Eli <eli@basis.ai> * wip * fix define signature * wip * wip * wip * wip * wip * wip * wip * lint * wip * test fix * fix merge error * lint * wip * respond to feedback * respond to feedback * clean up notebook * remove breakpoint * relax typing requirement * improve handling of method templates * lint * remove imprecise check * handle nested classes using qualname * feedback * fix scoping issue and add tests * address feedback * add xfail tests * lint --------- Co-authored-by: Eli <eli@basis.ai> Co-authored-by: Tim Cooijmans <cooijmans.tim@gmail.com> Co-authored-by: eb8680 <eb8680@users.noreply.github.com>
This was referenced Dec 29, 2025
* raise error when encoding terms or operations * change error type
* implemented record and replay fixtures for LLM calls * changed defaults to not rebuild fixtures, with dedicated make rule to rebuild * switched pytest to use request fixture instead of env
* remove program synthesis code * lint * mark xfail
* fixed failing test * made templates non-recursive by default + annotation for recursion * added docstring for IsRecursive * call IsRecursive.infer_annotations in tool's init * added check for sig.parameters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.