(feat) Always use json schema to validate response body when available

[danwritecode]

Currently, we DO validate the JSON response with json schema if one is given. However, this requires the model itself to support JSON schema. We should still allow users to provide JSON schema and we should validate the response body even in situations when the model doesn't support json schema.

The important thing here is that we do not pass "json_schema" to the model otherwise it will bomb. We should only use it for validation purposes. Currently in llm.rs we are doing exactly this, so some small tweaks are needed to make this a reality.

This also allows use to dynamically and automatically pass in the JSON schema to the end of the prompt so that you don't have to manually embed the JSON schema into the prompt itself. We again currently do this when building the LlmRequest struct.

[DarkStarStrix]

I'd also like to try this I worked on json validation in my own inference application so I can help with this one to

[danwritecode]

I'd also like to try this I worked on json validation in my own inference application so I can help with this one to

in the llm.rs service you'll see that I am already using a library to validate the json body using the json schema. so it's all there basically, we just have to make a change to not include json_schema to the model if it's not supported. Furthermore, where we construct the LlmRequest object, I am pushing the JSON schema into the end of the system prompt this way the user doesn't have to define the shape of the JSON themselves manually in the prompt itself.