Hi everyone,
I'm looking to use a model through API requests—like we commonly do these days—when local deployment isn’t feasible. I'm currently developing a new healthcare platform on my dev machine, and the underlying model is Gemma4B. However, with only an 8GB Nvidia GPU, it takes around 12 minutes to get a single response, which is far too slow.
Instead of running models locally, I want to make inference calls using an API key—like we used to do in the good old days. I tried Hugging Face, but wasn’t able to purchase credits for their inference service.
I'm not interested in setting up a Google Cloud endpoint or doing fine-tuning. I simply want to tap into a model’s existing knowledge to analyze symptoms from text or images as part of a comprehensive healthcare platform.
Thanks,
Eran
Hi everyone,
I'm looking to use a model through API requests—like we commonly do these days—when local deployment isn’t feasible. I'm currently developing a new healthcare platform on my dev machine, and the underlying model is Gemma4B. However, with only an 8GB Nvidia GPU, it takes around 12 minutes to get a single response, which is far too slow.
Instead of running models locally, I want to make inference calls using an API key—like we used to do in the good old days. I tried Hugging Face, but wasn’t able to purchase credits for their inference service.
I'm not interested in setting up a Google Cloud endpoint or doing fine-tuning. I simply want to tap into a model’s existing knowledge to analyze symptoms from text or images as part of a comprehensive healthcare platform.
Thanks,
Eran