Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion blog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "Posts about the great work we are doing at Anarchy"

## Posts

- **[Guided Distillation](https://anarchy.ai/blog/guided_distillation)** Read about how we finetuned a Pythia model to constrain the output space of a 70 million parameter Pythia model!
- **[Guided Distillation](https://anarchy.ai/blog/guided_distillation)** Read about how we fine-tuned a Pythia model to constrain the output space of a 70 million-parameter Pythia model!

- **[The REBEL Agent](https://anarchy.ai/blog/rebel)** Read about the REBEL agent, a tool that empowers LLMs to use external tools to solve compositional questions!

8 changes: 4 additions & 4 deletions blog/guided_distillation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: 🎻 Guided Distillation
description: 'Finetune a Pythia 70M model to only output within a '
---

The paper Efficient Guided Generation for Large Language Models provides a state machine based approach to constrain the output of a Large Language Model. To use the code from this paper, we used the Outlines python library. Using this library we were able to add three types of guiding to LLM generation on our LLM-VM repositiory
The paper Efficient Guided Generation for Large Language Models provides a state machine-based approach to constrain the output of a Large Language Model. To use the code from this paper, we used the Outlines Python library. Using this library we were able to add three types of guiding to LLM generation on our LLM-VM repository.

- **Regex** - The output of the underlying LLM will follow the inputted regex.
- **Type** - The output of the underlying LLM will be of the inputted type.
Expand All @@ -14,7 +14,7 @@ The paper Efficient Guided Generation for Large Language Models provides a state
from llm_vm.client import Client
import os

# Instantiate the client specifying which LLM you want to use. Gudied generation always uses an open source version of GPT2.
# Instantiate the client specifying which LLM you want to use. Guided generation always uses an open source version of GPT2.
client = Client()

# Regex-based guided generation
Expand All @@ -33,12 +33,12 @@ response = client.complete(prompt = 'How many presidents has the USA had?',
print(response)
```

In this blog we discuss how we distilled the GPT2 model down to a 70M parameter Pythia model and were able to limit the constrain the output space of the Pythia model.
In this blog, we discuss how we distilled the GPT2 model down to a 70M parameter Pythia model and were able to limit constrain of the output space of the Pythia model.




<Warning>Saving finetuned LLMs can result in a large amount of storage space. Make sure to keep an eye on how much you're saving</Warning>
<Warning>Saving fine-tuned LLMs can result in a large amount of storage space. Make sure to keep an eye on how much you're saving</Warning>

## References
https://arxiv.org/pdf/2307.09702v4.pdf
Expand Down
3 changes: 2 additions & 1 deletion contributing/community_development.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The first and most importantly step is to [join our discord](https://discord.com

Our aim is for this community to be not just for organization, but a teaching community.
Don't be afraid to speak up with questions, and don't shame people for asking them!
There are a mix of experts and n00bs there.
There is a mix of experts and n00bs there.

### Becoming an official contributor!

Expand All @@ -28,6 +28,7 @@ We reserve the right to revoke contributor status at any time for any reason (pl
## Bounty Program

Some tickets we'll pay you for closing! Look at the [ticket](https://github.com/anarchy-ai/LLM-VM/issues) labels to see how much the bounty is.

Note we can't guarantee that if you close a ticket we can pay you (only if you are from a non-US sanctioned country),
and it is up to you to reach out and be annoying about this. To get started, it is essential to read the [full guide in discord](https://discord.com/channels/1075227138766147656/1147542772824408074)

Expand Down
10 changes: 5 additions & 5 deletions contributing/getting_involved.mdx
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: "🔥 Become an Anarchist"
description: "Excited to develop new democratized AGI? Us too!"
description: "Excited to develop a new democratized AGI? Us too!"
---

This project is at it's very earliest stages, but we still welcome support!
We have a number of ways to get involved, from [working for us](jobs), to working on [tickets directly](community_development).
This project is at its very earliest stages, but we still welcome support!
We have a number of ways to get involved, from [working for us](jobs) to working on [tickets directly](community_development).

By far the easiest way to get a job is to work on tickets directly and become a top contributor and get noticed!
By far the easiest way to get a job is to work on tickets directly. Become a top contributor and get noticed!

As we want to encourage open-source AGI development, we have a bounty program to pay for open development
As we want to encourage open-source AGI development, we have a bounty program to pay for open development.
6 changes: 3 additions & 3 deletions contributing/jobs.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "👷 Work for us"
description: "Here we describe the process for getting a job and where to find open-jobs"
description: "Here we describe the process for getting a job and where to find open jobs"
---

Right now we're very early stage so we have limited jobs available.
Expand All @@ -19,5 +19,5 @@ Nevertheless, we're always on the lookout for exceptional talent.
* Compilers and GPU/driver engineers
* DevRel Profiles

We welcome applicants with non traditional backgrounds who strongly believe in the mission and want
to assist with marketing and sales.
We welcome applicants with non-traditional backgrounds who strongly believe in the mission and want
to assist with marketing and sales.
10 changes: 5 additions & 5 deletions get_started/install.mdx
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
---
title: 📀 Install
description: "You can install everything you need easily from the command line. Make sure check the [system requirements](system) and that you have [Python](https://www.python.org/downloads/release/python-3114/) installed."
description: "You can install everything you need easily from the command line. Make sure to check the [system requirements](system) and that you have [Python](https://www.python.org/downloads/release/python-3114/) installed."
---

## Installation
**Use git and pip to quickly install the LLM-VM**

### Get the code
You can download the code from our [Github](https://github.com/anarchy-ai/LLM-VM/archive/refs/heads/main.zip) repo manually, or navigate a CLI to your perferred directory and run:
You can download the code from our [Github](https://github.com/anarchy-ai/LLM-VM/archive/refs/heads/main.zip) repo manually, or navigate a CLI to your preferred directory and run:
```bash CLI Install
git clone https://github.com/anarchy-ai/LLM-VM.git
```

### Install the module
Once you've downloaded the code, getting it to work with python is easy, open a CLI and
Once you've downloaded the code, getting it to work with Python is easy, open a CLI and
<CodeGroup>
```bash Standard Install
cd LLM-VM
Expand All @@ -27,7 +27,7 @@ pip3 install --editable .

This will install both the LLM-VM and the HTTP server.

<Note>Installing with the ```--editable``` flag allows your changes to the codebase take effect without rebuilding with ```pip3 install .```</Note>
<Note>Installing with the ```--editable``` flag allows your changes to the codebase to take effect without rebuilding with ```pip3 install .```</Note>


<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
6 changes: 3 additions & 3 deletions get_started/quickstart/agents.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: 'Our AI agents expand what you can do with LLMs!'
## The REBEL Agent
**REcursion Based Extensible LLM**

Our REBEL agent takes a novel approach to answering complex questions. Using recursive reasoning, REBEL expands what LLMs can do with problem decomposition and tool use. In this way, we are able to answer questions requiring data LLMs were not directly trained on.
Our REBEL agent takes a novel approach to answering complex questions. Using recursive reasoning, REBEL expands on what LLMs can do with problem decomposition and tool use. In this way, we are able to answer questions requiring data that LLMs were not directly trained on.

### Running REBEL
**Getting started with REBEL is easy**
Expand All @@ -23,7 +23,7 @@ response = client.complete(
prompt = 'Is it warmer in Paris or Timbuktu and what are the temperatures in either city?',
context='',
openai_key=os.getenv("OPENAI_API_KEY"), #for REBEL we need an OpenAI key
tools=[{'description': 'Find the weather at a location and returns it in celcius.', #this tool list contains only one dictionary, therefore only one tool
tools=[{'description': 'Find the weather at a location and returns it in celsius.', #this tool list contains only one dictionary, therefore only one tool
'dynamic_params': {
"latitude": 'latitude of as a float',
"longitude": 'the longitude as a float'
Expand All @@ -39,7 +39,7 @@ Tools are defined by dictionaries that are added to the list ```tools```. These
|Field| Type | Description|
|-|-|-|
|```'description'```| string | A description of what the tool does|
|```'dynamic_params'```| dictionary | A dictionary containing key value pairs (paramter name : description) of the API endpoint's mutable parameters that need to be set by REBEL in order to answer a query|
|```'dynamic_params'```| dictionary | A dictionary containing key-value pairs (parameter name : description) of the API endpoint's mutable parameters that need to be set by REBEL in order to answer a query|
|```'method'```| string | ```GET``` or ```POST```, whichever is the type of the API endpoint|
|```'url'```| string | URL of the API endpoint that the given tool specifies|
|```'static_params'```| dictionary | Any parameters that are constant between all API calls. An API key/token is an example of this|
Expand Down
4 changes: 2 additions & 2 deletions get_started/quickstart/completions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ print(response)

</CodeGroup>

<Warning>Using OpenAI's models require an OpenAI API Key and may result in costs not associated with Anarchy's LLM-VM</Warning>
<Warning>Using OpenAI's models requires an OpenAI API Key and may result in costs not associated with Anarchy's LLM-VM</Warning>

## Supported Models
We support several open LLM model families. You can see which ones and the default models used below.
Expand All @@ -53,4 +53,4 @@ Just replace ```'chat_gpt'``` in the example with your desired model and you're
</Accordion>
For more information on selecting models visit our [Local LLMs](local_llms) section.

<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
24 changes: 12 additions & 12 deletions get_started/quickstart/finetuning.mdx
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
title: 🎻 Finetuning
description: 'Here we describe how to finetune LLMs and load previously trained models'
title: 🎻 Fine-tuning
description: 'Here we describe how to fine-tune LLMs and load previously trained models'
---

## Finetuning Models
The LLM-VM uses a data synthesis feature to engage a teacher/student model for finetuning super specific small models from larger more general ones through a simple process.
## Fine-tuning Models
The LLM-VM uses a data synthesis feature to engage a teacher/student model for fine-tuning super-specific small models from larger more general ones through a simple process.

1. A scalable call is made to OpenAI (or other large parameter LLM)
- 'give me fifty examples like the following query....'
2. The response is parsed into a training set for finetuning
3. The ```small_model``` is trained on the synthezied data set and saved
2. The response is parsed into a training set for fine-tuning
3. The ```small_model``` is trained on the synthesized data set and saved

```python Finetuning a Model
# import our client
Expand All @@ -28,11 +28,11 @@ response=client.complete(prompt="Answer question Q. ",
finetune=True,)
```

### Loading a finetuned LLM from disk
Loading a finetuned model into the LLM-VM for completions is very straightforward.
### Loading a fine-tuned LLM from disk
Loading a fine-tuned model into the LLM-VM for completions is very straightforward.

1. Specify the family of finetuned models in "big_model"
2. Specify the specific filename of the finetuned model you wish to load
1. Specify the family of fine-tuned models in "big_model"
2. Specify the specific filename of the fine-tuned model you wish to load
3. Run the ```load_finetune(model_filename)``` class function to load the model
4. Generate completions in the usual way
```python Loading a Finetuned Model
Expand All @@ -43,7 +43,7 @@ import os
# Instantiate the client specifying which LLM you want to use
client=Client(big_model='pythia')

# specify the file name of the finetuned model to load
# Specify the file name of the finetuned model to load
model_name='<filename_of_your_model>.pt'
client.load_finetune(model_name)

Expand All @@ -56,4 +56,4 @@ print(response)
<Warning>Saving finetuned LLMs can result in a large amount of storage space. Make sure to keep an eye on how much you're saving</Warning>


<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
8 changes: 4 additions & 4 deletions get_started/quickstart/local_llms.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ title: 🧰 Local LLMs
description: 'Here we explain how to choose non-default LLMs for use in the LLM-VM'
---

The LLM-VM suports several default models intended to make experimentation with LLMs accessible to everyone, but if you have the memory required, larger parameter models will perform far better!
The LLM-VM supports several default models intended to make experimentation with LLMs accessible to everyone, but if you have the memory required, larger parameter models will perform far better!

Here is an example if you want to use a large and small neo model for your teacher and student, and you have enough ram:
Here is an example if you want to use a large and small neo model for your teacher and student, and you have enough RAM:
```python Loading Non-default models
# import our client
from llm_vm.client import Client
Expand All @@ -22,6 +22,6 @@ response = client.complete(prompt = 'What is Anarchy?', context = '')
print(response)
# Anarchy is a political philosophy that advocates no government...
```
Now, keep in mind the [gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) is almost 42 GB in size, and would require at least as much RAM to use, in addition to the [gpt-neo-125m](https://huggingface.co/EleutherAI/gpt-neo-125m) which is another ~ 0.5 GB.
Now, keep in mind the [gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) is almost 42 GB in size and would require at least as much RAM to use, in addition to the [gpt-neo-125m](https://huggingface.co/EleutherAI/gpt-neo-125m) which is another ~ 0.5 GB.

<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
16 changes: 8 additions & 8 deletions get_started/quickstart/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ description: 'Our LLM-VM lets get going at your speed with the completion endpoi
# import our client
from llm_vm.client import Client

# Select which LLM you want to use, here we have openAI's
# Select which LLM you want to use, here we have OpenAI's
client=Client(big_model='chat_gpt')

# Put in your prompt and go!
Expand All @@ -20,7 +20,7 @@ response=client.complete(prompt='What is Anarchy?',
print(response)
# Anarchy is a political ideology that advocates for the absence of government...
```
Our Client allows you to interact with OpenAI's completion endpoint seemlessly.
Our Client allows you to interact with OpenAI's completion endpoint seamlessly.


## Locally Run an LLM
Expand Down Expand Up @@ -72,9 +72,9 @@ print(response)
# Anarchy is a political philosophy that advocates no government...
```

## Finetuning
### Finetuning an LLM
**Finetune intelligently with LLM-VM**
## Fine-tuning
### Fine-tuning an LLM
**Fine-tune intelligently with LLM-VM**

Our strategy pairs a small LLM with a larger LLM in a student/teacher relationship.

Expand All @@ -97,9 +97,9 @@ response=client.complete(prompt="Answer question Q. ",
```

### Loading a finetuned LLM from disk
**Quickly test and deploy your finetuned models with LLM-VM**
**Quickly test and deploy your fine-tuned models with LLM-VM**

Just specify the parent model, your finetuned filename, and you're almost there!
Just specify the parent model and your finetuned filename, and you're almost there!
```python
# import our client
from llm_vm.client import Client
Expand All @@ -118,4 +118,4 @@ response=client.complete(prompt='What is anarchy?',
print(response)
```

<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
6 changes: 3 additions & 3 deletions get_started/quickstart/server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@ description: 'Here you can find instructions on setting up our HTTP endpoint for
## LLM-VM Server
### Get up and running

All the functionality described in previous sections can be accessed almost entirely through an API call. First start ther server with the following command.
All the functionality described in previous sections can be accessed almost entirely through an API call. First start the server with the following command.

```bash
llm_vm_server
```
This will spin up the flask server with settings specifed in your ```settings.toml``` file!
This will spin up the flask server with settings specified in your ```settings.toml``` file!

### Specifying your models
**Save your development environment in a ```setting.toml``` file**
Expand All @@ -23,4 +23,4 @@ SMALL_MODEL = "bloom"
HOST = "127.0.0.1"
```
The server pulls from this file to determine which LLMs will be used, allowing for reliable behavior every server launch.
<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
6 changes: 3 additions & 3 deletions get_started/system.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ pip3 install .
pip3 install --editable .
```
</CodeGroup>
<Note>Installing with the ```--editable``` flag allows your changes to the codebase take effect without rebuilding with ```pip3 install .```</Note>
<Note>Installing with the ```--editable``` flag allows your changes to the codebase to take effect without rebuilding with ```pip3 install .```</Note>
### Supported Models
Which model you choose to use will be the biggest impact on your system's requirements.
Which model you choose to use will have the biggest impact on your system's requirements.

LLM-VM comes with support for some default models intended to get people experimenting with LLMs. You can see their specifications here.

Expand All @@ -49,4 +49,4 @@ We recommend making sure you have enough RAM to load the models.

Our section on [Local LLMs](quickstart/local_llms) has more information on specifying LLM models.

<Snippet file="github.mdx" />
<Snippet file="github.mdx" />
10 changes: 5 additions & 5 deletions reference/endpoint/create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The contents of the user group

<ResponseField name="team_id" type="number">
This is the internal ID for this user group. You don't need to record this
information, since you will not need to use it.
information since you will not need to use it.
</ResponseField>

<ResponseField name="token" type="string">
Expand All @@ -44,10 +44,10 @@ The contents of the user group

<ResponseField name="api_environment_tag" type="JSON or null">
This is the environment tag of the user group. Possible values are 'Customer'
and 'Testing'. User group id's must be unique to each environment, so you can
not create multiple user groups with with same id. If you have a production
customer and a test user group with the same id, you will be required to label
one as 'Customer' and another as 'Testing'
and 'Testing'. User group IDs must be unique to each environment, so you can
not create multiple user groups with the same ID. If you have a production
customer and a test user group with the same ID you will be required to label
one as 'Customer' and another as 'Testing'.
</ResponseField>

</Expandable>
Expand Down
Loading