Skip to content

report.md

Latest commit

 

History

History
51 lines (44 loc) · 4.36 KB

report.md

File metadata and controls

51 lines (44 loc) · 4.36 KB

Report

This report consists of the design decisions and instructions to execute the tasks

Method

Task 1: Conversational Core (Streaming & Cost Telemetry) This is a core task which involves

  1. Using the given model, I used openAI LLM which takes input from the user and responds with a response.
  2. The response is streamed token by token instead of user waiting till all tokens predicted and returned.
  3. The conversation history is preserved for the last 10 messages and the conversation is sent to the LLM which has context of the previous chat before responding.
  4. Once the response is received from the model, I calculate the KPIs like total input tokens, output tokens, cost and latency which is presented to the user on the screen.

Task 2: High-Performance Retrieval-Augmented QA This task is built on top of task 1 where I reused the task 1 components. This task involves

  1. Extracting text from a large pdf.
  2. Creating chunking mechanism where the extracted text is further divided into sentences as cutting the sentence in middle will impact the accuracy of the RAG
  3. The sentences are then divided into tokens, where I fixed the maximum number of tokens with an fixed overlap to preserve the sematic meaning between the sentences.
  4. The chunks are then further embedded into vectors using sentence transformers.
  5. These embeddeding along with the chunks and metadata are loaded into a chromaDB vector store.
  6. Now, the large PDF is divided into smaller chunks and loaded into vector store and the RAG is ready to take input from the user.
  7. The user input is then embedded and compared with the other vectors in the vector store using a similarity search and gets the top 5 relavent chunks.
  8. These relavent chunks along with the prompt is sent to the task 1 workflow which responds with the answer for the user query using the retrived chunks and inline citation.

Task 3: Autonomous Planning Agent with Tool Calling This task reuses the components from the previous tasks. This task involves

  1. This task uses available tools to complete the user query
  2. I have created 2 different tools, a weather tool which gives information regarding the weather details of a particular city and number of days required. The other tool was to get the flight details like price and timing from once city to the other city.
  3. The weather tool uses an external live API and the flights tool uses the mocked data
  4. When user asks a query regarding planning an itenary, the model reasons and looks for the available tools to complete the task.
  5. I have limited the agent reasoning steps to 10 which acts as a gaurdrail avoiding infinite loop which saves the cost and time.
  6. If the user query requires using multiple tools, the model continue reasoning until a final response is generated or the reasoning steps limit is reached.
  7. The modal displays its reasoning as it continous to generate the response.
  8. The final response is outputted as a JSON object where the schema is predefined in the prompt.

Task 4: Self-Healing Code Assistant This is the final task which reuse all the previous tasks components. This task involves

  1. I have created 2 new tools, one for executing the commands in the shell environment and the other tool to save the file in the current working directory to the disk.
  2. Before generating the code, it check the current OS and if python, pytest and rust were already installed in the machine. If they were not installed, the models stops immediatly and notifies the user.
  3. Once all the necessary frameworks were installed, the model then proceed to generate the code and writes to a file on the disk
  4. The model then executes the code and checks for the errors. If there are errors, those errors will be sent to the model as a conversation history and themodel will retry to resolve the issue.
  5. Once the issues are resolved, the model then executes the test cases and display the passes/failed tests to the user. If there are any failed testcases, the model then retries to resolve issue and execute the testcases again.
  6. The number of retries are restricted to 3 and after that the final result will be shown to the user.

Execution steps

  1. Install dependencies using pip install -r requirements.txt
  2. Once all the dependencies installed, execute main python main.py
  3. You will see different options to select.
  4. Depending on which task to execute, select the respective number.