Save cost in Llama Index pipeline using LLUMO compressor API
import os
: This module provides a way to interact with the operating system, including setting and retrieving environment variables.
from getpass import getpass
: This module allows us to securely prompt the user for sensitive information, such as API keys, without echoing the input back to the screen.
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext, Document
: These classes are core components of LlamaIndex. They are used for creating and querying vector indexes, reading documents, and configuring the indexing and querying process.
from llama_index.llms import OpenAI
: This class is used to interact with OpenAI’s API for generating responses from their language models.
import requests
: This library is used for making HTTP requests, which we will need to communicate with external APIs, including the LLUMO Compressor API.
import json
: This library is used for parsing and manipulating JSON data, which is often the format of data exchanged between APIs.
getpass
module to safely input our API keys and then store them in environment variables for later use.
getpass
: This module provides a way to securely prompt the user for input without echoing the input back to the screen. This is particularly useful for handling sensitive information like API keys.os
: This module provides a way to interact with the operating system, including setting environment variables.openai_api_key = getpass("Enter your OpenAI API key: ")
: This line prompts the user to enter their OpenAI API key. The input is not displayed on the screen for security reasons.llumo_api_key = getpass("Enter your LLUMO API key: ")
: Similarly, this line prompts the user to enter their LLUMO API key securely.os.environ['OPENAI_API_KEY'] = openai_api_key
: This line stores the OpenAI API key in an environment variable named OPENAI_API_KEY
.os.environ['LLUMO_API_KEY'] = llumo_api_key
: This line stores the LLUMO API key in an environment variable named LLUMO_API_KEY
.del openai_api_key
: This line deletes the variable openai_api_key
from memory to ensure that the API key is not accidentally exposed or misused later in the code.del llumo_api_key
: This line deletes the variable llumo_api_key
for the same reason.load_and_process_pdf(file_path):
: This line defines a function named load_and_process_pdf
that takes a single argument, file_path
, which is the path to the PDF file we want to read and process.raw_documents = SimpleDirectoryReader(input_files=[file_path]).load_data()
: This line uses LlamaIndex’s SimpleDirectoryReader
to load the PDF file. It returns a list of Document
objects, each representing a page or section of the PDF.Document
object in raw_documents
and split its text content into smaller chunks. This is done to ensure that each chunk is of a manageable size for processing and indexing.text_chunks = [doc.text[i:i+1000] for i in range(0, len(doc.text), 800)]
: This line creates chunks of 1000 characters with an overlap of 200 characters between chunks.documents.extend([Document(text=chunk) for chunk in text_chunks])
: This line creates new Document
objects for each text chunk and adds them to the documents
list.service_context = ServiceContext.from_defaults(llm=OpenAI(temperature=0, model_name="gpt-3.5-turbo"))
: This line sets up the service context for LlamaIndex, specifying the OpenAI model to use for text generation.index = VectorStoreIndex.from_documents(documents, service_context=service_context)
: This line creates a VectorStoreIndex
from the processed documents, using the specified service context.return index
: After processing and indexing the PDF content, this line returns the created index.os.getenv()
. The API key is essential for authenticating the request to the LLUMO API.application/json
) and the authorization token (Bearer {LLUMO_API_KEY}
).raise_for_status()
method raises an exception if the request fails.main
function serves as the entry point for our PDF Query Assistant. It integrates all the steps discussed previously, from uploading and processing a PDF file to querying the content and using LLUMO compression to optimize costs. We will go in details of each part of the main
function:
files.upload()
to upload a PDF file.load_and_process_pdf
function to extract text from the PDF and convert into
chunks.
compress_with_llumo
function to compress the context using LLUMO API.ChatOpenAI
) with specified parameters.main
function. This comprehensive function handles PDF uploading, text extraction, text chunking, embedding generation, similarity search, text compression with LLUMO, and generating a response to the user’s query using the compressed or original text.