Save cost in Langchain pipeline using LLUMO compressor API
getpass
module to safely input our API keys and then store them in environment variables for later use.
getpass
: This module provides a way to securely prompt the user for input without echoing the input back to the screen. This is particularly useful for handling sensitive information like API keys.os
: This module provides a way to interact with the operating system, including setting environment variables.openai_api_key = getpass("Enter your OpenAI API key: ")
: This line prompts the user to enter their OpenAI API key. The input is not displayed on the screen for security reasons.llumo_api_key = getpass("Enter your LLUMO API key: ")
: Similarly, this line prompts the user to enter their LLUMO API key securely.os.environ['OPENAI_API_KEY'] = openai_api_key
: This line stores the OpenAI API key in an environment variable named OPENAI_API_KEY
.os.environ['LLUMO_API_KEY'] = llumo_api_key
: This line stores the LLUMO API key in an environment variable named LLUMO_API_KEY
.del openai_api_key
: This line deletes the variable openai_api_key
from memory to ensure that the API key is not accidentally exposed or misused later in the code.del llumo_api_key
: This line deletes the variable llumo_api_key
for the same reason.load_pdf
, utilizes the PdfReader
class from the PyPDF2
library to achieve this. By looping through each page in the PDF and using the extract_text()
method, it collects all the text and returns it as a single string. This function is a key component of our pipeline, as it allows us to convert the PDF content into a format that can be further processed and analyzed using natural language processing tools.
from PyPDF2 import PdfReader
: This line imports the PdfReader
class from the PyPDF2
library. PdfReader
is used to read and manipulate PDF files.load_pdf
Function:
def load_pdf(pdf):
: This line defines a function named load_pdf
that takes a single argument, pdf
, which is the path to the PDF file we want to read.pdf_reader = PdfReader(pdf)
: This line creates an instance of PdfReader
for the given PDF file. This instance allows us to access the contents of the PDF.text = ""
: This line initializes an empty string variable text
that will be used to accumulate the extracted text from each page of the PDF.for page in pdf_reader.pages:
: This line starts a loop that iterates over each page in the PDF. pdf_reader.pages
is a list of all the pages in the PDF.text += page.extract_text()
: Within the loop, this line extracts the text content from the current page using the extract_text()
method and appends it to the text
variable. This method is provided by the PyPDF2
library and returns the text found on the page.return text
: After the loop has processed all the pages, this line returns the accumulated text as a single string.split_text
uses RecursiveCharacterTextSplitter
, imported above, to achieve this.
from langchain.text_splitter import RecursiveCharacterTextSplitter
: This line imports the RecursiveCharacterTextSplitter
class from the langchain.text_splitter
module. This class is designed to split text into smaller chunks based on specified parameters.split_text
Function:
def split_text(text):
: This line defines a function named split_text
that takes a single argument, text
, which is the large block of text we want to split into smaller chunks.text_splitter = RecursiveCharacterTextSplitter(
: This line initializes an instance of RecursiveCharacterTextSplitter
with specific parameters:
chunk_size=1000
: This parameter sets the maximum size of each text chunk to 1000 characters. This ensures that each chunk is not too large to handle efficiently.chunk_overlap=200
: This parameter sets the overlap between consecutive chunks to 200 characters. Overlapping chunks can help maintain context between chunks, which can be important for tasks like text analysis and question answering.length_function=len
: This parameter specifies the function used to measure the length of the text. In this case, the built-in len
function is used, which measures the length in characters.return text_splitter.split_text(text=text)
: This line uses the split_text
method of the RecursiveCharacterTextSplitter
instance to split the input text
into smaller chunks based on the specified parameters. The method returns a list of text chunks.load_embeddings
uses the OpenAIEmbeddings
class and the FAISS
vector store.
from langchain_openai import OpenAIEmbeddings
: This import statement brings in the OpenAIEmbeddings
class, which is used to generate embeddings for text using OpenAI’s models.from langchain_community.vectorstores import FAISS
: This import statement brings in the FAISS
class, which is used to create and manage a vector store for efficient similarity search.load_embeddings
Function:
def load_embeddings(store_name, chunks):
: This line defines a function named load_embeddings
that takes two arguments:
store_name
: A placeholder for the name of the vector store (not used in this specific function but can be useful for future extensions).chunks
: A list of text chunks for which embeddings will be generated.embeddings = OpenAIEmbeddings()
: This line initializes an instance of the OpenAIEmbeddings
class. This instance will be used to generate embeddings for the text chunks.vector_store = FAISS.from_texts(chunks, embedding=embeddings)
: This line creates a FAISS
vector store from the list of text chunks. The from_texts
method generates embeddings for each chunk using the embeddings
instance and stores them in the FAISS vector store. This vector store enables efficient similarity search, allowing us to quickly find the most relevant text chunks based on query embeddings.return vector_store
: This line returns the created FAISS vector store. The vector store contains the embeddings for all the text chunks and is ready for similarity search operations.os.getenv()
. The API key is essential for authenticating the request to the LLUMO API.application/json
) and the authorization token (Bearer {LLUMO_API_KEY}
).raise_for_status()
method raises an exception if the request fails.main
function serves as the entry point for our PDF Query Assistant. It integrates all the steps discussed previously, from uploading and processing a PDF file to querying the content and using LLUMO compression to optimize costs. We will go in details of each part of the main
function:
files.upload()
to upload a PDF file.load_pdf
function to extract text from the PDF.split_text
function to split the extracted text into manageable chunks.load_embeddings
function to create a vector store using the text chunks.compress_with_llumo
function to compress the context using LLUMO API.ChatOpenAI
) with specified parameters.load_qa_chain
.main
function. This comprehensive function handles PDF uploading, text extraction, text chunking, embedding generation, similarity search, text compression with LLUMO, and generating a response to the user’s query using the compressed or original text.