This document will guide you through integrating LLUMO AI API into your codebase and performing evaluations with both default and custom analytics. Follow the steps below to start using LLUMO AI’s evaluation capabilities within your application.

​
Overview

LLUMO AI provides a powerful API that helps you evaluate and analyze text inputs. By sending a request to the LLUMO API, you can obtain evaluation metrics such as confidence, clarity, and context. You can also customize these evaluations by defining your own analytics criteria in your workspace. The following sections cover integration details, including how to integrate the LLUMO API into your code and customize the evaluation process.

​
Integration Steps

​
Step 1: Log In to LLUMO AI

  1. Sign in to your LLUMO AI account https://app.llumo.ai using your credentials.
  1. Navigate to the “Eval LM” section and Go to “Connect API” where you can start the evaluation process for your AI model.
  2. Now, copy the code and Setup prompt evaluation API in your codebase.

​
Step 2: Install Required Libraries

Before integrating LLUMO AI’s API, ensure you have the requests library installed. You can install it using pip:

pip install requests

​
Step 3: Basic Setup

To integrate LLUMO API into your code, you’ll need to send a POST request to the endpoint provided by LLUMO AI.

​
API Endpoint & Authorization

  • API URL: https://app.llumo.ai/api/create-eval-analytics
  • Authorization: Use your unique API token in the Authorization header. Replace Bearer <your-token> with your actual Bearer key.

Example Header:

headers = {
    "Authorization": "Bearer <your-token>",
    "Content-Type": "application/json"
}

​
Request Body Format

The request body is a JSON object where you provide essential parameters like your workspaceID, the prompt, input, and output.

Example of default evaluation request:

data = {
    "prompt": "I am planning a trip to {{location}}, give me a travel plan",
    "input": {
        "location": "Germany"
    },
    "output": "Absolutely, Germany is a wonderful destination with a rich history, diverse culture, and beautiful landscapes. Here's a suggested travel plan for a memorable trip: Day 1-3: Berlin Day 1: Arrive in Berlin, check into your accommodation, and get acquainted with the city. Visit iconic landmarks such as the Brandenburg Gate, the Reichstag Building, and Checkpoint Charlie. Day 2: Explore Berlin's cultural scene. Visit Museum Island, home to several world-renowned museums including the Pergamon Museum and the Altes Museum. Take a stroll along the East Side Gallery to see remnants of the Berlin Wall adorned with street art. Day 3: Dive deeper into Berlin's history with a visit to the Holocaust Memorial and the Topography of Terror museum. Spend the evening exploring the vibrant nightlife of neighborhoods like Kreuzberg or Friedrichshain."
}

Explanation of fields:

  • prompt: Text or question you want to analyze.
  • input: Language or other settings (currently only language is supported).
  • output: The result or analysis based on your input prompt.

​
Step 4: Making the API Call

Use the requests.post() method to send your data to the API. Below is a sample code that demonstrates how to make the request.

Example Code:

import requests

# URL and headers
url = "https://app.llumo.ai/api/create-eval-analytics"
headers = {
    "Authorization": "Bearer <your-token>",
    "Content-Type": "application/json"
}

# Request body
data = {
    "prompt": "I am planning a trip to {{location}}, give me a travel plan",
    "input": {
        "location": "Germany"
    },
    "output": "Absolutely, Germany is a wonderful destination with a rich history, diverse culture, and beautiful landscapes. Here's a suggested travel plan for a memorable trip: Day 1-3: Berlin Day 1: Arrive in Berlin, check into your accommodation, and get acquainted with the city. Visit iconic landmarks such as the Brandenburg Gate, the Reichstag Building, and Checkpoint Charlie. Day 2: Explore Berlin's cultural scene. Visit Museum Island, home to several world-renowned museums including the Pergamon Museum and the Altes Museum. Take a stroll along the East Side Gallery to see remnants of the Berlin Wall adorned with street art. Day 3: Dive deeper into Berlin's history with a visit to the Holocaust Memorial and the Topography of Terror museum. Spend the evening exploring the vibrant nightlife of neighborhoods like Kreuzberg or Friedrichshain.",
}

# Make the POST request
response = requests.post(url, headers=headers, json=data)

# Output the response
print("Status Code:", response.status_code)
print("Response:", response.json())

​
Default Evaluation Metrics

Upon making a request to the LLUMO AI API, the response will include the following default evaluation metrics:

  • Confidence: Measures the confidence level of the generated output.
  • Clarity: Assesses the clarity of the generated text.
  • Context: Analyzes whether the output aligns with the context provided in the prompt.
  • Overall Score: A combined score based on the above metrics.

Example Response:

{
    "status": "success",
    "metrics": {
        "confidence": 0.92,
        "clarity": 0.85,
        "context": 0.88,
        "overallScore": 0.87
    }
}

​
Custom Evaluation Metrics

If you want to evaluate your data based on custom-defined analytics in your workspace, you can add a custom analytics name to the request body.

​
Adding Custom Analytics

To specify custom analytics, include the "analytics" field in the request body with an array of your custom analytics names.

Example:

data = {
    "prompt": "I am planning a trip to {{location}}, give me a travel plan",
    "input": {
        "location": "Germany"
    },
    "output": "Absolutely, Germany is a wonderful destination with a rich history, diverse culture, and beautiful landscapes. Here's a suggested travel plan for a memorable trip: Day 1-3: Berlin Day 1: Arrive in Berlin, check into your accommodation, and get acquainted with the city. Visit iconic landmarks such as the Brandenburg Gate, the Reichstag Building, and Checkpoint Charlie. Day 2: Explore Berlin's cultural scene. Visit Museum Island, home to several world-renowned museums including the Pergamon Museum and the Altes Museum. Take a stroll along the East Side Gallery to see remnants of the Berlin Wall adorned with street art. Day 3: Dive deeper into Berlin's history with a visit to the Holocaust Memorial and the Topography of Terror museum. Spend the evening exploring the vibrant nightlife of neighborhoods like Kreuzberg or Friedrichshain.",
    "analytics": ["Perplexity"]  # Custom analytics
}

In this case, "Perplexity" is a custom analytics name that you have set up in your workspace.

​
Custom Analytics Configuration

Before using custom analytics, ensure they are set up within your workspace on the LLUMO platform. If the analytics name is valid and configured, the API will return metrics for your custom analysis. You can refer to the guide on setting up custom analytics in the Custom Eval Playground here.

Example of Full API Call with Custom Analytics:

import requests

# URL and headers
url = "https://app.llumo.ai/api/create-eval-analytics"
headers = {
    "Authorization": "Bearer <your-token>",
    "Content-Type": "application/json"
}

# Request body with custom analytics
data = {
    "prompt": "I am planning a trip to {{location}}, give me a travel plan",
    "input": {
        "location": "Germany"
    },
    "output": "Absolutely, Germany is a wonderful destination with a rich history, diverse culture, and beautiful landscapes. Here's a suggested travel plan for a memorable trip: Day 1-3: Berlin Day 1: Arrive in Berlin, check into your accommodation, and get acquainted with the city. Visit iconic landmarks such as the Brandenburg Gate, the Reichstag Building, and Checkpoint Charlie. Day 2: Explore Berlin's cultural scene. Visit Museum Island, home to several world-renowned museums including the Pergamon Museum and the Altes Museum. Take a stroll along the East Side Gallery to see remnants of the Berlin Wall adorned with street art. Day 3: Dive deeper into Berlin's history with a visit to the Holocaust Memorial and the Topography of Terror museum. Spend the evening exploring the vibrant nightlife of neighborhoods like Kreuzberg or Friedrichshain.",
    "analytics": ["Perplexity"]  # Custom analytics
}

# Make the POST request
response = requests.post(url, headers=headers, json=data)

# Output the response
print("Status Code:", response.status_code)
print("Response:", response.json())

​
Sample Response for Custom Analytics

{
    "status": "success",
    "analyticsScore": {
        "Perplexity": 0.89,
        "confidence": 0.91,
        "clarity": 0.87,
        "context": 0.84,
        "overallScore": 0.89
    }
}

​
Conclusion

By following this documentation, you should be able to easily integrate LLUMO AI API into your codebase for evaluating text outputs. You can choose to use the default evaluation metrics or customize your evaluation by adding analytics configurations in your workspace. For any issues or further questions, consult the LLUMO AI API documentation or contact support for assistance.

​
FAQ

Q. How can I use custom evaluation metrics with the LLUMO API?

A. To use custom evaluation metrics, you need to add the analytics field in the request body when making an API call. For example, you can specify custom metrics like "Perplexity" by including the following in the request body:

"analytics": ["Perplexity"]

You can check our custom analytics support doc for more details.

Was this page helpful?

Suggest edits
Evaluate on 3-4 metricsRun Evaluation in Production