Skip to content

Conversation

@kevinmessiaen
Copy link
Member

Description

Bokeh >= 3.6 cause issue with rendering plots when saving to html file

How to reproduce

Run this code

import os

import giskard
from giskard.llm.client.openai import OpenAIClient

os.environ["OPENAI_API_KEY"] = "sk-..."

giskard.llm.set_llm_api("openai")
oc = OpenAIClient(model="gpt-4-turbo-preview")
giskard.llm.set_default_client(oc)

import pandas as pd
from giskard.rag import KnowledgeBase, generate_testset

# Load your data and initialize the KnowledgeBase
df = pd.read_csv("knowledge_base.csv")


knowledge_base = KnowledgeBase.from_pandas(df, columns=["q", "a"])

# Generate a testset with 10 questions & answers for each question types (this will take a while)
testset = generate_testset(
    knowledge_base,
    num_questions=10,
    language='en',  # optional, we'll auto detect if not provided
    agent_description="A customer support chatbot for company X", # helps generating better questions
)

# Save the generated testset
testset.save("my_testset.jsonl")

# You can easily load it back
from giskard.rag import QATestset

loaded_testset = QATestset.load("my_testset.jsonl")

# Convert it to a pandas dataframe
df = loaded_testset.to_pandas()

#%%
from giskard.rag import evaluate

# Wrap your RAG model
def get_answer_fn(question: str, history=None) -> str:
    """A function representing your RAG agent."""
    # Format appropriately the history for your RAG agent
    messages = history if history else []
    messages.append({"role": "user", "content": question})

    # Get the answer
    answer = "I don't know"  # could be langchain, llama_index, etc.

    return answer


# Run the evaluation and get a report
report = evaluate(get_answer_fn, testset=testset, knowledge_base=knowledge_base)

report.to_html('test.html')

Open the test.html generated file to see that the plot is not rendering when bokeh>3.6

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix
 Bokeh >= 3.6 cause issue with rendering plots when saving to html file
Copy link
Member

@henchaves henchaves left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested with bokeh==3.6.0 and bokeh==3.4.3. Indeed, the plot generated using the version 3.6.0 was broken.

LGTM!

@kevinmessiaen kevinmessiaen merged commit 74553db into main Sep 30, 2024
@kevinmessiaen kevinmessiaen deleted the fix/rag-plot-rendering branch September 30, 2024 01:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants