Giskard-AI · davidberenstein1957 · Apr 22, 2025 · Mar 27, 2025 · Mar 31, 2025 · Mar 31, 2025
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -16,7 +16,7 @@ repos:
         args:
           - "--config"
           - "pyproject.toml"
-        stages: [commit]
+        stages: [pre-commit]
   - repo: https://github.com/astral-sh/ruff-pre-commit
     rev: v0.1.6
     hooks:
@@ -33,4 +33,4 @@ repos:
     hooks:
       - id: ggshield
         language_version: python3
-        stages: [commit]
+        stages: [pre-commit]
diff --git a/README.md b/README.md
@@ -33,11 +33,11 @@ We officially support Python 3.9, 3.10 and 3.11.
 
 ______________________________________________________________________
 
-Giskard is an open-source Python library that **automatically detects performance, bias & security issues in AI applications**. The library covers LLM-based applications such as RAG agents, all the way to traditional ML models for tabular data. 
+Giskard is an open-source Python library that **automatically detects performance, bias & security issues in AI applications**. The library covers LLM-based applications such as RAG agents, all the way to traditional ML models for tabular data.
 
 ## Scan: Automatically assess your LLM-based agents for performance, bias & security issues ⤵️
 
-Issues detected include: 
+Issues detected include:
 - Hallucinations
 - Harmful content generation
 - Prompt injection
@@ -63,20 +63,20 @@ If you're testing a RAG application, you can get an even more in-depth assessmen
     - `Rewriter`: rewrite the user query to make it more relevant to the knowledge base or to account for chat history
     - `Router`: filter the query of the user based on his intentions
     - `Knowledge Base`: the set of documents given to the RAG to generate the answers
-  
+
 <p align="center">
   <img src="https://raw.githubusercontent.com/giskard-ai/giskard/main/readme/RAGET_updated.gif" alt="Test Suite Example" width="800">
 </p>
 
 
 Giskard works with any model, in any environment and integrates seamlessly with your favorite tools ⤵️ <br/>
- 
+
 <p align="center">
   <img width='600' src="https://raw.githubusercontent.com/giskard-ai/giskard/main/readme/tools_updated.png">
 </p>
 <br/>
 
-Looking for solutions to evaluate computer vision models? Check out [giskard-vision](https://github.com/Giskard-AI/giskard-vision), a library dedicated for computer vision tasks. 
+Looking for solutions to evaluate computer vision models? Check out [giskard-vision](https://github.com/Giskard-AI/giskard-vision), a library dedicated for computer vision tasks.
 
 # Contents
 
@@ -94,13 +94,13 @@ Let's build an agent that answers questions about climate change, based on the 2
 
 Before starting let's install the required libraries:
 ```sh
-pip install langchain tiktoken "pypdf<=3.17.0"
+pip install langchain-openai tiktoken "pypdf<=3.17.0"
 ```
 
 
 ```python
-from langchain import OpenAI, FAISS, PromptTemplate
-from langchain.embeddings import OpenAIEmbeddings
+from langchain import FAISS, PromptTemplate
+from langchain_openai import OpenAIEmbeddings, OpenAI
 from langchain.document_loaders import PyPDFLoader
 from langchain.chains import RetrievalQA
 from langchain.text_splitter import RecursiveCharacterTextSplitter
@@ -193,7 +193,7 @@ knowledge_base = KnowledgeBase.from_pandas(df, columns=["column_1", "column_2"])
 
 # Generate a testset with 10 questions & answers for each question types (this will take a while)
 testset = generate_testset(
-    knowledge_base, 
+    knowledge_base,
     num_questions=60,
     language='en',  # optional, we'll auto detect if not provided
     agent_description="A customer support chatbot for company X", # helps generating better questions
@@ -250,6 +250,6 @@ We thank the following companies which are sponsoring our project with monthly d
 **[Biolevate](https://www.biolevate.com/)**
 
 <img src="https://awsmp-logos.s3.amazonaws.com/seller-wgamx5z6umune/2d10badd2ccac49699096ea7fb986b98.png" alt="Biolevate logo" width="400"/>
- 
+
 
 
diff --git a/docs/community/contribution_guidelines/dev-environment.md b/docs/community/contribution_guidelines/dev-environment.md
@@ -2,24 +2,37 @@
 
 After cloning the repository here are the useful commands to set up the local environment:
 
+## Install dependencies
+
+We work with [PDM](https://pdm-project.org/en/latest/) as our package manager. You can find the installation instructions [here](https://pdm-project.org/en/latest/#installation).
+
+After installing PDM, we can install the dependencies by running:
+
+```shell
+pdm install
+```
+
 ## Pre-commit hooks installation
-Pre-commit hooks ensure that the basic code quality checks are performed before the code is committed
+Pre-commit hooks ensure that the basic code quality checks are performed before the code is committed. Pre-commit has already been installed as a dev dependency using `pdm`.
 
-The current pre-commit hool configuration is defined in `.pre-commit-config.yaml`
+The current pre-commit cool configuration is defined in `.pre-commit-config.yaml`.
 
 To install it run:
 
 ```shell
-brew install pre-commit
 pre-commit install
 ```
 
+Make sure to comment out `skip: true` in the `.pre-commit-config.yaml` file to enable the pre-commit hooks for `ggshield` to use GitGuardian.
+
 ## Run pre-commit hook manually to fix easy issues
-In case the build is failing because of the pre-commit checks that don't pass it's possible to fix easy issues by running
-```sh
+In case the build is failing because of the pre-commit checks that don't pass it's possible to fix easy issues by running:
+
+```shell
 pre-commit run --all-files
 ```
-and then committing the fixed files
+
+The pre-commit hooks are also automatically run when committing.
 
 ## Troubleshooting
 
@@ -28,7 +41,7 @@ and then committing the fixed files
 This is probably caused by a [known issue](https://github.com/microsoft/LightGBM/issues/4707) with `libomp>=12` and LightGBM on MacOS.
 You can avoid this problem by downgrading `libomp` to version 11.1.0, which is compatible with LightGBM.
 
-```sh
+```shell
 $ wget https://raw.githubusercontent.com/Homebrew/homebrew-core/fb8323f2b170bd4ae97e1bac9bf3e2983af3fdb0/Formula/libomp.rb
 $ brew install ./libomp.rb
 ```
diff --git a/docs/conf.py b/docs/conf.py
@@ -129,8 +129,8 @@
 
 sys.path.insert(0, os.path.abspath("../"))
 
-pygments_style = "github-dark"
-pygments_dark_style = "material"
+pygments_style = "default"
+pygments_dark_style = "native"
 
 
 # make github links resolve

diff --git a/docs/index.md b/docs/index.md
@@ -69,7 +69,6 @@ reference/rag-toolset/index
 reference/tests/index
 reference/slicing-functions/index
 reference/transformation-functions/index
-reference/push/index
 reference/suite/index
 ```
 

diff --git a/docs/integrations/avid/avid-integration-llm.ipynb b/docs/integrations/avid/avid-integration-llm.ipynb
@@ -50,7 +50,7 @@
       },
       "outputs": [],
       "source": [
-        "%pip install langchain pypdf faiss-cpu openai tiktoken avidtools"
+        "%pip install langchain-openai pypdf faiss-cpu openai tiktoken avidtools"
       ]
     },
     {
@@ -99,11 +99,10 @@
       },
       "outputs": [],
       "source": [
-        "from langchain.llms import OpenAI\n",
         "from langchain.prompts import PromptTemplate\n",
         "from langchain.vectorstores import FAISS\n",
         "from langchain.chains import RetrievalQA\n",
-        "from langchain.embeddings import OpenAIEmbeddings\n",
+        "from langchain_openai import OpenAIEmbeddings, OpenAI\n",
         "from langchain.document_loaders import PyPDFLoader\n",
         "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
         "\n",

diff --git a/docs/knowledge/catalogs/test-catalog/text_generation/index.rst b/docs/knowledge/catalogs/test-catalog/text_generation/index.rst
@@ -1,5 +1,5 @@
 Text generation tests
-=================
+=====================
 
 - **Injections**
 
@@ -14,9 +14,9 @@ Text generation tests
   - :func:`~.giskard.testing.tests.llm.test_llm_output_against_requirement_per_row`
   - :func:`~.giskard.testing.tests.llm.test_llm_single_output_against_requirement`
   - :func:`~.giskard.testing.tests.llm.test_llm_output_against_requirement`
-  
+
 - **Ground Truth**
-  
+
   - :func:`~.giskard.testing.tests.llm.test_llm_ground_truth`
   - :func:`~.giskard.testing.tests.llm.test_llm_ground_truth_similarity`
   - :func:`~.giskard.testing.tests.llm.test_llm_as_a_judge_ground_truth_similarity`

diff --git a/docs/knowledge/llm_vulnerabilities/index.rst b/docs/knowledge/llm_vulnerabilities/index.rst
@@ -19,6 +19,17 @@ Large Language Model (LLM) vulnerabilities are different from those in tradition
 
 By conducting a `Giskard Scan <../../open_source/scan/index.md>`_, you can proactively identify and address these vulnerabilities to ensure the reliability, fairness, and robustness of your LLMs.
 
+.. toctree::
+   :maxdepth: 1
+   :hidden:
+
+   hallucination/index
+   harmfulness/index
+   injection/index
+   robustness/index
+   formatting/index
+   disclosure/index
+   stereotypes/index
 
 How does the LLM Scan work?
 ===========================

diff --git a/docs/reference/index.rst b/docs/reference/index.rst
@@ -1,5 +1,5 @@
 API Reference
-===============
+=============
 
 .. toctree::
    :maxdepth: 1
@@ -10,6 +10,5 @@ API Reference
    tests/index
    slicing-functions/index
    transformation-functions/index
-   push/index
    suite/index
    rag-toolset/index