Skip to content

Testset generation lost when a single generation fail #2022

@kevinmessiaen

Description

@kevinmessiaen

Issue Type

Bug

Source

source

Giskard Library Version

2.15.0

OS Platform and Distribution

Macos

Python version

3.12.4

Installed python packages

No response

Current Behaviour?

When generating a testset, it takes around 45-60 minutes to go thru a doc and produce the topics. 

Then it starts generating the testset but... it's failing around 80% because of JSONDecode error and I have to restart again, including all the topic generation part.

We should implement a solution to save the progress, since it is very likely to have error like this when generating testset from very long docs.

Standalone code OR list down the steps to reproduce the issue

TODO

Relevant log output

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions