Running kotaemon without UI

[michalwelna0]

Hello there!

I have a case where I use kotaemon to answer some questions. However the amount of those questions are going to be large (around 100 up to 200). Moreover I would like to run it with sometimes different parameters (chunk_size, overlap, different reasoning method etc.)

I was thinking if there is a way to actually run it without a UI, because you can imagine that providing 200 questions, copy answers to excel file is tedious (not to mention doing it for different parameters).

Because what I wanted to do is to create some kind of grid search in the backend so I could simply define parameters and run it, questions would be asked, answers saved to excel file for each configuration. But I am struggling to figure out how to do it (of course if it's achievable).

I operate on repository installed without docker. I simply cloned repository, did pip install... as in instruction, indexed some documents using the UI, defined prompts there in the UI, so everything I have prepared for questioning (with the UI) but I wonder if I can automate this.

I was playing also with the cli approach. I managed to export the FullQAPipeline to YML file. I loaded it, the gradio UI was created but when I execute a message it raises an error (apparently it wants tu execute run() method from FullQAPipeline but it is not implemented yet (its parent ABC class has run() raise NotImplementedError.

So any thoughts, suggestions I would appreciate! Especially to tell me that what I want to have is not possible.