Skip to content

allow running hackingbuddygpt with llama-style models again #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 24, 2024
Merged

allow running hackingbuddygpt with llama-style models again #30

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
88324d4
capture stderr/stdout over SSH
andreashappe Apr 23, 2024
eef91c7
update readme.md
andreashappe Apr 23, 2024
54c044b
the 'minimal' usecase is actually not an abstrct base class
andreashappe Apr 24, 2024
a9bf484
mark the `perform_round` method as abstract
andreashappe Apr 24, 2024
66e8e10
reintroduce quick hack so that we can test non-openai LLMs easily
andreashappe Apr 24, 2024
b45bc77
Merge branch 'main' of github.com:ipa-lab/hackingBuddyGPT into other_…
andreashappe Apr 24, 2024
File filter

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
reintroduce quick hack so that we can test non-openai LLMs easily
  • Loading branch information
@andreashappe
andreashappe committed Apr 24, 2024
commit 66e8e10eb35f9f9d8fb6c0ec565200d13df665bf
9 changes: 7 additions & 2 deletions utils/openai/openai_llm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
from utils.configurable import configurable, parameter
from utils.llm_util import LLMResult, LLM


@configurable("openai-compatible-llm-api", "OpenAI-compatible LLM API")
@dataclass
class OpenAIConnection(LLM):
Expand Down Expand Up @@ -69,7 +68,13 @@ def get_response(self, prompt, *, retry: int = 0, **kwargs) -> LLMResult:
return LLMResult(result, prompt, result, toc - tic, tok_query, tok_res)

def encode(self, query) -> list[int]:
return tiktoken.encoding_for_model(self.model).encode(query)
# I know this is crappy for all non-openAI models but sadly this
# has to be good enough for now
if self.model.startswith("gpt-"):
encoding = tiktoken.encoding_for_model(self.model)
else:
encoding = tiktoken.encoding_for_model("gpt-3.5-turbo")
return encoding.encode(query)


@configurable("openai/gpt-3.5-turbo", "OpenAI GPT-3.5 Turbo")
Expand Down