Skip to content

feat: local tokenizer #5508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 22, 2025
Merged

feat: local tokenizer #5508

merged 3 commits into from
Oct 22, 2025
+218 −3

Conversation

@jif-oai
Copy link
Collaborator

@jif-oai jif-oai commented Oct 22, 2025

No description provided.

@jif-oai
Copy link
Collaborator Author

jif-oai commented Oct 22, 2025

@codex review
@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
2 similar comments
@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@chatgpt-codex-connector
Copy link
Contributor

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@jif-oai jif-oai merged commit fd0673e into main Oct 22, 2025
20 checks passed
@jif-oai jif-oai deleted the jif/local-tokenizer branch October 22, 2025 15:01
@github-actions github-actions bot locked and limited conversation to collaborators Oct 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

3 participants

@jif-oai @pakrym-oai