You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Could an '--approach auto' feature be enabled that would start optillm in such a way that requests could be routed automatically to any of the endpoints based upon the sending prefix?
example current config:
--approach bon
then requests are sent as bon-{model}
example auto config:
--approach auto
then requests are sent as bon-{model} or moa-{model} or rto-{model} etc.
if this is already implemented and I am missing the obvious then accept my apologies
Thank you for your amazing code!
--approach {none,mcts,bon,moa,rto,z3,self_consistency,pvg,rstar,cot_reflection,plansearch,leap,re2,cepo,mars,proxy,deepthink,json,deep_research,majority_voting,spl,router,web_search,mcp,readurls,executecode,coc,genselect,memory,longcepo}
Inference approach to use
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
optillm 0.3.12
Could an '--approach auto' feature be enabled that would start optillm in such a way that requests could be routed automatically to any of the endpoints based upon the sending prefix?
example current config:
--approach bon
then requests are sent as bon-{model}
example auto config:
--approach auto
then requests are sent as bon-{model} or moa-{model} or rto-{model} etc.
if this is already implemented and I am missing the obvious then accept my apologies
Thank you for your amazing code!
Beta Was this translation helpful? Give feedback.
All reactions