Skip to main content
Open In ColabOpen on GitHub

ChatFeatherlessAi

This will help you get started with FeatherlessAi chat models. For detailed documentation of all ChatFeatherlessAi features and configurations head to the API reference.

Overview​

Integration details​

ClassPackageLocalSerializableJS supportPackage downloadsPackage latest
ChatFeatherlessAilangchain-featherless-ai✅❌❌PyPI - DownloadsPyPI - Version

Model features​

Tool callingStructured outputJSON modeImage inputAudio inputVideo inputToken-level streamingNative asyncToken usageLogprobs
❌❌✅❌❌❌✅✅✅❌

Setup​

To access Featherless AI models you'll need to create a/an Featherless AI account, get an API key, and install the langchain-featherless-ai integration package.

Credentials​

Head to https://featherless.ai/ to sign up to FeatherlessAI and generate an API key. Once you've done this set the FEATHERLESSAI_API_KEY environment variable:

import getpass
import os

if not os.getenv("FEATHERLESSAI_API_KEY"):
    os.environ["FEATHERLESSAI_API_KEY"] = getpass.getpass(
        "Enter your FeatherlessAI API key: "
    )

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# os.environ["LANGSMITH_TRACING"] = "true"
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")

Installation​

The LangChain FeatherlessAi integration lives in the langchain-featherless-ai package:

%pip install -qU langchain-featherless-ai
Note: you may need to restart the kernel to use updated packages.

Instantiation​

Now we can instantiate our model object and generate chat completions:

from langchain_featherless_ai import ChatFeatherlessAi

llm = ChatFeatherlessAi(
    model="featherless-ai/Qwerky-72B",
    temperature=0.9,
    max_tokens=None,
    timeout=None,
)

Invocation​

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
c:\Python311\Lib\site-packages\pydantic\main.py:463: UserWarning: Pydantic serializer warnings:
  PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value=1747322408.706, input_type=float])
  return self.__pydantic_serializer__.to_python(
AIMessage(content="J'aime programmer.", additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 5, 'prompt_tokens': 27, 'total_tokens': 32, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'featherless-ai/Qwerky-72B', 'system_fingerprint': '', 'id': 'G1sgui', 'service_tier': None, 'finish_reason': 'stop', 'logprobs': None}, id='run--6ecbe184-c94e-4d03-bf75-9bd85b04ba5b-0', usage_metadata={'input_tokens': 27, 'output_tokens': 5, 'total_tokens': 32, 'input_token_details': {}, 'output_token_details': {}})
print(ai_msg.content)
J'aime programmer.

Chaining​

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)
API Reference:ChatPromptTemplate
c:\Python311\Lib\site-packages\pydantic\main.py:463: UserWarning: Pydantic serializer warnings:
  PydanticSerializationUnexpectedValue(Expected `int` - serialized value may not be as expected [input_value=1747322423.487, input_type=float])
  return self.__pydantic_serializer__.to_python(
AIMessage(content='Ich liebe Programmieren.', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 5, 'prompt_tokens': 22, 'total_tokens': 27, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'featherless-ai/Qwerky-72B', 'system_fingerprint': '', 'id': 'BoBqht', 'service_tier': None, 'finish_reason': 'stop', 'logprobs': None}, id='run--67464357-83d1-4591-9a62-303ed74b8148-0', usage_metadata={'input_tokens': 22, 'output_tokens': 5, 'total_tokens': 27, 'input_token_details': {}, 'output_token_details': {}})

API reference​

For detailed documentation of all ChatFeatherlessAi features and configurations head to the API reference

Was this page helpful?