Home
Guides

API Reference

base_completion

BaseCompletion Objects

class BaseCompletion()

Base class for handling completions. This class provides shared logic for creating completions,
both synchronously and asynchronously, and both streaming and non-streaming.

Attributes:

  • endpoint str - API endpoint for the completion request.
  • response_class Type - Class used for parsing the non-streaming response.
  • stream_response_class Type - Class used for parsing the streaming response.

create

@classmethod
def create(cls,
           model,
           prompt_or_messages=None,
           request_timeout=600,
           stream=False,
           **kwargs)

Create a completion or chat completion.

Arguments:

  • model str - Model name to use for the completion.
  • prompt_or_messages Union[str, List[ChatMessage]] - The prompt for Completion or a list of chat messages for ChatCompletion. If not specified, must specify either prompt or messages in kwargs.
  • request_timeout int, optional - Request timeout in seconds. Defaults to 600.
  • stream bool, optional - Whether to use streaming or not. Defaults to False.
  • **kwargs - Additional keyword arguments.

Returns:

Union[CompletionResponse, Generator[CompletionStreamResponse, None, None]]:
Depending on the stream argument, either returns a CompletionResponse
or a generator yielding CompletionStreamResponse.

acreate

@classmethod
def acreate(cls, model, *args, request_timeout=600, stream=False, **kwargs)

Asynchronously create a completion.

Arguments:

  • model str - Model name to use for the completion.
  • request_timeout int, optional - Request timeout in seconds. Defaults to 600.
  • stream bool, optional - Whether to use streaming or not. Defaults to False.
  • **kwargs - Additional keyword arguments.

Returns:

Union[CompletionResponse, AsyncGenerator[CompletionStreamResponse, None]]:
Depending on the stream argument, either returns a CompletionResponse or an async generator yielding CompletionStreamResponse.

completion

Completion Objects

class Completion(BaseCompletion)

Class for handling text completions.

chat_completion

ChatCompletion Objects

class ChatCompletion(BaseCompletion)

Class for handling chat completions.

api

Choice Objects

class Choice(BaseModel)

A completion choice.

Attributes:

  • index int - The index of the completion choice.
  • text str - The completion response.
  • logprobs float, optional - The log probabilities of the most likely tokens.
  • finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

CompletionResponse Objects

class CompletionResponse(BaseModel)

The response message from a /v1/completions call.

Attributes:

  • id str - A unique identifier of the response.
  • object str - The object type, which is always "text_completion".
  • created int - The Unix time in seconds when the response was generated.
  • choices List[Choice] - The list of generated completion choices.

CompletionResponseStreamChoice Objects

class CompletionResponseStreamChoice(BaseModel)

A streamed completion choice.

Attributes:

  • index int - The index of the completion choice.
  • text str - The completion response.
  • logprobs float, optional - The log probabilities of the most likely tokens.
  • finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

CompletionStreamResponse Objects

class CompletionStreamResponse(BaseModel)

The streamed response message from a /v1/completions call.

Attributes:

  • id str - A unique identifier of the response.
  • object str - The object type, which is always "text_completion".
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
    choices (List[CompletionResponseStreamChoice]):
    The list of streamed completion choices.

Model Objects

class Model(BaseModel)

A model deployed to the Fireworks platform.

Attributes:

  • id str - The model name.
  • object str - The object type, which is always "model".
  • created int - The Unix time in seconds when the model was generated.

ListModelsResponse Objects

class ListModelsResponse(BaseModel)

The response message from a /v1/models call.

Attributes:

  • object str - The object type, which is always "list".
  • data List[Model] - The list of models.

ChatMessage Objects

class ChatMessage(BaseModel)

A chat completion message.

Attributes:

  • role str - The role of the author of this message.
  • content str - The contents of the message.

ChatCompletionResponseChoice Objects

class ChatCompletionResponseChoice(BaseModel)

A chat completion choice generated by a chat model.

Attributes:

  • index int - The index of the chat completion choice.
  • message ChatMessage - The chat completion message.
  • finish_reason Optional[str] - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

UsageInfo Objects

class UsageInfo(BaseModel)

Usage statistics.

Attributes:

  • prompt_tokens int - The number of tokens in the prompt.
  • total_tokens int - The total number of tokens used in the request (prompt + completion).
  • completion_tokens Optional[int] - The number of tokens in the generated completion.

ChatCompletionResponse Objects

class ChatCompletionResponse(BaseModel)

The response message from a /v1/chat/completions call.

Attributes:

  • id str - A unique identifier of the response.
  • object str - The object type, which is always "chat.completion".
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
  • choices List[ChatCompletionResponseChoice] - The list of chat completion choices.
  • usage UsageInfo - Usage statistics for the chat completion.

DeltaMessage Objects

class DeltaMessage(BaseModel)

A message delta.

Attributes:

  • role str - The role of the author of this message.
  • content str - The contents of the chunk message.

ChatCompletionResponseStreamChoice Objects

class ChatCompletionResponseStreamChoice(BaseModel)

A streamed chat completion choice.

Attributes:

  • index int - The index of the chat completion choice.
  • delta DeltaMessage - The message delta.
  • finish_reason str - The reason the model stopped generating tokens. This will be "stop" if the model hit a natural stop point or a provided stop sequence, or "length" if the maximum number of tokens specified in the request was reached.

ChatCompletionStreamResponse Objects

class ChatCompletionStreamResponse(BaseModel)

The streamed response message from a /v1/chat/completions call.

Attributes:

  • id str - A unique identifier of the response.
  • object str - The object type, which is always "chat.completion".
  • created int - The Unix time in seconds when the response was generated.
  • model str - The model used for the chat completion.
    choices (List[ChatCompletionResponseStreamChoice]):
    The list of streamed chat completion choices.

model

Model Objects

class Model()

list

@classmethod
def list(cls, request_timeout=60)

Returns a list of available models.

Arguments:

  • request_timeout int, optional - The request timeout in seconds. Default is 60.

Returns:

  • ListModelsResponse - A list of available models.

log

set_console_log_level

def set_console_log_level(level: str) -> None

Controls console logging.

Arguments:

  • level - the minimum level that prints out to console.
    Supported values: [CRITICAL, FATAL, ERROR, WARN,
    WARNING, INFO, DEBUG]

error

PermissionError Objects

class PermissionError(FireworksError)

A permission denied error.

InvalidRequestError Objects

class InvalidRequestError(FireworksError)

A invalid request error.

AuthenticationError Objects

class AuthenticationError(FireworksError)

A authentication error.

RateLimitError Objects

class RateLimitError(FireworksError)

A rate limit error.

InternalServerError Objects

class InternalServerError(FireworksError)

An internal server error.

ServiceUnavailableError Objects

class ServiceUnavailableError(FireworksError)

A service unavailable error.