Custom base models

Users may upload custom base models in the Hugging Face model format.

Requirements

Fireworks currently supports the following model architectures:

The model files you will need to provide depend on the model architecture. In general, you will need the following files:

  • Model configuration: config.json
  • Model weights, in one of the following formats:
    • *.safetensors
    • *.bin
  • Weights index:*.index.json
  • Tokenizer file(s), e.g.
    • tokenizer.model
    • tokenizer.json
    • tokenizer_config.json

If the requisite files are not present, model deployment may fail.

Uploading a custom base model

To upload a custom base model, ensure all the required files are in a single directory. Then run:

firectl create model <MODEL_ID> /path/to/files

Creating a deployment

Custom base models may only be used with on-demand deployments. Just run:

firectl create deployment <MODEL_ID>

Chat completions

To enable the chat completions API for your custom base model, ensure your tokenizer_config.json contains a chat_template field. See the Hugging Face guide on Templates for Chat Models for details.