Private deployments

Creating a private deployment

Important: Before creating a private deployment, work with your Fireworks representative to confirm your capacity needs.

To create a private deployment, run:

firectl create deployment accounts/<account-id>/models/<model-id>

The second argument is the fully-qualified name of the base model used in the deployment. You may use any publicly available base model, which includes those in the fireworks account, or your own custom base model.

Make note of the deployment ID (the suffix after accounts/<account-id>/deployments/). You will need this ID to deploy models to this deployment.

NOTE: A base model can have multiple deployments. You must specify which deployment is used when querying a custom base model by explicitly deploying it to a specific deployment ID.

Deploying models

Once the deployment is created, you can now deploy models to it.

firectl deploy <model-id> --deployment-id <deployment-id>

Any PEFT addons you deploy must use the base model specified when you created the deployment.

You may also deploy the base model (if it's not already deployed elsewhere) to the deployment.

Querying the model

Querying a model in a private deployment is identical to using a public deployment. Simply specify the model name in your API request. See Programmatic access for details.

Deleting a private deployment

To delete a deployment, run:

firectl delete deployment <deployment-id>

WARNING: This will automatically undeploy all models deployed to this deployment.