Important: Before creating a private deployment, work with your Fireworks representative to confirm your capacity needs.
To create a private deployment, run:
firectl create deployment accounts/<account-id>/models/<model-id>
The second argument is the fully-qualified name of the base model used in the deployment. You may use any publicly available base model, which includes those in the
fireworks account, or your own custom base model.
Make note of the deployment ID (the suffix after
accounts/<account-id>/deployments/). You will need this ID to deploy models to this deployment.
NOTE: A base model can have multiple deployments. You must specify which deployment is used when querying a custom base model by explicitly deploying it to a specific deployment ID.
Once the deployment is created, you can now deploy models to it.
firectl deploy <model-id> --deployment-id <deployment-id>
Any PEFT addons you deploy must use the base model specified when you created the deployment.
You may also deploy the base model (if it's not already deployed elsewhere) to the deployment.
Querying a model in a private deployment is identical to using a public deployment. Simply specify the model name in your API request. See Programmatic access for details.
To delete a deployment, run:
firectl delete deployment <deployment-id>
WARNING: This will automatically undeploy all models deployed to this deployment.
Updated 4 months ago