Pay for exactly what you use

With Inferless, you only pay for the compute resources used to run your models. Our pricing is based on two factors:
‍
• Duration - You are charged for the total number of seconds your models are running in a healthy state, rounded up to the nearest second. The duration is calculated from when your model starts loading until it finishes processing a request.
• Machine type - We offer different machine types like A100, A10 and T4 GPUs. The price per second varies based on the machine type you choose for your model. More powerful machines cost more per second.

For example, let’s say you have kept autoscaling on for 2 machines as maximum concurrency and used dedicated A100 80GB as machine type.

It runs on 1 machine for 14,400 sec (4 hrs) and 2 machines for 10,800 sec (3 hrs).
With A100 costing $0.0014/sec, total usage is 25,200 sec.
Your monthly bill is 25,200 * $0.0014 = $35.28.

You can deploy any machine learning model that runs on GPU workloads. For example, any compute-intensive deep learning workloads across various applications from computer vision, NLP, recommendations and scientific computing. Some of the most popular models that our users have deployed with us are Llama2 13Bn, Stable diffusion control-net,Vicuna 7B etc

The distinction between shared and dedicated instances revolves around resource allocation and performance. Shared instances allocate GPU resources among several users, offering variable performance at a cost-effective rate, making them suitable for smaller or infrequent tasks. In contrast, dedicated instances grant users exclusive access to an entire GPU, delivering consistent high performance but at a higher cost. This setup is optimal for large-scale tasks or when data isolation is a priority. Your choice should hinge on workload demands, desired performance, and budget.

We provide a $30 free credit to help you kickstart. Since, we are currently in private beta, your use-case & stage needs to match with our criteria. You can read more about it here.

Customer data and privacy is our top priority. Inferless execution environments are completely isolated from each other using Docker containerization. This prevents any interaction between individual customer environments.
All log streams are separated securely using AWS CloudWatch Logs access controls. Logs are retained only for 30 days and then deleted as per our strict data retention policies.
The storage used for model hosting is encrypted using AES-256 encryption. Models and data are not shared across customers.

With Inferless, you only pay for what you use. When minimum replicas is set to zero, no machines are spun up. So you are not charged when there are no inference requests.

We run our workload on Nvidia on A100, A10, and T4 GPUs, so that you can have a blazing-fast inference.