# Host a model on GPUs

Deploy a model-serving container as a GPU-enabled service. This guide connects GPU setup, model configuration, storage, and network access.

> [!note] Requirements
>
> You will need the following to get started:
>
> - A model-serving image with a documented start command and network port
> - Model files or permission to download them from their source
> - The GPU memory and runtime requirements of the model and serving software

### 1. Choose GPU capacity and hosting

Use [Run GPU workloads](run-gpu-workloads) to choose Northflank cloud or your cloud. Compare the model's requirements with the available GPU configuration before deployment.

Use a service for an endpoint that must keep running. Model training or batch inference that ends can use a [job](https://northflank.com/docs/v1/application/run/run-an-image-once-or-on-a-schedule).

### 2. Configure the model service

Deploy the serving image and configure its [GPU resources](https://northflank.com/docs/v1/application/gpu-workloads/configure-and-optimise-workloads-for-gpus). Set the documented command and [runtime variables](https://northflank.com/docs/v1/application/run/inject-runtime-variables) for that image.

If the model source needs credentials, provide them through [secrets](https://northflank.com/docs/v1/application/secure/inject-secrets). Choose how the container obtains model files at startup.

If files must remain between container replacements, review [persistent volumes](https://northflank.com/docs/v1/application/databases-and-persistence/add-a-volume) and their limitations before using them. Do not assume that temporary container storage preserves downloaded model files.

### 3. Configure the endpoint

[Configure the serving port](https://northflank.com/docs/v1/application/network/configure-ports). Choose private access or [public access](https://northflank.com/docs/v1/application/network/expose-your-application) for the application's requirements.

Before exposing an endpoint publicly, configure the application's required authentication and [port security policies](https://northflank.com/docs/v1/application/network/add-security-policies-for-ports). Use a [health check](https://northflank.com/docs/v1/application/observe/configure-health-checks) that reflects whether the model server can handle requests.

### 4. Exercise a representative request

Wait for model loading to complete. Send a request that fits the model server's documented API and inspect the result.

Review [logs](https://northflank.com/docs/v1/application/observe/view-logs) and [metrics](https://northflank.com/docs/v1/application/observe/view-metrics). Make sure that startup time and resource use fit the intended workload before changing capacity or adding traffic.