> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nscale.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat

> This guide will walk you through integrating a chat model into your application using Nscale’s API. With our serverless architecture, you can focus on building your application without worrying about infrastructure management.

## Prerequisites

1. **Service Token:** Sign up on the Nscale platform to create your [**service token**](https://docs.nscale.com/docs/manage/service-tokens.md).
2. **Model Selection:** Choose a chat model from Nscale’s library.
   * Example: Qwen3.8 Flash Next `Qwen/Qwen3.8-Flash-Next-FP8`

## Step 1: Set up your environment

Before making requests, ensure you have the necessary tools installed for your language of choice:

For Python:
Install openai library

```bash theme={null}
pip install openai
```

For Typescript:
Install openai library

```bash theme={null}
npm install openai
```

For cURL:
Ensure cURL is installed on your system (it's usually pre-installed on most Unix-based systems).

## Step 2: Sending an inference request

Let's walk through an example where we summarise a blog post into 100 words.

**Request structure**

Each request to the Nscale Chat Completions API endpoint should include the following:

1. Headers:
   * `"Authorization": "Bearer <SERVICE-TOKEN>"`
   * `"Content-Type": "application/json"`
2. Payload:
   * `"model"`: `"<model id e.g., Qwen/Qwen3.8-Flash-Next-FP8>"`
   * `"messages"`: `"<array of messages to send to the model>"`

Example use case: Summarise a blog post

```python Python theme={null}
import os
import openai

nscale_service_token = os.getenv("NSCALE_SERVICE_TOKEN")
nscale_base_url = "https://inference.api.nscale.com/v1"

client = openai.OpenAI(
    api_key=nscale_service_token,
    base_url=nscale_base_url
)

blog_text = "Serverless inference simplifies access to AI models..."

response = client.chat.completions.create(
    model="Qwen/Qwen3.8-Flash-Next-FP8",
    messages=[
        {"role": "system", "content": "Provide a summary of the blog post in 100 words."},
        {"role": "user", "content": blog_text}
    ]
)

print(response.choices[0].message.content)
```

```typescript Typescript theme={null}
import OpenAI from 'openai';

const nscaleServiceToken = process.env.NSCALE_SERVICE_TOKEN;
const nscaleBaseUrl = "https://inference.api.nscale.com/v1";

const client = new OpenAI({
    apiKey: nscaleServiceToken,
    baseURL: nscaleBaseUrl
});

const blogText = "Serverless inference simplifies access to AI models...";

const response = await client.chat.completions.create({
    model: "Qwen/Qwen3.8-Flash-Next-FP8",
    messages: [
        { role: "system", content: "Provide a summary of the blog post in 100 words." },
        { role: "user", content: blogText }
    ]
});

console.log(response.choices[0].message.content);
```

```bash cURL theme={null}
curl https://inference.api.nscale.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $NSCALE_SERVICE_TOKEN" \
  -d '{
    "model": "Qwen/Qwen3.8-Flash-Next-FP8",
    "messages": [
        { "role": "system", "content": "Provide a summary of the blog post in 100 words." },
        { "role": "user", "content": "Serverless inference simplifies access to AI models..." }
    ]
  }'
```

## Step 3: Understanding the response

The API will return a JSON object containing the model's output and token usage:

Example Response:

```json theme={null}
{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "In this article, the author discusses the challenges of deploying Artificial Intelligence (AI) models in real-world applications..."
      }
    }
  ],
  "usage": {
    "completion_tokens": 175,
    "prompt_tokens": 1172,
    "total_tokens": 1347
  }
}
```

Key Fields:

* `choices`: An array of message objects containing the model's output.
* `usage`: An object containing the input (prompt\_tokens), output (completion\_tokens), and total number of tokens used.

## Step 4: Using the CLI for Chat Inferencing

You can also use the Nscale CLI to interact with chat models. This is a convenient way to test models or build command-line applications.

### Prerequisites

* Ensure you have the [Nscale CLI installed](https://docs.nscale.com/docs/cli/overview.md)

### Examples

Here are some examples of using the CLI for chat inferencing:

```bash theme={null}
# Generate a single response
nscale chat "What is machine learning?" -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8

# Start an interactive chat session
nscale chat -i -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8

# Use a custom system message
nscale chat --message "system:You are a creative storyteller" -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8

# Get usage statistics in JSON format
nscale chat "Explain quantum computing" --stats -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8

# Limit the response length
nscale chat "Write a story" --max-tokens 100 -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8

# Supply chat history
nscale chat -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8 \
  --message "system:You are a helpful assistant" \
  --message "user:What is your name?"

# Start interactive mode with chat history
nscale chat -i -t $NSCALE_SERVICE_TOKEN -m Qwen/Qwen3.8-Flash-Next-FP8 \
  --message "system:You are a helpful assistant" \
  --message "user:Hello" \
  --message "assistant:Hello! How can I help?"

# Use service token from environment variable
export NSCALE_SERVICE_TOKEN=your_service_token
nscale chat "What is machine learning?" -m Qwen/Qwen3.8-Flash-Next-FP8
```

For more details on CLI usage, refer to the [CLI documentation](https://docs.nscale.com/docs/cli/overview.md).

## Step 5: Monitoring and scaling

Nscale handles scaling automatically based on traffic patterns—no manual intervention needed! Use the Nscale Console to monitor:

* API usage by model
* Spend breakdowns

For custom models or high-throughput applications on dedicated endpoints, contact Nscale Support.

## Troubleshooting

Common status codes and their meanings:

| Status | Description | Response Format |
| - | - | - |
| 200 | Success (synchronous) | `application/json` response with completion |
| 201 | Success (streaming) | `text/event-stream` with delta updates |
| 401 | Invalid service token or unauthorized | Error object |
| 404 | Model not found or unavailable | Error object |
| 429 | Insufficient credit | Error object |
| 500 | Internal server error | Error object |
| 503 | Service temporarily unavailable | Error object |

### Success Response Format (200)

```json theme={null}
{
  "id": "cmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "Qwen/Qwen3.8-Flash-Next-FP8",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "..."
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 56,
    "completion_tokens": 31,
    "total_tokens": 87
  }
}
```

### Error Response Format

```json theme={null}
{
  "error": {
    "code": "TOO_MANY_REQUESTS",
    "message": "You have insufficient credit to run this request",
    "param": null,
    "error_type": "INSUFFICIENT_CREDIT"
  }
}
```

For the extensive list of error codes and handling, see the [error code page](https://nscale.mintlify.app/docs/faqs/error-codes)

By following this guide, you'll be able to easily integrate chat models into your application using Nscale's serverless inference service.

> [card] [Contact Support](mailto:helpdesk@nscale.com)
>
> Need assistance? Get help from our support team
