AI inference, hosted in France.
Serving models in production means fast responses, predictable costs and data you can trust. Run open-source LLMs and image models, from a single API call to your own serverless endpoint or a reserved cluster, on our own NVIDIA servers in Paris and Lyon.
OpenAI-compatible · 30+ open-source models · Billed per token or per second
What is AI inference?
AI inference is the moment a trained model answers a request: a chatbot replying to a user, an agent calling a tool, a model summarizing a contract or generating an image. Training happens once or from time to time. Inference runs every time someone uses your product.
That is why inference drives most of the cost and the user experience of an AI application. Three things matter: how fast each response arrives, how much each request costs at scale, and where your prompts and outputs are processed.
How LLM inference works
An LLM answers in two phases. During prefill, it reads your whole prompt at once and stores intermediate results in a KV cache. During decode, it generates the answer one token at a time, reusing that cache instead of reprocessing the prompt. Two metrics follow: time to first token, driven mostly by prefill and prompt length, and tokens per second, the decode speed your users see as the answer streams.
Training
Inference
Goal
Learn patterns from data
Answer requests with the trained model
When it runs
Once, or periodically
Continuously, every time your app is used
What matters
Throughput and total GPU hours
Latency, cost per request, availability
Typical setup
Multi-node GPU clusters
API, serverless endpoints or dedicated GPUs
What makes inference hard, and how we solve it.
Moving from a demo to millions of requests raises the same four questions for every team.
Latency
Fast responses, every time
FP8/FP4 quantization, tuned vLLM and proprietary CUDA kernels deliver low latency and high throughput on the latest NVIDIA GPUs, so your users get answers quickly.
Cost
Pay for what fits your traffic
Per token for variable traffic, per second for your own endpoints, dedicated GPUs for steady volume. On GPU time, you pay for compute, not for each input token.
Data residency
Your data stays in France
Arkane Cloud is a Sovereign AI Cloud: requests are processed on our own servers in Paris and Lyon, and your prompts and outputs are never used to train models.
No lock-in
Open models, standard API
An OpenAI-compatible API and open-source models: switch from another provider by changing a URL and a key, and leave just as easily.
Three ways to run AI inference on Arkane Cloud.
Start with a ready-made API, move to your own endpoint when you need control, and switch to dedicated GPUs when volume becomes steady. Same console, same account.
Per token
Models API
Call 30+ open-source models through one OpenAI-compatible endpoint. Nothing to deploy, nothing to size.
Best for: prototyping, variable traffic, testing many models
Explore the Models API →Per second
Serverless GPU
Deploy any catalog or Hugging Face model, including fine-tuned ones, on the GPU you choose, with scale-to-zero.
Best for: custom models, long prompts, RAG, steady traffic
Explore Serverless GPU →Dedicated
Cloud GPU & GPU Clusters
Run your own inference stack on NVIDIA GPU instances hosted in Europe, or reserve a cluster on our own servers in Paris and Lyon for high-volume production.
Best for: high, predictable volume and full control
Explore Cloud GPU → Explore GPU Clusters →Recommended GPUs
On Serverless GPU and reserved GPU Clusters, running on our own servers in Paris and Lyon: RTX PRO 6000 Blackwell (96 GB) for cost-efficient serving of small and mid-size models · B300 (288 GB HBM3e) for 70B+ models, the largest models and long contexts
Popular models
Kimi, DeepSeek, Qwen, GLM, MiniMax, Llama, Mistral, Gemma, GPT-OSS, plus FLUX and SDXL for images
See the full catalog →From first prompt to production inference.
Test
Compare models on your own prompts in the console playground.
Integrate
Point your OpenAI SDK to the Models API and ship your first feature.
Customize
Deploy your fine-tuned or custom model on Serverless GPU, billed per second.
Scale
Move steady traffic to dedicated Cloud GPU instances or a reserved cluster.
Call the Models API with the OpenAI SDK
Keep your code: set the Arkane Cloud base URL and your API key, then use any model ID from the catalog. Streaming responses are supported.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://console.arkanecloud.com/api/v1",
api_key=os.environ["ARKANE_CLOUD_API_KEY"],
)
completion = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4.1-Flash",
messages=[{"role": "user", "content": "Hello!"}],
)
Which inference option for your workload?
A quick guide to match common inference workloads with the right product.
Workload
Recommended
Why
Chatbot or assistant with variable traffic
No infrastructure, you only pay for the tokens used
RAG and long-document processing
Billed by GPU time, not per token: long input prompts add no per-token charges
Fine-tuned model in production
Deploy any vLLM-compatible Hugging Face model, scale to zero
High, steady request volume
Dedicated capacity and predictable costs
Image generation
FLUX and SDXL billed per image
Questions about AI inference
What is AI inference?
AI inference is the use of a trained model to produce an output from new input: answering a question, classifying a document, calling a tool or generating an image. It is what happens every time a user interacts with an AI application.
What is the difference between AI training and inference?
Training teaches a model from large datasets and usually runs on multi-node GPU clusters for hours or weeks. Inference uses the trained model to answer requests in real time, so latency, cost per request and availability matter most.
Which inference option should I choose on Arkane Cloud?
Use the Models API to call ready-made open-source models per token. Use Serverless GPU to deploy your own or fine-tuned model, billed per second. Use Cloud GPU or GPU Clusters when you need dedicated capacity for high, steady volume.
Is your inference API compatible with OpenAI?
Yes. The Models API and Serverless GPU endpoints follow the OpenAI API format, so most applications only need a new base URL and API key.
Can I run inference on my own fine-tuned model?
Yes. Paste any vLLM-compatible Hugging Face model ID, public or private, and deploy it on Serverless GPU on the GPU of your choice.
Where is my data processed?
The Models API, Serverless GPU and GPU Clusters run on our own NVIDIA GPU servers in colocation datacenters in Paris and Lyon, France. Cloud GPU instances are hosted in Europe. Your prompts and outputs are never used to train models.
How is inference billed?
The Models API is billed per million input, output and cached tokens, or per image. Serverless GPU and Cloud GPU are billed per second of GPU time. GPU Clusters are reserved for 2 to 5 years.
Other use cases
Run your models in France, today.
Test models in the playground, get your API key and move to production at your own pace.
Free playground: 10 free requests · No commitment