API & apps

wanna.sh speaks the OpenAI API, so apps and code that work with OpenAI can use your own models instead.

Connect an app

In any app with an “OpenAI-compatible” or “custom endpoint” setting, use this address:

Use auto as the model name to use whichever model is running, or an exact id from the model list. Full reference: interactive API docs.

Example code

# pip install openai
from openai import OpenAI

client = OpenAI(base_url="http://localhost:7860/v1", api_key="not-needed")

reply = client.chat.completions.create(
    model="auto",  # or an id from /v1/models
    messages=[{"role": "user", "content": "Write a haiku about GPUs"}],
)
print(reply.choices[0].message.content)

image = client.images.generate(prompt="a cat astronaut", size="512x512")
print(image.data[0].url)

with client.audio.speech.with_streaming_response.create(
    model="auto", voice="default", input="Hello there!"
) as r:
    r.stream_to_file("hello.mp3")

Endpoints

POST	/v1/chat/completions	Chat (streaming supported)
POST	/v1/completions	Raw text completion (GGUF models)
POST	/v1/images/generations	Make pictures (optional sampler: dpmpp_2m_karras, euler_a, euler, unipc, ddim, builtin)
POST	/v1/images/edits	Change a picture with a description
POST	/v1/audio/speech	Text to speech
POST	/v1/audio/transcriptions	Speech to text (json, text, srt, vtt, verbose_json)
POST	/v1/audio/translations	Speech in any language to English text
POST	/v1/videos	Start a video (poll GET /v1/videos/{id}, then /content)
GET	/v1/models	Installed models and what they do

API keys

Keys are optional right now. Turn on “Require API key” in Settings before sharing wanna.sh on your network.