Which GPUs are supported by the LocalAI skill?

The skill supports NVIDIA (via CUDA), AMD (via ROCm/HIP), and Intel GPUs, automatically selecting the appropriate container image based on detected hardware.

Can I run multiple LocalAI instances simultaneously?

Yes, the skill supports multi-instance management using the --instance flag, allowing you to run different models on different ports with isolated storage.

How do I add models to my LocalAI instance?

You can place your model files (such as GGUF or GGML) in the instance's specific configuration directory, usually found at ~/.config/localai/ /models.

Does this skill require a specific operating system?

This skill is specifically designed for Bazzite-based systems or environments using the ujust command runner and Podman Quadlets for container management.

LocalAI Manager

Name: LocalAI Manager
Author: atrawog

byatrawog

Data Science & ML

Manages LocalAI inference services with GPU acceleration using Podman Quadlet containers for local model hosting.

About

The LocalAI skill empowers Claude to deploy and manage a local, OpenAI-compatible API service on Bazzite-based systems. It automates the configuration and lifecycle management of LocalAI containers, including automatic hardware detection for NVIDIA, AMD, and Intel GPUs. This skill is essential for developers looking to run large language models, image generation, and audio transcription locally while maintaining a standardized API interface for other tools like Open WebUI, ensuring data privacy and reducing latency.

Key Features

Automatic GPU acceleration detection for NVIDIA, AMD, and Intel hardware
Standardized OpenAI-compatible API endpoints for chat, text, and embeddings
Seamless integration with the bazzite-ai bridge network for cross-container DNS
Containerized lifecycle management via Podman Quadlet and ujust commands
0 GitHub stars
Multi-instance support with isolated configurations and model storage

Use Cases

Configuring GPU-accelerated model inference on Fedora Bazzite or compatible systems
Setting up a local, private alternative to OpenAI's API for development and testing
Managing multiple isolated LocalAI instances for different project requirements and model sets

About

Key Features

Automatic GPU acceleration detection for NVIDIA, AMD, and Intel hardware
Standardized OpenAI-compatible API endpoints for chat, text, and embeddings
Seamless integration with the bazzite-ai bridge network for cross-container DNS
Containerized lifecycle management via Podman Quadlet and ujust commands
0 GitHub stars
Multi-instance support with isolated configurations and model storage

Use Cases

Configuring GPU-accelerated model inference on Fedora Bazzite or compatible systems
Setting up a local, private alternative to OpenAI's API for development and testing
Managing multiple isolated LocalAI instances for different project requirements and model sets