Problem solved

Locai solves the challenge of balancing cost, data privacy, latency, and continuity in AI inference by allowing organizations to offload routine tasks to local devices or on-prem servers while reserving cloud APIs for complex requests. It eliminates the need to choose between expensive per-token pricing models and the risk of exposing sensitive data to third-party clouds, instead giving full control over where inference happens through a single unified endpoint.

Use cases

• Route routine AI requests to end-user devices to eliminate per-token costs
• Keep sensitive data on-premises during inference for compliance
• Provide low-latency AI responses by computing on the device that sends the request
• Maintain AI functionality in disconnected environments like ships, remote clinics, or construction sites
• Deploy a company-wide AI workspace for chat and document search without third-party cloud syncing
• Orchestrate hybrid inference across edge devices, private cloud, and public APIs from a single control plane
• Build AI-powered products that run on customer devices while still using a consistent API endpoint

Audience

• Developers
• ML Engineers
• Enterprise
• Startups
• Hobbyists

Pros

• Enforces inference policy to balance cost, privacy, latency, and continuity
• Single OpenAI-compatible endpoint simplifies routing across diverse compute targets
• Full audit trail for governance and explainability
• Runs on existing user-owned infrastructure, avoiding third-party data exposure
• Supports offline or disconnected operation with local models

Cons

• Early development – not all advertised features are production-ready
• Currently limited to OpenAI-compatible API; may not support all model formats
• Windows and Linux support not yet available (only macOS at present)
• Pricing beyond the free tier remains unclear
• Requires technical expertise for on-premise deployment and fleet management