53ON AI Server: OpenAI API for Ryzen AI Max

Resumen

Pixel shares 53ON AI Server, a lightweight Node.js server for running local AI on AMD Ryzen AI Max hardware with OpenAI- and Anthropic-compatible APIs. Its Vulkan sparse KV cache lets memory scale as tokens arrive, reducing a 1B model with a 1M-token window from 8.24 GiB to 0.86 GiB at startup; the MIT-licensed project also offers model controls, API quotas, and cloud-provider relays.

DEV Community

Show HN: 53ON AI Server – OpenAI-compatible server for AMD Ryzen AI Max

Hi HN. I run my own AI at home on a Ryzen AI Max+ 395 mini PC with 128 GB of unified memory, and this is the server it talks to. It's a single Node.js process with no dependencies, on top of llama.cpp. It speaks the OpenAI and Anthropic APIs, and it has a panel to load models, give each client an API key with a token quota, and relay cloud providers when a local model isn't enough. We also publish our own llama.cpp build. The main change is that the KV cache takes memory as tokens arrive instead of reserving it all at load (sparse buffers on Vulkan). A 1B model with a 1M-token window starts with 0.86 GiB instead of 8.24 GiB. It's MIT. I've only tested it on Strix Halo with Vulkan (RADV) on Ubuntu 24.04. I built it with a lot of help from Claude. Feedback and bug reports are very welcome.
53ON AI Server: OpenAI API for Ryzen AI Max
Powered by Readybot.io

Lo último de Pixel

Incident Response: Look Beyond the Red Alert

Pixel’s DEV Community announcement explores how teams can fixate on one failing service while overlooking the real cause of an outage, such as a full …

Sach: AI Fact-Checks WhatsApp Forwards in India

Pixel’s announcement introduces Sach, a tool that checks forwarded messages in English, Hindi, Telugu, and 14 other Indian languages against official sources. It gives cited …

Java 25 Removes SecurityManager: What to Fix

Pixel’s announcement explains how Java 24’s JEP 486 permanently disabled the SecurityManager, breaking test workarounds that intercept System.exit. It recommends returning exit codes or throwing …