ZQN-12B: Open AI Model with 128K Context
ملخص
Pixel shares ZQN-12B, an early open model trained from scratch and compressed to 6.48 GiB, designed to run on an 8 GB GPU at about 25 tokens per second. The announcement also features the M3-GPTQ compression paper, with details on its quantization methods, 128K native context, and Apache 2.0 license.
DEV Community
ZQN-12B: our second open model, plus the M3-GPTQ compression paper
ZQN-12B is the second open model from ZAQERIS Ai, an independent AI research group in Cairo, Egypt. Trained from scratch 15.55B parameters, compressed with our ZAQERIS M3-GPTQ method to 6.48 GiB (3.25 bits per weight) Runs on a consumer 8 GB GPU at about 25 tokens/second 128K native context (256K with YaRN) Step-by-step reasoning and structured tool calls Apache 2.0 Model: https://huggingface.co/ZAQERIS/ZQN-12B Alongside the model we published the technical paper behind its compression: "ZAQERIS M3-GPTQ: Hessian-Aware Multi-Stage Residual Vector Quantization for Extreme Bitrate LLM Compression". It combines Block-Hadamard decorrelation, Hessian-salience outlier selection, multi-stage 8-dimensional residual vector quantization, and Block-8 Cholesky GPTQ error propagation. Paper: https://huggingface.co/ZAQERIS/ZQN-12B/blob/main/docs/papers/ZAQERIS_M3_GPTQ_PAPER.md This is an early checkpoint, not a frontier model, and we are honest about its limits on the model card. Feedback and contributions are welcome.
Powered by Readybot.io
