One Rust inference engine. One API. One toolchain — from a $100 phone to an H200 cluster. AI infrastructure you can own, run, and scale anywhere.
Team Sprapp — Sam Wong · Aiden Kee · S K Song · Keshav Adjovi Sim · Chong En Lim
Mentors — Dr. Ann Ang · Built in 28 days on Bitdeer 8×H200
The real barrier isn't training a model — it's deploying AI where it's actually needed: on hardware people already own, offline, without a cloud bill.
| Pain point | What it means | Oaica's answer |
|---|---|---|
| Training is complex | Weeks and budget burned on setup, eval, iteration. | We focus on deployment — take proven models, tune them, run them anywhere. |
| Cloud inference is expensive | Every API call has a meter. | Local inference cuts the meter — Oaica runs fully offline on phone, laptop, or DGX Spark. |
| Runtimes ignore real hardware | Most engines assume a data-center GPU. | PRISM targets hardware people actually own: $100 Android → H200 cluster. |
| SDKs multiply | Edge, laptop, cloud each need a different stack. | One engine, one API, one skill set. |
| Small models hallucinate | A 3B model asked trivia will guess. | Router + RAG: edge model emits tool calls; offline DB/API supplies facts. Zero hallucination in the answer path. |
A single generate → verify → filter → audit pipeline feeds all three submissions — not three unrelated projects.
Pipeline generates a synthetic Malay/SEA conversation + safety-audit dataset. Two-arm bias probe (raw Malay vs MT-English) vs a hand-authored Malaysia policy. 58,888 paired rows, gated HF release.
DeepSeek-V4-Flash 284B MoE + KAT-Coder 35B, SFT/GRPO/quant on Megatron-Bridge (NeMo's training core). Domain verticals: coder / legal / tutor LoRAs, Malay-sovereign 35B.
Multi-agent enterprise workflow automation (config-drift audit → rollout plan) on the NVIDIA agentic pattern: decompose → spawn → delegate → synthesize, driving our own fine-tuned models live.
SDG is the pipeline's safety-audit arm · Finetuning is its trained output · Agentic is that output put to work on stage.
Malay ~34M speakers, zero native NVIDIA safety-classifier coverage. We built the pipeline, not just a dataset.
flowchart LR
G["Generate
bandit-picked prompt variant
per-family autoresearch"] --> Q["Quality gate stack
heuristics → GBM classifier →
LLM-judge → KenLM → langid"]
Q --> S["Two-arm safety probe
raw Malay vs MT-English
Nemotron-3 + custom S23-S26 policy"]
S --> P["PII / license audit
handle-mask, contamination check"]
P --> R(["Gated release
5,411 rows · 2,422 public-safe
58,888 paired probe rows"])
curated samples in the corpus DB; 6.3B-token Malay CPT shard set, 5× the original target
mean_composite gain from prompt-style search vs +0.102 from sampling-param tuning — the real lever we found
Nemotron-3 content-safety + Aegis taxonomy as eval/filter signal only — never trained into our own weights
CPT → SFT → RL/GRPO on two architecturally distinct model families, plus a sovereign-language domain vertical.
Day-one architecture support (landed llama.cpp upstream, ran in production same day). Full-818 sweep pass@8 floor 74.3%. MTP speculative decode 75.2 tok/s (+34%). Custom I-Quality per-expert quant: 5.2× compression, ~114 GB deploy.
Qwen3.6-35B-A3B, Apache-2.0, security-specialist SFT, ~214 tok/s bf16. Malay-35B ties SEA-LION v4.5 (95% CI overlap), +16.8pp NLU composite. Domain verticals via LoRA: coder, legal, tutor.
A real tool-use agent loop — not a chatbot — built on the same decompose → spawn → delegate → synthesize pattern as NVIDIA NemoClaw, driving our own fine-tuned model family live.
flowchart LR
T["Task
'Audit staging vs prod config drift
and give me a safe rollout plan'"] --> D["Decompose"]
D --> A1["config_auditor"]
D --> A2["migration_author"]
D --> A3["rollout_planner"]
A1 --> SY["Synthesize"]
A2 --> SY
A3 --> SY
SY --> OUT(["One ordered plan
+ rollback command"])
Same pattern NVIDIA showcases with NemoClaw — applied to real infra ops (config-drift audit) instead of a toy demo. Runs on phone, PWA, or cluster — same code.
PRISM backend speaks the Ollama HTTP API — any Ollama-compatible agent CLI drives Oaica V4-Flash, KAT, Malay-35B as local models, verified live at ~26 tok/s.
Same API, tokenizer, and tool-call parser at every scale — tuned per tier instead of forcing data-center assumptions everywhere.
| Tier | Model | Size | Runs on | Headline number |
|---|---|---|---|---|
| 0.8B Phone | Qwen3.5-0.8B Q4_K_M | 508 MB | $100 Galaxy A06, entry Android | 3.61 tok/s offline |
| 4B Tool | Qwen3.5-4B + r16 LoRA | 2.6 GB | Tablet / phone, CPU-only | 94.4% select · 77.8% strict |
| 35B A3B | Qwen3.6-35B-A3B (i-nano/mini/compact) | 11–16 GB | DGX Spark, MacBook, 16 GB VRAM + 128 GB RAM | 256K → 1M ctx (TurboQuant) |
| V4-Flash | DeepSeek-V4-Flash 284B MoE, 13B active | 94–136 GB | H200 cluster, Spark upper-bound | 75.2 tok/s (+34% MTP) |
| 35B-class model, same harness | Decode tok/s | Prefill tok/s |
|---|---|---|
| base (Qwen3.6-35B-A3B) | 12.99 | 19.45 |
| sfc6 (Malay-35B, ours) | 12.97 | 19.12 |
| v45 (SEA-LION 27B) | 12.60 | 12.80 |
| qwen35 (Qwen3.5-27B) | 11.74 | 20.18 |
Apples-to-apples: all 4 on Bitdeer H200 GPU7, same script — our belebele +35.4pp fine-tune costs essentially zero decode speed (12.97 vs base 12.99 tok/s), and beats SEA-LION v4.5 on both decode and prefill.
128 GB unified memory in a desk-side box — where 35B A3B becomes genuinely useful without a rack of GPUs.
| Tier | VRAM | CPU RAM | Example hardware | Experience |
|---|---|---|---|---|
| CPU-only min | 0 GB | ~14 GB | 16 GB RAM laptop | ~5.75 tok/s decode |
| Minimum | 12 GB | 64 GB | RTX 4070 laptop / 4060 Ti | Loads; needs patience on long ctx |
| Recommended | 16 GB | 128 GB | RTX 4080/90, DGX/RTX Spark | Smooth 256K, LoRA hot-swap, interactive |
| Ideal | 128 GB UMA | shared | DGX/RTX Spark, high-end UMA Mac | Best throughput + context, one pool |
Demo beat: Oaica + 35B A3B i-mini Coder LoRA on a DGX/RTX Spark laptop, fully offline, 256K context.
BFCL-1240, measured on our own PRISM /v1/chat/completions endpoint. More merge components ≠ better — a clean finding, not an accident.
Oaica 4B Tool — tool-name selection on a 2.6 GB CPU-only GGUF (77.8% strict-pass, up from 61.1% via schema fix, zero retrain)
KAT-Coder bf16, single 80GB GPU — Apache-2.0, 3B active params/token, security-specialist SFT
MTP speculative decode on V4-Flash — 75.2 tok/s at 100% draft acceptance vs 56.3 baseline
Full 8-axis SEA-HELM breakdown, 4 models, complete. GlossoBench MS & official gate_axis.py composite shown honestly as separate, non-comparable measurements.
belebele (reading comprehension) is the axis this track exists to move — 84.5%, our single biggest jump in the family. Composite: 95% CI overlaps SEA-LION v4.5 — a statistical tie, +16.8pp NLU vs v4.5.
Malay-35B vs base on belebele — largest single-axis gain across the whole model family
Nanbeige SFT-3B beats SEA-LION E2B-IT 2B on SEA-HELM overall
paired safety rows — 3.6% vs 5.9% translation-loss (off-the-shelf vs our custom policy)
| Capability | Before | After |
|---|---|---|
| Model on cheap phone | None ran well | 3.61 tok/s on Galaxy A06 |
| 35B+ model on laptop | Needed data-center GPU / cloud | Runs on 16 GB VRAM + 128 GB CPU RAM |
| Coding model | Not loaded end-to-end | KAT i-compact 85.2% BFCL v4 |
| V4-Flash serving | Separate stack | Same PRISM engine + MTP recipe |
| Agent orchestration | Not built | 3-turn swarm, live in PWA |
| KV memory pressure | Big context = OOM | TurboQuant KV, ~4.5× compression |
In 28 days: phone → Spark → H200 cluster, one runtime, one API.
A 127 MB LoRA download turns a commodity chatbot into a specialist — the finance/healthcare/legal customisation pattern, made cheap.
94.4% tool-name selection (34/36) · 77.8% strict-pass (28/36)
Strict-pass went 61.1% → 77.8% from one schema-enrichment fix — zero retrain, pure deployment config.
Runs as a 2.6 GB GGUF Q4_K_M on tablet/phone CPU. Smoke test 5/5 PASS.
The SDG entry proves the pipeline produces training-quality data; the Finetuning entry is that data's downstream result; the Agentic entry is that model put to work.
flowchart TD
CORE(["Datagen pipeline
generate → verify → filter → audit"])
CORE --> SDG["SDG
Malay/SEA safety cross-check"]
CORE --> FT["Model Finetuning/Customisation
Oaica V4-Flash 284B + KAT 35B"]
CORE --> AG["Agentic Workflows
Oaica Code CLI"]
SDG --> SDGOUT(["Gated Malay safety eval set
5,411 rows (2,422 public-safe)"])
FT --> FTOUT(["SFT + GRPO + quant, shipped
day-one arch support, +34% MTP decode"])
AG --> AGOUT(["Live agentic demo
drives the whole model family on stage"])
Category filter → PII mask → license split. The provenance system caught and fixed its own leak before release — proof the rigor is real, not decorative.
flowchart TD
DQ[("Disagreement queue
5,588 rows")]
DQ --> CAT["Category hard-exclude
S24 royal defamation only"]
CAT -->|"177 excluded"| CATOUT["excluded: legal-risk"]
CAT -->|"5,411 pass"| PII["PII / handle scan
mask, don't drop"]
PII -->|"1,534 masked, incl.
1,390 raw handles missed earlier"| GATED(["GATED RELEASE
5,411 rows"])
PII --> SRC["Source-license filter
Twitter-sourced, unconfirmed terms"]
SRC -->|"3,057 excluded"| SRCOUT["excluded: unconfirmed terms"]
SRC -->|"2,422 pass"| PUBLIC(["PUBLIC-SAFE VARIANT
2,422 rows"])
rows, gated research-use release on Hugging Face — live now
raw Twitter handles our own audit caught and masked before shipping — same day
row public-safe variant, all Twitter-sourced rows dropped
Same-harness eval, calibrated on a known-good model first, diagnosed down to raw-response dumps — failures feed straight back into the gate.
flowchart LR
GEN["① Generate
bandit-picked prompt variant"] --> GATEX["② Gate
full quality-gate chain"]
GATEX --> TRAIN["③ Train
consumer run"]
TRAIN --> EVAL["④ Eval
real-harness only"]
EVAL --> DIAG["⑤ Diagnose
per-axis gaps, regressions"]
DIAG --> FIX["⑥ Fix the pipeline
not just the data"]
FIX --> GEN
DIAG -.->|"new filter"| GATEX
DIAG -.->|"harder pool"| GEN
EVAL -.->|"disagreement rows"| GOLD(["New eval sets
released as artifacts"])
Before generation runs at scale, a CPU-only autoresearch loop searches prompt variants against the same scorer that later gates the real corpus.
flowchart TD
CAND["Candidate prompt variants
control · reorder · exemplar · persona · format"] --> SCORE["score_sample() — CPU-only
textbook + structural + register quality"]
SCORE --> RANK{"gap_score(knob)
coordinate-ascent"}
RANK -->|"high-gap, keep testing"| CAND
RANK -->|"gap < 0.02, converged"| CONFIG[("optimal_configs.json
keyed by family × provider × quant")]
CONFIG --> DRIVERS["generation drivers adopt config"]
RANK -.-> FOUND(["Finding: prompt STYLE beats
sampling-param tuning"])
mean_composite gain from the winning prompt-style variant
mean_composite gain from the best sampling-parameter search alone — the wrong lever
Students learn on RTX. Developers prototype on DGX Spark. Labs scale on H200. Skill transfer only works if the runtime doesn't change underneath them.
H200 SXM training + cloud inference · DGX/RTX Spark 128 GB UMA fit for 35B A3B and V4-Flash i-micro · MTP speculative decode · BFCL-1240 on our own PRISM endpoint.
Nemotron-3 content-safety, Nemotron Ultra 550B, Nemotron-Reasoning-4B, NeMo Curator, Aegis, NIM/vLLM. Trained on Megatron-Bridge — NeMo's own training core — not overclaimed as "we used NeMo Framework."
Compute: Bitdeer 8×H200 SXM · 120× Intel Xeon Platinum 8580 cores · 2.0 TB DDR5-5600 · 62 TB NVMe
One open engine, one API, phone to data center. Students, indie hackers, rural clinics build without a cloud gatekeeper.
Same runtime on a $100 phone, a DGX Spark, or an H200 cluster. Upgrade when you need to — not because a platform forces you.
Local copilots on patient records and unpublished data, no cloud exfiltration. Deterministic, auditable, offline science.
Oaica turns AI from a cloud service you rent into infrastructure you can own, tune, and run anywhere.
28 days captured repeatable know-how we can re-run on the next model family, the next vertical, the next language.
Ship one Rust/WGSL engine from a $100 phone to an 8×H200 cluster. Compress KV with TurboQuant. Plan heterogeneous offload. Hot-swap LoRA without reloading base weights.
Benchmark tool-calling, coding, safety, long-context with GlossoBench. Move a model through CPT → SFT → RL/GRPO. Merge and ablate across three distinct architectures.
Turn a local model into a retrieval router that calls tools instead of hallucinating. Build a 3-turn agent swarm inside a PWA, deploy on Cloudflare Pages.
Native Malay content-safety classifier (no English-translation proxy). On-device ternary/GGUF converter inside a model hub. One-click LoRA marketplace inside Oaica.
Aegis / Nemotron taxonomy coverage for Bahasa Malaysia + Indonesia — the exact gap our SDG track exists to document.
Always-on Bitdeer 8×H200 node for CPT, SFT, RL/GRPO, classifier training, merge/ablate sweeps, V4-Flash speculative-decode work.
Longer judging slots for multi-device demos. A physical smoke-test room for hardware-tier demos.
PRISM + Oaica — a Rust inference engine and four model tiers, from a $100 Galaxy A06 to an H200 cluster, shipped in 28 days on one API.
This work was completed in part at the AI for Science Australian Open Hackathon, part of the Open Hackathons program.
The authors acknowledge OpenACC-Standard.org for their support. · @OpenACCorg #OpenHackathons