Notes

NVIDIA just added our Viettel Cloud Tech Blog post to the NVIDIA Dynamo docs as a case study, one of 8 alongside Amazon Ads, Pinterest, Cognition, Prime Intellect, DigitalOcean, Photoroom and H Company.

The post covers how we serve multiple LLM families on one H200 fleet with NVIDIA Dynamo, a component of NVIDIA DSX OS.

I wrote it with Doan Thinh Vo, and we did the work together with Henry Ma, Nan Gu, Kuldeep Sharma, Tao Liu, Azman Salleh and Que Nguyen from NVIDIA.

The models were never the problem. Every GPU-hour we lost, we lost in the serving layer above the engine. Results from our fleet:

• Engine chosen by measurement: on Qwen3.8-27B-FP8, SGLang gave 26-39% more throughput than vLLM at concurrency 32
• Autoscaling on KV-cache utilization instead of GPU utilization: TTFT p95 dropped from 44.5 s to 1.4-3.4 s
• Agentic endpoint on MiniMax-M3-NVFP4: ~4.5x throughput on the same 2 H200 nodes, and ITL p99 from 256 ms to 26 ms with prefill/decode disaggregation

If you run several model families on one fleet, we’d like to compare notes.

Full post on the Viettel Cloud Tech Blog, and the listing in NVIDIA Dynamo External Publications.

CNCF just published the 2026-2028 Ambassador cohort, and my name is on the list 🎉

58 new ambassadors this year, selected from more than 600 applications, joining a cohort of 312 worldwide. I’m the only one from Vietnam in this intake 🇻🇳

The post names translating the Cloud Native Glossary as one of the ways ambassadors give back. That’s where my path started, with the Vietnamese edition, and it’s the kind of work I want to keep doing: bringing more Vietnamese engineers from user to contributor.

Thank you Audra Montenegro and the CNCF team.

My first bug fix just landed in vLLM 😁

We hit the bug running prefill/decode disaggregation with NVIDIA Dynamo in production, and fixed it upstream: vllm-project/vllm#57734.

vLLM is one of the most widely used open-source engines for serving LLMs, with more than 93,000 stars on GitHub.

Thanks Nicolò Lucchesi for the review.

AICR v1.0 just shipped, and Viettel Cloud is on NVIDIA’s contributor list, next to Red Hat, Microsoft, Oracle Cloud, SUSE, Akamai, Mirantis and Sidero Labs.

As one of the contributors behind that name, I’m proud of it.

Figure shared by Yuan Chen in the #aicr channel.

AICR is open source under Apache 2.0, and it is built on four independent capabilities:

• Snapshot records observed cluster state, including Kubernetes, operating system, kernel, GPU, and topology information.
• Recipe describes the desired, version-locked component configuration and the constraints and validation phases that apply to it.
• Bundle renders the recipe into artifacts for the operator’s preferred deployment tooling.
• Validation compares the recipe with observed state and, where declared, runs deployment, conformance, and performance checks against the cluster.

AICR’s version 1.0 then commits to a stable compatibility contract across the CLI, the REST API, the Go SDK, the bundle layout and the artifact schemas, with committed baselines checked before merge, so integrators build on public interfaces instead of internal ones. Recipes and evidence are public at validation.aicr.run.

The numbers Yuan Chen shared in the #aicr channel with v1.0: 2,083 pull requests with 1,865 merged, around 180 merged per month since February, 1,031 issues with 854 closed, 94 contributors with 49 (52%) outside NVIDIA.

Going from using NVIDIA’s software to being listed as a contributor to it is a long road, and my team is at the start of it. If you run GPU Kubernetes and want to know which configuration a vendor actually validates, this is the project to watch: Yuan Chen and Christopher Haar have the AICR talk at KubeCon NA 2026, November 9-12 in Salt Lake City.

Theo dõi NVIDIA thời gian gần đây, mình thấy rõ là họ đang đi theo hướng “mở” hơn hẳn. Họ đưa project vào CNCF, open source dần các component của AI factory, và chủ động kéo doanh nghiệp dùng mã nguồn mở của họ thay vì chỉ dùng phần mềm đóng đi kèm phần cứng. Là CNCF Ambassador, mình rất vui khi thấy điều này. Team mình cũng có làm việc trực tiếp với NVIDIA về các dự án mã nguồn mở này.

Sắp tới ở KubeCon NA 2026 tại Salt Lake City (9–12/11), Yuan Chen và Christopher Haar (NVIDIA) sẽ trình bày về AICR (AI Cluster Runtime): các recipe GPU đã được validate cho Kubernetes, chạy được trên cloud, on-prem lẫn bare metal, và có thể query, audit được.

AICR là một component của DSX OS, và DSX OS là ví dụ rõ nhất cho hướng đi này: phần lớn component trong đó là mã nguồn mở, và mỗi component có thể cài độc lập.

Mình và team mình đã đặt challenge adopt DSX OS lên hạ tầng H200, đi qua từng component một.

Nhưng tham vọng không dừng ở chỗ dùng được. Mình muốn team mình, và rộng hơn là đơn vị mình làm việc, có thể dẫn đầu khu vực về khả năng adopt và làm chủ mã nguồn mở Hạ tầng AI, khi hệ thống gặp lỗi thì tự tìm ra nguyên nhân và gửi fix lên upstream, thay vì chờ vendor.

Ai để ý GitHub của mình thì sẽ thấy đã có những PR được merge ở KAI Scheduler, Workload-Map, Cluster Readiness Engine hay AICR.

Khi chính NVIDIA đã mở như vậy thì bên cạnh việc sở hữu nhiều GPU, ai hiểu và làm chủ phần mềm chạy trên đó nhanh hơn cũng sẽ có lợi thế rất lớn.

Over a year ago, our team was searching for a fractional GPU solution on Kubernetes: a way to share and partition GPUs across AI workloads instead of assigning whole devices. That search led us to HAMi. 🍉

And later we started running HAMi at Viettel Cloud: fractional GPUs for our AI Notebooks and inference workloads instead of assigning whole devices.

From adopter to advocate: since 2025 I’ve been sharing the HAMi story at Cloud Native Vietnam meetups, KCD, and Viet OpenInfra Days, and at KCD Vietnam 2026 I finally got to meet Reza Jelveh in person and had a talk with him.

Along the way, I’ve watched the project grow from CNCF Sandbox to CNCF Incubating.

Today this arrived: a Contributor Recognition box from dynamia.ai, a certificate signed by Jonathan Bryce (CNCF Executive Director) and swags.

Thank you to the HAMi maintainers and community, thank you Jimmy Song. Proud to be #1077 on this journey, and excited to keep both contributing and talking about it.

Glad to have a small piece in AICR v0.21: an l40-any accelerator overlay (NVIDIA/aicr#2366).

L40 was a registered accelerator type in AICR’s catalog with no overlay behind it, so aicr snapshot on a real L40 node would detect the hardware and then fail to resolve a recipe for it.

Worth pushing upstream when it hit on real hardware.

Shared with Mark Chmarny’s announcement: AICR v0.21 adds GB300 and Vera Rubin support, Kubernetes 1.37 compatibility, major GPU Operator and Dynamo upgrades, and stronger API, validation and bundling guarantees on the way to v1.

In April 2025 I started the Vietnamese track of the CNCF Cloud Native Glossary. There wasn’t one, and English was the first thing standing between a lot of Vietnamese engineers and a CNCF project. So I opened the tracking discussion, set up the dev-vi workflow from scratch, and started asking people to join me.

Seven contributors showed up, from different companies. We worked through 40+ terms. Ten weeks later the Vietnamese edition was live and announced on the CNCF blog.

The term count isn’t the part I’m proud of. It’s that curious engineers walked out of it with a real upstream CNCF track record. Several of them went on to become Kubernetes community members.

And it didn’t stop there. The same work continues in Kubernetes SIG Docs Vietnamese, translating Kubernetes’ own documentation. I’m an owner and approver there, which mostly means reading a lot of Vietnamese pull requests and trying to make the first-time contributor experience less intimidating than mine was.

Alongside that: contributions across CNCF projects, membership in the Kubernetes and HAMi orgs, submitting Viettel AI Platform to the new Kubernetes AI Conformance program, and talks at KCD & OpenInfra Days Vietnam, Cloud Native Hanoi and the CNCF 10th Birthday Bash, mostly about the thing I actually run: fractional GPU and AI workloads on Kubernetes.

Today I got the email.

I’m now a CNCF Ambassador.

The reason this one matters to me is written plainly on the CNCF Ambassadors page:

Ambassadors are an extension of CNCF, furthering the mission of ‘making cloud native ubiquitous’ through community leadership and mentorship.

That’s the direction I want my career to keep pointing in. Not just contributing myself, but shortening the path from “user” to “contributor” for engineers who currently believe that path isn’t open to them.

A word about where I do this from. I’m lucky to work at Viettel Networks (Viettel Group), a company that didn’t just talk about open source but put its name on it: a member of the Cloud Native Computing Foundation (CNCF), the OpenInfra Foundation and The Linux Foundation. I handle those relationships day to day, which means my community work and my day job point the same direction instead of competing. Not every engineer gets that. I don’t take it for granted.

And thank you to Khanh Tran, Vietnam’s first CNCF Ambassador, and my fellow owner in sig-docs-vi. The goal was never to duplicate what he built. It’s to double the bandwidth.

And to VIETOPENINFRA, the community that built the stages every one of those talks happened on.

Plenty of work ahead. Thanks to everyone who kept this ecosystem open long enough for someone like me to find a way in.

But today I’m taking a moment.

Thank you, CNCF. ❤️