# Get your VLM running in 3 simple steps on Intel CPUs

[Hugging Face](https://yomu.fyi/company/hugging-face) · Ezequiel Lanza, Helena, Nikita, Ella Charlaix, Ilyas Moutawwakil · Oct 15, 2025

**Type:** Tutorial

## Summary

Deploying Vision Language Models locally presents computational challenges despite providing data privacy and independence from external servers. Optimum Intel and OpenVINO address this by optimizing lightweight models like SmolVLM2-256M for low-resource hardware, including Intel CPUs and GPUs. The process involves exporting the model to the OpenVINO Intermediate Representation format, applying 8-bit weight-only or static quantization, and executing inference with the transformed model. In single-image CPU benchmarks on an Intel Core Ultra 7 265K processor, OpenVINO with 8-bit weight-only quantization reduced time to first token from 5.150 seconds under PyTorch down to 0.247 seconds. The optimized configuration also raised decoding throughput from 0.722 to 63.928 tokens per second.

## Context

Running Vision Language Models on local devices is computationally demanding, but local execution offers enhanced privacy, speed, and reliability without reliance on internet connections or external servers.

## Approach / What changed

Using Optimum Intel and OpenVINO, SmolVLM2-256M is converted to the OpenVINO IR format, quantized using either 8-bit Weight Only Quantization (WOQ) or mixed static quantization with a calibration dataset, and executed on Intel CPUs or GPUs.

## Takeaways

- Converting SmolVLM2-256M to OpenVINO IR cut time to first token on an Intel Core Ultra 7 CPU from 5.150s to 0.420s and boosted throughput from 0.722 to 47.237 tokens/s.
- Applying 8-bit Weight Only Quantization further improved performance on OpenVINO, achieving a 0.247s time to first token and 63.928 tokens/s decoding throughput.
- Static quantization requires a calibration step using representative data samples and can be applied specifically to vision embeddings while retaining weight-only quantization for other components.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/openvino-vlm)