Introduction

This document describes how to install and verify Ollama on Kunpeng servers.

Ollama is an open-source, lightweight inference framework designed for local deployment and running of large language models (LLMs). Deploying Ollama on Kunpeng servers can leverage the native support for GPT-Generated Unified Format (GGUF) models in the AArch64 architecture. This, combined with the Kunpeng multi-core processing capability and deep adaptation to the openEuler OS, enables instruction-level optimization.