Rate This Document
Findability
Accuracy
Completeness
Readability

Introduction

Numerical calculation performed by a computer has a precision loss and cannot be as accurate as mathematical formula calculation. The precision loss with industrial software is controllable and acceptable within a certain range. However, if the precision loss exceeds the acceptable range, software and hardware engineers need to locate the calculation phase where a precision loss occurs, which is complicated and inefficient. Based on the Kunpeng computing platform, this document describes the causes, remediation measures, and calculation methods of numerical calculation precision errors. Besides, it proposes an automatic and efficient analysis tool, which is of great benefit to developers and scientific researchers in related fields. It should be clarified that calculation errors and precision losses are determined by data calculation methods and computer data representation. This problem exists on all types of computing platforms, including x86 and ARM. The calculation result is usually generated based on the trade-off between performance and precision.