gemm_batch_?8?8s32

Perform batch general matrix multiplication. This function is described as follows:

$\text{[math]}$

The value of op(X) may be $\text{[math]}$ . alpha and beta are multiplication coefficients; op(A) is an m*k matrix, op(B) is a k*n matrix, and C is an m*n matrix.

A_offset is a scalar of fixed type int8_t. A_offset is added to all elements in op(A).

B_offset is a scalar of fixed type int8_t. B_offset is added to all elements in op(B).

C_offset has three definitions (the element type is int32_t):

A scalar. C_offset is added to each element in matrix C.
A row vector. C_offset is added to each row in matrix C.
A column vector. C_offset is added to each column in matrix C.

Note: Batch GEMM is applicable to small matrices with large batch sizes. It is less effective for large matrices with small batch sizes.

Interface Definition

C interface:

void cblas_gemm_batch_s8s8s32(const CBLAS_LAYOUT layout, const CBLAS_TRANSPOSE *transA_array,

const CBLAS_TRANSPOSE *transB_array, const CBLAS_OFFSET *offsetc_array, const BLASINT *m_array,

const BLASINT *n_array, const BLASINT *k_array, const float *alpha_array, const BLASINT8 **a_array,

const BLASINT *lda_array, const BLASINT8 *oa_array, const BLASINT8 **b_array, const BLASINT *ldb_array,

const BLASINT8 *ob_array, const float *beta, int32_t **c_array, const BLASINT *ldc_array, const int32_t **oc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_gemm_batch_u8u8s32(const CBLAS_LAYOUT layout, const CBLAS_TRANSPOSE *transA_array,

const CBLAS_TRANSPOSE *transB_array, const CBLAS_OFFSET *offsetc_array, const BLASINT *m_array,

const BLASINT *n_array, const BLASINT *k_array, const float *alpha_array, const BLASUINT8 **a_array,

const BLASINT *lda_array, const BLASINT8 *oa_array, const BLASUINT8 **b_array, const BLASINT *ldb_array,

const BLASINT8 *ob_array, const float *beta, int32_t **c_array, const BLASINT *ldc_array, const int32_t **oc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_gemm_batch_s8u8s32(const CBLAS_LAYOUT layout, const CBLAS_TRANSPOSE *transA_array,

const CBLAS_TRANSPOSE *transB_array, const CBLAS_OFFSET *offsetc_array, const BLASINT *m_array,

const BLASINT *n_array, const BLASINT *k_array, const float *alpha_array, const BLASINT8 **a_array,

const BLASINT *lda_array, const BLASINT8 *oa_array, const BLASUINT8 **b_array, const BLASINT *ldb_array,

const BLASINT8 *ob_array, const float *beta, int32_t **c_array, const BLASINT *ldc_array, const int32_t **oc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_gemm_batch_u8s8s32(const CBLAS_LAYOUT layout, const CBLAS_TRANSPOSE *transA_array,

const CBLAS_TRANSPOSE *transB_array, const CBLAS_OFFSET *offsetc_array, const BLASINT *m_array,

const BLASINT *n_array, const BLASINT *k_array, const float *alpha_array, const BLASUINT8 **a_array,

const BLASINT *lda_array, const BLASINT8 *oa_array, const BLASINT8 **b_array, const BLASINT *ldb_array,

const BLASINT8 *ob_array, const float *beta, int32_t **c_array, const BLASINT *ldc_array, const int32_t **oc_array,

const BLASINT group_count, const BLASINT *group_size);

Parameters

Parameter	Type	Description	Input/Output
layout	Enumeration type CBLAS_ORDER	Whether the matrix is in row-major order or column-major order.	Input
transA_array	Enumeration type CBLAS_TRANSPOSE	Transposition sequence of matrix A. Whether matrix A is a conventional matrix, a transposed matrix, or a conjugate matrix. If TransA = CblasNoTrans, $\text{[math]}$ . If TransA = CblasTrans, $\text{[math]}$ . If TransA = CblasConjTrans, $\text{[math]}$ . If TransA = CblasConjTrans, $\text{[math]}$ .	Input
transB_array	Enumeration type CBLAS_TRANSPOSE	Transposition sequence of matrix B. Whether matrix B is a conventional matrix, a transposed matrix, or a conjugate matrix. If TransB = CblasNoTrans, $\text{[math]}$ . If TransB = CblasTrans, $\text{[math]}$ . If TransB = CblasConjTrans, $\text{[math]}$ . If TransB = CblasConjTrans, $\text{[math]}$ .	Input
offsetc_array	Enumeration type CBLAS_OFFSET	Array of offsetc. If offsetc = CblasRowOffset, C_offset is a row vector. If offsetc = CblasColOffset, C_offset is a column vector. If offsetc = CblasFixOffset, C_offset is a scalar.	Input
m_array	Integer	Sequence of the number of rows in matrix op(A) and matrix C.	Input
n_array	Integer	Sequence of the number of columns in matrix op(B) and matrix C.	Input
k_array	Integer	Sequence of the number of columns in matrix op(A) and the number of rows in matrix op(B).	Input
alpha_array	Single-precision floating-point type.	Sequence of multiplication coefficients.	Input
a_array	The type is int8_t in _s88s32. The type is uint8_t in _u88s32.	Sequence of matrix A.	Input
lda_array	Integer	Sequence of the leading dimension of matrix A. For column-major matrices, if TransA = CblasNoTrans, lda must be at least max(1, m); otherwise, it must be at least max(1, k). For row-major matrices, if TransA = CblasNoTrans, lda must be at least max(1, k); otherwise, it must be at least max(1, m).	Input
oa_array	Integer	Sequence of the value of A_offset in the GEMM definition.	Input
b_array	The type is int8_t in _8s8s32. The type is uint8_t in _8u8s32.	Sequence of matrix B.	Input
ldb_array	Integer	Sequence of the leading dimension of matrix A. For column-major matrices, if TransB = CblasNoTrans, ldb must be at least max(1, k); otherwise, it must be at least max(1, n). For row-major matrices, if TransB = CblasNoTrans, ldb must be at least max(1, n); otherwise, it must be at least max(1, k).	Input
ob_array	Integer	Sequence of the value of B_offset in the GEMM definition.	Input
beta_array	Single-precision floating-point type.	Sequence of multiplication coefficients.	Input
c_array	Integer	Sequence of matrix C.	Input/Output
ldc_array	Integer	Sequence of the leading dimension of matrix C. For column-major matrices, ldc must be at least max(1, m); otherwise, it must be at least max(1, n).	Input
oc_array	Integer	Sequence of the value of C_offset in the GEMM definition.	Input
group_count	Integer	Number of groups. The value must be at least 0.	Input
group_size	Integer	Array with a size of group_count. The element group_size[i] specifies the number of matrices in group i.	Input

Dependencies

#include "kblas.h"

Examples

int m = 2, k = 3, n = 2, lda = 2, ldb = 3, ldc = 2; 
    int group_count = 2;
    int group_size[2] = {1, 1};
    int m_array[2] = {2, 2};
    int k_array[2] = {3, 3};
    int n_array[2] = {2, 2};
    int lda_array[2] = {2, 2};
    int ldb_array[2] = {3, 3};
    int ldc_array[2] = {2, 2};
    CBLAS_TRANSPOSE transA_array[2] = {CblasNoTrans, CblasNoTrans};
    CBLAS_TRANSPOSE transB_array[2] = {CblasNoTrans, CblasNoTrans};
    float alpha_array[2] = {1.0, 1.0};
    float beta_array[2] = {1.0, 1.0}; 
    CBLAS_OFFSET offestc_array[2] = {CblasFixOffset, CblasFixOffset};
    int32_t oc_array[2] = {0, 0};
    int8_t oa_array[2] = {0, 0};
    int8_t ob_array[2] = {0, 0};
    int8_t a_data[12] = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};
    int8_t b_data[12] = {1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1};
    int32_t c_data[8] = {1, 1, 1, 1, 1, 1, 1, 1};
    int8_t* a_array[2];
    int8_t* b_array[2];
    int32_t* c_array[2];
    a_array[0] = a_data;
    a_array[1] = a_data + m*k*group_size[0];
    b_array[0] = b_data;
    b_array[1] = b_data + k*n*group_size[0];
    c_array[0] = c_data;
    c_array[1] = c_data + m*n*group_size[0];
    cblas_gemm_batch_s8s8s32(CblasColMajor, transA_array, transB_array, offestc_array, m_array, n_array, k_array, alpha_array, a_array,
                      lda_array, oa_array, b_array, ldb_array, ob_array, beta_array, c_array, ldc_array, oc_array, group_count, group_size);

Parent topic: KML_BLAS-like extensions