Rate This Document

Findability

Accuracy

Completeness

Readability

?gemm_batch

Compute the product of a general matrix and a matrix in batches. This function is described as follows:

$\text{[math]}$

The value of op(X) may be $\text{[math]}$ . alpha and beta are multiplication coefficients; op(A) is an m*k matrix; op(B) is a k*n matrix, and C is an m*n matrix.

Note: Batch GEMM is applicable to large batch sizes of small matrices. It is less effective for small batch sizes of large matrices.

Interface Definition

C interface:

void cblas_sgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const float *alpha_array, const float **a_array, const BLASINT *lda_array, const float **b_array,

const BLASINT *ldb_array, const float *beta_array, float **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_dgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const double *alpha_array, const double **a_array, const BLASINT *lda_array, const double **b_array,

const BLASINT *ldb_array, const double *beta_array, double **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_cgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const float *alpha_array, const float **a_array, const BLASINT *lda_array, const float **b_array,

const BLASINT *ldb_array, const float *beta_array, float **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_zgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const double *alpha_array, const double **a_array, const BLASINT *lda_array, const double **b_array,

const BLASINT *ldb_array, const double *beta_array, double **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_hgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const __fp16 *alpha_array, const __fp16 **a_array, const BLASINT *lda_array, const __fp16 **b_array,

const BLASINT *ldb_array, const __fp16 *beta_array, __fp16 **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

void cblas_bgemm_batch(const enum CBLAS_ORDER order, const enum CBLAS_TRANSPOSE *transA_array,

const enum CBLAS_TRANSPOSE *transB_array, const BLASINT *m_array, const BLASINT *n_array, const BLASINT *k_array,

const __bf16 *alpha_array, const __bf16 **a_array, const BLASINT *lda_array, const __bf16 **b_array,

const BLASINT *ldb_array, const __bf16 *beta_array, __bf16 **c_array, const BLASINT *ldc_array,

const BLASINT group_count, const BLASINT *group_size);

Parameters

Parameter	Type	Description	Input/Output
order	Enumeration type CBLAS_ORDER	Whether the matrix is in row- or column-major order.	Input
transA_array	Enumeration type CBLAS_TRANSPOSE	Array of transposition states for matrix A. Whether matrix A is non-transposed, transposed, or conjugated. If TransA = CblasNoTrans, $\text{[math]}$ . If TransA = CblasTrans, $\text{[math]}$ . If TransA = CblasConjTrans, $\text{[math]}$ . If TransA = CblasConjTrans, $\text{[math]}$ .	Input
transB_array	Enumeration type CBLAS_TRANSPOSE	Array of transposition states for matrix B. Whether matrix B is non-transposed, transposed, or conjugated. If TransB = CblasNoTrans, $\text{[math]}$ . If TransB = CblasTrans, $\text{[math]}$ . If TransB = CblasConjTrans, $\text{[math]}$ . If TransB = CblasConjTrans, $\text{[math]}$ .	Input
m_array	Integer	Array indicating the number of rows in matrices op(A) and C.	Input
n_array	Integer	Array indicating the number of columns in matrices op(B) and C.	Input
k_array	Integer	Array indicating the number of columns in matrix op(A) and the number of rows in matrix op(B).	Input
alpha_array	Single-precision floating-point type for sgemm Double-precision floating-point type for dgemm FP16 type for hgemm BF16 type for bgemm	Array of multiplication coefficients.	Input
a_array	Single-precision floating-point type for sgemm Double-precision floating-point type for dgemm FP16 type for hgemm BF16 type for bgemm	Array of matrix A.	Input
lda_array	Integer	Array of leading dimensions for matrix A. For a column-store matrix with TransA = CblasNoTrans, lda must be at least max(1, m); otherwise, lda must be at least max(1, k). For a row-store matrix with TransA = CblasNoTrans, lda must be at least max(1, k); otherwise, lda must be at least max(1, m).	Input
b_array	Single-precision floating-point type for sgemm Double-precision floating-point type for dgemm FP16 type for hgemm BF16 type for bgemm	Array of matrix B.	Input
ldb_array	Integer	Sequence of the leading dimension of matrix B. For a column-store matrix with TransB = CblasNoTrans, ldb must be at least max(1, k); otherwise, ldb must be at least max(1, n). For a row-store matrix with TransB = CblasNoTrans, ldb must be at least max(1, n); otherwise, ldb must be at least max(1, k).	Input
beta_array	Single-precision floating-point type for sgemm Double-precision floating-point type for dgemm FP16 type for hgemm BF16 type for bgemm	Array of multiplication coefficients.	Input
c_array	Single-precision floating-point type for sgemm Double-precision floating-point type for dgemm FP16 type for hgemm BF16 type for bgemm	Array of matrix C.	Input/Output
ldc_array	Integer	Array of leading dimensions for matrix C. For a column-store matrix, ldc must be at least max(1, m); otherwise, ldc must be at least max(1, n).	Input
group_count	Integer	Number of groups. The value must be at least 0.	Input
group_size	Integer	Array with a size of group_count. The element group_size[i] specifies the number of matrices in the i-th group.	Input

Dependencies

#include "kblas.h"

Examples

    int m = 2, k = 3, n = 2, lda = 2, ldb = 3, ldc = 2; 
    int group_count = 2;
    int group_size[2] = {1, 1};
    int m_array[2] = {2, 2};
    int k_array[2] = {3, 3};
    int n_array[2] = {2, 2};
    int lda_array[2] = {2, 2};
    int ldb_array[2] = {3, 3};
    int ldc_array[2] = {2, 2};
    CBLAS_TRANSPOSE transA_array[2] = {CblasNoTrans, CblasNoTrans};
    CBLAS_TRANSPOSE transB_array[2] = {CblasNoTrans, CblasNoTrans};
    float alpha_array[2] = {1.0, 1.0};
    float beta_array[2] = {2.0, 2.0}; 
     /* 
     * A: 
     *     0.340188,       0.411647,       -0.222225, 
     *     -0.105617,      -0.302449,      0.053970, 
     *     0.283099,       -0.164777,      -0.022603, 
     *     0.298440,       0.268230,       0.128871, 
     * B: 
     *     -0.135216,      0.416195,
     *     0.013401,       0.135712,
     *     0.452230,       0.217297,
     *     -0.358397,      -0.257113, 
     *      0.106969,      -0.362768,
     *     -0.483699,      0.304177
     * C: 
     *     -0.343321,      0.498924,       0.112640,       -0.006417, 
     *     -0.099056,      -0.281743,      -0.203968,      0.472775
     */
    float a_data[12] = { 0.340188,  -0.105617, 0.283099,  0.298440, 0.411647,  -0.302449,
                         -0.164777, 0.268230,  -0.222225, 0.053970, -0.022603, 0.128871 };
    float b_data[12] = { -0.135216, 0.416195,  0.013401, 0.135712,  0.452230,  0.217297,
                         -0.358397, -0.257113, 0.106969, -0.362768, -0.483699, 0.304177 };
    float c_data[8] = { -0.343321, -0.099056, -0.370210, -0.391191, 0.498924, -0.281743, 0.012932, 0.339112 };
    float* a_array[2];
    float* b_array[2];
    float* c_array[2];
    a_array[0] = a_data;
    a_array[1] = a_data + m*k*group_size[0];
    b_array[0] = b_data;
    b_array[1] = b_data + k*n*group_size[0];
    c_array[0] = c_data;
    c_array[1] = c_data + m*n*group_size[0];
    cblas_sgemm_batch(CblasColMajor, transA_array, transB_array, m_array, n_array, k_array, alpha_array, a_array,
                      lda_array, b_array, ldb_array, beta_array, c_array, ldc_array, group_count, group_size);

Parent topic: KML_BLAS-like extensions