Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Spearman

Model API Type

Function API

ML API

def corr(dataset: Dataset[_], column: String, method: String): DataFrame

ML API

  • Function description

    Output the clustering result after you input sample data in the dataset format and call the fitPredict API.

  • Input and output
    1. Package name: org.apache.spark.ml.stat
    2. Class name: Correlation
    3. Method name: corr
    4. Input: training sample data (Dataset[_]). The following are mandatory fields.

      Parameter

      Value Type

      Description

      data

      Dataset[Vector]

      Matrix, which is stored by row.

      column

      String

      Specifies columns for correlation matrix calculation.

      method

      String

      Correlation matrix method, which can be spearman.
    5. Algorithm parameters

      Parameter

      Value Type

      Default Value

      Description

      method

      String

      pearson

      Method for solving the correlation matrix

      Code API example:

      1
      val mat_df = ml.stat.Correlation.corr(data,"matrix", method = "spearman")
    6. Output: Pearson correlation matrix

      Parameter

      Value Type

      Description

      df

      DataFrame

      Spearman correlation matrix. The column name is column + method.
  • Example
    val mat = stat.Correlation.corr(data, "matrix", "spearman")
Parent topic: Feature Engineering