Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Linear Regression

The Linear Regression algorithm provides ML APIs.

Model API Type

Function API

ML API

def fit(dataset: Dataset[_]):LinearRegressionModel

def fit(dataset: Dataset[_], paramMap: ParamMap): LinearRegressionModel

def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*):LinearRegressionModel

def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): Seq[LinearRegressionModel]

ML API

  • Function description

    Output the Linear Regression model after you input sample data in dataset format and call the fit API.

  • Input and output
    1. Package name: package org.apache.spark.ml.regression
    2. Class name: LinearRegression
    3. Method name: fit
    4. Input: training sample data (Dataset[_]). The following are mandatory fields.

      Parameter

      Value Type

      Default Value

      Description

      labelCol

      Double

      label

      Label

      featuresCol

      Vector

      features

      Feature label
    5. Parameters optimized based on native algorithms
      def setRegParam(value: Double): LinearRegression.this.type
def setFitIntercept(value: Boolean): LinearRegression.this.type
def setStandardization(value: Boolean): LinearRegression.this.type
def setElasticNetParam(value: Double): LinearRegression.this.type
def setMaxIter(value: Int): LinearRegression.this.type
def setTol(value: Double): LinearRegression.this.type
def setWeightCol(value: String): LinearRegression.this.type
def setSolver(value: String): LinearRegression.this.type
def setAggregationDepth(value: Int): LinearRegression.this.type
def setLoss(value: String): LinearRegression.this.type
def setEpsilon(value: Double): LinearRegression.this.type

      An example is provided as follows:

       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
      import org.apache.spark.ml.param.{ParamMap, ParamPair}

val linR = new LinearRegression()
// Define the def fit(dataset: Dataset[_], paramMap: ParamMap) API parameter.
val paramMap = ParamMap(linR.maxIter -> maxIter)
.put(linR.regParam, regParam)

// Define the def fit(dataset: Dataset[_], paramMaps: Array[ParamMap]): API parameter.
val paramMaps: Array[ParamMap] = new Array[ParamMap](2)
for (i <- 0 to  2) {
paramMaps(i) = ParamMap(linR.maxIter -> maxIter)
.put(linR.regParam, regParam)
}//Assign a value to paramMaps.

// Define the def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*) API parameter.
val regParamPair = ParamPair(linR.regParam, regParam)
val maxIterParamPair = ParamPair(linR.maxIter, maxIter)
val tolParamPair = ParamPair(linR.tol, tol)

// Call the fit APIs.
model = linR.fit(trainingData)
model = linR.fit(trainingData, paramMap)
models = linR.fit(trainingData, paramMaps)
model = linR.fit(trainingData, regParamPair, maxIterParamPair, tolParamPair)
    6. Output: LinearRegressionModel. The following table lists the field output in model prediction.

      Parameter

      Value Type

      Default Value

      Description

      predictionCol

      Int

      prediction

      predictionCol
  • Example
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    import org.apache.spark.ml.regression.LinearRegression

// Load training data
val training = spark.read.format("libsvm")
  .load("data/mllib/sample_linear_regression_data.txt")

val lr = new LinearRegression()
  .setMaxIter(10)
  .setRegParam(0.3)
  .setElasticNetParam(0.8)

// Fit the model
val lrModel = lr.fit(training)

// Summarize the model over the training set and print out some metrics
val trainingSummary = lrModel.summary