Java 类名：com.alibaba.alink.operator.batch.regression.GlmEvaluationBatchOp
Python 类名：GlmEvaluationBatchOp

功能介绍

GLM(Generalized Linear Model)又称为广义线性回归模型，是一种常用的统计模型，也是一种非线性模型族，许多常用的模型都属于广义线性回归。
它描述了响应和预测因子之间的非线性关系。广义线性回归模型具有线性回归模型的广义特征。响应变量遵循正态、二项式、泊松分布、伽马分布或逆高斯分布，链接函数f定义了μ和预测值的线性组合之间的关系。
GLM功能包括GLM训练，GLM预测(批和流)和GLM评估, 其中训练使用迭代最小二乘方法。

算法使用

| 分布 | 连接函数 | 对应算法 | | —- | —- | —- |

| 二项分布 | Logit | 逻辑回归 |

| 多项分布 | Logit | softmax |

| 高斯分布 | Identity | 线性回归 |

| Poisson分布 | Log | Possion回归 |

文献或出处

[1] https://en.wikipedia.org/wiki/Generalized_linear_model

代码示例

Python 代码

  1. from pyalink.alink import *
  3. import pandas as pd
  5. useLocalEnv(1)
  7. df = pd.DataFrame([
  8.     [1.6094,118.0000,69.0000,1.0000,2.0000],
  9.     [2.3026,58.0000,35.0000,1.0000,2.0000],
  10.     [2.7081,42.0000,26.0000,1.0000,2.0000],
  11.     [2.9957,35.0000,21.0000,1.0000,2.0000],
  12.     [3.4012,27.0000,18.0000,1.0000,2.0000],
  13.     [3.6889,25.0000,16.0000,1.0000,2.0000],
  14.     [4.0943,21.0000,13.0000,1.0000,2.0000],
  15.     [4.3820,19.0000,12.0000,1.0000,2.0000],
  16.     [4.6052,18.0000,12.0000,1.0000,2.0000]
  17. ])
  19. source = BatchOperator.fromDataframe(df, schemaStr='u double, lot1 double, lot2 double, offset double, weights double')
  21. featureColNames = ["lot1", "lot2"]
  22. labelColName = "u"
  24. # train
  25. train = GlmTrainBatchOp()\
  26.                 .setFamily("gamma")\
  27.                 .setLink("Log")\
  28.                 .setRegParam(0.3)\
  29.                 .setMaxIter(5)\
  30.                 .setFeatureCols(featureColNames)\
  31.                 .setLabelCol(labelColName)
  33. source.link(train)
  35. # predict
  36. predict =  GlmPredictBatchOp()\
  37.                 .setPredictionCol("pred")
  39. predict.linkFrom(train, source)
  42. # eval
  43. eval =  GlmEvaluationBatchOp()\
  44.                 .setFamily("gamma")\
  45.                 .setLink("Log")\
  46.                 .setRegParam(0.3)\
  47.                 .setMaxIter(5)\
  48.                 .setFeatureCols(featureColNames)\
  49.                 .setLabelCol(labelColName)
  51. eval.linkFrom(train, source)
  53. predict.lazyPrint(10)
  54. eval.print()

Java 代码

  1. import org.apache.flink.types.Row;
  3. import com.alibaba.alink.operator.batch.BatchOperator;
  4. import com.alibaba.alink.operator.batch.regression.GlmEvaluationBatchOp;
  5. import com.alibaba.alink.operator.batch.regression.GlmPredictBatchOp;
  6. import com.alibaba.alink.operator.batch.regression.GlmTrainBatchOp;
  7. import com.alibaba.alink.operator.batch.source.MemSourceBatchOp;
  8. import org.junit.Test;
  10. import java.util.Arrays;
  11. import java.util.List;
  13. public class GlmEvaluationBatchOpTest {
  14.     @Test
  15.     public void testGlmEvaluationBatchOp() throws Exception {
  16.         List <Row> df = Arrays.asList(
  17.             Row.of(1.6094, 118.0000, 69.0000, 1.0000, 2.0000),
  18.             Row.of(2.3026, 58.0000, 35.0000, 1.0000, 2.0000),
  19.             Row.of(2.7081, 42.0000, 26.0000, 1.0000, 2.0000),
  20.             Row.of(2.9957, 35.0000, 21.0000, 1.0000, 2.0000),
  21.             Row.of(3.4012, 27.0000, 18.0000, 1.0000, 2.0000),
  22.             Row.of(3.6889, 25.0000, 16.0000, 1.0000, 2.0000),
  23.             Row.of(4.0943, 21.0000, 13.0000, 1.0000, 2.0000),
  24.             Row.of(4.3820, 19.0000, 12.0000, 1.0000, 2.0000),
  25.             Row.of(4.6052, 18.0000, 12.0000, 1.0000, 2.0000)
  26.         );
  27.         BatchOperator <?> source = new MemSourceBatchOp(df,
  28.             "u double, lot1 double, lot2 double, offset double, weights double");
  29.         String[] featureColNames = new String[] {"lot1", "lot2"};
  30.         String labelColName = "u";
  31.         BatchOperator <?> train = new GlmTrainBatchOp()
  32.             .setFamily("gamma")
  33.             .setLink("Log")
  34.             .setRegParam(0.3)
  35.             .setMaxIter(5)
  36.             .setFeatureCols(featureColNames)
  37.             .setLabelCol(labelColName);
  38.         source.link(train);
  39.         BatchOperator <?> predict = new GlmPredictBatchOp()
  40.             .setPredictionCol("pred");
  41.         predict.linkFrom(train, source);
  42.         BatchOperator <?> eval = new GlmEvaluationBatchOp()
  43.             .setFamily("gamma")
  44.             .setLink("Log")
  45.             .setRegParam(0.3)
  46.             .setMaxIter(5)
  47.             .setFeatureCols(featureColNames)
  48.             .setLabelCol(labelColName);
  49.         eval.linkFrom(train, source);
  50.         predict.lazyPrint(10);
  51.         eval.print();
  52.     }
  53. }

运行结果

预测结果

| u | lot1 | lot2 | offset | weights | pred | | —- | —- | —- | —- | —- | —- |

| 0 | 1.6094 | 118.0 | 69.0 | 1.0 | 2.0 |

| 1 | 2.3026 | 58.0 | 35.0 | 1.0 | 2.0 |

| 2 | 2.7081 | 42.0 | 26.0 | 1.0 | 2.0 |

| 3 | 2.9957 | 35.0 | 21.0 | 1.0 | 2.0 |

| 4 | 3.4012 | 27.0 | 18.0 | 1.0 | 2.0 |

| 5 | 3.6889 | 25.0 | 16.0 | 1.0 | 2.0 |

| 6 | 4.0943 | 21.0 | 13.0 | 1.0 | 2.0 |

| 7 | 4.3820 | 19.0 | 12.0 | 1.0 | 2.0 |

| 8 | 4.6052 | 18.0 | 12.0 | 1.0 | 2.0 |

评估结果

| summary | | —- |

| {“rank”:3,”degreeOfFreedom”:6,”residualDegreeOfFreeDom”:6,”residualDegreeOfFreedomNull”:8,”aic”:9702.08856968678,”dispersion”:0.01600672089664272,”deviance”:0.09638590199190636,”nullDeviance”:0.8493577599031797,”coefficients”:[0.007797743508551773,-0.031175844426501245],”intercept”:1.6095243247335171,”coefficientStandardErrors”:[0.030385113783611032,0.05301723001061871,0.10937960484662167],”tValues”:[0.2566303869744822,-0.5880323136508093,14.715031444760513],”pValues”:[0.8060371545111102,0.5779564640149484,6.188226474801439E-6]} |