Flexible Regression And Smoothing : Using GAMLS...

The Generalized Additive Model for Location, Scale and Shape (GAMLSS) is an approach to statistical modelling and learning. GAMLSS is a modern distribution-based approach to (semiparametric) regression. A parametric distribution is assumed for the response (target) variable but the parameters of this distribution can vary according to explanatory variables using linear, nonlinear or smooth functions. In machine learning parlance, GAMLSS is a form of supervised machine learning.

Flexible regression and smoothing : using GAMLS...

In particular, the GAMLSS statistical framework enables flexible regression and smoothing models to be fitted to the data. The GAMLSS model assumes the response variable has any parametric distribution which might be heavy or light-tailed, and positively or negatively skewed. In addition, all the parameters of the distribution [location (e.g., mean), scale (e.g., variance) and shape (skewness and kurtosis)] can be modeled as linear, nonlinear or smooth functions of explanatory variables.

The functions cs() and scs() are using the cubic smoothing splines function smooth.spline() to do smoothing. They take a vector and return it with several attributes. The vector is used in the construction of the model matrix. The functions do not do the smoothing, but assigns the attributes to the vector to aid gamlss in the smoothing.The function doing the smoothing is gamlss.cs().This function use the R function smooth.spline() which is then used by the backfitting function additive.fit() which is based on the original GAM implementation described in Chambers and Hastie (1992).The function gamlss.scs() differs from the function cs() in that allows cross validation of the smoothing parameters unlike the cs() which fixes the effective degrees of freedom, df. Note that the recommended smoothing function is now the function pb() which allows the estimation of the smoothing parameters using a local maximum likelihood. The function pb() is based on the penalised beta splines (P-splines) of Eilers and Marx (1996).

Note that cs() and scs() functions behave differently at their default values that is if df and lambda are not specified.cs(x) by default will use 3 extra degrees of freedom for smoothing for x.scs(x) by default will estimate lambda (and the degrees of freedom) automatically using generalised cross validation (GCV).Note that if GCV is used the convergence of the gamlss model can be less stable compared to a model where the degrees of freedom are fixed. This will be true for small data sets.

As the explanatory variables can be included in any of the regression structures of all parameters, there are some procedures to select the additive terms. In this paper, we are using the so-called Strategy A [11,17], a stepwise-based method applied to select the terms for each model parameters based on the Akaike information criterion (AIC) [18]. This approach can be achieved using the stepGAICAll.A() function in the gamlss package.

Note that, for both regression structures, a P-spline [12,13] was considered due to the nonlinear relationship between xi and both parameters. The smoothing parameters λ for μ and σ are 3.23 and 2.85, respectively.

where the fitted smoothing parameter λ for μ is 17.47. Note that, although temperature was considered to model both regression structures, the smoothing function was only necessary to model the mode μ.

The key point within the discussion in application sections in papers that develop new (log-)location models is usually based on goodness-of-fit measures, such as AIC and BIC. Focusing on this specifically point, let us suppose a response variable Y that follows a Gaussian distribution, and an explanatory variable X which directly affects both the mean μ and standard deviation σ of Y. To fit such behaviors, should we build a location model or a heteroscedastic model (GAMLSS in other words) or propose a new location model? The natural choice here seems to be the GAMLSS (distributional regression) approach.

The step length for each of the parameters mu, sigma, nu or tau is very useful to aid convergence if the parameter has a fully parametric model.However using a step length is not theoretically justified if the model for the parameter includes one or more smoothing terms, (even thought it may give a very approximate result).

GAMs are simply a class of statistical Models in which the usual Linear relationship between the Response and Predictors are replaced by several Non linear smooth functions to model and capture the Non linearities in the data.These are also a flexible and smooth technique which helps us to fit Linear Models which can be either linearly or non linearly dependent on several Predictors \(X_i\) to capture Non linear relationships between Response and Predictors.In this article I am going to discuss the implementation of GAMs in R using the 'gam' package.Simply saying GAMs are just a Generalized version of Linear Models in which the Predictors \(X_i\) depend Linearly or Non linearly on some Smooth Non Linear functions like Splines , Polynomials or Step functions etc.

Generalized Additive Models are a very nice and effective way of fitting Linear Models which depends on some smooth and flexible Non linear functions fitted on some predictors to capture Non linear relationships in the data.Best part is that they lead to interpretable Models. We can easily mix terms in GAMs,some linear and some Non Linear terms and then compare those Models using the anova() function which performs a Anova test for goodness of fit.The Nonlinear terms on Predictors \(X_i\) can be anything from smoothing splines , natural cubic splines to polynomial functions or step functions etc. GAMs are additive in nature , which means there are no interaction terms in the Model. 041b061a72