Indicative Marking Scheme – Statistical Modelling
 
 
The following is meant only as a guide to the steps which can be taken in carrying out a regression analysis.  Circumstances may dictate other approaches, especially in the area of advanced Econometrics.  Students may adopt other approaches but the key aspect is one of analysis.
 
PART A   (overall weighting 30%)
 
This part is essentially a training ground for the second stage of the assessment.  It is expected that the student will undertake the basic steps of analysis required for an initial investigation of the data.  Mistakes will be expected as well as a lack of finesse in the statistical approaches.
 
The marks allocated are only indicative as it is the overall approach which is important.  So strengths in some areas may out way weaknesses or mistakes in others.
 
 
STEP 1  (10 %)
 
Decide on the model to be analysed     
      - choice of variables and whether lags are to be included
      - ‘a priori’ signs of the parameters i.e. anticipated sign of the slope coefficients or betas
      - mathematical form of the relationship
      - availability of suitable data
      - transformation/scaling of the data
 
 
STEP 2 (25%)
 
Check the integrity of the data
      - detection of outliers
      - detection of influential observations or points of leverage
      - tests for stability of the coefficients e.g. structural break
      - take corrective action if necessary
 
Use plots of standardised residuals against the explanatory variables, plots of HI, COOKD, DFITS etc.  Plots of standardised residuals/residuals can also indicate a problem with the structure e.g. non-linear relationship, structural breaks.  Normality test of residuals can also be useful for detecting problems either in the data or with the choice of variables.
 
 
 
 
STEP 3 (55%)
 
Initial interpretation of the regression results
      - correct sign of the coefficients
      - high R squared adjusted for multiple regression
      - significant F ratio (model fits the data)
      - significant t ratios for the coefficients (model contains relevant explanatory
        variables)
      - low error sum of squares
      - Durbin-Watson near 2
      -  normality test not rejected
      - evidence of influential observations
      - evidence of non-linearity
 
May need to revise the choice of variables in the light of the results e.g. variables whose coefficients have insignificant t-ratios maybe removed from the model (take care though). Would test this by using the WALD (F-test).  Use of  information criteria such as AIC to choose model.
 
 
STEP 4  (10%)
 
A sensible conclusion and statement of findings.
 
 
 
 
PART B  (weighting 70%)
 
Whilst there will be duplication of the initial stages of the analysis, it is expected that there will now be more rigour applied to the analysis in view of the availability of the  feedback from Part A.  This part also involves the analysis of a new data set.
 
 
STEP 1  (5 %)
 
Decide on the model to be analysed     
      - choice of variables and whether lags are to be included
      - ‘a priori’ signs of the parameters i.e. anticipated sign of the slope coefficients or betas
      - mathematical form of the relationship
      - availability of suitable data
      - transformation/scaling of the data
 
 
 
 
 
STEP 2 (10%)
 
Check the integrity of the data
      - detection of outliers
      - detection of influential observations or points of leverage
      - tests for stability of the coefficients e.g. structural break
      - take corrective action if necessary
 
Use plots of standardised residuals against the explanatory variables, plots of HI, COOKD, DFITS etc.  Plots of standardised residuals/residuals can also indicate a problem with the structure e.g. non-linear relationship, structural breaks.  Normality test of residuals can also be useful for detecting problems either in the data or with the choice of variables.
 
STEP 3 (25 %)
 
Full interpretation of regression results
      - correct sign of the coefficients
      - high R squared adjusted for multiple regression
      - significant F ratio (model fits the data)
      - significant t ratios for the coefficients (model contains relevant explanatory
        variables)
      - low error sum of squares
      - Durbin-Watson near 2
      -  normality test not rejected
      - evidence of influential observations
      - evidence of non-linearity
- construction of confidence intervals for the coefficients where applicable
      - stability of coefficients (Chow test)
      - forecasting (possible use of PRED)
      - confidence and prediction intervals for forecasts
      - could consider use of hold-out sample if there is enough data in order to test the forecasting accuracy of the model (might be left to last if revision of model takes place)
 
 
STEP 4  (50 %)
 
Check that the regression assumptions have been met
      - normality of residuals (Anderson-Darling etc, Jarque-Berra test, plus others)
      - Multicollinearity (Correlation, VIF, high R square and low t-ratios, auxiliary
        regression)
      - Heteroscedasticity (plot of residuals against explanatory variables, White’s test,
        Breusch-Pagan test, Park Test, Goldfield-Quandt Test etc)
      - Autocorrelation (Time series plot of residuals, DW, Correlogram, Ljung-Box
        Statistic, LM test etc)
-Ramsey test for model misspecification
- Revision of model in the light of the results, use of WALD Test and information criteria, re-testing of regression assumptions.
 
STEP 5  (10%)
 
A sensible conclusion and statement of findings together with any limitations etc of the analysis found.
     
There are also many other procedures and tests that can be undertaken in addition to the above.  These are only meant as guidelines. 
 
 
The marks allocated are only indicative as it is the overall approach which is important.  So strengths in some areas may out way weaknesses or mistakes in others.