M.R.L.N. Panchanana
(M.Sc Statistics from University of Hyderabad, India)
M.Tech (Compueter Science & Technology
from Jawaharlal Nehru University, Delhi)
AbstractFitting a parametric distribution to a data set is a key step in analyzing the data. In many statistical procedures like ANOVA, several hypothesis tests are performed assuming the data as normal. In this presentation, it is discussed how to perform statistical techniques using R Software, to fit a statistical distribution to oneday cricket Score made by Sachin Ramess Tendulkar. DNB (Does Not Batted) cases are not considered.
Introduction
Few
functions used in R® is useful in deciding, which Univariate
distribution is suitable to the specified variable from the data set. It provides
Univariate discrete and continuous distributions. Before going to perform, goodness of fit test,
it is advisable to know the characteristics of the data by descriptive
statistics methods like summary
and stem and
Density estimation methods like hist,
boxplot, ecdf, density, fitdistr,ks.test. Graphical illustrative methods are qqplot and qqline.
Oneday cricket Runs
> library(sas7bdat)
Read the data file as follows:SACHIN <- read.sas7bdat("D:/Modeling/sachin.sas7bdat")
The following commands invoke the function and computes mean, median, and different percentiles.
> summary(SACHIN)
RUNS
Min. : 0.00
1st Qu .
: 8.00
Median : 28.50
Mean : 40.77
3rd Qu . : 63.00
Max. :200.00
Sachin's Avarage is around 40 runs. Here, we considered every match whether he is out or notout.
With quantile function, distribution of runs at each percentile can be observed.
> quantile(sachin,seq(0, 1, by=.1))Median : 28.50
Mean : 40.77
3rd Qu . : 63.00
Max. :200.00
Sachin's Avarage is around 40 runs. Here, we considered every match whether he is out or notout.
With quantile function, distribution of runs at each percentile can be observed.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0.0 2.0 5.0 11.0 18.4 28.5 39.0 53.0 72.8 100.0 200.0
To have a look at the shape of the distribution of the Runs,
Stem and leaf plot can be generated with the following statements.
> sachin <-
as.numeric(SACHIN$RUNS)> stem(sachin)
0 |
00000000000000000000111111111111111111112222222222222222223333333333+46
1 |
00000011111111112222233344444445555555666677777788888889999
2 |
00011111111222233344445555667777778888888999
3 |
00000011112222223444555556666666777788889999999
4 | 0000011112334444555556777888899
5 | 0012222333334444557777
6 | 0112222233334555555677788999
7 | 0012347789
8 | 011122223455678899
9 | 011333334556778999
10 | 000000112455
11 | 001234457788
12 | 002234778
13 | 4789
14 | 0111366
15 | 2
16 | 3
17 | 5
18 | 6
19 |
20 | 0
The Graph shows the runs made by Sachin in nice formatt. For Example, Sachin made 200 runs is shown in the lowest part and he did not make any runs in 190s. Histograms and Nonparametric density estimation methods
+ xlab="Oneday match Runs",
+ ylab="count",
+ main="Oneday Runs made by Sachin Ramesh Tendulkar")
The output is:
> bins=c(seq(0,150,by=10),200)
> hist(sachin,breaks=bins,col="darkblue", border = " yellow ", xlab="Oneday match Runs", ylab="count", main="Oneday Runs made by Sachin Ramesh Tendulkar")
> lines(density(sachin), col= "red")
> rug(sachin)
The Output is as follows:> hist(sachin,breaks=bins,col="darkblue", border = " yellow ", xlab="Oneday match Runs", ylab="count", main="Oneday Runs made by Sachin Ramesh Tendulkar")
> lines(density(sachin), col= "red")
> rug(sachin)
You can observe that there is difference in the shape of the
histogram by giving the bins as 0-10, 11-20, …,150-200. With rugs() function
you can observe the data points on X-axis. You can clearly identify the runs
mad in each interval, particularly 100+ runs.
You can observe the Outliers in box plot after 149+ runs.
With the above statistics and graphs, the distribution is not Symmetric. Once again, we can look at summary statistics.
Load the library "fBasics" and look at the Statistics:
> library(fBasics)
> basicStats(sachin)
sachinnobs 452.000000
NAs 0.000000
Minimum 0.000000
Maximum 200.000000
1. Quartile 8.000000
3. Quartile 63.000000
Mean 40.765487
Median 28.500000
Sum 18426.000000
SE Mean 1.883299
LCL Mean 37.064357
UCL Mean 44.466617
Variance 1603.159959
Stdev 40.039480
Skewness 1.152907
Kurtosis 0.753945
With high variance, mean is almost equal to standard deviation. the distribution is heavily weighted far from the mean. Since skewness > 0, the distribution is right skewed as shown in the Histogram too, most values are concentrated on left side of the mean, with extreme values to the right. with all the above interpretation, Symmetric distributions will not be a fit to the data.
You can estimate the density function and Cumulative density function as follows:
> plot(density(sachin),main="Density estimate of sachin’s Oneday Runs")
Quantile-Quantile Plots
Before,
going to decide the distribution, examine with other graphical procedures,
Quantile-Quantile plots and Probability plots. The quantile of a sample is the
data point corresponding to a given fraction of the data. A one-sample quantile
plot looks like a cumulative sample distribution function. qqnorm()is used
to test the goodness of fit of a Normal distribution and qqplot() for any kind of other distribution. To plot
Q-Q plot for variable RUNS, the input is:
> x.teo<-rweibull(n=452,shape=0.83908325, scale=37.62877983)
> qqplot(sachin,x.teo,main="QQ-plot for Weibull distribution")
> abline(0,1)
fitdistr
Coverted 0 RUNS to 0.1; From all the above methods, the variable with positive numbers greater than zero may follow some asymmetric distributions, with more observations in the first intervals and decreasing number of observations in the later intervals.
> SACHIN1 <-read.sas7bdat("D:/Modeling/SYSTAT/Cluster/GRAPHS/Rdatastore/sachin1.sas7bdat")
> sachin1 <- as.numeric(SACHIN1$RUNS)
We can see, whether any of the weibul or exponential
distributions fit to the data with the help of fitdistr.
Parameters of the distribution can be calculated by maximum likelihood
estimation methods. The following
statements will fit distributions to the variable RUNS.
> library(MASS) The Output is:
rate
0.024527892
(0.001153695)
The Output is:
shape scale
0.83908325 37.62877983
( 0.03253613) ( 2.20929924)
Goodness of fit tests
> ks.test(sachin1,"pexp",rate=0.024527892)
The output is as follows:
One-sample Kolmogorov-Smirnov test
data: sachin1
D = 0.0917, p-value = 0.0009926
alternative hypothesis:
two-sided
>ks.test(sachin1,"pweibull",shape=0.83908325
,scale=37.62877983)
The output is as follows:
One-sample
Kolmogorov-Smirnov testdata: sachin1
D = 0.0596, p-value = 0.08023
alternative hypothesis: two-sided
We accept null
hypothesis that the data follow a Weibull distribution because the p-value is
enough higher than significance levels usually referred. Since shape
parameter of weibull is <1, it is harder to get Sachin out as his score
increases.
In an
unpublished work done by me and Prof Traimbakam Krishnan during 2004 to 2007,
analysis was done on censored data for several famous players. Not-outs are considered as right censored.The results will be kept in separate Article.
Conclusions
By using function available in R software, we can analyze
univariate data very well.
Similar kind of Analysis can be done on all famous oneday match
players.
References
[1] Johnson, N. L., Kotz, S., and, Balakrishnan, N. (1994). Univariate
continuous distributions.
Vol. 1, 2nd ed. :New York: John Wiley & Sons.
[2] Johnson, N. L., Kotz, S., and Balakrishnan, N. (1995). Univariate
continuous distributions.Vol.
2, 2nd ed. :New York: John Wiley & Sons.
[3] Johnson, N. L., Kotz, S., and Kemp, A. W. (1993). Univariate
discrete distributions. 2nd ed.: New
York: John Wiley & Sons.
[4] A. Law and D. Kelton. Simulation
Modelling and Analysis. McGraw Hill, New York, 1990.
[5] http://cran.r-project.org/doc/manuals/R-intro.pdf
Nice work! How do you handle not-outs if we want to understand probability of failure?
ReplyDeleteNot-outs can be considered as right censored and Survival Model will be fitted
ReplyDelete