Lecture 1
+Zheyuan Wu
+2023/1/18
+ +++It was exactly a year before, I finished the R course with markdown, +lol.
+
Introduction
+Check syllabus
+Discussion Board will be maintained
+Textbook is recommended (We will not include materials in Chapter
+11)
+You can access course material under modules
+R and RStudio
+Calculators, (including statistical functions is recommended but not
+required)
+Quizes with 2 drops 3 subsession absent maximum
About R (Base R)
+-
+
- R is a very powerful and popular statistical software +
- R is open source and FREE +
- R can be used on many OS: windows, Mac, Unix, Linux … +
- R is an efficient data managing and storage facility +
- R is the most widely used statistical software in research +
- R has a sharp learning curve and good documentation support +
Learning R
+-
+
- Installation and interface of R (and RStudio) +
- R console, script, workspace, working directory, libraries (base R +for most of the time) +
- R code: comments, help +
?lm## starting httpd help server ... done# R arise from python, the comment like this is really nice.-
+
- Data structure in R: scaler, vector, matrix, array, lists, data +frame +
- Data type in R：class (numeric, factor, character, user defined), +names, attributes, +
- Summary statistics： mean, sd, cor, var, plot. hist, summary +
- In and out of R: workspace, csv/excel data +
- Use RMarkdown to write dynamic/interactive statistical analysis +report +
Statistics is learning from data
+Statistc is the art and science of +learning from data.
+-
+
- Art comes form the various creative and informative ways to +visualize, summarize, and analyze data +
- Science comes form using applied and theoretic mathematics and +probability to make objective decisions. +
An analysis that does not contain both aspects is often +incomplete and difficult to understand and use.
+-
+
- Note, “statistics” is also the plural of “statistic” which is a +numerical fact of summary. Example: average/mean, variance, range, +… +
- Formally, a statistc is a function of data. +
- Key: understand and quantify +uncertainty/variability. +
Scope of Statistics
+Formulation of the problem Collecting the data:
+-
+
Observation study or an experiment?
+
+How many Subjects?
+
+How do we decide what subject to collect? what factors to +collect?
+Summarizing the data
+Numerical description
+
+Graphic description
+
Analyzing, interpreting, and communicating the results
+-
+
- Modeling
+
+ - Estimation
+
+ - Hypothesis testing
+
+ - Interpretation +
Lecture_10
+Zheyuan Wu
+2023-02-10
+ +++The last session is review session
+
Random variables
+A random variable is a function that associates with number +with each outcome of the sample space of a random experiment.
+A discrete random variable is a random variable whose sample +space has a finite or at most a countably infinite number of values
+Example: Toss a fair coin and let X be the number of tosses until the +third head occurs.
+\[ +S_X=\{3,4,5,6...\} infinite,ountable +\]
+A continuous random variable can take any value within a +finite or infinite interval of the real number line (-\(\infty\),\(\infty\))*
+Cumulative Distribution Function
+The cumulative distribution function (CDF) of a random +variable X is the function \[ +F(x)=P(X\leq x) +\]
+for \(x \in [-\infty,\infty]\) (in +real number line).
+Properties of the CDF:
+-
+
It is non-decreasing: If \(a\leq +b\), then \(F(a)\leq F(b)\), +because \([X\leq a]\subseteq[X\leq +b]\).
+\(F(-\infty)=0\) and $F()=1 +$.
+If \(a< b\), then \(P(a<X\leq b)=F(b)-F(a)\).
+
Example: Toss a fair coin 3 times and let X be the number of +heads:
+Outcomes: {HHH,HHT,…TTT} |S|=8,
+The PMF for x
+x | 0 1 2 3
+P(x)|(1/8)(3/8)(3/8)(1/8)
+The CDF:
+For \(x<0, F(x)=P(X\leq +x)=0\);
+For \(0\leq x<1, F(x)=P(X\leq +x)=P(X=0)=1/8\);
+For \(1\leq x<2, F(x)=P(X\leq +x)=P(X=0)+P(X=1)=1/2\);
+For \(2\leq x<3, F(x)=P(X\leq +x)=P(X=0)+P(X=1)+P(X=2)=7/8\);
+For \(3\leq x, F(x)=P(X\leq +x)=P(X=0)+P(X=1)+P(X=2)+P(X=3)=1\);
+Properties of the CDF of a discreate random variable
+Let \(x_1<x_2<x_3...\) denote +the possible values of X. Then
+-
+
F is a step function with jumps occurring only +at the values of x of \(S_X\). The size +of the jump at each x of \(S_X\) equals +p(x).
+The CDF can be obtained from the PMF:
+
\[ +F(x)=\sum_{x_i\leq x}p(x_i) +\]
+-
+
- The PMF can be obtained from the CDF: +
\[ +p(x_i)=F(x_i)-F(x_i-1) for i=2,3,... +\]
+-
+
- The probability of $a<Xb $ is given as +
\[ +P(a < X \leq b)=F(b)-F(a)=\sum_{a<x_i<b}p(x_i) +\]
+Example: Suppose the CDF of a random variable X is given by \[ +F(x)=\begin{cases}0,x<-1\\1/3,-1\leq x<1\\1/2,1\leq +x<2\\1,2\leq x \end{cases} +\]
+The places for jump (-1,1,2) is the change of P(x)
+Density Function for a Continuous Random variable
+-
+
For a continuous random variable P(X=x)=0 for all x.
+The CDF is a continuous function.
+
Example: We say X is the uniform in [0,1] random variable if X has +CDF
+\[ +F(x)=\begin{cases}0,x<0\\x,0\leq x<1\\1,1\leq x\end{cases} +\]
+The probability density function (PDF of a continuous random +variable X is a non-negative function f such that
+$P(a<X<b) $ = area under f between a and b = \(\int_a^b{f(x)dx}\)
+You can also write it with equal sign because +P(X=a)=P(X=b)=0
+Let X be a continuous random variable with PDF f(x) and CDF F(x). +Then
+-
+
\(\int_{-\infty}^{\infty}{f(x)dx}=1\)
+The CDF can be obtained form the PDF:
+
\[ +F(x)=P(X\leq x)=\int^x_{-\infty}f(y)dy +\]
+-
+
- The PDF can be obtained from the CDF: +
\[ +f(x)={d\over{dx}}F(x) +\]
+Lecture_11
+Zheyuan Wu
+2023-02-13
+ +Expected value
+For a discrete random variable x with sample space \(S_X\) and PMF \(p(x)\). The expected value is
+\[ +E(X)=\mu X=\sum_{x\in S_X}xp(x) +\]
+Example: Let X be the random variable with PMF
+| x | +-1 | +1 | +2 | +
|---|---|---|---|
| p(x) | +1/3 | +1/6 | +1/2 | +
Lecture_12
+Zheyuan Wu
+2023-02-15
+ +Variance and Standard Deviation
+-
+
- The variance \(\sigma_X^2\),or +Var(x), of a random variable X is +
\[ +\begin{aligned} +\sigma_X^2&=E[(X-\mu_X)^2]\\ +&=E[x^2-2\mu X+\mu_X^2]\\ +&=E(X^2)-2\mu_XE(X)+\mu_X^2\\ +&=E(X^2)-2\mu_X^2+\mu_X^2\\ +&=E(X^2)-[E(X)^2] +\end{aligned} +\]
+square distance between the data and expected +value/center
+where \(\mu_x=E(X)\) is the expected +value of X.
+An easier formula to use is
+\[ +\sigma_X^2=E(X^2)-[E(X)]^2 +\]
+-
+
- The standard deviation of X is the positive square root of the +variance: +
\[ +\sigma_X=\sqrt{\sigma_X^2} +\]
+Example: Let X be the random variable with PMF
+| x | +-1 | +1 | +2 | +
|---|---|---|---|
| p(x) | +1/3 | +1/6 | +1/2 | +
\[ +E(X)=-1\times{1\over 3}+1\times{1\over 6}...={5\over 6} +\]
+Method 1:
+\[ +\begin{aligned} +Var(X)=\sigma_X^2&=E[(X-E(x))^2]\\ +&=(-1-{5\over 6})^2\times {1\over 3}+(1-{5\over 6})^2\times {1\over +6}+(2-{5\over 6})^2\times {1\over 2}\\ +&=1.81 +\end{aligned} +\] Method 2:
+\[ +\begin{aligned} +\sigma_X^2&=E(X^2)-[E(X)]^2\\ +&=(-1)^2*{1\over 3}+(1)^2*{1\over 6}+(2)^2*{1\over 2}-({5\over +6})^2\\ +&=2.5-({5\over 6})^2\\ +&=1.81 +\end{aligned} +\]
+For continuous variable:
+Let \(X\sim U(-3,2)\). Find +Var(X).
+\[ +E(X)=-{1\over 2}(middle~point)\\ +f(x)={1\over 5},x\in[-3,2]\\ +\]
+\[ +\begin{aligned} +E(X^2)&=\int_{-\infty}^{\infty}x^2f(x)dx\\ +&=\int_{-3}^{2}x^2{1\over 5}dx\\ +&={1\over 15}(2^3-(-3)^2)\\ +&={7\over3} +\end{aligned} +\]
+\[ +Var(X)={7\over 3}-(-{1\over 2})^2=2.08 +\]
+Method 2:
+\[ +Var(X)=\int_{-\infty}^{\infty}(x-\mu_X)^2f(x)dx=... +\]
+Transform of random variable
+If the variance of X is \(\sigma_X^2\) and \(Y=a+bX\), then
+\[ +\sigma_Y^2=b^2\sigma_X^2 +\]
+Population Percentiles
+-
+
- For continuous random variables. +
Let X be a continuous random variable with CDF F and \(\alpha\) a number between 0 and 1. the +100(1-\(\alpha\))-th percentile of X is +\(x_\alpha\) such that
+\[ +F(x_\alpha)=P(X\le x_\alpha)=1-\alpha +\]
+alpha as the upper area of the probability, eg. \(x_0.05\) is the area of top 0.05, as 95th +percentile
+Example: Let X be a continuous random variable with PDF
+\[ +f(x)\begin{cases}e^{-x},~~~x>0\\0,~~~ elsewhere\end{cases} +\]
+Find the IQR.
+\[ +F(Q_1)=0.25=0.288\\ +1-e^x=0.25\\ +F(Q_3)=0.75=1.386\\ +1-e^x=0.75\\ +IQR=Q_3-Q_1=1.098 +\]
+Moment generating function
+\[ +M_x(t)=E(e^{tx}),~~~t\in(-h,h)for~some~h>0 +\]
+\[ +M_x(0)=1\\ +{dM_x(t)\over dt}=M'(0)=E({d\over dx}e_{tx})=E(xe^{tx})=E(x)\\ +M^k(0)=E(x^k) +\]
+Models for Discrete random variables
+Bernoulli Distribution
+-
+
A Bernoulli trial or experiment is one whose outcome is +either success or a faliure.
+A Bernoulli random variable X takes the value of 1 if +the outcome is a success and 0 if the outcome is a failure.
+The probability of success is p.
+Notation: \(X\sim +Bern(p).\)
+
The PMF:
+|x|0|1| |p(x)|1-p|p|
+Lecture_13
+Zheyuan Wu
+2023-02-17
+ +Binomial Distribution
+-
+
A binomial experiment is when n Bernoulli experiments, +each having prbability of success p, are performed +independently.
+The binomial random variable Y is the number of success +in the n Bernoulli trials.
+The parameters of a binomial random variable are n and p
+Notation:\(Y\sim +Bin(n,p)\).
+
The PMF:
+\[ +p(y)=P(Y-y)=\begin{pmatrix}n\\y\end{pmatrix}p^y(1-p^{n-y}),~~~y=0,1,2,3,4,...n +\]
+Example: Suppose 70% of all purchases in a certain store are made +with a credit card. Let Y denote the number of credit card used in the +next 10 purchases. What is \(P(5\leq Y +\leq8)\)
+\[ +Y\sim Bin(10,0.7) +\]
+p=0.7
+n=10
+result=0
+for (y in 5:8){
+# R will computer the PMF, P(Y=y):
+ result=result+dbinom(y,n,p)
+}
+result## [1] 0.8033427# R will also computer the CDF, P(y<=y):
+pbinom(8,n,p)-pbinom(4,n,p)## [1] 0.8033427The expected value and variance of \(Y\sim +Bin(n,p)\) are
+\[ +E(Y)=np~~~~~~~\sigma_Y^2=np(1-p) +\]
+Example: what is the expected number of credit card purchases? What +is the variance?
+\[ +E(Y)=n\times p=10\times 0.7=7\\ +Var(Y)=10\times 0.7\times(1-0.7)=2.1 +\]
+Geometric Distribution
+Construct based on Bernoulli trails
+-
+
In a geometric experiment, independent Bernoulli trails, +each with probability of success p, are performed until the first +success occurs.
+The geometric random variable X is the number of trails +up to and including the first success.
+Notation: \(X\sim +Geo(p)\)
+
The PMF:
+\[ +p(x)=P(X=x)=(1-p)^{x-1}p,~~~~~x=1,2,3.... +\]
+The \((1-p)^{x-1}\) is the +probability of failure in x-1 trails.
+The CDF: \[ +F(x)=P(X\le x)=1-(1-p)^x~~~~~x=1,2,3,... +\]
+The expected value and variance of \(X\sim +Geo(p)\) are
+\[ +E(X)={1\over p}~~~~~~~\sigma_X^2={1-p \over p^2} +\]
+Example: Suppose you need to find a store that carries a special +printer ink. You know that of the stores that carry printer ink, 15% of +them carry the special ink. You randomly call each store until one has +the ink you need.
+-
+
- What is the probability that you first find the special ink at the +third store you call? +
Let x denote the number of calls until you find the ink.
+\[ +X\sim Geo(0.15)\\ +P(X=3)=(1-0.15)^20.15=0.108 +\]
+-
+
- What is the probability you first find the special ink in 3 calls or +less? +
\[ +P(X\le 3)=1-(1-0.15)^3=0.386 +\]
+-
+
- What is the expected number of calls until you first find the +special ink? What is the variance? +
\[ +E(X)={1\over0.15}=6.67\\ +\sigma_x^2={1-0.15\over 0.15^2}=37.78 +\]
+Negative Binomial Distribution
+-
+
In a negative binomial experiment, independent Bernoulli +trials, each with probability of success p, are performed until the rth +success occurs.
+The negative binomial random variable Y is the total +number of trials up to and including the rth success.
+Notation: \(Y\sim +NB(r,p)\)
+The sample space of Y is \(S_Y=\{r,r+1,r+2...\}\)
+
The PMF:
+\[ +p(y)=P(Y=y)=\begin{pmatrix}y-1\\r-1\end{pmatrix}p^r(1-p)^{y-r} +\]
+Example: An oil company conducts a geological study that indicates +that an exploratory oil well should have a 20% chance of striking +oil.
+-
+
- What is the probability the first strike comes on the third well +drilled? +
Let X denote the number of wells until first strike
+\[ +X\sim Geo(0.2) +P(X=3)=(1-0.2)^20.2=0.128 +\]
+-
+
- What is the probability that the third strike comes on the seventh +well drilled? +
Let Y denote the number of wells until third strike.
+\[ +Y\sim NB(3,0.2)\\ +P(Y=7)=\begin{pmatrix}6\\2\end{pmatrix}0.2^2(1-0.2)^{7-3}0.2=0.049 +\]
+y=7
+r=3
+# y-r is the number of failure to got the xth success
+p=0.2
+dnbinom(y-r,r,p)## [1] 0.049152The expected value and variance of \(Y\sim +NB(r,p)\) are
+\[ +E(Y)={r\over p}~~~~~~~\sigma_Y^2={r(1-p)\over p^2} +\]
+-
+
- What is the expected number of wells to be drilled until striking +oil for the third time? +
\[ +\mu_Y=E(Y)={3\over 0.2}=15 +\]
+Hypergeometrix distribution
+An extension of geometric distribution, \(Geo(p)=NB(1,p)\)
+-
+
Suppose a population consists of \(M_1\) (number of failures) objects labeled +1 and \(M_2\) (number of success) +objects labeled 0, and that a sample of size n is selected at random +without replacement.
+The hypergeometric random variable X is the number of +objects labeled 1 in the sample.
+Notation: \(X\sim +Hyp(M_1,M_2,n)\)
+
The PMF:
+\[ +p(x)=P(X=x)={\begin{pmatrix}M_1\\x\end{pmatrix}\begin{pmatrix}M_2\\n-x\end{pmatrix}\over\begin{pmatrix}M_1+M_2\\n\end{pmatrix}} +\]
+\(\begin{pmatrix}M_1\\x\end{pmatrix}\) is the +number of ways getting x failures from \(M_1\) failures in the population
+\(\begin{pmatrix}M_2\\n-x\end{pmatrix}\) is +the number of ways getting n-x successes from \(M_2\) successes in the population
+\(\begin{pmatrix}M_1+M_2\\n\end{pmatrix}\) is +the number of ways getting n items from \(M_1+M_2\) items in the population
+The sample space of X:
+\[ +S_X=\{max(0,n-M_2),....min(n-M_1)\} +\]
+Example: A crate contains 50 light bulbs of which 5 are defective and +45 are not. A quality control inspector randomly samples 4 bulbs without +replace- ment. Let X be the number of defective bulbs in the sample.
+\[ +X\sim HyperG(M_1,M_2,n),M_1=5,M_2=45,n=4\\ +S_X=\{0,1,2,3,4\},0=max(0,4-45),4=min(4,5) +\]
+Find the probability that less than 3 bulbs are defective.
+\[ +P(X=x)={\begin{pmatrix}5\\x\end{pmatrix}\begin{pmatrix}45\\4-x\end{pmatrix}\over\begin{pmatrix}50\\4\end{pmatrix}},~~~x=0,1,2,3,4\\ +P(X<2)=P(X=0)+P(X=1)=P(X\leq 1)\\ +\]
+dhyper(0,5,45,4)+dhyper(1,5,45,4)## [1] 0.9550369phyper(1,5,45,4)## [1] 0.9550369The expected value and variance of \(X\sim +Hyp(M_1,M_2<n)\) are
+\[ +E(X)={nM_1\over N}~~~~~~~\sigma_X^2={nM_1\over N}(i-{M_1\over +N})({N-n\over N-1}) +\]
+Where \(N=M_1+M_2\)
+Lecture_14
+Zheyuan Wu
+2023-02-20
+ +++recap from last week.
+
sampling with replacement
+independent Bernoulli trails
+\(X\sim Ber(p)\), \(E(X)=P\), \(Var(X)=p(1-p)\), \(P(X=1)=p\)
+\(Y\sim Bin(n,p)\), \(P(X=k)=\begin{pmatrix}n\\k\end{pmatrix}p^k(1-p)^{1-k},k=0,1,2,3,...n\), +\(E(X)=np\), \(Var(X)=np(1-p)\)
+\(X\sim Geometric(p)\), \(P(X=k)=(1-p)^{k-1}p\), \(k=1,2,3,..\infty\), \(E(X)={1\over p}\), \(Var(X)={1-p\over p^2}\)
+\(Y\sim NB(r,p)\) Negative binomial +distribution, \(P(X=k)={\begin{pmatrix}k-1\\r-1\end{pmatrix}}p^{r-1}(1-p)^{k-r},k=r,r+1,....\infty\), +\(E(X)={r\over p}\), \(Var(X)={r(1-p)\over p^2}\)
+sampling without replacement
+p changed, event dependent on each other
+\(X\sim HyperG (M_1,M_2,n)\), \(P(X=k)={\begin{pmatrix}M_1\\k\end{pmatrix}\begin{pmatrix}M_2\\n-k\end{pmatrix}\over\begin{pmatrix}M_1+M_2\\n\end{pmatrix} +},k=min(0,n-M_2),1,2,3,...max(n,M_1)\), \(E(X)={nM_1\over N}\), \(Var(X)={nM_1\over N}(1-{M_1\over N})({N-n\over +N-1})\), n is the selected items, N-n is the number of +non-selected items.
+The hypergeometric distribution can sometimes be approximated using +the binomial distribution.
+For large population size N, the difference between sampling with and +without replacement is very small.
+-
+
Suppose $X \sim Hyp(M_1,M_2,n)$ and n $N \le 0.05$ where $N = M_1 + M_2$. Then $P(X = x) \simeq P(Y = x)$ where $Y ∼Bin(n,p = M_1 N )$.
+
Example: Suppose M1 = 100, M2 = 900, and n = 25. Find P(X = 3).
+#Using the hypergeometric distribution:
+dhyper(3,100,900,25)## [1] 0.229574#Using the binomial approximation:
+dbinom(3,25,0.1)## [1] 0.2264973Possion Distribution
+-
+
The Poisson distribution is used to model the probability that a +number of events occur in an interval of time or space.
+The Poisson random variable X denotes the number of +events that occurred.
+Notation \(X\sim Poisson +(\lambda)\)
+
The PMF:
+\[ +p(x)=P(X=x)={e^{-\lambda}\lambda^x\over +x!},~~~~~~~x=0,1,2,3...\lambda>0 +\]
+The mean and variance for \(X\sim +Poisson(\lambda)\) are
+\[ +E(X)=\lambda~~~~~~~\sigma_X^2=\lambda +\]
+-
+
R will compute the PMF, P(X = x): dpois(x,λ)
+R will also compute the CDF, P(X ≤x): ppois(x,λ)
+R will provide a sample of n Poisson random variables: +rpois(n,λ)
+
Example: Suppose that a person taking Vitamin C supplements contracts +an average of three colds per year, and that this average increases to +five colds per year for persons not taking Vitamin C supplements. +Suppose further that the number of colds a person contracts in a year is +a Poisson random variable.
+-
+
- Find the probability of no more than two colds for a person taking +sup- plements and a person not taking supplements. +
\(X_1\)= number of colds for patient +taking VC, \(X_1\sim Poisson(3)\)
+\(X_2\)= number of colds for patient +not taking VC, \(X_2\sim +Poisson(5)\)
+\(P(X_1\leq)=P(X_1=0)+P(X_1=1)+P(X=2)=\)
+## first is x, second is lambda
+ppois(2,3)## [1] 0.4231901\(P(X_2\le 2)=0.125\)
+ppois(2,5)## [1] 0.124652-
+
- Suppose 70% of the population takes Vitamin C supplements. Find the +probability that a randomly selected person will have no more than two +colds in a given year. +
\(A\) = taking VC, \(A^c\)= not taking VC
+X = number of codes of a randomly selected person
+\[ +\begin{aligned} +P(X\le 2)&=P(x\le2,A)+P(X\le2,A^c)\\ +&=P(X\le 2|A)P(A)+P(X\le 2|A^c)P(A^c)\\ +&=0.423\times 0.7+0.125\times 0.3\\ +&=0.334 +\end{aligned} +\]
+Exponential distribution
+-
+
The exponential distribution is often used to model +lifetimes of equipment or waiting times until events occur.
+Notation \(X\sim +Exp(\lambda)\)
+
PDF:
+\[ +f(x)=\begin{cases}\lambda e^{-\lambda +x},~~~x\geq0\\0,~~~~~~~~~~~x<0\end{cases} +\]
+The mean and variance for \(X\sim +Poisson(\lambda)\) are
+\[ +E(X)=\lambda~~~~~~~\sigma^2_X=\lambda +\]
+CDF:
+For \(0\leq X\)
+\[ +F(X)=P(X\le x)\int^X_{-\infty}f(x)dx=\int^x_0 \lambda e^{-\lambda t}dt\\ +let~u=\lambda t +=\int^{\lambda x}_0 e^{-u}du\\ +=-e^{\lambda x}-1-e^0\\ +=1-e^{-\lambda x} +\]
+the expected value and variance of \(X\sim +Exp(\lambda)\) are
+\[ +E(X)={1\over \lambda}~~~~~~~\sigma_X^2={1\over \lambda^2} +\]
+Example: Let X be the amount of time (in minutes) a postal clerk +spends with his or her customer. The time spent has an exponential +distribution with λ = 0.25.
+-
+
- Find the expected time spent with each customer. +
\[ +E(X)={1\over 0.25}=4 +\]
+-
+
- What is the probability the clerk spends between 2 and 4 minutes +with a customer? +
\[ +P(2\le X<4)=\int^4_2 \lambda e^{-\lambda x}dx=\int_2^4 +0.25e^{-0.25x}=0.239\\ +P(2\le X<4)=F(4)-F(2)=(1-e^{-0.24\times 4})=0.239 +\]
+An exponential random variable X has the memory-less property:
+\[ +P(X > s + t |X > s) = P(X > t) +\]
+Lecture_15
+Zheyuan Wu
+2023-02-22
+ +Normal Distribution
+From Binomial Distribution
+\(X \sim Bin(n,p)\)
+-
+
Consider a fixed number of independent Bernoulli trials with the +same probability of success.
+Binomial r.v.: Let X =the number of successes in n independent +Bernoulli trials, each of which has prob. p to success.
+
Eg: the number heads when flip a coin twice, the number of boys among +100 newborns ….
+-
+
- PDF:\(f(x)=P(X=x)=\begin{pmatrix}n\\x\end{pmatrix}p^xq^{n-x}\) +for \(x=0,1,2,3...n\) +
…. CDF:
+Normal Distribution
+Motivating Example of Normal Distribution
+-
+
The number of success in about Binomial distributions heaviliy +depends on the number of trials, n.
+Instead, if we consider the proportion of success,
+
\[ +Y=X/n +\]
+its expected value is \(E(Y)=p\) no +mater what n is.
+And \(Var(Y)=p(1-p)/n\),$ Var (Y)$ +decreases as n increases.
+From Discrete to Continuous Distribution
+-
+
The discrete jumps in the cdf become continuous increments. So +cdf is smooth for continuous random variable.
+From discrete random variable, the jump size is the pmf, For +continuous version,, we can take the derivative of cdf to define +pdf.
+For discrete r.v., the sum of jump size is 1. For continuous +r.v., we have \(\int^{\infty}_{-\infty}f(x)dx=1\)
+
For continuous random variable:
+-
+
\(f(x)\) is NOT the jump size +but the rate of increase, \(f(x)=lim_{\epsilon->0}{P(x-\epsilon<X<x+\epsilon)\over +2\epsilon}\), which is \({dF(x)\over +dx}\).
+\(f(x)\neq P(X=x)\)
+\(f(x)\) may not be smaller than +1.
+\(P(a<X<b)=\int^b_af(x)dx\), the +probability is the area under pdf curve.
+\(P(X=x)=0\) for any given +x.
+
Normal Distribution
+\[ +X\sim N(\mu,\sigma^2) +\]
+-
+
- Normal random variable: a continuous random variable with pdf +
\[ +f(x)={1\over\sqrt{2\pi\sigma^2}}e^{-{1\over2\sigma^2}(x-\mu)^2},~~~x\in +R, +\] where \(\mu\in R\) and \(\sigma^2>0\)
+-
+
- CDF: +
\[ +F(x)=\int^x_{-\infty}f(t)dt,~~~x\in R +\]
+-
+
It can be shown that \(E(X)=\mu\) and \(Var(X)=\sigma^2\).
+A normal density (pdf) is fully specified by only two parameters: +\(\mu,\sigma^2\)
+Every normal distribution is symmetric about its +mean, μ.
+Changing μ: shifts the “bell-shaped” curve left and +right
+Changing σ: shrinks/stretches the curve
+There are an infinite number of normal distributions (just change +the mean and variance/standard deviation).
+
Standard Normal:
+\(\mu=0,\sigma^2=1\). denoted by +\(Z\sim(0,1)\)
+The standard normal distribution has pdf:
+\[ +f(x)={1\over\sqrt2\pi}e^{-{1\over2}x^2},~~~-\infty<x<\infty +\]
+-
+
We often use \(\phi\)(·) and +\(\Phi\)(·) to denote the pdf and cdf +of the standard normal distribution.
+If X ∼N(μ,σ2), then \({X-\mu\over\sigma} \sim N(0,1)\), which is +often denoted by Z.
+cdf of Normal is in-explicit and can be evaluated using tables +(referred to as Normal table or Z table) or computer. R: pnorm(x, mean, +sd)
+
Calculation
+Classic values:
+\(Z\sim N(0,1),P(Z<1)\)
+pnorm(1,0,1)## [1] 0.8413447\(Z\sim N(0,1),P(-1<Z<1)\)
+pnorm(1,0,1)-pnorm(-1,0,1)## [1] 0.6826895\(Z\sim N(0,1),P(-2<Z<2)\)
+pnorm(2,0,1)-pnorm(-2,0,1)## [1] 0.9544997\(Z\sim N(0,1),P(-3<Z<3)\)
+pnorm(3,0,1)-pnorm(-3,0,1)## [1] 0.9973002-
+
Obtain the cut-off values (or range) of the random variable to +satisfy certain probability
+Ex: Find the z-value such that P(Z < z) = 0.95
+
qnorm(0.95,0,1)## [1] 1.644854-
+
Ex: Find the interval [−z,z] s.t. P(−z < Z < z) = 0.90 +solve z s.t. P(Z > z) = 0.05: z = 1.645
+0.95? (z=1.960) 0.99? (z=2.576)
+Ex: Let \(X \sim N(0.5,0.052)\). +Find the interval [a,b] such that P(a < X < b)=0.95.
+
b=qnorm(0.975,0.5,0.05)
+a=qnorm(0.025,0.5,0.05)
+b## [1] 0.5979982a## [1] 0.4020018Properties of Normal
+-
+
\({X-\mu\over\sigma}\sim +N(0,1)\)
+\(a+bX\sim +N(a+b\mu,b^2\sigma^2)\)
+Let \(x_\alpha\) denote the +\((1 −\alpha)-100\)th percentile of X, +and let \(z_\alpha\) denote the \((1 −\alpha)-100\)th percentile of Z. +Then
+
\[ +x_\alpha = \mu + \sigma z_\alpha +\]
+Example: Tire Tread Thickness
+A machine manufactures tires with an initial tread thickness that is +normally distributed with mean 10 mm and standard deviation 2 mm. The +tire has a 50,000-mile warranty. In order to last for 50,000 miles the +tread thickness must initially be at least 7.9 mm. If the initial +thickness of tread is measured to be less than 7.9 mm, then the tire is +sold as an alternative brand with a warranty of less than 50,000 +miles.
+-
+
- Find the proportion of tires sold under the alternative brand. +
Let X be the Tread Thickness
+\[ +P(X<7.9)= +\]
+pnorm(7.9,10,2)## [1] 0.1468591-
+
- The demand for the alternative brand of tires is such that 30% of +the total output should be sold under the alternative brand name. What +should the critical thickness, originally 7.9 mm, be set at in order to +meet the demand? +
qnorm(0.3,10,2)## [1] 8.951199Checking Normality: QQ plot
+QQ plot is a plot to compare the empirical quantiles against +theoretical quantiles. It is often used to check the distribution +assumption.
+The most commonly used QQ plot is QQ plot for checking normality, +which is also referred to as Normal plot or Normal probability plot.
+-
+
Standardize data to find z-scores. \(z_i\)
+Rank the z-scores from the smallest to the largest. Use the rank +to calculate the empirical quantile for each point. \(k/(n + 1) or (k −0.375)/(n + 1 +−0.75)\)
+Find the z-value from Normal Table corresponding to the empirical +quantile of each point. \(z_i = \Phi−1(k/(n + +1))\).
+Plot the z-scores in the first step against these z-values. +(x-axis: ̃zi, y-axis: $$z_i)
+
One may replace Φ in step 3 by the cdf of any specified distribution, +to check the distribution assumption.
+qqnorm(x)
+qqline(x)Lecture_16
+Zheyuan Wu
+2023-02-24
+ +Joint Probability Distributions
+-
+
Often we are not interested in the behavior of an individual +random variable.
+Instead, we may be interested in the relationship between two +components of a bivariate random variable.
+
Example: X = number of hours study per week
+Joint and Marginal PMF
+The joint probability mass function (or joint PMF) of the +jointly discrete random variable X and Y is
+\[ +p(x,y)=P(X=x,Y=y) +\]
+-
+
- If \(S=\{(x_1,y_1),(x_2,y_2)....\}\) is the +sample space of (X,Y), then +
\[ +p(x_i,y_i)\geq0~for~all~i\\\ +\sum_{(x_i,y_i)\in S}p(x_i,y_i)=1\\ +p(x<X\leq b,c<Y\le d)=\sum_{i:a,x_i\le b,c<y_i\leq d}p(x_i,y_i) +\]
+-
+
- The distributions of the individual random variables are called +marginal distributions. The marginal PMFs can be found using the joint +PMF. +
\[ +p_X(x)=\sum_{y\in S_Y}p(x,y)\\ +p_Y(y)=\sum_{x\in S_X}p(x,y) +\]
+Example: Measurements for the length and width of plastic covers for +CDs are rounded to the nearest mm.
+X = length of a randomly selected CD cover
+Y = width of a randomly selected CD cover
+The possible values of X are 129, 130, and 131 mm. The possible +values of Y are 120 and 121 mm. The joint PMF for X and Y is
+ y
+ 120 121x 129 0.12 0.08 130 0.42 0.28 131 0.06 0.04
+-
+
- Verify that this is a valid joint PMF +
The sum of probability is 1, or each item is greater than 0.
+-
+
- Find \(P(129.5 \le X \le 131.5, Y \geq +121)\). +
\[ +=P(X=130,Y=121)+P(X=131,Y=121)=0.28+0.06=0.32 +\]
+-
+
- Find \(P(129.5\le X\le131.5)\) +
\[ +=P(X=130)+P(X=131)=0.42+0.28+0.06+0.04=0.8 +\]
+-
+
- Find the marginal PMF of Y. +
| y | +120 | +121 | +
|---|---|---|
| p(y) | +0.6 | +0.4 | +
Lecture_17
+Zheyuan Wu
+2023-02-27
+ +Conditional Distributions
+Recall: \(P(B | A)\) is the +probability that event B occurs conditional on knowing that event A has +occurred.
+\[ +P(B|A)={P(B\cap A)\over P(A)} +\]
+A & B are independent
+if \(P(B\cap A)=P(B)\times +P(A)\)
+We can extend the concept of conditional probability to PMFs and +PDFs.
+Conditional PMFs
+For jointly discrete random variables X and Y , the conditional PMF +of Y given X = x (given a fixed value of X) is
+\[ +p_{Y|X=x}(Y)=P(Y=y|X=x)={p(x,y)\over p_X(x)} +\]
+The conditional PMF is a proper PMF:
+-
+
\(pY|X=x(y)=1\)
+\(\sum_y +p_{Y|X=x}(y)=1\)
+We can obtain conditional means and variances.
+
Example: CD example with joint PMF
+| x\ y | +120 | +121 | +\(PX(x)\) | +
|---|---|---|---|
| 129 | +0.12 | +0.08 | +0.2 | +
| 130 | +0.42 | +0.28 | +0.7 | +
| 131 | +0.06 | +0.04 | +0.1 | +
-
+
- Find the conditional PMF of Y given X = 130. +
Find the marginal distribution first
+| y | +120 | +121 | +
|---|---|---|
| \(p_{Y|X=130}(y)\) | +0.42/0.4 | +0.28/0.7 | +
| + | 0.6 | +0.4 | +
\[ +P_{Y|X=130}(120)={P(X=130,Y=120)\over P(X=130)}=0.6 +\]
+-
+
- Verify that the conditional PMF is valid. +
They are valid because they add up to one and none of them are +negative.
+-
+
- Find the conditional expected value of Y given X = 130. +
\[ +E(Y|X=130)=\sum y\times p_{Y|X=130}(y)=120\times 0.6+121\times 0.4=120.4 +\]
+-
+
- Find the conditional variance of Y given X = 130. +
\[ +Var(Y|X=130)=E(Y^2|X=130)-[E(Y|X=130)]^2=0.24\\ +E(Y^2|X=130)=\sum y^2\times p_{Y|X=130}(y)=120^2\times 0.6+121^2\times +0.4 +\]
+Note: We can compute the marginal PMF using the conditional PMF.
+\[ +p_Y (y) =\sum_x P_{Y|X=x}(y)P_X(x) +\]
+Conditional PDFs
+For jointly continuous random variables X and Y , the conditional PDF +of Y given X = x is
+\[ +f_{Y |X=x}(y) = {f(x, y)\over f_X(x)} +\]
+The conditional PDF is a proper PDF:
+-
+
\(\int_{-\infty}^{\infty} f_{Y |X=x}(y) +dy = 1\)
+\(P(a < Y < b | X = x) = +\int_{a}^{b} f_{Y |X=x}(y) dy\), which is the area under the +conditional PDF curve, \(f_{Y|X=x}(y)\) +from a to b.
+We can obtain conditional means and variances.
+
Example: Let (X, Y ) be jointly continuous random variables with +joint PDF
+\[ +f(x,y)=\begin{cases}{12\over 7}(x^2+xy),~~~0\leq x\leq1,0\leq +y\leq1\\0,~~~~~~elsewhere\end{cases} +\]
+-
+
- Find \(f_{Y|X=0.5}(y)\). +
\[ +P_X(x)={12\over 7}(x^2+{1\over 2}x) +\]
+\[ +f_{Y|X=0.5}(y)={f(x,y)\over f_X(x)}={{12\over 7}(0.5^2+0.5y)\over +{12\over 7}(0.5^2+{1\over 2}0.5)}={0.25+0.5y\over 0.25+0.25}={1\over +2}+y~~~~~ +0\leq y\leq1 +\]
+-
+
- Find \(P(0<Y<0.5|X=0.5)\). +
You can also write the equation in this way
+\[ +F_{Y|X=0.5}(0.5)-F_{Y|X=0.5}(0) +\]
+\[ +\int_0^{0.5}f_{Y|X=0.5}(y)dy=\int_0^{0.5}{1\over 2}+y~dy=0.375 +\]
+-
+
- Find \(E(Y|X=0.5)\). +
\[ +\int^{\infty}_{-\infty}y\times f_{Y|X=0.5}(y)dy=\int ^1_0y\times +({1\over 2}+y)~dy=0.583 +\]
+Note: We an computer the marginal PDF using the conditional PDF:
+\[ +f_Y(y)=\int^{\infty}_{-\infty}f_{Y|X=x}(y)f_X(x)~dx +\]
+weighted average of \(f_{Y|X=x}(y)\)
+Independent Random variables
+Recall:
+-
+
The joint PMF for two jointly discrete random variables X and Y +is \(p(x, y)\).
+The joint PDF for two jointly continuous random variables X and Y +is \(f(x, y)\).
+
Definition: Two random variables X and Y are independent +if
+\[ +P(X\in A, Y\in B)=P(X\in A)P(Y\in B) +\] For any set of A and B..
+Result
+-
+
- If X and Y are discrete, then X and Y are independent if and only if +for all x and y +
\[ +p(x, y) = p_X(x)p_Y (y) +\]
+-
+
- If X and Y are continuous, then X and Y are independent if and only +if for all x and y +
\[ +f(x, y) = f_X(x)f_Y (y) +\]
+Example: Let X and Y be discrete random variables with the joint PDF +below. Are X and Y independent?
+| x | +1 | +2 | +3 | +
|---|---|---|---|
| 0 | +0.10 | +0.20 | +0.15 | +
| 1 | +0.15 | +0.15 | +0.25 | +
Lecture_18
+Zheyuan Wu
+2023-03-01
+ +Example: For a cylinder selected at random from a production line, +let X be the cylinder’s height and Y the cylinder’s radius. Suppose X +and Y have the joint PDF below. Are X and Y independent?
+\[ +f(x, y) = \begin{cases}{3\over 8}{x\over y^2}, 1 \le x \le 3,{1\over +2}\le y{3\over 4}\\0, elsewhere\end {cases} +\]
+Note: Continuous random variables X and Y will be independent if and +only if we can write the joint PDF as
+\[ +\newcommand{\indep}{\perp \!\!\! \perp} +f(x, y) = g(x)h(y).~~~g(x)\indep h(y) +\]
+The support of x and y shall also be independent
+Example:
+\[ +f(x,y)={3\over 8}{x\over y^2} +\]
+If the support of x and y are independent, they are independent from +each other with constant \(c={3\over +8}\).
+Theorem: If X and Y are jointly discrete, then X and Y are +independent if and only if
+-
+
\(p_{Y |X=x}(y) = p_Y (y)\), +probability of Y does not depends on X=x
+\(p_{X|Y =y}(x) = p_X(x)\), +probability of X does not depends on Y=y
+
Note: The preceding theorem holds for jointly continuous random +variables with PDFs replacing PMFs.
+Theorem: Let X and Y be independent. Then
+-
+
\(E(Y | X = x) = E(Y )\) does +not depend on the value of x.
+\(g(X)\) and \(h(Y )\) are independent.
+\(E[g(X)h(Y )] = E[g(X)]E[h(Y +)]\), for \(E(XY)=E(X)E(Y)=\int \int +xyf(xy)dxdy=\int xf(x)dx\int yf(y)dy\)
+
Example: Find the expected volume of a randomly selected cylinder +form the production line.
+\(volume=\pi r^2 h=\pi y^2x\)
+Let X denote the height of cylinder, Y denotes the radius
+\[ +\newcommand{\indep}{\perp \!\!\! \perp} +X\indep Y\\ +E(\pi Y^2 X)=\pi E(Y^2)E(X)=\pi{26\over 12}{3\over 8}={13\over 16}\pi\\ +E(X)=\int^3_1x{1\over 4}xdx={26\over 12}\\ +E(Y^2)=\int^{3\over 4}_{1\over 2}y^2{3\over 2}{1\over y^2}dy={3\over 8} +\]
+Note: Independence extends to 3 or more random variables.
+-
+
- Jointly discrete random variables X1, . . . , Xn are independent if +and only if +
\[ +p(x1, . . . , xn) = p_{X1}(x1)· · · p_{Xn}(xn) +\]
+-
+
- Jointly continuous random variables X1, . . . , Xn are independent +if and only if +
\[ +f(x1, . . . , xn) = f_{X1}(x1)· · · f_{Xn}(xn) +\]
+If X1, . . . , Xn are independent and have the same distribution we +say they are independent and identically distributed, or iid.
+Expected Value of Functions of Random Variables
+We can use joint PMFs and PDFs to obtain expected values of functions +of random variables.
+-
+
- If X and Y are discrete +
\[ +E[h(X, Y )] = \sum_{x\in S_X}\sum_{y\in S_Y}h(x, y)p(x, y) +\]
+-
+
- If X and Y are continuous +
\[ +E[h(X, Y )] +=\int^{\infty}_{-\infty}\int^{\infty}_{-\infty}h(x,y)f(x,y)dx~dy +\]
+Example: Suppose X and Y have the joint PMF below. Find \(E(XY )\).
+| x\y | +-1 | +3 | +8 | +
|---|---|---|---|
| -1 | +0.1 | +0 | +0.4 | +
| 2 | +0.2 | +0.3 | +0 | +
The support of x depends on Y (When y=3, x cannot be -1). \(P(X=-i|Y=3)\)
+\[ +E(XY)=\sum_{x\in S_X}\sum_{y\in S_Y}xyp_{x,y}=(-1)\times (-1)\times +0.1...+2\times 8\times 0=-1.7 +\]
+Example: Suppose X and Y have the joint PDF below. Find E(XY ).
+\[ +f(x, y) = \begin{cases}{1\over 2}, 0 \leq x \leq 2, 0 \leq y \leq 1 +\\0, elsewhere\end{cases} +\]
+$$ XY\
+E(XY)=1_02_0xy{1}dx~dy={1} $$
+Lecture_19
+Zheyuan Wu
+2023-03-03
+ +A university uses an admissions index to rank applicants. The +admissions index is a numerical score calculated by multiplying the ACT +score by 10, the high school GPA by 100, and then adding the two. +Suppose that for this university the mean ACT score is 29 and the mean +high school GPA is 3.72. What is the expected value of the admissions +index?
+Let $X_1 $ denote the ACT score, \(X_2\) to denote GPA
+\[ +E(X_1)=29,E(X_2)=3.72\\ +I=10X_1+100X_2\\ +E(I)=10E(X_1)+100E(X_2)=10\times 29+100\times 3.72=662 +\]
+Result 3: (continuous on previous chapters):
+Suppose \(X_1,...X_n\) are iid +Bern(p), the \(E(\hat p)=p\)
+\[ +E(\hat p)=E({X_1+...+X_n\over n})\\ +E(X_1)=\mu=p +\]
+Covariance
+Question: Is \(Var(X+Y)=Var(X)+Var(Y)\) ?
+Not true in general. only true if X and Y are +uncorrelated/independent
+When random variables X and Y are dependent, computing Var(X + Y ) +involves the covariance.
+\[ +Cov(X,Y)=\sigma_{X,Y}=E[(X-\mu_X)(Y-\mu_Y)]=E(XY)=\mu_X\mu_Y\\ +Var(X)=E[(X-\mu_X)(X-\mu_X)]\\ +Cov(X,X)=Var(X) +\]
+-
+
- \(Cov(X,Y)>0\): +
Greater value of X, mainly correspond to greater values of Y.
+-
+
- \(Cov(X,Y)<0\): +
Greater value of X, mainly correspond to less values of Y.
+Properties:
+-
+
\(Cov(X, Y ) = Cov(Y, +X)\)
+\(Cov(X, X) = Var(X)\)
+If X and Y are independent, then \(Cov(X, Y ) = 0\).
+
\[ +Cov(X,Y)=E(XY) +-\mu_X\mu_y=E(X)E(Y)-\mu_X\mu_Y=0 +\]
+When \(Cov(X,Y)=0\), it doesn’t +means that the X, Y are independent…
+\[ +P(X=-1)=P(X=1)={1\over2}\\Y=X^2 +\]
+-
+
- $Cov(aX + b, cY + d) = acCov(X, Y) $ for any real numbers a, b, c, +and d. +
\[ +Cov(ax+b,cY+d)=E[(ax+b-aE(X)-b)(cY+d-cE(Y)-d)]\\ +=E[a(X-E(X))(cY-E(Y))]=acCov(X,Y) +\]
+Example: Suppose X and Y have the joint PMF below. Find Cov(X, Y +).
+| x\y | +-1 | +3 | +8 | +
|---|---|---|---|
| -1 | +0.1 | +0 | +0.4 | +
| 2 | +0.2 | +0.3 | +0 | +
\[ +E(X)=-1\times 0.5+2\times -0.5=0.5\\ +E(Y)=-1\times 0.3+3\times 0.3+8\times0.4=3.8\\ +E(XY)=-1.7\\ +Cov(X,Y)=E(XY)-E(X)E(Y)\\ +=-1.7-0.5\times 3.8=-3.6 +\]
+Variance of Sums of Random Variables
+Let \(\sigma_1^2\) and \(\sigma_2^2\) be the variances of X1 and X2, +respectively.
+-
+
- If X1 and X2 are independent, +
\[ +Var(X1 + X2) = \sigma^2_1 + \sigma^2_2\\ +Var(X1 − X2) = \sigma^2_1 + \sigma^2_2 +\]
+-
+
- If X1 and X2 are dependent, +
\[ +Var(X1 + X2) = \sigma^2_1 + \sigma^2_2 + 2Cov(X1, X2)\\ +Var(X1 - X2) = \sigma^2_1 + \sigma^2_2 - 2Cov(X1, X2) +\]
+-
+
- If \(X_1, . . . , X_n\) are random +variables with variances \(\sigma_1^2...,\sigma_n^2\) +
\[ +Var(a_1X_1+...+a_nX_n)=a^2_1\sigma_1^2+...+a^2_n\sigma^2_n+\sum_i\sum_{j\neq +i}a_ia_jCov(X_i,X_j) +\]
+for constants \(a_1, . . . +a_n\).
+If \(X_1,....X_n\) are independent, +\(Var(\sum +a_iX_i)=\sum^n_{i=1}a_i^2\sigma_i^2\)
+If \(X_1,....X_n\) are iid, \(Var(\sum +a_iX_i)=\sigma^2\sum^n_{i=1}a_i^2\), \(Var(\bar X)=\sigma^2{1\over n}\)
+Example: Suppose X and Y have the joint PMF below. Find Var(2X + Y +).
+|x\y-1 3 8 |-1 0.1 0 0.4 2 0.2 0.3 0
+\[ +Var(2X+Y)=Var(2X)+Var(Y)+2Cov(2X,Y)\\ +=4Var(X)+Var(Y)+4Cov(X,Y)\\ +=8.76 +\]
+Result 1: Let \(X_1,....X_n\) be iid +with common variance \(\sigma^2\). +then
+\[ +Var(\sum^n_{i=1}X_i)=n\sigma^2\\ +Var(\bar X)=Var({1\over n}\sum^n_{i=1}X_i)={\sigma^2\over n} +\]
+Result 2: Suppose \(X_1,....X_n\), +are iid Bern(p). Then
+\[ +Var(\hat p)={p(1-p)\over n} +\]
+Person’s Correlation Coefficient
+The correlation coefficient of X and Y is
+\[ +\row +\]
+Lecture 2
+Zheyuan Wu
+2023/1/20
+ +++assignment posted, due next week. based on today’s lecture. Looking +forward a little bit.
+
On example 5, P(+| Normal) = 5%, P(Symptom| All population) = 0.1%, +P(+| Symptom) = 99%, P(+ |All population) = 5.094%, P(Normal| +) = +98.15469%
+Simpson’s Paradox
+The basic distribution of population may result skew in general +probabilities.
+Popluation and Samples (Chapter I)
+Population: a collection of all objects/items/human/animals about +which information is sought.
+Sample: a subset of the population that is actually observed. +(Sampling refers to the process of selecting in a number of population +units and recording their characteristics.)
+Parameter: a numerical characteristic of the population for a +specified variables that we are interested to learn.
+Statistic: a numerical function of the sample data that is used to +estimate the unknown parameter of the population.
+Population – Sampling –> Sample Parameters <– Inference– +Statistics
+Example
+Interest: Is the content of lead in a lake within the safety limit? +Population: All locations in the lake Sample: 30 selected location for +examination Parameter: Average lead concentration in the lake Statistic: +Average lead concentration of our 30 sampled locations
+Why Sampling
+Can we collect all population units? - Yes, census, but costly, +time-consuming, rarely use.
+We can used the sample statistics to provide estimates of the +population parameters.
+Scope of statistc: - Point estimation; Confidence Intervals, Testing, +Prediction.
+Lecture_20
+Zheyuan Wu
+2023-03-06
+ +Example from previous week.
+Example: Let \(X_1, X_2\) be iid +\(N(0, 1)\) and let \(Y = 4X_1 + X_2\).
+-
+
- Find Var(Y ). +
\[ +\begin{aligned} +Var(Y)&=Var(4X_1+X_2)\\ +&=Var(4X_1)+Var(X_2)+2Cov(4X_1,x_2)\\ +&=4^2+Var(X_1)+Var(X_2)+0(from~iid)\\ +&=4^2\times 1 + 1+0\\ +&=17 +\end{aligned} +\]
+-
+
- Find \(Cov(X1, Y )\). +
\[ +\begin{aligned} +Cov(X_1,Y)&=Cov(X_1,4X_1+X_2)\\ +&=Cov(X_1,4X_1)+Cov(X_1,X_2)\\ +&=4Cov(X_1,X_1)+Cov(X_1,X_2)\\ +&=4Var(X_1)+0\\ +&=4+0\\ +&=4 +\end{aligned} +\]
+-
+
- Find \(Corr(X_1,Y)\). +
\[ +\sigma(X_1)=1,\sigma(Y)=\sqrt{Var(Y)}=\sqrt{17}\\ +Corr(X_1,Y)={Cov(X_1,Y)\over \sigma(X_1)\sigma(Y)}=0.97 +\]
+Note: Correlation and Covariance only measure linear dependence. +Corr(X,Y)=0 doesn’t imply that X and Y are independent.
+\(Corr(X,Y)\) is also written as +\(\rho(X,Y)\) pearson correlation
+Sample Covariance and Correlation
+If\((X_1,Y_1),...,(X_n,Y_n)\) is a +sample from the bivariate distribution of (X,Y),
+-
+
- the sample covariance is +
\[ +S_{X,Y}={1\over n-1}\sum_{i=1}^n(X_i-\bar X)(Y_i-\bar Y) +\]
+-
+
- the sample correlation coefficient is +
\[ +r_{X,Y}={S_{X,Y}\over S_X S_Y} +\]
+where \(\bar X\) and \(\bar Y\) are the sample means of X and Y +and \(S_X\) and \(S_Y\) are the sample standard deviation of +X and Y.
+-
+
In R, the sample covariance is given by cov(x,y).
+In R, the sample correlation coefficient is given by +cor(x,y).
+
Example: Consider the data set below.
+x | 10 11 18 16 21 y | 9 7 4 11 8
+x<-c(10,11,18,16,21)
+y<-c(9,7,4,11,8)
+cov (x,y)## [1] -2.45cor(x,y)## [1] -0.2031884Demo of Bivariate Normal Distribution
+Jimin Ding
+2023-03-04
+ +library(rgl) ## For interactive 3D plots
+library(mvtnorm) ## For multivariate normal distributions
+# ?dmvnorm ## check help for distribution of multivariate function
+# dmvnorm ## type the function name to see how the author define the function in details.
+library(ggplot2) ## For high-level R plots1. Joint Distribution of Bivariate Normal
+Let’s first draw the density for the bivariate normal distribution +with \(E(X)=0\), \(E(Y)=1\), \(var(X)=1\), \(var(Y)=4\), and \(Corr(X,Y)=0.5\). Note that \(Cov(X,Y)=0.5\times \sqrt{1}\times +\sqrt{4}=1\).
+CovXY=matrix(c(1,1,1,4),nrow=2)
+x=seq(from=-3,to=3,by=0.1) ## range of X
+y=seq(from=-7,to=7,by=0.1) ## range of Y
+xy=cbind(x=rep(x,each=length(y)),y=rep(y,times=length(x))) ## pairs of (x,y) in matrix
+fxy=dmvnorm(xy,mean=c(0,1),sigma=CovXY) ## a vector of joint density function at (x,y)
+z=matrix(fxy,byrow=TRUE,nrow=length(x),ncol=length(y)) ## reshape to a matrix
+
+persp3d(x,y,z,col='steelblue',alpha=0.7) ## 3D plot in rgl library, alpha<1 for transparent color
+
+planes3d(a=0,b=0,c=-1,d=0.08,alpha=0.5, col='green') ## plane st. joint pdf=0.08
+planes3d(a=0,b=0,c=-1,d=0.06,alpha=0.5, col='green') ## plane st. joint pdf=0.06
+planes3d(a=0,b=0,c=-1,d=0.04,alpha=0.5, col='green') ## plane st. joint pdf=0.04
+
+grid3d(c("x", "y", "z")) ## add grid in a 3D plotLet’s see this bivariate normal density in a movie.
+movie3d(spin3d(axis=c(0,0,1),rpm=4), duration = 15, movie ="BiNorm", type="gif",
+ dir="./Rmovies",verbose = FALSE)knitr::include_graphics("./Rmovies/BiNorm.gif") ## include animation in HTML output
The following widget has an interactive 3D plot of the bivariate +normal density.
+rglwidget(elementId = "BiNorm") ## Create an interative widget in HTML outputclipplanes3d(a=0,b=0,c=-1,d=0.04) ## slice the bell horizontally
+rglwidget(elementId = "BiNorm_cut")2. Conditional Distributions.
+We consider f(x,1), so the density at \(y=1\)
+persp3d(x, y, z, col='steelblue', alpha=0.7)
+planes3d(a=0, b=-1, c=0, d=1, alpha=0.5, col='green')
+grid3d(c("x", "y", "z"))
+rglwidget(elementId = "BiNorm_cond")clipplanes3d(a=0,b=-1,c=0,d=1) ## slice the bell vertically (conditional distribution)
+rglwidget(elementId = "BiNorm_cond_cut")movie3d(spin3d(axis=c(0,0,1),rpm=4), duration = 15, movie ="BiNorm_cond", type="gif",
+ dir="./Rmovies",verbose = FALSE)knitr::include_graphics("./Rmovies/BiNorm_cond.gif") ## include animation in HTML output
If we scale this curve by the probability that \(y\) at(around) \(y=1\), then we have the conditional density +\(f(x|y=1)\).
+3. Marginal Distributions
+Projecting all these bell curves to the x-axis, we have
+plot(x,z[,80],type='l',ylab='f(x,fixed y)') ## this curve is for y=1.
+for (i in 1:dim(z)[2]){
+ lines(x,z[,i])
+}
The marginal density of \(X\) is the +weighted aggregation of all curves in above figure, where the weight is +probability that \(Y=y\).
+4. Data from a Bivariate Normal Distribution
+Now we generate a random sample from above Binormal distributions
+## Use rmvnorm function to generate multivariate normal data
+XYi=data.frame(rmvnorm(1000,mean=c(0,1),sigma=CovXY))
+names(XYi)=c('x','y') ## assign column namesCheck the scatter plot of data with 4 contour lines, sample mean +point and populational mean point.
+plot(XYi[,1],XYi[,2],pch=20,col=rgb(0.2,0.2,1,alpha=0.3),
+ xlab='x',ylab='y',main='Bivariate Normal Random Variables')
+
+## Add the means of X and Y on the plot
+points(mean(XYi[,1]),mean(XYi[,2]),col='red',pch=19) ## sample mean
+points(0,1,col='green',pch=2,cex=1.5) ## population mean by green triangle
+contour(x,y,z,add=T,nlevel=4,labcex=1,lwd=2) ## contour lines 
We see the darkest part of x-axis is around 0 and that of y-axis is +around 1. The contour lines close to the center are also denser. The +following 3D histogram and 2D heat map also show similar +observation.
+library(plot3D)
+## Create cuts:
+x_c = cut(XYi[,1], 20)
+y_c = cut(XYi[,2], 20)
+## Calculate joint counts at cut levels:
+z_c <- table(x_c, y_c)
+
+## Plot as a 3D histogram:
+hist3D(z=z_c, border="black")
## Plot as a 2D heatmap:
+image2D(z=z_c, border="black")
We draw a histogram of \(X\) given +\(Y\) is around 1. First define a +horizontal strip: \(Y\) is within \(1-delta \sim 1+\delta\). Second we draw +histogram of all \(X\) in the defined +strip, which looks like normal.
+delta=0.5
+yv=1
+
+library(ggplot2)
+ggplot(XYi, aes(x = x, y = y)) +geom_point()+
+ geom_point(x=0,y=1,col="red",shape="*",size=15)+
+ geom_ribbon(aes(ymin=yv-delta,ymax=yv+delta),alpha=0.2,col="blue")
hist(XYi[(XYi[,2]>yv-delta &XYi[,2]<yv+delta),1],
+ main="Histogram of X given Y",xlab="")
If we push the observations to the Y-axis, we can find the marginal +histogram for \(Y\). Similarly, +marginal histogram for \(X\) is plotted +at the bottom. (The points are jiggled a little to see the +frequency.)
+library(ggplot2)
+scatter <- qplot(x,y, data=XYi) +
+ scale_x_continuous(limits=c(min(x),max(x))) +
+ scale_y_continuous(limits=c(min(y),max(y))) +
+ geom_rug(col=rgb(.5,0,0,alpha=.2))+
+ geom_point(x=0,y=1,col="red",shape="*",size=15)
+scatter
+
+# Plot the scatter plot with marginal histograms
+library(ggExtra)
+p = ggplot(XYi, aes(x = x, y = y)) +geom_point()+
+ geom_point(x=0,y=1,col="red",shape="*",size=15)
+ggMarginal(p, type = "histogram")
5. Conditional Mean Function
+Consider \(E(Y|X=x)\) as a function +of \(x\). For example, we take vertical +strips at \(x=-1.5,x=0\) and \(x=1.5\), and display histograms of \(Y\) conditional on these values of \(X\). You can see how the conditional mean +of \(Y\) increases as the value of +\(X\) increases.
+delta=0.25
+xv=c(-1.5,0,1.5)
+#xv=0
+
+library(ggplot2)
+p1=ggplot(XYi, aes(x = x, y = y)) +geom_point()+
+ geom_point(x=0,y=1,col="red",shape="*",size=15)+
+ geom_ribbon(aes(xmin=xv[1]-delta,xmax=xv[1]+delta),alpha=0.2,col="blue")+
+ geom_ribbon(aes(xmin=xv[2]-delta,xmax=xv[2]+delta),alpha=0.2,col="blue")+
+ geom_ribbon(aes(xmin=xv[3]-delta,xmax=xv[3]+delta),alpha=0.2,col="blue")
+
+histdat=c()
+ybar.cond=c()
+for (i in 1:length(xv)){
+ yi=XYi[(XYi[,1]>xv[i]-delta &XYi[,1]<xv[i]+delta),2]
+ histdat=rbind(histdat, cbind(x=xv[i],y=yi) )
+ ybar.cond=rbind(ybar.cond, cbind(x=xv[i],ybar=mean(yi)) )
+}
+
+histdat=as.data.frame(histdat)
+ybar.cond=as.data.frame(ybar.cond)
+
+p2=ggplot(histdat, aes(x = y, fill = factor(x))) +
+ geom_histogram(aes(y = ..density..))+ ## plot histogram for each value of x
+ geom_density()+ ## overlay density
+ facet_grid(cols =vars(x))+ ## histogram by group in different plots
+ geom_vline(data=ybar.cond,aes(xintercept = ybar),colour = "black")+
+ theme(legend.position = "none")+ ## remove legend
+ coord_flip() ## rotate the plot
+
+
+library(gridExtra) ## for arrangment of ggplots
+grid.arrange(p1, p2, nrow=2)## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
This is the idea of a REGRESSION model. 
Source: Introductory
+Statistics
Lecture_22
+Zheyuan Wu
+2023-03-10
+ +Recap on exams
+Exponential distribution is usually used to measure the distribution +of ages. The supports of normal distribution includes negative +values…
+Law of Large Numbers
+-
+
- \(\bar X\) and \(E(X)\). +
how well \(\bar X\) can estimate the +\(E(X)\).
+The Law of Large Numbers (LLN): Let \(X_1,...,X_n\) be iid and let g be a +function such that \(-\infty<E[g(X_1)]<\infty\). Then for +any $ > 0$
+\[ +P(|{1\over n}\sum^n_{i=1}g(X_i)-E[g(X_1)]|>\epsilon)\to 0~as~n\to +\infty +\]
+For a large n, \({1\over n}\sum +g(X_1)\) is very close to E(X) with a high probability.
+-
+
We say that \({1\over +n}\sum^n_{i=1}g(X_i)\) converges in probability to \(E[g(X_1)]\).
+In particular, if \(-\infty< E(X_1) +< \infty\), then \(\bar X\) +converges in probability to \(E(X_1)\)
+
-
+
For very large n, there will be high. prob. that \(\bar X={1\over n}\sum X_i\) will be close +to \(E(X_1)\).
+Reasonable to use \(\bar X\) to +estimate \(E(X_1)\)
+
-
+
We call \(\bar X\) a +consistent estimator of \(E(X_1)\).
+Since \(\hat p\) is also a +sample mean, from the LLN we also have that
+
\(\hat p\) converges in probability +to p.
+Note: For many distributions \(-\infty<E(X)<\infty\), but there are +exceptions.
+Example: Let X be a random variable with PDF \(f(x)={1\over x^2}, 1\leq X<\infty\).
+\[ +E(X)=\int^{\infty}_{-\infty}xf(x)~dx=\infty +\]
+Limitation of LLN:
+LLN does not provide the speed of convergence, or how close \(\bar X\) to \(E(X)\).
+Convolutions
+The convolution of two independent random variables refers +to the distribution of their sum.
+if X is independent of X+Y is convolution
+Example: Let \(X \sim Bin(n_1,p)\) +and let \(Y \sim Bin(n_2,p)\). If X and +Y are independent, then \(X + Y \sim Bin(n_1 + +n_2,p)\).
+Let \(S_i\) \(T_j\) denote the success or failure in a +Bernoulli trial.
+\[ +X=\sum^{n_1}_{i=1}S_i,Y=\sum^{n_2}_{i=1}T_i +\]
+\[ +X+Y=\sum^{n_1}_{i=1}S_i+\sum^{n_2}_{i=1}T_i\sim Bin((n_1+n_2),p) +\]
+Other examples: Let X and Y be independent random variables. i. If +\(X ∼Poisson(\lambda_1)\) and \(Y ∼Poisson(\lambda_2)\), then \(X+Y ∼Poisson(\lambda_1 +\lambda_2)\). ii. +If \(X ∼N(\mu_1,\sigma^2_1 )\) and +\(Y ∼N(\mu_2,\sigma^2_2 )\), then \(X + Y ∼N(μ_1+ μ_2.\sigma^2_1 + \sigma^2_2 +)\).
+Result 1: Let \(X_1,...,X_n\) be +independent random variables with \(X_i +∼N(\mu_i,\sigma^2_i )\) and let \(Y = +a_1X_1 + ···+ a_nX_n\) for constants \(a_1,...,a_n\). Then
+-
+
\(Y \sim N(\mu_Y ,\sigma^2_Y +)\)
+\(\mu_Y = a_1\mu_1 + ···+ +a_n\mu_n\)
+\(\sigma^2_Y = a^2_1\sigma^2_1 + ···+ +a^2_n\sigma^2_n\)
+
Example: History suggests that scores on the Math portion of the SAT +are normally distributed with a mean of 529 and a variance of 5732. +History also suggests that scores on the Verbal portion of the SAT are +normally distributed with a mean of 474 and a variance of 6368. Select +two students at random. Let X denote the first student’s Math score, and +let Y denote the second student’s Verbal score. What is P(X > Y +)?
+Since X is independent of Y
+\[ +P(X>Y)=P(X-Y>0) +\]
+\[ +\mu=E(X)-E(Y)=529-474=55 +\]
+\[ +\sigma^2=Var(X)+Var(Y)=5732+6368=12100 +\]
+Result 2: Let \(X_1,...,X_n\) be iid +\(N(\mu,\sigma^2)\) and let \(\bar X\) be the sample mean. then
+\[ +\bar X\sim N(\mu_{\bar X},\sigma^2_{\bar X}) +\]
+where \(\mu_{\bar X}=\mu\) and \(\sigma_{\bar X}^2={\sigma^2\over n}\)
+Example: Suppose we want to estimate the mean of a normal population +whose variance is known to be 4. What sample size should be used to +ensure that \(\bar X\) lies within 0.5 +units of the population mean with probability 0.90? \[ +\bar X\sim N(\mu,{4\over n})\to {\bar X-\mu\over \sqrt{4\over n}}\sim +N(0,1) +\]
+\[ +P({-0.5\over \sqrt{4\over n}}<{\bar X-\mu\over \sqrt{4\over +n}}<{0.5\over \sqrt{4\over n}}) +\]
+In R:
+qnorm(0.95,0,1)## [1] 1.644854Lecture_23
+Zheyuan Wu
+2023-03-20
+ +Recap for Law of Large numbers
+The Central Limit Theorem
+-
+
In general, finding the exact distribution of \(\hat X\) is difficult.
+However, for large sample sizes, the distribution of \(\hat X\) can be approximated
+
Central Limit Theorem (CLT): Let \(X_1,...X_n\) be iid with finite mean \(\mu\) and finite variance \(\sigma^2\). Then for large enough n,
+-
+
- $X $ has approximately a normal distribution with mean \(\mu\) and variance \(\sigma^2\over n\) +
\[ +\bar X\dot\sim N(\mu,{\sigma^2\over n}) +\]
+-
+
- \(T=X_1+...+X_n\) has approximately +a normal distribution with mean \(n\mu\) and variance \(n\mu\). +
\[ +T=X_1+...+X_n\dot\sim N(n\mu,n\sigma^2) +\]
+approximate/estimate mean would be better as n increase, In practice, +worked well for \(n\geq 30\).
+Key result we use for inference
+Example: The level of impurity in a randomly selected batch of +chemicals is a random variable with \(\mu\) = 4.0% and \(\sigma\) = 1.5%. For a random sample of 50 +batches, find
+-
+
- an approximation to the probability that the average level of +impurity is between 3.5% and 3.8%. +
Let \(X_1,..,X_{50}\) be the +impurety level in the batches.
+\(X_1,..,X_{50}\) are iid with mean +\(\mu=4\%\), and \(\sigma^2=(1.5\%)^2\)
+By Central Limit Theorem: \(\bar X \dot\sim +N(0.04,{0.015^2\over 50})\)
+\[ +P(3.5\% < \bar X<3.8\%)=0.1636782 +\]
+pnorm(3.8,4,sqrt(1.5^2/50))-pnorm(3.5,4,sqrt(1.5^2/50))## [1] 0.1636782-
+
- an approximation to the 95th percentile of the average impurity +level. +
qnorm(0.95,4,sqrt(1.5^2/50))## [1] 4.348926Remark \(\bar X\) is more +concentrated around \(\mu\) than +individual \(X_i\).
+Normal Approximation to the Binomial
+DeMoivre-Laplace Theorem: If \(T\sim Bin +(n,p)\) then, for large enought n,
+\[ +T\dot\sim N(np,np(1-p)) +\]
+Recall \(T=\sum^n_{i=1}X_i\), where +X: iid \(Ber(p)\)
+with \(E(X_i)=p, Var(X_i)=p(1-p)\). +if \(n\geq 30\)
+Continuity correlation: Let \(T\sim Bin(n,p)\) and let \(Y\sim N(np,np(1-p))\).
+The condition for CLT is \(np\geq +5\), and \(n(1-p)\geq 5\), +derived from \(n\geq 30\)
+To correct the approximation using a continuous random variable to a +discrete random variable.
+-
+
\(P(T\leq k)\approx P(Y\leq +k+0.5)\),\(T\sim Bin(n,p),Y\sim +N(np,np(1-p))\)
+\(P(T= k)\approx P(k-0.5< Y < +k+0.5)\)
+\(P(T\geq k)\approx P(Y\geq +k-0.5)\)
+
Example: Suppose that only 60% of all drivers wear seat belts at all +times. In a random sample of 500 drivers let X denote the number of +drivers who wear seat belt at all times.
+-
+
- State the exact distribution of X and use R to find \(P(270 \leq X \leq 320)\). +
\(X\sim Bin(n,p)\) where \(n=500,p=0.6\)
+pbinom(320,500,0.6)-pbinom(269,500,0.6)## [1] 0.9671151# use 269 to include the 270.-
+
- Approximate \(P(270 \leq X \leq +320)\) using the continuity correction. +
By Central Limit Theorem, \(X\dot \sim +N(np,np(1-p))\), with \(np=500\times +0.6,np(1-p)=500\times (1-0.6)\times 0.6=Var\)
+\[ +\approx P(270\leq X\leq 320)\\ +=P(X<320.5)-P(X<269.5) +\]
+In R
+pnorm(320.5,500*0.6,sqrt(500*0.6*0.4))-pnorm(269.5,500*0.6,sqrt(500*0.6*0.4))## [1] 0.9666716When sample size is low, the result might not be as valid as +expected.
+Use big sample for approximation.
+Lecture_24
+Zheyuan Wu
+2023-03-22
+ +Estimation
+Goal: to estimate unknown population parameters
+-
+
model free (sample mean to estimate population mean, sample +variance to estimate population variance…)
+model based (start from a prob distribution then used data to +estimate unknown pane.)
+
We will focus first on point estimation.
+-
+
- estimate unknown parameter with a single value. +
Notation:
+-
+
\(\theta\) is the +unknown population parameter of interest.
+\(X_1,...X_n\) is the sample +before values are observed.
+Random variable.
+\(x_1,...x_n\) are the +observed sample values or data.
+Fixed values (not random)
+\(\hat \theta\) is a quantity +used to estimate θ.
+\(\hat \theta\) \(X_1,...X_n\) is called an +estimator, a function of random variables
+\(\hat \theta\) \(x_1,...x_n\) is called an +estimate, a function of observed values.
+For \(\theta=\mu\), \(\hat \theta(X_1,...X_n)={1\over +n}\sum^n_{i=1}X_i\).
+
Unbiased Estimators
+Often there are multiple ways of estimating a parameter.
+-
+
How do we choose one estimator over another?
+What are some properties of “good” estimators?
+
Definition: An estimator \(\hat +\theta\) of \(\theta\) is +unbiased for θ if
+\[ +E_{\theta}(\hat\theta)=\theta +\]
+Defnition: The bias of \(\hat +\theta\) is
+\[ +bias(\hat\theta)=E_{\theta}(\hat\theta)-\theta +\]
+Examples:
+-
+
\(\bar X\) is an unbiased +estimator of \(\mu=E(X_i)\).
+\(\hat p\) is an unbiased +estimator of p.
+Let \(X_i,..,X_n\) be iid with +common variance \(\sigma^2\). +Then
+
\[ +S^2={1\over n-1}\sum^n_{i=1}(X_i-\bar X)^2 +\]
+is unbiased for \(\sigma^2\)
+Note: S is a biased estimator of \(\sigma\)
+Definition: The standard error of an estimator \(\hat \theta\) is
+\[ +\sigma_{\hat \theta}=\sqrt{Var_{\theta}(\hat \theta)} +\]
+The estimated standard error is denoted by \(S_{\hat\theta}\)
+Example: Let \(\bar X_1\) and \(S_1^2\) be the sample mean and variance of +a simple random sample of size \(n_1\) +from a population with mean \(\mu_1\) +and variance \(\sigma_1^2\). Let \(\bar X_2\) and \(S_2^2\) be the sample mean and variance of +a simple random sample of size \(n_2\) +from a population with mean \(\mu_2\) +and variance \(\sigma^2_2\). Assume the +two samples are independent.
+-
+
- Show that\(\bar X_1-\bar X_2\) is +an unbiased estimator of \(\mu_1-\mu_2\). +
Lecture_25
+Zheyuan Wu
+2023-03-24
+ +Mean Square Error
+The mean square error (MSE) of an estimator \(\hat \theta\) for \(\theta\) is
+\[ +\begin{aligned} +MSE(\hat \theta)&=E_\theta[(\hat \theta - \theta)^2]\\ +&=E((\hat \theta-E(\hat \theta)+E(\hat \theta)-\theta)^2)\\ +&=E[(\hat\theta-E(\hat +\theta))^2+(E(\hat\theta)-\theta)^2-2((\hat\theta-E(\hat \theta))\times +(E(\hat\theta)-\theta))]\\ +&=Var_\theta(\hat \theta)+[bias(\hat \theta)]^2 +\end{aligned} +\]
+-
+
- If there are multiple estmator for \(\theta\), the estimatro with the smallest +MSE is preferred. +
Note: The MSE can also be expressed as
+\[ +MSE(\hat\theta)=Var_\theta(\hat \theta)+[bias(\hat \theta)]^2 +\]
+For unbiased estimator:
+For unbiased estimators, the preferred estimator is the one with the +smallest variance. Even it have larger bias
+Method of Moments Estimation
+Method of moments estimation is a model based estimation +technique.
+-
+
It equates theoretical moments with sample moments/
+This yields a set of equation to solve for the +estimators.
+
The kth theoretical moment
+\[ +\mu_k=E(X^k) +\]
+The kth sample moment:
+\[ +\hat \mu_k={1\over n}\sum^n_i=1{X_i^k} +\]
+Example: Let \(X_1,...X_n\) be iid +\(Poisson(\lambda)\).
+-
+
- Find the method of moments estimator of \(\lambda\). +
k=1, only need to use the \(1^{th}\) +moment.
+\(\mu_1=E(X_i)\), for \(i=1,...,n\), Note \(E(X_i)=\lambda\)
+\(\hat \mu_1={1\over +n}\sum^n_{i=1}{X_i^k}\)
+Setting \(\hat \mu=\bar X\) is the +moment estimator for \(\lambda\)
+-
+
- Is the estimator unbiased? +
DNE.
+Example: Let \(X_1,...X_n\) be iid +from the uniform on \([\alpha, \beta]\) +distribution. Find the method of moment estimator of \(\alpha\) and \(\beta\) using
+\[ +\mu_1 = E(X_i)={\alpha+\beta\over 2}\\ +\mu_2=E(X_i^2)=Var(X_i)+[bias(X_i)]^2={(\beta-\alpha)^2\over +12}+({\alpha+\beta\over 2})^2\\ +\]
+Setting \(\hat \mu_1={1\over +n}\sum^n_{i=i}{X_i}\), \(\hat +\mu_1={1\over n}\sum^n_{i=1}{X_i^2}\)
+Then solve the equation.
+Note: The textbook uses a slightly different approach for the second +sample moment.
+Maimum Likelihood Estimation
+Maximum likelihood (ML) estimation is a model based estimation +technique.
+-
+
Answers the question “what value of the parameter is most likely +to have generated the data?”
+Maximizes the probability of the observed data.
+
Suppose \(X_1, . . . , X_n\) are iid +random variables with PDF \(f(x_i| +\theta)\) or PMF \(p(x_i| +\theta)\).
+The likelihood function is
+\[ +lik(\theta)=\prod^n_{i=1}f(x_i|\theta)~or~lik(\theta)=\prod^n_{i=1}p(x_i|\theta) +\]
+Treat as a function of \(\theta\), +Joint pdf of \(X_1, . . . , X_n\) +evaluated at the observed values: \(X_1, . . . +, X_n\)
+-
+
The value of θ that maximizes \(lik(\theta)\) is the maximum likelihood +estimator \((MLE) \hat +\theta\).
+It is often easier to maximize the log of the likelihood +function.
+
\[ +\mathcal{L} (\theta)=\sum^n-{i=1}log[f(x_i|\theta)] +\]
+Example: Let \(X_1, . . . , X_n\) be +iid \(N(\theta,1)\). Find the
+Recall: \(f(x_i|\theta)={1\over +\sqrt{2\pi\sigma^2}}exp\{-{1\over +2\sigma^2}(x_1-\theta)^2\}\)
+\(={1\over \sqrt{2\pi}}exp\{-{1\over +2}(x_1-\theta)^2\}\)
+\[ +log[f(x_i|\theta)]=-{1\over2}(x_1-\theta)^2+log({1\over \sqrt{2\pi}}) +\]
+\[ +{\partial +\mathcal{L}(\theta)\over\partial\theta}={1\over2}\sum^n_{i=1}2(x_i-\theta)(-1)=\sum^n_{i=1}x_i-n\theta +\]
+\[ +{\partial^2 \mathcal{L}(\theta)\over\partial^2\theta}=-n<0 +\]
+so
+\[ +\hat \theta={1\over n}\sum^n_{i=1}x_i +\]
+maximize the likelihood function. Hence, \(\hat \theta=\bar X\) is the MLE
+Lecture_26
+Zheyuan Wu
+2023-03-27
+ +Example from last week lectures
+Example: Let \(X_1,...,X_n\) be iid +\(uniform(0,\theta)\).
+\[ +f(x)\begin{cases}{1\over \theta}~~~0\leq x\leq +\theta,\\0,~~~elsewhere\end{cases} +\]
+-
+
- Find the method of moments estimator of \(\theta\). +
k=1, only need the first moment
+\[ +\mu_1=E(X_1)={\theta\over 2}\\ +\hat\mu_1={1\over n}\sum_{i=1}^nX_i +\]
+Setting \(\mu_1=\hat \mu_1\), we +have \({\theta\over 2}={1\over +n}\sum_{i=1}^nX_i\Rightarrow\hat\theta=2\bar X\)
+-
+
- Find the MLE of \(\theta\). +
\[ +lik(\theta)=\prod^n_{i=1}f(x_i|\theta)=\begin{cases}{1\over\theta^n},~~~0\leq +X_1,...,X_n\leq \theta\\0,~~~ elsewhere\end{cases} +\]
+Introduction to Confidence Intervals
+We can use the sample mean \(\bar +X\) as a point estimate for the population mean \(\mu\).
+-
+
LLN: \(\bar X\) approximates +\(\mu\) more accurately as the sample +size increases.
+CLT: allows us to access the probability that \(\bar X\) will be within a certain distance +from \(\mu\).
+
use only \(\bar X\)to estimate \(\mu\), it is not very informative since it +does not quantify accuracy.
+A confidence interval is an interval for which we can +asserts, with a given degree of confidence/certainty, that it includes +the true value of the parameter being estimated.
+Z Confidence Intervals
+-
+
- Confidence intervals that use percentiles from the standard normal +distribution. +
\[ +z_\alpha=100(1-\alpha)-th~percentile +\]
+\(z_\alpha\) is also called as upper +percentile sometime.
+For example:
+\[ +\alpha =0.025\\ +z_\alpha=qnorm(1-\alpha)=1.96 +\]
+Let \(X_1,...,X_n\) be iid random +variables with parameter \(\theta\).
+-
+
- By CLT, many estimators \(\hat +\theta\) of \(\theta\) will be +approximately normally distributed. +
\[ +\hat\theta\sim N(\theta,\sigma_{\hat\theta}^2), approximately\\ +{\hat\theta-\theta\over \sigma_{\hat\theta}}\sim N(0,1), approximately +\]
+you can also replace \(\sigma_{\hat\theta}\) by \(S_{\hat\theta}\) (estimated standard error) +if \(\sigma_{\hat\theta}\) is +unknown
+-
+
- This implies: +
\[ +P(-1.96<{\hat\theta-\theta\over \sigma_{\hat\theta}}<1.96)=0.95 +\]
+-
+
- This results in a 95% confidence interval for \(\theta\): +
\[ +\hat \theta-1.96S_{\hat \theta}\leq \theta\leq \hat\theta+1.96S_{\hat +\theta} +\]
+\[ +{\hat\theta-\theta\over \sigma_{\hat\theta}}\leq1.96 +\Leftrightarrow \hat\theta-\theta\leq1.96\sigma_{\hat\theta} +\Leftrightarrow \theta\geq\hat \theta-1.96\sigma_{\hat\theta} +\]
+\[ +{\hat\theta-\theta\over \sigma_{\hat\theta}}\geq-1.96 +\Leftrightarrow \hat\theta-\theta\geq -1.96\sigma_{\hat\theta} +\Leftrightarrow \theta\leq\hat \theta+1.96\sigma_{\hat\theta} +\]
+you can replace \(\sigma_{\hat\theta}\) by \(S_{\hat \theta}\)
+-
+
- The general \((1-\alpha)100%\) +confidence interval for \(\theta\) +is +
\[ +\hat\theta-z_{\alpha/2}S_{\hat\theta}\leq\theta\leq\hat\theta+z_{\alpha/2}S_{\hat\theta} +\]
+gives us a range of possible values of the true parameter \(\theta\), such that we are \((1-\alpha)100%\) confident that it contains +the true \(\theta\)
+Common choice of \(\alpha\): 0.01 +(99%), 0.05 (95%), 0.01 (90%).
+-
+
- Z confidence interval will generally be used for proportions \((\theta=p)\). +
only works well if \(\sigma\) is +known in mean estimation or sample size is very large.
+T Confidence Intervals
+-
+
Confidence intervals that use percentiles from the T +distribution.
+\(T_v\) is a random variable +from the T distribution with v degrees of freedom. (or df)
+
degree of freedom often depends on the sample size.
+\[ +\lim_{v\to+\infty}t_v\sim N(0,1) +\]
+-
+
In R:
+dt(x, v) gives the PDF of \(T_v\) for x
+pt(x, v) gives the CDF of \(T_v\) for x
+qt(s, v) gives the s100 percentile of \(T_v\)
+rt(n, v) generates a random sample of n \(T_v\) random variables
+Many estimator \(\hat\theta\) of +\(\theta\) satisfy
+
\[ +{\hat\theta-\theta\over S_{\hat\theta}}\sim T_v +\]
+when you replace the estimated by \(S_{\hat +\theta}\), the result shall have greater variability.
+\({\hat\theta-\theta\over +S_{\hat\theta}}\) has a greater variability than \({\hat\theta-\theta\over +\sigma_{\hat\theta}}\)
+-
+
- This gives a \((1-\alpha)100\%\) +confidence interval for \(\theta\): +
\[ +(\hat\theta-t_{v,\alpha/2}S_{\hat\theta},\hat\theta+t_{v,\alpha/2}S_{\hat\theta}) +\]
+Notes:
+-
+
T confidence intervals will generally be used for means.
+\(t_{v,\alpha/2}\to +z_{\alpha/2}\) as \(n\to +\infty\)
+Common values of \(\alpha\) are +0.1,0.05,and 0.01
+
Lecture_27
+Zheyuan Wu
+2023-03-29
+ +Confidence Intervals for Proportions
+Example: Suppose we take a survey of 100 American adults and find +that 89 of them favor expanding solar energy. How would we estimate the +true proportion of American adults who favor expanding solar energy?
+estimate sample proportion by \(\hat +p={89\over 100}\) so sample proportion, 89%. Is a point estimator +of p. However, the \(\hat p\) is random +if we re-sample.
+Now: We’ll provide an interval of plausible values for this +proportion.
+Let \(X \sim Bin(n,p)\) and let +\(\hat p={X\over n}\). Then use the +Central Limit Theorem,
+\[ +{\hat p -p \over \sqrt{\hat p(1-p)\hat p \over n}}\dot\sim N(0,1) +\]
+\(p\) is the mean of \(\hat p\) and the \(\sqrt{\hat p(1-p)\hat p \over n}\) is the +estimated standard error of \(\hat +p\).
+A \((1-\alpha)100\%\) confidence +interval for p.
+\[ +P(-1.96\leq {\hat p -p \over \sqrt{\hat p(1-p)\hat p \over n}}\leq +1.96)=95\% +\]
+rearrange the equation above, you can get:
+\[ +\begin{pmatrix}\hat p-z_{\alpha/2} \sqrt{\hat p(1-p)\hat p \over n},\hat +p+z_{\alpha/2} \sqrt{\hat p(1-p)\hat p \over n}\end {pmatrix} +\]
+where \(z_{\alpha/2} \sqrt{\hat p(1-p)\hat +p \over n}\) is called the margin of error (ME)
+This confidence interval works well as long as \(n\hat p\geq 8\) (number of observed +successes) and \(n(1-\hat p)\geq 8\) +(number of observed failures) In general, the larger \(np\), \(n(1-p)\), the better Central Limit Theorem +works.
+Recall \(z_{\alpha /2}\) can be
+found form R using qnorm(1-a/2, 0, 1).
Example: Find the 95% confidence interval for the proportion of +American adults who favor expanding solar energy.
+for this example, we can get 95% confidence interval that p will lies +in
+(0.89+qnorm(0.975)*sqrt(0.89*(1-0.89)/100))## [1] 0.9513253(0.89+qnorm(0.025)*sqrt(0.89*(1-0.89)/100))## [1] 0.8286747When we increase the sample size, we can narrow the range of +estimation
+(0.89+qnorm(0.975)*sqrt(0.89*(1-0.89)/1000))## [1] 0.9093928(0.89+qnorm(0.025)*sqrt(0.89*(1-0.89)/1000))## [1] 0.8706072How do we interpret this confidence interval?
+-
+
The true value of the parameter p is fixed, not random.
+Before collecting data, the confidence interval is +random.
+After collecting data, the confidence interval is not random.
+The calculated Confidence interval either contains the true p or +not. So we don’t say the probability that the given confidence interval +contains p.
+We say we are 95% confident p is in the +interval.
+If we take many samples and compute 95% confidence +intervals from each sample, about 95% of those intervals will contain +the true value of p.
+
Precision
+The precision in the estimation of p is quantified by the margin of +error or the length of the confidence interval (2ME).
+For proportions:
+\[ +MOE=z_{\alpha/2}\sqrt{\hat p (1-\hat p)\over n} +\]
+MOE depends on \(\alpha\), as \(\alpha\) increase, Margin of error +decrease. MOE also depends on \(n\), as +\(n\) increases, Margin of error +decrease.
+How can we make our confidence interval shorter?
+-
+
- Reduce the confidence level. +
qnorm(0.995)## [1] 2.575829qnorm(0.975)## [1] 1.959964qnorm(0.95)## [1] 1.644854-
+
- Increase the sample size +
L is desired length of Confidence Interval.
+\(L =2z_{\alpha/2}\sqrt{\hat p(1-\hat +p)\over n} \to n={4 z_{\alpha/2}\hat p(1-\hat p)\over L^2}\)
+Choose n so that
+\[ +n\geq {4z^2_{\alpha/2}\hat p_{pr}(1-\hat p_{pr})\over L^2} +\]
+(The value of n is round up already)
+-
+
\(\hat p_{pr}\) is preliminary +estimate of p.
+If no \(\hat p_{pr}\) exists, +use \(\hat p_{pr}=0.5\). (the worst +case that maximizes \(\hat p(1-\hat +p)\))
+
Example: Find the sample size required for a 95% confidence interval +of length 0.06 for the proportion of American adults who favor expanding +solar energy.
+\[ +L=0.06, z_{\alpha/2}=0.96,\hat p=0.89\\ +n\geq {4\times(1.96)^2\times 0.89\times +(1-0.89)\over0.06^2}=417.88\approx 418 +\]
+The worst case, use \(\hat +p=0.5\)
+\[ +n\geq {4\times(1.96)^2\times 0.5\times (1-0.5)\over +0.06^2}=1067.1\approx 1068 +\]
+Lecture_28
+Zheyuan Wu
+2023-03-31
+ +Confidnece Interval for the Mean
+Let \(X_1,... X_n\) be a simple +random sample from a population with mean \(\mu\) and variance \(\sigma ^2\).
+\(\bar X\) is the sample mean, \(S^2\) is the sample variance
+Z Confidence Interval for the Mean
+If the population variance \(\sigma^2\) is known, then by CLT
+\[ +{\bar X-\mu\over \sigma/\sqrt{n}}\dot\sim N(0,1) +\]
+This is equivalent to
+\[ +\bar X \dot\sim N(\mu,{\sigma^2\over n}) +\]
+\[ +P(-z_{\alpha/2}\leq {\bar X-\mu\over \sigma/\sqrt{n}}\leq +z_{\alpha/2})=1-\alpha\\ +P(-z_{\alpha/2}\cdot \sigma/\sqrt{n}\leq \bar X-\mu\leq +z_{\alpha/2}\cdot \sigma/\sqrt{n})=1-\alpha\\ +P( \bar X-z_{\alpha/2}\cdot \sigma/\sqrt{n}\leq\mu\leq \bar +X+z_{\alpha/2}\cdot \sigma/\sqrt{n})=1-\alpha +\]
+A \((1-\alpha)100\%\) confidence +interval for \(\mu\) is
+\[ +\begin{pmatrix}\bar X-z_{\alpha/2}{\sigma\over \sqrt{n}},\bar +X+z_{\alpha/2}{\sigma\over \sqrt{n}}\end{pmatrix} +\]
+\(z_{\alpha/2}{\sigma\over +\sqrt{n}}\) is the Margin of Error.
+This confidence interval only works if
+-
+
\(n\geq 30\) (from central limit +theorem) or the population has a normal distribution
+\(\sigma^2\) is known
+
This method is not really commonly used…
+T Confidence Interval for the Mean
+If we make the additional assumption that \(X-1,...,x-n\) are iid \(N(\mu,\sigma^2)\), then
+\[ +{\bar X-\mu\over S/\sqrt{n}}\sim T_{n-1} +\]
+We replace \(\sigma\) by sample +standard deviation S. with n-1 degree of freedom
+A \((1-\alpha)100%\) confidence +interval for \(\mu\) is
+\[ +\begin{pmatrix}\bar X-t_{n-1,\alpha/2}{S\over \sqrt{n}},\bar +X-t_{n-1,\alpha/2}{S\over \sqrt{n}}\end {pmatrix} +\]
+Replace \(z_{\alpha/2}\) with \(t_{n-1,\alpha/2}\) (critical value) \(z_{\alpha/2}<t_{n-1}\).
+Notes:
+-
+
If the population is normal, this confidence interval hods for +all n.
+If the population is not normal, this confidence interval will +still work well if \(n\geq +30\)
+\(t_{n-1,\alpha/2}\) can be +found in R using
qt(1-\alpha/2, n-1)
+
Example: The following data include temperatures for a random sample +of 10 time points in New York in September 1973. Find a 90% confidence +interval for the average temperature.
+temp=c(81,72,67,88,93,96,84,82,82,67)
+qqnorm(temp)
+qqline(temp)
This looks fine for normality
+\[ +n=10,\bar X=81.20,S=10.01\\ +t_{9,0.05}=qt(0.95,9)=1.833\\ +ME=1.833\times{10.01\over\sqrt{10}}=5.80\\ +81.20\pm5.8=[75.4,87.00] +\]
+Interpretation:
+We are 90% confident that the average temperature in New York in +Sep. 1973 is between 75.4 and 87.00 degree.
+We can obtain T confidence intervals in R:
+confint(lm(temp ~ 1),level =0.9)## 5 % 95 %
+## (Intercept) 75.39804 87.00196# lm stands for linear model, level is the confidence interval. 90% in this case)Precision
+For means, the margin of error is
+\[ +MOE=z_{\alpha/2}{\sigma\over \sqrt{n}}~ or~ MOE=t_{n-1,\alpha/2} {S\over +\sqrt{n}} +\]
+L is the desired length of Confidence Interval.
+\(L=2\times z_{\alpha/2}\times {\sigma\over +\sqrt{n}}\to n=(2\times z_{\alpha/2}\times {\sigma\over +L})^2\)
+To make the confidence interval shorter, we can increase the sample +size.
+Choose n so that
+\[ +n\geq(2z_{\alpha/2}{S_{pr}\over L})^2 +\]
+-
+
- \(S_{pr}\) is a preliminary +estimate of \(\sigma\). +
Note: Often a preliminary estimate of \(\sigma\) is not available.
+-
+
Guess a likely range of values that the variable will +take.
+Use \(S_{pr}={range\over 3.5}\) +(if uniform) \(S_{pr}={range\over 4}\) +(is normal)
+
Example: What sample size should we use to obtain a 90% confidence +interval of length 5 for the average temperature in New York in +1973?
+\[ +L=5, S_{pr}=10.01,z_{\alpha/2}=qnorm(0.95)=1.645\\ +n\geq[2\times 1.645\times{0.01\over 5}]^2=43.38=44 +\]
+Always round up.
+Confidence Intervals for the Variance
+Example: Suppose we want a confidence interval for the variance in +temperature in New York in 1973.
+\(\sigma^2\)= true variance, \(S^2\)= sample variance to estimate \(\sigma^2\)
+It turns out that if the population has a normal distribution
+\[ +{(n-1)S^2\over \sigma^2}\sim\chi^2_{n-1} +\]
+The chi-square Distribution
+If \(Z_1,...,Z_n\) are iid with +\(N(0,1)\) then \(Z_1^2,...,Z_n^2\sim \chi_n^2\).
+-
+
\(X\sim \chi_v^2\) has the \(\chi^2\) distribution with v degrees of +freedom
+Plot of \(\chi_v^2\) +densities:
+-
+
Positive support
+Non-symmetric
+
+
library(ggplot2)
+
+ggplot(data.frame(x = c(0, 20)), aes(x = x)) +
+ stat_function(fun = dchisq, args = list(df = 1))
ggplot(data.frame(x = c(0, 20)), aes(x = x)) +
+ stat_function(fun = dchisq, args = list(df = 2))
ggplot(data.frame(x = c(0, 20)), aes(x = x)) +
+ stat_function(fun = dchisq, args = list(df = 4))
ggplot(data.frame(x = c(0, 20)), aes(x = x)) +
+ stat_function(fun = dchisq, args = list(df = 8))
Confidence Interval
+A \((1-\alpha)100\%\) confidence +interval for \(\sigma^2\) is
+\[ +{(n-1)S^2\over \chi_{n-1,\alpha/2}^2}<\sigma^2<{(n-1)S^2\over +\chi_{n-1,1-\alpha/2}^2} +\]
+percentiles for the \(\chi^2\) +distribution can be found in R:
+qchisq(p, v) gives \(\chi^2_{v,1-p}\)
Examples continued from last week
+Example: Find the 95% confidence interval for the variance in +temperature in New York in 1973
+Use data continued from last chapter.
+\[ +S=10.01,n=10,S^2=10.01^2=100.20 +\]
+\[ +\alpha=0.05,\alpha/2=0.025,df=n-1=9 +\]
+\[ +\chi^2_{9,0.025}\approx19.023,\chi^2_{9,0.975}\approx 2.700 +\]
+qchisq(0.975,9) # use 1-p instead of p## [1] 19.02277qchisq(0.025,9)## [1] 2.700389\[ +({9\times 10.01^2\over 19.023},{9\times 10.01^2\over +2.7})=(47.41,334.00) +\]
+(47.41,334.00) is the estimation for \(\sigma^2\) (population variance)
+Interpretation: We are 95% confident that the true temperature +variance falls between 47.41 and 334.
+Notes:
+-
+
- We can also obtain a \((1-\alpha)100\%\) confidence interval for +the standard deviation +
\[ +\sqrt{(n-1)S^2\over +\chi^2_{n-1,\alpha/2}}<\sigma<\sqrt{(n-1)S^2\over +\chi^2_{n-1,1-\alpha/2}} +\]
+Example: Find the 95% confidence interval for the standard deviation +in temperature in New York in 1973.
+\[ +(\sqrt{47,41},\sqrt{334})=(6.89,1828) +\]
+-
+
- The confidence intervals for the variance and standard deviation +only hold if the population had a normal distribution. +
Even if n is large, we still need normality.
+Review: Confidence Interval for Proportion:
+-
+
- A \(100(1-\alpha)\%\) Confidence +interval for \(\hat p\) is \(\hat p \pm z_{\alpha/2}\sqrt{\hat p (1-\hat +p)/n}\) +
Use z test for proportion, and mean when population variance +is known
+Use t test for mean when population variance is +unknown
+Use chi square test for variance
+Lecture_3
+Zheyuan Wu
+2023-01-23
+ +Example from last class
+To select representative sample when there is groups of people:
+-
+
Separate sample to different group (eg.SRS of size 80 from group +from 800 Group A, then 15 from group from 150 Group B..)
+Combine all 3 groups to get 100 clients <- stratified +sampling.
+
In R
+y1 = sample (seq(1,800),size=80)
+y2 = sample (seq(801,950),size=15)
+y3 = sample (seq(951,1000),size=5)
+# in R, you may use c() to represent array.
+y = c(y1,y2,y3)statistical population
+v_i_ = response from the ith client in the population v_i_ += {1:yes,0,no} statistical population = {v_1_,v_2_,…v_1000_}
+Lecture_30
+Zheyuan Wu
+2023-04-05
+ +Introduction to Hypothesis Testing
+-
+
Hypothesis tests allow us to assess evidence provided by the data +in favor of some claim about a population parameter.
+Often we want to decide if the observed value of +a statistic (calculated from data) is consistent with some +hypothesized value of a parameter (true population +parameter is usually unknown).
+Can the difference (often observed) between the observed and +hypothesized values be attributed to random chance?
+
Example: A well established genetic model suggests that a particular +experimental cross of lupine should have blue flowers with probability +0.75 and white flowers with probability 0.25. In an experiment with 200 +seeds, 142 are blue flowering. Is this consistent with the model?
+The hypothesis value of \(p=0.75\),
+observed statistic \(\hat p={142\over +200}=0.71\)
+A statistical hypothesis test involves two opposing hypotheses.
+-
+
The null hypothesis, \(H_0\):
+-
+
Often a statement of “no effect” or “no difference”, the status +quo
+A test is designed to assess the strength of evidence against +\(H_0\).
+
+The alternative hypothesis, \(H_a\):
+
* Often states that there is an effect or a difference.
+
+* Test procedures do not treat the hypotheses equally.-
+
Test procedures do not treat the hypotheses equally.
+Favor/Reject the \(H_0\) (null +hypothesis) assume \(H_0\) is true +values we find evidence to reject \(H_0\).
+
\(H_a:p=0.75\), \(H_0:p\neq 0.75\)
+Example:
+A hypothesis test had two possible conclusions
+-
+
Reject \(H_0\), and conclude +\(H_a\) is true, there is sufficient +evidence against the \(H_0\), and we +can claim the \(H_a\) is true.
+Fail to reject \(H_0\)
+-
+
Either \(H_0\) or \(H_a\) could be true.
+Or there is no sufficient evidence against \(H_0\).
+
+
Since we are using sample data to draw a conclusion about a +population parameter, we may make an incorrect conclusion (by random +chance)
+-
+
A Type I error occurs when we reject \(H_0\) when \(H_0\) is true.
+A Type II error occurs when we fail to reject +\(H_0\) when \(H_0\) is false.
+
| Decision\Truth | +Truth \(H_0\) is true | +\(H_0\) is false (\(H_a\) is true) | +
|---|---|---|
| Do not reject \(H_0\) | +True negative | +Type II error (false negative) | +
| Reject \(H-0\) | +Type I error (false positive) | +True positive | +
A (very) simplified example: Suppose we receive a large shipment of +parts. We take a random sample of parts in the shipment in order to +determine whether to accept or reject the shipment.
+\(H_0\): shipment of parts is +good
+\(H_a\): shipment of parts is +bad
+-
+
We could accept a shipment of bad parts. (Type II error)
+We could reject a shipment of good parts. (Type I error)
+
Can we prevent Type I and Type II errors form happening?
+-
+
No. But we can control the probabilities that they +occur.
+\(\alpha =\) probability of +having type I error
+\(\beta =\) probability of +having type II error
+As \(\alpha\) decreases, \(\beta\) increases (and vice versa). In real +life, we usually control alpha first.
+It is easiest to control \(\alpha\).
+We will call \(\alpha\) the +level of significant of a hypothesis test. \(\alpha\) is determined by the research and +is a subjective choice.
+
Example: Consider a criminal trial
+-
+
\(H_0\): Defendant is +innocent.
+\(H_a\): Defendant is +guilty.
+Reject \(H_0\): Find the +defendant is guilty.
+Do not reject \(H_0\): Not +sufficient evidence to show that the defendant is guilty. (not +guilty)
+Type I error: Find a innocent person guilty.
+Type II error: Find a innocent person not guilty.
+
Hypothesis Tests
+A hypothesis test allows us to asses the evidence provided by the +data in favor of some claim about an unknown population parameter.
+-
+
Our hypothesis tests will be performed with level of +significance \(\alpha\). The most +common choices are 0.01,0.05,0.1.
+We will cover hypothesis tests for proportions, means, and +variances.
+
Procedure:
+-
+
State the hypotheses.
+Compute a test statistic.
+
-
+
Based on an esimation of the parameter.
+Has a known distribution of the estimation of \(H_0\),
+
-
+
- Reach a conclusion to the test. +
-
+
Rejection rule or region
+p-value
+
-
+
- State your conclusion in the contest of the problem. +
Hypothesis Tests for Proportions
+Example: In 1995, 40% of adults aged 18 years or older reported that +they had “a great deal” of confidence in the public schools. On June 1, +2005, the Gallup Organization released results of a poll in which 372 of +1004 adults aged 18 years or older stated that they had “a great deal” +of confidence in public schools. Does the evidence suggest at the \(\alpha\) = 0.05 significance level that the +proportion of adults aged 18 years or older having “a great deal” of +confidence in the public schools is lower in 2005 than in 1995?
+\(p=\) percentage of adults having +“a great deal” of confidence in the public schools.
+Let \(X\sim Bin(n,p)\) and let \(\hat p={X\over n}\) be the sample +proportion. (The number of adults being confidence in public school +among the sampled 1004 adults)
+-
+
- State the hypotheses +
We assume \(p_0\) be 40%
+\[ +H_0;p=p_0~~~H_0:p=p_0~~~H_0:p=p_0\\ +H_a;p>p_0~~~H_a:p<p_0~~~H_a:p\neq p_0\\ +\]
+The first is called one-sided right/upper test
+The second is called one-sided left/lower test
+The third is called two-sided test/two tail test
+Example:
+\(H_0:p=0.4\)
+\(H_a:p<0.4\)
+-
+
- Computer the test statistic +
\[ +Z_{H_0}={\hat p-p_0\over \sqrt{p_0(i-p_0)\over n}} +\]
+\(\sqrt{p_0(i-p_0)\over n}\) is the +standard error of \(\hat p\)
+Condition: \(np_0\geq 5\) and \(n(i-p_0)\geq5\)
+Example:
+\[ +\hat p={372\over 1004}=0.371\\ +Z_{H_0}={\hat p-p_0\over \sqrt{p_0(i-p_0)\over +n}}={0.371-0.4\over\sqrt{0.4(1-0.4)\over 1004} }=-1.876 +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(Z_{H_0}\dot \sim N(0,1)\).
+We want to determine if the observed value of \(Z_{H_0}\) is unusual for a \(N(0,1)\) random variable.
+If the observed value of \(Z_{H_0}\) is unusual, then we reject \(H_0\).
+
i. Rejection rule or region:
+
+ $H_a:p>p_0$
+
+ $H_a:p<p_0$
+
+ $H_a:p\neq p_0$
+
+ii. p-value:
+
+ - Using a p-value conveys the strength of the evidence agianst $H_0$.
+
+ - The p-value is the probability of observing what was obserbed if $H_0$ is true.
+
+ $H_a:p>p_0$
+
+ $H_a:p<p_0$
+
+ $H_a:p\neq p_0$
+
+ Lecture_31
+Zheyuan Wu
+2023-04-07
+ +Hypothesis Tests for Proportions
+Example: In 1995, 40% of adults aged 18 years or older reported that +they had “a great deal” of confidence in the public schools. On June 1, +2005, the Gallup Organization released results of a poll in which 372 of +1004 adults aged 18 years or older stated that they had “a great deal” +of confidence in public schools. Does the evidence suggest at the \(\alpha\) = 0.05 significance level that the +proportion of adults aged 18 years or older having “a great deal” of +confidence in the public schools is lower in 2005 than in 1995?
+\(p=\) percentage of adults having +“a great deal” of confidence in the public schools.
+Let \(X\sim Bin(n,p)\) and let \(\hat p={X\over n}\) be the sample +proportion. (The number of adults being confidence in public school +among the sampled 1004 adults)
+-
+
- State the hypotheses +
We assume \(p_0\) be 40%
+\[ +H_0;p=p_0~~~H_0:p=p_0~~~H_0:p=p_0\\ +H_a;p>p_0~~~H_a:p<p_0~~~H_a:p\neq p_0\\ +\]
+The first is called one-sided right/upper test
+The second is called one-sided left/lower test
+The third is called two-sided test/two tail test
+Example:
+\(H_0:p=0.4\)
+\(H_a:p<0.4\)
+-
+
- Computer the test statistic +
\[ +Z_{H_0}={\hat p-p_0\over \sqrt{p_0(i-p_0)\over n}} +\]
+\(\sqrt{p_0(i-p_0)\over n}\) is the +standard error of \(\hat p\)
+Condition: \(np_0\geq 5\) and \(n(i-p_0)\geq5\)
+Example:
+\[ +\hat p={372\over 1004}=0.371\\ +Z_{H_0}={\hat p-p_0\over \sqrt{p_0(i-p_0)\over +n}}={0.371-0.4\over\sqrt{0.4(1-0.4)\over 1004} }=-1.876 +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(Z_{H_0}\dot \sim N(0,1)\).
+We want to determine if the observed value of \(Z_{H_0}\) is unusual for a \(N(0,1)\) random variable.
+If the observed value of \(Z_{H_0}\) is unusual, then we reject \(H_0\).
+
i. Rejection rule or region:
+
+ $H_a:p>p_0$
+
+ * Reject $H_0$ if $z_{H_0}\geq z_{\alpha}$
+
+ $H_a:p<p_0$
+
+ * Reject $H_0$ if $z_{H_0}\leq z_{\alpha}$
+
+ $H_a:p\neq p_0$
+
+ * Reject $H_0$ if $z_{H_0}\leq z_{\alpha/2} or z_{H_0}\geq z_{\alpha/2} $
+
+ii. p-value:
+
+ - Using a p-value conveys the strength of the evidence agianst $H_0$.
+
+ - The p-value is the probability of observing what was obserbed if $H_0$ is true.
+
+ $H_a:p>p_0$
+
+ * p-value=$P(z\geq z_{H_0})$
+
+ $H_a:p<p_0$
+
+ * p-value=$P(z\leq z_{H_0})$
+
+ $H_a:p\neq p_0$
+
+ * p-value=$P(z\geq abs(z_{H_0})+$P(z\leq -abs(z_{H_0})$
+
+ - Reject $H_0$ if p-value $\leq \alpha$
+ Example:
+-
+
- State the conclusion in the context of the problem. +
Example:
+Hypothesis test for Means
+Example: The Centers for Disease Control (CDC) reported on trends in +weight, height and body mass index from the 1960’s through 2002. The +general trend was that Americans were much heavier and slightly taller +in 2002 as compared to 1960. In 2002, the mean weight for men was +reported at 191 pounds. Suppose that an investigator hypothesizes that +weights are even higher in 2006 (i.e., that the trend continued over the +subsequent 4 years). A random sample of 45 American males was recruited +in 2006 and their weights were measured. The sample mean weight was +197.1 pounds, and the sample standard deviation was 25.6 pounds. Use +\(\alpha\) = 0.01.
+Let \(X_1,...,X_n\) be a simple +random sample form a popluation and let \(\bar +X\) and \(S^2\) denote the +sample mean and sample variance, respectively.
+-
+
- State the hypotheses +
Lecture_32
+Zheyuan Wu
+2023-04-10
+ +Hypothesis test for Means (continue from last week)
+Example: The Centers for Disease Control (CDC) reported on trends in +weight, height and body mass index from the 1960’s through 2002. The +general trend was that Americans were much heavier and slightly taller +in 2002 as compared to 1960. In 2002, the mean weight for men was +reported at 191 pounds. Suppose that an investigator hypothesizes that +weights are even higher in 2006 (i.e., that the trend continued over the +subsequent 4 years). A random sample of 45 American males was recruited +in 2006 and their weights were measured. The sample mean weight was +197.1 pounds, and the sample standard deviation was 25.6 pounds. Use +\(\alpha\) = 0.01.
+Let \(X_1,...,X_n\) be a simple +random sample form a population and let \(\bar +X\) and \(S^2\) denote the +sample mean and sample variance, respectively.
+-
+
- State the hypotheses +
\[ +H_0:\mu=\mu_0~~~H_0:\mu=\mu_0~~~H_0:\mu=\mu_0\\ +H_a:\mu>\mu_0~~~H_a:\mu<\mu_0~~~H_a:\mu\neq \mu_0\\ +\]
+Example:
+\[ +H_0:\mu=191,H_a:\mu>191 +\]
+because they already assumes that the weights are higher
+-
+
- Compute the test statistic +
\[ +T_{H_0}={\bar X-\mu_0\over S/\sqrt{n}} +\]
+\(S/\sqrt{n}\) is the standard error +of \(\bar X\).
+Condition: the population is normal or \(n\geq 30\)
+Example:
+\[ +n=45\geq 30,\bar X=197.1,\mu_0=191,S=25.6 +\]
+\[ +T_{H_0}={197.1-191\over 25.6/\sqrt{45}}=1.598 +\]
+-
+
- Reach a conclusion +
-
+
- If \(H_0\) is true, \(T_{H_0}\sim T_{n-1}\) +
- We want to determine if the observed value of \(T_{H_0}\) is unusual for a \(T_{n-1}\) random variable. +
- If the observed value of \(T_{H_0}\) is unusual (determined by \(\alpha\)), then we reject \(H_0\).
+
+
-
+
- Rejection rule or region:
+\(H_a:\mu>\mu_0\)
+
+
-
+
- Reject \(H_0\) if \(T_{H_0}\geq T_{n-1,\alpha}\)
+\(H_a:\mu<\mu_0\)
+
+ - Reject \(H_0\) if \(T_{H_0}\leq -T_{n-1,\alpha}\)
+\(H_a:\mu\neq\mu_0\)
+
+ - Reject \(H_0\) if \(T_{H_0}\geq T_{n-1,\alpha/2}~or~T_{H_0}\leq
+-T_{n-1,\alpha/2}\Leftrightarrow |T_{H_0}|\geq
+T_{n-1,\alpha/2}\)
+
+
-
+
- p-value:
+\(H_a:\mu>\mu_0\)
+
+
-
+
- p-value = \(P(T\geq
+T_{H_0})\)
+\(H_a:\mu<\mu_0\)
+
+ - p-value = \(P(T\leq
+-T_{H_0})\)
+\(H_a:\mu\neq\mu_0\)
+
+ - p-value=\(P(T\geq abs(T_{H_0}))+P(T\leq +-abs(T_{H_0}))\) +
Reject \(H_0\) if p-value\(\leq \alpha\)
+Example:
+\[ +\alpha=0.01,T_{H_0}=1.598,n=45(df=44)\\ +H_a:\mu>191 +\]
+-
+
Reject rule Since \(T_{H_0}=1.598<2.414\), do not reject +\(H_0\).
+P value fail to reject \(H_0\).
+State the conclusion in the context of the problem.
+
Example:
+We do not have sufficient evidence to conclude that the average +weight in 2006 was greater than that in 2001.
+We can use R to compute test statistics and p-values
+Example: Suppose we want to test
+\(H_0:\mu=25\)
+\(H_a:\mu\neq 25\)
+using the data:
+\[ +22.76~24.08~31.23~25.28~25.52~31.86~26.84~19.94~22.25~23.22 +\]
+x <- c(22.76, 24.08, 31.23, 25.28, 25.52, 31.86, 26.84, 19.94, 22.25, 23.22)
+t.test(x,mu=25,alternative = "two.sided")##
+## One Sample t-test
+##
+## data: x
+## t = 0.24708, df = 9, p-value = 0.8104
+## alternative hypothesis: true mean is not equal to 25
+## 95 percent confidence interval:
+## 22.56965 28.02635
+## sample estimates:
+## mean of x
+## 25.298# or "greater" or "less"Lecture_33
+Zheyuan Wu
+2023-04-12
+ +Hypothesis test for Variance
+Example: A company produces metal pipes of a standard length. Five +years ago it tested its production quality and found that the lengths of +the pipes produced were normally distributed with a standard deviation +of 1.1 cm. They want to test whether they are still meeting this level +of quality by testing a random sample of 30 pipes. They found the sample +standard deviation for the 30 pipes was 1.2 cm. Use \(\alpha\) = 0.10.
+Let \(X_1,...,X_n\) be iid normal +with variance \(\sigma^2\). Let \(S^2\) denote the sample variance. (or a +random sample of size n from normal.)
+-
+
- State the hypotheses +
\[ +H_0:\sigma^2=\sigma^2_0~~~H_0:\sigma^2=\sigma^2_0~~~H_0:\sigma^2=\sigma^2_0\\ +H_a:\sigma^2>\sigma^2_0~~~H_a:\sigma^2<\sigma^2_0~~~H_a:\sigma^2\neq +\sigma^2_0\\ +\]
+Example:
+\(\sigma^2=\) variation of pipe +length currently.
+\[ +H_0:\sigma^2=1.1^2\\ +H_a:\sigma^2> +1.1^2(quality~gets~worse)~~\sigma^2<1.1^2(quality~gets~better)\\ +H_a:\sigma^2\neq 1.1^2 +\]
+iid = same distribution and independent.
+If the \(H_0\) hypothesis is true, +we shall expect \(S^2\) should be +closer to \(\sigma^2\), and \(S^2\over\sigma^2\), shall be close to +1.
+-
+
- Compute the test statistic +
\[ +\chi^2_{H_0}={(n-1)S^2\over \sigma^2_0} +\]
+Condition the population is normal
+Example:
+\[ +\chi_{H_0}^2={(30-1)\times (1.2)^2\over (1.1)^2}=34.512 +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(\chi_{H_0}^2\sim \chi^2_{n-1}\).
+We want to determine if the observed value of \(\chi_{H_0}^2\) is unusual for a \(\chi_{n-1}^2\) random variable.
+-
+
- Rejection rule or region: +
\(H_a:\sigma^2>\sigma_0^2\)
+-
+
- Reject \(H_0\) if \(\chi_{X_0}^2\geq \chi_{n-1,\alpha}^2\) +
\(H_a:\sigma^2<\sigma_0^2\)
+-
+
- Reject \(H_0\) if \(\chi_{X_0}^2\leq \chi_{n-1,1-\alpha}^2\) +Use 1-a because the chi-square distribution is not +symmetric +
\(H_a:\sigma^2\neq \sigma^2_0\)
+-
+
- Reject \(H_0\) if \(\chi_{X_0}^2\geq \chi_{n-1,\alpha/2}^2\) or +\(\chi_{X_0}^2\leq +\chi_{n-1,1-\alpha/2}^2\) +
-
+
- p value: +
\(H_a:\sigma^2>\sigma_0^2\)
+-
+
- p-value = \(P(\chi_{n-1,\alpha}^2\geq +\chi_{H_0}^2)\) +
\(H_a:\sigma^2<\sigma_0^2\)
+-
+
- p-value = \(P(\chi_{n-1,1-\alpha}^2\leq +\chi_{H_0}^2)\) +
\(H_a:\sigma^2\neq\sigma_0^2\)
+-
+
- p-value = \(min(2\times +P(\chi_{n-1,\alpha/2}^2\geq \chi_{H_0}^2),2\times +P(\chi_{n-1,1-\alpha/2}^2\leq \chi_{H_0}^2))\) +
Reject \(H_0\) if p-value \(\leq \alpha\) (the \(\alpha\) is the significance level of the +test that controls the type I error)
+
Example:
+\[ +\alpha=0.1,\chi_{H_0}^2=34.512,n=30,df=29\\ +H_0:\sigma^2=1.21,vs.H_a\sigma^2\neq 1.21 +\]
+Method 1: reject \(H_0\) if \(\chi_{H_0}^2>\chi_{n-1,\alpha/2}^2\) or +\(\chi_{H_0}^2<\chi_{n-1,1-\alpha/2}^2\)
+\(\chi_{n-1,\alpha/2}^2\) = +17.70837
+# yes, you are right that R would calculate the left tail, but chi square test would do the right tail. lol. 0.05 is 1-0.95
+qchisq(0.05,df=29)## [1] 17.70837\(\chi_{n-1,1-\alpha/2}^2\) = +42.55697
+qchisq(0.95,df=29)## [1] 42.55697Failed to reject \(H_0\) since \(17.708<\chi_{H_0}^2=34.512<42.557\)
+Method 2: p-value=P(observing more extreme | \(H_0\))=\(2P(\chi^2\geq \chi_{H_0}^2)~or~2P(\chi^2 \leq +\chi_{H_0}^2)\)
+\[ +P(\chi^2\geq34.512)=1-pchisq(34.512,df=29)=0.221\\ +P(\chi^2\leq34.512)=1-pchisq(34.512,df=29)=0.779 +\]
+p-value \(=min\{2\times 0.221,2\times +0.779\}\) which is greater than \(\alpha\)
+-
+
- State the conclusion in the context of the problem +
Example:
+We don’t have sufficient evidence to conclude that the variance in +pipe length is different from 1.21 \(cm^2\)
+Notes:
+-
+
- To perform a hypothesis test, the investigator selects a value of +$$. +
\(\alpha\) is the probability of +Type I error
+-
+
- To control $= $ the probability of a Type II error, the sample size +can be increased. +
power of test = \(1-\beta\) = +P(reject \(H_0\) | \(H_0\) is false)
+higher power is better. (increasing \(\alpha\) would reduce \(\beta\), thus increase the power), increase +n increase the power.
+Confidence Interval and Hypothesis Tests
+Confidence intervals can be used to perform two-sides hypothesis +tests.
+Suppose we want to test the following hypotheses using significance +level \(\alpha=0.05\).
+$H_0:\mu=\mu_0$
+$H_a:\mu\neq \mu_0$-
+
A 95% confidence interval contains the plausible values of \(\mu\).
+If \(\mu_0\) is in the +confidence interval, do not reject \(H_0\).
+If \(\mu_0\) is not the +confidence interval, reject \(H_0\).
+
Example: Use a 90% confidence interval to test whether the variance +in pipe length differs from \(1.1^2\) +with \(\alpha =0.10\).
+Comparing Two Means
+In this section, we will discuss comparing means for two +populations.
+-
+
Let \(X_1,...,X_{n_1}\) be a +simple random sample from a population with mean \(\mu_1\) and variance \(\sigma_1^2\).
+Let \(X_1,..,X_{n_2}\) be a +simple random sample from a population with mean \(\mu_2\) and variance \(\sigma_2^2\).
+We will assume the two sided samples are independent.
+Our inference will be focused on \(\mu_1-\mu_2\).
+
Case 1: Equal Variances
+-
+
Assume \(\sigma_1^2=\sigma_2^2=\sigma^2\).
+A pooled estimator of the common variance \(\sigma^2\) is
+
\[ +S^2_p={(n_1-1)S^2_1+(n_2-1)S_2^2\over n_1+n_2-2} +\]
+-
+
- If both population are normal, then +
\[ +{(\bar X_1-\bar X_2)-(\mu_1-\mu_2)\over \sqrt{S_p^2({1\over n_1}+{1\over +n_2})}}\sim T_{n_1+n_2-2} +\]
+-
+
- In practice, our inference will hold if +
-
+
the populations are not normal but \(n_1\geq 30\) and \(n_2\geq 30\)
+\(S_1^2\) and \(s_2^2\) are “close enough”:
+
\[ +{max\{S_1^2,S_2^2\}\over +min\{S_1^2,S_2^2\}}<\begin{cases}5~if~n_1,n_2\approx +7\\3~if~n_1,n_2\approx15\\2~if~n_1,n_2\approx 30\\\end{cases} +\]
+Example: During the 2003 season, Major League Baseball took steps to +speed up the play of baseball games in order to maintain fan interest. +For a sample of 32 games played during the summer of 2002 and a sample +of 35 games played during the summer of 2003, the sample mean duration +of the games was computed. For games in 2002, the sample mean was 172 +minutes with a standard deviation of 10.1 minutes. For games in 2003, +the sample mean was 166 minutes with a standard deviation of 12.2 +minutes.
+Population 1:
+Population 2:
+Check assumptions:
+Lecture_34
+Zheyuan Wu
+2023-04-14
+ +Comparing Two Means
+In this section, we will discuss comparing means for two +populations.
+-
+
Let \(X_1,...,X_{n_1}\) be a +simple random sample from a population with mean \(\mu_1\) and variance \(\sigma_1^2\).
+Let \(X_1,..,X_{n_2}\) be a +simple random sample from a population with mean \(\mu_2\) and variance \(\sigma_2^2\).
+We will assume the two sided samples are independent.
+Our inference will be focused on \(\mu_1-\mu_2\).
+
Case 1: Equal Variances
+-
+
Assume \(\sigma_1^2=\sigma_2^2=\sigma^2\).
+A pooled estimator of the common variance \(\sigma^2\) is
+
\[ +\begin{aligned} +S^2_p&={(n_1-1)S^2_1+(n_2-1)S_2^2\over n_1+n_2-2}\\ +&={\sum_{i=1}^{n_1}(X_{1i}-\bar X_1)+\sum_{i=1}^{n_2}(X_{2i}-\bar +X_2)\over n_1 + n_2 -2} +\end{aligned} +\]
+a weighted average of \(S_1^2 and +S_2^2\)
+-
+
- If both population are normal, then +
\[ +{(\bar X_1-\bar X_2)-(\mu_1-\mu_2)\over \sqrt{S_p^2({1\over n_1}+{1\over +n_2})}}\sim T_{n_1+n_2-2} +\]
+\[ +\bar X_1\sim N(\mu_1,{\sigma_1^2\over n_1}),\bar X_2\sim +N(\mu_2,{\sigma_2^2\over n_2})\\ +\]
+Since \(\bar X_1\) is independent +from \(\bar X_2\)
+\[ +\bar X_1-\bar X_2\sim N(\mu_1-\mu_2,{\sigma^2_1\over +n_1}+{\sigma^2_2\over n_2}) +\]
+-
+
- In practice, our inference will hold if +
-
+
the populations are not normal but \(n_1\geq 30\) and \(n_2\geq 30\)
+\(S_1^2\) and \(s_2^2\) are “close enough”:
+
\[ +{max\{S_1^2,S_2^2\}\over +min\{S_1^2,S_2^2\}}<\begin{cases}5~if~n_1,n_2\approx +7\\3~if~n_1,n_2\approx15\\2~if~n_1,n_2\approx 30\\\end{cases} +\]
+Example: During the 2003 season, Major League Baseball took steps to +speed up the play of baseball games in order to maintain fan interest. +For a sample of 32 games played during the summer of 2002 and a sample +of 35 games played during the summer of 2003, the sample mean duration +of the games was computed. For games in 2002, the sample mean was 172 +minutes with a standard deviation of 10.1 minutes. For games in 2003, +the sample mean was 166 minutes with a standard deviation of 12.2 +minutes.
+Population 1:
+Population 2:
+Check assumptions:
+Confidence Interval for \(\mu_1-\mu_2\)
+A \((1-\alpha)100\%\) confidence +interval for \(\mu_1-\mu_2\) is
+\[ +(\bar X_1-\bar X_2)\pm t_{n_1+n_2-2,\alpha/2}\sqrt{S^2_p({1\over +n_1}+{1\over n_2})} +\]
+Example: Computer a 95% confidence interval for the difference in +average game time for 2002 and 2003
+Lecture_35
+Zheyuan Wu
+2023-04-17
+ +Comparing Two Means in R
+Suppose we have the following data from two independent samples:
+Sample 1 20.0 12.4 23.6 19.5 33.5 11.7 13.6 18.2 11.8 21.8
+Sample 2 16.8 21.7 31.5 22.1 23.8 17.1 20.0 17.1
+First, read the sample into two vectors x1 and x2:
+x1 <- c(20.0, 12.4, 23.6, 19.5, 33.5, 11.7, 13.6, 18.2, 11.8, 21.8)
+x2 <- c(16.8, 21.7, 31.5, 22.1, 23.8, 17.1, 20.0, 17.1)To test the following hypotheses
+\[ +H_0 : \mu_1 − \mu_2 = 2 \\ +H_a : \mu_1 − \mu_2 < 2 +\]
+using the equal variances test use the following R command:
+t.test(x1, x2, mu = 2, alternative = "less", var.equal = T)##
+## Two Sample t-test
+##
+## data: x1 and x2
+## t = -1.621, df = 16, p-value = 0.06227
+## alternative hypothesis: true difference in means is less than 2
+## 95 percent confidence interval:
+## -Inf 2.358326
+## sample estimates:
+## mean of x mean of y
+## 18.6100 21.2625or classical T test
+Two Sample t-test
+To test the following hypotheses
+\[ +H_0 : \mu_1 − \mu_2 = 0 \\ +H_a : \mu_1 − \mu_2 = 0 +\] using the unequal variances test use the following R +command:
+t.test(x1, x2, mu = 0, alternative = "two.sided")##
+## Welch Two Sample t-test
+##
+## data: x1 and x2
+## t = -0.9594, df = 15.873, p-value = 0.3517
+## alternative hypothesis: true difference in means is not equal to 0
+## 95 percent confidence interval:
+## -8.517329 3.212329
+## sample estimates:
+## mean of x mean of y
+## 18.6100 21.2625Note: Using alternative = "two.sided" also provides a
+confidence interval. To change the confidence level to 99%, add the
+option conf.level = 0.99.
Lecture_34
+Zheyuan Wu
+2023-04-19
+ +Paired Data
+Paired data arise from an alternative sampling design used for the +comparison of two means.
+-
+
Each data point in the first sample is matched with a unique data +point in the second sample.
+Studies with paired data can reduce uncontrolled +variability.
+
Example: A certain lake has been designated for pollution clean-up. +One way to assess the effectiveness of the clean-up measures is to +randomly select a number n of locations from which water samples are +taken and analyzed both before and after the clean-up.
+The paired data is the condition before and after +clean-up
+Example: Another way of designing the study is to select a random +sample of \(n_1\) locations from which +water samples are taken to assess the water quality before the clean-up +measures, and a different random sample of \(n_2\) locations which will serve to assess +the water quality after the clean-up measures.
+In such case we have independent sample. \(n_1\) and \(n_2\) could be different
+Example: Suppose a sample of 10 students were given a diagnostic test +before studying a particular module and then again after completing the +module. We want to find out if, in general, our teaching leads to +improvements in students knowledge/skills (i.e. test scores). The scores +are as follows:
+| Student | +Pre | +Post | +
|---|---|---|
| 1 | +18 | +22 | +
| 2 | +21 | +25 | +
| 3 | +16 | +17 | +
| 4 | +22 | +24 | +
| 5 | +19 | +16 | +
| 6 | +24 | +29 | +
| 7 | +17 | +20 | +
| 8 | +21 | +23 | +
| 9 | +23 | +19 | +
| 10 | +18 | +20 | +
Analyzing paired data involves analyzing the differences +between the observations for each pair, D.
+Notation:
+-
+
\(\mu_D=\mu_1-\mu_2=\) +population mean difference
+\(\bar D=\) sample mean +difference
+\(S_D=\) sample standard +deviation of the difference
+\(n=\) number of pairs
+
Example:
+\[ +n=10,~\bar D=1.6,~S_D=2.95 +\]
+\[ +H_0:\mu_D=0\\ +H_a:\mu_D>0 +\]
+Result:
+If the difference are normally distributed, then
+\[ +{\bar D-\mu_D\over S_D/\sqrt{n}}\sim T_{n-1} +\]
+\(S_D/\sqrt{n}\) is the standard +error of \(\bar D\)
+The calculation similar as one-sample t test, but testing the +difference.
+-
+
- Our methods will also work if the differences are not normal but +\(n\geq 30\). +
Confidence Interval
+A \((a-\alpha)100\%\) confidence +interval for \(\mu_D=\mu_1-\mu_2\) +is
+\[ +\bar D\pm t_{n-1,\alpha/2}{S_D\over \sqrt{n}} +\]
+Example: What is a 90% confidence interval for the average +improvement in test scores?
+\[ +n=10,~n-1=9 +\]
+\[ +t_{9,0.05}=qt(0.95,df=9)=1.833\\ +\]
+So a 90% confidence interval for \(\mu_0\) is
+\[ +1.6\pm 1.833\times {2.95\over \sqrt{10}}=(-0.11,3.31) +\]
+We are 90% confident that the average improvement is between -0.11 +and 3.31.
+Hypothesis Test
+-
+
- State the hypotheses +
\[ +H_0:\mu_D=\Delta_0~~~H_0:\mu_D=\Delta_0~~~H_0:\mu_D=\Delta_0\\ +H_a:\mu_D>\Delta_0~~~H_a:\mu_D<\Delta_0~~~H_0:\mu_D\neq\Delta_0 +\]
+-
+
- Compute the test statistic +
\[ +T_{H_0}={\bar D-\mu_D\over S_D/\sqrt{n}} +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(T_{H_0}\sim T_{n-1}\).
+We can find rejection rules and p-values using the same method as +our previous T test.
+
-
+
- State the conclusion in the context fo the problem. +
Example: Test whether the average improvement is greater than 1 point +using \(\alpha\) = 0.10.
+-
+
- State the hypothesis +
\[ +H_0:\mu_0=1\\ +H_a:\mu_0>1 +\]
+-
+
+
\[ +T_{H_0}={\bar D-\mu_D\over S_D/\sqrt{n}}={1.6-1\over +2.95/\sqrt{10}}=0.643 +\]
+-
+
+
-
+
Method 1: Reject \(H_0\) if +\(T_{H_0}>qt(0.9,df=9)\)
+Method 2: Reject \(H_0\) if +p-value is less than \(\alpha\) = +0.10,
+
p-value is
+\[ +1-pt(0.643,df=9)=0.2681 +\]
+-
+
+
We do not have sufficient evidence that the average improvement is +greater than 1 point.
+In R
+pre <- c(18,21,16,22,19,24,17,21,23,18)
+post <- c(22,25,17,24,16,29,20,23,19,20)
+ t.test(post, pre, mu = 1, paired = T, alternative = "greater")##
+## Paired t-test
+##
+## data: post and pre
+## t = 0.64286, df = 9, p-value = 0.2682
+## alternative hypothesis: true mean difference is greater than 1
+## 95 percent confidence interval:
+## -0.1109054 Inf
+## sample estimates:
+## mean difference
+## 1.6Comparing Two Proportions
+We can compare proportions from two populations using data from two +independent samples.
+-
+
Let \(X_1\sim Bin(n_1,p_1)\) and +let \(\hat p_1={X_1\over n_1}\), \(\sim N(p_1,{p_1(1-p_1)\over +n_1})\)
+Let \(X_2\sim Bin(n_2,p_2)\) and +let \(\hat p_2={X_2\over n_2}\), \(\sim N(p_2,{p_2(1-p_2)\over +n_12})\)
+
Result: If \(n_1\hat p_1\geq 8\), +\((1-n_1)\hat p_1\geq 8\), \(n_2\hat p_2\geq8\), \((1-n_2)\hat p_2\geq8\), then
+\[ +{(\hat p_1-\hat p_2)-(p_1-p_2)\over \sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}}}\dot \sim N(0,1) +\]
+\(\sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}}\) is the standard +error.
+Example: Time magazine reported the result of a telephone poll of 800 +adult Americans. The question posed of the Americans who were surveyed +was: “Should the federal tax on cigarettes be raised to pay for health +care reform?” Non-smokers Smokers $ n_1 = 605,~n_2 = 195 $ 351 said +“yes” 41 said “yes”
+Confidence Interval
+A $(1-) 100% $ confidence interval for \(p_1-p_2\) is
+\[ +(\hat p_1-\hat p_2)\pm z_{\alpha/2}\sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}} +\]
+Example: Find 95% confidence interval for the difference in the +proportions of non-smokers and smokers who say “yes”.
+Lecture_37
+Zheyuan Wu
+2023-04-21
+ +Comparing Two Proportions
+We can compare proportions from two populations using data from two +independent samples.
+-
+
Let \(X_1\sim Bin(n_1,p_1)\) and +let \(\hat p_1={X_1\over n_1}\), \(\sim N(p_1,{p_1(1-p_1)\over +n_1})\)
+Let \(X_2\sim Bin(n_2,p_2)\) and +let \(\hat p_2={X_2\over n_2}\), \(\sim N(p_2,{p_2(1-p_2)\over +n_12})\)
+
Result: If \(n_1\hat p_1\geq 8\), +\((1-n_1)\hat p_1\geq 8\), \(n_2\hat p_2\geq8\), \((1-n_2)\hat p_2\geq8\), then
+\[ +{(\hat p_1-\hat p_2)-(p_1-p_2)\over \sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}}}\dot \sim N(0,1) +\]
+\(\sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}}\) is the standard +error.
+Example: Time magazine reported the result of a telephone poll of 800 +adult Americans. The question posed of the Americans who were surveyed +was: “Should the federal tax on cigarettes be raised to pay for health +care reform?” Non-smokers Smokers $ n_1 = 605,~n_2 = 195 $ 351 said +“yes” 41 said “yes”
+\(\hat p_1={351\over 60.5}=0.58\), +\(\hat p_2={41\over 195}=0.21\)
+Confidence Interval
+A $(1-) 100% $ confidence interval for \(p_1-p_2\) is
+\[ +(\hat p_1-\hat p_2)\pm z_{\alpha/2}\sqrt{{\hat p_1(1-\hat p_1)\over +n_1}+{\hat p_2(1-\hat p_2)\over n_2}} +\]
+Example: Find 95% confidence interval for the difference in the +proportions of non-smokers and smokers who say “yes”.
+\[ +Z_{0.025}=1.96=qnorm(0.975)\\ +(0.58-0.21\pm 1.96\times ) +\]
+Hypothesis Test
+-
+
- State the hypotheses +
\[ +H_0:p_1-p_2=\Delta_0~~H_0:p_1-p_2=\Delta_0~~H_0:p_1-p_2=\Delta_0\\ +H_a:p_1-p_2>\Delta_0~~H_a:p_1-p_2<\Delta_0~~H_a:p_1-p_2\neq\Delta_0\\ +\]
+-
+
- Compute the test statistic +
-
+
- If \(\Delta_0\neq 0\): +
\[ +Z_{H_0}^{P_1P_2}={(\hat p_1-\hat p_2)-\Delta_0\over \sqrt{{\hat +p_1(1-\hat p_1)\over n_1}+{\hat p_2(1-\hat p_2)\over n_2}}} +\]
+-
+
- If \(\Delta_0=0\): +
\[ +Z_{H_0}^{P_1P_2}={\hat p_1-\hat p_2\over \sqrt{\hat p(1-\hat p)({1\over +n_1}+{1\over n_2})}} +\]
+where
+\[ +\hat p={n_1\hat p_1+n_2\hat p_2\over n_1+n_2} +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(Z_{H_0}^{P_1P_2}\dot \sim N(0,1)\)
+We can find rejection rules and p-values using the same method as +our previous Z test.
+
-
+
- State the conclusion in the context of the problem. +
Example: Use \(\alpha = 0.05\) to +test whether the proportion of non-smokers who say “yes” is greater than +the proportion of smokers.
+\[ +H_0:p_1=p_2,H_a:p_1>p_2 +\]
+\[ +\hat p={351+41\over 605+195}=0.49\\ +\]
+\[ +Z_{H_0}^P={(0.58-0.21)-0\over\sqrt{0.49\times(1-0.49)\times({1\over605 +}+{1\over 195})}}=8.988 +\]
+Method 1. Reject rule: if \(Z_{H_0}^p>\alpha\) reject \(H_0\)
+Method 2. p-value: \(P(Z>8.988)=1-pnrom(8.988)\approx0\), +reject \(H_0\)
+We have sufficient evidence that the proportion of the non smokers +who say yes is greater than that of smokers.
+In R
+The prop.test function in R will perform the hypothesis test in the +case of \(\Delta_0 = 0\).
+x <- c(351,41)
+n <- c(605,195)
+prop.test(x, n, alternative = "greater", correct = F)##
+## 2-sample test for equality of proportions without continuity correction
+##
+## data: x out of n
+## X-squared = 80.746, df = 1, p-value < 2.2e-16
+## alternative hypothesis: greater
+## 95 percent confidence interval:
+## 0.3116585 1.0000000
+## sample estimates:
+## prop 1 prop 2
+## 0.5801653 0.2102564prop.test(x, n, alternative = "two.sided", correct = F)##
+## 2-sample test for equality of proportions without continuity correction
+##
+## data: x out of n
+## X-squared = 80.746, df = 1, p-value < 2.2e-16
+## alternative hypothesis: two.sided
+## 95 percent confidence interval:
+## 0.3004992 0.4393185
+## sample estimates:
+## prop 1 prop 2
+## 0.5801653 0.2102564In practice, use continuity correction.
+In large n, the result of correction and non corrections are +similar
+Analysis of Variance (ANOVA)
+-
+
Previously, we’ve learned how to do a hypothesis test comparing +two means.
+Now, we would like to compare means across more than two +populations.
+Why don’t we just do multiple pairwise comparisons?
+-
+
We will lose control of type I error (\(\alpha\))
+very likely to identify a false significance by chance.
+
+ANOVA uses a single hypothesis test to check whether the means +across several populations are equal.
+
\[ +H0 : \mu_1 = \mu_2 = · · · = \mu_k\\ +Ha: at~least~one mean~is~different +\]
+Example: A psychologist predicts that students will learn most +effectively with a constant background sound, as opposed to an +unpredictable sound or no sound at all. She randomly divides fifteen +students into three groups of five. All students study a passage of text +for 30 minutes. Those in group 1 study with background sound at a +constant volume in the background. Those in group 2 study with noise +that changes volume periodically and randomly. Those in group 3 study +with no sound at all. After studying, all students take a 10 point +multiple choice test over the material. Their scores are in the table +below.
+| + | Constant Sound | +Random Sound | +No Sound | +
|---|---|---|---|
| + | 7 | +5 | +2 | +
| + | 4 | +5 | +4 | +
| + | 6 | +3 | +6 | +
| + | 8 | +1 | +1 | +
| + | 9 | +4 | +2 | +
| Sample Mean | +6.8 | +3.6 | +3.0 | +
| Sample Std. Dev. | +1.9 | +1.7 | +2.0 | +
Idea behind ANOVA:
+-
+
If H0 is true (\(\mu_1 = \mu_2 = +\mu_3\)), the variability between the sample means should be +small. (sample means shall be similar with each other)
+If \(H_0\) is false, the +variability between the sample means should be large.
+The variability between the sample means is called the +variability between groups.
+In order to determine if the variability between groups is large +or small, we need to compare it to the variability within each +group.
+This is why this method is called Analysis of Variance.
+
When \(H_0:\mu_1 = \mu_2 = \mu_3\) +is false, the between groups variability will be much greater than the +with in groups variability.
+When \(H_0:\mu_1 = \mu_2 = \mu_3\) +is true, the between groups variability will be about the same as the +within groups variability.
+The Details
+Suppose we have independent random samples form k populations:
+\[ +\begin {matrix} +X_{11},X_{12},...,X_{1n_1}\\ +X_{21},X_{22},...,X_{2n_2}\\ +...\\ +X_{k1},X_{k2},...,X_{kn_k}\\ +\end{matrix} +\]
+Notation
+-
+
\(\bar X_i\) is the sample mean +for the ith random sample.
+\(S_i^2\) is the sample variance +for the ith random sample.
+\(\bar X\) is the overall sample +mean. (also called ground mean)
+\(N=n_1+...+n_k\) is the overall +sample size.
+
Examples (sound):
+\[ +N=5+5+5=15\\ +\bar X=4.47\\ +n_1=5,\bar X_1=6.8,S_1=1.9\\ +n_2=..,\bar X_2=..,S_2=..\\ +n_3=..,\bar X_3=..,S_3=..\\ +\]
+Variability between groups:
+\[ +SSTr=\sum_{i=1}^kn_i(\bar X_i-\bar X)^2 +\]
+\[ +MSTr={SSTr\over k-1} +\]
+-
+
SSTr is called the treatment sum of squares.
+MSTr is called the mean squares for treatment.
+
Example: What are the SSTr and MSTr for the sound example?
+\[ +SSTr=5(6.8-4.47)^2+5(3.6-4.47)^2+5(3.0-4.47)^2=41.73\\ +MSTr={41.3\over3-1}=20.87 +\]
+Variability within groups:
+\[ +SSE=\sum _{i=1}^k\sum_{j=1}^{n-i}(X_{ij}-\bar X_{i})^2=\sum +_{i=1}^k(n_i-1)S_i^2 +\]
+\[ +MSE={SSE\over N-k} +\]
+-
+
SSE is called the error sum of squares.
+MSE is called the mean squares for the error.
+
Example: What are SSE and MSE for the sound example?
+\[ +SSE=4\times 1.9^2+4\times 1.7^2+4\times 2.0^2=42.0\\ +MSE={42.0\over 15-3}=3.5 +\]
+Test statistic
+The test statistic \(F_{H_0}\) is +the ratio of the MSTr and MSE:
+\[ +F_{H_0}={MSTr\over MSE} +\]
+-
+
\(H_0\) is true: when \(F_{H_0}\) is small
+\(H_a\) is true: when \(F_{H_0}\) is large
+If \(H_0\) is true, \(F_{H_0}\) has an F distribution.
+– The F distribution is skewed to the right. – The F distribution is +frequently used when the test statistic is a ratio. – The shape is +determined by two degrees of freedom: ν1 and ν2.
+The degrees of freedom:
+
– \(ν1 = DF_{SST r} = k − 1\)
+– \(ν2 = DF_{SSE} = N − k\)
+Example: What is the test statistic and what are the degrees of +freedom for the sound example?
+\[ +F={20.87\over 3.5}=5.96\\ +v_1=2,v_2=12 +\]
+Conclusion
+-
+
- Rejection Rule: Reject \(H_0\) if +\(F-{H_0} \geq +F_{k−1,N−k,\alpha}\) +
In R: qf(1-a, ν1, ν2) gives \(F_{ν1,ν2,α}\).
-
+
- p-value: p-value = \(P(F_{k−1,N−k} > +F_{H_0})\) +
In R: pf(x, ν1, ν2) gives \(P(F_{ν1,ν2} \leq x)\).
Example: What is the conclusion for the sound example? Use \(\alpha = 0.05\).
+Method 1: Rejection rule
+qf(0.95,2,12)## [1] 3.885294which is less than 5.96, reject \(H_0\)
+Method 2: p-value
+pf(5.96,2,12)## [1] 0.9840588Lecture_38
+Zheyuan Wu
+2023-04-24
+ +In R: pf(x, v_1, v_2) gives \(P(F_{v_1,v_2}\leq x)\).
Example: What is the conclusion for the sound example? Use \(\alpha = 0.05\).
+Summary:
+-
+
We can summarize our calculations in an ANOVA table.
+Some final notation:
+
\[ +SST = SSTr + SSE\\ +DF_{SST} = DF_{SST r} + DF_{SSE} = N − 1 +\]
+-
+
\(SST\) is called the total sum +of squares. (\(SSTr\) is the +variability between groups, the \(SSE\) +is variability within group)
+The ANOVA table partitions the total sum of squares into two +parts: the error and the treatment.
+
| Source of Variation | +Sum of Squares | +Degrees of Freedom | +Mean Square | +F | +p-value | +
|---|---|---|---|---|---|
| Treatment | +\(SST_r\) | +\(k-1\) | +\(MSTr\) | +\(F_{H_0}\) | ++ |
| Error | +\(SSE\) | +\(N − k\) | +\(MSE\) | ++ | + |
| Total | +\(SST\) | +\(N − 1\) | ++ | + | + |
-
+
- \(F_{H_0}\) is the test statistic +in ANOVA test. +
Example: The ANOVA table for the sound data is:
+| Source of Variation | +Sum of Squares | +Degree of Freedom | +Mean Square | +F | +p-value | +
|---|---|---|---|---|---|
| Treatment | +41.73 | +2 | +20.87 | +5.96 | +0.0159 | +
| Error | +42 | +15-3=12 | +3.5 | ++ | + |
| Total | +83.73 | +15-1=14 | ++ | + | + |
Assumptions:
+-
+
The k samples are independent.
+The variances of the k populations are equal.
+The k populations are normally distributed (or$ n_i $ for all +i)
+
In R:
+score <- c(7,4,6,8,9,5,5,3,1,4,2,4,6,1,2)
+group <- c(rep(1,5), rep(2,5), rep(3,5))
+group## [1] 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3# aov is analysis of variance
+fit <- aov(score ~ as.factor(group))
+anova(fit)## Analysis of Variance Table
+##
+## Response: score
+## Df Sum Sq Mean Sq F value Pr(>F)
+## as.factor(group) 2 41.733 20.867 5.9619 0.01593 *
+## Residuals 12 42.000 3.500
+## ---
+## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# The as.factor is treatment, residuals is errorExample: Researchers recruited overweight subject’s and randomly +assigned them to one of four popular diet plans: Atkins, Ornish, Weight +Watchers, and Zone. Each subject’s weight loss was measured after one +year.
+| Diet | +Sample Mean weight Loss (in pounds) | +n | +
|---|---|---|
| Atkins | +3.92 | +21 | +
| Ornish | +6.56 | +20 | +
| Weight Watchers | +4.59 | +26 | +
| Zone | +4.88 | +26 | +
-
+
- Fill in the empty cells of the ANOVA table. +
| Source of Variation | +Sum of Squares | +Degrees of Freedom | +Mean Square | +F | +p-value | +
|---|---|---|---|---|---|
| Treatment | +78 | +4-1=3 | +78/3=26 | +0.539 | ++ |
| Error | +4294 | +93-4=89 | +4294/89=48.25 | ++ | + |
| Total | +4372 | +93-1=92 | ++ | + | + |
-
+
- Tests whether the average weight losses for the four diets are +different. Use \(\alpha = 0.10\). +
Let \(\mu\) denote the average +weight losses for the ith diet, \(i=1,2,3,4\)
+\(H_0:\mu_1=\mu_2=\mu_3=\mu_4\)
+\(H_a:\) at lease one of them is +different
+\[ +F={26\over48.25}=0.539 +\]
+the p-value is
+1-pf(0.539,3,89)## [1] 0.6568001We failed to reject \(H_0\) because +we don’t have sufficient evidence of a difference in average weight loss +for four is different.
+We can only make type I error in the ANOVA test, type I error +is our assumption is true, but we reject it, type II error is that our +assumption is false, but we failed to reject it.
+Simple Linear Regression
+Simple linear regression allows us to investigate the relationship +between two variables Y and X.
+-
+
Y = the response, outcome, or dependent variable
+X = the explanatory, predictor, or independent variable
+
Goal: To describe how \(E(Y | X = +x)\) varies as a function of x.
+Example: The data set heights.txt (on Canvas) contains the heights +(in inches) of 1375 mother/daughter pairs.
+heights<-read.csv("./heights.txt",sep=" ")
+head(heights)## Mheight Dheight
+## 1 59.7 55.1
+## 2 58.2 56.5
+## 3 60.6 56.0
+## 4 60.7 56.8
+## 5 61.8 56.0
+## 6 55.5 57.9X is the daughter’s height, Y is the mother’s height.
+Recall: We already have some tools for looking at relationships +between two variables.
+-
+
Scatter plots
+
+plot(heights$Mheight, heights$Dheight, xlab="Mother’s Height", ylab="Daughter’s Height")
+Covariance =$ Cov(X, Y )$ and sample covariance = \(S_{X,Y}\)
+
+cov(heights$Mheight, heights$Dheight)## [1] 3.004806
+Correlation = Corr(X, Y ) and sample correlation = rX,Y
+
+cor(heights$Mheight, heights$Dheight)## [1] 0.4907094
+
Regression Model
+The simple linear regression model is
+\[ +Y = \alpha_1 + \beta_1 X + \epsilon +\] The error variable ε:
+-
+
has a N(0, σ2 ε ) distribution
+\(Cov(X, \epsilon) = +0\)
+\(\sigma_\epsilon^2\) is the +same for all values of x.
+
Results:
+-
+
- \(E(Y | X = x) = \alpha_1 + \beta_1 +x\) +
\[ +E(Y|X=x)=E(\alpha_1+\beta_1 x +\epsilon|x)=\alpha+\beta +E(X|x)+E(\epsilon|x) +\]
+\(E(\epsilon|x)=0\)
+-
+
- The distribution of \(Y | X = x\) +is \(N(\alpha_1 + \beta_1 x, σ^2_\epsilon +)\). +
The Method of Least Squares
+We will use least squares to estimate the parameters of the linear +regression model. Let \((X_1, Y_1), . . . +,(X_n, Y_n)\) be a random bivariate sample from the +population.
+Make the distance of observed point and prediction line is as small +as possible
+\[ +SSE = \sum_{i=1}^n(y_i-\hat y_i)^2=\sum_{i=1}^n(y_i-(\hat +\alpha_1+\hat\beta_1x_i))^2=\sum_{i=1}^n \hat \epsilon_i^2 +\]
+These vertical distances are the estimated errors and are called +residuals:
+\[ +\hat \epsilon _i=Y_i-\hat Y_i +\]
+Minimizing the SSE provides the estimates:
+\[ +\hat \beta_1={S_{X,Y}\over S_X^2},\hat\alpha_1=\bar Y-\hat\beta_1\bar X +\]
+To ensure the fitted line passed the overall mean point \((\bar X,\bar Y)\)
+derivative is required for computing these values by hand… +(minimizing the value means the derivative is zero)
+We can also use the SSE to estimate \(\sigma_\epsilon^2\):
+\[ +S_\epsilon^2={SSE\over n-2} +\]
+\(n-2\) is the df of residual.
+We can obtain these estimates using R.
+model <- lm(Dheight~Mheight, data = heights)
+model$coefficients## (Intercept) Mheight
+## 29.917437 0.541747model$residuals## 1 2 3 4 5 6
+## -7.159733372 -4.947112855 -6.747305683 -6.001480384 -7.397402096 -2.084395924
+## 7 8 9 10 11 12
+## -2.830221223 -3.088667039 -3.867889947 -3.859540545 -4.009636959 -5.159733372
+## 13 14 15 16 17 18
+## -4.180510464 -5.138956280 -5.430606878 -4.684781579 -5.551576798 -5.868275602
+## 19 20 21 22 23 24
+## -6.847498510 -7.314293728 -2.688667039 -2.042841740 -3.284588752 -2.159540545
+## 25 26 27 28 29 30
+## -3.772161062 -3.555462258 -3.443034568 -3.143034568 -3.080510464 -3.959733372
+## 31 32 33 34 35 36
+## -3.451383970 -3.738956280 -4.484781579 -6.981088947 -1.197016441 -2.030414051
+## 37 38 39 40 41 42
+## -1.330414051 -0.897016441 -2.363811660 -2.255462258 -1.938763453 -2.317986361
+## 43 44 45 46 47 48
+## -2.617986361 -1.592938154 -2.559733372 -2.280510464 -2.988859867 -2.143034568
+## 49 50 51 52 53 54
+## -3.213908074 -2.759733372 -2.505558671 -3.043034568 -2.343034568 -2.697209269
+## 55 56 57 58 59 60
+## -2.713908074 -3.013908074 -3.638956280 -3.909829786 -2.830606878 -2.793130982
+## 61 62 63 64 65 66
+## -3.422257476 -3.447305683 -3.501480384 -3.855655085 -3.122257476 -3.347305683
+## 67 68 69 70 71 72
+## -3.276432177 -3.726528591 -3.851576798 -3.918179188 -3.364004487 -3.889052694
+## 73 74 75 76 77 78
+## -3.889052694 -3.434877993 -3.726528591 -4.026528591 -4.276625004 -4.184974407
+## 79 80 81 82 83 84
+## -3.705751499 -5.101673211 -4.364197315 -5.389245522 -6.451962452 0.036381168
+## 85 86 87 88 89 90
+## 0.290555869 -0.088667039 -1.030414051 -0.384588752 -0.576239349 -0.305365844
+## 91 92 93 94 95 96
+## -0.651191143 -0.955462258 -1.326335763 -1.117986361 -0.838763453 -1.472161062
+## 97 98 99 100 101 102
+## -0.655462258 -0.938763453 -1.226335763 -1.201287556 -1.680510464 -1.305558671
+## 103 104 105 106 107 108
+## -2.313908074 -1.597209269 -1.188859867 -2.005558671 -1.343034568 -1.197209269
+## 109 110 111 112 113 114
+## -2.213908074 -2.368082775 -1.613908074 -1.288859867 -1.988859867 -1.959733372
+## 115 116 117 118 119 120
+## -1.934685166 -1.893130982 -2.030606878 -2.130606878 -1.522257476 -2.484781579
+## 121 122 123 124 125 126
+## -1.793130982 -1.738956280 -1.847305683 -2.801480384 -1.630606878 -2.247305683
+## 127 128 129 130 131 132
+## -2.030606878 -2.909829786 -2.701480384 -2.222257476 -2.322257476 -2.655655085
+## 133 134 135 136 137 138
+## -2.176432177 -2.209829786 -2.409829786 -2.772353890 -3.197402096 -2.372353890
+## 139 140 141 142 143 144
+## -3.451576798 -3.189052694 -2.897402096 -2.489052694 -2.764004487 -2.934877993
+## 145 146 147 148 149 150
+## -2.880703292 -2.551576798 -2.751576798 -2.918179188 -3.397402096 -3.297402096
+## 151 152 153 154 155 156
+## -3.189052694 -3.414100901 -3.122450303 -3.614100901 -2.659926200 -2.868275602
+## 157 158 159 160 161 162
+## -3.622450303 -2.759926200 -3.259926200 -3.839149108 -3.314100901 -2.859926200
+## 163 164 165 166 167 168
+## -2.914100901 -2.805751499 -3.584974407 -3.776625004 -3.314100901 -2.930799706
+## 169 170 171 172 173 174
+## -3.418372016 -4.210022614 -3.880896119 -3.872546717 -4.426721418 -4.055847912
+## 175 176 177 178 179 180
+## -3.618372016 -3.847498510 -4.110022614 -3.955847912 -3.347498510 -3.247498510
+## 181 182 183 184 185 186
+## -3.772546717 -3.755847912 -3.935070820 -3.635070820 -3.847498510 -3.635070820
+## 187 188 189 190 191 192
+## -4.343420223 -4.176817832 -4.522643131 -4.893516636 -5.318564843 -5.735263648
+## 193 194 195 196 197 198
+## -5.543613050 -6.535456475 0.619682363 0.632110053 0.440459455 -0.063811660
+## 199 200 201 202 203 204
+## 0.352887145 -0.092938154 0.082013639 -0.772161062 -0.726335763 -0.026335763
+## 205 206 207 208 209 210
+## -0.563811660 -0.668082775 -0.588859867 -0.513908074 -0.705558671 -0.680510464
+## 211 212 213 214 215 216
+## -1.368082775 0.019489536 -1.213908074 -0.568082775 -0.868082775 -0.080510464
+## 217 218 219 220 221 222
+## -0.751383970 -0.659733372 -0.651383970 -0.943034568 -0.888859867 -1.043034568
+## 223 224 225 226 227 228
+## -0.488859867 -1.022257476 -0.793130982 -1.201480384 -0.893130982 -1.476432177
+## 229 230 231 232 233 234
+## -1.230606878 -1.855655085 -1.293130982 -1.709829786 -0.793130982 -1.409829786
+## 235 236 237 238 239 240
+## -1.355655085 -1.909829786 -1.101480384 -1.493130982 -1.155655085 -1.530606878
+## 241 242 243 244 245 246
+## -1.655655085 -2.180703292 -1.851576798 -1.543227395 -1.497402096 -1.843227395
+## 247 248 249 250 251 252
+## -1.726528591 -1.164004487 -2.397402096 -2.243227395 -1.672353890 -1.589052694
+## 253 254 255 256 257 258
+## -1.851576798 -1.651576798 -1.526528591 -1.789052694 -2.034877993 -1.897402096
+## 259 260 261 262 263 264
+## -2.297402096 -1.564004487 -1.589052694 -1.526528591 -1.480703292 -1.626528591
+## 265 266 267 268 269 270
+## -1.118179188 -1.543227395 -1.918179188 -1.626528591 -1.772353890 -2.126528591
+## 271 272 273 274 275 276
+## -2.351576798 -1.543227395 -1.864004487 -1.918179188 -1.380703292 -2.122450303
+## 277 278 279 280 281 282
+## -2.839149108 -1.976625004 -2.459926200 -2.439149108 -2.139149108 -1.859926200
+## 283 284 285 286 287 288
+## -2.039149108 -2.059926200 -2.476625004 -2.139149108 -2.539149108 -2.668275602
+## 289 290 291 292 293 294
+## -2.368275602 -2.384974407 -2.184974407 -2.093323809 -2.322450303 -2.630799706
+## 295 296 297 298 299 300
+## -2.393323809 -1.976625004 -2.622450303 -2.059926200 -1.714100901 -2.622450303
+## 301 302 303 304 305 306
+## -2.693323809 -1.759926200 -3.210022614 -3.335070820 -2.735070820 -2.935070820
+## 307 308 309 310 311 312
+## -3.535070820 -2.672546717 -3.335070820 -2.618372016 -2.472546717 -3.326721418
+## 313 314 315 316 317 318
+## -3.155847912 -3.172546717 -2.864197315 -2.735070820 -2.526721418 -3.776817832
+## 319 320 321 322 323 324
+## -3.160119027 -2.889245522 -2.843420223 -3.343420223 -3.122643131 -3.351769625
+## 325 326 327 328 329 330
+## -3.243420223 -3.460119027 -3.864390142 -4.510215441 -3.939341935 -3.585167234
+## 331 332 333 334 335 336
+## -4.185167234 -4.218564843 -4.143613050 -4.097787751 -4.597787751 1.186284754
+## 337 338 339 340 341 342
+## 1.194634156 1.223760651 1.315411248 0.998712444 0.390363041 0.927838938
+## 343 344 345 346 347 348
+## 0.807061846 0.427838938 0.790363041 0.282013639 0.973664237 0.690363041
+## 349 350 351 352 353 354
+## 0.573664237 1.252887145 0.282013639 -0.268082775 0.486091926 -0.059733372
+## 355 356 357 358 359 360
+## 0.002790731 0.611140133 0.156965432 0.619489536 0.611140133 -0.059733372
+## 361 362 363 364 365 366
+## 0.048616030 0.194441329 0.219489536 0.202790731 0.348616030 0.156965432
+## 367 368 369 370 371 372
+## 0.419489536 0.586091926 0.086091926 0.602790731 0.286091926 0.015218421
+## 373 374 375 376 377 378
+## -0.093130982 -0.147305683 0.023567823 -0.130606878 0.223567823 -0.530606878
+## 379 380 381 382 383 384
+## -0.476432177 -0.801480384 -0.484781579 -0.009829786 -0.801480384 -0.555655085
+## 385 386 387 388 389 390
+## 0.006869018 -0.176432177 -0.230606878 -0.038956280 0.052694317 -0.538956280
+## 391 392 393 394 395 396
+## 0.161043720 -0.176432177 0.377742524 -0.393130982 -0.184781579 -0.047305683
+## 397 398 399 400 401 402
+## 0.069393122 0.006869018 0.061043720 -0.701480384 -0.347305683 0.023567823
+## 403 404 405 406 407 408
+## 0.177742524 0.477742524 -1.343227395 -1.397402096 -0.572353890 -0.489052694
+## 409 410 411 412 413 414
+## -0.834877993 -0.618179188 -0.951576798 -0.472353890 -0.643227395 -1.089052694
+## 415 416 417 418 419 420
+## -0.734877993 -0.472353890 -0.780703292 -0.818179188 -0.564004487 -1.026528591
+## 421 422 423 424 425 426
+## -1.189052694 -0.964004487 -1.243227395 -1.097402096 -0.443227395 -0.626528591
+## 427 428 429 430 431 432
+## -0.980703292 -0.918179188 -0.864004487 -0.626528591 -0.889052694 -0.651576798
+## 433 434 435 436 437 438
+## -0.564004487 -0.418179188 -1.051576798 -0.718179188 -1.243227395 -1.026528591
+## 439 440 441 442 443 444
+## -0.543227395 -1.205751499 -1.130799706 -1.359926200 -1.505751499 -1.276625004
+## 445 446 447 448 449 450
+## -1.030799706 -1.214100901 -1.193323809 -1.576625004 -1.093323809 -1.614100901
+## 451 452 453 454 455 456
+## -1.339149108 -0.814100901 -1.276625004 -1.322450303 -0.868275602 -0.959926200
+## 457 458 459 460 461 462
+## -1.584974407 -1.176625004 -1.622450303 -1.122450303 -1.193323809 -1.114100901
+## 463 464 465 466 467 468
+## -1.559926200 -1.893323809 -1.130799706 -1.030799706 -1.584974407 -1.259926200
+## 469 470 471 472 473 474
+## -1.168275602 -0.759926200 -1.793323809 -1.030799706 -0.984974407 -1.284974407
+## 475 476 477 478 479 480
+## -1.955847912 -1.826721418 -1.935070820 -1.347498510 -2.210022614 -2.335070820
+## 481 482 483 484 485 486
+## -1.972546717 -1.826721418 -1.980896119 -1.701673211 -2.001673211 -2.110022614
+## 487 488 489 490 491 492
+## -1.964197315 -1.947498510 -1.418372016 -1.526721418 -2.264197315 -2.372546717
+## 493 494 495 496 497 498
+## -1.980896119 -2.118372016 -2.218372016 -1.655847912 -1.547498510 -1.818372016
+## 499 500 501 502 503 504
+## -3.076817832 -2.805944326 -2.605944326 -2.151769625 -2.168468430 -2.360119027
+## 505 506 507 508 509 510
+## -2.922643131 -2.189245522 -1.689245522 -2.668468430 -2.705944326 -2.289245522
+## 511 512 513 514 515 516
+## -2.814293728 -2.639341935 -2.785167234 -2.485167234 -3.301866039 -3.164390142
+## 517 518 519 520 521 522
+## -2.610215441 -2.556040740 -2.601866039 -3.056040740 -3.185167234 -3.426914246
+## 523 524 525 526 527 528
+## -3.481088947 -3.306137154 -3.877010659 2.665507662 2.723760651 2.277935352
+## 529 530 531 532 533 534
+## 2.902983559 2.848808857 1.661236547 2.252887145 1.636188340 2.207061846
+## 535 536 537 538 539 540
+## 2.007061846 1.498712444 1.548616030 0.956965432 1.331917225 1.419489536
+## 541 542 543 544 545 546
+## 0.948616030 1.865314834 1.556965432 1.040266628 1.165314834 1.556965432
+## 547 548 549 550 551 552
+## 1.548616030 1.411140133 0.902790731 1.619489536 1.111140133 1.802790731
+## 553 554 555 556 557 558
+## 1.511140133 1.286091926 1.006869018 0.598519616 0.144344915 1.269393122
+## 559 560 561 562 563 564
+## 1.423567823 1.169393122 0.290170214 0.706869018 0.806869018 1.069393122
+## 565 566 567 568 569 570
+## 0.977742524 0.723567823 0.998519616 0.623567823 0.298519616 0.598519616
+## 571 572 573 574 575 576
+## 1.315218421 0.915218421 0.623567823 1.044344915 0.290170214 0.861043720
+## 577 578 579 580 581 582
+## 0.544344915 1.023567823 0.869393122 1.423567823 1.223567823 0.690170214
+## 583 584 585 586 587 588
+## 0.544344915 -0.251576798 0.127646110 0.081820812 0.365122007 0.210947306
+## 589 590 591 592 593 594
+## -0.018179188 -0.026528591 0.227646110 0.119296708 0.419296708 0.410947306
+## 595 596 597 598 599 600
+## 0.081820812 -0.051576798 0.256772605 0.035995513 0.081820812 0.102597904
+## 601 602 603 604 605 606
+## 0.827646110 0.581820812 0.373471409 0.681820812 0.627646110 -0.251576798
+## 607 608 609 610 611 612
+## 0.527646110 0.448423202 -0.018179188 0.527646110 0.065122007 0.473471409
+## 613 614 615 616 617 618
+## 0.781820812 0.073471409 0.319296708 0.248423202 0.935995513 -0.134877993
+## 619 620 621 622 623 624
+## 0.273471409 0.119296708 0.002597904 0.302597904 0.148423202 0.210947306
+## 625 626 627 628 629 630
+## 0.627646110 -0.784974407 -0.230799706 -0.268275602 -0.439149108 -0.459926200
+## 631 632 633 634 635 636
+## -0.493323809 -0.159926200 0.077549697 -0.776625004 -0.230799706 0.094248501
+## 637 638 639 640 641 642
+## -0.114100901 0.094248501 -0.239149108 -0.639149108 0.094248501 -0.105751499
+## 643 644 645 646 647 648
+## 0.085899099 -0.739149108 -0.459926200 -0.505751499 -0.576625004 -0.414100901
+## 649 650 651 652 653 654
+## -0.693323809 -0.884974407 -0.276625004 -0.384974407 -0.368275602 -0.405751499
+## 655 656 657 658 659 660
+## -0.530799706 -0.084974407 -0.305751499 -0.468275602 0.094248501 -0.076625004
+## 661 662 663 664 665 666
+## -0.993323809 -0.205751499 -0.093323809 -0.005751499 -0.193323809 0.085899099
+## 667 668 669 670 671 672
+## -0.514100901 0.194248501 -0.684974407 -0.059926200 -0.839149108 -0.818372016
+## 673 674 675 676 677 678
+## -1.010022614 -1.118372016 -0.980896119 -0.555847912 -0.664197315 -0.772546717
+## 679 680 681 682 683 684
+## -0.972546717 -1.126721418 -0.455847912 -0.872546717 -0.526721418 -1.126721418
+## 685 686 687 688 689 690
+## -1.047498510 -1.380896119 -0.310022614 -0.735070820 -0.872546717 -0.501673211
+## 691 692 693 694 695 696
+## -1.235070820 -0.980896119 -1.080896119 -0.655847912 -0.818372016 -1.172546717
+## 697 698 699 700 701 702
+## -1.110022614 -1.226721418 -0.401673211 -0.801673211 -1.326721418 -0.664197315
+## 703 704 705 706 707 708
+## -1.135070820 -1.110022614 -0.764197315 -1.543420223 -1.389245522 -1.422643131
+## 709 710 711 712 713 714
+## -1.768468430 -1.597594924 -1.560119027 -1.168468430 -1.560119027 -1.368468430
+## 715 716 717 718 719 720
+## -0.789245522 -1.576817832 -1.643420223 -1.976817832 -1.660119027 -1.189245522
+## 721 722 723 724 725 726
+## -1.543420223 -1.051769625 -1.343420223 -2.001866039 -2.130992533 -1.939341935
+## 727 728 729 730 731 732
+## -2.001866039 -1.330992533 -1.793516636 -2.056040740 -1.839341935 -1.530992533
+## 733 734 735 736 737 738
+## -1.764390142 -2.351962452 -2.335263648 -2.589438349 -2.335263648 3.177935352
+## 739 740 741 742 743 744
+## 3.940459455 3.007061846 2.598712444 2.873664237 3.182013639 2.356965432
+## 745 746 747 748 749 750
+## 2.548616030 2.486091926 2.565314834 2.056965432 2.640266628 1.498519616
+## 751 752 753 754 755 756
+## 1.298519616 1.490170214 1.398519616 1.561043720 2.223567823 2.069393122
+## 757 758 759 760 761 762
+## 2.115218421 1.598519616 1.690170214 1.461043720 1.898519616 1.606869018
+## 763 764 765 766 767 768
+## 1.952694317 2.377742524 1.915218421 1.361043720 2.077742524 1.290170214
+## 769 770 771 772 773 774
+## 2.006869018 1.498519616 1.915218421 1.998519616 2.206869018 1.110947306
+## 775 776 777 778 779 780
+## 1.335995513 0.956772605 1.610947306 0.810947306 1.356772605 0.710947306
+## 781 782 783 784 785 786
+## 1.610947306 1.202597904 1.002597904 1.573471409 0.656772605 1.727646110
+## 787 788 789 790 791 792
+## 1.835995513 1.502597904 1.156772605 1.227646110 1.465122007 1.835995513
+## 793 794 795 796 797 798
+## 1.619296708 1.735995513 1.456772605 1.335995513 1.665122007 1.573471409
+## 799 800 801 802 803 804
+## 1.235995513 1.381820812 1.481820812 1.202597904 1.419296708 0.965122007
+## 805 806 807 808 809 810
+## 0.548423202 0.869200294 0.485899099 0.306676191 0.206676191 0.469200294
+## 811 812 813 814 815 816
+## 0.331724398 0.106676191 0.660850892 0.269200294 0.785899099 0.260850892
+## 817 818 819 820 821 822
+## 0.640073800 0.169200294 1.240073800 0.469200294 0.785899099 0.785899099
+## 823 824 825 826 827 828
+## 0.715025593 0.060850892 0.506676191 0.694248501 0.831724398 0.469200294
+## 829 830 831 832 833 834
+## 0.706676191 0.706676191 0.223374996 0.640073800 0.685899099 0.769200294
+## 835 836 837 838 839 840
+## 0.423374996 1.177549697 0.060850892 0.915025593 0.460850892 0.731724398
+## 841 842 843 844 845 846
+## 0.823374996 0.623374996 0.740073800 -0.155847912 0.435802685 -0.164197315
+## 847 848 849 850 851 852
+## 0.219103881 -0.235070820 0.089977386 0.381627984 0.652501490 0.364929180
+## 853 854 855 856 857 858
+## 0.164929180 -0.280896119 0.244152088 -0.380896119 0.364929180 0.389977386
+## 859 860 861 862 863 864
+## -0.235070820 0.544152088 -0.072546717 -0.272546717 0.635802685 -0.026721418
+## 865 866 867 868 869 870
+## -0.047498510 0.581627984 -0.164197315 0.698326789 -0.101673211 0.444152088
+## 871 872 873 874 875 876
+## -0.080896119 0.344152088 0.289977386 0.373278582 -0.318372016 -0.126721418
+## 877 878 879 880 881 882
+## -0.235070820 0.473278582 0.581627984 -0.264197315 0.481627984 0.264929180
+## 883 884 885 886 887 888
+## 0.198326789 -0.260119027 -0.543420223 -0.543420223 -0.460119027 -0.868468430
+## 889 890 891 892 893 894
+## -0.314293728 -0.622643131 -0.868468430 -0.389245522 -0.543420223 -0.143420223
+## 895 896 897 898 899 900
+## -0.343420223 0.202405076 -0.051769625 -0.143420223 -0.822643131 -0.868468430
+## 901 902 903 904 905 906
+## -0.576817832 -0.822643131 -0.522643131 -0.697594924 -0.489245522 -0.822643131
+## 907 908 909 910 911 912
+## -0.376817832 -0.589245522 -0.760119027 -0.651769625 -0.089245522 -1.318564843
+## 913 914 915 916 917 918
+## -0.393516636 -1.410215441 -1.156040740 -0.930992533 -0.664390142 -0.710215441
+## 919 920 921 922 923 924
+## -1.001866039 -0.801866039 -1.301866039 -1.264390142 -0.493516636 -0.539341935
+## 925 926 927 928 929 930
+## -0.701866039 -0.847691338 -1.056040740 -1.085167234 -0.901866039 -1.147691338
+## 931 932 933 934 935 936
+## -1.251962452 -1.672739544 -1.535263648 -1.589438349 -1.931185360 -1.947884165
+## 937 938 939 940 941 942
+## 4.052887145 3.394441329 3.756965432 3.048616030 3.811140133 3.519489536
+## 943 944 945 946 947 948
+## 3.219489536 3.477742524 2.661043720 2.398519616 3.369393122 2.244344915
+## 949 950 951 952 953 954
+## 2.623567823 3.261043720 3.315218421 2.861043720 3.006869018 1.756772605
+## 955 956 957 958 959 960
+## 1.927646110 2.019296708 1.848423202 2.165122007 1.848423202 2.219296708
+## 961 962 963 964 965 966
+## 2.327646110 1.973471409 1.819296708 2.502597904 2.510947306 2.181820812
+## 967 968 969 970 971 972
+## 1.765122007 2.781820812 2.935995513 2.235995513 1.648423202 1.160850892
+## 973 974 975 976 977 978
+## 1.523374996 1.223374996 1.306676191 1.515025593 1.423374996 2.015025593
+## 979 980 981 982 983 984
+## 1.931724398 2.031724398 2.131724398 1.706676191 1.460850892 1.385899099
+## 985 986 987 988 989 990
+## 1.915025593 1.623374996 1.985899099 1.523374996 2.240073800 2.031724398
+## 991 992 993 994 995 996
+## 2.240073800 1.677549697 2.085899099 2.231724398 1.615025593 1.160850892
+## 997 998 999 1000 1001 1002
+## 1.694248501 1.606676191 1.631724398 1.698326789 0.619103881 1.064929180
+## 1003 1004 1005 1006 1007 1008
+## 1.489977386 1.135802685 1.535802685 0.619103881 1.435802685 0.973278582
+## 1009 1010 1011 1012 1013 1014
+## 1.435802685 1.164929180 1.244152088 0.973278582 1.419103881 0.764929180
+## 1015 1016 1017 1018 1019 1020
+## 0.989977386 0.664929180 1.281627984 0.981627984 1.198326789 1.498326789
+## 1021 1022 1023 1024 1025 1026
+## 1.019103881 1.027453283 1.144152088 1.281627984 1.089977386 0.464929180
+## 1027 1028 1029 1030 1031 1032
+## 1.064929180 1.035802685 1.381627984 0.889977386 1.044152088 0.973278582
+## 1033 1034 1035 1036 1037 1038
+## 1.152501490 1.152501490 0.656579777 0.885706272 0.931531570 0.410754478
+## 1039 1040 1041 1042 1043 1044
+## 1.102405076 0.885706272 0.685706272 0.823182168 0.885706272 0.277356869
+## 1045 1046 1047 1048 1049 1050
+## 0.556579777 1.156579777 0.431531570 0.985706272 0.723182168 0.731531570
+## 1051 1052 1053 1054 1055 1056
+## 0.477356869 0.331531570 0.277356869 1.094055674 0.477356869 0.456579777
+## 1057 1058 1059 1060 1061 1062
+## 0.948230375 0.523182168 1.110754478 0.748230375 0.443959260 0.360658065
+## 1063 1064 1065 1066 1067 1068
+## 0.260658065 -0.264390142 -0.064390142 0.098133961 -0.210215441 -0.110215441
+## 1069 1070 1071 1072 1073 1074
+## -0.239341935 -0.410215441 0.014832766 -0.356040740 0.043959260 0.443959260
+## 1075 1076 1077 1078 1079 1080
+## 0.469007467 0.098133961 -0.681088947 -0.560311855 -0.372739544 -0.672739544
+## 1081 1082 1083 1084 1085 1086
+## -0.272739544 -0.681088947 -0.489438349 -0.906137154 -0.560311855 -0.635263648
+## 1087 1088 1089 1090 1091 1092
+## -0.097787751 0.064736352 -1.393709464 -0.731185360 -1.402058866 5.236188340
+## 1093 1094 1095 1096 1097 1098
+## 4.256965432 4.819489536 4.586091926 4.215218421 3.990170214 4.323567823
+## 1099 1100 1101 1102 1103 1104
+## 3.915218421 3.961043720 3.198519616 3.190170214 3.290170214 4.044344915
+## 1105 1106 1107 1108 1109 1110
+## 3.552694317 3.027646110 3.835995513 3.335995513 3.102597904 3.119296708
+## 1111 1112 1113 1114 1115 1116
+## 2.756772605 3.402597904 2.710947306 2.865122007 3.435995513 3.273471409
+## 1117 1118 1119 1120 1121 1122
+## 3.148423202 3.481820812 3.556772605 3.248423202 2.615025593 3.085899099
+## 1123 1124 1125 1126 1127 1128
+## 2.794248501 2.223374996 2.660850892 3.031724398 3.185899099 2.931724398
+## 1129 1130 1131 1132 1133 1134
+## 2.940073800 2.469200294 2.740073800 3.177549697 3.031724398 2.885899099
+## 1135 1136 1137 1138 1139 1140
+## 2.306676191 2.115025593 3.140073800 2.560850892 2.615025593 2.369200294
+## 1141 1142 1143 1144 1145 1146
+## 2.885899099 2.515025593 2.406676191 1.719103881 2.498326789 2.198326789
+## 1147 1148 1149 1150 1151 1152
+## 2.273278582 2.252501490 1.681627984 2.381627984 1.919103881 1.827453283
+## 1153 1154 1155 1156 1157 1158
+## 1.719103881 1.464929180 1.835802685 2.189977386 2.298326789 2.364929180
+## 1159 1160 1161 1162 1163 1164
+## 2.273278582 1.952501490 2.144152088 1.919103881 2.481627984 1.464929180
+## 1165 1166 1167 1168 1169 1170
+## 1.989977386 2.119103881 2.435802685 1.889977386 1.681627984 2.852501490
+## 1171 1172 1173 1174 1175 1176
+## 2.281627984 1.573278582 1.877356869 0.977356869 0.977356869 1.785706272
+## 1177 1178 1179 1180 1181 1182
+## 1.456579777 1.956579777 1.948230375 1.494055674 1.248230375 1.794055674
+## 1183 1184 1185 1186 1187 1188
+## 1.777356869 1.402405076 1.685706272 2.002405076 1.285706272 1.194055674
+## 1189 1190 1191 1192 1193 1194
+## 1.885706272 1.602405076 1.948230375 1.494055674 1.023182168 1.756579777
+## 1195 1196 1197 1198 1199 1200
+## 0.789784559 0.869007467 1.235609858 0.852308662 0.781435157 1.143959260
+## 1201 1202 1203 1204 1205 1206
+## 1.114832766 0.760658065 1.181435157 1.406483364 1.314832766 1.135609858
+## 1207 1208 1209 1210 1211 1212
+## 1.314832766 1.014832766 1.660658065 0.598133961 0.293862846 0.348037548
+## 1213 1214 1215 1216 1217 1218
+## 0.673085754 0.164736352 0.548037548 0.302212249 0.448037548 -0.160311855
+## 1219 1220 1221 1222 1223 1224
+## 0.702212249 0.973085754 0.218911053 0.856386950 0.364736352 0.102212249
+## 1225 1226 1227 1228 1229 1230
+## 0.093862846 0.810561651 -0.377010659 0.377164042 0.097941134 -0.443805878
+## 1231 1232 1233 1234 1235 1236
+## -0.364582970 5.331917225 5.719489536 4.652694317 4.661043720 4.915218421
+## 1237 1238 1239 1240 1241 1242
+## 5.152694317 4.615218421 5.044344915 4.010947306 4.619296708 4.935995513
+## 1243 1244 1245 1246 1247 1248
+## 4.527646110 3.965122007 4.665122007 4.381820812 4.210947306 4.240073800
+## 1249 1250 1251 1252 1253 1254
+## 3.969200294 4.285899099 3.877549697 4.177549697 3.006676191 3.777549697
+## 1255 1256 1257 1258 1259 1260
+## 3.894248501 2.464929180 2.727453283 3.598326789 2.919103881 3.852501490
+## 1261 1262 1263 1264 1265 1266
+## 3.073278582 3.852501490 2.619103881 3.489977386 2.773278582 3.398326789
+## 1267 1268 1269 1270 1271 1272
+## 3.744152088 2.952501490 3.498326789 3.744152088 3.102405076 2.931531570
+## 1273 1274 1275 1276 1277 1278
+## 2.539880973 1.977356869 1.923182168 2.602405076 2.085706272 2.231531570
+## 1279 1280 1281 1282 1283 1284
+## 3.102405076 2.077356869 3.002405076 2.931531570 3.256579777 2.585706272
+## 1285 1286 1287 1288 1289 1290
+## 2.343959260 1.543959260 1.381435157 1.689784559 2.389784559 2.181435157
+## 1291 1292 1293 1294 1295 1296
+## 2.714832766 1.760658065 1.869007467 2.560658065 1.869007467 1.002212249
+## 1297 1298 1299 1300 1301 1302
+## 1.448037548 0.939688145 1.756386950 1.518911053 1.248037548 1.673085754
+## 1303 1304 1305 1306 1307 1308
+## 1.110561651 1.585513444 1.531338743 1.331338743 0.614639938 0.914639938
+## 1309 1310 1311 1312 1313 1314
+## -0.135456475 -0.102251694 4.548423202 4.702597904 4.931724398 4.969200294
+## 1315 1316 1317 1318 1319 1320
+## 4.815025593 4.923374996 4.915025593 5.185899099 4.244152088 4.635802685
+## 1321 1322 1323 1324 1325 1326
+## 3.844152088 4.198326789 4.473278582 4.644152088 3.594055674 3.931531570
+## 1327 1328 1329 1330 1331 1332
+## 3.877356869 3.131531570 3.077356869 3.106483364 3.169007467 3.289784559
+## 1333 1334 1335 1336 1337 1338
+## 3.135609858 3.260658065 2.843959260 2.727260456 2.448037548 2.439688145
+## 1339 1340 1341 1342 1343 1344
+## 2.285513444 1.914639938 2.477164042 2.114639938 2.252115835 1.272892927
+## 1345 1346 1347 1348 1349 1350
+## 0.268621812 6.085899099 5.198326789 5.189977386 5.010754478 4.494055674
+## 1351 1352 1353 1354 1355 1356
+## 5.202405076 4.423182168 4.494055674 4.560658065 3.981435157 4.352308662
+## 1357 1358 1359 1360 1361 1362
+## 3.318911053 3.627260456 3.310561651 2.552115835 2.672892927 1.576971214
+## 1363 1364 1365 1366 1367 1368
+## 7.123374996 6.235802685 5.823182168 4.002212249 4.477164042 3.460465237
+## 1369 1370 1371 1372 1373 1374
+## 3.881242329 2.831145916 2.747844720 6.469007467 5.564736352 2.726874801
+## 1375
+## 9.052501490sum(model$residuals^2)/model$df.residual## [1] 5.136167# sum(model$residuals^2) is SSE, df.residual is n-2Note: The least squares estimates are also the maximum likelihood +estimates of the parameters.
+Interpretation
+-
+
- The estimated intercept, \(\hat\alpha_1\), is the estimated average +value of Y when X = 0. +
Example: \(\hat\alpha_1=29.917\)
+The estimated average height of women chose mothers heights are 0 +inches is 29.917 inches.
+-
+
- The estimated slope, \(\hat +\beta_1\), is the estimated change in the average of \(Y | X = x\) for a one-unit increase in +x. +
Example: \(\hat \beta_1=0.542\)
+The Mothers who are 1 inch taller have daughter who have 5.42 inches +on average.
+Lecture_39
+Zheyuan Wu
+2023-04-26
+ +Inference for \(\beta_1\)
+Since \(\beta_1\) captures the +relationship between X and Y , we would like to be able to do inference +on this parameter.
+Confidence interval or test about \(\beta_1\), (not \(\hat\beta_1\))
+Result: If \(Y | X = x\) has a +normal distribution and \(S_{\hat +\beta_1}\) is the estimated standard error of \(\hat \beta _1\), then
+\[ +{\hat \beta_1-\beta_1\over S_{\hat \beta_1}}\sim T_{n-2} +\]
+\(S_{\hat \beta_1}\) is the standard +error of \(\hat \beta_1\). depend on +\(\sigma_\epsilon^2\). You will not +required to calculate the value by hand in this course.
+This holds approximately if \(Y | X = +x\) is not normal, but \(n \geq +30\).
+Confidence Interval for \(\beta_1\)
+A \((1-\alpha)100\)% confidence +interval for \(\beta_1\) is
+\[ +\hat \beta_1\pm t_{n-1,\alpha/2}S_{\hat \beta_1} +\]
+In R:
+confint(model, level =0.95)## 2.5 % 97.5 %
+## (Intercept) 26.7346495 33.1002241
+## Mheight 0.4908201 0.5926739The intercept is \(\alpha_1\), +Mheight is \(\beta_1\)
+Hypothesis Test for \(\beta_1\)
+-
+
- State the hypotheses +
\[ +H_0:\beta_1=\beta_{1,0},H_0:\beta_1=\beta_{1,0},H_0:\beta_1=\beta_{1,0}\\ +H_a:\beta_1>\beta_{1,0},H_a:\beta_1<\beta_{1,0},H_a:\beta_1\neq\beta_{1,0} +\]
+\(H_a:\beta_1\neq\beta_{1,0}\) means +there is no linear relationship between Y and X.
+\(H_a:\beta_1>\beta_{1,0}\) means +Y and X are positively correlated, or Y will increase as X +increases.
+\(H_a:\beta_1<\beta_{1,0}\) means +Y and X are negatively correlated, or Y will decrease as X +increases.
+-
+
- Compute the test statistic +
\[ +T_{H_0}={\hat\beta_1-\beta_{1,0}\over S_{\hat\beta_1}} +\]
+-
+
- Reach a conclusion +
-
+
If \(H_0\) is true, \(T_{H_0}\sim T_{n-2}\)
+We can find rejection rules and p-values using the same method as +our previous T tests.
+
-
+
- State the conclusion in the context of the problem. +
Example: Test whether mothers’ height and daughters’ height have a +positive linear relationship using \(\alpha = +0.05\).
+summary(model)##
+## Call:
+## lm(formula = Dheight ~ Mheight, data = heights)
+##
+## Residuals:
+## Min 1Q Median 3Q Max
+## -7.397 -1.529 0.036 1.492 9.053
+##
+## Coefficients:
+## Estimate Std. Error t value Pr(>|t|)
+## (Intercept) 29.91744 1.62247 18.44 <2e-16 ***
+## Mheight 0.54175 0.02596 20.87 <2e-16 ***
+## ---
+## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+##
+## Residual standard error: 2.266 on 1373 degrees of freedom
+## Multiple R-squared: 0.2408, Adjusted R-squared: 0.2402
+## F-statistic: 435.5 on 1 and 1373 DF, p-value: < 2.2e-16\(\hat\beta_1=0.54175>0\), +standard error of \(\hat \beta_1\) is +0.02596
+\(T_{H_0}=20.87\), p-value is +realllly small.
+The p-value is two-sided by default in R.
+\(H_0:\beta_1=0\), \(H_a:\beta_1>0\)
+$T_{H_0}={ 0.54175 }=20.87 $
+\(p-value=2\div (2\times +10^{-16})<0.05\)
+Reject \(H_0\) p-value is smaller +than \(\alpha\)
+Prediction
+The estimated regression line is
+\[ +\hat Y=\hat \alpha_1+\hat \beta_1 x +\]
+We can use this regression line to predict values of Y for specific +values of x. We will denote this predicted value with \(\hat \mu_{Y|X}(x)\).
+Example: What is the predicted height for a woman whose mother is 69 +inches tall?
+\(\hat y=\hat +\alpha_1+\hat\beta_1x=29.917+0.542\times 69=67.315\)
+\(\hat \mu_{Y|X=69}=\hat +E(Y|X=69)=67.315\) inches.
+This value represents two types of prediction.
+-
+
- The average value of Y for the sub-population with \(X=x\) +
Average height of all women whose mothers are 69 +inches
+Smaller error due to averaging
+-
+
- An individual value of Y when \(X=x\) +
The height of an individual (future observation) +women whose mother is 69 inches.
+Higher error for individual prediction
+The errors and intervals associated with these two cases are +different.
+Lecture_4
+Zheyuan Wu
+2023-01-25
+ +extra office hour from undergraduates next week.
+Averages
+-
+
- let \(v_i\), … \(v_n\) to denote the valued in population of +our variable of interest. The population average or +population mean is +
\[ +\mu = {1\over{N}}\sum^n_{i=1}{v_i} +\]
+you may keep two digit for accuracy in homework.
+-
+
- sample mean (or sample average) +
\[ +\bar{x} = {1\over{n}}\sum^n_{i=1}{x_i} +\]
+Variance and standard deviation
+-
+
- The population variance measures the amount of intrinsic +variability in the population (average distance from the population +mean) +
\[ +\sigma^2={1\over{N}}\sum^N_{i=1}{(v_i-\mu)^2} +\]
+The square function is simpler than absolute value (square +function is differentiable and looks nice.)
+-
+
We can also demote the population variance as \(\sigma^2_X\) or Var(X), the variance of the +random variable X.
+The population standard deviation is the square root of +the population variance.
+
\[ +\sigma = \sqrt{\sigma^2} +\]
+unit of variance: (original unit)^2
+unit of standard deviation: original unit
-
+
- The sample variance \[ +S^2={1\over{n-1}}\sum^n_{i=1}{(x_i-\bar{x})^2} +\] +
we use n-1 to make estimation unbiaseness of S^2 for ^2. (we +don’t have full scope for the data anymore when we use sample to +esitmate population)
+-
+
- The sample standard deviation \[ +S=\sqrt{S^2} +\] +
if all the sample response are the same, S=0. (no variability in +data)
+In R
+Sample data
+temperature<-c(1,6,3,4,5,2,7)
+mean(temperature)## [1] 4var(temperature)## [1] 4.666667sd(temperature)## [1] 2.160247Percentile
+The \(x_{(i)}\) is the 100(\({i-0.5}\over n\))-th sample +percentile. The 0.5 here is used to avoid 100 +percentile.
+In R
+sort(temperature)## [1] 1 2 3 4 5 6 7Some percentiles hare of particular interest:
+-
+
- 50th percentile = median = \(\widetilde{x}\) +
- 25th percentile = lower sample quantile = \(q_1\) +
- 75th percentile = higher sample quantile = \(q_3\) +
In R
+quantile(temperature,0.25)## 25%
+## 2.5quantile(temperature,0.75)## 75%
+## 5.5# median
+quantile(temperature,0.5)## 50%
+## 4median(temperature)## [1] 4-
+
- The R command summary provides five number summary for the data +\(min,q1,median,q2,max\) +
summary(temperature)## Min. 1st Qu. Median Mean 3rd Qu. Max.
+## 1.0 2.5 4.0 4.0 5.5 7.0sample average is sensitive for outliers, while the median is +less sensitive to outliers.
+The sample interquartile range (IQR) is an alternative +measure of variability. \[ +IQR=q_3-q_1 +\] Boxplots
+\[ +max=min(q_3+1.5\times IQR,actual\ max) \\ +min=max(q_1-1.5\times IQR,actual\ min) +\] exceptions are called outliers.
+Lecture_40
+Zheyuan Wu
+2023-04-28
+ +Prediction
+The estimated regression line is
+\[ +\hat Y=\hat \alpha_1+\hat \beta_1 x +\]
+We can use this regression line to predict values of Y for specific +values of x. We will denote this predicted value with \(\hat \mu_{Y|X}(x)\).
+Example: What is the predicted height for a woman whose mother is 69 +inches tall?
+\(\hat y=\hat +\alpha_1+\hat\beta_1x=29.917+0.542\times 69=67.315\)
+\(\hat \mu_{Y|X=69}=\hat +E(Y|X=69)=67.315\) inches.
+This value represents two types of prediction.
+-
+
- The average value of Y for the sub-population with \(X=x\) +
Average height of all women whose mothers are 69 +inches
+Smaller error due to averaging
+-
+
- An individual value of Y when \(X=x\) +
The height of an individual (future observation) +women whose mother is 69 inches.
+Higher error for individual prediction
+The errors and intervals associated with these two cases are +different.
+It is easier to estimate the average for populations (smaller error, +narrower confidence interval) than to predict a response for an +individual (larger error, larger confidence interval).
+Confidence Interval for an Average Response
+A \((1-\alpha)100\)% confidence +interval is
+\[ +\hat \mu_{Y|X}(x)\pm t_{n-2,\alpha/2}S_{\epsilon}\sqrt{{1\over +n}+{(x-\bar X)^2\over \sum(X_i-\bar X)^2}} +\]
+The margin of error of average response interval will converge to +zero as \(n\to \infty\)
+-
+
The margin of error depends on variation of response +(error).
+Confidence interval at the center of predictors, and wider as x +moves away from \(\bar X\)
+Confidence interval is smaller as sample increase
+Larger variance of X leads to a smaller Confidence interval. +(more “type” of sample collected.)
+
Prediction Interval for an Individual Response
+A \((1-\alpha)100\)% prediction +interval is
+\[ +\hat \mu_{Y|X}(x)\pm t_{n-2,\alpha/2}S_{\epsilon}\sqrt{1+{1\over +n}+{(x-\bar X)^2\over \sum(X_i-\bar X)^2}} +\]
+The margin of error of prediction interval will not converge to zero +as \(n\to \infty\)
+The prediction band is larger in prediction than average response
+In R
+Both of these intervals can be computed using R.
+Example: Compute and interpret the 95% confidence interval and 95% +prediction interval when mother’s height is 69 inches.
+t <- data.frame(Mheight=69)
+predict(model, t, interval = "confidence", level = 0.95)## fit lwr upr
+## 1 67.29798 66.94365 67.65231predict(model, t, interval = "prediction", level = 0.95)## fit lwr upr
+## 1 67.29798 62.83808 71.75789Caution: Don’t do prediction for values of x that are far outside of +the range of the data.
+fit is the \(\hat\mu_{Y|X}\) of +data
+Interpretation:
+-
+
Average interval: We are 95% confident that the average +height of women whose mother are 69 inches tall is between +66.94 an 67.65.
+Prediction interval: We are 9% confident that a +particular women whose mother is 69 inches tall is between +62.84 and 71.76 inches.
+
Lecture_5
+Zheyuan Wu
+2023-01-25
+ +++first exam cover chapter two.
+
Mean and Will Rogers Paradox
+++“There are three kinds of lies: lies, damned lies, and +statistics.”
+
When we change an observation (eg.4) from Group A to Group B, both +mean increased.
+Save life without treatment.
+# Life span of two groups
+Healthy=c(60,70,80,90,100)
+Cancer=c(45,50,55,60,65)
+mean(Healthy)## [1] 80mean(Cancer)## [1] 55By changing diagnose method or imaging techniques.
+# Change the 60 to the other group
+Healthy=c(70,80,90,100)
+Cancer=c(45,50,55,60,60,65)
+mean(Healthy)## [1] 85mean(Cancer)## [1] 55.83333Mean is sensitive to outlier and may not represent a +“typical” value of the group.
+Introduction to Probability
+Experiments, outcomes, sample space, and events
+Event: Pulling out an orange ball (doesn’t matter which one).
+Random draw: Put my hand in the bowl and pull ball out without +looking.
+record the color and put them back.
+Experiment: single, “random” trail
+-
+
- Pull ball out of bowl +
- Flipping coin +
Outcomes: observable, potential result of the trial
+-
+
- Getting a blue ball +
- Getting a head +
Sample Space: Set of all outcomes.
+-
+
- set(blue, orange, black, white …) +
- set(head, tail) +
Event: any subset of the sample space (subset of sample space)
+-
+
- set(blue) +
- set(tail) +
Probability of an event: the relative frequency with which that event +can be expected to occur
+WE could also think of probability as being a function that takes +every event and assigned between 1 and 0.
+\[ +P(X) +\]
+Union, Intersection, complement, disjoint Events
+Subset: collection of any outcomes
+Elementary event: an event consisting of a single outcome
+empty set: set that does not have any element.
+Complement of event of A are all the space other than event A +(denoted by \(A^c\))
+Multiple events
+Intersection:
+\[ +C = A \cup B +\] Mutrually exclusive event: events that don’t have any overlap +(\(C = A \cup B =\phi\)) (independent +from each other)
+Union:
+\[ +C = A \cap B +\]
+Difference and Subset: A-B = the event have the outcomes contains A +but not B
+if \(A\cup B=A\),then A is a subset +of B, denoted by (\(A \subseteq +B\))
+We may also use this notation for sum in union / intersection +operations
+\[ +\cup^k_{i=1}A_i = A_1\cup A_2 \cup A_3 ... \cup A_k +\] Union otherwise..
+Union… At least one sample is acceptable
+Intersect… All the sample is acceptable
+Set operations
+-
+
Commutative Laws: \[ +A\cup B=B\cup A,A\cap B=B \cap A +\]
+Associative Laws: \[ +(A\cup B)\cup C=A\cup(B\cup C),(A\cap B)\cap C=A\cap(B\cap C) +\]
+Distributive Laws: \[ +(A\cup B)\cap C=(A\cap C)\cup(B\cap C),(A\cap B)\cup C=(A\cup +C)\cap(B\cup C) +\]
+De Morgan’s Laws: \[ +(A\cup B)^c=A^c\cap B^c, (A\cap B)^c=A^c\cup B^c +\]
+
Axioms of probability
+We want probability to be relative frequency with which we expect the +event to occur
+Axiom 1: \(P(S)=1\)
+Axiom 2: \(0\leq P(A) \leq 1\)
+Axiom 3: \(P(A\cup B)=P(A)+P(B)\) if +they are multrally exclusive.
+Lecture_6
+Zheyuan Wu
+2023-01-30
+ +Introduction to Probability
+-
+
\(P(\phi)\) is 0.
+For any finite collection \(E_1,E_2,...E_m\) of disjoint event \[ +P(E_1 \cup E_2 \cup .... \cup E_m)=P(E_1)+P(E_2)+...+P(E_m) +\]
+If \(A\subseteq B\), then \(P(A)\leq P(B)\)
+\(P(A^c)=1-P(A)\)
+$P(AB)=P(A)+P(B)-P(AB) $
+\(P(A\cup B \cup +C)=P(A)+P(B)+P(C)-P(A\cap B)-P(A\cap C)-P(B\cap C)+P(A\cap B\cap +C)\)
+
Counting
+The Multiplication Rule: If a task can be completed in K +stages and stage i has n_i outcomes, regardless of the outcomes of the +previous stages, then the task has \(n_1,n_2,..n_k\) outcomes.
+Example. coin tossed 10 times.
+\(2^{10}=1024\)
+Select first second, and third place winder from a group of 4 +final-list. (select without replacement.)
+\(4\times 3\times 2=24\)
+Equal likely outcome
+-
+
The probability of an event \(E\) is the likelihood of the occurrence of +E, This is denoted \(P(E)\). (\(0\leq P(E) \leq 1\))
+Let S be a sample space with N outcomes that are equally likely +to occur. Then the probability of each outcome is 1/N.
+If \(N(E)\) denotes the number +of outcomes in the Event \(E\), +then
+
\[ +P(E)={{N(E)}\over N} +\]
+Example: Suppose we roll 2 die separately, the probability of rolling +6 is.
+\(E=\{(1,5),(2,4),(3,3),(4,2),(5,1)\},N=5\)
+\(P(E)=5/36=0.139\)
+Permutations and Combinations
+-
+
If a distinction is made between the outcomes of the stages, we +say the outcome are ordered. Otherwise we say the outcomes are +unordered.
+The ordered outcomes are called permutations of +k units. The number of permutations of k unites +selected from a group of n units is denoted by \(P_{k,n}\).
+The unorderd outcomes are called combinations of +k units. The number of combinations of k units +selected from a group of n units is denoted by \((^n_k)\).
+
To compute the number of permutations of k units:
+\[ +P_{k,n}=n(n-1)(n-2)...(n-k+1)={n!\over{(n-k)!}} +\] where \(m!=m(m-1)(m-2)..(2)(1)\) and \(0!=1\).
+and \(P_{n,n}=n!\)
+To compute the number of combinations of k units
+\[ +\begin{pmatrix}n\\k\end{pmatrix}={P_{k,n}\over{P_{k,k}}}={n!\over{k!(n-k)!}} +\] choose k form n
+Example. Select two card from a deck of 52 cards.
+-
+
- When the first card to player 1 and second card to player 2 +
\[ +P_{2,52}={52!\over{(52-2)!}}=52\times 51=2652 +\] * When both card to player 1
+\[ +C_{2,52}=\begin{pmatrix}52\\2\end{pmatrix}={{52\times 51}\over 2!}=1326 +\]
+Lecture_7
+Zheyuan Wu
+2023-02-01
+ +Continuing on Counting
+We can generalize the idea of combinations: the number of +arrangements of n units into r groups of sizes \(n_1,n_2,n_3...n_r\) is \[ +n=n_1+n_2+n_3...+n_r\\ +\begin{pmatrix}n\\n_1,n_2..n_r\end{pmatrix}={n!\over{n_1!n_2!n_3!..n_r!}} +\]
+Example, ways to build group of 2,3,3 from 8 students:
+\[ +\begin{pmatrix}8\\2,3,3\end{pmatrix}={8!\over{2!3!3!}}=560 +\]
+Example, a communication system consist of 13 antennas work as long +as no two of them arrange next to each other. suppose 5 of them stop +functioning.
+-
+
- number of arrangement for non functioning antennas +
8 functioning, 5 non functioning 9 possible locations for non +functioning antennas
+\[ +\begin{pmatrix}9\\5\end{pmatrix}={9!\over{5!4!}}=126 +\]
+(b)probability for the system still functional
+total arrangement for the antenna \[ +\begin{pmatrix}13\\5\end{pmatrix}={13!\over{5!8!}}=1287\\ +P(funcitonal)=126/1287 +\]
+Probability Mass Function
+The probability mass function of a discrete random variable +X is a list of the probabilities p(x) for each value x +in the sample pace \(S_x\) of X.
+Rolling two dies. \[ +P(x=2)={1\over{36}}...P(x=12)={1\over{36}} +\]
+Example: random sample of size n =3 from 10 sample, where 4 of them +are defective.
+x | 1 | 2 | 3 |
+p(x)|
+for x=0, we select 3 non-defective from 6 non-defective sample, +divided by the general chance of selecting 3 item from 10.
+\[ +P(x=0)={\begin{pmatrix}6\\3\end{pmatrix}\over{\begin{pmatrix}10\\3\end{pmatrix}}}=0.167 +\]
+for x=1, we select 2 non-defective from 6 non-defective sample, 1 +defective from 4 defective, then divided by the general chance of +selecting 3 item from 10
+\[ +P(x=1)={\begin{pmatrix}6\\2\end{pmatrix}\begin{pmatrix}4\\1\end{pmatrix}\over{\begin{pmatrix}10\\3\end{pmatrix}}}=0.5 +\]
+for x=2,… \[ +P(x=2)={\begin{pmatrix}6\\1\end{pmatrix}\begin{pmatrix}4\\2\end{pmatrix}\over{\begin{pmatrix}10\\3\end{pmatrix}}}=0.3 +\]
+for x=3,… \[ +P(x=3)={\begin{pmatrix}4\\3\end{pmatrix}\over{\begin{pmatrix}10\\3\end{pmatrix}}}=0.033 +\]
+\[ +P(x=0)+P(x=1)+P(x=2)+P(x=3)=1 +\]
+Simulation of experiment in R
+# replace = TRUE means we take sample with replacement (no put back operation)
+x = sample(0:3, size = 10000, replace = TRUE, prob = c(0.167,0.5,0.3,0.033))
+table(x)/10000## x
+## 0 1 2 3
+## 0.1648 0.5012 0.3015 0.0325Conditional probability and dependent event
+Conditional probability: the relative frequency we can expect an +event to occur under the condition that additional, preexisting +information is known about some other event. Denoted by \(P(A|B)\) (probability of A given B)
+During the process, Event B becomes the new sample space.
+\[ +P(A|B)=P(A\cap B)/P(B) +\]
+Lecture_8
+Zheyuan Wu
+2023-02-03
+ +Conditional Probability
+Independent events
+Independent events: if the occurrence of one event +does not change the probability of the other event occurring.
+\[ +P(A|B)=P(A \cap B)/P(B)=P(A)\\ +P(A\cap B)=P(A)\times P(B) +\]
+Properties of Independence
+-
+
If A is independent of B, B is also independent of A
+If A is independent of B, A is also independent of \(B^c\)
+
\[ +Proof: P(A|B)={P(A\cap B^c)\over{P(B^c)}}\\ +={P(A)-P(A\cap B)\over{1-P(B)}}\\ +={P(A)-P(A)\times P(B)\over{1-P(B)}}\\ +=P(A){(1-P(B))\over{1-P(B)}}\\ +=P(A) +\]
+-
+
If A is independent of B, \(A^c\)is also independent of B
+If A is independent of B, \(A^c\)is also independent of \(B^c\)
+Any event is independent of the empty event
+
Mutually Exclusive vs Independence
+For two mutually exclusive event \(P(A\cap +B)\) is zero. To be independent \(P(A|B)=P(A)\) But \(P(A|B)\) is zero because \(P(A\cap B)\) is zero.
+Example: Let A, B and C be three disjoint event with P(A)=0.2, +P(B)=-0.3, P(C)=0.5. Find \(P({A\cap B^c +\over{B\cup C}})\)
+\[ +P({A\cap B^c \over{B\cup C}})={P((A\cup B^c)\cap(B\cup C))\over{P(B\cup +C)}}\\ +={P(\phi \cup C)\over P(B\cup C)}\\ +={0.5\over 0.8}\\ +={5\over 8} +\]
+Rules of probabilites
+Multiplication Rule
+When tow events were independent we could write
+\[ +P(A\cap B)=P(A|B)\times P(B)\\ +=P(B|A)\times P(A)\\ +=P(A)\times P(B) +\]
+This was because \(P(A)=P(A|B)\), +which is not generally true.
+Addition Rule
+For two mutually exclusive events,
+\[ +P(A\cup B)=P(A)+P(B) +\]
+if they are not mutually exclusive.
+\[ +P(A\cup B)=P(A)+P(B)-P(A\cap B) +\]
+Complement Rule
+For any envent
+\[ +P(A)=S-P(A^c)=1-P(A^c) +\]
+Lecture_9
+Zheyuan Wu
+2023-02-06
+ +Bayes’ Rule
+-
+
Describe the probability of an event, based on prior knowledge of +conditions that might be related to the event.
+Can be used to update belief, given some evidence
+
Example: patient with positive test
++= positive diagnose, D= Disease, \(D^c\)=Normal, \(+^c\)=negative diagnose
+Given, the sensitivity of the test is: \(P(+|D)=0.99\),\(P(-|D^c)=0.95\) And the doctor said \(P(D)=0.001\)
+Goal:\(P(D|+)={P(D\cap ++)\over{P(+)}}\) (The positive of having cancer given positive +test result.)
+\[ +\begin{align} +P(D|+)&={P(D\cap +)\over{P(+)}}\\ +&={P(D)\times P(+|D)\over{P(+|D)+P(+|D^c)}}\\ +&={0.00099\over{0.00099+0.04995}}\\ +&=0.0194 +\end{align} +\]
+Baye’s Rule (Inverse Conditional Probability)
+\[ +P(B|A)={P(A|B)P(B)\over{P(A|B)P(B)+P(A|B^c)P(B^c)}} +\]
+First partition the sample space by B, and partition again by A.
+\[ +P(A|B)\neq 1-P(A|B^c) +\]
+Because they don’t add up to one, and they are not compliment of each +other.
+