Probabilities and Statistics translation [webpage]
1. Probabilities and Statistics refresher
⟶
2. Introduction to Probability and Combinatorics
⟶
3. Sample space ― The set of all possible outcomes of an experiment is known as the sample space of the experiment and is denoted by S.
⟶
4. Event ― Any subset E of the sample space is known as an event. That is, an event is a set consisting of possible outcomes of the experiment. If the outcome of the experiment is contained in E, then we say that E has occurred.
⟶
5. Axioms of probability For each event E, we denote P(E) as the probability of event E occuring.
⟶
6. Axiom 1 ― Every probability is between 0 and 1 included, i.e:
⟶
7. Axiom 2 ― The probability that at least one of the elementary events in the entire sample space will occur is 1, i.e:
⟶
8. Axiom 3 ― For any sequence of mutually exclusive events E1,...,En, we have:
⟶
9. Permutation ― A permutation is an arrangement of r objects from a pool of n objects, in a given order. The number of such arrangements is given by P(n,r), defined as:
⟶
10. Combination ― A combination is an arrangement of r objects from a pool of n objects, where the order does not matter. The number of such arrangements is given by C(n,r), defined as:
⟶
11. Remark: we note that for 0⩽r⩽n, we have P(n,r)⩾C(n,r)
⟶
12. Conditional Probability
⟶
13. Bayes' rule ― For events A and B such that P(B)>0, we have:
⟶
14. Remark: we have P(A∩B)=P(A)P(B|A)=P(A|B)P(B)
⟶
15. Partition ― Let {Ai,i∈[[1,n]]} be such that for all i, Ai≠∅. We say that {Ai} is a partition if we have:
⟶
16. Remark: for any event B in the sample space, we have P(B)=n∑i=1P(B|Ai)P(Ai).
⟶
17. Extended form of Bayes' rule ― Let {Ai,i∈[[1,n]]} be a partition of the sample space. We have:
⟶
18. Independence ― Two events A and B are independent if and only if we have:
⟶
19. Random Variables
⟶
20. Definitions
⟶
21. Random variable ― A random variable, often noted X, is a function that maps every element in a sample space to a real line.
⟶
22. Cumulative distribution function (CDF) ― The cumulative distribution function F, which is monotonically non-decreasing and is such that limx→−∞F(x)=0 and limx→+∞F(x)=1, is defined as:
⟶
23. Remark: we have P(a<X⩽B)=F(b)−F(a).
⟶
24. Probability density function (PDF) ― The probability density function f is the probability that X takes on values between two adjacent realizations of the random variable.
⟶
25. Relationships involving the PDF and CDF ― Here are the important properties to know in the discrete (D) and the continuous (C) cases.
⟶
26. [Case, CDF F, PDF f, Properties of PDF]
⟶
27. Expectation and Moments of the Distribution ― Here are the expressions of the expected value E[X], generalized expected value E[g(X)], kth moment E[Xk] and characteristic function ψ(ω) for the discrete and continuous cases:
⟶
28. Variance ― The variance of a random variable, often noted Var(X) or σ2, is a measure of the spread of its distribution function. It is determined as follows:
⟶
29. Standard deviation ― The standard deviation of a random variable, often noted σ, is a measure of the spread of its distribution function which is compatible with the units of the actual random variable. It is determined as follows:
⟶
30. Transformation of random variables ― Let the variables X and Y be linked by some function. By noting fX and fY the distribution function of X and Y respectively, we have:
⟶
31. Leibniz integral rule ― Let g be a function of x and potentially c, and a,b boundaries that may depend on c. We have:
⟶
32. Probability Distributions
⟶
33. Chebyshev's inequality ― Let X be a random variable with expected value μ. For k,σ>0, we have the following inequality:
⟶
34. Main distributions ― Here are the main distributions to have in mind:
⟶
35. [Type, Distribution]
⟶
36. Jointly Distributed Random Variables
⟶
37. Marginal density and cumulative distribution ― From the joint density probability function fXY , we have
⟶
38. [Case, Marginal density, Cumulative function]
⟶
39. Conditional density ― The conditional density of X with respect to Y, often noted fX|Y, is defined as follows:
⟶
40. Independence ― Two random variables X and Y are said to be independent if we have:
⟶
41. Covariance ― We define the covariance of two random variables X and Y, that we note σ2XY or more commonly Cov(X,Y), as follows:
⟶
42. Correlation ― By noting σX,σY the standard deviations of X and Y, we define the correlation between the random variables X and Y, noted ρXY, as follows:
⟶
43. Remark 1: we note that for any random variables X,Y, we have ρXY∈[−1,1].
⟶
44. Remark 2: If X and Y are independent, then ρXY=0.
⟶
45. Parameter estimation
⟶
46. Definitions
⟶
47. Random sample ― A random sample is a collection of n random variables X1,...,Xn that are independent and identically distributed with X.
⟶
48. Estimator ― An estimator is a function of the data that is used to infer the value of an unknown parameter in a statistical model.
⟶
49. Bias ― The bias of an estimator ^θ is defined as being the difference between the expected value of the distribution of ^θ and the true value, i.e.:
⟶
50. Remark: an estimator is said to be unbiased when we have E[^θ]=θ.
⟶
51. Estimating the mean
⟶
52. Sample mean ― The sample mean of a random sample is used to estimate the true mean μ of a distribution, is often noted ¯¯¯¯¯X and is defined as follows:
⟶
53. Remark: the sample mean is unbiased, i.e E[¯¯¯¯¯X]=μ.
⟶
54. Central Limit Theorem ― Let us have a random sample X1,...,Xn following a given distribution with mean μ and variance σ2, then we have:
⟶
55. Estimating the variance
⟶
56. Sample variance ― The sample variance of a random sample is used to estimate the true variance σ2 of a distribution, is often noted s2 or ^σ2 and is defined as follows:
⟶
57. Remark: the sample variance is unbiased, i.e E[s2]=σ2.
⟶
58. Chi-Squared relation with sample variance ― Let s2 be the sample variance of a random sample. We have:
⟶
59. [Introduction, Sample space, Event, Permutation]
⟶
60. [Conditional probability, Bayes' rule, Independence]
⟶
61. [Random variables, Definitions, Expectation, Variance]
⟶
62. [Probability distributions, Chebyshev's inequality, Main distributions]
⟶
63. [Jointly distributed random variables, Density, Covariance, Correlation]
⟶
64. [Parameter estimation, Mean, Variance]
⟶