Expected value and variance of step of random walk with barriers
up vote
2
down vote
favorite
I have to simulate the following game
Suppose that two players A and B each start with a stake of $5, and bet $0.5 on consecutive coin flips. The game ends when either one of the players has won all the money, that amounts to $10. Let $S_n$ be the fortune of player A at time n. Then ${S_n, n gt 0 }$ is a symmetric random walk with absorbing barriers at 0 and 10. Estimate $E[S_n]$ and $V[S_n]$ for $n = 50$.
I made a program and that's ok. Now I would to compare my results with the theoretical values. I don't know almost anything in probability, so that's just a curiosity. My question is
Which are the values of $E[S_n]$ and $V[S_n]$ in general? And for $n = 50$?
If there's no closed form, I will appreciate also an approximation (I got $E[S_n] approx 5$ and $V[S_{50}] approx 4.8$)
Thanks in advance.
probability random-walk means
|
show 4 more comments
up vote
2
down vote
favorite
I have to simulate the following game
Suppose that two players A and B each start with a stake of $5, and bet $0.5 on consecutive coin flips. The game ends when either one of the players has won all the money, that amounts to $10. Let $S_n$ be the fortune of player A at time n. Then ${S_n, n gt 0 }$ is a symmetric random walk with absorbing barriers at 0 and 10. Estimate $E[S_n]$ and $V[S_n]$ for $n = 50$.
I made a program and that's ok. Now I would to compare my results with the theoretical values. I don't know almost anything in probability, so that's just a curiosity. My question is
Which are the values of $E[S_n]$ and $V[S_n]$ in general? And for $n = 50$?
If there's no closed form, I will appreciate also an approximation (I got $E[S_n] approx 5$ and $V[S_{50}] approx 4.8$)
Thanks in advance.
probability random-walk means
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02
|
show 4 more comments
up vote
2
down vote
favorite
up vote
2
down vote
favorite
I have to simulate the following game
Suppose that two players A and B each start with a stake of $5, and bet $0.5 on consecutive coin flips. The game ends when either one of the players has won all the money, that amounts to $10. Let $S_n$ be the fortune of player A at time n. Then ${S_n, n gt 0 }$ is a symmetric random walk with absorbing barriers at 0 and 10. Estimate $E[S_n]$ and $V[S_n]$ for $n = 50$.
I made a program and that's ok. Now I would to compare my results with the theoretical values. I don't know almost anything in probability, so that's just a curiosity. My question is
Which are the values of $E[S_n]$ and $V[S_n]$ in general? And for $n = 50$?
If there's no closed form, I will appreciate also an approximation (I got $E[S_n] approx 5$ and $V[S_{50}] approx 4.8$)
Thanks in advance.
probability random-walk means
I have to simulate the following game
Suppose that two players A and B each start with a stake of $5, and bet $0.5 on consecutive coin flips. The game ends when either one of the players has won all the money, that amounts to $10. Let $S_n$ be the fortune of player A at time n. Then ${S_n, n gt 0 }$ is a symmetric random walk with absorbing barriers at 0 and 10. Estimate $E[S_n]$ and $V[S_n]$ for $n = 50$.
I made a program and that's ok. Now I would to compare my results with the theoretical values. I don't know almost anything in probability, so that's just a curiosity. My question is
Which are the values of $E[S_n]$ and $V[S_n]$ in general? And for $n = 50$?
If there's no closed form, I will appreciate also an approximation (I got $E[S_n] approx 5$ and $V[S_{50}] approx 4.8$)
Thanks in advance.
probability random-walk means
probability random-walk means
edited Nov 22 at 16:25
asked Nov 22 at 14:57
Marco All-in Nervo
34918
34918
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02
|
show 4 more comments
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02
|
show 4 more comments
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
Modeling the process
Define $mathcal{S}$ as the set of possible values for the Markov chain:
$$mathcal{S} = {0, 0.5, 1, 1.5, …, 9.5, 10}$$
Note that $S_0=5$ and $S_n in mathcal{S}$ for all $n in {0, 1, 2, …}$.
We have
$$S_{n+1} = S_n + A_n quad forall n in {0, 1, 2, ...} $$
where
$$ A_n = left{ begin{array}{ll}
(1/2)B_n &mbox{ if $S_n notin {0, 10}$} \
0 & mbox{ otherwise}
end{array}
right.$$
where ${B_n}_{n=0}^{infty}$ is an i.i.d. sequence with $P[B_n=1]=P[B_n=-1]=1/2$.
Then
$$boxed{E[A_n|S_n=s] = 0 quad, forall s in mathcal{S}} quad (Eq. 1) $$
Mean
So for each $n in {0, 1, 2, ...}$ we have
begin{align}
E[S_{n+1}] &overset{(a)}{=} sum_{s in mathcal{S}}E[S_{n+1}|S_n=s]P[S_n=s] \
&overset{(b)}{=} sum_{s in mathcal{S}}E[S_n + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}E[s + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}(s + E[A_n|S_n=s])P[S_n=s] \
&overset{(c)}{=} sum_{s in mathcal{S}}sP[S_n=s] \
&overset{(d)}{=} E[S_n]
end{align}
where (a) holds by the law of total expectation; (b) holds by the fact $S_{n+1}=S_n+A_n$; (c) holds by Eq. (1); (d) holds by definition of expectation.
Since $E[S_0]=5$ we conclude:
$$boxed{E[S_n]=5 quad forall n in {0, 1, 2, … }}$$
Limiting variance
We know $E[S_n]=5$ for all $n$ and so
$$Var(S_n) = E[(S_n-5)^2] = sum_{s in mathcal{S}}(s-5)^2P[S_n=s] $$
Since the process is equally likely to end up at state $0$ or $10$ we have
begin{align}
lim_{nrightarrowinfty} P[S_n=0] &= 1/2\
lim_{nrightarrowinfty} P[S_n=10] &= 1/2\
lim_{nrightarrowinfty} P[S_n=s] &= 0 quad forall s notin {0, 10}
end{align}
so
$$ boxed{lim_{nrightarrowinfty} Var(S_n) = (0-5)^2(1/2) + (10-5)^2(1/2) = 25} $$
Details on variance
Squaring the equation $S_{n+1} = S_n + A_n$ gives
$$S_{n+1}^2 = (S_n+A_n)^2 = S_n^2 + 2S_nA_n + A_n^2 $$
So
$$E[S_{n+1}^2|S_n] = S_n^2 + 2S_nE[A_n|S_n] + E[A_n^2|S_n] = S_n^2 + 0 + (1/4)1_{{S_n notin{0, 10}}}$$
where $1_{{S_n notin{0, 10}}}$ is an indicator function that is 1 if $S_n notin {0,10}$ and is 0 else. So
$$E[S_{n+1}^2] = E[S_n^2] + (1/4)P[S_n notin {0,10}]$$
Subtracting 25 from both sides gives
$$ Var(S_{n+1}) = Var(S_n) + (1/4)P[S_n notin {0,10}]$$
and $Var(S_0)=0$ so
$$ boxed{Var(S_n) = (1/4)sum_{i=0}^{n-1} P[S_i notin {0,10}] quad forall nin {1, 2, 3, ...} } $$
Since $P[S_i notin {0,10}] = 1$ for $i in {0, 1, 2, 3, ..., 9}$ we have $$boxed{Var(S_1)=1/4, Var(S_2)=2/4, Var(S_3) = 3/4, ..., Var(S_{10})= 10/4}$$
On the other hand:
$$ Var(S_{11}) = 10/4 + (1/4)underbrace{(1-2(1/2)^{10})}_{P[S_{10}notin{0,10}]}$$
In general, the variance increases as $nrightarrowinfty$ to approach a
limiting value of $25$. It is possible to compute $P[S_i notin {0,10}]$ for all $i$ (for example, by taking powers of a transition probability matrix), but this calculation is more involved.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3009239%2fexpected-value-and-variance-of-step-of-random-walk-with-barriers%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
Modeling the process
Define $mathcal{S}$ as the set of possible values for the Markov chain:
$$mathcal{S} = {0, 0.5, 1, 1.5, …, 9.5, 10}$$
Note that $S_0=5$ and $S_n in mathcal{S}$ for all $n in {0, 1, 2, …}$.
We have
$$S_{n+1} = S_n + A_n quad forall n in {0, 1, 2, ...} $$
where
$$ A_n = left{ begin{array}{ll}
(1/2)B_n &mbox{ if $S_n notin {0, 10}$} \
0 & mbox{ otherwise}
end{array}
right.$$
where ${B_n}_{n=0}^{infty}$ is an i.i.d. sequence with $P[B_n=1]=P[B_n=-1]=1/2$.
Then
$$boxed{E[A_n|S_n=s] = 0 quad, forall s in mathcal{S}} quad (Eq. 1) $$
Mean
So for each $n in {0, 1, 2, ...}$ we have
begin{align}
E[S_{n+1}] &overset{(a)}{=} sum_{s in mathcal{S}}E[S_{n+1}|S_n=s]P[S_n=s] \
&overset{(b)}{=} sum_{s in mathcal{S}}E[S_n + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}E[s + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}(s + E[A_n|S_n=s])P[S_n=s] \
&overset{(c)}{=} sum_{s in mathcal{S}}sP[S_n=s] \
&overset{(d)}{=} E[S_n]
end{align}
where (a) holds by the law of total expectation; (b) holds by the fact $S_{n+1}=S_n+A_n$; (c) holds by Eq. (1); (d) holds by definition of expectation.
Since $E[S_0]=5$ we conclude:
$$boxed{E[S_n]=5 quad forall n in {0, 1, 2, … }}$$
Limiting variance
We know $E[S_n]=5$ for all $n$ and so
$$Var(S_n) = E[(S_n-5)^2] = sum_{s in mathcal{S}}(s-5)^2P[S_n=s] $$
Since the process is equally likely to end up at state $0$ or $10$ we have
begin{align}
lim_{nrightarrowinfty} P[S_n=0] &= 1/2\
lim_{nrightarrowinfty} P[S_n=10] &= 1/2\
lim_{nrightarrowinfty} P[S_n=s] &= 0 quad forall s notin {0, 10}
end{align}
so
$$ boxed{lim_{nrightarrowinfty} Var(S_n) = (0-5)^2(1/2) + (10-5)^2(1/2) = 25} $$
Details on variance
Squaring the equation $S_{n+1} = S_n + A_n$ gives
$$S_{n+1}^2 = (S_n+A_n)^2 = S_n^2 + 2S_nA_n + A_n^2 $$
So
$$E[S_{n+1}^2|S_n] = S_n^2 + 2S_nE[A_n|S_n] + E[A_n^2|S_n] = S_n^2 + 0 + (1/4)1_{{S_n notin{0, 10}}}$$
where $1_{{S_n notin{0, 10}}}$ is an indicator function that is 1 if $S_n notin {0,10}$ and is 0 else. So
$$E[S_{n+1}^2] = E[S_n^2] + (1/4)P[S_n notin {0,10}]$$
Subtracting 25 from both sides gives
$$ Var(S_{n+1}) = Var(S_n) + (1/4)P[S_n notin {0,10}]$$
and $Var(S_0)=0$ so
$$ boxed{Var(S_n) = (1/4)sum_{i=0}^{n-1} P[S_i notin {0,10}] quad forall nin {1, 2, 3, ...} } $$
Since $P[S_i notin {0,10}] = 1$ for $i in {0, 1, 2, 3, ..., 9}$ we have $$boxed{Var(S_1)=1/4, Var(S_2)=2/4, Var(S_3) = 3/4, ..., Var(S_{10})= 10/4}$$
On the other hand:
$$ Var(S_{11}) = 10/4 + (1/4)underbrace{(1-2(1/2)^{10})}_{P[S_{10}notin{0,10}]}$$
In general, the variance increases as $nrightarrowinfty$ to approach a
limiting value of $25$. It is possible to compute $P[S_i notin {0,10}]$ for all $i$ (for example, by taking powers of a transition probability matrix), but this calculation is more involved.
add a comment |
up vote
1
down vote
accepted
Modeling the process
Define $mathcal{S}$ as the set of possible values for the Markov chain:
$$mathcal{S} = {0, 0.5, 1, 1.5, …, 9.5, 10}$$
Note that $S_0=5$ and $S_n in mathcal{S}$ for all $n in {0, 1, 2, …}$.
We have
$$S_{n+1} = S_n + A_n quad forall n in {0, 1, 2, ...} $$
where
$$ A_n = left{ begin{array}{ll}
(1/2)B_n &mbox{ if $S_n notin {0, 10}$} \
0 & mbox{ otherwise}
end{array}
right.$$
where ${B_n}_{n=0}^{infty}$ is an i.i.d. sequence with $P[B_n=1]=P[B_n=-1]=1/2$.
Then
$$boxed{E[A_n|S_n=s] = 0 quad, forall s in mathcal{S}} quad (Eq. 1) $$
Mean
So for each $n in {0, 1, 2, ...}$ we have
begin{align}
E[S_{n+1}] &overset{(a)}{=} sum_{s in mathcal{S}}E[S_{n+1}|S_n=s]P[S_n=s] \
&overset{(b)}{=} sum_{s in mathcal{S}}E[S_n + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}E[s + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}(s + E[A_n|S_n=s])P[S_n=s] \
&overset{(c)}{=} sum_{s in mathcal{S}}sP[S_n=s] \
&overset{(d)}{=} E[S_n]
end{align}
where (a) holds by the law of total expectation; (b) holds by the fact $S_{n+1}=S_n+A_n$; (c) holds by Eq. (1); (d) holds by definition of expectation.
Since $E[S_0]=5$ we conclude:
$$boxed{E[S_n]=5 quad forall n in {0, 1, 2, … }}$$
Limiting variance
We know $E[S_n]=5$ for all $n$ and so
$$Var(S_n) = E[(S_n-5)^2] = sum_{s in mathcal{S}}(s-5)^2P[S_n=s] $$
Since the process is equally likely to end up at state $0$ or $10$ we have
begin{align}
lim_{nrightarrowinfty} P[S_n=0] &= 1/2\
lim_{nrightarrowinfty} P[S_n=10] &= 1/2\
lim_{nrightarrowinfty} P[S_n=s] &= 0 quad forall s notin {0, 10}
end{align}
so
$$ boxed{lim_{nrightarrowinfty} Var(S_n) = (0-5)^2(1/2) + (10-5)^2(1/2) = 25} $$
Details on variance
Squaring the equation $S_{n+1} = S_n + A_n$ gives
$$S_{n+1}^2 = (S_n+A_n)^2 = S_n^2 + 2S_nA_n + A_n^2 $$
So
$$E[S_{n+1}^2|S_n] = S_n^2 + 2S_nE[A_n|S_n] + E[A_n^2|S_n] = S_n^2 + 0 + (1/4)1_{{S_n notin{0, 10}}}$$
where $1_{{S_n notin{0, 10}}}$ is an indicator function that is 1 if $S_n notin {0,10}$ and is 0 else. So
$$E[S_{n+1}^2] = E[S_n^2] + (1/4)P[S_n notin {0,10}]$$
Subtracting 25 from both sides gives
$$ Var(S_{n+1}) = Var(S_n) + (1/4)P[S_n notin {0,10}]$$
and $Var(S_0)=0$ so
$$ boxed{Var(S_n) = (1/4)sum_{i=0}^{n-1} P[S_i notin {0,10}] quad forall nin {1, 2, 3, ...} } $$
Since $P[S_i notin {0,10}] = 1$ for $i in {0, 1, 2, 3, ..., 9}$ we have $$boxed{Var(S_1)=1/4, Var(S_2)=2/4, Var(S_3) = 3/4, ..., Var(S_{10})= 10/4}$$
On the other hand:
$$ Var(S_{11}) = 10/4 + (1/4)underbrace{(1-2(1/2)^{10})}_{P[S_{10}notin{0,10}]}$$
In general, the variance increases as $nrightarrowinfty$ to approach a
limiting value of $25$. It is possible to compute $P[S_i notin {0,10}]$ for all $i$ (for example, by taking powers of a transition probability matrix), but this calculation is more involved.
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
Modeling the process
Define $mathcal{S}$ as the set of possible values for the Markov chain:
$$mathcal{S} = {0, 0.5, 1, 1.5, …, 9.5, 10}$$
Note that $S_0=5$ and $S_n in mathcal{S}$ for all $n in {0, 1, 2, …}$.
We have
$$S_{n+1} = S_n + A_n quad forall n in {0, 1, 2, ...} $$
where
$$ A_n = left{ begin{array}{ll}
(1/2)B_n &mbox{ if $S_n notin {0, 10}$} \
0 & mbox{ otherwise}
end{array}
right.$$
where ${B_n}_{n=0}^{infty}$ is an i.i.d. sequence with $P[B_n=1]=P[B_n=-1]=1/2$.
Then
$$boxed{E[A_n|S_n=s] = 0 quad, forall s in mathcal{S}} quad (Eq. 1) $$
Mean
So for each $n in {0, 1, 2, ...}$ we have
begin{align}
E[S_{n+1}] &overset{(a)}{=} sum_{s in mathcal{S}}E[S_{n+1}|S_n=s]P[S_n=s] \
&overset{(b)}{=} sum_{s in mathcal{S}}E[S_n + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}E[s + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}(s + E[A_n|S_n=s])P[S_n=s] \
&overset{(c)}{=} sum_{s in mathcal{S}}sP[S_n=s] \
&overset{(d)}{=} E[S_n]
end{align}
where (a) holds by the law of total expectation; (b) holds by the fact $S_{n+1}=S_n+A_n$; (c) holds by Eq. (1); (d) holds by definition of expectation.
Since $E[S_0]=5$ we conclude:
$$boxed{E[S_n]=5 quad forall n in {0, 1, 2, … }}$$
Limiting variance
We know $E[S_n]=5$ for all $n$ and so
$$Var(S_n) = E[(S_n-5)^2] = sum_{s in mathcal{S}}(s-5)^2P[S_n=s] $$
Since the process is equally likely to end up at state $0$ or $10$ we have
begin{align}
lim_{nrightarrowinfty} P[S_n=0] &= 1/2\
lim_{nrightarrowinfty} P[S_n=10] &= 1/2\
lim_{nrightarrowinfty} P[S_n=s] &= 0 quad forall s notin {0, 10}
end{align}
so
$$ boxed{lim_{nrightarrowinfty} Var(S_n) = (0-5)^2(1/2) + (10-5)^2(1/2) = 25} $$
Details on variance
Squaring the equation $S_{n+1} = S_n + A_n$ gives
$$S_{n+1}^2 = (S_n+A_n)^2 = S_n^2 + 2S_nA_n + A_n^2 $$
So
$$E[S_{n+1}^2|S_n] = S_n^2 + 2S_nE[A_n|S_n] + E[A_n^2|S_n] = S_n^2 + 0 + (1/4)1_{{S_n notin{0, 10}}}$$
where $1_{{S_n notin{0, 10}}}$ is an indicator function that is 1 if $S_n notin {0,10}$ and is 0 else. So
$$E[S_{n+1}^2] = E[S_n^2] + (1/4)P[S_n notin {0,10}]$$
Subtracting 25 from both sides gives
$$ Var(S_{n+1}) = Var(S_n) + (1/4)P[S_n notin {0,10}]$$
and $Var(S_0)=0$ so
$$ boxed{Var(S_n) = (1/4)sum_{i=0}^{n-1} P[S_i notin {0,10}] quad forall nin {1, 2, 3, ...} } $$
Since $P[S_i notin {0,10}] = 1$ for $i in {0, 1, 2, 3, ..., 9}$ we have $$boxed{Var(S_1)=1/4, Var(S_2)=2/4, Var(S_3) = 3/4, ..., Var(S_{10})= 10/4}$$
On the other hand:
$$ Var(S_{11}) = 10/4 + (1/4)underbrace{(1-2(1/2)^{10})}_{P[S_{10}notin{0,10}]}$$
In general, the variance increases as $nrightarrowinfty$ to approach a
limiting value of $25$. It is possible to compute $P[S_i notin {0,10}]$ for all $i$ (for example, by taking powers of a transition probability matrix), but this calculation is more involved.
Modeling the process
Define $mathcal{S}$ as the set of possible values for the Markov chain:
$$mathcal{S} = {0, 0.5, 1, 1.5, …, 9.5, 10}$$
Note that $S_0=5$ and $S_n in mathcal{S}$ for all $n in {0, 1, 2, …}$.
We have
$$S_{n+1} = S_n + A_n quad forall n in {0, 1, 2, ...} $$
where
$$ A_n = left{ begin{array}{ll}
(1/2)B_n &mbox{ if $S_n notin {0, 10}$} \
0 & mbox{ otherwise}
end{array}
right.$$
where ${B_n}_{n=0}^{infty}$ is an i.i.d. sequence with $P[B_n=1]=P[B_n=-1]=1/2$.
Then
$$boxed{E[A_n|S_n=s] = 0 quad, forall s in mathcal{S}} quad (Eq. 1) $$
Mean
So for each $n in {0, 1, 2, ...}$ we have
begin{align}
E[S_{n+1}] &overset{(a)}{=} sum_{s in mathcal{S}}E[S_{n+1}|S_n=s]P[S_n=s] \
&overset{(b)}{=} sum_{s in mathcal{S}}E[S_n + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}E[s + A_n|S_n=s]P[S_n=s] \
&= sum_{s in mathcal{S}}(s + E[A_n|S_n=s])P[S_n=s] \
&overset{(c)}{=} sum_{s in mathcal{S}}sP[S_n=s] \
&overset{(d)}{=} E[S_n]
end{align}
where (a) holds by the law of total expectation; (b) holds by the fact $S_{n+1}=S_n+A_n$; (c) holds by Eq. (1); (d) holds by definition of expectation.
Since $E[S_0]=5$ we conclude:
$$boxed{E[S_n]=5 quad forall n in {0, 1, 2, … }}$$
Limiting variance
We know $E[S_n]=5$ for all $n$ and so
$$Var(S_n) = E[(S_n-5)^2] = sum_{s in mathcal{S}}(s-5)^2P[S_n=s] $$
Since the process is equally likely to end up at state $0$ or $10$ we have
begin{align}
lim_{nrightarrowinfty} P[S_n=0] &= 1/2\
lim_{nrightarrowinfty} P[S_n=10] &= 1/2\
lim_{nrightarrowinfty} P[S_n=s] &= 0 quad forall s notin {0, 10}
end{align}
so
$$ boxed{lim_{nrightarrowinfty} Var(S_n) = (0-5)^2(1/2) + (10-5)^2(1/2) = 25} $$
Details on variance
Squaring the equation $S_{n+1} = S_n + A_n$ gives
$$S_{n+1}^2 = (S_n+A_n)^2 = S_n^2 + 2S_nA_n + A_n^2 $$
So
$$E[S_{n+1}^2|S_n] = S_n^2 + 2S_nE[A_n|S_n] + E[A_n^2|S_n] = S_n^2 + 0 + (1/4)1_{{S_n notin{0, 10}}}$$
where $1_{{S_n notin{0, 10}}}$ is an indicator function that is 1 if $S_n notin {0,10}$ and is 0 else. So
$$E[S_{n+1}^2] = E[S_n^2] + (1/4)P[S_n notin {0,10}]$$
Subtracting 25 from both sides gives
$$ Var(S_{n+1}) = Var(S_n) + (1/4)P[S_n notin {0,10}]$$
and $Var(S_0)=0$ so
$$ boxed{Var(S_n) = (1/4)sum_{i=0}^{n-1} P[S_i notin {0,10}] quad forall nin {1, 2, 3, ...} } $$
Since $P[S_i notin {0,10}] = 1$ for $i in {0, 1, 2, 3, ..., 9}$ we have $$boxed{Var(S_1)=1/4, Var(S_2)=2/4, Var(S_3) = 3/4, ..., Var(S_{10})= 10/4}$$
On the other hand:
$$ Var(S_{11}) = 10/4 + (1/4)underbrace{(1-2(1/2)^{10})}_{P[S_{10}notin{0,10}]}$$
In general, the variance increases as $nrightarrowinfty$ to approach a
limiting value of $25$. It is possible to compute $P[S_i notin {0,10}]$ for all $i$ (for example, by taking powers of a transition probability matrix), but this calculation is more involved.
edited Nov 23 at 19:51
answered Nov 23 at 19:40
Michael
13.2k11325
13.2k11325
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3009239%2fexpected-value-and-variance-of-step-of-random-walk-with-barriers%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
If you have simulated, I'm sure you can guess the (very simple) value of $E[S_n]$. The proper way to simulate that is to run multiple (say, 10000 or more) experiments, get a time series for each experiment $i$ to obtain the vector $(S_0^{(i)}, S_1^{(i)}, S_2^{(i)}, ..., S_{100}^{(i)})$, average those values and plot the averaged values $frac{1}{10000}sum_{i=1}^{10000}S_n^{(i)}$ versus $n in {0, 1, 2, ..., 100}$.
– Michael
Nov 22 at 15:25
Yes I did, but now I want to know the theoretical results
– Marco All-in Nervo
Nov 22 at 15:34
So from those experiments you should be able to guess the value of $E[S_n]$ for each $n$ and your guess is...
– Michael
Nov 22 at 15:34
My guess is 5 but I want to know if it agrees with the theory
– Marco All-in Nervo
Nov 22 at 15:56
Yes $E[S_n]=5$ for all $n$ since this is a martingale, $S_{n+1} = S_n + A_n$ where $A_n=0$ if $S_n in {0, 10}$ and $E[A_n|S_n]=0$ regardless the value of $S_n$. You might want to provide your simulated guesses for $E[S_{50}]$ and $Var(S_{50})$ in the question itself. Here I assume coin flips are independent and equally likely to be heads or tails.
– Michael
Nov 22 at 16:02