Two-look Gaussian channel
$begingroup$
I'm reading through a solution from Elements of Information Theory by Thomas A. Cover. This is the two-look Gaussian channel, where the input to the channel is $X$ and the output is $(Y_1, Y_2)$.
$Y_1 = X + Z_1$
$Y_2 = X + Z_2$
where $(Z_1, Z_2) = Normal(0, K)$, where $K = begin{bmatrix} N & N rho \ N rho & N end{bmatrix}$
There is a power constraint on $X$ : $Var(X) = P$
Now, I understand that the channel capacity is maximized when the distribution of $X$ is Gaussian - $Normal (0, P)$. And since $(Y_1, Y_2)$ are combinations of $X$ and $(Z_1, Z_2)$, they must also be normally distributed.
The solution says that $(Y_1, Y_2)$ is distributed as $Normalleft(
0, begin{bmatrix} N + P & N rho + P \ N rho + P & N + P end{bmatrix} right)$
How is this distribution of $(Y_1, Y_2)$ obtained? Why does $P$ get added to all elements of the covariance matrix?
probability random-variables information-theory entropy
asked Dec 5 '18 at 3:33
User42User42
334
$endgroup$
add a comment |
$begingroup$
I'm reading through a solution from Elements of Information Theory by Thomas A. Cover. This is the two-look Gaussian channel, where the input to the channel is $X$ and the output is $(Y_1, Y_2)$.
$Y_1 = X + Z_1$
$Y_2 = X + Z_2$
where $(Z_1, Z_2) = Normal(0, K)$, where $K = begin{bmatrix} N & N rho \ N rho & N end{bmatrix}$
There is a power constraint on $X$ : $Var(X) = P$
Now, I understand that the channel capacity is maximized when the distribution of $X$ is Gaussian - $Normal (0, P)$. And since $(Y_1, Y_2)$ are combinations of $X$ and $(Z_1, Z_2)$, they must also be normally distributed.
The solution says that $(Y_1, Y_2)$ is distributed as $Normalleft(
0, begin{bmatrix} N + P & N rho + P \ N rho + P & N + P end{bmatrix} right)$
How is this distribution of $(Y_1, Y_2)$ obtained? Why does $P$ get added to all elements of the covariance matrix?
probability random-variables information-theory entropy
asked Dec 5 '18 at 3:33
User42User42
334
$endgroup$
1
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
1
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
1
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09
add a comment |
$begingroup$
I'm reading through a solution from Elements of Information Theory by Thomas A. Cover. This is the two-look Gaussian channel, where the input to the channel is $X$ and the output is $(Y_1, Y_2)$.
$Y_1 = X + Z_1$
$Y_2 = X + Z_2$
where $(Z_1, Z_2) = Normal(0, K)$, where $K = begin{bmatrix} N & N rho \ N rho & N end{bmatrix}$
There is a power constraint on $X$ : $Var(X) = P$
Now, I understand that the channel capacity is maximized when the distribution of $X$ is Gaussian - $Normal (0, P)$. And since $(Y_1, Y_2)$ are combinations of $X$ and $(Z_1, Z_2)$, they must also be normally distributed.
The solution says that $(Y_1, Y_2)$ is distributed as $Normalleft(
0, begin{bmatrix} N + P & N rho + P \ N rho + P & N + P end{bmatrix} right)$
How is this distribution of $(Y_1, Y_2)$ obtained? Why does $P$ get added to all elements of the covariance matrix?
probability random-variables information-theory entropy
asked Dec 5 '18 at 3:33
User42User42
334
$endgroup$
I'm reading through a solution from Elements of Information Theory by Thomas A. Cover. This is the two-look Gaussian channel, where the input to the channel is $X$ and the output is $(Y_1, Y_2)$.
$Y_1 = X + Z_1$
$Y_2 = X + Z_2$
where $(Z_1, Z_2) = Normal(0, K)$, where $K = begin{bmatrix} N & N rho \ N rho & N end{bmatrix}$
There is a power constraint on $X$ : $Var(X) = P$
Now, I understand that the channel capacity is maximized when the distribution of $X$ is Gaussian - $Normal (0, P)$. And since $(Y_1, Y_2)$ are combinations of $X$ and $(Z_1, Z_2)$, they must also be normally distributed.
The solution says that $(Y_1, Y_2)$ is distributed as $Normalleft(
0, begin{bmatrix} N + P & N rho + P \ N rho + P & N + P end{bmatrix} right)$
How is this distribution of $(Y_1, Y_2)$ obtained? Why does $P$ get added to all elements of the covariance matrix?
probability random-variables information-theory entropy
probability random-variables information-theory entropy
asked Dec 5 '18 at 3:33
User42User42
334
asked Dec 5 '18 at 3:33
User42User42
334
asked Dec 5 '18 at 3:33
User42User42
334
asked Dec 5 '18 at 3:33
User42User42
334
asked Dec 5 '18 at 3:33
User42User42
334
334
1
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
1
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
1
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09
add a comment |
1
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
1
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
1
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09
1
1
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
1
1
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
1
1
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
It is known that the Gaussian noises $Z_1, Z_2$ are zero mean. If we try to find the variance of each of the $Y_1, Y_2$ pairs, we could write the elements of the covariance matrix $L$ of the form:
$$
L_{11} = E[Y_1Y_1] = E[(X+Z_1)(X+Z_1)] = E[X^2] + 2E[XZ_1] + E[Z^2] = P + 0 + N = P+N
$$
$$
L_{12} = E[Y_1Y_2] = E[(X+Z_1)(X+Z_2)] = E[X^2] + E[XZ_1] + E[XZ_2] + E[Z_1Z_2] = P + 0 + 0 + rho N = P+ rho N
$$
The other two elements of the covariance matrix can be obtained through symmetry.
answered Dec 5 '18 at 15:47
User42User42
334
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3026574%2ftwo-look-gaussian-channel%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
It is known that the Gaussian noises $Z_1, Z_2$ are zero mean. If we try to find the variance of each of the $Y_1, Y_2$ pairs, we could write the elements of the covariance matrix $L$ of the form:
$$
L_{11} = E[Y_1Y_1] = E[(X+Z_1)(X+Z_1)] = E[X^2] + 2E[XZ_1] + E[Z^2] = P + 0 + N = P+N
$$
$$
L_{12} = E[Y_1Y_2] = E[(X+Z_1)(X+Z_2)] = E[X^2] + E[XZ_1] + E[XZ_2] + E[Z_1Z_2] = P + 0 + 0 + rho N = P+ rho N
$$
The other two elements of the covariance matrix can be obtained through symmetry.
answered Dec 5 '18 at 15:47
User42User42
334
$endgroup$
add a comment |
$begingroup$
It is known that the Gaussian noises $Z_1, Z_2$ are zero mean. If we try to find the variance of each of the $Y_1, Y_2$ pairs, we could write the elements of the covariance matrix $L$ of the form:
$$
L_{11} = E[Y_1Y_1] = E[(X+Z_1)(X+Z_1)] = E[X^2] + 2E[XZ_1] + E[Z^2] = P + 0 + N = P+N
$$
$$
L_{12} = E[Y_1Y_2] = E[(X+Z_1)(X+Z_2)] = E[X^2] + E[XZ_1] + E[XZ_2] + E[Z_1Z_2] = P + 0 + 0 + rho N = P+ rho N
$$
The other two elements of the covariance matrix can be obtained through symmetry.
answered Dec 5 '18 at 15:47
User42User42
334
$endgroup$
add a comment |
$begingroup$
It is known that the Gaussian noises $Z_1, Z_2$ are zero mean. If we try to find the variance of each of the $Y_1, Y_2$ pairs, we could write the elements of the covariance matrix $L$ of the form:
$$
L_{11} = E[Y_1Y_1] = E[(X+Z_1)(X+Z_1)] = E[X^2] + 2E[XZ_1] + E[Z^2] = P + 0 + N = P+N
$$
$$
L_{12} = E[Y_1Y_2] = E[(X+Z_1)(X+Z_2)] = E[X^2] + E[XZ_1] + E[XZ_2] + E[Z_1Z_2] = P + 0 + 0 + rho N = P+ rho N
$$
The other two elements of the covariance matrix can be obtained through symmetry.
answered Dec 5 '18 at 15:47
User42User42
334
$endgroup$
It is known that the Gaussian noises $Z_1, Z_2$ are zero mean. If we try to find the variance of each of the $Y_1, Y_2$ pairs, we could write the elements of the covariance matrix $L$ of the form:
$$
L_{11} = E[Y_1Y_1] = E[(X+Z_1)(X+Z_1)] = E[X^2] + 2E[XZ_1] + E[Z^2] = P + 0 + N = P+N
$$
$$
L_{12} = E[Y_1Y_2] = E[(X+Z_1)(X+Z_2)] = E[X^2] + E[XZ_1] + E[XZ_2] + E[Z_1Z_2] = P + 0 + 0 + rho N = P+ rho N
$$
The other two elements of the covariance matrix can be obtained through symmetry.
answered Dec 5 '18 at 15:47
User42User42
334
answered Dec 5 '18 at 15:47
User42User42
334
answered Dec 5 '18 at 15:47
User42User42
334
answered Dec 5 '18 at 15:47
User42User42
334
334
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3026574%2ftwo-look-gaussian-channel%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
So you know $(Y_1,Y_2)$ has a normal distribution. You only need to find it's mean vector and the correlations matrix. But remember that for Gaussian Channel $X$ and $Z$ are assumed not to be correlated. . . So try to compute the variance and correlations by any formula you know, use the correlation 0 to cancel terms...
$endgroup$
– Joaquin San
Dec 5 '18 at 3:44
1
$begingroup$
Thanks, got it!
$endgroup$
– User42
Dec 5 '18 at 3:48
1
$begingroup$
If you've found the answer, please either wrote yourself it as answer and accept it, or delete the question, so it does not appear as waiting for an answer.
$endgroup$
– leonbloy
Dec 5 '18 at 14:09