Poisson adaptation of IRT model: how to interpret parameters?
$begingroup$
for my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get the perfect score of 0. With every extra step they need, their score increases by 1.
To measure the skill of the participants, we use an IRT (item response theory) model. Since the test isn't dichotomous, scores are discrete, scores on every item can range from 0 to $infty$, and scores seem roughly normally distributed, we adapted the IRT model to use a poisson distribution.
That is, we use the normal MaxLogLikelihood estimation, but instead of using the 2-parameter IRT fomula(1)
$$
P(X|a,b,theta) = {e^{a(theta - b)} over 1+ e^{a(theta - b)}}
$$
we calculate P according to the regular poisson formula (2)
$$
P(X=n) = {lambda^n e^{-lambda} over !n}
$$
where $lambda$ is... well, that's the question. Based on a textbook I have been using (3)
$$
lambda = e^{(b - atheta)}
$$
My questions are about the relationship between $b$, $lambda$ and $theta$:
- does the formula at (3) make sense, or should it be something else?
- in the 2plm IRT model (1) the roles of the parameters are very clear to me: $theta$ is assumed/defined to be N(0,1) so ranges roughly from -3 to 3. $b$ is defined such that if $b = theta$, the chance of getting a correct answer is exactly 50%. And $a$ defines the slope. Makes perfect sense. In the poisson model however, I lack these definitions. If $b$ still represents the difficulty of an item, and suppose $b = theta$, what $lambda$ should I expect? What does $a$ represent? I'm finding it hard to wrap my head around the conceptual meaning.
In short: keeping $theta$ defined as N(0,1), what would be the best/most sensible way to define the other parameters of a poisson-based IRT model?
p.s. first post here, feel free to give other feedback.
mathematical-modeling poisson-distribution
$endgroup$
add a comment |
$begingroup$
for my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get the perfect score of 0. With every extra step they need, their score increases by 1.
To measure the skill of the participants, we use an IRT (item response theory) model. Since the test isn't dichotomous, scores are discrete, scores on every item can range from 0 to $infty$, and scores seem roughly normally distributed, we adapted the IRT model to use a poisson distribution.
That is, we use the normal MaxLogLikelihood estimation, but instead of using the 2-parameter IRT fomula(1)
$$
P(X|a,b,theta) = {e^{a(theta - b)} over 1+ e^{a(theta - b)}}
$$
we calculate P according to the regular poisson formula (2)
$$
P(X=n) = {lambda^n e^{-lambda} over !n}
$$
where $lambda$ is... well, that's the question. Based on a textbook I have been using (3)
$$
lambda = e^{(b - atheta)}
$$
My questions are about the relationship between $b$, $lambda$ and $theta$:
- does the formula at (3) make sense, or should it be something else?
- in the 2plm IRT model (1) the roles of the parameters are very clear to me: $theta$ is assumed/defined to be N(0,1) so ranges roughly from -3 to 3. $b$ is defined such that if $b = theta$, the chance of getting a correct answer is exactly 50%. And $a$ defines the slope. Makes perfect sense. In the poisson model however, I lack these definitions. If $b$ still represents the difficulty of an item, and suppose $b = theta$, what $lambda$ should I expect? What does $a$ represent? I'm finding it hard to wrap my head around the conceptual meaning.
In short: keeping $theta$ defined as N(0,1), what would be the best/most sensible way to define the other parameters of a poisson-based IRT model?
p.s. first post here, feel free to give other feedback.
mathematical-modeling poisson-distribution
$endgroup$
add a comment |
$begingroup$
for my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get the perfect score of 0. With every extra step they need, their score increases by 1.
To measure the skill of the participants, we use an IRT (item response theory) model. Since the test isn't dichotomous, scores are discrete, scores on every item can range from 0 to $infty$, and scores seem roughly normally distributed, we adapted the IRT model to use a poisson distribution.
That is, we use the normal MaxLogLikelihood estimation, but instead of using the 2-parameter IRT fomula(1)
$$
P(X|a,b,theta) = {e^{a(theta - b)} over 1+ e^{a(theta - b)}}
$$
we calculate P according to the regular poisson formula (2)
$$
P(X=n) = {lambda^n e^{-lambda} over !n}
$$
where $lambda$ is... well, that's the question. Based on a textbook I have been using (3)
$$
lambda = e^{(b - atheta)}
$$
My questions are about the relationship between $b$, $lambda$ and $theta$:
- does the formula at (3) make sense, or should it be something else?
- in the 2plm IRT model (1) the roles of the parameters are very clear to me: $theta$ is assumed/defined to be N(0,1) so ranges roughly from -3 to 3. $b$ is defined such that if $b = theta$, the chance of getting a correct answer is exactly 50%. And $a$ defines the slope. Makes perfect sense. In the poisson model however, I lack these definitions. If $b$ still represents the difficulty of an item, and suppose $b = theta$, what $lambda$ should I expect? What does $a$ represent? I'm finding it hard to wrap my head around the conceptual meaning.
In short: keeping $theta$ defined as N(0,1), what would be the best/most sensible way to define the other parameters of a poisson-based IRT model?
p.s. first post here, feel free to give other feedback.
mathematical-modeling poisson-distribution
$endgroup$
for my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get the perfect score of 0. With every extra step they need, their score increases by 1.
To measure the skill of the participants, we use an IRT (item response theory) model. Since the test isn't dichotomous, scores are discrete, scores on every item can range from 0 to $infty$, and scores seem roughly normally distributed, we adapted the IRT model to use a poisson distribution.
That is, we use the normal MaxLogLikelihood estimation, but instead of using the 2-parameter IRT fomula(1)
$$
P(X|a,b,theta) = {e^{a(theta - b)} over 1+ e^{a(theta - b)}}
$$
we calculate P according to the regular poisson formula (2)
$$
P(X=n) = {lambda^n e^{-lambda} over !n}
$$
where $lambda$ is... well, that's the question. Based on a textbook I have been using (3)
$$
lambda = e^{(b - atheta)}
$$
My questions are about the relationship between $b$, $lambda$ and $theta$:
- does the formula at (3) make sense, or should it be something else?
- in the 2plm IRT model (1) the roles of the parameters are very clear to me: $theta$ is assumed/defined to be N(0,1) so ranges roughly from -3 to 3. $b$ is defined such that if $b = theta$, the chance of getting a correct answer is exactly 50%. And $a$ defines the slope. Makes perfect sense. In the poisson model however, I lack these definitions. If $b$ still represents the difficulty of an item, and suppose $b = theta$, what $lambda$ should I expect? What does $a$ represent? I'm finding it hard to wrap my head around the conceptual meaning.
In short: keeping $theta$ defined as N(0,1), what would be the best/most sensible way to define the other parameters of a poisson-based IRT model?
p.s. first post here, feel free to give other feedback.
mathematical-modeling poisson-distribution
mathematical-modeling poisson-distribution
asked Dec 12 '18 at 12:31
JumbomanJumboman
1
1
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
If you use the more commonly used IRT representation:
$$
lambda=e^{ a(theta-b)}
$$
and you fill in $b=theta$, you obtain:
$lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $theta$ represents inability, since a large value for $lambda$ represents a large number of tries.
If you would parameterize the model as
$$
lambda=e^{a(b-theta)}
$$
you would have a $theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3036625%2fpoisson-adaptation-of-irt-model-how-to-interpret-parameters%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If you use the more commonly used IRT representation:
$$
lambda=e^{ a(theta-b)}
$$
and you fill in $b=theta$, you obtain:
$lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $theta$ represents inability, since a large value for $lambda$ represents a large number of tries.
If you would parameterize the model as
$$
lambda=e^{a(b-theta)}
$$
you would have a $theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.
$endgroup$
add a comment |
$begingroup$
If you use the more commonly used IRT representation:
$$
lambda=e^{ a(theta-b)}
$$
and you fill in $b=theta$, you obtain:
$lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $theta$ represents inability, since a large value for $lambda$ represents a large number of tries.
If you would parameterize the model as
$$
lambda=e^{a(b-theta)}
$$
you would have a $theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.
$endgroup$
add a comment |
$begingroup$
If you use the more commonly used IRT representation:
$$
lambda=e^{ a(theta-b)}
$$
and you fill in $b=theta$, you obtain:
$lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $theta$ represents inability, since a large value for $lambda$ represents a large number of tries.
If you would parameterize the model as
$$
lambda=e^{a(b-theta)}
$$
you would have a $theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.
$endgroup$
If you use the more commonly used IRT representation:
$$
lambda=e^{ a(theta-b)}
$$
and you fill in $b=theta$, you obtain:
$lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $theta$ represents inability, since a large value for $lambda$ represents a large number of tries.
If you would parameterize the model as
$$
lambda=e^{a(b-theta)}
$$
you would have a $theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.
answered Dec 13 '18 at 10:35
StéphanieStéphanie
111
111
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3036625%2fpoisson-adaptation-of-irt-model-how-to-interpret-parameters%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown