Jtdylktuy

Let $A in M_{2times 2}(mathbb{C})$ and $det(A)=1DeclareMathOperator{tr}{tr}$

Prove that $det(A^2+A-I_2)+det(A^2+I_2) = 5$

using Cayley-Hamilton Theorem $A^2-tr(A)A+det(A)I_2=0$

$detbig(tr(A)A-det(A)I_2-I_2big) + detbig(A(A+A^{-1})big)=5$

$detbig(tr(A)A-I_2(det(A)+1)big)+det(A+A^{-1})=5$

using https://math.stackexchange.com/q/1937052 $det(A+B)=det A+det B+det A⋅tr(A^{-1}B) $

$tr(A)det(A)-big(det(A)+1big)det(I_2)+tr(A)big(-det(A)-1big)^{-1}tr(I^{-1}A)+det(A)+det(A^{-1})+tr(A^2)=5
$

$tr(A)-2-tr(A)^20.5+1+1+tr(A^2)=5$

$tr(A)-tr(A)^20.5+trbig(tr(A)A-det(A)I_2big)=5$

$tr(A)-tr(A)^20.5+tr(A)^2-tr(I_2)=5$

$tr(A)+0.5tr(A)^2-2=5$

This is where I am stuck. And also I don't know how to prove the identity which I cited.

You're on the good track.

Let $t=operatorname{tr}(A)$. Then, from $det(A)=1$, we know by Cayley-Hamilton that
$$
A^2-tA+I_2=0
$$
Thus
$$
A^2+A-I_2=tA-I_2+A-I_2=(t+1)A-2I_2
$$
and
$$
A^2+I_2=tA
$$
Thus $det(A^2+I_2)=t^2det(A)=t^2$.

The formula you cite tells you that
$$
det(X+Y)=det(X)+det(Y)+det(X)operatorname{tr}(X^{-1}Y)
$$
and we can apply it to $X=-2I_2$, $Y=(t+1)A$. Thus
$$
det(X)=4,qquad det(Y)=(t+1)^2,qquad X^{-1}Y=-frac{1}{2}(t+1)A,
qquad operatorname{tr}(X^{-1}Y)=-frac{1}{2}t(t+1)
$$
Thus the expression on the left-hand side becomes
$$
4+(t+1)^2-2t(t+1)+t^2=5
$$

Let $p(x):=det(x,I-A)$ be the characteristic polynomial of $A$, where $I$ is a short-hand notation for $I_2$. Then, show that
$$det(A^2+A-I)+det(A^2+I)=pleft(phiright),pleft(bar{phi}right)+p(+text{i}),p(-text{i}),,$$
where $phi:=dfrac{-1+sqrt{5}}{2}$ and $bar{phi}:=dfrac{-1-sqrt{5}}{2}$.
Since $det(A)=1$, $p(x)=x^2-bx+1$ for some $binmathbb{C}$.

Because $phi$ and $bar{phi}$ are roots of the polynomial $x^2+x-1$, we have
$$p(t)=t^2-bt+1=(1-t)-bt+1=2-(1+b)t$$
for $tin{phi,bar{phi}}$. This shows that
$$p(phi),p(bar{phi})=big(2-(1+b)phibig),big(2-(1+b)bar{phi}big)=4-2(1+b)(phi+bar{phi})+(1+b)^2phibar{phi},.$$
That is,
$$p(phi),p(bar{phi})=4-2(1+b)(-1)+(1+b)^2(-1)=5-b^2,.$$

Similarly, $+text{i}$ and $-text{i}$ are roots of the polynomial $x^2+1$, we have
$$p(t)=t^2-bt+1=(-1)-bt+1=-bt$$
for $tin{+text{i},-text{i}}$. Hence,
$$p(+text{i}),p(-text{i})=(-btext{i})(+btext{i})=b^2,.$$
The claim follows immediately.

Below is a proof of the cited identity, i.e., $$det(A+B)=det(A)+det(B)+det(A),text{tr}(A^{-1}B)$$ for any $A,Bintext{Mat}_{2times 2}(K)$ such that $A$ is invertible. Here, $K$ is any field.

Since $A$ is invertible,
$$det(A+B)=det(A),det(I+A^{-1}B),.$$
The characteristic polynomial of $A^{-1}B$ is
$$q(x):=det(x,I-A^{-1}B)=x^2-text{tr}(A^{-1}B),x+det(A^{-1}B),.$$
Thus,
$$det(I+A^{-1}B)=(-1)^2,det(-I-A^{-1}B)=q(-1)=1+text{tr}(A^{-1}B)+det(A^{-1}B),.$$
Ergo,
$$begin{align}det(A+B)&=det(A),Big(1+text{tr}(A^{-1}B)+det(A^{-1}B)Big)\&=det(A)+det(A),det(A^{-1}B)+det(A),text{tr}(A^{-1}B)\&=det(A)+det(B)+det(A),text{tr}(A^{-1}B),.end{align}$$

By brute force, calling

$$
A = left(begin{array}{cc}a_1 & a_2 \ a_3 & a_4end{array}right)
$$

we have

$$
det A = 1to a_1 a_4 = 1 + a_2 a_3
$$

and also

$$
detleft(A^2+A-I_2right)+detleft(A^2+I_2right) = 2 a_4^2 a_1^2+a_4 a_1^2+a_4^2 a_1-a_2 a_3 a_1-4 a_2 a_3 a_4 a_1+a_4 a_1-a_1+2 a_2^2 a_3^2-a_2 a_3-a_2 a_3 a_4-a_4+2
$$

and after substituting $ a_1 a_4 = 1 + a_2 a_3$ we obtain the result

$$
detleft(A^2+A-I_2right)+detleft(A^2+I_2right) = 5
$$