> A:=Matrix([[a,b],[c,d]]);

> B:=Matrix([[p,q],[r,s]]);

> with(linalg);

Warning, the protected names norm and trace have been redefined and unprotected

> trace(A.B);

> trace(B.A);

We find trace of AB=trace of BA .

> a*p+b*r+c*q+d*s;

> (A.B-B.A).(A.B-B.A);

>

> expand((b*r-c*q)*(a*q+b*s-p*b-q*d)+(a*q+b*s-p*b-q*d)*(c*q-b*r));

> expand((c*p+d*r-r*a-s*c)*(b*r-c*q)+(c*q-b*r)*(c*p+d*r-r*a-s*c));

> expand((b*r-c*q)^2+(a*q+b*s-p*b-q*d)*(c*p+d*r-r*a-s*c)-((a*q+b*s-p*b-q*d)*(c*p+d*r-r*a-s*c)+(c*q-b*r)^2));

We find (AB-BA)(AB-BA) is scalar.

> C:=Matrix([[1,0],[0,2]]);

> A.C-C.A;

Find all marices commuting with are

>

> A^(-1).B.A;

> simplify(expand((d*p/(a*d-b*c)-b*r/(a*d-b*c))*b+(d*q/(a*d-b*c)-b*s/(a*d-b*c))*d));

> -(-b*d*p+b^2*r-d^2*q+d*b*%?)/(a*d-b*c);

> expand((-c*p/(a*d-b*c)+a*r/(a*d-b*c))*a+(-c*q/(a*d-b*c)+a*s/(a*d-b*c))*c);

We find if a=b=c=d=0 then A^(-1)BA is diagonal. But A <>0 so A^(-1)BA can never be diagonal.

> restart:

> S:=Matrix([[a,b],[c,d]]);

> A:=Matrix([[1,0],[1,1]]);

> S^(-1).A.S;

> expand((d/(a*d-b*c)-b/(a*d-b*c))*b-b*d/(a*d-b*c));

If b=0 then S^(-1).A.S triangular.