(b) $a^2 + 2b$
For $p(x) = x^2 - ax - b$: $\alpha + \beta = a$ and $\alpha\beta = -b$.
$$\alpha^2 + \beta^2 = (\alpha+\beta)^2 - 2\alpha\beta = a^2 - 2(-b) = a^2 + 2b$$
Source: Chapter 2, Section 2.3
Use the identity $\alpha^2+\beta^2=(\alpha+\beta)^2-2\alpha\beta$. From the polynomial $x^2-ax-b$ (comparing with $ax^2+bx+c$): sum of zeroes $=a$, product of zeroes $=-b$. Substituting gives $a^2+2b$. Watch the sign of the product carefully — it is a common error point.