That Makes it Easy

I just had my mind blown.  In my summer course, Random Processes, I had a homework problem regarding estimation theory and determining 2nd order statistics about linear combinations of jointly Gaussian random variables. Part of the problem is to determine the Covariance of these linear combinations. As worked away I had an incredible realization: Covariance behaves like an inner product!

Check it out, it satisfies the properties of an inner produce on the probability space:
1) It's bilinear
\[
Cov\lbrack\gamma A + \delta B,Z\rbrack = \gamma Cov\lbrack A,Z\rbrack +\delta Cov\lbrack B,Z\rbrack\]
2) It's symmetric
\[
Cov\lbrack A,B\rbrack = Cov\lbrack B,A\rbrack\]
3) It's positive semi-definite (or what I think is better said non-negative definite, but that's for another post)
\[
Var\lbrack A\rbrack =Cov\lbrack A,A\rbrack\ge 0\]
Where $Cov\lbrack A,A\rbrack = 0$ implies that $A$ is a constant random variable.

So it's not exactly an inner product--there isn't the existence of a single zero. Rather, all constant random variables behave like zero. Apparently, this defines a Quotient Space--a vector space with an subspace $N$ that forms an equivalence class with $0$--and Covariance is an inner product over such a space.

Anyway, I looked this up on Wikipedia and, sure enough, there is a little section titled "Relationship to inner products" (which pointed me towards the article on quotient spaces). I guess it's not any huge discovery. I wasn't expecting that. But I am a little peeved that Variance and Covariance were never taught this way, or that this cool perspective was never even mentioned. I feel as though I pushed my understanding and intuition of Covariance way ahead by seeing this little change of face, and it would have been a huge help when taking probability courses in the past.