Why are the carboxylic acid groups of the amino acids so much more acidic (pka~ 2) than a carboxylic acid such as acetic acid (pka=4.76)?

The groups you mention are withdrawing electrons with inductive effect. Because there is no conjugation between the acids (pk-2 and pka 4.76), nitrogen makes it impossible to bind to the carboxyl group. Acids are affected in two ways, the first since the electron subtraction of charged oxygen molecules, and the second way by polarizing the O-H bond with carboxylic acid, this helps the protons to split easily.