Understanding the Limitations of Mathematical Reasoning in Large Language Models