God commanded the sign and seal of the covenant be given to infants, which was circumcision but is now baptism. In line with this command, the vast majority of the church throughout the entire history of the church has baptized the children of believers. To not baptize the children of believers comes from the relatively recent Anabaptist movement (16th century), and has, over time, leavened its way into various Protestant traditions.
Clearly, there is much more that could be addressed concerning infant baptism, including answers to the typical objections. For further reading I recommend these books: Bryan Chapell, Why Do We Baptize Infants?; Robert Rayburn, What About Baptism?; Jay Adams, The Meaning and Mode of Baptism; and Francis Schaeffer, Baptism. If you have any questions about the sacrament, please feel free to ask me or any of the elders.
In Christ, Roger