There have been a number of different positions amongst Christians as to what happens to the souls of children who die. Augustine postulated that unbaptized infants were punished eternally in hell, but to a lesser degree. His views softened over time. For centuries, the Roman Catholic Church unofficially taught that the souls of deceased children go to a place called Limbo. The Greek fathers did not discuss this topic at length, but some believed children who died went to some place distinct from both heaven and hell. The stance defended here is that the destination of little children who pass away enter into the presence of God in heaven.
If Christ is the only way to heaven as He claims to be, faith is required in order to follow Him, yet babies are not able to place their trust in anyone, then it might seem to some that those who die are excluded from heaven. However, we would be hard pressed to find Christians who actually believe that today. The idea is not palatable to us at all. While not explicitly answered in Scripture, there are a number of clues which seem to indicate that babies are recipients of God's grace and excluded from eternal punishment.
Scripture implicitly affirms the concept that we call the age of accountability (Deuteronomy 1:39; Isaiah 7:15-16). This is relative due to the fact that everybody develops differently. Thus, there exists a period of time during childhood in which God in His mercy does not count sins against a person due to lacking reason and understanding. While babies are born with a sin nature, they still cannot actually commit acts of evil.
Jesus Christ stated that the kingdom of God is not only for little ones, but also for those who become like children in their faith (Mark 10:13-15; Luke 18:15-17). If babies were to be sentenced to hell upon death, then why did He welcome them? The gospels record Christ having compassion on children as He called them to His side and sat them at His feet in the crowds (Matthew 18:3; 19:14-15). If they were to be sentenced to hell by God for original sin, then why did Jesus commend their simplicity? John the Baptist is said to have been indwelt with the Spirit of God even in the womb (Luke 1:15), and receiving the Spirit is always viewed as evidence of regeneration (Luke 1:41, 67; Acts 9:17).
Consider the mourning of King David when the Lord took away his newborn child as discipline for acts of adultery and murder (2 Samuel 12:22-23). Even in the midst of his selfish and brutish conduct, we see God exercising mercy as his own life was spared. The Lord is in charge of life itself. The text of 2 Samuel affirms that both would go to the same place at the moment of physical death. Moreover, Hebrews 11:32-33 tells us David entered into the presence of God with the rest of the saints. Why should we believe that the souls of deceased children go to some place other than heaven?
God would not condemn to hell somebody who never even had a chance to live. We must also keep in mind those who have severe cognitive disabilities. Though it is a teaching of Scripture that God has inscribed a moral law into the hearts of men, every person needs to have the mental capacity to act. Each man will be judged according to his own conduct (2 Corinthians 5:10; 1 Peter 1:17). Babies cannot do anything on their own. What could God judge them for? Our hope lies in the goodness and graciousness of God. That is our one and only hope, from here to eternity. Children are by no means an exception to that rule.