Judeo-Christian tradition does affirm the scriptural concept that we call the age of accountability (Deuteronomy 1:39; Isaiah 7:15-16; James 4:17). This is relative due to the fact that everybody develops differently. Thus, there exists a period of time in childhood where God does not count sins against a person due to lacking reason and understanding. They have no knowledge of good and evil. While babies are born with a sin nature, they still cannot commit acts of evil. Additionally, our Lord Jesus Christ stated that the kingdom of God is not only for little ones, but also for those who become like children in their faith (Mark 10:13-15; Luke 18:15-17).
The God of the Bible is a God of love, mercy, righteousness, and truth. He would not condemn to hell somebody who never even had a chance to live, let alone a chance to accept the gospel and grow spiritually. We must also keep in mind those who have severe cognitive disabilities. Though it is a teaching of Scripture that God has inscribed a moral law into the hearts of all people, each individual nevertheless needs to be capable of understanding it. Each person will be judged according to his or her own conduct (2 Corinthians 5:10; 1 Peter 1:17). Babies can accomplish neither good nor evil. So what could God judge them for?
Of course, divine revelation does not seal every crevice of human perception and inquiry regarding what happens in the afterlife. But I still think believing that God allows the souls of innocent babies into heaven is the most reasonable stance to endorse.