why do babies hate sleeping?

I'm sitting at what feels like a fork in the road.  Technically, I'm sitting on the couch in the living room listening to my sweet baby boy cry his eyes out in his bed. 

We aren't even trying to do the cry-it-out method.  I rocked all my babies to sleep for the first year or longer of their lives.  I don't mind doing it, IF they stay asleep and that brings us to the problem.  Jackson is not staying asleep.

Every night we deal with one of the following:

A.   he wakes up less than an hour after laying him down for no good reason, but usually goes back down fairly easily

B.    he wakes up between 1am-3am, we feed him and he is wide awake and ready to play.  It usually takes over an hour to get him settled down and back to sleep.

Both scenarios are not good.  I'm not sure what to do at this point. 

I have two older, busy kids, so it's hard to keep a routine, but he is usually bathed, fed, and rocked between eight and nine o'clock. 
He has never slept all the way through the night, which I know at 8.5 months he perfectly capable of doing.  I just don't know what to do. 

I feel like every decision I make is turning him into a bad sleeper.  Should I let him cry? For how long?  We usually leave him for about ten or fifteen minutes and he only gets louder and louder and angrier and angrier. 

I remember with Abigail we tried the cry-it-out method and she was literally scared of her room and bed for a few days.  We only let her cry for thirty minutes, but it was scaring to her. 

Austin was and is my awesome sleeper.  Right around age one he lost all interest in being rocked to sleep.  We could just lay him down and he would go straight to sleep and sleep ten or more hours. 

The BIG, HUGE, MAJOR difference between my first two kiddos and Jackson is the PACI.  Jackson never took to the paci and believe me I tried hard.  My first two were paci babies big time.  I LOVED it.  It helped sooth them at night, it kept them quiet in public, it just made my life easier.  Breaking them of it was a little hard, but not as hard as a baby that won't sleep.



Here is Abigail at 1 with her paci.




Here is Austin at 1 with his best friends, the paci and the blankie.




And here's Jack-Jack with no paci or blanket

 
Jackson hasn't developed any self-soothing habits as far as I can tell.  Austin also had a blanket that he is still very attached to.  Jackson doesn't have anything like that.
Why is sleeping so hard for babies?  The type of humans that need sleep the most and they hate sleeping!!  I just don't get it. 

While I was typing this Brent got Jackson out of his bed.  I just don't want to teach him that crying will get him what he wants, but I also don't want him to feel insecure and abandoned.  Or what if something is bothering him, like ears or teeth?  I don't think so, but what if? 

Being a parent is so hard, even the third time around!!