Or it could be the proper amount of lining material- preferably linen or hemp, to stiffen it just the right amount. Let's face it- these things are not lightweight garments, and the weight/mass of the fabric is what helps them to keep their shape.
It would appear that the U shaped insert may be too short, or that the back of your doublet is too low - or a combination thereof. One or the other (or both) must be higher in order to support the shape of the garment at the point of the shoulders. The U-insert (Brustfleck) can reach to as high as your collar bones. The back of the doublet can touch the nape of your neck. This will help pull your shoulder seams in and prevent them from falling off onto the top of your arms.
If you do what Cherilyn says and mae it longer, that goes a long way. Without the placket in the brustfleck, the "u" should be as high as possible on the shoulder, maybe as much as 2-3 inches from the base of the neck, and should rest just outside the nipples by no more than an inch or so. On mine, I have an open side and a closed side; a dedicated stitch on my right and a combination of hook and eye with a top tie on the left so that I can open it up. Having it stitched nice and flat stretches it to the right size, and that way can fit over our expanding bellies properly! BTW, three layers are sufficient:
top wool color+bottom wool color+cotton backing
I would avoid cotton, as linen and hemp breathe sooo much better. But Ben is right- 3 layers of fabric ought to do it. Hard part is finding heavy linen/hemp- try Hemp Traders on line, the 12 oz dark is awesome, I just got 12 yards!