Preheat the oven to 350°F and line a large baking sheet with parchment paper and set aside. If you only have a 9x13-inch baking sheet, you'll want to use two baking sheets for two batches.
In a medium bowl, whisk together 2 cups all-purpose flour, ¼ cup unsweetened cocoa powder, ¼ cup black cocoa powder, 1 (3.4oz) box instant chocolate fudge pudding mix, 2 tablespoons powdered milk, ½ teaspoon salt, and ¼ teaspoon baking soda. Set aside.
Add together 1 cup granulated sugar and ½ cup salted butter to a large bowl or stand mixer fitted with a paddle attachment and cream for 1 minute.
Add in 1 large egg and beat until combined. Beat in 1 additional large egg yolk, 2 ounces whole milk, and 1½ teaspoons pure vanilla extract and mix just until combined.
Add half the dry ingredients to the wet ingredients and beat just until incorporated. Then, add the remaining dry ingredients and mix until thoroughly combined. If dry ingredients are settling at the bottom of the bowl, DO NOT keep mixing with your mixer, instead use your hands to mix the ingredients into the dough evenly - this will prevent overmixing.
Fold in ¾ cup Reese's Pieces and ¾ cup Reese's Peanut Butter Chips until evenly combined.
Scoop the dough with a #10 cookie scoop into 9 portions, about 4 to 4.2 ounces each. Then, gently press the dough flat and shape it into a bowl.
Add a 1-ounce ball of peanut butter filling to each cookie dough bowl.
Then, gently use your hands to fold and work the dough up and around the sides and top of the peanut butter filling. Roll the ball of dough between your palms to smooth and round out.
Place the cookies on the baking sheet, usually with the sealed side down. Place cookie dough balls 3 to 4 inches apart on the prepared baking sheet. Optional: Add some additional Reese's Pieces and peanut butter chips to the tops of the cookie dough balls to give them a more finished look.
Bake for 10 to 12 until the edges are just set, and the top has a matte look. The cookies will look slightly underdone; this is normal, as they will continue to cook as they cool.
Allow the cookies to cool on the pan for 15 minutes before transferring them to a wire rack with a spatula to cool completely. These cookies are very soft right after baking and will firm up while they cool.