Combine the warm water and yeast, letting it sit for around 10 minutes until tiny bubbles form on the surface and the liquid thickens slightly, indicating the yeast is activated.
Using a stand mixer with a paddle attachment, slowly mix the yeast mixture with the rest of the dough ingredients, from milk to flour.
Transfer the dough to a well-greased bowl, turning to coat it completely. Cover tightly with plastic wrap and let it rise until it doubles in size, which should take about 1 to 1 1/2 hours.
After rising, remove the dough from the bowl and knead it for 1-2 minutes, adding flour as needed to prevent sticking. Roll the dough into a large rectangle.
For the filling, mix together the pumpkin pie spice, through vanilla, and spread this mixture evenly over the dough rectangle. Combine the brown sugar and cinnamon, and sprinkle this mixture on top.
Starting from the short end, roll up the dough into a log.
Slice the log into individual rolls using a long piece of dental floss for clean cuts. Place the dental floss under the roll, cross the ends over the top, and pull to slice. Repeat until you have 15-18 pieces.
Arrange the slices in a glass casserole dish coated with cooking spray, ensuring they fit snugly. Cover with plastic wrap and let rise for another 30-60 minutes, or until nearly doubled in size.
Bake the rolls in a preheated 375°F oven for 20-25 minutes, or until golden brown.
Prepare the icing by mixing together the icing ingredients. Spread the icing over the warm rolls before serving.