Step 1: In a large mixing bowl, beat the softened unsalted butter and creamy peanut butter with an electric mixer on medium speed until light and fluffy, about 2-3 minutes.
Step 2: Gradually add the sifted powdered sugar, one cup at a time, mixing on low speed until just combined after each addition. Scrape down the sides of the bowl as needed.
Step 3: Pour in the milk (or heavy cream) and vanilla extract. Increase mixer speed to medium-high and beat until the frosting is smooth, creamy, and well combined, about 1-2 minutes.
Step 4: If the frosting is too thick for your desired use, add a tiny splash more milk or cream (1 teaspoon at a time) until the desired consistency is reached. If it's too thin, gradually add more sifted powdered sugar (1 tablespoon at a time) until it thickens.
Step 5: Use immediately to frost your favorite cakes, cupcakes, or brownies, or store as directed in the notes below.