Whisk together sweetened condensed milk, milk and cornstarch in a small saucepan. Cook and whisk over medium heat, until mixture comes to a full boil. Turn off heat and whisk in eggs until combined. Strain pudding into a large bowl, whisk in vanilla until combined. Chill bowl over an ice bath or in the refrigerator/freezer until cold. When cold, fold in Cool Whip until combined.
Layer wafers, bananas in pudding in a glass bowl. I used all of my pudding and bananas, but had leftover wafers. Top pudding with additional Cool Whip, if desired; sprinkle with Butterfinger. Chill until cold.