Before the Bunsen burner is turned on, the electricity from the battery flows to the lightbulb and lights it up. The electrons flowing through the wire are experiencing some resistance, but not enough to cause a major impact to the brightness of the bulb. When the flame is on, however, the heat excites the atoms in the wire which causes them to vibrate significantly. The extra bouncing around inside the wire creates more collisions, and therefore more resistance. The lightbulb dims when the flow of electrons is reduced.
You probably aren’t torching your electrical wires, so this tutorial setup might seem a little impractical. But it illustrates a simple physics phenomenon that challenges engineers anytime they need to conduct electricity in an extremely hot environment.