Bernoulli's principle is a statement about the conservation of energy in a moving fluid. It says that the energy of a given volume of fluid consists of three parts:
kinetic energy = v^2/2*density
potential energy = g*h*density
pressure energy = P
When energy is conserved (meaning no losses to friction, viscosity or compression), the sum of these parts is constant although the individual parts may vary.
When you combine Bernoulli's principle with the continuity principle (which basically says the volume flow rate is constant throughout a flow channel despite changes in the flow area), the hose behavior becomes more understandable.
With the hose end open, the volume flow rate is determined by the faucet valve area and the large pressure drop across it. (Note there is an energy loss at the valve. The flow rate is the same on either side of the valve, velocities are about the same, but the pressure decreases to zero.) Water passes through the hose with some velocity and zero pressure, and exits with the same values.
When you apply your thumb, you change the system. You raise the pressure in the hose moderately compared to the supply pressure (say 10 psi compared to 50 psi) and the volume flow rate decreases, but only a little. Flow continuity means the velocity beyond your thumb increases. So it exits with higher velocity and zero pressure. This is roughly consistent with Bernoulli, because passage through your thumb's restriction reduces P (from 10 psi to zero) and increases v. The pressure energy is exchanged for velocity energy.
Mark's explanation is good but I have to comment on one minor point that is easily and often misunderstood: There is no tendency for air parcels to try to reunite with their old neighbors after wing passage. See the ref.