4. Dressage. Classical dressage is a solid foundation for everyone in any discipline and not only develops your horse in a physical sense but also in a mental sense. You do not have to compete or even achieve third level, but learning the basics of dressage is of great benefit to you and your horse. The building blocks of the training scale are relaxation, rhythm, suppleness, developed using patterns and exercises that encourage the aforementioned, as guided (only) by the rider. When you achieve these and your horse starts picking up contact with your hand (of their own accord), you have another tool in your toolbox to use when your horse is anxious and tense. This allows you the ability to then ask your horse to relax, supple, and to focus on a specific task. After your horse has learned to initiate contact as a result of progressive schooling, you can start to ask them to initiate that contact - when they are on the bit, you have a great deal of control. Essentially your horse is giving control up to you and you have the ability to direct their forward movement and provide the leadership they require and in doing so, you maintain control of the situation and create a relaxed horse with a calm, focused mind.
If, in the moment, your horse is still too worked up, get off and work on the ground until he's calm enough to work under-saddle. You're not letting your horse "win" by getting off. Ever. If you do not have the skill level or the capacity to deal with something under-saddle, staying on the horse is not going to change that and in fact you might create further damage by remaining in the saddle if you are not handling the situation correctly. Instead, get off and leave it for another day - a day when you have more skill or are in a better frame of mind to deal with what is being presented. On the ground, where you are (in most cases) safer, you can also work at it from another angle, such as using groundwork exercises. This might allow you greater influence over your horse and even might allow you to influence your horse to the point where it is then safe to re-mount.
A poorly fitting saddle or bit (have a professional - not just your instructor - saddle fitter evaluate your saddle and someone with experience studying bits evaluate your bit and its suitability to the oral conformation of your horse)
A rider's hard hands
Consult professionals to figure out the problem. Horses might head-toss if the bit is stabbing the roof of their mouth or pinching their tongue, or if an ill-fitting saddle is pinching their shoulders or resting on their whithers. It might even be something as simple as bugs driving your horse to the point of insanity. Consider all the possible reasons for a head-tosser, rather than pinning it on the horse just being stupid (etc etc). They are behaving the way they are for a specific reason, and I can assure you it's not just to p*ss you off. Your horse is a reflection of you, its rider.