When that voice rises, I pay attention. If it is no more than a murmur that disappears when I am gearing up and getting my brain into riding mode (very Zen-like, I might add), then I ride. If the feeling persists and/or gets louder, I turn the bike off, strip off my gear and get into the car. I don't ignore those feelings. Sometimes the feeling lasts days. When my wife asks why I'm not riding, she understands my reply: "It doesn't feel right to ride.".
I'm dressed for a crash everytime I'm on the bike, head to toe and everything in between. But just because I've got my gear on means I have any desire to test it. That goes for the track too. I am more accepting of the idea of going down on the track. Good leathers, controlled environment, good safety crew and medical a couple minutes away do a lot to help ease concerns. Even then, going down is not on the schedule of things I want to try. If I do, it is due to a failure in my ability and if so, fine. That is the risk I take. But I am not and will never go to the track with the idea that "Hey, I'm looking forward to testing my leathers today!".
I don't buy the idea that all riders go down. If you go down on the street, it is not always avoidable and sometimes circumstances are beyond your control. Gravel, sand, right-of-way violation, etc. But the rest (turn too hot, acting like an idiot, pushing limits on the street that shouldn't be pushed, poor maintenance, etc), is avoidable and should have been. Most of these incidents are the result of the rider ratcheting beyond their limits too quickly or too far. We all need to push limits in order to expand them but finding that fine edge that doesn't push us from exhiliration to pavement is the trick. I've been on that edge and I tickle the envelope of it very, very gently. When I go to the track, the same attitude will prevail.
I don't care if I'm the last guy in the pack. I'm riding at my limits. Not the bikes in front of me and not at the control rider's. Mine. I know my bike and I know myself. When I start to push, I will only do it when everything is aligned and working for me. If something feels off, I don't want to be on that edge and compound discomfort into disaster.
Just my two and rambling cents on the matter.