A church is NOT a pastor. A church is a community. I know that my in-laws will not attend my church because the pastor jokes about their college football rival from the pulpit but most people become attached to the members of their Sunday school class, the programs and opportunities for ministry. Some even view their home church as a live market for selling insurance.
If the sermons push (or pierce) the envelop, are you really going to yank your children out the church where they have made friends? Where you have invested thousands of dollars in tithes? Where those you come in contact with challenge you to a more serious faith or greater compassion? Where your admittedly fiery and all too human pastor has provided sage advise and perspective in personal matters? Is your self-righteous indignation so great and humility so poor?
If you answer yes to any of these questions I will refrain from questioning the depth of your faith but will reserve the right to consider the quality of your humanity.