Text to Ideology or Text to Party Status?
Recent papers have used support-vector machines with word features to classify political texts by ideology. Our own work on this topic led us to hypothesize that such classifiers are sensitive not to expressions of ideology but rather to expressions of attack and defense, opposition and government. We test this hypothesis by training on one set of parliamentary speeches and testing on another in which party roles have been interchanged, and we find that the performance of the classifier completely disintegrates. Moreover, some features that are indicative of each party ‘swap sides’ with the change of government. Our results suggest that the language of attack and defense, of government and opposition, will dominate and confound any sensitivity to ideology in these kinds of classifiers.