The open category problem arises when a test query (e.g., image) belongs to a novel class/object category that was not present in the training data. We want our classifiers to detect these novel category objects and flag them for alternative handling. This is part of a larger goal of endowing every ML system with a model of its own competence so that it can reject queries that it is not competent to answer.

A natural approach to open category detection is to train an anomaly detector to raise an alarm when the input query is an anomaly with respect to the training data. This talk will review several deep anomaly detection methods including auto-encoder reconstruction error, hybrid supervised/auto-encoder models, and adversarial reciprocal points. It will also compare the results of applying standard anomaly detectors to the representations learned by various deep learning methods.

The results show that a simple score, the maximum of the logit scores of the supervised classifier, is competitive with and sometimes better than these anomaly detection approaches. We perform a series of experiments to understand why and conclude that the logit score can be interpreted as a measure of "object familiarity" in computer vision. Objects that lack familiar features are classified as novel.