The responses of inferior temporal (IT) neurons may depend on the behavioral context of the stimuli; e.g. in Konorski tasks responses to two successively presented physically identical stimuli can be markedly different. This effect has been interpreted as being linked to the behavioral task, and to be involved in short-term memory and/or the temporal comparison of successively presented stimuli. We tested whether this behavioral context effect also occurs when the monkey is not executing a Konorski task, i.e. no temporal comparison of stimuli is being performed. Responses of the same IT neurons under two behavioral conditions were compared using the same temporal stimulus sequence (but different stimuli): a Konorski task and a Fixation task. We found that the occurrence of the behavioral context effect did not depend on the execution of the short-term memory task. The observed decline in the level of responses to repeated presentation of similar stimuli is interpreted as being a passive mechanism involved in recency detection, which occurs even if the recency information is not useful for the task. The importance of these results in the interpretation of “task-related” neuronal responses is discussed.