An existence and regularity theorem is proved for integral equations of convolution type which contain hysteresis nonlinearities. On the basis of this result, frequency-domain stability criteria are derived for feedback systems with a linear infinite-dimensional system in the forward path and a hysteresis nonlinearity in the feedback path. These stability criteria are reminiscent of the classical circle criterion which applies to static sector-bounded nonlinearities. The class of hysteresis operators under consideration contains many standard hysteresis nonlinearities which are important in control engineering such as backlash (or play), plastic-elastic (or stop) and Prandtl operators. Whilst the main results are developed in the context of integral equations of convolution type, applications to well-posed state space systems are also considered.