The stochastic volatility jump diffusion model with jumps in both return and volatility leads to a two-dimensional partial integro-differential equation (PIDE). We exploit a fast exponential time integration scheme to solve this PIDE. After spatial discretization and temporal integration, the solution of the PIDE can be formulated as the action of an exponential of a block Toeplitz matrix on a vector. The shift-invert Arnoldi method is employed to approximate this product. To reduce the computational cost, matrix splitting is combined with the multigrid method to deal with the shift-invert matrix-vector product in each inner iteration. Numerical results show that our proposed scheme is more robust and efficient than the existing high accurate implicit-explicit Euler-based extrapolation scheme.