The Ultra Weak Variational Formulation (UWVF) of the Helmholtz equationprovides a variational framework suitable for discretization using plane wave solutionsof an appropriate adjoint equation. Currently convergence of the method is only proved on the boundary of the domain. However substantial computational evidence exists showing that the method also converges throughout the domain of the Helmholtz equation. In this paper we exploit the fact that the UWVF is essentially an upwind discontinuous Galerkin method to prove convergence of the solution in the special case where there is no absorbing medium present. We also provide some other estimates in the case when absorption is present, and give some simple numerical results to test the estimates. We expect that similar techniques can be used to prove error estimates for the UWVF applied to Maxwell's equations and elasticity.