In 1969, Baumslag introduced a family of parafree groups Gi,j which share many properties with the free group of rank 2. The isomorphism problem for the family Gi,j is known to be difficult; a few small partial results have been found so far. In this paper, we compute the twisted Alexander ideals of the groups Gi,j associated with non-abelian representations into $SL(2,{\mathbb Z}_2)$. Using the twisted Alexander ideals, we prove that several pairs of groups among Gi,j are not isomorphic. As a consequence, we solve the isomorphism problem for sub-families containing infinitely many groups Gi,j.