Anatomical and electrophysiological studies have provided us with detailed information regarding the extent and topography of the primary (V1) and secondary (V2) visual areas in primates. The consensus about the V1 and V2 maps, however, is in sharp contrast with controversies regarding the organization of the cortical areas lying immediately rostral to V2. In this review, we address the contentious issue of the extent of the third visual area (V3). Specifically, we will argue for the existence of both ventral (V3v) and dorsal (V3d) segments of V3, which are located, respectively, adjacent to the anterior border of ventral and dorsal V2. V3v and V3d would together constitute a single functional area with a complete representation of both upper and lower visual hemifields. Another contentious issue is the organization of the parietal-occipital (PO) area, which also borders the rostral edge of the medial portion of dorsal V2. Different from V1, V2, and V3, which exhibit a topography based on the defined lines of isoeccentricity and isopolar representation, area PO only has a systematic representation of polar angles, with an emphasis on the peripheral visual field (isoeccentricity lines are not well defined). Based on the connectivity patterns of area PO with distinct cytochrome oxidase modules in V2, we propose a subdivision of the dorsal stream of visual information processing into lateral and medial domains. In this model, area PO constitutes the first processing instance of the dorsal-medial stream, coding for the full-field flow of visual cues during navigation. Finally, we compare our findings with those in other species of Old and New World monkeys and argue that larger animals, such as macaque and capuchin monkeys, have similar organizations of the areas rostral to V2, which is different from that in smaller New World monkeys.