A mirror pyramid consists of a set of flat mirror faces arranged around an axis of symmetry, inclined to form a pyramid. By strategically positioning a number of conventional cameras around a mirror pyramid, the viewpoints for the individual camera's mirror images can be colocated at a single point within the pyramid, effectively forming a virtual camera with a wide field of view. Mirror pyramid-based panoramic cameras have a number of attractive properties, including single-viewpoint imaging, high resolution and video rate capture. Currently existing designs realize a single viewpoint within each mirror pyramid. In order to capture panoramas from multiple viewpoints with these designs, the entire physical setup would need to be relocated or duplicated. The former solution lacks the capability of video rate imaging, and the latter leads to bulky designs due to the multiple mirror pyramids. In this paper we propose a method for generalizing existing designs such that multiple viewpoints can be created in a single mirror pyramid. This enables simultaneous multiview panoramic video rate imaging with a compact design.