At a distance of 2 Kms from Vijayawada Railway Station, Mogalrajapuram Caves are rock cut sanctuaries dating back to the 5th century AD. The caves are situated in the center of the city.
Out of 5 caves, only one cave is in good condition. The Mogalarajapuram temple has an Ardhanarisvara statue of Lord Shiva which is thought to be the earliest of this kind in South. One of the caves has an overhanging cornice with artificial windows. The idols of Lord Nataraja and Vinayaka in one cave are still in good condition. It is also a place of archaeological importance.