These natural caves were discovered in 1807 by William King George of the Geological Survey of India. According to locals, once a cowherd lost his cow while grazing. He soon realized that his cow had fallen down. While searching for the cow, the cowherd found these caves and came across a Linga inside it. The Linga is considered synonymous to Lord Shiva, who saved the cow. After this, a small Shiva temple was built right outside this cave.