It was originally a temple containing idols of many dieties including some Hindu dieties and planetary gods like shani etc.
That stone is supposedly a shiv linga which the prophet of islam tried to break but that resulted in terrible consequences and hence ended up worshipping it.
