Hare Krsna,
As with any material things, knowledge also gets lost, hidden or covered from time to time. In case of transcendental knowledge pertaining to Krsna consciousness, such knowledge is sometimes deliberately hidden or covered so that unqualified people do not have access to it. Just as a physician prescribes medication to the patients according to their health conditions and ability to intake them, transcendental knowledge is also presented to the population according to their material ailments and ability to absorb them. This is precisely the reason why Krsna’s esoteric and most confidential pastimes (such as rasa lila, etc.). are not revealed fully or discussed openly. I am sure there could be many extremely confidential pastimes of Sri Sri Radha and Krsna that we have never heard of or read in the books of our acaryas. This is because conditioned souls in this age of Kali Yuga are not fully qualified to appreciate them. People conditioned by the three modes of material nature do not have the right level of purification to appreciate pure transcendental topics. As a result, the Supreme Lord and His bonafide representatives deliberately hide such literature giving the impression that they are lost to time.
One good example is the discovery of Sri Brahma Samhita by Sri Caitanya Mahaprabhu in the temple of Adi Kesava. In the temple of Adi-kesava, Sri Caitanya Mahaprabhu discussed spiritual matters among highly advanced devotees. While there, He found a chapter of the Brahma-samhita. Sri Caitanya Mahaprabhu was greatly happy to find a chapter of that scripture, and symptoms of ecstatic transformation – trembling, tears, perspiration, trance, and jubilation – were manifest in His body. There is no scripture equal to the Brahma-samhita as far as the final spiritual conclusion is concerned. Indeed, that scripture is the supreme revelation of the glories of Lord Govinda, for it reveals the topmost knowledge about Him. Since all conclusions are briefly presented in the Brahma-samhita, it is essential among all the Vaisnava literatures. Sri Caitanya Mahaprabhu copied the Brahma-samhita, and then with great pleasure He went to a place known as Ananta Padmanabha. (Sri Caitanya Caritamrta)
The fact that the Lord found only one chapter of the Brahma Samhita is not sheer coincidence. It is possible that the other chapters of the book are deliberately withheld from conditioned people of this earthly planets. The book surely manifests in its full glory in the higher planets where people are more qualified to receive transcendental topics.
I have heard of a similar phenomenon with Srimad Bhagavatam. It is said that the version of the Srimad-Bhagavatam that’s available to human society on this earth contains only eighteen thousand verses. On higher planets, where the lifespan and mental capacity of the inhabitants is far superior to ours, the Bhagavatam has many more verses. (Unfortunately, I do not have any source of reference).
This is one reason why transcendental knowledge is hidden or lost. Another reason, as Srila Prabhupada explains, is due to the nature of this material world. The time factor is so pressing that in the course of time everything within this material world is spoiled or lost. In BG 14.10, Lord Krsna explains that sometimes the mode of goodness becomes prominent, defeating the modes of passion and ignorance. Sometimes the mode of passion defeats goodness and ignorance, and at other times ignorance defeats goodness and passion. In this way there is always competition for supremacy. In such a battle of supremacy among the three modes, knowledge may get lost or deviated. This could also explain how we moved from sruti to smrti. In the previous yugas, transcendental knowledge was spread through the process of hearing (sruti). Suta Goswami heard Srimad Bhagavatam spoken by Sukadeva Goswami to Pariksit Maharaja, and then shared them verbatim with the sages of Naimisaranya. The hearers heard it perfectly, retained it perfectly, and disseminated perfectly. Later on considering that the people of Kali Yuga have limited cognitive power, the transcendental literatures were put down in written format for easy retainment by the people. Even then, the knowledge and its core essence get deviated, manipulated, misunderstood, misinterpreted, or sinisterly repurposed to suit the malafide intentions of some people. One example is how the personality of Lord Krsna is misunderstood by impersonalists to be impersonal.
Therefore, the Supreme Lord appears from time to time to reestablish dharma or real religious principles. Reestablishing dharma means reestablishing transcendental knowledge also.
“To deliver the pious and to annihilate the miscreants, as well as to reestablish the principles of religion, I Myself appear, millennium after millennium.” (BG 4.8)
Similarly in this particular pastime of Lord Kapiladeva, He appears to reestablish Sankhya philosophy which in the course of time was mysteriously lost.
“Similarly, Kapila also said that the system of Sāṅkhya philosophy was not exactly being introduced by Him; it was already current, but in course of time it was mysteriously lost, and therefore He appeared to reintroduce it. That is the purpose of the incarnation of Godhead. Yadā yadā hi dharmasya glānir bhavati bhārata. Dharma means the real occupation of the living entity. When there is a discrepancy in the eternal occupation of the living entity, the Lord comes and introduces the real occupation of life.” (SB 3.24.37 Purport)
Only the Supreme Lord and His bonafide representatives can establish real religious principles.
“Real religious principles are enacted by the Supreme Personality of Godhead. Although fully situated in the mode of goodness, even the great ṛṣis who occupy the topmost planets cannot ascertain the real religious principles, nor can the demigods or the leaders of Siddhaloka, to say nothing of the asuras, ordinary human beings, Vidyādharas and Cāraṇas.” (SB 6.3.19)
Therefore, it is extremely important for us as devotees to receive the transcendental knowledge in true spirit and preach to others in the same spirit without any deviation. This is the true meaning of following the disciplic succession. Srila Prabhupada suffixed Bhagavad Gita with ‘AS IT IS’; this is not some coincidence or marketing/branding propaganda. It has a deep significance, and that is propagating transcendental knowledge in its true spirit as received through disciplic succession. In this way, we all can all play our part in protecting such knowledge from being lost again to the flowing rivers of eternal time.
All glories to Srila Prabhupada!
Krsna Katha 