How many great TV shows have you discovered in season 3 or later? I started watching Game of Thrones after they had released 5 seasons. Pat Flynn had released at least 100 episodes of his podcast before I even knew he existed. I discovered Hard Core History years after Dan Carlin started producing it.
It's natural that it's going to take some time for you to hit critical mass and generate word-of-mouth.
With very few exceptions, your first couple of iterations with not be smash hits.
It took Tim Urban from WaitButWhy a dozen posts till he got his first big hit ('Why Generation Y Yuppies Are Unhappy')