Understanding how the brain encodes representations for a given stimuli is a key question in neuroscience and has influenced the development of artificial neural networks with brain-like learning abilities. Recently, learning representations by capturing similarity between input samples has been studied to answer this question in the context of learning downstream features from the input. However, this approach has not been studied in the case of a generative paradigm, crucial for explaining top-down interactions in sensory processing, consistent with the predictive abilities of our neural circuitry. We propose a similarity matching framework for generative modeling. We show that representation learning under this scheme can be achieved by maximizing similarity between the input kernel and a latent kernel which leads to an implicit generative model arising from learning the kernel structure in the latent space. We argue that the framework can be used to learn input manifold structures, potentially giving insights into task representations in the brain. Finally, we suggest a neurally plausible architecture to learn the model parameters linking representation learning using similarity matching with predictive coding.