In this talk I will summarize our recent developments and results of experiments testing reasoning abilities of neural networks. I will first frame these results into a broader picture to highlight the importance of figuring out how neural networks can reason and then I will present two vignettes of  our research which show two different modes by which neural networks can solve problems that require reasoning.