The advent of machine and deep learning algorithms on petascale supercomputers is accelerating the pace of discovery in astrophysics and poses significant challenges to the interpretability of these deep neural networks. Researchers have created a novel visualization of a deep neural network output during training as it is learning to classify galaxy images as either spiral or elliptical.