Deep learning with convolutional neural networks (CNN) is a powerful technique with wide-ranging applications. It has largely replaced traditional computer vision as the go-to method for solving image-analysis and classification problems. At its essence, however, training a CNN is an enormous global optimization problem which, like all optimizations, can fall victim to local extrema. We'll discuss ways of mitigating this issue using computer vision to add spatial context information to restrict the domain of optimization. These techniques not only speed up the training, but also improve the overall performance of the networks. We'll demonstrate results on real-world classification and segmentation problems.