This work aims to derive a non-trivial breakdown point for an algorithm for training a single hiddenlayer Neural Network. In pursuit of this goal, we propose two algorithms for training a network with ReLU activations. The first approach utilizes the partitioning property of the ReLU function while the second approach utilizes the convexity of the activation function.