The rapid and accurate identification of radionuclides brings crucial information for nuclear monitoring to diagnose unknown radiological scenes. Recent studies have used a deep learning approach based on neural networks to develop algorithms that perform well in terms of accuracy and computation time and can also identify radionuclides with a limited number of photons. However, it has been shown that conventional neural networks are not necessarily robust, in the sense that a small particular perturbation of the input data can mislead the networks. A specific learning procedure is necessary to overcome this lack of robustness. In this paper, we show that small perturbations intentionally injected into gamma-ray spectra, with respect to the Poisson statistics, are able to fool the network. We propose applying a robust learning procedure, called "adversarial learning". We evaluate this procedure using a CdTe detector, namely Caliste-HD. We train a Convolutional Neural Network (CNN) with a synthetic database composed of simulated spectra and we test its performance on real data acquired with a Caliste detector.