Abstract: With the rapid growth of the UAV market, UAVs have been widely used in aerial photography, agricultural plant protection, power inspection, forest fire prevention, high-altitude fire fighting, emergency communication, and UAV logistics. However, “black flight” incidents of unlicensed flights and random flights frequently occur, which results in severe security risks to civil aviation airports, sensitive targets, and major activities. Moreover, owing to their characteristics of maneuverability, intelligent control, and low cost, UAVs can be easily used for criminal activities, which threatens public and national security. How to effectively detect UAVs and implement effective measures for UAVs, especially “black-flying” UAVs, is an active and difficult problem that needs to be urgently solved, and it is also an important research area in the field of anti-UAV systems. The research and development of anti-UAV systems is an important focus in national public security, and UAV identification is one of the key technologies in anti-UAV systems. Aiming at the problem of how to recognize UAVs, a sound-recognition method based on a convolutional neural network (CNN) was proposed. The UAV anti-jamming technology based on acoustic signals is not easily affected by an UAV size, shelter, ambient light, and ground clutter, and sound is an inherent attribute of UAVs, which is also applicable to UAVs in a radio-silence state. In this study, UAV sounds, bird sounds, and human voice within 100 m were collected and preprocessed; then the mel frequency cepstral coefficient and gammatone frequency cepstral coefficient eigenvalues were extracted. Support vector machine (SVM) and CNN models were designed to recognize UAV sounds and other sounds. The experimental results show that the SVM and CNN accuracies are 93.3% and 96.7%, respectively. To further verify the recognition ability of the designed CNN, it was tested on some Urbansound8K datasets, and its accuracy reached 90%. The experimental results show that a CNN is feasible for UAV recognition, and it has a better recognition performance than a SVM.