Cetacean density can be estimated from passively-collected acoustic data via received calls attributed to the species of interest. Acoustic equipment can be used to gather data continuously for long periods, at high sampling rates, and over multiple channels, resulting in vast datasets. Humans are assumed to be the gold-standard for extracting features of interest from acoustic data, but automatic detection and classification software is necessary for large datasets where human auditing is not feasible. Here, we compare the abilities of two freely-available software (PAMGuard and the Low Frequency Detection and Classification System (LFDCS)) to detect and classify Argentinian southern right whale (Eubalaena australis) upcalls based on 4417 human ground-truthed calls. The number of detected calls varied substantially between the two software (PAMGuard recorded approximately the same number of true positives as LFDCS, but up to ten times as many false positives), and between separate runs of LFDCS with different parameter values (the results of which varied by up to a factor of four). The resulting differences between detected and truly-present calls can significantly impact subsequent density estimates. While it is possible to apply correction multipliers to outputs obtained from automated software, the aim is to minimise the amount of extrapolation required in order to maintain robust results. When using automatic detection algorithms, it is therefore essential that a rigorous, data-based detector performance analysis is conducted.