We are developing a machine learning system based on high-resolution images of Earth and Mars for classifying periglacial landscape features, detecting their temporal changes, and assessing their global distribution as well as their potential as indicator for climate conditions and changes. Earth periglacial landscape phenomena such as ice wedge polygons are closely linked to repeated freeze-thaw cycles, and the presence of water and ice in the subsurface. Ice wedge polygons, which are widespread in Arctic lowlands, constitute an important indicator for ground ice content. Ground ice makes permafrost vulnerable to thaw and subsidence, thus leading to massive changes in topography, hydrology, and biogeochemical processes [1]. Moreover, variations in permafrost extent due to climate warming in Earth’s polar regions cause changes in the aforementioned periglacial features. On Mars, similar young landforms such as ice-wedge polygons and debris flows are found [2]. Large volumes of excess ice are known to exist in the shallow subsurface of mid-latitude regions [3]. A major debate focusses on whether similar freeze-thaw cycles thawed this excess ice in the geologically recent past. If true, this would be conflicting with the current Martian environment, which ostensibly prevents the generation of liquid water, and would therefore have implications for the recent hydrologic past of Mars. With liquid water also intrinsically linked to the climate evolution and the potential habitability of Mars, the investigation of the aforementioned landforms becomes essential. Moreover, the present-day surface of Mars experiences changes linked to H2O and CO2 ice, which are unlikely to be the result of aqueous processes [4, 5]. Detecting the magnitude and timing of these changes would enable the estimation of the related process rates [6, 7] and the testing of hypotheses regarding the formation mechanism. Quantification of periglacial features on regional- to global-scale has not been done for either of the planets so far. These features as well as their changes can be tracked with high resolution remote sensing across large regions using big data approaches of image processing, classification and feature detection. For Earth the detection and tracking of periglacial features would provide invaluable insights into periglacial and permafrost dynamics as well as in the potential for permafrost vulnerabilities to thaw in a warming world. For Mars, determining the distribution of such landforms, as well as the spatial relationship to each other and to external parameters such as topography, would provide clues for their formation and thus elucidate the role of liquid water in the recent past. The key task is to map selected features across large regions using large high-resolution datasets, which makes automated methods for detection and classification of landforms essential. We are developing a powerful machine learning system that will identify the most appropriate image features for each landform. It will be trained on images for different cases of periglacial landforms. Ice wedge polygons are morphologically very similar on both planets, which besides the opportunity to conduct analogue studies, also provides the possibility to combine training datasets from both planets. Our system will be validated by manual identification of periglacial features on images, as well as with the existing validation datasets of both planets. Our project is exploring the potential for a machine learning system to detect periglacial phenomena by exploiting big datasets of large regions of Earth and Mars. The resulting global-scale mapping of ice wedge polygons will provide insights regarding the permafrost vulnerabilities to changing climates on Earth, as well as about the recent role of liquid water on Mars. The former are linked to life and biogeochemical processes on Earth, while the latter to the evolution of climate and potential habitability of Mars.