Social media and AI can measure the aesthetic quality of landscapes

To measure an ecosystem’s beauty and the well-being it produces for people can help inform public environmental policy. Scientists at EPFL and Wageningen University in the Netherlands have developed a novel modeling approach for ecosystem assessments based on deep learning and millions of Flickr photos.

The extent to which we enjoy an outdoor activity—such as hiking in the mountains, going for a run or paddleboarding—depends largely on the beauty of the surrounding ecosystem. For example, landscapes that contain a crystal blue sea, rolling hills covered in yellow and lavender, or a stream trickling across a bed of rocks can be a boon to both our physical and mental health. This sense of well-being is one of the factors examined in ecosystem service (ES) assessments, which quantify the contributions of landscapes to people’s well-being in order to inform environmental policy. To support these assessments, a team of scientists from EPFL and Wageningen University have developed a new type of model that uses artificial intelligence to incorporate people’s aesthetic enjoyment of a landscape. Their model—drawing on some nine million pictures of British landscapes posted on Flickr—can be easily reproduced on a large scale since it employs data available on social media. The model is the first to include insights into how individuals appreciate landscapes at such a large scale while remaining as accurate as current methods. Their research appears in Scientific Reports, a Nature publication.

To develop their model, the scientists trained a deep-learning algorithm on over 200,000 photos of landscapes in Great Britain obtained from the Scenic-Or-Not database. These photos, comprising a geographically representative dataset of Great Britain, were rated according to their aesthetic quality, or “scenicness,” through a crowdsourced survey. This approach enabled the scientists to include factors addressing how individuals enjoy landscapes personally—factors that are missing from conventional large-scale ES assessments. The research team then ran their deep-learning model, which is based on neural networks, on over nine million Flickr pictures, also integrating other AI-based models in their predictions of scenic beauty. Finally, they compared the output from their model with the results of a more conventional, environmental indicator-based model.

A more accurate model

The scientists compiled their findings on maps of Great Britain with color codes indicating the level of scenicness. Both models found that Snowdonia National Park in Wales, the Lake District in England and the Scottish Highlands were areas of particularly high aesthetic value and well-being. “The results from the two models are roughly the same at a resolution of 5 km²,” says Devis Tuia, an associate professor at EPFL’s Environmental Computational Science and Earth Observation Laboratory. “They both also clearly identified urban areas like London and Glasgow as being less attractive.” But at a resolution of 500 m², discrepancies emerge and the Flickr model stands out as being more accurate. For example, Greater London, Richmond Park and Heathrow airport are predicted as being very scenic areas by the conventional model, whereas the Flickr model more accurately classifies them as decidedly un-scenic.

An entirely new way to assess the environment and how people interact with it

Thanks to the combination of social media and deep learning, the scientists’ model can also evaluate how people’s appreciation of the aesthetic quality of a landscape changes over time. In an additional experiment, the research team looked at natural park areas in Great Britain known for their exceptional beauty, such as the Lake District, the Pembrokeshire coast in Wales and the Cairngorms in Scotland. This experiment allowed them to study how aesthetic factors are linked to the seasons. The “snow” attribute, for example, aligns with weather reports over the same periods—the new model accurately showed the 2009–10 winter as being particularly snowy. The scientists even saw that the prevalence of “snow” increased around the weekend when people were more likely to visit snowy landscapes, while the prevalence of “asphalt” remained relatively constant throughout the week. “This indicates that the use of social media-based data provides a combination of information about the state of the environment and how people interact with it,” says Tuia. “Such information has never before been obtained with such a high degree of accuracy.”

Ilan Havinga, a Ph.D. student at Wageningen University, adds: “It’s not easy to measure on a large scale how the aesthetic quality of a landscape contributes to people’s well-being. Our research provides a technology-based method for modeling people’s aesthetic enjoyment while incorporating the most important factor—the individuals themselves.” The next step will be to see whether the model can be applied to other countries, given how different their landscapes and cultures can be. Scientists will need to find a way to train AI algorithms using locally relevant criteria. Projects to that effect are already under way in the Netherlands, Spain and other European countries, in order to support environmental conservation policies across Europe.