Urban Land Cover Segmentation:
Domain Adaptation from Chinese to Canadian Cities using LoveDA + U-Net

A part of my ongoing exploration of ML, this time in computer vision! CV and segmentation models have applications in area I'm interested in, like urban planning and environmental monitoring, and so I decided to try it out when I found the LoveDA dataset. LoveDA consists of both a rural and a urban China dataset, so it's often used for segmentation models and testing their transferability to a new environment.

In this project, I extended the urban vs. rural scope to land and culture differences across countries by applying LoveDA-trained models to Toronto imagery data. The goal was to understand what label classes transfer/fail and why, and whether/how that differs within LoveDA and in the Canadian extension. I also took the opportunity to learn about and build my own U-Net CNN. :)

Datasets

The LoveDA dataset has 5,987 images in the aforementioned urban-rural split across several Chinese municipalities at a 0.3m spatial resolution. Each image has a mask labelling each pixel as one of 7 classes: background, building, road, water, barren, forest, and agriculture.

For the Toronto exploration, samples of the Toronto Open Data orthophoto were taken at a similar resolution and run through trained models. See ontario_data/ folder for the images.

Model Architecture

The model used was a U-Net model with a ResNet-34 encoder from segmentation-models-pytorch, which was pretrained on ImageNet. Training used a combined cross-entropy + Dice loss to ensure better pixel predictions and overall area match, with Adam as the optimizer. Class weights inversely proportional to pixel frequency were applied to CE loss to prevent the model from ignoring rare classes in favour of dominant ones.

A U-Net implemented from scratch in PyTorch was also built (following the original Ronneberger et al. (2015) paper) and trained on LoveDA alongside the pretrained version for architectural comparison. See my_unet/ folder for details.

Key Findings

For additional details on LoveDA domain adaptation discussion, see adaptation_results.md

• Model asymmetry
  • Urban -> Rural transfer degrades significantly (0.4967 -> 0.2801 mIoU), especially due to class imbalances for underrepresented classes without a distinct texture like barren and forest, which collapsed to <0.1 IoU.
  • Rural -> Urban transfer shows no change (0.4526 -> 0.4546 mIoU), likely indicating the learning of more generalizable features from an access to a more balanced and varied dataset. Some IoUs dropped for classes like background and barren, likely due to domain gaps in what gets labeled as those classes between the two domains.
• Joint training closes both sides' gaps as the evaluation on its own full data shows improved IoUs across all classes. But the urban -> rural gap/asymmetry remains as performance still drops on joint -> rural.
• Rural model works best on Toronto images, having sharper boundaries and more accurate class identification than either the urban or joint model. The urban model performed the worst (predicting most surfaces, especially impervious asphalt, as buildings), with the joint in between. See image below!
  • This actually likely indicates negative transfer by urban on joint due to learning not as generalizable features, especially in a North American suburbs context. It is also possible that the layout and shapes of areas in Toronto suburbs are closer to that of rural than urban China.
• Spatial coarsening: due to being semantic instead of instance segmentation focused, same class objects merging together is relatively common. However, there are increased coarsening issues in the Toronto inferences, likely due to a domain gap with the different sizes and placements of structures and surfaces (ex. single-family house suburbs, or more surface parking lots).
• Water has the best class IoU overall, likely due to having a distinctive texture and colour. The one LoveDA exception is during urban->rural (0.3443 IoU instead of 0.6+ usually) likely due to contextual differences between urban water bodies (rivers, reservoirs) and rural ones (irrigation canals, paddy fields). Certain domain gaps also exist crossing over to Toronto, as sometimes things like shadows or very green grass are misclassified as water.

Limitations and Extensions

• The Ontario orthophoto samples do not have a ground truth mask like the LoveDA data does, and thus their analysis is done only through qualitative observations.
• In this project, background classes were evaluated as part of the mIoU, and so are not directly comparable to the LoveDA challenges, which typically exclude background.
• This project analyzed the model performance differences between Chinese urban-rural areas and Toronto areas. Certain results such as model asymmetries may be limited to this specific combination of domains, and analysis on other places around the world would be meaningful as well.
• Joint training was done by simply concatenating urban and rural datasets, and other domain adaptation techniques can be explored as extensions. Additional exploration into areas like instance segmentation may also be of interest.