CV benchmark
Five attributes, one label set, evaluated like a benchmark
The target attributes below are the real annotation protocol used for labeling, not an aspirational spec — each was chosen because it is both grounded in prior street-view literature and answerable from a single monocular frame.
Target attributes
Sidewalk presence
sidewalk_presence
A physically distinct, raised or demarcated pedestrian path running alongside the road, on either side, visible anywhere in frame.
Precedent: Weld, Jang, Zeng, Kim, Guo, Froehlich (2019), ASSETS '19 — Project Sidewalk
Road surface type (paved / unpaved)
road_surface_type
The carriageway surface material: paved (asphalt, concrete, paver blocks) vs. unpaved (dirt, gravel, loose aggregate).
Precedent: Randhawa et al. (2024/2025), arXiv:2410.19874 — global Mapillary road-surface classification
Street vegetation (ordinal, 3-level)
street_vegetation
Coarse ordinal simplification of the Green View Index: high (>25% of frame), medium (5-25%), low/none.
Precedent: Li, Zhang, Li et al. (2015), Landscape and Urban Planning — Green View Index
Street lighting presence
street_lighting
At least one dedicated street-lighting pole or fixture visible anywhere in frame (not headlights, not shop signage).
Immediate building frontage density
building_frontage_density
Density/form of buildings directly fronting the street in the visible frame: dense continuous multi-story frontage vs. open/low-rise/detached.
Source: data/labels/schema.json and docs/annotation_protocol.md
Label set
Every attribute above uses the same five-way label set, including explicit codes for occlusion and unusable images rather than forcing an annotator to guess.
positive
Attribute clearly present
negative
Attribute clearly absent
uncertain
Visible, but genuinely ambiguous even to a careful annotator
not_visible
Camera geometry/obstruction means this part of the scene isn't shown
unusable
Image itself is unusable — no attribute in this image gets a real label
Candidate architectures
These are candidates to be benchmarked against each other, not results. None has been trained on this project’s data yet.
Swin Transformer (small/base)
Direct precedent: Randhawa et al. (2024/2025) used this architecture for global Mapillary road-surface classification (F1 91–97%).
ResNet-50 / ConvNeXt baseline
Standard CNN baseline for multi-attribute image classification, used as a sanity-check floor against the transformer candidate.
CLIP-based linear probe
Cheap baseline exploiting pretrained vision-language representations before committing compute to full fine-tuning.
Leaderboard
Not yet measured
No model has been trained or evaluated on labeled UrbanLens India imagery yet, so there is no accuracy, F1, or Cohen’s kappa to report for any attribute or architecture. Inter-rater agreement on the labels themselves is also pending — see /methods.
see docs/annotation_protocol.md