Massachusetts Building Analysis Dashboard

Operation: Advanced Multi-Stage Spatial Join

An enhanced, multi-stage process was implemented to accurately enrich building footprints with NSI point data. This updated methodology features flexible handling of mixed-use properties, a precise nearest-neighbor buffer match, and systematic occupancy conflict detection to ensure data quality.

• Strategy 1: Intelligent Single-Family Matching
  • For buildings classified as 'Single Family', the process now flexibly considers both residential (RES) and commercial (COM) NSI points inside. This accommodates mixed-use scenarios like in-home businesses.
  • If one point is found, a direct one-to-one match is made.
  • If multiple points are found, their attributes are aggregated to create a composite profile, replacing the previous centroid-based selection. This robustly handles properties with multiple distinct units (e.g., a house with a separate commercial unit).
• Strategy 2: Standard Aggregation for Other Buildings
  • For all other building types (multi-family, commercial, etc.), all NSI points falling within the footprint are used.
  • Their attributes are aggregated to create a comprehensive profile for the building:
    • Value & Area (`structure_value`, `nsi_sqft`): Summed to get a total.
    • Stories (`nsi_num_story`): The maximum value is taken.
    • Characteristics (`year_built`, `material_type`): The statistical mode (most frequent value) is used.
• Strategy 3: Nearest Neighbor Buffer Match
  • For NSI points that remain unmatched, this strategy finds the single nearest building polygon within a 5-meter radius.
  • This ensures each point is uniquely assigned to its closest building, correcting for minor spatial inaccuracies. A single building can "absorb" multiple nearby points via this method.
  • A configurable option also allows buildings already matched in earlier stages to absorb additional nearby points, capturing features like adjacent garages or utility structures.
• Extra Feature: Systematic Occupancy Conflict Detection
  • Throughout the process, the script will actively compare the land use category of the NSI point (e.g., 'Commercial') against the category of the building polygon it falls into (e.g., 'Residential').

Operation: Soil Enrichment & Area-Weighted Spatial Join

• Part A: Preparing the Enhanced Soil Layer
  • 1. Component Filtering: Reads comp.txt and selects only the single Dominant Component (highest percentage) for each Map Unit.
  • 2. Horizon Filtering: Reads chorizon.txt and selects only Topsoil properties (depth < 10cm) to capture engineering characteristics relevant to foundations.
  • 3. Merge: Attributes are merged onto the Soil Shapefile to create a single, simplified soil layer.
• Part B: Spatial Intersection (EPSG:26986)
  • Data is projected to Mass State Plane (Meters) for accurate area measurement.
  • A Polygon-on-Polygon intersection (predicate='intersects') is performed between buildings and soil layers.
• Part C: Area-Weighted Conflict Resolution
  • Problem: Some buildings straddle the boundary between two or more soil map units.
  • Solution: The script calculates the exact Overlap Area for every match. If a building touches multiple soil units, it is assigned to the one with the largest intersection area.

Operation: Dictionary Mapping & Type Cleaning

Mapping specific NSI foundation codes to broader str_fdn_type for standardized Carbon Analysis.

Operation: Spatial Join (Polygons Preserved)

• 1. Priority-Based Deduplication: Filters permits and selects the "best" record per address.
  Priority Rule: Closed/Completed > Open > Most Recent Date.
• 2. CRS Alignment (EPSG:2249): Both datasets are projected to MA State Plane (Meters) for precise distance calculation.
• 3. Nearest Neighbor Join (5m Radius): Uses sjoin_nearest to find the single closest permit within 5 meters (from polygon edge).
  Matching Statistics (Total: 5,018):
  ● Exact Matches (In Polygon): 4,922 (98.1%)
  ● Buffer Matches (<5m): 96 (1.9%)
• 4. Non-Destructive Merge: New attributes (`DEMOLITION_TYPE`, `DATE`, `STATUS`) are joined back using Index Alignment. This guarantees zero data loss and perfectly preserves original Polygon geometry.

Operation: High-Performance Spatial Join & Logic-Based Fusion

• 1. Centroid-Based Spatial Indexing: To optimize performance and accuracy, building geometries were converted to Centroids before performing a `within` spatial join against the MassGIS Parcel polygons.
• 2. Temporal Conflict Resolution (Latest Year Heuristic):
  Handling parcels with multiple build years:
  • When a single building matched multiple parcel records (potential duplicates or subdivisions), the system prioritized the most recent construction year.
  • Method: Data was sorted by [BUILD_ID, YEAR_BUILT] in [Ascending, Descending] order, retaining only the top record.
• 3. Data Cleaning: Outliers were removed by filtering massgis_yr_built to the valid range of 1630 - 2025. (1,709 invalid values detected and removed).
• 4. Source Prioritization Strategy:

  The decision logic for merging MassGIS and NSI year data is illustrated below: