Geospatial Map Viewer Platform

🏗️ System Architecture

Frontend

Next.js

Kepler.gl (Modified)

React + TypeScript

MapLibre GL

Backend

NestJS

Tippecanoe (Async)

Data Processing

Authentication

Storage

PostgreSQL (Metadata)

AWS S3 (Data/Tiles)

AWS SQS (Queue)

Redis (Cache)

⚡ Key Features

📁 Multi-Project

Create unlimited projects with separate map configurations and data

🔐 Role-Based Access

RBAC per project - admin, editor, viewer roles

🔄 Auto Tile Generation

JSON upload → automatic Tippecanoe conversion → MVT tiles

📊 Modified Kepler.gl

Save projects to database (original only loads in browser)

⚡ Async Processing

SQS queue handles large data without blocking UI

🗺️ Vector Tiles (MVT)

Optimized for fast rendering of large datasets

🔄 Data Upload Workflow

Upload

JSON/GeoJSON file

Validate

Check format

Queue

SQS message

Process

Tippecanoe

Store

S3 + DB

Ready

MVT tiles

📦 Data Storage Structure (S3)

# S3 Bucket Structure

s3://geospatial-viewer-data/
├── projects/
│   ├── {project_id}/
│   │   ├── raw/
│   │   │   ├── data_001.geojson
│   │   │   └── data_002.json
│   │   ├── tiles/
│   │   │   ├── data_001/{z}/{x}/{y}.mvt
│   │   │   └── data_002/{z}/{x}/{y}.mvt
│   │   └── metadata.json
│   └── ...
└── temp/
    └── (processing queue files)
            

🗄️ Database Schema (PostgreSQL)

-- Projects Table
CREATE TABLE projects (
    id UUID PRIMARY KEY,
    name VARCHAR(255),
    config JSONB,          -- Kepler.gl config
    owner_id UUID,
    created_at TIMESTAMP
);

-- Project Members (RBAC)
CREATE TABLE project_members (
    project_id UUID,
    user_id UUID,
    role VARCHAR(20),      -- admin/editor/viewer
    PRIMARY KEY (project_id, user_id)
);

-- Data Layers
CREATE TABLE data_layers (
    id UUID PRIMARY KEY,
    project_id UUID,
    name VARCHAR(255),
    s3_raw_path VARCHAR(500),
    s3_tiles_path VARCHAR(500),
    status VARCHAR(20),    -- pending/processing/ready
    created_at TIMESTAMP
);
            

🔬 Modified Kepler.gl Integration

❌ Original Kepler.gl

Data loads in browser only
No persistence between sessions
Manual export/import required
Limited to browser memory

✅ Modified Version

Auto-save projects to database
Server-side data conversion
Multi-user collaboration
Handles GB+ datasets
Role-based access control

⚙️ Backend Processing (NestJS + SQS)

// NestJS Queue Processor for Tile Generation
@Processor('tile-generation')
export class TileGenerationProcessor {
  
  @Process()
  async handleTileGeneration(job: Job<TileJobDto>) {
    const { layerId, s3RawPath, projectId } = job.data;
    
    // 1. Download from S3
    const rawData = await this.s3Service.download(s3RawPath);
    
    // 2. Run Tippecanoe (async)
    const tilesPath = await this.tippecanoeService.convert(rawData, {
      maxZoom: 14,
      minZoom: 0,
      layerName: layerId
    });
    
    // 3. Upload tiles to S3
    await this.s3Service.uploadTiles(tilesPath, projectId, layerId);
    
    // 4. Update database status
    await this.dataLayerService.updateStatus(layerId, 'ready');
  }
}
            