High-speed OnPrem image detection and intelligence response system for object detection, OCR, entity extraction, and response automation
Satellite imagery of border regions, critical infrastructure, and asset or troop movements is highly classified. Routing this data to external or cloud-based AI platforms for processing would require it to leave our secured environment, creating serious national-security, data-sovereignty, and confidentiality concerns.
Most vision and OCR models are trained on general, worldwide datasets. While strong at generic object and text recognition, they do not fully understand Indian border terrain, locally relevant vehicle and asset types, regional number-plate and container-marking formats, or the multi-script text (Devanagari and other regional scripts) found on assets in our area of operations. A model may understand imagery in general but still lack the context in which it is actually being used.
We learned that a powerful detection model on its own was not enough. Turning a raw satellite scene into an actionable threat assessment requires several layers working together — image validation, tiling, object detection, text extraction, geo-referencing, classification, and entity linking. The model is only one part of a much larger intelligence system.
Hosting detection, OCR, and classification models on our own GPU servers introduced challenges around server connectivity, model availability, and service interruptions — unacceptable for a platform expected to monitor continuously. We needed additional monitoring and infrastructure hardening to make the services stable and reliable.
Imagery comes in specialised geospatial formats (GeoTIFF, NITF, HDF5, GeoPackage) with varying band counts, resolutions, and coordinate systems, and the useful text on assets is embedded inside the pixels. This information cannot be analysed directly by a generic model and must first be validated, tiled, cleaned, and structured.
Large scenes must be broken into tiles for processing, which causes objects and text near tile edges to be cut in half and missed by naive detection. Beyond that, a detection can look confident yet still be a false positive or a misclassified threat. Technical metrics alone were not sufficient — analyst and operator feedback was essential to confirm whether detections were operationally relevant and useful.
Source imagery can be extremely large (up to 10 GB per scene) and very high-resolution. Analysing it end to end is compute-intensive and, without careful optimisation, too slow to support real-time, round-the-clock threat monitoring.
Our object-detection, OCR, and classification models run on-premises within a private, controlled, air-gap-friendly environment. Instead of relying on a single model, specialised models handle threat detection, asset-marking extraction, and threat classification, while all sensitive imagery and intelligence stays inside our security perimeter.
Object detection (YOLOv8, accelerated with TensorRT on GPU with an ONNX fallback on CPU) locates assets and threats; OCR (PaddleOCR with a two-stage detect-then-recognise pass) reads number plates, container IDs, and other markings; and a classification model labels each detection as Attacking, Defensive, Neutral, or Unknown. Together these automate what was previously slow, manual image analysis.
Adaptive tiling with overlap prevents edge-split objects, and cross-tile Weighted Box Fusion re-joins detections that span tile boundaries — fixing the objects that naive processing misses. Pixel coordinates are projected into real-world latitude/longitude using tile geo-metadata, and extracted text is linked to the nearest detected object so every reading is tied to a specific asset and location.
Detections and OCR reads are stored with geospatial indexes (PostgreSQL + PostGIS), allowing the platform to track assets across missions, flag repeat sightings, and place every finding on a map rather than treating it as an isolated result. This makes the intelligence contextual and mission-specific rather than a set of disconnected detections.
We are continuously improving the models with locally relevant terrain, asset types, and multi-script/multi-language text so the system better reflects the actual environment in which it operates and produces more contextually relevant results.
The platform is deployed in an operational setting where analysts review real detections, confirm or reject them, and flag missed or misclassified threats. This feedback helps us identify blind spots, incorrect assumptions, and areas where detection quality can be improved.
Per-class confidence thresholds and uncertainty estimation (Monte-Carlo dropout / ensemble voting) surface low-confidence results for review, while a full observability stack (Prometheus and Grafana, structured logging, distributed tracing) and MLflow-based experiment tracking let us compare different models on our own mission data and monitor reliability continuously.
The on-premises architecture keeps classified geospatial data close to where it is stored and removes our dependency on external AI services for processing intelligence.
Rather than isolated bounding boxes, the platform produces findings tied to real coordinates and specific assets, tracked consistently across missions — including automatic flagging of repeat sightings of the same asset.
Analysts have surfaced false positives, missed threats, and edge cases that technical testing alone would not catch, and their feedback feeds directly back into model improvement.
Ongoing adaptation to local terrain, asset types, and multi-script text is steadily improving accuracy in the specific environment that matters to us.
Automated object detection and OCR extract threats and asset markings directly from raw imagery, sharply reducing manual image interpretation, saving analyst time, and speeding up the path from image to actionable intelligence — supporting genuine 24/7 monitoring.
We can evaluate different models on real mission data by detection quality, processing time, and infrastructure cost. This reduces our dependency on any single AI model provider.
The same pipeline extends well beyond border monitoring — to disaster and infrastructure monitoring, agriculture and land-use analysis, logistics and supply-chain tracking, and other geospatial-analytics use cases — and lays the groundwork for predictive, pattern-level intelligence across many missions rather than one image at a time.
Keeps classified satellite imagery and intelligence data within our own secured environment
Operates within existing defense-grade security and access-control infrastructure (clearance-based access)
Provides greater control over data access, storage, and retention
Provides better visibility and control over the complete AI processing workflow, from ingestion to output
Supports air-gapped and sovereign deployment with no reliance on external cloud AI services
Enables continuous customisation of the AI for Indian defense and border-security needs