Retrofit Architecture for AI Physical Security: A Technology Briefing on VMS Integration, ONVIF/RTSP Standards, NVR Compatibility, Latency Budgets, and the Decision Between Add-On Inference and Rip-and-Replace

AI physical security retrofit architecture is the layered decision a security director makes when the existing camera fleet works, the network exists, the VMS is paid for, and the only thing missing is the detection layer. Buyers who treat that as a hardware-procurement problem (rip out the old, install the new) end up paying for infrastructure they already have. Buyers who treat it as an integration problem (add inference to what is already deployed) end up with the latency, privacy, and cost profile of edge AI without the capital outlay of a greenfield install.

This briefing is a primary-source-grounded reference for security directors, IT architects, and procurement teams evaluating how to add AI detection to an existing physical security stack. It defines the five integration patterns that dominate real-world deployments, walks the standards layer (ONVIF Profiles S, T, M, and A; IETF RFC 7826 RTSP 2.0; the underlying RTP and SRTP transports), traces the latency arithmetic across each pattern, and surfaces the procurement implications under FAR 52.204-25, the NIST AI Risk Management Framework, and the emerging IEC 62676-4:2025 video surveillance application standard. It complements the related Intelligence briefings on where AI inference runs and the architectural shift from CCTV recording to real-time detection.

LIVE CAM-07 · EXTERIOR ZONE

Why retrofit became the default AI physical security deployment pattern in 2026

Retrofitting AI detection onto an existing camera fleet is the dominant pattern because three structural forces converged between 2023 and 2026 to make the alternative economically and operationally indefensible for most buyers.

First, the installed base is too large to displace. The global surveillance camera installed base passed one billion units in 2021 and is on a trajectory toward two billion by the end of the decade, according to tracking data from IHS Markit (now part of S&P Global Market Intelligence) and confirmed by Omdia. In the United States alone the installed base is estimated at roughly eighty-five million cameras. The vast majority of those cameras are IP-based, were installed within the last seven to ten years, and have a depreciable service life of seven to fifteen years from installation. The economic logic of full replacement collapses against that timeline: a fleet installed in 2020 is not depreciated, not end-of-life, and not budgeted to be replaced. Asking a customer to rip out two-thirds of an unfinished depreciation schedule to deploy AI is asking for a capital decision that finance teams correctly resist.

Second, the standards layer matured. ONVIF, the open industry forum founded in 2008 by Axis Communications, Bosch, and Sony, has certified more than 25,000 conformant products across 500-plus member companies, with ONVIF Profile S (basic video streaming) deployed on more than 33,000 devices. Profile T, introduced in 2018, brought H.265 support, advanced motion detection, and metadata streaming into the same interoperability layer. Profile M, finalized in 2021, standardized analytics metadata exchange between cameras, analytics engines, and management software. Profile A, focused on advanced access control configuration, rounds out the modern multi-profile stack. The practical consequence is that an analytics-capable appliance can attach to a Hanwha camera, an Axis camera, a Hikvision camera (where supply-chain rules permit), a Bosch camera, and a Vivotek camera through one standardized protocol surface rather than five proprietary ones.

Third, edge inference silicon caught up to the workload. A modern on-premises appliance built around an NVIDIA Jetson AGX Orin module delivers up to 275 TOPS (trillion operations per second) of sustained AI compute in a 60-watt power envelope, comfortably enough to process dozens of standard-definition or 1080p RTSP feeds with a typical computer-vision detection model. Hailo accelerator silicon delivers 26 TOPS at roughly 2.5 watts of typical draw. Axis Communications shipped its ARTPEC-9 system-on-chip with on-camera deep-learning capability in early 2025. The capacity to run real-time detection on premises, on commodity edge hardware, without re-routing video to a cloud endpoint, removed the last technical argument for replacing cameras to get AI: the AI no longer needs to live in the camera, because it can live in an appliance the camera already feeds.

The economic, standards, and silicon vectors together produced the 2026 baseline: retrofit is the default. Greenfield deployments still occur (new facilities, new wings, security upgrades that coincide with broader infrastructure refresh), but the marginal-camera decision for an existing facility tilts decisively toward integration over replacement. The total-cost-of-ownership briefing walks through the financial implications in more detail; this briefing focuses on the technology.

The standards layer: ONVIF, RTSP, RTP, and how they interoperate

The standards layer is what makes retrofit architecturally possible. Without it, every analytics vendor would need to write camera-specific drivers for every model from every manufacturer, and the integration economics would collapse. The relevant standards stack has three layers, and procurement teams should understand all three before they evaluate a vendor.

At the transport layer, RTSP and RTP do the actual streaming. The Real-Time Streaming Protocol, originally specified in IETF RFC 2326 in 1998 and modernized in RFC 7826 (RTSP 2.0) in 2016, is the application-layer protocol that lets a client (an analytics appliance, a VMS, or a network video recorder) request, pause, and tear down a live media stream from a server (the camera). RTSP is the control channel; the actual media flows over the Real-Time Transport Protocol (RTP, RFC 3550) or its secure variant SRTP. The combination is what every analytics appliance uses to ingest video from an IP camera that supports the standard, which is essentially every IP camera shipped in the last decade.

At the codec layer, H.264 and H.265 dominate the encoded payload. H.264 (MPEG-4 Part 10, AVC) has been the workhorse video codec for IP surveillance since the early 2010s. H.265 (HEVC) offers roughly fifty percent better compression at equivalent visual quality and is now standard on cameras shipped after about 2017. The codec choice matters for retrofit because decoding is computationally non-trivial: H.265 demands roughly twice the decode cost of H.264 per frame, and that decode work either runs on the appliance's CPU, a hardware decoder block on the GPU or AI accelerator, or both. Appliances designed for retrofit deployments specify hardware decode acceleration explicitly; ones that decode on CPU are typically constrained to a smaller concurrent camera count.

At the interoperability layer, ONVIF specifies the API contract. ONVIF defines profiles, each of which is a named conformance level that guarantees a specific set of capabilities. Profile S, introduced in 2011, guarantees basic video streaming and PTZ control. Profile T, introduced in 2018, adds H.265 support, advanced video streaming with imaging settings, motion alarm and tampering events, and metadata streaming. Profile M, finalized in 2021, adds analytics-metadata transport and event subscription, which is the standardized surface an analytics engine uses to publish detection results back into a VMS or orchestration layer. Profile A handles advanced access-control configuration. ONVIF announced in 2024 that it would sunset Profile S in favor of Profile T because Profile S's authentication mechanisms no longer meet current cybersecurity recommendations; deployments still relying on Profile S should plan a Profile T transition path.

The interaction of these three layers is what makes retrofit work. An analytics appliance discovers a camera via ONVIF, opens an RTSP control session against it, ingests an H.264 or H.265 stream over RTP, decodes the frames with hardware acceleration, runs inference on the decoded frames, and publishes detection metadata back through ONVIF Profile M to whatever VMS the operator already runs. No proprietary integration is required at any layer. The vendor lock-in tax that characterized the analog CCTV era is structurally absent.

The five retrofit integration patterns, mapped to where AI sits in the stack

Real-world AI retrofit deployments in 2026 fall into five architectural patterns. The patterns differ in where inference runs, what data crosses the customer network boundary, what failure modes apply, and which legacy components remain in service. Buyers who can identify the pattern a vendor proposes can stress-test the latency, privacy, and resilience implications before signing a contract.

The patterns are not mutually exclusive. Large multi-site deployments commonly mix Patterns 1 and 2 (on-camera detection for very high-value sightlines, on-appliance detection for everything else) or Patterns 2 and 4 (on-appliance detection at the primary site, cloud-bridge detection at remote locations where appliance footprint is impractical). What separates a defensible architecture from an undisciplined one is whether the operator can articulate, by detection category, which pattern handles it and why.

Latency budget math for retrofit pipelines

The latency budget of a retrofit AI pipeline is the sum of seven stages, and the integration pattern determines which stages dominate. Understanding the arithmetic separates buyers who can stress-test vendor claims from buyers who accept marketing latency numbers at face value.

Frame capture at the sensor takes one to five milliseconds. In-camera encoding to H.264 or H.265 adds five to fifteen milliseconds depending on codec, profile, and bitrate. The Axis Communications white paper on latency in live network video surveillance (June 2024) documents the encoding contribution in detail and confirms that H.265 with B-frames pays a higher decoder-side buffering cost than H.264, even when the encoder-side latency looks similar. Network transport from the camera to the analytics appliance over RTSP and RTP adds a sub-millisecond LAN hop in a well-designed network, or one to five milliseconds if the camera is across a managed switch boundary. Decoding the H.264 or H.265 frame on the appliance takes five to twenty milliseconds, depending on whether decode runs on hardware blocks (typical) or CPU (fallback). Inference itself takes ten to forty milliseconds on modern edge accelerators for a standard detection model. Post-processing (non-maximum suppression, tracking association, confidence thresholding, overlay generation) adds two to ten milliseconds. Dispatch to the operator notification or VMS event channel adds one to ten milliseconds.

Pattern 2 (on-premises appliance sidecar) running an INT8-quantized detection model on a Jetson AGX Orin or comparable Hailo-based appliance routinely lands between twenty and fifty milliseconds end-to-end for the inference pipeline alone. The detection-to-alert latency that downstream responders care about (the time from a behavior occurring to a notification arriving on a phone or operator console) is dominated by the dispatch and notification path, not the inference pipeline. Pattern 4 (cloud bridge) adds the network upload to the cloud ingress (50 to 150 milliseconds depending on network conditions), queue wait at the inference service (10 to 50 milliseconds), and the return network leg (50 to 150 milliseconds). The structural penalty of cloud routing is the round-trip itself, not the cloud GPU's per-frame inference time.

A peer-reviewed study published in Computers in early 2026 documented sub-second end-to-end latency for an RTSP-to-YOLOv11 pipeline in a defense surveillance context, using a hybrid RTSP-WebRTC transport to bound the variability. The study's measurement methodology underscores a point that vendor marketing rarely makes explicit: latency in live video surveillance is the sum of many small contributors, and the variance matters at least as much as the mean. A pipeline whose mean latency is 80 milliseconds but whose 99th-percentile latency is 600 milliseconds will miss roughly one frame in a hundred under adversarial conditions. For weapon detection and active-shooter alerting, that variance is the binding constraint.

When retrofit does not work: the four failure modes that force replacement

Retrofit is the dominant pattern, but it is not universal. Four failure modes force a partial or full camera replacement, and procurement teams should identify them during the assessment phase rather than discover them during deployment.

Sensor resolution below detection requirement. Computer-vision detection models have a minimum effective pixel count for the target object. A drawn firearm at twenty feet from a 1MP camera projects to roughly thirty by twenty pixels at typical lens settings, which is below the reliable detection threshold for most current models. A 4MP or 8MP camera at the same distance produces a 60-by-40 or 120-by-80 pixel projection, which sits comfortably inside the operating range. Cameras installed before about 2015 are commonly 1MP or 2MP and may fall below the detection threshold for fine-object workloads regardless of the AI behind them. Wide-area detection (people, vehicles, perimeter intrusion) is more forgiving than fine-object detection (weapons, license plates, specific behavioral signals), and the sensor-to-workload mapping should be done lens by lens before a retrofit RFP is finalized. The briefing on computer vision under adversarial conditions covers the resolution-related failure modes in depth.

Codec limitations and bandwidth ceilings. Cameras that support only MJPEG or legacy MPEG-4 (not H.264 or H.265) and cannot be firmware-upgraded to a current codec generate streams that consume substantially more network bandwidth and consume more decode resources at the appliance. A fleet of older MJPEG-only cameras can still be integrated, but the appliance capacity it consumes per camera is two to four times that of H.265 streams, which compresses the appliance's concurrent-camera capacity. At a certain ratio of legacy-codec cameras to modern ones, the math tips toward camera replacement.

Supply-chain compliance constraints. Federal facilities, defense contractors, critical infrastructure operators, and an increasing set of state and municipal buyers face restrictions under NDAA Section 889 and FAR 52.204-25 on cameras manufactured by certain Chinese vendors. Retrofitting AI onto a camera that the buyer cannot legally retain is not retrofit; it is a procurement violation in slow motion. The supply-chain status of every camera in the fleet should be verified at the bill-of-materials level before integration planning begins.

Inadequate lens, mount, or vantage point. A camera that points the wrong direction, has the wrong field of view, or is mounted at a height or angle that occludes the relevant scene cannot be rescued by AI. The detection model evaluates the frames it receives; it cannot synthesize information that is not in the frame. Site walks during the assessment phase should identify which cameras are in defensible vantage points and which are not. Cameras in the wrong place are camera-replacement candidates; cameras in the right place with the wrong AI are not.

VMS compatibility matrix: how AI sidecar appliances integrate with the dominant video management platforms

The Video Management System layer is the operator-facing console where security staff watch live feeds, search recorded video, configure alerts, and manage user access. A retrofit AI deployment must integrate with the existing VMS in a way that preserves the operator's workflow. Five integration surfaces dominate.

The pragmatic lesson from production deployments is that the VMS does not have to change for AI retrofit to work. The most common architecture is Pattern 2 (appliance sidecar) with ONVIF Profile M event publication into the existing VMS, plus an operator-facing sidecar console for detailed event review, evidence packaging, and analytics-specific configuration. The existing VMS remains the system of record for video; the sidecar appliance is the system of record for detection events. Operators continue to use the console they know; investigators and risk managers gain a parallel interface optimized for detection workflow.

Cloud-native VMS platforms are the slightly more constrained case because they tend to favor first-party analytics over third-party sidecar integrations. Buyers selecting a cloud-native VMS should treat its analytics roadmap, its webhook/API surface, and its ONVIF Profile M support level as procurement criteria, not as deployment-time discoveries. The procurement methodology briefing covers the proof-of-concept design for testing VMS-AI integration before purchase.

Procurement implications: supply chain, AI governance, and biometric privacy

The procurement layer of retrofit architecture is where the technology decisions intersect with the compliance regime. Three frameworks dominate.

NDAA Section 889 and FAR 52.204-25. Federal contractors and recipients of federal funding are prohibited from purchasing, installing, or retaining covered telecommunications equipment from certain identified Chinese vendors, including Hikvision, Dahua, and their resold OEMs. Retrofit projects in federal-adjacent environments must verify the bill of materials of the camera fleet before integration; an AI sidecar that integrates with a Hikvision OEM camera at a federal site creates compliance exposure that no analytics-layer guarantee can remediate. The Intelligence Hub maintains a deeper camera supply-chain compliance briefing on the specific FAR clauses and certifications required.

NIST AI Risk Management Framework. The NIST AI RMF (publication 100-1, January 2023) is becoming a de facto procurement reference for AI-augmented physical security. Federal buyers and an increasing set of state and enterprise buyers require vendors to map their systems to the RMF's seven trustworthiness characteristics, including validity and reliability, safety, security and resilience, accountability and transparency, explainability and interpretability, privacy enhancement, and fairness with managed harmful bias. Retrofit deployments map cleanly to the RMF because the inference-location decision (on-camera, on-appliance, on-cloud) is itself one of the variables the RMF asks the buyer to characterize and document.

State biometric privacy statutes. Illinois BIPA, Texas CUBI, Washington's MHMDA, and the growing patchwork of state biometric and consumer privacy laws regulate the collection, transmission, and storage of biometric identifiers. Pattern 2 (on-premises appliance sidecar) is the architecturally cleanest fit because the appliance computes detection signals locally, discards frames after evaluation, and does not produce or retain biometric identifiers. Pattern 4 (cloud bridge) creates a more complex compliance profile because full frames cross the customer-to-cloud boundary; whether that constitutes biometric data transmission depends on whether the cloud inference includes face embedding or other identifier extraction. The biometric privacy briefing covers the state patchwork in detail; the retrofit pattern decision is upstream of the compliance posture.

How to evaluate retrofit versus replacement on a real fleet

The retrofit-vs-replace decision should be made at the camera level, not the fleet level. Different cameras in the same facility commonly land on different sides of the line. The following five-step framework structures the assessment so procurement teams can defend the decision in front of finance.

Step one: enumerate the fleet. Inventory every camera by manufacturer, model, firmware version, ONVIF profile conformance, codec support, resolution, lens specification, mounting location, and field of view. Most facilities cannot produce this inventory from existing documentation; the inventory itself is the deliverable that makes the rest of the assessment possible.

Step two: map detection workload to camera capability. For each detection category (weapon, person, vehicle, perimeter intrusion, fall, loitering, restricted-zone entry), identify which cameras can deliver the resolution, frame rate, and vantage required. Cameras that cannot are replacement candidates regardless of their general condition; cameras that can are retrofit candidates regardless of their age.

Step three: verify supply-chain compliance. For each camera, verify NDAA Section 889 status, manufacturer ownership chain, and applicable state or federal procurement restrictions. Non-compliant cameras are replacement candidates regardless of technical capability.

Step four: model the appliance footprint. Given the retained camera count, codec mix, and resolution distribution, model the on-premises appliance count and specifications required for the target inference workload. Modern Jetson AGX Orin or Hailo-based appliances commonly support 25 to 60 concurrent 1080p H.265 feeds depending on model selection and detection density. Multi-appliance deployments are standard at facility scale; the appliance topology drops out of the camera math.

Step five: stress-test the proof of concept. Deploy the candidate retrofit architecture against a representative subset of cameras (typically 10 to 25 percent of the fleet) and measure latency, detection accuracy, false-positive rate, network impact, and operator workflow under live conditions. The proof of concept is the artifact that defends the rollout decision to the steering committee. The proof-of-concept methodology briefing walks the structure of the test plan.

The output of the five-step framework is a per-camera disposition: retrofit (keep the camera, attach to analytics appliance), replace (remove the camera, install a current-generation IP camera, attach to analytics appliance), or remove (the camera serves no defensible detection workload and should be decommissioned). In a typical enterprise facility, the disposition splits run roughly 70 to 80 percent retrofit, 15 to 25 percent replace, and a small remainder remove. The cost profile of that mix is dramatically below a full replacement, which is why retrofit became the dominant pattern in the first place.

Frequently asked questions about AI physical security retrofit and VMS integration

Continue the research

This briefing covers the integration-architecture layer of AI physical security retrofit. For deeper reading on adjacent pieces of the stack:

• Edge vs. Cloud AI Inference — where the model runs, latency arithmetic across architectures, and the four hybrid patterns that complement the retrofit decision.
• AI Video Analytics vs. Traditional CCTV — the foundational shift from recording-first to detection-first architectures that underpins the retrofit case.
• How AI Gun Detection Works — the model architecture and runtime layer that the retrofit pattern delivers into.
• AI Physical Security Total Cost of Ownership — the financial model for retrofit versus replacement across hardware, license, integration, and lifecycle.
• NDAA Section 889 and FAR 52.204-25 — the camera supply-chain compliance briefing that gates retrofit decisions in federal and federal-adjacent environments.
• The Definitive 2026 Guide to Proactive Computer Vision — the cross-cutting reference that situates retrofit within the broader proactive-detection paradigm.
• How IntelliSee deploys — the operational view of the appliance-sidecar retrofit pattern in production environments. To structure a retrofit assessment for your facility, contact the IntelliSee team.