Why Traditional Monitoring May Be Too Slow for AI Infrastructure
ShriSR Research & Insights | October 2026
Category: AI Infrastructure | Data Center Operations | Power & Thermal Management
The Emerging Thermal Challenge
Artificial intelligence is transforming the power and cooling requirements of modern data centers. GPU-intensive workloads, particularly Large Language Model (LLM) training, introduce significant variations in computational demand, electrical consumption, and heat generation.
Modern AI racks can consume more than 100 kW, concentrating substantial thermal loads within relatively small physical spaces. NVIDIA’s DGX H100, for example, has a maximum system power rating of approximately 10.2 kW. Multiple systems operating within a single rack can therefore create significant cooling demands. [1]
Unlike traditional enterprise applications, AI training workloads may exhibit rapid changes in GPU utilization as they transition between computation, communication, data loading, and checkpointing.
These changes can translate into variations in power consumption and heat generation.
The challenge is no longer simply providing sufficient cooling capacity. It is detecting and responding to rapidly changing thermal conditions before they threaten infrastructure availability.
1. AI Training Creates Dynamic Power and Thermal Loads
LLM training involves computational phases such as forward propagation, backward propagation, gradient synchronization, and checkpointing.
Each phase can impose different demands on GPUs, CPUs, memory, storage, and networking infrastructure.
During intensive GPU computation, power consumption can approach the equipment’s configured power limits. During communication or data-processing phases, consumption may decrease.
For example, NVIDIA’s H100 SXM GPU supports a thermal design power of up to 700 W. Eight such GPUs, together with supporting processors, memory, networking, and power conversion components, can create substantial system-level power demand. [2]
Almost all electrical energy consumed by computing equipment ultimately becomes heat.
Consider an AI rack whose electrical consumption increases from 80 kW to 110 kW during a training session.
The additional 30 kW represents an equivalent increase in heat that the cooling infrastructure must eventually remove.
Although this example is illustrative, it demonstrates an important operational issue: the thermal load generated by an AI rack can change considerably during a training session.
Facility cooling systems must accommodate these variations while maintaining equipment inlet temperatures within recommended operating limits.
ASHRAE’s data center environmental guidelines provide the engineering foundation for managing these conditions. [3]
2. The Seven-Minute Cooling Vulnerability
One of the most significant operational risks occurs during utility power disturbances.
Most enterprise data centers use Uninterruptible Power Supply (UPS) systems to maintain electrical continuity for critical computing infrastructure.
When utility power fails, UPS systems immediately support protected IT loads while standby generators start and assume the electrical load.
However, cooling infrastructure may operate on separate electrical distribution systems.
Depending on the facility design, chillers, pumps, Computer Room Air Conditioning (CRAC) units, Computer Room Air Handling (CRAH) units, and associated equipment may experience temporary interruptions or reduced operating capacity.
Although generator startup and electrical transfer typically occur much faster, restarting and stabilizing cooling equipment may require additional time.
In certain facility configurations, the complete restoration of cooling capacity could take several minutes.
Consider a scenario in which full cooling capacity is unavailable for seven minutes while AI servers continue operating on UPS-supported power.
A rack consuming 100 kW would generate:
100 kW × 420 seconds = 42 megajoules of thermal energy
This is equivalent to approximately 11.7 kWh.
Not all this energy immediately increases inlet-air temperature. Some is absorbed by equipment, coolant, air, and surrounding structures. Nevertheless, the example illustrates the substantial thermal energy that must be managed during a cooling interruption.
ASHRAE provides guidance on data center cooling infrastructure and equipment thermal requirements. Its publications reinforce the importance of understanding the interaction between electrical infrastructure, cooling systems, and operating conditions. [3][4]
The seven-minute period is an illustrative facility-specific scenario, not an industry-standard generator transfer time.
The critical risk is that IT equipment may remain energized while the infrastructure responsible for removing its heat is temporarily impaired.
3. Localized Heat Can Threaten Neighboring Racks
Data center cooling is often evaluated using room-level temperatures, airflow measurements, and overall cooling capacity.
However, AI workloads can generate highly concentrated thermal loads.
A rack containing GPU servers running an intensive training workload may generate substantially more heat than adjacent racks operating conventional applications.
Under normal conditions, cooling infrastructure removes this heat through controlled airflow or liquid-cooling systems.
During reduced cooling availability, temperatures can increase rapidly.
In air-cooled environments, inadequate airflow or containment can allow hot exhaust air to recirculate toward equipment inlets.
Consequently, a thermal problem originating in one rack may affect neighboring racks.
The extent of this effect depends on several factors:
Physical proximity and rack orientation.
Hot-aisle and cold-aisle containment.
Airflow direction and recirculation.
Cooling equipment placement and available capacity.
Shared cooling zones.
Rack power density and thermal characteristics.
ASHRAE’s environmental guidelines emphasize the importance of maintaining appropriate equipment inlet conditions and managing airflow. [3]
A thermal incident in one AI rack should therefore be evaluated as a potential cooling-zone incident, not merely an isolated equipment alarm.
Monitoring platforms should identify neighboring racks and other equipment dependent on the same cooling infrastructure.
This requires combining environmental telemetry with accurate physical rack locations and cooling-system relationships.
4. Why Traditional Polling Can Miss Critical Events
Data center monitoring systems commonly collect operational information using protocols such as SNMP, NETCONF/YANG, gNMI/OpenConfig, and Redfish.
These technologies provide access to equipment temperatures, electrical consumption, operational status, and other measurements.
However, the effectiveness of monitoring depends not only on the protocol but also on the collection mechanism.
Many platforms continue to rely heavily on periodic polling.
A monitoring system may collect temperature and power measurements every 60, 120, or 300 seconds.
Consider a rack whose temperature is polled every five minutes.
If a cooling failure occurs immediately after a successful poll, the monitoring platform may not observe the resulting temperature increase until the next scheduled collection.
During that interval, the rack could experience a substantial thermal excursion.
Similarly, a short-lived power spike may occur entirely between polling cycles.
The monitoring system may be functioning correctly while remaining unaware of important events occurring between measurements.
This limitation is particularly relevant when infrastructure conditions change faster than the configured polling interval.
The OpenConfig gNMI specification explicitly supports alternatives to traditional polling, including streaming subscriptions and updates generated when monitored values change. [5]
NETCONF also supports asynchronous event notifications through standardized mechanisms. [6]
The underlying issue is therefore not that traditional management protocols are inadequate.
It is that periodic polling alone may be insufficient for detecting rapidly developing thermal and electrical conditions.
Depending on device capabilities and configuration, telemetry may be delivered through periodic sampling or updates associated with changes in monitored values.
This can reduce dependence on infrequent polling cycles.
However, gNMI ON_CHANGE support is implementation-dependent, and not every numerical measurement is necessarily available through change-driven subscriptions. [5]
Temperature thresholds approaching equipment operating limits.
Sudden increases in rack or PDU power consumption.
Power consumption exceeding defined capacity thresholds.
Cooling equipment failures or reduced cooling capacity.
Unexpected temperature differences between neighboring racks.
Utility power failures and changes in UPS operating state.
Where supported, alerts should incorporate rate-of-change analysis rather than relying exclusively on absolute thresholds.
For example, a rapid temperature increase may justify an early warning even before the rack reaches its maximum permitted inlet temperature.
Event-driven monitoring does not guarantee instantaneous detection. Sensor response times, device processing, notification configuration, and network latency remain important.
Nevertheless, properly implemented notifications can alert operators significantly earlier than infrequent polling.
The objective is to alert operators when a dangerous condition begins developing—not merely when the next polling cycle discovers it.
6. Colocation Data Centers: Monitoring Without Tenant Access
Colocation facilities introduce another challenge.
The facility operator may be responsible for cooling and environmental conditions without having management access to tenant-owned computing equipment.
Tenants may prohibit direct access to SNMP, Redfish, NETCONF, or gNMI interfaces because of security policies, contractual restrictions, or network isolation requirements.
Consequently, operators may lack visibility into GPU utilization, server temperatures, and equipment-level electrical consumption.
ShriSR recommends deploying independent IoT-based environmental sensors where direct access to tenant equipment is unavailable.
These sensors can measure:
Rack inlet temperatures at multiple heights.
Rack exhaust temperatures.
Temperature changes in adjacent racks.
Humidity and other environmental conditions.
Rack or branch-circuit power consumption, where separately metered.
Sensors can communicate through an independent wired or wireless monitoring network without accessing the tenant’s management infrastructure.
This provides environmental visibility while preserving tenant security boundaries.
ASHRAE’s environmental guidelines provide a basis for selecting measurement locations and evaluating equipment inlet conditions. [3]
Independent sensors cannot directly measure internal GPU temperatures or utilization without access to the relevant device telemetry.
Nevertheless, they can help operators detect developing thermal conditions that would otherwise remain invisible.
7. From Individual Alarms to Location-Aware Thermal Intelligence
Detecting an elevated temperature is useful.
Understanding its potential impact is considerably more valuable.
When a rack reports a rapid temperature increase, operators should be able to determine:
Where the affected rack is located.
Which racks are physically adjacent.
Which cooling equipment serves the rack.
Whether neighboring racks are experiencing similar temperature increases.
Whether cooling capacity has decreased.
Whether electrical demand has increased.
This requires correlating monitoring data with the physical and operational relationships within the data center.
A location-aware infrastructure model or Digital Twin can provide this context.
For example, when a cooling unit fails, the monitoring system could identify all racks dependent on that unit and prioritize their temperature measurements.
If several racks simultaneously report rising inlet temperatures, the system could recognize a cooling-zone incident rather than treating each measurement as an unrelated alarm.
More advanced implementations could incorporate airflow analysis, historical measurements, and thermal simulation to estimate the possible spread of overheating.
The value lies in connecting telemetry to physical infrastructure relationships so that operators understand both the event and its potential consequences.
⸻
ShriSR Recommendations
AI data centers require monitoring strategies that account for rapidly changing power demand, concentrated thermal loads, and temporary cooling disruptions.
ShriSR recommends five operational priorities.
1. Combine polling with event-driven telemetry. Use SNMP Traps/Informs, gNMI subscriptions, NETCONF notifications, and Redfish events wherever supported. Retain periodic polling for baseline measurements, historical analysis, and verification.
2. Monitor temperature and power rates of change. Detect rapidly developing conditions before absolute alarm thresholds are reached.
3. Correlate alerts with physical rack locations. Identify neighboring racks, shared cooling infrastructure, and potentially affected cooling zones.
4. Deploy independent sensors in colocation environments. Use IoT-based environmental monitoring when tenant security policies prevent direct access to equipment telemetry.
5. Validate cooling continuity during power transitions. Test utility failures, UPS operation, generator transfers, cooling equipment restart, and restoration of full cooling capacity. Establish workload-throttling or shutdown procedures where necessary.
Event-driven monitoring should complement—not replace—appropriate cooling design, redundancy, equipment thermal protection, and emergency operating procedures.
Conclusion: The Heat Trap Is a Visibility Problem
AI workloads are changing the relationship between computing, electrical power, and cooling.
GPU-intensive training can create substantial fluctuations in power consumption and concentrated thermal loads.
Temporary cooling interruptions can amplify these risks, particularly when AI equipment continues operating on UPS-supported power.
Traditional monitoring systems may eventually detect these conditions.
The more important question is whether they detect them early enough.
AI data center monitoring must evolve from periodically observing infrastructure conditions to detecting meaningful changes, understanding their physical impact, and enabling timely intervention.
For data center operators, the objective is straightforward:
Detect the heat. Understand its impact. Respond before it spreads.
⸻
References
[1] NVIDIA — DGX H100 Datasheet Technical specifications and maximum system power requirements for NVIDIA DGX H100 AI infrastructure. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/nvidia-dgx-h100-datasheet.pdf
[3] ASHRAE — Data Centers and Telecommunication Facilities Engineering guidance on data center environmental conditions, equipment inlet temperatures, airflow, and cooling design. https://handbook.ashrae.org/Handbooks/A15/SI/a15_ch19/a15_ch19_si.aspx
[4] ASHRAE TC 9.9 — Data Center Power Equipment Thermal Guidelines and Best Practices Technical guidance on thermal considerations associated with data center power infrastructure. https://www.ashrae.org/file%20library/technical%20resources/bookstore/ashrae_tc0909_power_white_paper_22_june_2016_revised.pdf
[7] IETF RFC 3413 — Simple Network Management Protocol Applications Defines SNMP notification originators and receivers, including traps and informs. https://www.rfc-editor.org/rfc/rfc3413.html
[8] DMTF — Redfish Specification Defines standardized management interfaces, event services, and telemetry-related capabilities for IT infrastructure. https://redfish.dmtf.org/
⸻
About ShriSR
Shri S R Software Solutions Pvt. Ltd. (ShriSR) provides independent research, technology assessments, and consulting services for network and data center infrastructure.
Our research covers infrastructure monitoring, observability, network automation, power and environmental intelligence, Digital Twin technologies, and multi-vendor infrastructure management.
ShriSR Research & Insights | www.shrisr.com
Discuss your data center requirements and how ShriSR can help.
