Distribution of Cooling Airflow in a Raised-Floor Data Center
Subas V. Patankar, Pb.D. Kailash C. Karki, Ph.D.
ABSTRACT For reliable operation of computer
equipment in a data center; adequate cooling air must besupplied to the equipment. The distribution of cooling air through the perforated tiles in a raised-floor data center is governed by the fluid mechanics of the underfloor space. The pressure variation in that space is shown to be the cause of nonuniform distribution of airflow. The various factors that influence the distribution are discussed. The efect offloor height and tile open area is illustrated through the results for a simple configuration. The use of variable tile open area and other issues are also discussed. Calculated distributions of airflow rates are used to explain some of the observed behavior in data centers on the basis of fluid mechanics principles. INTRODUCTION
Raised-floor data centers are commonly used to house computer servers, telecommunications equipment, and data storage systems. The equipment dissipates a significant amount of heat and must be maintained at acceptable temperatures for reliable operation. It is not sufficient that the data center as a whole receives the required amount of cooling air. Each piece of equipment must be given the amount of cooling air that corresponds to its heat load. Therefore, special attention must be paid to the distribution of cooling air in the data center. This paper uses the fluid mechanics of the underfloor space to calculate the distribution of the airflow rate through the perforated tiles. The effect of different factors such as the floor height and the open area of the perforated tiles is discussed.
THE RAISED-FLOOR CONCEPT
Raised-floor data centers use the underfloor plenum below a raised floor to supply cooling air to the computer equipment. As shown in Figure 1, the computer room air conditioner (CRAC) units push cold air into the plenum, from where it is introduced into the computer room via perforated floor tiles, tile cutouts, and other openings. The raised-floor design offers considerable flexibility in placing the computer equipment above the raised floor. The underfloor plenum serves as the distribution chamber for the cooling air. Without the need for any ducting, cooling air can be delivered to any location simply by replacing a solid tioor tile by a perforated tile. A common arrangement for the perforated tiles and the computer equipment is the so-called “hot aisle-cold aisle” layout, which is shown in the plan view of Figure 1. Perforated tiles are placed in a region called the cold aisle. On each side of the cold aisle, computer racks are placed with their intake sides facing the cold aisle. A hot aisle is the region between the back ends of two rows ofracks. The cooling air delivered by the perforated tiles is drawn into the intake side of the racks. This air heats up inside the racks and is exhausted from the back of the racks into the hot aisle. From the hot aisle, the heated air returns to the CRAC units.
REQUIREMENTS FOR AIRFLOW DISTRIBUTION A necessary condition for good thermal management is to supply the required airflow through the perforated tiie(s) located near the inlet of each computer server. The heat load can vary significantly across the computer room, and it changes with the addition or reconfigurationof hardware. For all computer servers to operate reliably, the data center design
Suhas Patankar is the president and Kailash Karki is a principal engineer at Innovative Research, Inc., Plymouth, Mim.
To CRAC Unit
150 CFM, 55OF
150 CFM, 55OF
Figure 2 Insuficient cooling air-ow.
Figure 1 A schematic of a raised-Joor data center:
must ensure that the cooling air distributesproperly, that is, the distribution of airflow rates through perforated tiles meets the cooling air needs of the equipment on the raised floor. When adequate airflow is not supplied through the perforated tiles, the internal fans in the server racks tend to draw air from the ceiling space. Since most of this air originates in the hot aisle, its temperatureis high. Thus, the cooling of the upper parts of the server racks is seriously compromised. This behavior is schematically shown in Figure 2. Although the picture is a simplified representation of what really happens, it does capture the main physical phenomenon. Thus, the key to satisfactory cooling in a data center is to deliver the required amount of cooling airflow at the inlet of each server. If the temperature rise of the air flowing through the server is to be limited to 20"F, the airflow requirement can be calculated from Required airflow in CFM = 154 x (the server heat load in kW). This formula is appropriate at sea level. For higher altitudes, the required airflow should be multiplied by the ratio (atmospheric pressure at sea level) / (atmospheric pressure at the local altitude).
At first sight, it may appear that, once the plenum is pressurized (by the inflow from the CRAC units), each perforated tile will deliver the same amount of airflow (at least when the perforated tiles are identical in construction).Actually, there is a significant variation in the flow rates from different perforated tiles. There are many factors that are responsible for the variation. A major factor is the fact that different perforated tiles are located at different distances from the CRAC unit. Further, the pattern of the airflow distribution is somewhat counterintuitive. One may expect more flow near the CRAC unit and less away from it. In reality, there is very little flow near the CRAC and very large flow through the perforated tiles located far away. As a result, the computer equipment placed near the CRAC does not get much cooling air.
PRESSURE VARIATIONS IN THE UNDERFLOOR SPACE
The flow rate through a perforated tile depends on the pressure drop across the tile, that is, the difference between the plenum pressure just below the tile and the room pressure above the raised floor. Pressure variations within the computer room are generally small compared to the pressure drop across the perforated tiles. Thus, relative to the plenum, the pressure just above the perforated tiles can be assumed to be uniform. The flow rates through the perforated tiles, therefore, depend directly on the plenum pressure just below the tile. The nonunifonnity in the airflow distributionis caused by the horizontal pressure variations under the raised floor.
ROLE OF THE FLOW FIELD UNDER THE RAISED FLOOR
Interestingly, the distribution of the cooling airflow through the perforated tiles is governed by the fluid mechanics of the space below the raised floor. It is not the large, visible, above-floor space that controls this flow distribution. It is the air movement in the tiny underfloor space that decides how much air will emerge from each perforated tile.
THE BASIC CAUSE OF FLOW MALDISTRIBUTION
The main reason for nonuniform distribution of airflow through the perforated tiles (and its counterintuitive nature) can be understood from the simple example shown in Figure 3. Here, the flow in the vicinity of one CRAC unit is shown. The CRAC flow enters the plenum in a vertically downward direction, turns 90 degrees, and then proceeds horizontally. As the flow moves under the perforated tiles, some air exits from
ASHRAE Transactions: Symposia
Velocity decreases, pressure increases
Figure 3 The basic cause offlow maldistribution.
the plenum into the computer room. The horizontal air velocity in the vicinity of the CRAC is very high since it has to carry the entire flow delivered by the CRAC. As air leaks out of the perforated tiles, successively less and less flow moves in the horizontal direction. Thus, the horizontal velocity decreases from left to right. According to laws of fluid mechanics, a velocity decrease is accompanied by a pressure increase. (A simple form of this behavior is expressed by the well-known Bernoulli equation.) Thus, the static pressure in the plenum increases from left to right. Now, it is easy to see why the perforated tiles near the CRAC give smaller flow than those far away. Under some conditions, the nonuniformity of airflow distribution is so severe that the flow through the perforated tiles near the CRAC is not simply small but even negative. That is, the static pressure in the plenum in the region close to the CRAC is actually less than the room pressure. The high velocity stream in the plenum entrains the room air through the perforated tiles located close to the CRAC. Obviously, the computer equipment placed in this area will not be properly cooled.
Figure 4 Effect of raised-floor height on airflow rates.
ments in a real-life data center. The CFD technique used is based on the methodology described in Patankar (1980). In this paper, instead of presenting results for complex layouts, simplified scenarios are used to provide a qualitative understanding of the flow parameters and to establish some general trends. This is described in the following sections.
EFFECT OF THE RAISED-FLOOR HEIGHT
As seen from Figure 3, the variation of static pressure in the plenum results from the variation of the horizontal velocity. It, therefore, follows that, if the magnitude of the maximum velocity is small, the corresponding pressure variations will be mild. The magnitude of the velocity depends on the height of the raised floor (which decides the area available for the flow supplied by the CRAC). For the same CRAC flow rate, the magnitude of the velocities in a 24-inch (0.61 m) raised floor will be only half as much as that in a 12-inch (0.3 m) raised floor. The result is that a larger height will lead to a more uniform pressure distribution. The pressure distribution in the plenum governs the distribution of airflow through the perforated tiles. Thus, larger floor heights lead to a more uniform airflow distribution. This effect is shown in Figure 4 for a particular case. The CFM supplied by each tile is plotted for a row of 15 perforated tiles in front of a CRAC. Each curve correspondsto a different height of the raised floor. Whereas the flow distribution is quite nonuniform for small heights such as 6 or 12 inches (O. 15 or 0.3 m), it becomes progressively more uniform as larger heights are used. For the small heights, the nonuniform distribution includes regions of negative flow, where air is drawn from the computer room into the plenum. An additional observation is relevant here. What is truly significant is the actual flow area available for the air. If a 12inch (0.61 m) raised floor space is filled with cables and other obstructions, it may behave like an 8-inch or 6-inch (0.2 m or
FACTORS AFFECTING THE AIRFLOW DISTRIBUTION
Whereas the main cause of flow nonuniformity is explained by the simple example in Figure 3, a number of factors influence the distributionof airflow through perforated tiles. They include the locations of the CRACs and the corresponding spreading of the underfloor flow to various perforated tile locations; collision or merging of the air streams coming from different CRACs; and the flow disturbance caused by underfloor blockages such as pipes and cable trays. Geometrical factors, such as the height of the raised floor and the amount of open area of the perforated tiles, also affect the flow distribution. . A quantitative evaluation of all these factors in a real data center requires a detailed numerical calculation of the threedimensional flow field in the underfloor space. Karki et al. (2003) present a computational fluid dynamics (CFD) model ofthe underfloor space and compare the results with measureASHRAE Transactions: Symposia
Variable Open Area Uniform Open Area
0 20 o!
Q 1 0 1 1 1 2 1 3 1 4 1 5
1 1 1 io ' Tile Number
f Figure 5 Effect o tile open area on airflow rates.
O. 15 m) floor leading to more severe nonuniform distribution of airflow.
EFFECT OF THE OPEN AREA OF THE PERFORATED TILES
The flow resistance of the perforated tiles depends on their open area. For a given airflow rate, a tile with only 10% open area will require a much greater pressure drop than a tile with 25% open area. As described above, the nonuniformity in the airflow distribution is caused by the horizontal pressure variations in the plenum. These variations should be judged in comparison with the pressure drop across the perforated tiles. A pressure variation that is significantfor a 25% open tile may not be very significant for a 10% open tile (in comparison with the much larger pressure drop across it.) It, therefore, follows that restrictive tiles (such as 10% open) give a nearly uniform airflow distribution, while open tiles (25%) create nonuniformities. The open tiles readily respond to the pressure variations in the plenum. The restrictive tiles (due to the large pressure drop across them) are insensitive to the pressure variations. This behavior is shown in Figure 5 for a typical case. Again, the CFM supplied by each tile is plotted for a row of 15 perforated tiles in front of a CRAC. Each curve corresponds to a particular value of the fractional open area of the perforated tiles. The flow distribution is significantly nonuniform for more open tiles such as 50% or 70%. They lead to large regions of negative flow near the CRAC. The flow uniformity improves as more restrictive tiles are used. Some additional comments are relevant here. First, there is a common misconception that having more open tiles increases the airflow rate. Obviously,for the same staticpressure in theplenum, more open tiles will produce more airflow than more restrictive tiles. However, the static pressure in the
1 1 1011
1 I I 1 2 1 3 1 4 1 5
Figure 6 Flow rates for a variable tile open area arrangement.
plenum is not a given constant; it is a result of the flow resistance of the tiles. The airflow rate is controlled by the amount of flow the CRAC blower is able to supply. For the blower, the controlling resistance is internal to the CRAC unit. For most situations, the additional flow resistance offered by the perforated tiles is rather insignificant. By lowering this resistance, it is not possible to increase the flow. In general, the open area of the perforated tiles will influence the flow uniformity (as seen in Figure 5), not the amount of flow. Second, the flow resistance of more restrictive tiles (such as 10% open) is comparable to the flow resistance of other flow openings such as cable cutouts and cracks. When restrictive tiles are used, the air will have an increased tendency to leak through these other openings. This may be undesirable in some circumstances. Third, when very restrictive tiles (for example, 5% open) are used, their flow resistance does influence the flow delivered by the CRAC blower. In this case, the plenum pressure becomes so high that the blower gives a smaller flow rate against the high back pressure. Under such conditions, the usual assumption of constant airflow supplied by a CRAC does not hold.
ASHRAE Transactions: Symposia
USE OF VARYING OPEN AREA OF PERFORATED TILES
As seen above, the use of restrictive tiles leads to a more uniform airflow distribution. However, this may have some undesirable side effects. Instead of using restrictive tiles everywhere, one can make selectiveuse of restrictive and open tiles in different locations to get the desired airflow distribution. For making the flow distributionuniform, it is necessary to encourage the flow near the CRAC and discourageit atpositions away from the CRAC. Figure 6 shows one possible arrangement, where the perforated tiles have a large open area near the CRAC and they progressively become restrictive away from the CRAC. This layout is compared with the use of 25% open tiles everywhere.From the calculated airflow rates, it can be seen that a much greater uniformity is achieved by employing varying open areas.
Karki, K.C., A. Radmehr, and S.V. Patankar. 2003. Use of computational fluid dynamics for calculating flow rates through perforated tiles in raised-floor data centers. Znt. J. ofHVAC&R Research, Vol. 9, pp. 153-166. Patankar, S.V. 1980. Numerical Heat Transfer and Fluid Flow. Taylor and Francis.
DISCUSSION Jim VanGilder, Staff Mechanical Engineer, APC, Billerica, Mass.: With many perforated tiles in a data center of the 56% variety, the AP through the tile falls to the point where pressure variations in the room need to be considered in order to correctly predict tile flow rates. Therefore, the whole room would need to be simulated.That said, even with careful simulation, I wouldn’t recommend the use of the large-open-area tiles in most cases, as tile airflow is generally very nonuniform and small changes to room layout can produce large changes in cooling airflow distribution. Kailash C. Karki: The airflow rates through perforated tiles can be predicted solely from the knowledge of pressure distribution in the underfloorplenum only if the pressure variations above the raised floor are small compared to the pressure drop across the perforated tiles. This topic has been discussed in detail in Reference 1. The airstream exiting”a perforated tile produces a pressure variation of one velocity head above the raised floor. Thus, the assumption in the present approach is valid as long as the pressure drop across the perforated tiles is significantly larger than one velocity head. For a 25% open tile, the pressure drop is about 43 velocity heads, whereas for a 56% open tile, it is about 5 velocity heads. Thus, if all perforated tiles were 56% open, the present approach will produce somewhat approximate results, and the precise pressure distribution above the raised floor will be required to produce more accurate results. However, as Mr. VanGilder has pointed out, use of perforated tiles with large open areas leads to highly nonuniform and sensitive airflow distribution. In a practical data center design, a large majority of perforated tiles would be 25% open (or close to it) and only a few tiles at selected locations would have larger open area. For these conditions, the pressure variations above the raised floor can be ignored and, thus, the airflow rates can be calculated solely from the knowledge of the flow field in the plenum. Mike Kearney, Chairman of the Board, Michael C. Kearney & Associates, Inc., St. Louis Mo.: What degree of obstruction underfloor produces results on slide?
EFFECT OF UNDERFLOOR OBSTRUCTIONS A raised-floor data center usually has underfloor obstructions such as pipes, cable trays, and structural beams. The obstructions cause a disturbance in the airflow pattern under the floor, influence the pressure distribution, and thus affect the airflow coming out of the perforated tiles. Since the obstructions reduce the area available for the flow, the air velocity increases and leads to more significant pressure changes. Usually, the static pressure will increase on the upstream side of an obstruction and decrease on the downstream side. Because of this effect, two adjacent perforated tiles may give very different airflow rates. CONCLUDING REMARKS
It is shown that the key to a satisfactorydata center design is to deliver the required amount of cooling airflow at the inlet of each computer. The distribution of the cooling airflow through the perforated tiles is governed by the fluid mechanics of the underfloor space. The pressure variations in the underfloor space can cause significant variations in the distribution of airflow through the perforated tiles. These pressure variations result from the velocity variations as the air travels from the computer room air-conditioner (CRAC) to the perforated tiles. The airflow distribution is influenced by raised-floor height, perforated tile layout, tile open area, placement of CRACs, and underfloor blockages. In particular, larger floor heights give a more uniform distribution than smaller floor heights. Also, restrictive tiles (small open area) tend to make the airflow distribution uniform, while significant nonuniformities occur when more open tiles are used.
ASHRAE Transactions: Symposia
Karki: Underfloor obstructions affect the airflow distribution when their sizes are comparable to the plenum height. In the present example (Figure Al), the plenum height is 12 in. and the diameter of the obstruction (a circular pipe) is 6 in. (Note
that the degree to which an obstruction affects the airflow distribution depends not only on the size of the obstruction, but also on its location.)
ASHRAE Transactions: Symposia