Simulation and experimental research on the optimization of airflow organization and energy saving in data centers using air deflectors

.


Introduction
According to China Industry Information Network's report on the global data center construction industry development trend and market scale forecast in 2018 [1], the total amount of global data will increase from 16.1 ZB in 2016 to 163 ZB (approximately 18 billion GB) in 2025.The amount of data will grow 10 times in 10 years, with a compound growth rate of 26%.As the demand for computing power continues to grow, the safe and stable operation of data centers, as well as energy conservation and emission reduction, are currently hot issues in academia and the industry.
Data centers are high-energy-consuming buildings.According to statistics, the data center industry accounts for 2% of the world's total energy consumption [2], and a data center's average power consumption is about 30 times that of a standard office [3].As the number of data centers further increases, the energy consumption of data centers doubles every five years [4][5][6].By 2015, the total energy consumption of global data centers accounted for 4.5% of the world's total energy consumption and is expected to reach 8% by 2030 [7].In addition to the increasing construction volume of data centers, another reason for the high energy consumption of data centers is their low overall energy efficiency.Power usage effectiveness (PUE) is a commonly used indicator to measure the energy utilization efficiency of data centers.PUE is the ratio of the total power consumption of data center equipment to the power consumption of IT equipment.The closer the value is to 1, the higher is the energy efficiency [8].The power consumption of data centers includes IT equipment, cooling systems, lighting systems, power supply systems, etc.Among them, the cooling system and the IT power supply system consume the most significant energy.For PUE values of 1.8 and 3.0, the electricity consumption of the cooling system accounts for 35% and 55%, respectively [9].Reducing the energy consumption of the cooling system will directly reduce the PUE value.
The internal cooling environment of data centers is a necessary condition to ensure the normal operation of equipment, and the rationality of airflow organization has a great impact on energy consumption levels.Studies have shown that an unreasonable layout of data centers will lead to chaotic airflow, resulting in nearly 60% waste of the cooling capacity [10].Therefore, reasonable planning of airflow organization design and reduction of local hot spots have become important measures for data centers' energy conservation.The methods of airflow organization in data centers can be divided into long-distance cooling and short-distance cooling according to the length of the cooling path.For long-distance cooling, the cooling air generated by the computer room air conditioner (CRAC) or computer room air handler (CRAH) can be sent to the cold channel through the perforated floor or ceiling by the form of raised floor or ceiling air supply, and finally, it can exchange heat with the cabinet in the distance, and the hot air after heat exchange will gather in the hot channel and circulate back to CRAC (or CRAH) accordingly.For short-distance cooling, the distance between the CRAC (or CRAH) equipment (such as inter-row air conditioners, etc.) and the cabinets is greatly shortened, which effectively reduces/eliminates hot air recirculation and results in higher cooling efficiency.
Raised-floor air supply is currently the most commonly used air-cooling method in data centers [11].Due to the long transportation distance of the cooling air, the speed and pressure of the air are affected by many factors and usually lead to uneven airflow organization.It not only causes high energy consumption and low energy efficiency of the cooling system but also easily cause local server overheating [12] (called "local hot spots") and excessively high CPU temperature.According to statistics, for every 2 ℃ increase in CPU temperature after the CPU reaches a safe temperature, reliability is reduced by 10% until the CPU is damaged [13].Therefore, optimizing the airflow organization of the data center is of great significance to ensure server performance and avoid downtime [14].
The poor airflow organization in data centers mainly includes three aspects: 1) uneven airflow distribution, 2) airflow loss, and 3) an imbalance between airflow supply and demand.Uneven airflow distribution is reflected in chaotic temperature distribution, which is usually accompanied by uneven wind pressure distribution.The location of the CRAC and cabinet rows, the height of the floor plenum, and the opening ratio of the perforated floor are the main influencing factors.The uniformity of air supply can be improved by optimizing the parameters of the static plenum [15][16][17][18] and the opening ratio of the perforated floor [19][20][21][22][23][24][25][26].Increasing the depth of the static plenum can reduce the pressure gradient of the airflow, thereby improving the uniformity of the air supply, but there is a limit to the maximum depth.The generally recommended depth of a data center's static plenum is between 600 mm and 1080 mm.Also, the pressure loss of the airflow is inversely proportional to the opening ratio of the perforated floor.An opening ratio of 25% can effectively improve the uniformity of the air supply on the perforated floor [27].In addition, the uniformity of the airflow on the perforated floor can also be improved by installing a fan [28].To solve the problem of airflow loss, methods such as channel sealing and gap sealing, among others, can be used to reduce the cooling power consumption of data centers by 8.8%-47% [29,30].The reason for the imbalance between airflow supply and demand in data centers is mainly caused by the mismatch between the actual airflow entering a cabinet and the total airflow demand of the cabinet.When the airflow supply is greater than the airflow demand, the excess cold air directly returns to the CRAC to form an airflow short-circuit [31].And because the air on the perforated floor flows out at a high vertical speed, the horizontal component of the airflow speed for the lowermost servers is small and the airflow will directly pass by the lowermost servers, resulting in insufficient airflow rate, and therefore the servers cannot be effectively cooled.When the airflow supply is less than the airflow demand, hot air will flow back [32] and the servers in the upper part of the cabinet cannot draw the demanded air volume due to insufficient cold air power, which will also cause a poor cooling effect [33].
In actual operation and maintenance, servers in most data centers are arranged in the middle and lower parts of cabinets, and it is very easy for the air temperature to rise due to the mismatch between the cooling airflow supply and demand.Alkharabsheh et al.'s research pointed out that a 6.8% reduction in a server's airflow rate caused a 10 ℃ rise in certain parts of the server [34], seriously affecting the efficient operation and computing functions of the server.Currently, most of the research on the thermal management of data centers is at the room scale, and further research is needed on the optimal airflow distribution at the cabinet scale.In order to solve the above problems, this work studied a cabinet-scale airflow organization optimization method based on air deflectors.With the installation of deflectors under the perforated floor, the air outlet direction from the perforated floor was controlled in a targeted manner, and part of the air volume was directed to the bottom server.Combining numerical simulations and experimental verification, this study obtained optimal design parameters and layouts of deflectors under different working conditions.By modeling and simulation of data centers, the impact of installing deflectors on the cabinet-scale cooling airflow velocity field and temperature field was analyzed, and the energy efficiency improvement of the CRACs and cooling system was evaluated.The relevant conclusion of this study can provide guidance and suggestions for improving the airflow organization of data center.

Experimental platform and multi-condition airflow rate experiment 2.1. Experimental platform and equipment
In order to achieve controllability and repeatability of the experimental environment, an artificial environment cabin was used to create precise working conditions to simulate the airflow organization experiment in a data center.The environmental cabin is located in the Tianjin University, as shown in Figure 1.The dimension of the environmental cabin was 4100 mm (length) × 4000 mm (width) × 4000 mm (height), and the ceiling height was 3.25 m.The floor of the environmental cabin was made of polyurethane foam with an aluminum-plate surface, and the outer protective structure had good thermal insulation properties.The environmental cabin was equipped with precision air conditioning, which can accurately control air temperature, humidity, and volume.In order to accurately create the on-site air supply situation of a static pressure plenum to a data center, a closed chamber was built inside the environmental cabin to simulate an actual data center.The floor plan of the environmental cabin and the closed chamber is shown in Figure 2. A static plenum with a height of 700 mm was built in the closed chamber to deliver the cooling airflow.The experimental server cabinet adopted the standard 42U form of 600 mm (length) × 800 mm (width) × 2000 mm (height).The cabinet was 600 mm away from the south wall and 450 mm away from the west wall.The cooling airflow generated by the air conditioner was sent into the static pressure plenum by a fan.At the end of the air duct, the cooling air flowed out through the perforated floor and exchanged heat with the servers.The layout of the data center laboratory is shown in Figure 3.

Experimental study on airflow rate distribution
In order to reveal the influence of different layouts of servers in a cabinet on airflow rate distribution, this study used an impeller anemometer to obtain the inlet air speed at different locations of the servers, and collect data on the airflow rates for the servers based on the air inlet area.The servers were placed in four layouts, as shown in Figure 4: A) servers concentrated in the lower part of the cabinet, B) servers concentrated in the middle of the cabinet, C) servers concentrated in the upper part of the cabinet, and D) servers in a dispersed arrangement.The test result is shown in Table 1.The A-③, C-①, and D-① servers had small cooling airflow rates, which were 182.77m³/h, 192.26 m³/h, and 189.89 m³/h, respectively.Therefore, the servers at the top and bottom of the cabinet were prone to insufficient cooling airflow rates.The insufficient cooling airflow rate for the upper servers of the cabinet was caused by the attenuation of the upward airflow speed, which can be solved by increasing the airflow rate through the perforated floor.For the servers at the bottom of the cabinet, the vertical velocity component of the air coming out of the floor was large and the horizontal velocity component was small, resulting in insufficient airflow rate entering the bottom servers.If the airflow rate was increased, although this would supply more inlet airflow rate for the upper servers, it would also further increase the vertical speed of the cooling airflow, making the problem of insufficient airflow rate for the bottom servers more significant.Taking into account the supply and demand balance for effective cooling airflow in the cabinet, the optimization of the airflow organization of the entire cabinet needs to prioritize the solving of the problem of insufficient airflow rate for the bottom servers.The deflectors were a passive measure to guide the airflow.The aim is to lead the airflow into place with insufficient airflow rates.They acted as a diversion to achieve a more economical and reasonable heat dissipation effect.The deflectors included cabinet-level and channel-level deflectors.The cabinet-level deflectors are classified into front deflectors and rear deflectors based on their positions.The installation angle and size of the deflectors were key factors that can affect airflow distribution.Therefore, this paper adopts parameter sensitivity analysis to explore the impact of different deflector widths and angles on the airflow rate distribution for servers in the upper, middle and lower parts of the cabinet.The deflector widths of 100 mm, 200 mm and 300 mm and angles between deflector and the perforated floor of 30°, 45° and 60°.The different deflector installation layouts are shown in Figure 5. Three servers of the same model were placed in the upper, middle and lower positions in the same cabinet.The corresponding server numbers were ① for the upper server, ② for the middle server, and ③ for the lower server.An impeller anemometer was used to obtain the air inlet speed, and combined with the air inlet area of the server, the airflow rates for the three servers were obtained, and the test result is shown in Table 2.The standard deviation of the airflow rate for each server in the same cabinet was used to evaluate the uniformity of the airflow rates.Without the deflector, the standard deviation of the airflow rate distribution of each server was 10.28.The horizontal speed of the airflow passing through server ③ was too small, its airflow rate was much smaller than the average airflow rates for other servers, and hence the cooling effect for server ③ was not good and local hot spots formed easily.When different deflector layouts were used, the airflow rate for the lowermost server improved to varying degrees.According to Table 2, it can be seen that when the width of the deflector was 100 mm and the angle between the deflector and the perforated floor was 45°, the standard deviation of the airflow rate distribution of each server was the smallest and the airflow was the most uniform.Therefore, based on the increased airflow rate for the lowest server and the improved uniformity of the airflow rate distribution in the cabinet, it is recommended to use a deflector with a width of 100 mm at an angle of 45° with the perforated floor for the single-side air supply to the cabinet.

Laboratory simulation research
In order to further research the improvement effect of deflectors on the airflow organization under different scenarios, this research established an ANSYS dynamic simulation model and carried out simulations of various working conditions.Based on the experimentally verified ANSYS airflow organization model, the original experimental plan for the single-side air supply to a cabinet was expanded to simulate the working condition of the double-side cabinets, and to investigate the effect of deflectors for the optimization of the deflector layout.

Establishment and verification of dynamic simulation model of airflow organization
The laboratory model built using the ANSYS software is shown in Figure 6.Based on the state of fluid flowing through key equipment and the fluid dynamics theory, corresponding boundary conditions can be set in ANSYS.The specific boundary conditions selected for the model established in this research are shown in Table 3.The software was used to obtain the intake airflow rate for each server under four different server layouts and compare it with the experimental data in Table 1.  Figure 7 shows the relative errors between the simulation results and the experimental results.It can be seen that the errors between the simulation and the experimental results were mostly less than 5%.It only reached −7.12% and 10.59% under two working conditions.One reason for the large error was instrumental error.The accuracy of the impeller anemometer was 0.1.Therefore, there would be a certain instrumental error in the measurement of the wind speed.Another reason was the operational error caused by human factors, such as the slight error in setting the angle between the impeller anemometer and the direction of the cooling airflow during the test.However, it can be seen that the relative errors between the simulation results and the experimental results were still within the allowable range of engineering errors, which was <15% [35].Therefore, the simulation model established using the ANSYS method was accurate.In the following subsections, the established ANSYS model was used to carry out simulations of various working conditions to analyze the impact of the deflectors on cabinet airflow organization.

Simulation model of single cabinet full of servers
Based on the verified simulation model, this study expanded the laboratory cabinet model to a cabinet full of servers and simulated the airflow rate distribution in a single cabinet under different deflector layouts.The size and installation method of the deflector are the same as those in Section2.The simulation result is organized as shown in Figure 8.It can be found that when a deflector with a width of 100 mm at an angle of 45° to the perforated floor was used, the airflow distribution in the cabinet was the most uniform, and the simulation result was consistent with the experimental result for the non-full cabinet.At the same deflector angle, the larger the deflector width, the more uneven the cabinet airflow was.In addition, when the overall airflow rate was small, the airflow rate allocated for the upper server was too small, due to airflow attenuation.and can easily cause the generation of local hot spots.However, the simulation did not show an insufficient airflow rate for the bottom server.This may be due to the relatively small air supply speed, which did not cause the bottom jet to directly pass over the bottom server.

Simulation model of dual cabinets full of servers
The single-side air supply to a cabinet is generally used for data centers with small size or that are planned to be expanded later.For large data centers, it is more common to supply air to each side of dual cabinets.Therefore, based on the singlecabinet simulation model, this study constructed a model for supplying air to dual cabinets.The air supply channel was located between the cabinets, as shown in Figure 9.For the system structure of bilateral air supply, nine different deflector layouts were designed.Taking into account cabinet symmetry and the server layout, for each working condition, the angles of the deflectors on both sides of the perforated floor were the same.The specific layouts of the deflectors are shown in Figure 10.Using the ANSYS model, this study simulated the airflow rate distribution in dual cabinets full of servers under different deflector layouts.The result of the simulation is shown in Figure 11.It can be seen that for the air supply to each side of the dual cabinets, the deflector with a width of 100 mm at an angle of 30° with the perforated floor had a better effect on airflow organization, which greatly improved the airflow rate for the lowermost server.In addition, this ensured better airflow uniformity.

Simulation of data center
Through the above experiments and simulations, the optimal deflector layouts under different air supply modes were obtained.In order to verify the conclusion obtained, this study modeled the actual data and used the optimal deflector layouts obtained to optimize airflow organization and verify their effect.cabinet was slightly different in size, with the most commonly used cabinet dimension being 600 mm (width) × 1000 mm (length) × 2200 mm (height).The layout of the servers in each cabinet varied greatly.The data center adopted the airflow organization form of air supply under the floor of the static pressure plenum and air return on the ceiling.The air supply was controlled by 20 CRACs, two of which were redundant units.Eight CRACs were installed on both sides of the data center for opposite blowing.After flowing through the static pressure plenum, the air was supplied to the date center through the perforated floor.The CRAC was the P2100GC2MS1R model of the Emerson brand with a rated cooling capacity of 100 kW.The height of the plenum was 700 mm.A perforated floor with a size of 600 mm × 600mm, and an opening ratio of 45% were used.The cabinet adopted a forward-and-rear air supply mode.The simulation model of the data center is shown in Figure 12.

Verification of data center model
In order to verify the accuracy of the ANSYS model, 24 ambient temperature measurement points were arranged in the data center.They are placed at a height of 1.5 m from the ground.Their specific locations are shown in Figure 13.The temperature values monitored by the 24 measurement points in the data center were used as the standard for accuracy inspection, and the test and simulation data results were compared, as shown in Figure 14.It can be seen that the relative error of temperature at each measuring point was less than 13%.The normalized mean bias error (NMBE) and coefficient of variation of mean square error (CVRMSE) commonly used in the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) Guideline 14-2014 were used to evaluate the model's errors.The calculation method is shown in Equations ( 1) and (2).
where   is the measured data,   is the simulation data, N is the number of data,   ���� is the average value of the measured data and p = 1.
From the calculations, the NMBE of the temperature from the 24 measuring points was −2.74% and the CVRMSE was 5.87%, which were far lower than the corresponding limits in ASHRAE of <5% and <15%, respectively.This shows that the established ANSYS model was acceptable and can be applied to the subsequent optimization research.

Deflector optimization
Most of the servers in the data center were placed in the middle and lower parts of the cabinets, and the cooling air was supplied in between the cabinets.Based on the model optimization results in the previous section, the deflector width was set to 100 mm and installed at an angle of 30° to the perforated floor.The deflectors in the entire data center were arranged to compare the effects before and after optimization.The specific model is shown in Figure 15.After the simulations, the results of the temperature field and cabinet airflow rate before and after deflectors were installed were compared, as shown in Figure 16 and Table 4, respectively.It can be seen that after the deflectors were installed, the airflow rate entering the cabinets increased by 16.98%, and more cooling airflow participated in directly cooling the server.The temperature field of the data center is shown in Figure 17.It can be seen that after the deflectors were installed, although the average temperature of the overall environment of the data center did not change significantly, temperature uniformity slightly improved and the airflow organization of the data center improved to a certain extent.The distribution of the air velocity field in the data center is shown in Figure 18.After the deflectors were installed, the wind speed at the servers' entrance increased significantly and the airflow speed at different positions in the cabinets became more uniform.Installing deflectors reduced the excessive airflow rates or low-speed eddy currents in local areas of the data center.The cooling airflow exchanged heat with the servers more effectively, thus achieving a good cooling effect.In addition, from the air inlet surface velocity distribution in the cabinets in Figure 19, it can be seen that the inlet airflow rate for servers at the top and bottom of the cabinets improved after deflectors were installed, which effectively alleviated the excessive horizontal air velocity for servers at the bottom of the cabinets due to the excessive air supply jet speed and low horizontal split speed for servers under the cabinet.After the deflectors were installed, the airflow involved in cooling the servers increased.Therefore, for the same outlet air temperature for the servers, the cooling air temperature from the CRACs increased, thereby saving energy consumption.Based on the difference in the simulation results of airflow rates before and after deflectors were installed, assuming that the air outlet temperature for the servers was 45 ℃ [12], it can be seen that for the same outlet air temperature, the servers' inlet air temperature after the deflectors were installed was 0.46 ℃ higher than the air temperature without the deflectors installed, as from Equation (3) below: where Q is server power in kW, 2 is the server's return air temperature in ℃, 1 is the server's air supply temperature in ℃, c is the specific heat capacity of air in kJ/kg.K, and  is air mass flow rate in kg/s.
Based on relevant research [36], for every 1 ℃ increase in the air supply's temperature, the energy consumption of CRACs in data centers can be saved by 4.3%, and the installation of the deflectors in this study improved the energy efficiency of the air conditioning system by 1.98%.

Conclusion
Aiming at the problem of high PUE in data centers due to poor airflow organization, this study proposed a simple and economical solution based on deflectors.Combining field experiments and simulations, the status of airflow distribution in server cabinets under various working conditions was analyzed.For different scenarios of single-side air supply and double-side air supply, the size and installation angle of the deflectors were optimized.Based on the modeled cases, the effects of the deflectors on improving airflow organization and the energy efficiency of the data center were explored.The specific conclusion of this study is as follows: 1) The optimal deflector layout had a positive effect on improving the airflow organization in the cabinet.

2)
For the single-side air supply to a cabinet, it is recommended to use a layout with a deflector width of 100 mm at an angle of 45° with the perforated floor.
Other than increasing the airflow rate of the lowermost server, this also improved the uniformity of the airflow rate distribution in the cabinet.

3)
For the double-side air supply to cabinets, it is recommended to use a layout with a deflector width of 100 mm at an angle of 30° with the perforated floor, which enhanced the uniformity of the air low distribution in the cabinets.

4)
Based on the simulation of an actual data center, optimizing the arrangement of deflectors effectively improved the uniformity of the airflow organization's velocity field and temperature field in the data center and improved the energy efficiency of the air conditioning system by 1.98%.

Figure 2 .
Figure 2. Floor plan for airflow organization experiment.

Figure 4 .
Figure 4. Different server layouts in the cabinet.

Figure 7 .
Figure 7. Error comparison between simulation and experimental results.

Figure 8 .
Figure 8. Simulation result of airflow rate distribution in a single cabinet full of servers under different deflector layouts.

Figure 11 .
Figure 11.Simulation result of airflow rate distribution of dual cabinets full of servers under different deflector layouts.

Figure 12 .
Figure 12.Simulation model of data center.

Figure 13 .
Figure 13.Layout of ambient temperature measurement points in data center.

Figure 14 .
Figure 14.Comparison of results for model verification.

Figure 15 .
Figure 15.Models of data center with and without deflectors.

Figure 16 .Table 4 .Figure 17 .
Figure 16.Impact on temperature in data center before and after installing deflectors.

Figure 18 .
Figure 18.Velocity fields before and after installation of deflectors in data center.

Figure 19 .
Figure 19.Air inlet surface velocity field in cabinets.

Table 1 .
Server airflow rates under different server layouts.

Table 2 .
Experimental result of server airflow rate distribution under different deflector layouts.

Table 3 .
Laboratory simulation model's boundary condition settings.