High-Performance Computing
Transforming power grid operations via high-performance computing
Simulation is a fundamental element in power grid operations. However, due to the complexity and large data volume involved, power grid simulation is very slow. For example, state estimation and contingency analysis can not keep up with Supervisory Control and Data Acquisition, or SCADA, measurement cycles, and dynamic simulation is largely limited to an off-line application. What this means is that grid reliabiliy is limited, as is our ability to fully utilize the grid’s capacity. Real-time grid simulation aims to improve the reliability, efficiency and functionality of grid operations by applying high-performance computing techniques and advanced computing hardware to power grid equations.
Real-time simulations
Figure 1. Integrated real-time operations platform for state estimation and grid simulation. Applying high-performance computing to power grid simulation enables real-time state estimation, faster-than-real-time dynamic simulation and dynamic contingency analysis. Click for a larger image.
At Pacific Northwest National Laboratory, we believe that traditional power grid algorithms can be reformulated and applied to high-performance computing platforms. Applying high-performance computing techniques and advanced computing hardware involves two major aspects: reformulation of power grid equations and parallelization of computational processes.
Power grid operations include many important functions. "State estimation" is central for driving other key functions, e.g., contingency analysis, optimal power flow and automatic generation control. State estimation typically receives telemetered data from the SCADA system every four seconds and extrapolates a full set of grid conditions for operators based on the grid's current configuration and a theoretically based engineering power flow solution.
Figure 2. The iterative conjugate gradient algorithm outperforms state-of-the-art direct algorithm, SuperLU, on 128-processor SGI Altix. Conjugate gradient algorithm performs well on scalability and absolute performance. Execution time increases with SuperLU running on more than two processors. The shortest time to solution using conjugate gradient (16 processors) is 4.75 times better than the best time using SuperLU (2 processors). Click for a larger image.
With today's computers and algorithms, the process of a medium-sized state estimation problem can be updated only about every two minutes — much slower than the SCADA measurement cycle. Contingency analysis assesses the effect of various combinations of power system component failures based on state estimates and can be updated only about every five minutes.
This is not fast enough to predict system status because a power grid could become unstable and collapse within seconds. In addition, both of these analyses are based on static power flow models, and dynamic information is not included. Adding another layer of complexity to power grid operations are markets, which also interact with the physical power grid.
Figure 3. Fully parallelized state estimation package: 10x speedup of state estimation process is achieved when running on 16 Cray MTA-2 parallel processors versus single MTA-2 processor. Click for a larger image.
We aim to speed up these complex algorithms, bringing them into the reach of real-time grid operations by reformulating and converting traditional power grid equations to high-performance computing platforms such as PC clusters, reconfigurable hardware, scalable multicore shared memory computers, and multithreaded architectures. Parallel computing is an essential ingredient to taking advantage of high-performance multi-processor computing platforms. The improved performance is expected to have a huge impact on how power grids are operated and managed, ultimately leading to better reliability and asset utilization in the power industry.
The new computational capabilities will be developed, tested and demonstrated on the comprehensive grid operations platform in the Electricity Infrastructure Operations Center (EIOC), a new Pacific Northwest National Laboratory facility for developing and testing technologies that enhance energy infrastructure and operations. Pacific Northwest National Laboratory has a wealth of computing resources representing both leading and emerging classes of high-end computer architectures.
The Advanced Computing Technology Laboratory at Pacific Northwest National Laboratory hosts a range of architectures such as the 128-processor SGI Altix with scalable non-uniform memory access shared memory. Shared memory is useful for efficient implementation of sparse matrix and irregular computations like those arising in power systems. Another, smaller configuration of the SGI Altix with directly connected field-programmable gate arrays is available to explore hybrid and reconfigurable computing options. Other high performance computing research is being conducted at Pacific Northwest National Laboratory’s Environmental Molecular Sciences Lab.
The Cray MTA parallel multithreaded shared-memory architecture, which, like the SGI Altix, provides shared memory hardware in the uniform memory access configuration, is also available. Future Cray architectures, like the Cray XMT (also known as "Eldorado") scalable multithreaded system with advanced compiler technology, are expected to offer excellent support for the latency hiding and fine-grain parallelism lacking in traditional architectures.
Pacific Northwest National Laboratory's primary areas of research in high performance computing for power systems include:
- Parallelized State Estimation — The objective is to reduce solution time of state estimation from minutes to seconds, comparable with SCADA measurement cycles (typically ~4 seconds). Parallelized state estimation would enable the timely update of power grid status as soon as new measurement comes in. This enables operators to see what’s currently happening on the grid with minimal delay.
- Parallel Contingency Analysis — Contingency cases can be allocated to multiple processors, and computation can be performed naturally in parallel. Challenges lie in minimizing communication overhead added from managing multiple processors and balancing computational load among the processors. This enables a myriad of "what-if" questions to be answered in an instant.
- Dynamic State Estimation — Dynamic state estimation reformulates state estimation problems with dynamics included. It estimates dynamic states, e.g., generator speeds and rotor angles, instead of static states of bus voltages and phase angles.
- Look-Ahead Dynamic Simulation — Dynamic equations are parallelized to enable dynamic simulation on multiple processors. The goal is to significantly increase the speed of dynamic simulation so it can be performed faster than real-time and one can look into the future to predict the dynamic status of the power grid. Together with Dynamic State Estimation, this will allow operators to view the trajectory of the grid- even identifying severe contingencies that have already begun but have not yet matured (such as various voltage instability issues).
Publications and Presentations
Huang Z, RT Guttromson, J Nieplocha, and RG Pratt. 2007. "Transforming Power Grid Operations." Scientific Computing 24(5):22-27.
Zhou N, Z Huang, J Nieplocha, and TB Nguyen. 2006. "Wide-Area Situational Awareness of Power Grids with Limited Phasor Measurements." In Third International Conference on Critical Infrastructures, Alexandria, VA, September 24 to 27, 2006. International Institute for Critical Infrstructures, Blacksburg, VA.
