April 16, 2021       Video

Distribution grids are currently challenged by the rampant integration of distributed energy resources (DER). Scheduling DERs via an optimal power flow problem (OPF) in real time and at scale under uncertainty is a formidable task. Prompted by the success of deep neural networks (DNNs) in other fields, this talk presents two learning-based schemes for near-optimal DER operation. The first solution engages a DNN to predict the solutions of an OPF given the anticipated demands and renewable generation. Different from the generic learning setup, the training dataset here (namely past OPF solutions) features rich yet largely unexploited structure. To leverage prior information, we train a DNN to match not only the OPF solutions, but also their partial derivatives with respect to the input parameters of the OPF. Sensitivity analyses for non-convex and convexified renditions of the OPF show how such derivatives can be readily computed from the OPF solutions. The proposed sensitivity-informed DNN features sample efficiency improvements at a modest computational increase. Nonetheless, this two-stage OPF-then-learn approach may not be suitable for DER operation when the OPF input parameters are changing rapidly. To deal with such setups, we put forth an alternative OPF-and-learn scheme. Here the DNN is not trained to mimic the OPF minimizers but is rather organically integrated into a stochastic OPF formulation. A key advantage of this second DNN is that it can be driven by partial OPF inputs or proxies being the measurements the utility can collect from the grid in real time.

April 30, 2021       Video

Optimization of electric power grids is a challenging, large-scale, non-convex problem. In order to optimize assets across these networks on fast operational timescales, the problem is typically simplified using linear models or other heuristics - resulting in increased cost of operation and potentially decreased reliability. Much work has been performed on improving these models through convexifications and other approximations, but here we take an alternate approach. We use the abundance of data from grid operators or generated offline to train machine learning models that can calculate optimal grid setpoints without even solving an optimization problem. By using an interesting application of a neural network and post-processing procedure, feasible, near-optimal solutions to the AC Optimal Power Flow problem can be obtained in milliseconds.

May 14, 2021       Video

Integration of variable renewable and distributed energy resources in the grid makes demand and supply conditions uncertain. In this talk, we explore customized algorithm to tackle risk-sensitive electricity market clearing problems, where power delivery risk is modeled via the conditional value at risk (CVaR) measure. The market clearing formulations are such that they allow a system operator to effectively explore the cost-reliability tradeoff. We discuss algorithmic architectures and their convergence properties to solve these risk-sensitive optimization problems at scale. The first half of this talk will focus on a CVaR-sensitive optimization problem that can be cast as a large linear program. For this problem, we propose and analyze an algorithm that shares parallels and differences with Benders decomposition. The second half of this talk will focus on another CVaR-sensitive problem for which we propose and analyze sample complexity of a stochastic primal-dual algorithm.

May 28, 2021       Video

For seven years we have studied stranded power (curtailment and negative pricing), a large (TWh) and rapidly growing phenomena (>30% CAGR), characterizing its temporal and spatial structure in power grids with growing renewable generation. Our goal is to exploit this power for productive use, both reducing the associated carbon-emissions for that use (e.g. cloud computing), and increasing grid renewable absorption. Our work shows that dispatchable computing loads that exploit stranded power can improve renewable absorption significantly. High-performance and cloud computing workloads can be supported with high quality-of-service and superior economics (total-cost-of-ownership) can be achieved despite intermittent power. Such loads can now exceed 1 gigawatt per site. In 2021, variations of these ideas are achieving acceptance and early deployment.

The rise of adaptive loads optimized for selfish purpose raises significant questions as to acceptable and constructive behavior for both grid stability and operations (e.g. social welfare and decarbonization). We have explored the impact of adaptive cloud compute load on power prices and carbon emissions, varying coupling model (none, selfish optimization, and grid optimization). The results show that computing adaptive loads can disturb grid dispatch and both increase carbon emissions and inflict financial harm on other customers. At today’s cloud-computing penetration levels, such effects would be significant and increasing.

More generally, these studies expose fundamental challenges with large-scale adaptive load to make it a pillar of increasing renewable absorption. If possible, we will discuss major economic and operational obstacles to creating adaptive load (flexibility) and effective coupling interfaces (e.g. service and markets). These challenges underpin an emerging “struggle for control” between competing spheres of optimization; directly, the tensions between societal aspirations for grid decarbonization and acceptable power cost.

October 1, 2021       Video

The talk focuses on the synthesis of data-based online algorithms to optimize the behavior of networked systems – with a specific focus on power grids – operating in uncertain, dynamically changing environments, and with human-in-the-loop components. Formalizing the optimization objectives through a time-varying optimization problem, the talk considers the design of online algorithms with the following features: i) concurrent learning modules are utilized to learn the cost function on-the-fly (using infrequent functional evaluations, and leveraging Gaussian Processes), and ii) algorithms are implemented in closed-loop with the plant to bypass the need for an accurate knowledge of the algebraic map of the system. Leveraging contraction arguments, the performance of the online algorithm is analyzed in terms of tracking of an optimal solution trajectory implicitly defined by the time-varying optimization problem; in particular, results in terms of convergence in expectation and in high-probability are presented, with the latter derived by adopting a sub-Weibull assumption on measurement errors and the error in the reconstruction of the cost. Additional results are offered in terms of cumulative fixed-point residual. The online algorithm is applied to solve real-time demand response problems in power grids, where one would like to strike a balance between maximizing the profit for providing grid services and minimizing the (unknown) discomfort function of the end-users. Additional examples include real-time optimal power flow applications.

October 15, 2021       Video

A rising challenge in control of large-scale control systems such as the energy and the transportation networks is to address autonomous decision making of interacting agents. In this setting, a Nash equilibrium is a stable solution outcome in the sense that no agent finds it profitable to unilaterally deviate from her decision. Due to geographic distance, privacy concerns or simply the scale of these systems, each agent can only base her decision on local measurements. Hence, a fundamental question is: do agents learn to play a Nash equilibrium strategy based only on local information? I will motivate this question with problems arising in energy markets and discuss my work on addressing it for two classes of games, those with continuous and finite action spaces. I will present our algorithms for learning Nash equilibria in these games, discuss their convergence and show their applications in in energy markets. This is a joint work with my colleague T. Tatarenko, my former and current doctoral students O. Karaca and P.G. Sessa.

October 29, 2021       Video

As the complexity and uncertainty of modern energy systems grow, we are seeing a growing need for good quality data for improving system and market operations. Traditionally, data in energy operations has largely been regarded as a free and highly accessible commodity, which is in significant contrast to the rising concerns about data privacy. Therefore, recent academic research has been investigating various forms of data markets to incentivize data exchanges.

In this talk, I will be presenting our recent work on a specific type of data market that is based on linear regression frameworks. The chosen use case is wind power forecasting, in which wind agents have the potential to improve their forecasts through gaining access to measurement data from neighboring wind agents. Based on how data payments are determined, I will introduce two market clearing mechanisms: 1) a cooperative game theoretic profit allocation, and 2) a lasso (least absolute shrinkage and selection operator) payment scheme. A comparison of the two payment mechanisms will be presented to show the difference in their preserved market properties and in their levels of incentive for the data sellers and data buyers.

Joint work with Pierre Pinson and Jalal Kazempour.

November 1, 2021       Video

This talk discusses an electricity market clearing formulation that seeks to remunerate spatio-temporal, load-shifting flexibility provided by data centers (DaCes). Load-shifting flexibility is a key asset for power grid operators as they aim to integrate larger amounts of intermittent renewable power and to decarbonize the grid. Central to our study is the concept of virtual links, which provide non- physical pathways that can be used by DaCes to shift power loads (by shifting computing loads) across space and time. We use virtual links to show that the clearing formulation treats DaCes as prosumers that simultaneously request load and provide a load-shifting flexibility service. Our analysis also reveals that DaCes are remunerated for the provision of load-shifting flexibility based on nodal price differences (across space and time). We also show that DaCe flexibility helps relieve space-time price volatility and show that the clearing formulation satisfies fundamental economic properties that are expected from coordinated markets (e.g., provides a competitive equilibrium and achieves revenue adequacy and cost recovery). The concepts presented are applicable to other key market players that can offer space-time shifting flexibility such as distributed manufacturing facilities and storage systems. Case studies are presented to demonstrate these properties.

November 12, 2021       Video

The growing climate impact of increased Greenhouse Gas Emissions and CO₂ levels in Earth’s atmosphere highlights the value and importance of technologies that reduce such impact. Electricity generation is a significant contributor to global CO₂ emissions, and the datacenter industry is expected to reach anywhere from 3 to 13% of global electricity demand by 2030. Datacenters have the potential to facilitate grid decarbonization in a manner different from isolated power loads, and Google has set on a mission to increase and scale carbon awareness via new technology solutions. We present Google’s globally-distributed Carbon-Intelligent Computing System, which harnesses datacenter load flexibility, not only to reduce emissions, but also to contribute to more robust, resilient, and cost-efficient decarbonization of the grid through energy system integration. We share some conclusions derived from operational data and discuss the key technical challenges that drive carbon-aware designs and their future enhancements. By calling out opportunities for “greening” the cloud industry, we hope to inspire academia and industry to pursue diverse and system-specific carbon-aware solutions.

November 19, 2021       Video

Recent years have witnessed rapid transformations of contemporary advances in machine learning (ML) and data science to aid the transition of energy systems into a truly sustainable, resilient, and distributed infrastructure. A blind application of the latest-and-greatest ML algorithms to solve stylized grid operation problems, however, may fail to recognize the underlying physics models or safety constraint requirements. This talk will introduce two examples of bridging physics-aware and risk-aware ML advances into efficient and reliable grid operations. First, we develop a topology-aware approach using graph neural networks (GNNs) to predict the price and line congestion as the outputs of real-time AC optimal power flow problem. Building upon the underlying relation between prices and topology, this proposed solution significantly reduces the model complexity of existing end-to-end ML methods while efficiently adapting to varying grid topology. Second, we put forth a risk-aware ML method to ensure the safety guarantees of data-driven, scalable reactive power dispatch policies in distribution grids. An adaptive sampling method is leveraged to improve the gradient computation of conditional variance-at-risk (Cvar) and attain guaranteed voltage violation performance. Our proposed algorithms have demonstrated the advantages of incorporating power system modeling into the design of learning-enabled power system operations.

December 3, 2021       Video

We formulate a generic network game as a generalized Nash equilibrium problem. Relying on normalized Nash equilibrium as solution concept, we provide a parametrized proximal algorithm to span many equilibrium points. Complexifying the setting, we consider an information structure in which the agents in the network can withhold some local information from sensitive data, resulting in private coupling constraints. The convergence of the algorithm and deviations in the players’ strategies at equilibrium are formally analyzed. In addition, duality theory extension enables to use the algorithm to coordinate the agents through a fully distributed pricing mechanism, on one specific equilibrium with desirable properties at the system level (economic efficiency, fairness, etc.). To that purpose, the game is recast as a hierarchical decomposition problem, and a procedure is detailed to compute the equilibrium that minimizes a secondary cost function capturing system level properties. An application is presented to a peer-to-peer energy trading problem, under complete and incomplete information. Under incomplete information, assuming that the agents anticipate the form of the market equilibrium, an aggregative noncooperative stochastic game is formulated to model the communication mechanism between the agents on top of the peer-to-peer energy trading game. Analytical and numerical analysis are provided to capture the impact of privacy constraints on the generalized Nash equilibria.

December 10, 2021       Video

The wide-scale electrification of the transportation sector won’t be possible without careful planning and coordination with the power grid and the companies that manage its operation. If left unmanaged, the uncoordinated charging of EVs at increased levels of penetration will amplify existing peak loads, potentially outstripping the grid’s current capacity to meet demand. In this talk, I’ll present findings from the OptimizEV Project—a real-world pilot study in Upstate New York exploring a novel approach to coordinated residential EV charging. As one of its primary objectives, the OptimizEV platform seeks to harness the latent flexibility in EV charging by offering EV owners monetary incentives to delay the time required to charge their EVs. Each time an EV owner initiates a charging session, they can use their smartphone to specify how long they intend to leave their vehicle plugged in by selecting from a “menu of deadlines” that offers lower electricity prices the longer they’re willing to delay the time required to charge their car. Given a collection of active charging requests, a smart charging system dynamically optimizes the power being drawn by every EV in real time to minimize their collective strain on the grid, while ensuring that every customer’s car is fully charged by its deadline. Customers get their energy when they need it by and the smart charging system can optimally coordinate the delivery of that energy to avoid spikes in demand. I’ll describe important lessons learned from the OptimizEV Project related to customer behavior, EV charging characteristics, and EV transportation patterns.

This is joint work with Polina Alexeenko and New York State Electric and Gas (NYSEG).

December 17, 2021       Video

We consider the combination of a network design and graph partitioning model in a multilevel framework for determining the optimal network expansion and the optimal zonal configuration of zonal pricing electricity markets. The two classical discrete optimization problems of network design and graph partitioning together with nonlinearities due to economic modeling yield extremely challenging mixed-integer nonlinear multilevel models for which we develop two problem-tailored solution techniques. The first approach relies on an equivalent bilevel formulation and a standard KKT transformation thereof including novel primal-dual bound tightening techniques, whereas the second is a tailored Benders-like decomposition approach. We prove for both methods that they yield global optimal solutions. Afterward, we compare the approaches in a numerical study and show that the tailored Benders approach clearly outperforms the standard KKT transformation. Finally, we present some case study that illustrates the economic effects that are captured in our model and discuss some recent results for the Germany electricity sector.

March 11, 2022       Video

The water and power networks are heavily interdependent. The water network requires power for extraction, treatment, and distribution processes. The power network requires water for thermoelectric generation, i.e., cooling and emissions scrubbing, as well as fuel extraction and processing. By and large, these networks are operated independently, but there is a growing body of research beginning to demonstrate the benefits of operating these networks jointly. In this talk, I will describe our research to develop operational planning and real-time control strategies for coupled water-power distribution networks under uncertainty. Specifically, we formulate chance constrained and robust optimization problems that simultaneously determine water pumping schedules along with the parameters of real-time control policies that can be used to respond to water and power demand forecast errors. We show how the water network can help the power network manage voltage deviations resulting from intermittent solar PV production and provide frequency regulation to the bulk power network, while satisfying the water demand and the physical constraints of both networks. This is joint work with Anna Stuhlmacher.

March 18, 2022       Video

Integration of volatile and uncertain renewable energy resources and synergistic energy storage and demand response technologies motivates the pursuit of stochastic electricity market designs to accommodate these resources and technologies efficiently and reliably. However, until recently the primary means of achieving these goals rested on the use of traditional optimization techniques and, in particular, on stochastic optimization and its proxies. However, recent developments make it possible to “re-invent” stochastic electricity markets borrowing results from data science, machine learning and financial engineering. This presentation will first describe how stressing of load and renewable time series can be achieved using principal component analysis and generative adversarial networks, and then how these stressed time series could be integrated into a stochastic electricity market design. Building on this result, we will describe a machine learning approach to stochastic-market clearing that affords both market- and operation-feasible solutions and reduces computational requirements.

April 1, 2022       Video

The extensive monitoring of the power grid, together with the proliferation of technologies for intelligent metering, control and computations, is allowing us to record, and possibly exploit, the context under which power grids are operated. In this talk, we will show examples of how ideas from the fields of machine learning, optimization and decision-making can be put to work together to develop context-driven methods that improve the solution to challenging problems in power system operations. We will discuss a variety of context-driven methods of different complexity, some of which are surprising for their apparent simplicity but remarkable benefits in terms of computational costs and/or social welfare, proving the huge potential value in the contextual data still to be unleashed.

April 15, 2022       Video

Making use of modern black-box AI tools is potentially transformational for online optimization and control. However, such machine-learned algorithms typically do not have formal guarantees on their worst-case performance, stability, or safety. So, while their performance may improve upon traditional approaches in “typical” cases, they may perform arbitrarily worse in scenarios where the training examples are not representative due to, e.g., distribution shift or unrepresentative training data. This represents a significant drawback when considering the use of AI tools for energy systems and autonomous cities, which are safety-critical. A challenging open question is thus: Is it possible to provide guarantees that allow black-box AI tools to be used in safety-critical applications? In this talk, I will introduce recent work that aims to develop algorithms that make use of black-box AI tools to provide good performance in the typical case while integrating the “untrusted advice” from these algorithms into traditional algorithms to ensure formal worst-case guarantees. Specifically, we will discuss the use of black-box untrusted advice in the context of online convex body chasing, online non-convex optimization, and linear quadratic control, identifying both novel algorithms and fundamental limits in each case.

April 29, 2022       Video

A large and rising share of carbon dioxide emissions is attributable to the electricity sector. The electrification of heat and transportation will have broad consequences for our energy systems and for the electricity sector. Improperly managed, massive electrification could destabilize power grids, cause blackouts and dramatically increase the cost of electricity. Carefully integrated, however, heat and transportation could represent a sorely needed flexibility resource as we turn towards generation sources that rely heavily on the wind and sun.

The research presented here focuses on building advanced computational tools for sustainable energy systems. These tools are developed and deployed in research-grade, end-to-end, prototype software systems for real-world energy and carbon management applications. Two currently active areas of interest that will be discussed in this talk are (1) building tools to track emissions in the US electricity system and (2) experimenting with building HVAC systems on the Stanford campus in the context of the COOLER Research Program. COOLER’s goal is to make large, modern buildings more energy-efficient, flexible, low carbon, and resilient using data, optimization, and control.

May 13, 2022       Video

With the push towards more sustainable electric power systems, renewable generation resources, which are usually connected via power electronics, increasingly replace power plants that are based on synchronous machines. This, on one hand, changes the interactions of the generation resources with the grid and has impacts on the system dynamics, but the power electronics also allow for, or even require, new ways of power system planning and operation. In this talk, we will provide examples of where optimization can be leveraged at different time scales to integrate increased levels of renewable generation taking into account their capabilities but also addressing the newly arising challenges.