Autonomous Demand Response Using Stochastic Differential Gamesby Najmeh Forouzandehmehr, Mohammad Esmalifalak, Hamed Mohsenian-Rad, Zhu Han

IEEE Trans. Smart Grid


Computer Science (all)


This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.


Autonomous Demand Response Using

Stochastic Differential Games

Najmeh Forouzandehmehr, Mohammad Esmalifalak, Student Member, IEEE,

Hamed Mohsenian-Rad, Senior Member, IEEE, and Zhu Han, Fellow, IEEE

Abstract—Demand response (DR) programs are implemented to encourage consumers to reduce their electricity demand when needed, e.g., at peak-load hours, by adjusting their controllable load. In this paper, our focus is on controllable load types that are associated with dynamic systems and can be modeled using differential equations. Examples of such load types include heating, ventilation, and air conditioning; water heating; and refrigeration. In this regard, we propose a new DR model based on a two-level differential game framework. At the beginning of each

DR interval, the price is decided by the upper level (aggregator, utility, or market) given the total demand of users in the lower level. At the lower level, for each player (residential or commercial buildings that are equipped with automated load control systems and local renewable generators), given the price from the upper level, the electricity usage of air conditioning unit, and the battery storage charging/discharging schedules, are controlled in order to minimize the user’s total electricity cost. The optimal user strategies are derived using the stochastic Hamilton–Jacobi–

Bellman equations. We also show that the proposed game can converge to a feedback Nash equilibrium. Based on the effect of real-time pricing on users’ daily demand profile, the simulation results demonstrate the properties of the proposed game and show how we can optimize consumers’ electricity cost in the presence of time-varying prices.

Index Terms—Autonomous demand response, real-time pricing, smart building, stochastic differential game.

NOMENCLATURE u1 Power draw from battery for building usage. u2 Air conditioner usage of electricity. x1 Energy stored in the battery array. x2 Indoor temperature of building. xdi Desired temperature. w Renewable output prediction.

Manuscript received December 23, 2013; revised May 30, 2014 and August 4, 2014; accepted September 6, 2014. This work was supported in part by the U.S. National Science Foundation (NSF) under Grant CNS-0910461, Grant CNS-0905556, Grant CNS-0953377,

Grant ECCS-1028782, Grant CNS-1117560, Grant CNS-1265268, and Grant

ECCS 1253516; in part by the Qatar National Research Fund; and in part by the Electric Power Analytics Consortium at the University of Houston.

Paper no. TSG-00933-2013.

N. Forouzandehmehr was with the Electrical and Computer Engineering

Department, University of Houston, Houston, TX 77004 USA. She is now with Samsung Research America, San Jose, CA 95134 USA.

M. Esmalifalak and Z. Han are with the Electrical and Computer

Engineering Department, University of Houston, Houston, TX 77004 USA (e-mail:

H. Mohsenian-Rad is with the Department of Electrical Engineering,

University of California at Riverside, Riverside, CA 92521 USA.

Color versions of one or more of the figures in this paper are available online at

Digital Object Identifier 10.1109/TSG.2014.2357346 e Prediction error. l Uncontrollable load of a building. β Battery leakage rate.  Factor of inertia. γ Coefficient of performance. tOD Outside temperature. p Market spot price. d Total power consumption of the building. g Total power generation of the building. μ Building utility function.

L Building expected total utility function. h Terminal condition. v Building value function.

N Number of buildings.


DEMAND response (DR) programs are implementedby utility companies to control the energy consumption at the customer side of the meter. Two popular DR approaches are direct load control (DLC) and smart pricing.

In DLC [1]–[4], an aggregator can remotely control the operations and energy consumption of certain consumer appliances.

In contrast, in smart pricing, users are encouraged to individually and voluntarily manage their load, e.g., by reducing their consumption at peak price hours. This can be done using automated energy consumption scheduling (ECS) devices [5].

For each user, the ECS finds the best load schedule to minimize the user’s electricity cost while fulfilling the user’s energy needs. This can lead to autonomous DR programs that burden a minimal control overhead on utilities.

A common analytical tool to study autonomous DR systems is game theory [6], that provides a framework to study rational interactions and outcome in a distributed manner.

In [7], a stochastic game is developed to model an hourly energy auction in which generators and consumers participate as adaptive agents. Chen et al. [8] proposed a game theoretic DR scheme to develop a distributed load prediction system that involves user participation. Chen et al. [9] employed the Cournot game model to analyze the market effect of a DR aggregator on both shifting and reducing deferrable loads. Dave et al. [10] developed a hybrid dayahead and real-time consumption scheduling for a number of houses that participate in a demand side program based on game theory. The interaction between the service provider and the users is modeled as a Stackelberg game in [11] to 1949-3053 c© 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.

See for more information.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2 IEEE TRANSACTIONS ON SMART GRID derive the optimal real-time electricity price and each user’s optimal power consumption. In [12], the real-time price-based

DR management is evaluated for residential appliances via stochastic optimization and robust optimization approaches.