In other words, the electrical energy grid changes significantly from a demand-side driven to a supply-side driven system due to an increasing amount of distributed energy resources. In this sense a promising approach for the integration of distributed energy resources within the low and medium voltage network can be an efficient and robust demand side response management.
Consequently, the demand side management (DSM) optimization becomes a promising solution for matching demand and supply including volatile generation and flexibility on the demand side. Concepts such as Demand Response (DR) are the main core of many research activities. The Demand Side Management (DSM) is usually considered as a process of shifting of energy consumption from peak hours to off-peak times. DSM doesn’t always reduce total energy consumption, but it helps to meet energy demand and supply. For example, it balances variable generation from renewables (such as solar and wind) when energy demand differs from renewable generation .
One of limitation of electricity power is that generally, electrical energy cannot be stored because of a large number of economic or physical feasibility limits. Thus, it must be produced in the quantity needed. It is exactly the main objective of DSM – to equilibrate production and consumption of energy.
DSM originated after oil crisis in the 1970s. Then, energy demand was met relying on forecasts, which were often made with a ruler and double-log paper. In other words, demand side was largely disconnected from the market. Consumers were mostly simple users of energy sources. They received electricity from energy grid and paid for it. Gradually, situation is changing. After petrol shock in 1973, Demand Side Optimization has become more important. Most countries tried to develop programs to reduce dependence on oil and to promote energy efficiency and alternative energy sources. Nowadays, energy consumers are more proactive. They want to optimize electricity consumption so as to reduce their expenses.
The process of DSM activities usually follows the integrated approach. DSM sends signals to end-use systems to shed load depending on system conditions. This allows for very precise tuning of demand to ensure that it matches supply at every period, reduces capital expenditures for the utility. Critical system conditions could be peak times, or in areas with levels of variable renewable energy, during times when demand must be adjusted upward to avoid over generation or downward to help with high needs. Consequently, the analysis and optimization on the demand side focuses on the involvement of the customer and fits to the vision of a customer centric energy grid.
According to literature , DSM can be divided into 3 categories:
Energy efficiency means usage of less power due to more efficient load-intensive appliances such as water heaters, refrigerators, or washing machines.
Strategic Load Growth refers to a general increase in energy consumption. Load growth may involve increased market share of loads which can be served by fuel switching from fuels to electricity such as heat pumps, induction cooker and microwave oven.
Demand Response (DR) identifies the short-term relationship between price and quantity when the actions and interactions of substitutes and complements are considered. Currently, the term DR is used in a broad sense, in relation to electricity end-use, and is attributed to a wide range of control signals such as prices, resources availability and network security .
Fig. 1 sums up DSM categories (based upon ).
Figure 1: Categorization of Demand Side Management (based upon )
Demand Side Management and Demand Response
We define DR as part of DSM similar to  and , as the “voluntary changes by end-consumers or producers or at storages of their usual electricity/gas flow patterns
in response to market signals such as time-variable prices, incentive payments” or beforehand given agreements between customers and third parties. Such pattern changes are possible due to flexibility on the demand side. Such flexibility might be provided for example through electrical or thermal storages where demand is decoupled from generation, but also from other flexible loads, such as EVs.
In electricity markets, traditional DSM programs are slowly getting replaced with DR programs. A good example of demand response implication and reducing electricity peak demand is the introduction of “Time of Use Tariffs” in France. Its aim was to apply a fixed rate with different time units depending on hours and seasons. “Time of Use Tariffs” in France included 3 parts:
- «Green tariffs» (1956) for large firms or buildings (La Defense): Many prices options according to season/hour and localisation/use.
- «Off-peak hours» (1965) tariffs for residential market and business– special tariffs from 10 PM to 6AM week days and on Sundays.
- «Peak day step back» (EJP) (1982) for residential market was introduced to decrease consumption at critical times (22 days of 18 hours between 1st November and 31th March). It established high price during this period and low price the rest of the year. Currently, EJP replaced by TEMPO (6 price’s levels according to hour and season).
This program showed high results: «off-peak hours» tariffs reduce peak consumption by about 20% and customers with «Peak day stepping back» tariffs reduce their consumption by 50% during peak period (4% of total residential consumption) .
Direct Load Control vs. Indirect Load Control
In general DSM and DR concepts can be distinguished between direct and indirect load control. Indirect load control implies an incentive, such as a price signal. Such signal might motivate the consumer to shift its consumption into times of lower prices. Direct load control rather means an agreement between the customer and a third party that allows the party to directly control the loads of the customer based upon the beforehand made agreement .
For field installations the most promising solution which finds well acceptance in research and industry is the automated demand response (OpenADR) protocol which is now a de-facto standard for DR concepts .
Several recent research activities that use mathematical optimization techniques for DR refer both to direct and indirect load control. These research topics are related to the optimization and coordination of the operation supply and demand units throughout a time horizon, e.g. an offline day-ahead scheduling under consideration of flexibility. The flexibility is achieved through temporal shifts over a Horizon T. Such problems are very generally known as the Portfolio Balancing problem.
Demand Side Management in different time horizons (short to long term)
The classic unit commitment problem is mainly short term but can be solved also for medium and long term problems. As shown in fig.
2, similarly as presented in , we can distinguish Demand Side Management according to its time line.
Spinning Reserve in this context refers to primary and secondary and even tertiary control, which is usually done by power plants. However, in DSM, loads can be virtually aggregated and act as negative spinning reserve for frequency control. The time horizon is in between seconds and minutes.
2: Different time horizon in DSM, based upon 
If the DR optimization problem is based upon a static Time-of-use price schedule, the optimization problem is typically shift from short- to medium or long term. Customers reschedule and rearrange their processes and behavior in order to avoid consuming energy during periods of high prices, such as in-between 5pm – 7pm. These periods and the related prices are available for the customer and are typically arranged months before. In other words, a static price schedule is applied, whereby short term DR uses a dynamic (even real-time) price schedule.
Integrated Demand Side Management
Nowadays, DSM technologies become increasingly feasible due to the integration of information and communications technology and the power system, new terms such as Integrated Demand Side Management (IDSM), or smart grid.
Smart grid gives new opportunity of remote control services that allow the network operator to switch off high electricity consumption devices (for example, air conditioners, hot water tanks, heat pumps) for a limited period during peak demand without causing major failure for the consumer.
For example, French company Voltalis offers to residential customers, the ‘Bluepod’ box, a device which switches off electrical heating and space conditioning appliances.
If demand exceeds electricity production, transmission network operator (Réseau de Transport d'Electricité, RTE) contacts Voltalis, which can withdraw demand in real time by modulating electricity consumption in many households via ‘Bluepod’ .
Challenges and Requirements for Demand Side Management and Demand Response in Optimization
The above mentioned general description of the portfolio balancing problem for city districts and neighborhoods incorporate several challenges both for the mathematical method and the overall approach. First, there is usually a high heterogeneity of participants and devices that must be taken into account. Residential buildings, but also industrial consumers might take part of the portfolio balancing. The load and flexibility of such units diversify within their granularity of time, their amplitude and their criticalness. Second, a city district contains in general a high number of participants and devices which lead to a computational intensive problem with an increasing portfolio size. Consequently, a mathematical optimization must be able to handle a large amount of heterogeneous participants. Third, referring to the concept of demand response and in particular to direct load control, it is an important requirement for the method to ensure data privacy. Fourth, the coordination within city districts usually needs to integrate both local (customer) and global (system) level objectives. In respect to this challenge the method requires an approach for both satisfying global and local objectives. Fifth, depending on the kind of installed devices on the demand side the mathematical optimization method might have to be able to take care of on/off devices leading to an Integer related problem formulation.
Research Paper and Solver
Indirect load control on the demand side is for example studied in  and . In particular  is a very recent example for showing the operation scheduling of Plug-in electric vehicles coordinated by an aggregator agent. The MILP is solved within GAMS Build 21.1.2. using the CPLEX 12.5.1 solver .
This research satisfies all of the mentioned requirements. As mentioned in the challenges above a central optimization becomes hard to solve with an increasing portfolio size. Indirect and direct load control for scheduling loads on the demand side by using a distributed algorithm is hence an active field of research. Consequently many research papers, such as [3, 5, 8–13] propose distributed optimization demand response techniques for (residential) energy demand side management. Decomposition methods such as in  or  use dual decomposition (DD) or the alternating direction method of multipliers (ADMM) such as in [3, 12]. For both DD and ADMM in particular challenges and requirements 1) – 4) are taken into account. 
The residential demand side energy management in  for example used the matlab environment in combination with ILOG CPLEX 12.2 to solve the optimization problems. The ADMM problems were solved using CVX, a package for specifying and solving convex programs , . Looking into integer related problem formulations authors in  propose a column generation approach for direct load control which is solved using the object-oriented Python Interface of Gurobi . Research in  performs a decentralized robust ILP optimization for balancing a portfolio within a microgrid. The optimization uses the CPLEX package within Java. Both  and  are other examples for satisfying all mentioned challenges 1) - 5) and the resulting requirements.  uses a MILP formulation for the optimal control of a residential microgrid using the Gurobi solver as well through the object-oriented interface for Java. Further, authors in  perform a distributed optimization via a multi-agent system using the Java agent development framework (JADE) . However, each local agent solves its own local MILP optimization using MOSEK .
 P. Palensky and D. Dietrich, “Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads,” Industrial Informatics, IEEE Transactions on, vol. 7, no. 3, pp. 381–388, 2011.
 Smart Grid Task Force, “Regulatory Recommendations for the Deployment of Flexibility: SGTF-EG3 Report,” https://ec.europa.eu/energy/sites/ener/files/documents/EG3%20Final%20-%20January%202015.pdf, 2015.
 Morten Juelsgaard, “Utilizing Distributed Resources in Smart Grids A Coordination Approachin Smart Grids A Coordination Approach: A Coordination Approach,” Dissertation, Aalborg University, Denmark, 2014.
 OpenADR Alliance. Available: http://www.openadr.org/ (2016, Feb. 19).
 A. Safdarian, M. Fotuhi-Firuzabad, and M. Lehtonen, “A Distributed Algorithm for Managing Residential Demand Response in Smart Grids,” IEEE Trans. Ind. Inf, p. 1, 2014.
 I. Momber, S. Wogrin, and Gomez San Roman, T, “Retail Pricing: A Bilevel Program for PEV Aggregator Decisions Using Indirect Load Control,” Power Systems, IEEE Transactions on, vol. 31, no. 1, pp. 464–473, 2016.
 IBM Corporation, IBM CPLEX Optimizer - United States. Available: http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/ (2016, Feb. 18).
 N. Rahbari-Asr and M.-Y. Chow, “Cooperative Distributed Demand Management for Community Charging of PHEV/PEVs Based on KKT Conditions and Consensus Networks,” IEEE Trans. Ind. Inf, vol. 10, no. 3, pp. 1907–1916, 2014.
 del Real, Alejandro J, A. Arce, and C. Bordons, “An Integrated Framework for Distributed Model Predictive Control of Large-Scale Power Networks,” IEEE Trans. Ind. Inf, vol. 10, no. 1, pp. 197–209, 2014.
 E. Kuznetsova, C. Ruiz, Y.-F. Li, and E. Zio, “Analysis of robust optimization for decentralized microgrid energy management under uncertainty,” International Journal of Electrical Power & Energy Systems, vol. 64, pp. 815–832, 2015.
 Elizaveta Kuznetsova, “Microgrid Agent-Based Modelling And Optimization Under Uncertainty,” Dissertation, Universite de Versailles, 2014.
 M. Kraning, E. Chu, J. Lavaei, and S. P. Boyd, Dynamic network energy management via proximal message passing.
 Y. J. Jhi and M. D. Ilic, “Multi-Layered Optimization Of Demand Resources Using Lagrange Dual Decomposition,” Smart Grid, IEEE Transactions on, vol. 4, no. 4, pp. 2081–2088, 2013.
 B. Moradzadeh and K. Tomsovic, “Two-Stage Residential Energy Management Considering Network Operational Constraints,” IEEE Trans. Smart Grid, vol. 4, no. 4, pp. 2339–2346, 2013.
 Michael Grant and Stephen Boyd, CVX: Matlab software for disciplined convex programming, version 2.0 beta. http://cvxr.com/cvx.
 Michael Grant and Stephen Boyd. Graph implementations for nonsmooth convex programs, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), V. Blondel, S. Boyd, and H. Kimura, editors, pages 95-110, Lecture Notes in Control and Information Sciences, Springer, 2008. http://stanford.edu/~boyd/graph_dcp.html.
 H. Harb, J.-N. Paprott, P. Matthes, T. Schütz, R. Streblow, and D. Müller, “Decentralized scheduling strategy of heating systems for balancing the residual load,” Building and Environment, vol. 86, no. 0, pp. 132–140, http://www.sciencedirect.com/science/article/pii/S0360132314004260, 2015.
 Gurobi, Gurobi Optimization, Inc. Available: http://www.gurobi.com/ (2015, Jun. 15).
 P. O. Kriett and M. Salani, “Optimal control of a residential microgrid,” Energy, vol. 42, no. 1, pp. 321–330, 2012.
 N. Blaauwbroek, P. H. Nguyen, M. J. Konsman, Huaizhou Shi, Kamphuis, R. I. G, and W. L. Kling, “Decentralized Resource Allocation and Load Scheduling for Multicommodity Smart Energy Systems,” Sustainable Energy, IEEE Transactions on, vol. 6, no. 4, pp. 1506–1514, 2015.
 Telecom Italia SpA, Jade Site | Java Agent DEvelopment Framework Available: http://jade.tilab.com. Available: http://jade.tilab.com/ (2016, Feb. 18).
 MOSEK ApS MOSEK Optimization Toolbox [Online]. Available: http://mosek.com/. Available: https://www.mosek.com/ (2016, Feb. 18).
Mr Michael Diekerhof, RWTH Aachen University
Prof Antonello Monti, RWTH Aachen University
Electricity market: Demand Response and price optimization
One of the main research objectives in Demand Response (DR) is the design and implementation of technologies and mechanisms to lower the electricity consumption via energy efficiency measures, and to improve the electricity consumption via demand shifting. Increasing energy efficiency requires a reduction of energy demand peaks by shifting part of the energy consumption in off-peak hours. This can be done via DR mechanisms and load control.
Demand shifting can provide a number of advantages to the energy system :
- Load management can improve system security by allowing a demand reduction in emergency situations.
- In periods of peak loads even a limited reduction in demand can lead to significant reductions in electricity prices on the market.
- If users receive information about prices, energy consumption becomes more closely related to the energy cost, thus increasing market efficiency: the demand is moved from periods of high load (typically associated with high prices) to periods of low load.
- Load management can limit the need for expensive and polluting power generators, leading to better environmental conditions.
Potential benefits and implementation schemes for DR mechanisms are well documented in literature. DR programs can be defined as methods to induce deviations from the usual consumption pattern in response to stimuli, such as dynamic prices, incentives for load reductions, tax exemptions, or subsidies. They can be divided in two main groups: price-based and incentive-based mechanisms ,  and .
- Price-based demand response is related to the changes in energy consumption by customers in response to the variations in their purchase prices. This group includes DR mechanisms like Time-of-Use (ToU) pricing, Real Time Pricing (RTP) and Critical-Peak Pricing (CPP) rates. If the price varies significantly, customers can respond to the price structure with changes in their pattern of energy use. They can reduce their energy costs by adjusting the time of the energy usage by increasing consumption in periods of lower prices and reducing consumption when prices are higher. ToU mechanisms define different prices for electricity usage during different periods: the tariffs reflect the average cost of generating and delivering power during those periods. For RTP the price of electricity is defined for shorter periods of time, usually 1 h, again reflecting the changes in the wholesale price of electricity. In RTP customers usually have the information about prices. CPP is a hybrid ToU RTP program. This mechanisms is based on the real time cost of energy in peak price periods, and has various methods of implementation.
- Incentive-based demand response consists in programs with fixed or time varying incentives for customers in addition to their electricity tariffs. Incentive-Based programs (IB) include Direct Load Control (DLC), Interruptible/Curtailable service (I/C), Emergency Demand Response program (EDR), Capacity market Program (CAP), Demand Bidding (DB) and Ancillary Service (A/S) programs. Classical IB programs include DLC and I/C programs. Market-Based IB programs include EDR, DB, CAP, and the A/S programs. In classical IBP, customers receive participation payments (e.g. discount rate) for their participation in the programs. In Market-Based programs, participants receive money for the amount of their load reduction during critical conditions. In I/C programs, participants are asked to reduce their load to fixed values and participants who do not respond can pay penalties based on the program conditions. DB are programs in which consumers are encouraged to change their energy consumption pattern and decline their peak load in return for financial rewards and to avoid penalties. In EDR programs, customers are paid incentives for load reductions duringemergency conditions.
Demand Response mechanisms and load control in the electricity market represent an important area of research at international level, and the market liberalization is opening new perspectives. This calls for the development of methodologies and tools that energy providers can use to define specific business models and pricing schemes.
Every actor in the electricity market has different objectives. For example, retailers and generators aim to maximize their own profit by reducing their costs. In contrast, customers would like their electricity bills as low as possible. Game theoretical methods can also be used to capture the conflicting economic interests of the retailer and their consumers. Authors in  propose optimization models for the maximization of the expected market profits for the retailer and the minimization of the electricity cost for the consumer.
One implementation approach of DR mechanisms in the electricity market consists in defining economically and environmentally sustainable energy pricing schemes. In this field, optimization approaches to define dynamic prices have been proposed, and they focus on the definition of day-ahead prices for a period of 24 hours and for a single customer (or a single group of homogeneous customers). In , the response of a non-linear mathematical model is analyzed for the calculation of the optimal prices for electricity assuming default customers under different scenarios over a 24h period.  defines a model of an electric energy service provider in the environment of the deregulated electricity market. This problem studies the impact on the profits of several factors, such as the price strategy, the discount on tariffs and the elasticity of customer demand functions always over a 24h period.
Consumers may decide to modify their load profile to reduce their electricity costs. For this reason, it is important to analyze the effect that the market structure has on the elasticity demand for electricity.  proposes an elastic model to characterize the demand-response behavior and load management with ToU programs and it describes how the consumers behavior can be modeled using a matrix of self and cross-elasticities.  and  take into account also other schemes, and rely on the elastic model proposed in  to model the demand-response behavior.  assesses the impacts of ToU tariffs on a dataset of residential users in terms of changes in electricity demand, price savings, peak load shifting and peak electricity demand at sub-station level.
Response of the customers to the DR programs affects the daily load curve. Therefore, the Load Duration Curve (LDC) changes due to the responsiveness of the customers over a year and even the participation of the customers in DR programs can have considerable effects on the LDC : the effects of DR need to be investigated over the daily time horizon.  has adapted elasticity model mentioned above to ToU based prices and considered scenarios over a 24h period to better identify trends and assess how the characteristics of the market and the customers affect the consumption annual profiles.
Consumption and cost awareness has an important role for the effectiveness of demand response schemes for pricing optimization.  describes a system architecture for monitoring the electricity consumption and displaying consumption profiles to increase awareness. [13, 14] study how customers respond to price changes, and which price indicators are more relevant on this respect.
 A. De Filippo, M. Lombardi, and M. Milano, Non-linear Optimization of Business Models in the Electricity Market. Springer International Publishing, 2016.
 M. Albadi and E. El-Saadany, “Demand response in electricity markets: An overview,” in In Proceedings of IEEE power engineering society general meeting, 2007, pp. 1–5. M. Albadi and E. El-Saadany, “A summary of demand response in electricity markets,” Electric power systems research, vol. 78, no. 11, pp. 1989–1996, 2008.
 P. Palensky and D. Dietrich, “Demand side management: Demand response,” Intelligent Energy Systems, and Smart Loads, vol. 7, no. 3, pp. 381–388, 2011.
 J.S. Vardakas, N. Zorba, and C.V. Verikoukis, “ A Survey on Demand Response Programs in Smart Grids: Pricing Methods and Optimization Algorithms”, IEEE Communication Surveys & Tutorials, Vol. 17, no. 1, 2015.
 D. Kirschen, G. Strbac, P. Cumperayot, and D. de Paiva Mendes, “Factoring the elasticity of demand in electricity prices,” Power Systems, IEEE Transactions on, vol. 15, no. 2, pp. 612–617, May 2000.
 H. Aalami, M. Moghaddam, and G. Yousefi, “Demand response modeling considering interruptible/curtailable loads and capacity market programs,” Applied Energy, vol. 87, no. 1, pp. 243–250, 2010.
 H. Aalami, M. Moghaddam, and G. Yousefi, “Modeling and prioritizing demand response programs in power markets,” Electric Power Systems Research, vol. 80, no. 4, pp. 426–435, 2010.
 J. Torriti, “Price-based demand side management: Assessing the impacts of time-of-use tariffs on residential electricity demand and peak shifting in northern italy,” Energy, vol. 44, no. 1, pp. 576 – 583, 2012.
 J. Yusta, I. Ramrez-Rosado, J. Dominguez-Navarro, and J. Perez-Vidal, “Optimal electricity price calculation model for retailers in a deregulated market,” International Journal of Electrical Power & Energy Systems, vol. 27, no. 56, pp. 437 – 447, 2005.
 M. Samadi, M. H. Javidi, and M. S. Ghazizadeh, “The effect of time-based demand response program on ldc and reliability of power system,” in Electrical Engineering (ICEE), 2013 21st Iranian Conference on, May 2013, pp. 1–6.
 R. Tanaka, M. Schmidt, C. Ahlund, and Y. Takamatsu. An energy awareness study in a smart city lessons learned. In In Proc. of Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), 2014 IEEE Ninth International Conference on, pages 1–4. IEEE, 2014.
 I. Koichiro. Do consumers respond to marginal or average price? evidence from nonlinear electricity pricing. American Economic Review, 104(2):537—563, 2014.
 S. Borenstein. To what electricity price do consumers respond? residential demand elasticity under increasing-block pricing. Preliminary Draft April, 30, 2009.
 M. Zugno, J. M. Morales, P. Pinson, and H. Madsen, “A bilevel model for electricity retailers participation in a demand response market environment,” Energy Econ., vol. 36, pp. 182–197, Mar. 2013.
Allegra De Filippo, Michele Lombardi and Michela Milano, University of Bologna Wei-Yu Chiu; Hongjian Sun; H.V. Poor (2012). “Demand-side energy storage system management in smart grid”. 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm): no.73, 78, p.5–8, 2012.
 Lund, Peter D; Lindgren, Juuso; Mikkola, Jani; Salpakari, Jyri (2015). “Review of energy system flexibility measures to enable high levels of variable renewable electricity”. Renewable and Sustainable Energy Reviews. no. 45, p.785–807, 2015.
 A. A. Garcia, “Demand Side Integration,” Working Group C6.09, Cigre Technical Report, August 2011
 Lampropoulos I, Kling W L, Ribeiro P F and van den Berg J (2013). “History of demand side management and classification of demand response control schemes”. IEEE Power & Energy Society General Meeting (Vancouver BC), 2013.
 OECD, “OECD Economic Surveys: France 2011”, March 11, 2011.
 EDF Group: Demand Response and Pricing in France: EDF’s experience, New regulation, Main goals. 2007, April. Available http://smartenergydemand.eu/wp-content/uploads/2011/05/EDF-07-04-10-Dynamic-Pricing-in-France.ppt (2017, May 14).
Dr. Elizaveta Lebedeva, University of Tartu
Dr. Michael Diekerhof, RWTH Aachen Universit
Prof Antonello Monti, RWTH Aachen University