Future power systems with a large penetration of fluctuating renewable energy production from wind and solar power generation call for demand flexibility. In Denmark, for instance, on average 44\% of the power load in 2017 was covered by wind power production, and during several hours the wind production was well above 100% of the electricity load, which was possible partly because of the flexibility of the widely used district heating systems.
Heating and cooling represent a huge part of the total energy consumption. According to 2014 Eurostat figures, in the EU around 30\% of the primary energy is used to produce heat, and 40\% is used for electricity, including electricity for heat production . The dynamics and inertia of thermal systems and the low-cost storage capabilities for hot water, implies that district heating (DH) system are capable of playing an important role in the future intelligent and integrated energy system. As mentioned above, in Denmark DH systems already play a very important role in the integration of the fluctuating renewable energy production and for providing energy balancing services to the power grid.
Historically DH systems are often considered as single systems, but this is rather due to the historic emphasis on energy supply as subsystems of different supply sources (e.g., gas, coal, and electricity). However, today they act as a key element for integrating the different energy systems, and they provide some of the needed flexibility to the power system . DH systems also provide an eminent possibility of using excess heat from e.g. industrial production and cooling in super markets.
This section briefly describes the operational optimization problems involved in various parts of DH systems, and some methodologies and tools for solving these problems will be indicated. For operational convenience we will split the discussion into a number of subsystems, which can provide flexibility to the overall DH system. Each subsystem leads to an optimization problem and calls for adequate related methodologies for providing the optimal operation. The optimal operation of the following subsystems are considered:
In general, DH systems often consist of a spectrum of different possibilities for heat production. For example, thermal solar plants are becoming increasingly popular nowadays. However, the solar energy production is often hard to predict, and hence this calls for methods like probabilistic forecasting and optimization under uncertainty . Here methods like stochastic programming and stochastic control theory are obvious for solving the operation problem in near real time. Therefore, the operation of the total DH system can be considered as a set of nested stochastic programming and control problems, which are presented in more detail in the remainder of this section. In the following description only district heating will be considered, but almost the same principles can be used for district cooling.
The portfolio of production units in a DH system, comprising of combined heat and power (CHP) plants and heat-only units, can be used to react to current state of the energy system and thus increase efficiency and reduce imbalances. In periods with high generation of intermittent renewable power resources, the generation can be shifted to heat-only units, which maybe even consume power (e.g. heat pumps) to lower the imbalance in the grid, while fulfilling the heat demand. In periods with less power production from wind and photovoltaic, CHP plants can provide power to the market while producing heat. Thus, the coupling of the operation of the district heating system to the electricity markets is important . The key to reducing costs in the operational production is by considering all production units as a portfolio to make use of the flexibility. By optimizing the entire portfolio, the interplay between the units can be used to further reduce costs and increase income from the market. During the optimization several restrictions have to be considered, such as the capacities of the producing units and connected thermal storages as well as technical characteristics of the units (e.g. start up/shut down times and costs).
In recent years, the production of heat from the installed small CHP plants has slightly decreased in favor of heat-only units such as boilers and heat pumps, due to the reduction of the electricity prices. The design of today's electricity market forces CHP producers to present power production offers one day before the actual energy delivery. Consequently, forecast uncertainties in prices and heat demand must be considered for an optimal planning of DH systems. Furthermore, the above mentioned increase in solar thermal production introduces an additional source of uncertainty from the heat production side.
To efficiently operate this mixture of heat production units while reducing the operational costs, several optimization techniques such as mixed integer linear programming, Lagrangian relaxation, heuristics, or fuzzy linear programming have been proposed. However, the use of mixed integer linear programming prevails over the other methods due to the easy implementation of these programs in available commercial solvers. In addition, the formulation of two-stage stochastic and robust mixed integer linear programming problems allows the integration of uncertainty in the optimization problem yielding in better operation plans for CHP plants [5,6]. The use of two-stage stochastic programs to optimize the heat production of different heat-only, storage and CHP units translates into more flexibility in the real-time operation . Finally, stochastic programming has been proven to be an effective tool to make use of DH networks to integrate the uncertain production from renewable energy sources .
The problem of determining the optimal operation of DH network relates to finding the optimal combination of flow and temperature profiles that provide the minimal operational cost. Pumping costs are, however, often an order of magnitude smaller than the costs related to the heat loss induced by having a too high supply temperature profile in the network . Consequently, a reasonable control strategy for DH networks is to keep the supply temperature from the district heating plant as low as possible. This is in particular the case, if the heat production takes place at a CHP plant [10,11].
The control is subject to some constraints. For instance, the total heat requirement for all consumers must be supplied at any time and location, such that each individual consumer is guaranteed some minimum supply temperature. A lower supply temperature leads typically to large savings, since this implies lower heat losses from the transmission and distribution networks as well as lower production costs.
As described in , the optimal operational problem can be formulated as a stochastic problem which can be solved using dynamic programming. Furthermore, given probabilistic forecasts for the heat load, cf. , and stochastic models for the dynamics in the network, the problem can be described as a problem which can be solved using stochastic control theory.
A DH system is an example of a non-stationary system, implying that model parameters have to be time varying, e.g., the time-delay from the plants to the end-users is unknown and time-varying. Therefore, the methods used in conventional predictive control theory have to be modified . The modified controllers have been incorporated in a software package, PRESS (HeatTO), developed at the Technical University of Denmark. PRESS (HeatTO) has been applied and tested, e.g. at Vestkraft in Esbjerg, Denmark, and significant savings have been documented .
The end-users in DH system can provide flexibility by storing energy in the thermal mass of the buildings or in a local water tank. In  it is shown how nested stochastic control problems can be defined such that the thermal mass of buildings can provide services to the future smart grid. This is further explained in .
Furthermore, in order to avoid, e.g., costly upgrades of the existing network in large cities, it will be more and more important to control the maximum energy used within a certain interval. This can be obtained by a control that directs the maximum flow towards specific end-users or districts.
Dynamic tariffs provide another option for enabling flexibility in DH networks. For instance, to reduce the peak consumption in the morning, an extra price or penalty can be utilized during peak hours.
It is suggested in e.g.  that dynamic electricity prices can be used to control the electricity consumption and hence to enable the needed flexibility for integrating large shares of fluctuating and intermittent renewable power generation.
Time-varying price signals are an example of a penalty signal that can be linked to the optimization and control problem in order to arrive at a cost optimal solution by demand response. Another example are real-time marginal CO2 signals that can be used as a penalty signal linked to the optimization problem. Then the optimal solution will minimize the CO2 emission associated with the optimal control or operation.
Different penalty signals will lead to different optimal solutions for the problem and the choice depends on the context or societal ambition. Three of the most obvious penalty signals are the following:
It is clear that a DH system with controllers defined by an objective of minimizing the total emission would in general lead to an increased use of energy. However, this may happen during periods with, e.g., a large amount of wind power production and where the alternative would be to stop some wind turbines.
 J. Kiviluoma, S. Heinen, H. Qazi, H. Madsen, G. Strbac, C. Kang, N. Zhang, D. Patteeuw, T. Naegler, Harnessing flexibility from hot and cold: heat storage and hybrid systems can play a major role, IEEE Power and Energy Magazine 15 (1) (2017) 25-33.
 P. Meibom, K. B. Hilger, H. Madsen, D. Vinther, Energy comes together in Denmark: The key to a future fossil-free danish power system, IEEE Power and Energy Magazine 11 (5) (2013) 46-55.
 J. M. Morales, A. J. Conejo, H. Madsen, P. Pinson, M. Zugno, Integrating renewables in electricity markets: operational problems, Vol. 205, Springer Science & Business Media, 2013.
 H. Lund, S. Werner, R. Wiltshire, S. Svendsen, J. E. Thorsen, F. Hvelplund, B. V. Mathiesen, 4th generation district heating (4GDH): Integrating smart thermal grids into future sustainable energy systems, Energy 68 (2014) 1 - 11.
 I. Dimoulkas, M. Amelin, Probabilistic day-ahead CHP operation scheduling, in: 2015 IEEE Power & Energy Society General Meeting, 2015, pp. 1-5.
 M. Zugno, J. M. Morales, H. Madsen, Commitment and dispatch of heat and power units via affinely adjustable robust optimization, Computers & Operations Research 75 (2016) 191-201.
 M. G. Nielsen, J. M. Morales, M. Zugno, T. E. Pedersen, H. Madsen, Economic valuation of heat pumps and electric boilers in the danish energy system, Applied Energy 167 (2016) 189-200.
 A. Hellmers, M. Zugno, A. Skajaa, J. M. Morales, Operational strategies for a portfolio of wind farms and CHP plants in a two-price balancing market, IEEE Transactions on Power Systems 31 (3) (2016) 2182-2191.
 G. Sandou, S. Font, S. Tebbani, A. Hiret, C. Mondon, Predictive control of a complex district heating network, in: IEEE Conference on Decision and Control, Vol. 44, 2005, p. 7372.
 H. Madsen, H. Tangen, K. Sejling, O. P. Palsson, Models and Methods for Optimization of District Heating Systems: Part I: Models and Identification Methods, Institut for Matematisk Statistik og Operationsanalyse, Technical University of Denmark, 1990.
 H. Madsen, K. Sejling, H. T. Søgaard, O. P. Palsson, On ow and supply temperature control in district heating systems, Heat Recovery Systems and CHP 14 (6) (1994) 613-620.
 H. A. Nielsen, H. Madsen, Modelling the heat consumption in district heating systems using a grey-box approach, Energy and Buildings 38 (1) (2006) 63-71.
 V. Peterka, Predictor-based self-tuning control, Automatica 20 (1) (1984) 39-50.
 R. Halvgaard, N. K. Poulsen, H. Madsen, J. B. Jørgensen, Economic model predictive control for building climate control in a smart grid, in: 2012 IEEE PES Innovative Smart Grid Technologies (ISGT), 2012, pp. 1-6.
 H. Madsen, J. Parvizi, R. Halvgaard, L. E. Sokoler, J. B. Jørgensen, L. H. Hansen, K. B. Hilger, Control of electricity loads in future electric energy systems, in: Handbook of Clean Energy Systems, Wiley Online Library, 2014.
Prof. Henrik Madsen, Technical University of Denmark
Dr. Ignacio Blanco, Technical University of Denmark
Dr. Daniela Guericke, Technical University of Denmark