1. Introduction
Global demand for clean and renewable energy sources, such as wind and solar power, is motivated by growing concerns about environmental sustainability and climate change. Like many other industrialized nations, the United States has committed to reducing harmful carbon emissions while increasing the proportion of power generated by non-fossil-based sources over the next few decades. Specifically, it plans to (1) cut carbon emissions in half by 2030, (2) reach 100% clean electricity by 2035, and (3) achieve net-zero carbon emissions by 2050 [20]. However, these ambitious goals can only be attained if the supply and reliability of renewable energy sources are sufficient to meet consumer demand while remaining economically competitive with fossil fuel sources. Some in the operations community have considered optimal investment strategies for renewable energy technologies, such as wind and solar [Reference Aflaki and Netessine3, Reference Kök, Shang and Yücel30]. Wind energy is playing an increasingly important role in the move towards more sustainable forms of energy because it is clean, accessible, affordable, and inexhaustible. Wind energy is produced by a wind turbine—a system designed to convert kinetic energy (wind) into mechanical energy to generate electricity. Wind turbines are typically comprises a structural tower, the rotor and hub (including three turbine blades), and the nacelle, which houses most of the mechanical and electrical systems, including the gear box and generator, among others. A collection of wind turbines operating in proximity to one another is known as a wind farm.
Although wind farms can be land-based (onshore) or sea-based (offshore), the vast majority of wind generation in the USA originates from onshore systems [Reference Musial, Spitsen, Duffy, Beiter, Marquis, Hammond and Shields47]. New onshore wind farm installations are very costly, as they require significant capital investments, supporting infrastructure and connection services, and these costs are substantially higher for offshore wind farms [Reference Papadopoulos, Coit and Ezzat52]. However, due to escalating demand for wind energy and depletion of available land resources, offshore wind has received a great deal of attention over the past two decades. Though it is more costly, offshore wind holds a few advantages over its onshore counterpart. First, stronger and more consistent wind resources are available offshore, thereby increasing generating capacity and improving reliability. Second, offshore wind farms are usually located in vast open water areas, allowing for large-scale installations. Finally, because offshore wind farms are distant from dense population centers, their negative visual and auditory impacts are significantly reduced. Despite these advantages, wide-scale adoption of offshore wind has not yet been realized due to significant capital investment costs and its levelized cost of energy (LCOE)—an aggregate economic measure that is used to compare lifetime power generation costs associated with different technologies [Reference Short, Packey and Holt65]. The three components of this aggregate cost are broadly categorized as capital expenditures, operational expenditures, including operations and maintenance (O&M) costs, and decommissioning expenditures. O&M costs typically account for 20–50% and 35–50% of the cost of producing onshore and offshore wind energy, respectively [Reference Stehly, Beiter and Duffy67, Reference Wiser and Bolinger74]. Moreover, the replacement of large, critical components such as turbine blades, gear boxes, and generators constitutes as much as 75% of total O&M costs [Reference Maples, Saur, Hand, van de Pietermen and Obdam40, Reference uit het Broek, Veldman, Fazi and Greijdanus70]. Therefore, the viability of large-scale offshore wind generation hinges on a reduction of O&M costs and, ultimately, the LCOE.
While similarities exist between the maintenance of onshore and offshore wind farms, offshore farms are especially difficult to maintain due to accessibility issues, environmental conditions (e.g., wind speed and wave height), safety concerns, and the need for costly, specialized crews and equipment to perform operations. Accessibility is usually associated with their distance from the shore, sea conditions, and availability of sea vessels. When unanticipated failures occur, helicopters are often deployed to reduce delays in accessing the turbines. Another unique feature of offshore wind farm maintenance is the need for different types of vessels to carry crews, replacement parts, and equipment. In addition to crew transfer vessels (CTVs) and service operational vessels (SOVs), a jack-up vessel (JUV) is essential for replacement of large, critical components such as turbine blades and generators [Reference uit het Broek, Veldman, Fazi and Greijdanus70]. Because it may be cost-prohibitive for a wind farm operator to own such resources, they must be chartered from external providers. Dalgic et al. [Reference Dalgic, Lazakis, Dinwoodie, McMillan and Revie15–Reference Dalgic, Lazakis, Turan and Judah17] have documented several complicating factors associated with the use of JUVs. The accessibility and availability issues associated with offshore wind farm maintenance are aptly detailed in [Reference Hofmann25, Reference Papadopoulos, Coit and Ezzat52, Reference Ren, Verma, Li, Teuwen and Jiang57, Reference Rinaldi, Thies and Johanning58, Reference Seyr and Muskulus61, Reference Shafiee62].
These considerations motivate our work here in which we seek to prescribe optimal replacement policies for offshore wind farms that minimize the long-run expected total discounted setup, replacement, and lost production costs. Critical to our approach is the inclusion of both economic and stochastic dependence between the turbines in a wind farm. Economic dependence is present because significant setup costs are incurred any time replacements are performed. These costs can be shared among multiple turbines, allowing operators to exploit economies of scale. Stochastic dependence refers to the association of the turbines’ degradation processes stemming from their common operation in a randomly varying, exogenous environment. The repair or replacement of turbine components can be either preventive (occurring before a failure) or reactive (occurring after a failure); in either case, a per-replacement cost is incurred. Substantial downtime costs (in the form of lost revenue) are also incurred whenever a major component fails and/or a turbine is taken offline to perform replacements. We formulate and analyze a Markov decision process (MDP) model to prescribe optimal replacement policies and provide practical insights to wind farm operators.
1.1. Relevant literature
An offshore wind farm can be viewed as a collection of complex, progressively degrading multiunit systems operating in a dynamic environment. A multiunit system can refer either to one system with multiple (dependent or independent) components or a collection of systems that are dependent in some way; we adopt the latter view for offshore wind farms. Many other researchers embrace the first view in order to devise tractable maintenance policies for multiple independent units. For instance, Tian and Liao [Reference Tian and Liao68] examined policies for general multiunit systems with several identical, independent units and numerically obtained dual threshold-type policies. Zhang et al. [Reference Zhang, Wu, Li and Lee80] decomposed a multiunit system and established an opportunistic dual-threshold policy for the maintenance of each single-unit system. Arts and Basten [Reference Arts and Basten7] proposed a condition-based maintenance (CBM) model for each component of a two-component system independently by considering both scheduled and unscheduled downtime. After prescribing optimal maintenance policies for each component, they synchronized scheduled downtime in order to maintain both units simultaneously. Such decomposition procedures usually lead to tractable maintenance policies; however, it is well-known that interdependent, degrading systems present significant challenges to strategic maintenance planning [Reference Dekker, Wildeman and van der Duyn Schouten19, Reference Ohashi and Nishida49, Reference Wang, Hou, Qin, Peng and Zhou73, Reference Zhu, Xiang and Zeng83], especially when multiple types of dependence exist. Olde Keizer et al. [Reference Olde Keizer, Flapper and Teunter50] surveyed several dependence types prevalent in CBM strategies for degrading systems. Here, we focus primarily on economic and stochastic dependence.
First, economic dependence exists in multiunit systems generally, and in offshore wind farms specifically, because maintenance operations are typically performed jointly on several units to share setup costs and reduce the total downtime overall. Ko and Byon [Reference Ko and Byon34] considered the problem of maintaining a large number of homogeneous units that degrade independently over time but are economically dependent due to shared setup costs. Salari and Makis [Reference Salari and Makis59] examined economic dependence between the units of a production system in which the production rate of each unit decreases as the degradation level increases. Zhu et al. [Reference Zhu, Peng and van Houtum81] considered a multicomponent system with heterogeneous, independently degrading units and significant shared maintenance setup costs. Zhu et al. [Reference Zhu and Xiang82, Reference Zhu, Xiang and Zeng83] formulated a multistage stochastic integer program to optimize the maintenance of a multicomponent system subject to economic dependence. They derived structural results for a simpler two-stage version of the model to enhance tractability. Related to economic dependence is resource-based dependence, in which units are linked through a shared and limited set of resources [Reference Abbou and Makis1, Reference Kurt and Kharoufeh36, Reference Shafiee and Sørensen63]. Olde Keizer et al. [Reference Olde Keizer, Teunter and Veldman51] used a MDP model to jointly optimize maintenance activities and spare parts inventory of a multicomponent system in which the components share identical spares. Maintenance is feasible only if spares are available, thereby inducing resource-based dependence. They obtained optimal replacement and ordering policies that minimize the total long-run average cost per time unit. Similarly, de Pater and Mitici [Reference de Pater and Mitici18] considered multiple repairable components whose maintenance activities are linked via the availability of spares. They integrated remaining useful life (RUL) estimation with a limited inventory of spares to devise a predictive maintenance model subject to resource-based dependence. Wind turbines may be forced to sit idle awaiting the resources needed to restore them to working condition, resulting in economic losses in the form of lost revenues.
Second, stochastic dependence can exist between the units in a multiunit system in at least three ways: via load sharing, dependence of degradation processes due to structural dependencies or the association of degradation processes via a common operating environment. The first type exists when the failure or degradation of one unit influences the lifetimes of other units. For example, if additional power production is required of one wind turbine due to the failure of another, load sharing can lead to accelerated degradation and premature failure of the operational unit [Reference Berrade, Scarf and Cavalcante9, Reference uit het Broek, Teunter and Veldman69, Reference Zhang, Li and Coit76, Reference Zhang, Fouladirad and Barros78, Reference Zhang, Wu, Li and Lee80]. Structural dependence refers to the physical configuration of components within the system, for example, the components are connected in series, parallel, or some other manner [Reference Andersen, Andersen, Kulahci and Nielsen5, Reference Liu, Pandey, Wang and Zhao38, Reference Yousefi, Coit and Zhu75, Reference Zhang, Fouladirad, Barros and Zhang79]. Third, stochastic dependence can be induced by exogenous factors, such as environmental conditions, that simultaneously influence the degradation processes of individual units. Reliability and availability indices, as well as optimal maintenance strategies, for single-unit systems operating in a randomly evolving environment have been well studied [Reference Kharoufeh and Cox31–Reference Kharoufeh and Mixon33, Reference Kurt and Kharoufeh35, Reference Zhang, Deng, Liu and Zhang77]. Random shock processes have often been used to model the operating environment and its impact on competing degradation processes [Reference Cha, Finkelstein and Levitin14, Reference Jiang, Feng and Coit28, Reference Rafiee, Feng and Coit55, Reference Rafiee, Feng and Coit56, Reference Yousefi, Coit and Zhu75]. Of particular interest to our work here is a model by Ulukuş et al. [Reference Ulukuş, Kharoufeh and Maillart71], who examined the optimal replacement of a single-unit system subject to environment-driven degradation. Using an MDP model, they established the existence of an optimal degradation-based threshold replacement policy for each environment state. Furthermore, they conjectured conditions for which the thresholds are monotone; however, this conjecture was based only on empirical evidence. Abdul-Malak et al. [Reference Abdul-Malak and Kharoufeh2] extended the model in [Reference Ulukuş, Kharoufeh and Maillart71] to the case of multiple units operating in a shared environment, where stochastic dependence stems from the association of degradation processes via the common environment. They established monotonicity of the value function over the entire state space, although they did not characterize the optimal policy. Instead, numerical approximations were used to illustrate the replacement policies, and for the special case of a single unit, they provided a partial resolution to the conjecture in [Reference Ulukuş, Kharoufeh and Maillart71] and proved conditions needed to ensure that the replacement thresholds are monotone (for each environment state). Our work differs from prior models in that we seek to establish structured replacement policies that consider the following: (1) economic dependence due to shared setup costs, (2) stochastic dependence induced by a common, randomly evolving environment, and (3) the impact of production losses that also depend on the environment state.
1.2. Summary of contributions and results
Our main contributions and results are summarized as follows:
(1) The problem we study has a unique feature: The turbines in the farm are exposed to a common environment that evolves randomly over time. In particular, the degradation processes of the turbines are associated with each other through their common exposure to the environment. Therefore, in addition to economic dependence induced by the sharing of setup costs, there is stochastic dependence among the turbines. We create and analyze an MDP model that incorporates economic and stochastic dependence, both of which significantly influence maintenance strategies.
(2) We establish structural properties that lead to a characterization of the optimal replacement policy. First, we show that insights from the analysis of single-unit systems carry over to our setting. Specifically, for a single turbine, the optimal policy is monotone in its degradation level when the degradation levels of all other turbines are fixed, as seen in Figure 1(a). For a fixed turbine 1 degradation level (level A), there is a turbine 2 degradation threshold (level B) below which it is not optimal to replace and above which it is optimal to replace. Other models assume that it is always optimal to replace a failed system, implying that a replacement threshold always exists. However, in our setting, it is possible that replacement of a failed turbine is not prescribed due to significant shared setup costs. Second, for the special case of two turbines, we provide a novel result that, for a given level of degradation of one turbine, the optimal policy is monotone in the degradation level of the other turbine beyond a certain point. Referring to Figure 1(b), for a fixed turbine 2 degradation level (level D), beyond the point X, there is a threshold degradation level of turbine 1 (level C) below which it is not optimal to replace turbine 2, but above which it is optimal. Similar to the prior case, it may not be optimal to replace turbine 2 when turbine 1 is failed. Third, we show that these thresholds (if they exist) are monotone in the degradation level of turbine 2. That is, in Figure 1(c), when the degradation level of turbine 2 is level G (level H), the optimal threshold for replacing turbine 2 is level F (level E) over the degradation level of turbine 1. Our results show that level E is necessarily less than or equal to level F. As a byproduct, the region where the optimal policy replaces both turbines (the opportunistic region) is completely characterized by a monotone curve (depicted by the dotted curve in Figure 6) in the space of degradation levels of the two turbines.
(3) Our numerical results provide a procedure to assess the value of incorporating dependence and adequately modeling the environment, as well as the consequences of incorrect assumptions on the replacement policies and costs. Our results provide insights as to how the replacement policies depend on the environment and underlying assumptions.
The remainder of the paper is organized as follows. In Section 2, we describe the degradation model and formulate the replacement problem using an MDP model. Section 3 establishes properties of the cost function and the structure of the optimal replacement policy. In Section 4, numerical examples are used to illustrate replacement policies and demonstrate the value of incorporating dependence and the environment. Finally, some concluding remarks are provided in Section 5.
2. Model and problem formulation
In this section, we first provide a mathematical model of wind turbine degradation due to normal operation and the influence of a randomly evolving environment. Subsequently, we formulate a stochastic optimization problem to minimize the expected total discounted setup, replacement, and downtime costs over an infinite horizon.
2.1. Degradation model
Consider a finite collection $\mathcal{N}:=\{1,\ldots,N\}$ of N wind turbines operating in an offshore wind farm. Critical wind turbine components (e.g., generators, transformers, rotors, and turbine blades) degrade over time due to normal usage and the influence of their ambient operating environment. For example, it is well known that ocean air is corrosive to wind turbine blades due to its higher salt content [Reference Dong, Yuan and Ma22]. In what follows, the phrase “turbine degradation” refers to the degradation of a single, critical component whose replacement requires substantial time, effort, and resources. That is, we do not specify which component but refer more generally to the turbine’s degradation. Each turbine degrades until its cumulative level of degradation first reaches, or exceeds, a critical deterministic threshold ξ (ξ > 0), at which time the turbine is said to be failed. In the problem formulation that follows, we consider discrete decision epochs $m\in \mathbb{Z}_+:=\{0,1,2,\ldots\}$ and define a period as the time between two consecutive decision epochs. Let $X_n(m)$ denote the degradation of turbine $n\in\mathcal{N}$ at decision epoch m with state space $[0,\xi]$. Here, level 0 means a turbine is in as-good-as-new condition. Next, define the vector-valued process $\mathbb{X}=\{\left(X_1(m),\ldots,X_N(m)\right):m\in \mathbb{Z}_+\}$ describing the joint degradation process of all N turbines with continuous state space $\mathcal{X}:=[0,\xi]^N$. At decision epoch m, turbine n is said to be working if $X_n(m)\in [0,\xi)$ and failed if $X_n(m)\ge\xi$. Hereafter, and without loss of generality, we set the failure threshold ξ = 1. Stochastic processes are used extensively in the reliability and maintenance optimization communities to model the temporal and/or spatial evolution of degradation in systems. For a single system, $\{X_n(m):m\in \mathbb{Z}_+\}$ is often assumed to evolve as a discrete-time Markov chain (DTMC) on a finite state space, or when a continuous-time, continuous-state process is more appropriate, degradation can be modeled using gamma, Wiener, or other Lévy processes [Reference Andrawus, Watson and Kishk6, Reference Besnard and Bertling10, Reference Liu, Pandey, Wang and Zhao38, Reference Nielsen and Sørensen48].
For our purposes here, it is appropriate to model not only degradation but also the influence of a randomly varying environment on the evolution of degradation. Wind turbines are often located in proximity to one another within a wind farm; thus, they experience not only similar environmental conditions but also wake effects resulting from changes in wind speed [Reference Ding21]. Although the turbines operate independently, their degradation processes exhibit stochastic dependence due to their exposure to a common environment. Monbet et al. [Reference Monbet, Ailliot and Prevosto44] surveyed stochastic models—many of which are Markovian—that can be used to model wind conditions and the sea state from time series data. Although the environment realistically evolves continuously in time and space, researchers have assumed that it evolves over finitely many discrete states. Representative examples include [Reference Abdul-Malak and Kharoufeh2, Reference Bian, Gebraeel and Kharoufeh11, Reference Kharoufeh and Cox31, Reference Kharoufeh, Finkelstein and Mixon32, Reference Ulukuş, Kharoufeh and Maillart71]. In those specific models, each environment state maps to a unique, positive degradation rate. That is, if the environment occupies state i, then the system degrades linearly at a rate $r(i) \gt 0$. Consequently, in the absence of any type of maintenance intervention, the degradation process evolves as a piecewise linear function of time (almost surely). In contrast to those models, our main theoretical results do not impose such restrictive assumptions but only require that future degradation can be predicted using the current level of degradation and the environment state. The discrete environment states are essentially aggregations of several factors, including wind speed, ambient temperature, relative humidity, wave height, and others. Hence, these states can be viewed as approximations of the true (continuous) environment states over time. For example, if the environment evolves as a diffusion model with a randomly evolving drift coefficient, the distinct environment states can be viewed as the average behavior of the diffusion process within an interval of time. Flory et al. [Reference Flory, Kharoufeh and Gebraeel23] employed Markov chain Monte Carlo methods to model such behavior.
Let J(m) denote the state of the environment at decision epoch m and assume $\mathbb{J}:=\{J(m):m\in\mathbb{Z}_+\}$ evolves as a temporally homogeneous DTMC on the finite state space $\mathcal{L}:=\{1,\ldots,L\}$ with $L\in \mathbb{N}$. The environment states are ordered such that state 1 represents the most favorable condition for maintenance activities and state L represents the least favorable condition. To account for accessibility, resource availability, and maintenance feasibility issues, we can append $\mathcal{L}$ with additional states that reflect conditions under which maintenance cannot be performed (e.g., when wind speeds and/or wave heights are excessive, transport vessels are unavailable and/or crews are unavailable). In such cases, the only feasible action is “do nothing” and wait until the next decision epoch. The one-step transition probability matrix (TPM) of $\mathbb{J}$ is denoted by $\mathbf{P}=[p_{\ell k}]$, where $p_{\ell k}$ is the conditional probability that the environment state is k at decision epoch m + 1, given that it was $\ell$ at epoch m for any $m\in \mathbb{Z}_+$. The number of distinct environment states (L) can be estimated using the well-known Bayesian information criterion [Reference Flory, Kharoufeh and Gebraeel23, Reference Schwarz60]. Anastasiou and Tsekos [Reference Anastasiou and Tsekos4] concluded that six-to-eight distinct states are sufficient to model environmental conditions using a first-order, stationary Markov chain and noted that the stationarity assumption has been employed to model wind speed and wave height in other maritime applications [Reference Bern and Houmb8, Reference MacCallum and Bolland39]. It is well-known that weather conditions exhibit strong seasonality effects, as highlighted and modeled by Byon et al. [Reference Byon and Ding12, Reference Byon, Ntaimo and Ding13]. To account for these effects, one can employ a different DTMC model for different seasons of the year. The transition probabilities can be estimated using proportions data [Reference Jones29, Reference McGuire41] or by mining time series data [Reference Song, Geng, Kusiak and Xu66]. A realization of the joint process $(\mathbb{X},\mathbb{J})$ is an $(N+1)$-dimensional vector $(\textbf{x},\ell)$, where $\textbf{x}=(x_1,\ldots,x_N)\in\mathcal{X}$ and $\ell\in \mathcal{L}$.
We assume that future degradation can be predicted via the current state $(\textbf{x},\ell)$. For each turbine $n\in \mathcal{N}$ and a triplet $(x_n,\ell,t)$, let $f_n:[0,1]\times \mathcal{L} \times \mathbb{R}_{+}\to \mathbb{R}_{+}$ be a generic mapping that predicts the degradation of turbine n after t time units when its current degradation level is xn and the environment state is $\ell$. It can assume any number of functional forms (e.g., linear, piecewise linear, exponential, etc.); however, we impose no such assumption. This mapping satisfies the initial condition $f_n(x_n,\ell,0)=x_n$, and we impose the natural assumption that, for each $n\in \mathcal{N}$, fn is monotone nondecreasing in $(x_n,\ell,t)$, i.e., $f_n(x_n,\ell,t)\le f_n(x^{\prime}_n,\ell^{\prime},t^{\prime})$ for any $x_n\le x^{\prime}_n$, $\ell\le \ell^{\prime}$, and $t\le t^{\prime}$. That is, future degradation is assumed to be higher than the present level of degradation, and because the state space $\mathcal{L}$ is ordered, higher environment states lead to accelerated degradation of the units; however, we impose no additional assumptions on fn. Condition monitoring or sensor data (e.g., from a Supervisory Control and Data Acquisition (SCADA) system) can be used to estimate the evolution of degradation and/or the RUL of components using the techniques developed in [Reference Bian, Gebraeel and Kharoufeh11, Reference Gebraeel, Lawley, Li and Ryan24], for example. Papadopoulos et al. [Reference Papadopoulos, Coit and Ezzat53] suggest that when a dynamic environment is considered, the RUL can be estimated using the models developed by Bian et al. [Reference Bian, Gebraeel and Kharoufeh11].
2.2. Problem formulation
Maples et al. [Reference Maples, Saur, Hand, van de Pietermen and Obdam40] note that the majority of wind turbine O&M costs are due to the replacement of critical components (e.g., generators, transformers, rotors, and turbine blades). Thus, optimally timed replacements that exploit economies of scale can reduce O&M costs for wind farms and the overall cost of wind energy to consumers. Here, we present an MDP model to formulate the problem of optimally replacing wind turbines with the objective of minimizing the expected total discounted, setup, replacement and downtime costs over an infinite horizon.
Periodic inspections, which are assumed to be costless and instantaneous, are performed to reveal the state of each turbine perfectly (i.e., the state is completely observable). Such observations can be obtained, for example, by using sensors attached to critical components and tracking important signals of degradation, such as vibration or acoustic signals [Reference Bian, Gebraeel and Kharoufeh11]. A decision to replace or not replace a unit is made by a decision maker (DM) just after an inspection such that $\mathbb{Z}_+=\{0,1,\ldots\}$ is the set of decision epochs, and the time between any two consecutive inspections is a period in the MDP model. The state of the system is the $(N+1)$-dimensional vector $(\textbf{x},\ell)$ whose state space is $\Gamma:=\mathcal{X} \times \mathcal{L}$. For any decision epoch $m\in \mathbb{Z}_+$, let $a_n\in \{0,1\}$ denote the action taken for turbine $n\in \mathcal{N}$, where $a_n=0$ means “do nothing” and $a_n=1$ means “replace.” Therefore, the N-dimensional vector $\textbf{a}=(a_1,\ldots,a_N)$ describes the actions taken by the DM, and the action space is $\mathcal{A}:=\{0,1\}^N$. Whenever replacement is prescribed, the turbine must be shut down for one full period. Replacement activities commence immediately after the inspection and end just prior to the subsequent inspection. We assume that a period is sufficiently long to ensure that replacements are completed during a single period. Because the average time to complete a major replacement is between 2 to 7 days (see Maples et al. [Reference Maples, Saur, Hand, van de Pietermen and Obdam40]), the time between two consecutive decision epochs is 1 week in our model. However, it is important to note that the length of the period can be adjusted to account for different time scales when the model is used to prescribe the optimal timing of maintenance actions for minor components. In such cases, the period length can be set to 1 day. All replacements are assumed to be perfect and reset the degradation to level 0. For the remainder of the paper, vector inequalities are assumed to hold component-wise. That is, $\textbf{a}\le \textbf{a}^{\prime}$ implies $a_n\le a_n^{\prime}$, and $\textbf{x}\le \textbf{x}^{\prime}$ implies $x_n\le x_n^{\prime}$ for each $n\in \mathcal{N}$.
Let ${\hat{\textbf{x}}}_n(x_n,\ell,a_n)$ be the predicted level of degradation of turbine n at the start of the next period when the current degradation and environment states are xn and $\ell$, respectively, and action an is taken. Then,
Following a replacement, degradation is returned to level 0 (an as-good-as-new condition), and without intervention, the turbine either fails or assumes some higher degradation level $f_n(x_n,\ell,1)$.
Next, we describe the setup and other costs associated with replacements in an offshore wind farm. If at least one unit is replaced, a setup cost cs ($c_s \gt 0$), which comprises significant equipment and crew transfer costs, is incurred. For offshore wind farms, the distance between the base station and the shoreline is usually significant such that helicopters and SOVs and/or JUVs must be used to transport needed equipment and crews to the farm. Additionally, heavy equipment is required to perform replacements; hence, the setup cost cs can be substantial (in excess of £250,000 per day of operation; see [Reference McMorland, Flannigan, Carroll, Collu, McMillan, Leithead and Coraddu42, Reference Shields, Marsh, Stefek, Oteri, Gould, Rouxel, Diaz, Molinero, Moser and Malvik64]). However, this cost can be shared if multiple turbines are replaced in the same period. The shared setup cost induces economic dependence between the turbines. Second, a per-replacement cost cr ($c_r \gt 0$) is incurred for every turbine that is replaced. This cost compromises the procurement cost of a new component, as well as the labor hours needed to complete the replacement. Finally, a downtime cost is incurred for any turbine that either (1) fails in the time interval between two inspections or (2) is taken offline so that a replacement can be performed. This cost corresponds to lost power production due to turbine unavailability [Reference Maples, Saur, Hand, van de Pietermen and Obdam40]. As noted by Wan et al. [Reference Wan, Ela and Orwig72], power output is a function of wind speed and is typically characterized by a monotone increasing, S-shaped curve that is convex at low wind speeds, concave after an inflection point, and flat beyond an upper speed threshold [Reference Hwangbo, Johnson and Ding26]; therefore, we assume the downtime cost rate is a function of the environment state $\ell$. Let $c_d(\ell)$ denote the downtime cost per unit time per turbine when the environment is in state $\ell$. For example, if wind speed instantiates the environment, disjoint intervals of speed may correspond to different regions of the power output curve, as depicted in Figure 2(a). Three distinct wind speed ranges (low, medium, and high) result in distinct regions of power output.
Figure 2(b) illustrates how the wind speed intervals can be mapped to environment states in order to associate a downtime cost rate with each state. Wind speed can be mapped to power production using the well-known method of bins, as prescribed by the International Electrotechnical Commission [27]. In this specific illustration with three environment states, we divide the wind speed range into three intervals of equal length, with the lowest environment state (state 1) corresponding to wind speeds that exceed the cutoff speed; therefore, we assign a positive downtime cost rate for this state. We assume the downtime cost rate $c_d(\ell)$ is strictly increasing in $\ell\in \mathcal{L}$ and bounded from above. That is, $0 \lt c_d(1) \lt \cdots \lt c_d(L) \lt \infty$.
We now describe the degradation dynamics of a particular turbine $n\in \mathcal{N}$ with and without replacement. Figure 3(a) and 3(b) depicts scenarios in which the environment state is fixed at $\ell$ throughout the period. From Figure 3(a), we see that at decision epoch m, the turbine can either be failed (point A) or working (point B).
For either case, if replacement is prescribed at time m, the turbine is down for the entire period and is restored to a new condition at decision epoch m + 1 (point C); hence, the cost $c_d(\ell)+c_r$ is incurred in each of the two scenarios of Figure 3(a). Figure 3(b) depicts the same two scenarios in which the turbine is either failed (point A) or working (point B) at time m. Here, if replacement is not prescribed at decision epoch m and the unit is failed, it remains failed at m + 1 so that the incurred cost is $c_d(\ell)$. On the other hand, if the turbine is working at time m (point B) and replacement is not prescribed, two possible scenarios (represented by the dotted lines) may occur. First, the turbine continues degrading and reaches the critical threshold (ξ = 1) before decision epoch m + 1 (point D). In this case, the turbine is down from the crossing time (point D) until the start of the next period (point E). This downtime is denoted by $\tau_n(x_n,\ell)$ and is obtained via the predictive function $f_n(x_n,\ell,t)$. Specifically, let $\tau_n(x_n,\ell)$ denote the predicted within-period downtime if no replacement is performed ($a_n=0$), xn is the degradation at the start of the period, and the environment state is $\ell$. Then,
For this first scenario, the downtime cost $c_d(\ell)\cdot \tau_n(x_n,\ell)$ is incurred for turbine n. For the second scenario, the turbine is working at time m (point B), but its level of degradation does not reach the failure threshold before decision epoch m + 1. Rather, it continues operating for the entire period and evolves to a higher level of degradation at decision epoch m + 1 (point F). In this case, $\tau_n(x_n,\ell)=0$ so that no downtime cost is incurred. Finally, we note that if the turbine is failed at decision epoch m (point A) and replacement is not prescribed, it remains failed at decision epoch m + 1 (point E). Since $f_n(1,\ell,t)=1$ for all $\ell\in \mathcal{L}$ and $t\ge 0$, it follows that $\tau_n(1,\ell)=1$. Therefore, the cost $c_d(\ell)\cdot 1=c_d(\ell)$ is incurred.
For notational convenience, define an indicator function $\mathbb{I}(\textbf{a})$, where $\mathbb{I}(\textbf{a})=1$ if $a_n=1$ for some $n\in \mathcal{N}$ (at least one replacement is performed) and $\mathbb{I}(\textbf{a})=0$ if $a_n=0$ for all $n\in \mathcal{N}$ (no replacements are performed). The one-step immediate cost is then
The first term on the RHS of Equation (3) represents the setup cost. The second term is the sum of replacement costs and power production losses incurred for all replaced turbines. Finally, the third term accounts for the downtime costs estimated by the predictive functions for turbines that are not replaced.
All immediate costs are discounted at a fixed rate λ $(0 \lt \lambda \lt 1)$. The objective is to minimize the expected total discounted setup, replacement, and downtime costs over an infinite horizon. Let $\mathcal{P}$ denote the set of all nonanticipative replacement policies. Starting in state $(\textbf{x},\ell)$, the DM solves
where $\textbf{a}_{\pi}(\textbf{x}_m,\ell_m)$ denotes the action prescribed by policy $\pi\in \mathcal{P}$ in state $(\textbf{x}_m,\ell_m)$ at decision epoch m. Equation (4) is equivalent to
where $\hat{\textbf{x}}(\textbf{x},\ell,\textbf{a})=\left({\hat{\textbf{x}}}_1(x_1,\ell,a_1),\ldots,{\hat{\textbf{x}}}_N(x_N,\ell,a_N)\right)$ is the N-dimensional vector of predicted degradation levels at the start of the next period. Note that the state space Γ is Borel-measurable, and the action space $\mathcal{A}$ is finite. Additionally, the immediate costs are strictly positive and bounded, and the problem is discounted. Therefore, by Theorems 6.2 and 6.3 of [Reference Puterman54], there exists an optimal, deterministic stationary replacement policy, and the value iteration (VI) algorithm converges to the optimal value. Next, define the optimal action in state $(\textbf{x},\ell)$ by
That is, if there exist multiple optimal actions in state $(\textbf{x},\ell)$, we take the minimum action; hence, Equation (6) has a unique solution. In Section 3, we establish structural properties and provide some practical insights into this important operational problem.
3. Structural results and insights
In this section, we establish some properties of the value function $V(\textbf{x},\ell)$ and the optimal replacement policy of the MDP model presented in Section 2. Before proceeding to the main results, we first provide two lemmas that describe useful properties of the immediate and downtime costs. The proofs of all results are contained in the Appendix.
Lemma 3.1. For each $n\in \mathcal{N}$, $\tau_n(x_n,\ell)$ is monotone nondecreasing in $x_n\in [0,1]$ and $\ell\in \mathcal{L}$.
Lemma 3.1 implies that, for each turbine n, the first crossing time to the failure threshold is shorter starting from higher degradation and environment states. Thus, the resulting downtime within a period is longer. Before proceeding to Lemma 3.3, we first review the notion of discrete concavity, which is more fully elucidated by Murota et al. [Reference Murota45, Reference Murota and Tamura46].
Definition 3.2. Let $X\subseteq \mathbb{Z}$. A function $f:X\to \mathbb{R}$ is said to be discrete concave on its domain X if and only if $X\ne \emptyset$ and
Discrete concavity is useful for characterizing structural properties of the value function. Due to the nature of power output of wind turbines, henceforth, we assume that the downtime cost rate $c_d(\ell)$ is discrete concave on its domain $\mathcal{L}$. Furthermore, to simplify notation, let $\delta_n(x_n,\ell):=c_d(\ell)\cdot \tau_n(x_n,\ell)$ for each $n\in \mathcal{N}$. We first examine properties of the immediate cost function, $c(\textbf{x},\ell,\textbf{a})$.
Lemma 3.3. For each $\textbf{a}\in \mathcal{A}$, the immediate cost function $c(\textbf{x},\ell,\textbf{a})$ is
(a) monotone nondecreasing in $(\textbf{x},\ell)\in \Gamma$ and
(b) discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ if $\delta_n(x_n,\ell)$ is discrete concave on $\mathcal{L}$ for each $n\in \mathcal{N}$.
Lemma 3.3(a) asserts that the immediate costs are monotone increasing as the degradation level and environment state increase. That is, it is more costly to operate a system that is more degraded and/or operating in a more detrimental environment. However, Lemma 3.3(b) asserts that the marginal cost increases are diminishing in these two dimensions if $\delta_n(x_n,\ell)$ is discrete concave on its domain. While this assumption may appear restrictive on first glance, it is appropriate in this context due to the nature of power output as a function of wind speed and other relevant conditions. Next, we present structural properties of the value function.
3.1. Value function properties
We now examine important properties of the value function $V(\textbf{x},\ell)$. Proposition 3.5 posits that the monotonicity and discrete concavity of Lemma 3.3 extend to the value function if the transition matrix P possesses the increasing failure rate (IFR) property.
Definition 3.4. Let $\mathbf{P}=[p_{\ell k}]$ be the one-step TPM of a DTMC with state space $\mathcal{L}=\{1,\ldots,L\}$. Then P is said to be IFR if
is nondecreasing in $\ell\in \mathcal{L}$ for each $k\in \mathcal{L}$.
Definition 3.4 implies that, starting from a higher environment state, the DTMC is more likely to transition to even higher states. Next, we state the main result related to $V(\textbf{x},\ell)$.
Proposition 3.5. If P is IFR, then $V(\textbf{x},\ell)$ is
(a) monotone nondecreasing in $(\textbf{x},\ell) \in \Gamma$ and
(b) discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ if $\delta_n(x_n,\ell)$ is discrete concave on $\mathcal{L}$ for each $n\in \mathcal{N}$.
Proposition 3.5(a) asserts that if P is IFR, then the cost starting from state $(\textbf{x},\ell)$ is nondecreasing over the entire state space. That is, for a fixed environment state $\ell$, the cost function is monotone nondecreasing in degradation x, and for a fixed level of degradation x, the cost is monotone nondecreasing in $\ell$. However, the costs do not increase without bound. Proposition 3.5(b) asserts that the marginal costs diminish as the environment state increases, a property that follows directly from the discrete concavity of the downtime cost rate function $c_d(\ell)$ on $\mathcal{L}$.
Naturally, one would like to examine the sensitivity of the value function to changes in the model’s cost parameters. For example, how sensitive is the value function to an increase in the setup cost cs in light of the fact that this substantial cost can be shared by performing group replacements? To answer this question, we employ the notion of comparative statics, as described by Milgrom and Shannon [Reference Milgrom and Shannon43]. Let w be any one of the nonnegative cost parameters cs, cr, or $c_d(\ell)$. The functions $c(\textbf{x},\ell,\textbf{a},w)$ and $V(\textbf{x},\ell,w)$ denote, respectively, the immediate cost and value functions associated with parameter w. Proposition 3.6 asserts monotonicity in all three cost parameters.
Proposition 3.6. If $c(\textbf{x},\ell,\textbf{a},w)$ is monotone nondecreasing in w for each $(\textbf{x},\ell)\in \Gamma$ and $\textbf{a}\in \mathcal{A}$, then $V(\textbf{x},\ell,w)$ is monotone nondecreasing in w.
Consider a case in which the DM can reduce the setup cost by decreasing the cost of preparing the JUV and helicopter. Proposition 3.6 asserts that optimally replacing the turbines incurs less cost in this case, that is, $V(\textbf{x},\ell,c_s)\le V(\textbf{x},\ell,c^{\prime}_s)$ for each $(\textbf{x},\ell)\in \Gamma$ and any $c_s\le c^{\prime}_s$. Next, we provide useful structural properties of the optimal wind farm replacement policy.
3.2. Characterizing the optimal replacement policy
Here, we present the main results of this section by establishing structural properties of the optimal replacement policy for a wind farm. For notational convenience, let
be the set of all turbines for which replacement is optimal in state $(\textbf{x},\ell)$. Similarly, let $\mathcal{N}\setminus \mathcal{V}(\textbf{x},\ell)$ be the set of turbines for which it is optimal to do nothing in state $(\textbf{x},\ell)$.
Theorem 3.7 For each $(\textbf{x},\ell)\in \Gamma$, consider a degradation vector $\textbf{y}\in \mathcal{X}$ such that $y_n\ge x_n$ for $n\in\mathcal{V}(\textbf{x},\ell)$ and $y_n=x_n$ for $n\in\mathcal{N}\setminus \mathcal{V}(\textbf{x},\ell)$. If P is IFR, then $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$.
Theorem 3.7 asserts that, for a given environment state $\ell$, the replacement region extends in each dimension for which it is optimal to replace in degradation level x. The result is elucidated by way of a simple example. Consider a wind farm comprising three turbines (N = 3), and suppose that in state $(\textbf{x},\ell)=(0.5,0.5,0.5,\ell)$, it is optimal to replace turbines 1 and 2 but not to replace turbine 3, that is, $\textbf{a}^*(\textbf{x},\ell)=(1,1,0)$. Figure 4(a) illustrates that these same actions are optimal for any state $(\textbf{y},\ell)$ in the shaded region (extension of the optimal actions in two dimensions).
That is, $\textbf{a}^*(\textbf{y},\ell)=(1,1,0)$ for any degradation level $\textbf{y}\in [0.5,1]\times[0.5,1]\times \{0.5\}$. Figure 4(b) illustrates a different scenario in which the state is again $(\textbf{x},\ell)=(0.5,0.5,0.5,\ell)$, but it is optimal to replace all three turbines, that is, $\textbf{a}^*(\textbf{x},\ell)=(1,1,1)$. For this case, we see that action $(1,1,1)$ is optimal for any degradation level $\textbf{y}\in [0.5,1]\times[0.5,1]\times [0.5,1]$, so the optimal actions extend in three dimensions to states with higher levels of degradation.
Now for a given environment state $\ell$, consider a single turbine $n^{\prime}\in \mathcal{N}$ with degradation level $x_{n^{\prime}}$ and fix the degradation levels of all other turbines $n\in \mathcal{N}\setminus \{n^{\prime}\}$. The following corollary characterizes the behavior of the optimal action $a^*_{n^{\prime}}(\textbf{x},\ell)$ for each environment state.
Corollary 3.8. Suppose that P is IFR and let $n^{\prime}\in \mathcal{N}$. Fix $\ell \in \mathcal{L}$ and $x_n\in [0,1]$ for each $n\in \mathcal{N}\setminus \{n^{\prime}\}$. Then the optimal action $a^*_{n^{\prime}}(\textbf{x},\ell)$ is monotone nondecreasing in $x_{n^{\prime}}$.
As a consequence of economic dependence, it is possible that replacement of turbine n ʹ is not optimal for any level of degradation, that is, a degradation-based threshold policy does not exist. However, if reactive replacement is required whenever unit n ʹ is found to be failed, Corollary 3.8 implies the existence of a degradation threshold $x^*_{n^{\prime}}$ such that
Equation (7) mirrors the optimal replacement policy described in Theorem 2 of [Reference Ulukuş, Kharoufeh and Maillart71] for a single degrading system that is influenced by its random environment; however, their model assumes that an immediate, reactive replacement is performed whenever a failure is revealed. Our model shows that, for some turbines, delaying replacement may be optimal due to the economic dependence induced by the shared setup cost (cs) and the turbines’ stochastic dependence induced by their common environment. That is, the threshold for replacing a turbine may depend not only that turbine’s degradation but also on the degradation levels of the other turbines in the wind farm. This interesting feature of our model is more transparent in the case of N = 2 turbines.
3.3. Special case: N = 2
Structural properties of the optimal replacement policy over the entire space Γ are not immediately obvious when a wind farm contains three or more turbines. Fortunately, Theorem 3.7 can be used to establish the dependence of an optimal replacement threshold of a particular turbine on the degradation of the other turbine in a two-turbine wind farm (N = 2). Before formalizing this interesting result, it is helpful to consider a graphical depiction of the two-turbine case in Figure 5. For $\ell$ fixed, the black circles in Figure 5 represent degradation levels $(x_1,x_2)\in [0,1]\times [0,1]$. From Theorem 3.7, we can deduce the following. If the optimal action at point A is $(0,1)$, then $(0,1)$ remains optimal for any point along the line segment AD. Specifically, for a fixed degradation level x 1, if it is optimal to replace turbine 2 at A, then it is optimal to replace turbine 2 for each point along AD. Similarly, if the optimal action at point B is $(1,0)$ (replace turbine 1 but not turbine 2), then $(1,0)$ remains optimal for any point along the line segment BE. Finally, at point C, the optimal action is $(1,1)$, replace both turbines. Then it is also optimal to replace both turbines for every point contained in the shaded region.
Next, we seek to provide structure to replacement regions over the degradation space $\mathcal{X}$. For a fixed environment state $\ell\in \mathcal{L}$, let $x^*_2(x_1,\ell)$ denote the optimal replacement threshold for turbine 2 when the degradation of turbine 1 is $x_1\in [0,1]$. Similarly, let $x^*_1(x_2,\ell)$ be the optimal replacement threshold for turbine 1 when the degradation of turbine 2 is $x_2\in [0,1]$. When they exist, let
That is, $b_1(x_2,\ell)$ ($b_2(x_1,\ell)$) is the minimum level of turbine 1 (turbine 2) degradation that makes group replacement optimal when the degradation of turbine 2 (turbine 1) is x 2 (x 1). The next proposition asserts that the optimal replacement threshold of one turbine is monotone decreasing in the degradation level of the other turbine beyond these special points $b_1(1,\ell)$ and $b_2(1,\ell)$.
Proposition 3.9. For each $\ell\in \mathcal{L}$, if P is IFR and $\textbf{a}^*(1,1,\ell)=(1,1)$, then
(a) $x^*_2(x_1,\ell)$ is monotone nonincreasing in $x_1\in \{x_1: b_1(1,\ell)\le x_1\le 1\}$ and
(b) $x^*_1(x_2,\ell)$ is monotone nonincreasing in $x_2\in \{x_2: b_2(1,\ell)\le x_2\le 1\}$.
Proposition 3.9 establishes the fact that the replacement threshold of one turbine depends on the degradation state of the other turbine. Figure 6 depicts the region in which it is optimal to replace turbine 2, that is, $\left\{(x_1,x_2):\textbf{a}^*(x_1,x_2,\ell)\in \{(0,1),(1,1)\}\right\}$. The figure illustrates that the replacement threshold for turbine 2 is monotone decreasing in x 1 for any $x_1\ge b_1(1,\ell)$.
Figure 6 also highlights the fact that it is advisable for the DM to decrease the optimal threshold of turbine 2 as turbine 1 reaches higher degradation levels. The results in Proposition 3.9 confirm that the replacement thresholds for both turbines are monotone decreasing. In Section 4, we provide some numerical examples illustrating the effects of economic and stochastic dependence on the optimal turbine replacement policies.
4. Numerical illustrations
Here, we illustrate the structure of optimal replacement policies for a two-turbine wind farm using our MDP model. While our main results are valid for any finite number of turbines, we choose to illustrate the policies for N = 2 since they can be easily visualized in the xy-plane. It is worth noting that, although the state space grows exponentially in the number of turbines N, approximate dynamic programming (ADP) and state-space reduction techniques can be employed to solve larger instances, as was done in [Reference Abdul-Malak and Kharoufeh2]. First, we provide numerical examples demonstrating how these policies differ depending on the magnitude of the setup cost (cs) and the prevailing environment state ($\ell$). Second, we assess the importance of incorporating economic and stochastic dependence when evaluating turbine replacement policies. To that end, we consider simpler models that either ignore these features altogether or use fewer environment states than are needed to adequately characterize the environment. We begin by describing a baseline example, which considers a wind farm with two wind turbines. There exists ample evidence to suggest that 6–8 states are sufficient to model environmental conditions [Reference Anastasiou and Tsekos4]; hence, our baseline example contains six distinct environment states.
4.1. A baseline example
Suppose there are two turbines in an offshore wind farm (N = 2), and the turbines are exposed to an environment whose evolution is modeled by a DTMC with state space $\mathcal{L}=\{1,\ldots,6\}$ and TPM
It can be shown that P possesses the IFR property. For this illustration, it is assumed that, for each turbine $n\in \mathcal{N}$, the DM predicts future degradation using the linear predictive function
where $r_n(\ell)$ is the degradation rate of turbine n in environment state $\ell\in \mathcal{L}$. The form of fn is immaterial as long as it satisfies the nonnegativity and monotonicity assumptions stated in Section 2. It is difficult to obtain proprietary wind farm O&M records to parameterize our model. Therefore, notional setup costs, downtime cost rates, and degradation rates were chosen relative in scale but are not intended to reflect any real data provided to the authors. The linear degradation rates were chosen such that $r_n(i) \lt r_n(j)$ whenever i < j, and their magnitudes are in proportion to the failure threshold ξ = 1. Because the time scale of one period is on the order of 1 week, the rates of degradation are relatively small as compared to the failure threshold of 1. Similarly, the downtime cost rates are selected such that $c_d(i) \lt c_d(j)$ whenever i < j. The setup and replacement costs are $c_s=5$ and $c_r=8$, respectively. Table 1 lists all other notional parameter values.
To implement the VI algorithm of [Reference Puterman54], we discretized the continuous degradation space $\mathcal{X}$ (cf. Kushner and Dupuis [Reference Kushner and Dupuis37]). Specifically, for each $n\in \mathcal{N}=\{1,2\}$, the degradation interval $[0,1]$ was uniformly discretized into 101 states so that $x_n\in \{0.00,0.01,\ldots,0.99,1.00\}$, and we let $\mathcal{X}^{\prime}:=\{0.00,0.01,\ldots,0.99,1.00\}^2$. Then for each $(\textbf{x},\ell)$ in $\mathcal{X}^{\prime}\times \mathcal{L}$, let $v^k(\textbf{x},\ell)$ denote the kth iterate of the VI algorithm with $v^0(\textbf{x},\ell)=0$. The algorithm terminates at iteration k $(k\in \mathbb{N}$) if
where $\epsilon_v \gt 0$ is a tolerance level. For each implementation of the VI algorithm, we set $\epsilon_v = 0.01$ and λ = 0.99. The VI algorithm was coded in Python 3.9 and executed on a personal computer with a 3.40 GHz processor and 8 GB of RAM. Next, we examine the sensitivity of the optimal replacement policy to each environment state and the setup cost.
4.2. Baseline example sensitivity
Starting in state $(x_1,x_2,\ell)=(0,0,1)$, the VI algorithm yields an optimal total discounted cost of 3,044.20 for the baseline example. For a fixed environment state, replacement regions are the regions in which at least one turbine is replaced (shaded regions). The opportunistic region is the region in which both turbines are replaced (dark shaded region). Figure 7 depicts the replacement regions in different environment states.
Figure 7(b)–(f) shows that higher environment states lead to larger replacement regions, as these states imply accelerated degradation and higher power production rates. Hence, in higher environment states, the DM opts to lower the replacement threshold to avoid high production losses caused by turbine failures. However, in Figure 7(a), the largest replacement region is for environment state $\ell=1$ since the production and degradation rates for this state are least among all environment states; hence, the DM prefers to perform the preventive replacements much sooner. Such a scenario may correspond to the overhaul of large, critical components in a wind farm, as major replacements are typically performed in the summer months when environment conditions are milder. Figure 7(a) also illustrates the result in Proposition 3.9—the optimal replacement threshold of one turbine is decreasing in the degradation state of the other turbine when both are in the opportunistic region. However, we also observe nonintuitive behavior for the optimal replacement policy in Figure 7; the optimal replacement threshold of each turbine is not necessarily decreasing over the entire degradation state space. For example, in Figure 7(a), for a fixed degradation state of turbine 2 ($x_2=0.4$), the DM opts to replace turbine 2 at point B but not to replace at points A and C. Thus, by increasing the degradation state of turbine 1, the optimal replacement threshold for turbine 2 decreases first (moving from point A to B) and then increases (moving from point B to C). One possible explanation for this behavior is that, at point C, turbine 2 is closer to the opportunistic region and will reach this region sooner. Thus, it is best for the DM to postpone replacement of turbine 2 until turbine 1 further degrades so that a group replacement can be performed and setup costs can be shared.
In the first instance of the baseline example, the setup cost ($c_s=5$) was relatively low compared to the downtime cost rates. This scenario corresponds to the case when the required resources for replacement are available (e.g., no cost for chartering a JUV and helicopter) and the majority of the setup cost is fueling. For the sake of comparison, we also examined a scenario with a relatively high setup cost corresponding to the case when a helicopter and JUV must be chartered from a spot market, and the total setup cost exceeds the highest capacity of power production for a turbine ($c_s = 80$). Figure 8 shows how the optimal replacement regions shrink with a substantial increase in the setup cost. Figure 8(c)–(f) shows that for some degradation levels, the DM may allow the other turbine to remain failed and postpone replacement. That is, for higher states of degradation, replacing only one turbine is not optimal, and the DM is more inclined to perform group replacement due to the magnitude of the setup cost.
One would expect the opportunistic region to be larger in higher environment states since the DM opts to perform the group replacement sooner to avoid higher power production losses. However, in Figures 7 and 8, the opportunistic region does not necessarily grow with the environment state. There are two reasons for this nonintuitive behavior. First, if the environment state is 1, then the likelihood of transitioning to higher states is low. Hence, it is expected that, in subsequent periods, the environment remains in state 1 with a higher probability. Second, the downtime cost in environment state 1 is low; hence, the DM prefers to replace both turbines in an environment state that leads to a lower production loss. In the next subsection, we show how models with incorrect assumptions impact the optimal replacement policies.
4.3. Value of correct modeling
Here, our aim is to assess the magnitude of cost increases resulting from incorrect model assumptions. The MDP model incorporates economic dependence via the setup cost cs and stochastic dependence via the association of degradation processes to the turbines’ common environment. Here, we compare the expected total discounted costs obtained by two distinct models and compare those with the costs obtained by the model of Section 2. In the first model, the DM simply ignores both types of dependence and assumes that replacement decisions are made for each turbine independently. In the second model, the DM includes dependence but assumes the environment can be described by a simple two-state DTMC, thereby ignoring any intermediate environment states.
Let $\pi^*$ be the optimal replacement policy obtained by the MDP model of Section 2, and let πi be the optimal replacement policy obtained by Model i, for $i=1,2$. Table 2 summarizes how the models differ with regard to their underlying assumptions.
The full model incorporates both economic and stochastic dependence and all six environment states. After obtaining the optimal policy for each model using the VI algorithm, we simulated the respective costs of the modified policies (π 1 and π 2) on the full model using a large number of sample paths. Along each sample path, the total discounted cost was computed, and these values were compared. The simulation run length, which corresponds to the number of decision epochs, is denoted by M ($M\in \mathbb{N})$. Since the expected one-step costs are bounded, and the cost function is discounted, the simulation run length can be determined a priori to ensure that the total discounted cost is accurate to a fixed constant. The simulation run length M was chosen such that $M\ge \left[\ln((1-\lambda)\epsilon_s/C)/\ln(\lambda)\right]-1$, where C is any valid upper bound on the expected one-step costs and λ is the discount factor. For all numerical examples, M was chosen to correspond to $\epsilon_s = 0.01$, λ = 0.99, and $C=c_s+N[c_r+c_d(L)]$. For the baseline example, $M\ge 1428$, so we chose M = 1500.
To compute the average simulated total discounted cost under each policy, we selected the number of replications (R) to achieve a desired margin of error using the well-known relationship
where $z_{\alpha/2}$ is the critical z-value above which lies area $\alpha/2$ under the standard normal density function, $\hat{\sigma}$ is the estimated standard deviation, and Δ is the desired margin of error. For each of the following models, we chose a 95% confidence level (α = 0.05, $z_{\alpha/2}=1.96$) and $\Delta=1$. The simulation models were coded in the MATLAB R2022a computing environment and executed on a personal computer with an 8-core CPU and 8 GB of RAM. We next describe the modified MDP formulations for Models 1 and 2, along with the corresponding numerical results.
Model 1. The first modified model ignores both economic and stochastic dependence; therefore, the problem can be decomposed into two independent, single-turbine problems. For each decomposed problem, the setup and replacement costs are assessed every time a replacement is performed. Although both turbines are assumed to operate in environments governed by the same transition matrix P, their environments (and degradation processes) evolve independently. Thus, for turbine $n\in \{1,2\}$, the state is $(x_n,\ell_n)\in [0,1]\times \mathcal{L}$, $a_n\in \{0,1\}$ is the action, and the immediate cost function is
The Bellman equation for the decomposed MDP model n is
where
Figure 9 illustrates the behavior of the Model 1 optimal replacement policy (π 1) for each environment state. We observe that the optimal replacement threshold of one turbine does not depend on the other’s degradation state in the absence of economic and stochastic dependence. Moreover, the optimal replacement thresholds decrease with the environment state. That is, in environments that induce higher degradation and power production rates, preventive replacements are prescribed at lower levels of degradation for both turbines.
Next, we seek to assess how the Model 1 optimal policy (π 1) performs under the assumptions of the full model (i.e., when economic and stochastic dependence are present). To this end, we apply policy π 1 to a simulation of the full model and compare its resulting simulated total discounted cost. We first performed 5,000 replications of the simulation model (each with $M=1,500$ decision epochs) and obtained a sample standard deviation $\hat{\sigma}\approx256$. By Equation (10), the number of replications needed to ensure $\Delta=1$ is $R=251,763$; hence, we selected $R=252,000$. The policy π 1 was applied in the simulation of the full model, and the simulated total discounted costs for three different initial states are reported in Table 4. The simulated costs are also compared with those obtained using VI on the full model, which includes both economic and stochastic dependence.
Model 2. Like the full model, this second modification incorporates both economic and stochastic dependence (via shared setup costs and a common environment). However, in this case, the DM lacks sufficient information to assess the number of environment states and the degradation rates associated with each state. Recall that the full model assumes that the environment can transition among six distinct states. In Model 2, the DM uses only two states to describe the environment ($\mathcal{L} = \{1,2\}$) and estimates a degradation rate for each state. Table 3 summarizes the downtime cost rates and degradation rates corresponding to each environment state. Note that the values used for states 1 and 2 in Model 2 correspond to those of the extreme states 1 and 6 from Table 1. For this new two-state model, the DM scales the original transition probabilities, p 11, p 16, p 61, and p 66, to construct the new TPM
where, for example, $\hat{p}_{11}$ was obtained by $\hat{p}_{11}=p_{11}/(p_{11}+p_{16})\approx 0.98$. All other components of this MDP model, including the immediate cost function and the Bellman equation, are identical to the MDP model defined in Section 2. Figure 10 depicts the optimal replacement policy (π 2) obtained by VI algorithm for each of the two environment states. To compare the costs of this policy to the full model, it was necessary to create a mapping between each of the six states in the full model to the two states of the modified environment process. Specifically, the replacement policy for $\ell=1$ (Figure 10(a)) was used for states 1, 2, and 3 in the full model. The replacement policy for $\ell=2$ (Figure 10(b)) was used for states 4, 5, and 6 in the full model.
We first performed 5,000 replications of the simulation model (each with $M=1,500$ decision epochs) and obtained a sample standard deviation $\hat{\sigma}\approx 251$. By Equation (10), the number of replications needed to ensure $\Delta=1$ is $R=242,024$; hence, we selected $R=243,000$. As before, we apply policy π 2 to a simulation of the full model and compare its resulting simulated total discounted cost. The simulated total discounted costs for six different initial states are summarized in Table 4 and compared with those obtained using VI on the full model.
Table 4 compares the expected total discounted costs of the full model and the simulated total discounted costs of Models 1 and 2 starting from initial states $(0,0,1)$, $(0,0,2)$, $(0,0,3)$, $(0,0,4)$, $(0,0,5)$, and $(0,0,6)$. This table reveals several interesting results. First, the policies resulting from Model 1, in which both economic and stochastic dependence are ignored, yield average total discounted costs that are just over 8% higher than those generated by the full model, indicating the importance of sharing setup costs and consideration of how the turbines’ degradation processes are related via their shared environment. Second, we note that the policies resulting from Model 2, which includes both types of dependence but fails to adequately characterize the environment process, result in cost increases on the order of 5%. These results provide evidence that accounting for all possible environment scenarios can be important when assessing the lifetime replacement costs of an offshore wind turbine. Finally, we observe a general trend that the total costs are increasing in the initial environment state (assuming both turbines start in an as-good-as-new condition). This observation is in agreement with Proposition 3.5(a).
4.4. Comparisons with heuristic policies
Here, we assess the performance of the optimal replacement policy obtained from our model with three distinct and common heuristic policies:
• Reactive replacement policy. Using this policy, the DM replaces the component reactively, that is, immediately following a failure.
• Age-replacement policy. The classical age-replacement policy prescribes replacement either at failure or at a chosen deterministic age T, whichever occurs first. The replacement age is taken to be the expected lifetime of the unit, which is typically the manufacturer’s mean time-to-failure.
• Fixed interval policy. This policy performs a replacement every T units, whether a replacement is warranted or not. Here, the deterministic inter-replacement time T is taken to be the manufacturer’s mean time-to-failure.
We compared the total costs generated by the MDP policies with those of the heuristic policies starting from six distinct initial states (corresponding to the six environment states). The results in Table 5 show that the total discounted costs obtained by the MDP model are approximately 17% lower than those obtained using reactive replacement, 14% lower than those obtained using the age-replacement policy, and over 40% lower than those obtained using a fixed interval policy. These results highlight the value of using the optimal policy as compared to simple heuristic policies.
5. Conclusions and discussion
We devised and analyzed an MDP model to optimally prescribe the replacement times of critical components in an offshore wind farm. Our model includes both economic and stochastic dependence between the turbines operating in the farm. Economic dependence is induced by significant setup costs that can be shared by performing opportunistic replacements; stochastic dependence exists in the turbines’ degradation processes due to their common exposure to a randomly varying environment. The main results may have significant practical implications for offshore wind farm operators who seek to reduce turbine downtime and ensure sustained power production capacity. We established the monotonicity of actions for each turbine in the farm (fixing the degradation of all other turbines). Most interesting is the fact that the optimal replacement time of one turbine depends not only on its own level of degradation but also on the degradation levels of the other turbines in the farm. Furthermore, beyond a certain point, the replacement thresholds are monotone. We showed that it may be more beneficial to delay replacements and reduce overall costs by performing group replacements when the setup costs can be shared among multiple turbines. This result is important since the cost of chartering or deploying service operations vessels and/or JUVs is significant. Numerical examples illustrated the structure of replacement policies in the case of two turbines and showed that monotone policies emerge beyond certain degradation thresholds for both turbines. We also demonstrated the value of including dependencies and accounting for the influence of the environment. When compared with three different heuristic policies, it was shown that the optimal policy substantially reduces the total discounted costs.
While the insights gained from the MDP model are valuable, it can be further expanded and improved to incorporate additional contextual information (in the form of constraints) related to the maintenance of offshore wind farms. Several factors influence the feasibility of performing maintenance operations, including the wind speed and wave height, availability of specialized resources (vessels and/or helicopters), and availability of crews to perform the work. Specifically, safety limits for wind speed and wave height dictate conditions for which vessels can safely operate in the sea, and when conditions are unsafe, operations are deemed infeasible. To incorporate this realism into our model, one need only to append an additional state—an indicator variable that assumes the value 0 or 1—indicating if weather conditions are suitable for maintenance operations. When conditions are too harsh, the indicator is assigned a value 0 and maintenance cannot be performed. In this case, the only feasible action is “do nothing” and wait until the next decision epoch. Another critical factor affecting the feasibility of maintenance is the availability of specialized resources (e.g., SOVs, CTVs, JUVs, and helicopters) or work crews. Likewise, these constraints can be included in the MDP model by adding indicator variables to the state vector, which indicate whether a resource or crew are available at each decision epoch. However, the inclusion of these additional components in the state vector increases the dimensionality of the problem to the extent that it may become intractable or the structural results might be lost. Another complication is that the number of states grows exponentially in the number of turbines in the wind farm. To deal with these issues, one can consider ADP techniques, along with state–space reduction strategies, as demonstrated by Abdul-Malak and Kharoufeh [Reference Abdul-Malak and Kharoufeh2].
Although the model presented herein focused on the replacement of major components in an offshore wind turbine, it can be modified to prescribe the timing of other maintenance actions (e.g., minor or medium repairs) for components that experience more frequent failures than major components such as generators and gearboxes. The MDP model employs an assumption that replacement takes one period (up to 1 week) to complete and that all replacements are completed before the next decision epoch. Because the average time needed to repair or replace a minor component is typically shorter than that of a major component, the length of time between decision epochs can be chosen to ensure that the maintenance action is completed within one period. For example, minor repairs or replacements can usually be completed within 1 day, whereas major replacements may require 2–7 days to complete. Thus, the time scale of the MDP model can be modified to account for minor repairs or replacements by setting the period length to 1 day. It is important to note that the per-period costs and the degradation function must be modified in this case. It is expected that the opportunistic region for minor or medium repairs will be larger than the opportunistic region for replacing major components.
Finally, in our framework, the DM’s objective is to minimize the expected total discounted costs, which include the setup and replacement costs, as well as the costs associated with turbine downtime. However, if the objective is to maximize expected profit over a planning horizon, it is necessary to modify the MDP model by incorporating pricing and production rate information into the one-step profit. To this end, let $C_n(\ell)\in \mathbb{R_+}$ be the nominal rate of power production for turbine n when the environment state is $\ell$ and ρm ($\rho_m \gt 0$) the price at decision epoch m. It can be assumed that a turbine produces power at its nominal rate when it is operating. The price can be incorporated by expanding the state to $(\textbf{x},\ell,\rho)$ and using pricing information, for example, from regional transmission operators such as PJM (https://www.pjm.com) to estimate the probability that the price transitions from ρm to $\rho_{m+1}$ in one period. By including the dynamic and stochastic price, the DM may consider maximization of expected profits. During each period, revenue is only accrued for a turbine if it is operational; no revenue is included during downtime. For instance, at the beginning of a period, if turbine n is failed $(x_n=1)$ or is scheduled for replacement $(a_n=1)$, there is no earned revenue because the turbine is down for the entire period. However, if turbine n is operational at the start of period m and no replacement is scheduled, the one-step revenue can be calculated as $C_n(\ell)\rho_m\left[1-\tau_n(x_n,\ell)\right]$, where $\tau_n(x_n,\ell)$ is the predicted downtime during that period. Finally, the profit in each period is determined by subtracting the applicable setup and replacement costs (if there is a need for replacement) from the generated revenues.
Acknowledgements
The authors wish to express their gratitude to two anonymous referees and an Editorial Board Member for their insightful comments and helpful suggestions that have improved this work.
Appendix A.
Proof of Lemma 3.1
Proof. For each $t\ge 0$ and $(\textbf{x},\ell),(\textbf{x}^{\prime},\ell^{\prime}) \in \Gamma$ such that $(\textbf{x},\ell) \le (\textbf{x}^{\prime},\ell^{\prime})$, $f_n(x_n,\ell,t)\le f_n(x^{\prime}_n,\ell^{\prime},t)$. Thus, we have
It follows that
and subsequently,
Proof of Lemma 3.3
Proof. For any given $\textbf{x}\le \textbf{x}^{\prime}$, $\ell\le \ell^{\prime}$ and $n\in \mathcal{N}$, if $a_n=0$, by Lemma 3.1, we have
and if $a_n =1$,
since $c_d(\ell)$ is strictly increasing in $\ell$. As Equations (A.1) and (A.2) hold for each $n\in \mathcal{N}$, then for any $\textbf{a} \in \mathcal{A}$,
Lemma 3.3(a) follows immediately. Now, for $\textbf{a}\in \mathcal{A}$ and $\textbf{x}\in \mathcal{X}$, $c(\textbf{x},\ell,\textbf{a})$ is a linear function of $c_d(\ell)$ and $\delta_n(x_n,\ell):=c_d(\ell)\tau_n(x_n(\ell)$, which are discrete concave on $\mathcal{L}$. Thus, Lemma 3.3(b) follows as well.
Proof of Proposition 3.5
Proof. For $(\textbf{x},\ell) \in \Gamma$, denote the mth iterate of the VI algorithm by $v^m(\textbf{x},\ell)$. We prove Proposition 3.5(a) by induction on m. Take $v^0(\textbf{x},\ell) = 0$ for all $(\textbf{x},\ell) \in \Gamma$. Therefore, for any pairs of $(\textbf{x},\ell),(\textbf{x}^{\prime},\ell^{\prime}) \in \Gamma$ such that $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$, by Lemma 3.3, we have
since $c(\textbf{x},\ell,\textbf{a})$ is monotone nondecreasing in $(\textbf{x},\ell) \in \Gamma$ for each $\textbf{a} \in \mathcal{A}$. For the induction hypothesis, assume $v^m(\textbf{x},\ell) \leq v^m(\textbf{x}^{\prime},\ell^{\prime})$. From Equation (1), $\hat{\textbf{x}}(\ell,\textbf{a}) \leq \hat{\textbf{x}}^{\prime}(\ell^{\prime},\textbf{a})$ for any given $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$ and $\textbf{a} \in \mathcal{A}$. Since P is IFR, by Lemma 4.7 of [Reference Puterman54], we have
for any given $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$ and $\textbf{a} \in \mathcal{A}$. By Lemma 3.3, we add $c(\textbf{x},\ell,\textbf{a})$ and $c(\textbf{x}^{\prime},\ell^{\prime},\textbf{a})$ to the left-hand side and right-hand side of the above inequality, respectively, to obtain
Minimizing both sides over $\textbf{a} \in \mathcal{A}$ results in $v^{m+1}(\textbf{x},\ell) \leq v^{m+1}(\textbf{x}^{\prime},\ell^{\prime})$. By Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell)\to V(\textbf{x},\ell)$, as $m\to\infty$, and Proposition 3.5(a) follows. Similarly, we prove Proposition 3.5(b) by induction on m. Take $v^0(\textbf{x},\ell) = 0$ for all $(\textbf{x},\ell) \in \Gamma$. Therefore, for each $(\textbf{x},\ell) \in \Gamma$, we have
Inequalities (A.3) and (A.4) hold by Lemma 3.3(b) and the fact that the min operator preserves the discrete concavity property. For the induction hypothesis, assume $v^m(\textbf{x},\ell)$ is discrete concave on $\mathcal{L}$ for each $\textbf{x} \in \mathcal{X}$. Define $Q(\textbf{x},\ell,\textbf{a})$ as the cost-to-go function in state $(\textbf{x},\ell)$ under action a, that is,
Then, for $(\textbf{x},\ell)\in \Gamma$, let $Q^m(\textbf{x},\ell,\textbf{a})$ denote the cost-to-go function with respect to $\textbf{a}\in \mathcal{A}$ and the mth iterate of the VI algorithm. That is,
Since $Q^m(\textbf{x},\ell,\textbf{a})$ is the sum of two discrete concave functions, it is discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ and $\textbf{a}\in \mathcal{A}$. Then, for each $\textbf{x}\in \mathcal{X}$, we have
Inequality (A.6) holds since the min operator preserves the discrete concavity property. By Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell)\to V(\textbf{x},\ell)$ as $m\to\infty$, and the proof is complete.
Proof of Proposition 3.6
Proof. For any given $(\textbf{x},\ell) \in \Gamma$, denote the mth iterate of the VI algorithm associated with parameter w by $v^m(\textbf{x},\ell,w)$. We prove the proposition by induction on m. Take $v^0(\textbf{x},\ell,w) = 0$ for all w and $(\textbf{x},\ell) \in \Gamma$. For each $(\textbf{x},\ell) \in \Gamma$ and any $w \leq w^{\prime}$, we have
since $c(\textbf{x},\ell,\textbf{a},w)$ is in monotone nondecreasing in w for each $ (\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$. For the induction hypothesis, assume $v^m(\textbf{x},\ell,w) \leq v^m(\textbf{x},\ell,w^{\prime})$. Then, for each $(\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$, we have
Minimizing both sides over $\textbf{a} \in \mathcal{A}$ results in $v^{m+1}(\textbf{x},\ell,w) \leq v^{m+1}(\textbf{x},\ell,w^{\prime})$. Finally, by Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell,w)\to V(\textbf{x},\ell,w)$, as $m\to\infty$, and the proof is complete.
Proof of Theorem 3.7
Proof. For a given $(\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$, we define the following sets:
For example, the set $\mathcal{I}(\textbf{x},\ell,\textbf{a})$ is the collection of all turbines for which the corresponding action is replacement in both $\textbf{a}^*(\textbf{x},\ell)$ and a. For notational convenience, we simply write $\mathcal{I}$, $\mathcal{J}$, $\mathcal{G}$, and $\mathcal{H}$ and suppress the dependence of these sets on $(\textbf{x},\ell,\textbf{a})$. Note that, by the definitions (6) and (A.5),
Next, we show that $c(\textbf{y},\ell,\textbf{a})-c(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell))\ge c(\textbf{x},\ell,\textbf{a})-c(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$.
Equality (A.8) holds by the definitions of y and $\mathcal{G}$, and inequality (A.9) holds by Lemma 3.1. Moreover, ${\hat{\textbf{y}}}(\ell,\textbf{a}^*(\textbf{x},\ell)) = \hat{\textbf{x}}(\ell,\textbf{a}^*(\textbf{x},\ell))$ and ${\hat{\textbf{y}}}(\ell,\textbf{a}) \geq \hat{\textbf{x}}(\ell,\textbf{a})$ since for all $n \in \mathcal{N}$
where $f_n(x_n,\ell),\ n\in \mathcal{N}$ is assumed to be monotone increasing in xn. Subsequently, we have
and by Proposition 3.5(a), we have
since ${\hat{\textbf{y}}}(\ell,\textbf{a}) \geq \hat{\textbf{x}}(\ell,\textbf{a})$. Thus, for any given a, we have
Inequality (A.12) holds by Equations (A.9), (A.10), and (A.11). From the above argument, we conclude
which implies
Furthermore, by Definition (6),
Therefore, by Equations (A.14) and (A.15), we have
Equality (A.16) implies that, if the degradation state is y, taking the optimal action associated with x leads to the minimum cost. In order to show that $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$, it is sufficient to prove that there does not exist any $\textbf{a}^{\prime}\in \mathcal{A}$ such that
We will prove this assertion by contradiction. Suppose there exists an $\textbf{a}^{\prime}\in \mathcal{A}$ satisfying Equation (A.17), and let us compare $Q(\textbf{x},\ell,\textbf{a}^{\prime})$ and $Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$ under two distinct cases:
Case 1: Suppose $Q(\textbf{x},\ell,\textbf{a}^{\prime}) \leq Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$. In this case, by Definition (6), we conclude $\textbf{a}^*(\textbf{x},\ell) \leq \textbf{a}^{\prime}$, which contradicts the inequality in Equation (A.17).
Case 2: Suppose $Q(\textbf{x},\ell,\textbf{a}^{\prime}) \gt Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$. Similar to the arguments above, for the given $\textbf{a}^{\prime} \lt \textbf{a}^*(\textbf{x},\ell)$, by inequality (A.13), we see that
That is, $Q(\textbf{y},\ell,\textbf{a}^{\prime}) \gt Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell))$, which contradicts the equality in Equation (A.17). As a contradiction occurs in both cases, we conclude that there is no $\textbf{a}^{\prime}\in \mathcal{A}$ satisfying the conditions in Equation (A.17); therefore, $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$.
Proof of Corollary 3.8
Proof. For a given $n^{\prime} \in \mathcal{N}$, if $a^*_{n'}(\textbf{x},\ell) = 1$, by Theorem 3.7, $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$ for a y such that $y_{n^{\prime}}\geq x_{n^{\prime}}$ and $y_n=x_n$ for $n \in \mathcal{N} \setminus \{n^{\prime}\}$, that is, $a^*_{n^{\prime}}(\textbf{y},\ell) = 1$. If $a^*_{n^{\prime}}(\textbf{x},\ell) = 0$, then $a^*_{n^{\prime}}(\textbf{y},\ell) \in \{0,1\}$ for any $\textbf{y} \geq \textbf{x}$. Therefore, $a^*_{n^{\prime}}(\textbf{x},\ell)$ is monotone nondecreasing in $x_{n^{\prime}}$ for each xn, $n \in \mathcal{N} \setminus \{n^{\prime}\}$ and $\ell$.
Proof of Proposition 3.9
Proof. For a given $\ell \in \mathcal{L}$, $\textbf{a}^*(1,1,\ell)=(1,1)$, thus $b_1(1,\ell)$ exists, and we have
and by Definition (8)
By Corollary 3.8 and Equation (A.18), the replacement threshold $x^*_2(x_1,\ell)$ exists on the degradation space $[b_1(1,\ell),1] \times [0,1]$. For a given $(x_1,x_2)$ in the degradation space $[0,b_1(1,\ell))\times [0,1)$, if $\textbf{a}^*(x_1,x_2,\ell)=(1,1)$, then by Theorem 3.7, $\textbf{a}^*(x_1,1,\ell)=(1,1)$, which contradicts Equation (A.19). Hence, there is no point in space $[0,b_1(1,\ell)) \times [0,1)$, for which the optimal action is to replace both turbines, that is,
By Theorem 3.7 and Equation (A.18), we have
That is,
By contradiction, assume that $x^*_2(x_1,\ell)$ is strictly increasing in x 1. Then, there must exist a point $(x_1,x_2) \in [b_1(1,\ell),1) \times [0,1]$ for which $a^*_2(x_1,x_2)=1$, and
for any ϵ > 0. If $a^*_2(x_1,x_2)=1$, then by Equation (A.20), $\textbf{a}^*(x_1,x_2)=(1,1)$, and by Theorem 3.7, $\textbf{a}^*(x_1+\epsilon,x_2)=(1,1)$, which contradicts Equation (A.21).