Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-27T17:26:31.190Z Has data issue: false hasContentIssue false

Structured Replacement Policies for Offshore Wind Turbines

Published online by Cambridge University Press:  02 October 2023

Morteza Soltani
Affiliation:
Department of Industrial Engineering, Clemson University, Clemson, SC, USA
Jeffrey P. Kharoufeh*
Affiliation:
Department of Industrial Engineering, Clemson University, Clemson, SC, USA
Amin Khademi
Affiliation:
Department of Industrial Engineering, Clemson University, Clemson, SC, USA
*
Corresponding author: Jeffrey P. Kharoufeh; Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of optimally maintaining an offshore wind farm in which major components progressively degrade over time due to normal usage and exposure to a randomly varying environment. The turbines exhibit both economic and stochastic dependence due to shared maintenance setup costs and their common environment. Our aim is to identify optimal replacement policies that minimize the expected total discounted setup, replacement, and lost power production costs over an infinite horizon. The problem is formulated using a Markov decision process (MDP) model from which we establish monotonicity of the cost function jointly in the degradation level and environment state and characterize the structure of the optimal replacement policy. For the special case of a two-turbine farm, we prove that the replacement threshold of one turbine depends not only on its own state of degradation but also on the state of degradation of the other turbine in the farm. This result yields a complete characterization of the replacement policy of both turbines by a monotone curve. The policies characterized herein can be used to optimally prescribe timely replacements of major components and suggest when it is most beneficial to share costly maintenance resources.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press.

1. Introduction

Global demand for clean and renewable energy sources, such as wind and solar power, is motivated by growing concerns about environmental sustainability and climate change. Like many other industrialized nations, the United States has committed to reducing harmful carbon emissions while increasing the proportion of power generated by non-fossil-based sources over the next few decades. Specifically, it plans to (1) cut carbon emissions in half by 2030, (2) reach 100% clean electricity by 2035, and (3) achieve net-zero carbon emissions by 2050 [20]. However, these ambitious goals can only be attained if the supply and reliability of renewable energy sources are sufficient to meet consumer demand while remaining economically competitive with fossil fuel sources. Some in the operations community have considered optimal investment strategies for renewable energy technologies, such as wind and solar [Reference Aflaki and Netessine3, Reference Kök, Shang and Yücel30]. Wind energy is playing an increasingly important role in the move towards more sustainable forms of energy because it is clean, accessible, affordable, and inexhaustible. Wind energy is produced by a wind turbine—a system designed to convert kinetic energy (wind) into mechanical energy to generate electricity. Wind turbines are typically comprises a structural tower, the rotor and hub (including three turbine blades), and the nacelle, which houses most of the mechanical and electrical systems, including the gear box and generator, among others. A collection of wind turbines operating in proximity to one another is known as a wind farm.

Although wind farms can be land-based (onshore) or sea-based (offshore), the vast majority of wind generation in the USA originates from onshore systems [Reference Musial, Spitsen, Duffy, Beiter, Marquis, Hammond and Shields47]. New onshore wind farm installations are very costly, as they require significant capital investments, supporting infrastructure and connection services, and these costs are substantially higher for offshore wind farms [Reference Papadopoulos, Coit and Ezzat52]. However, due to escalating demand for wind energy and depletion of available land resources, offshore wind has received a great deal of attention over the past two decades. Though it is more costly, offshore wind holds a few advantages over its onshore counterpart. First, stronger and more consistent wind resources are available offshore, thereby increasing generating capacity and improving reliability. Second, offshore wind farms are usually located in vast open water areas, allowing for large-scale installations. Finally, because offshore wind farms are distant from dense population centers, their negative visual and auditory impacts are significantly reduced. Despite these advantages, wide-scale adoption of offshore wind has not yet been realized due to significant capital investment costs and its levelized cost of energy (LCOE)—an aggregate economic measure that is used to compare lifetime power generation costs associated with different technologies [Reference Short, Packey and Holt65]. The three components of this aggregate cost are broadly categorized as capital expenditures, operational expenditures, including operations and maintenance (O&M) costs, and decommissioning expenditures. O&M costs typically account for 20–50% and 35–50% of the cost of producing onshore and offshore wind energy, respectively [Reference Stehly, Beiter and Duffy67, Reference Wiser and Bolinger74]. Moreover, the replacement of large, critical components such as turbine blades, gear boxes, and generators constitutes as much as 75% of total O&M costs [Reference Maples, Saur, Hand, van de Pietermen and Obdam40, Reference uit het Broek, Veldman, Fazi and Greijdanus70]. Therefore, the viability of large-scale offshore wind generation hinges on a reduction of O&M costs and, ultimately, the LCOE.

While similarities exist between the maintenance of onshore and offshore wind farms, offshore farms are especially difficult to maintain due to accessibility issues, environmental conditions (e.g., wind speed and wave height), safety concerns, and the need for costly, specialized crews and equipment to perform operations. Accessibility is usually associated with their distance from the shore, sea conditions, and availability of sea vessels. When unanticipated failures occur, helicopters are often deployed to reduce delays in accessing the turbines. Another unique feature of offshore wind farm maintenance is the need for different types of vessels to carry crews, replacement parts, and equipment. In addition to crew transfer vessels (CTVs) and service operational vessels (SOVs), a jack-up vessel (JUV) is essential for replacement of large, critical components such as turbine blades and generators [Reference uit het Broek, Veldman, Fazi and Greijdanus70]. Because it may be cost-prohibitive for a wind farm operator to own such resources, they must be chartered from external providers. Dalgic et al. [Reference Dalgic, Lazakis, Dinwoodie, McMillan and Revie15Reference Dalgic, Lazakis, Turan and Judah17] have documented several complicating factors associated with the use of JUVs. The accessibility and availability issues associated with offshore wind farm maintenance are aptly detailed in [Reference Hofmann25, Reference Papadopoulos, Coit and Ezzat52, Reference Ren, Verma, Li, Teuwen and Jiang57, Reference Rinaldi, Thies and Johanning58, Reference Seyr and Muskulus61, Reference Shafiee62].

These considerations motivate our work here in which we seek to prescribe optimal replacement policies for offshore wind farms that minimize the long-run expected total discounted setup, replacement, and lost production costs. Critical to our approach is the inclusion of both economic and stochastic dependence between the turbines in a wind farm. Economic dependence is present because significant setup costs are incurred any time replacements are performed. These costs can be shared among multiple turbines, allowing operators to exploit economies of scale. Stochastic dependence refers to the association of the turbines’ degradation processes stemming from their common operation in a randomly varying, exogenous environment. The repair or replacement of turbine components can be either preventive (occurring before a failure) or reactive (occurring after a failure); in either case, a per-replacement cost is incurred. Substantial downtime costs (in the form of lost revenue) are also incurred whenever a major component fails and/or a turbine is taken offline to perform replacements. We formulate and analyze a Markov decision process (MDP) model to prescribe optimal replacement policies and provide practical insights to wind farm operators.

1.1. Relevant literature

An offshore wind farm can be viewed as a collection of complex, progressively degrading multiunit systems operating in a dynamic environment. A multiunit system can refer either to one system with multiple (dependent or independent) components or a collection of systems that are dependent in some way; we adopt the latter view for offshore wind farms. Many other researchers embrace the first view in order to devise tractable maintenance policies for multiple independent units. For instance, Tian and Liao [Reference Tian and Liao68] examined policies for general multiunit systems with several identical, independent units and numerically obtained dual threshold-type policies. Zhang et al. [Reference Zhang, Wu, Li and Lee80] decomposed a multiunit system and established an opportunistic dual-threshold policy for the maintenance of each single-unit system. Arts and Basten [Reference Arts and Basten7] proposed a condition-based maintenance (CBM) model for each component of a two-component system independently by considering both scheduled and unscheduled downtime. After prescribing optimal maintenance policies for each component, they synchronized scheduled downtime in order to maintain both units simultaneously. Such decomposition procedures usually lead to tractable maintenance policies; however, it is well-known that interdependent, degrading systems present significant challenges to strategic maintenance planning [Reference Dekker, Wildeman and van der Duyn Schouten19, Reference Ohashi and Nishida49, Reference Wang, Hou, Qin, Peng and Zhou73, Reference Zhu, Xiang and Zeng83], especially when multiple types of dependence exist. Olde Keizer et al. [Reference Olde Keizer, Flapper and Teunter50] surveyed several dependence types prevalent in CBM strategies for degrading systems. Here, we focus primarily on economic and stochastic dependence.

First, economic dependence exists in multiunit systems generally, and in offshore wind farms specifically, because maintenance operations are typically performed jointly on several units to share setup costs and reduce the total downtime overall. Ko and Byon [Reference Ko and Byon34] considered the problem of maintaining a large number of homogeneous units that degrade independently over time but are economically dependent due to shared setup costs. Salari and Makis [Reference Salari and Makis59] examined economic dependence between the units of a production system in which the production rate of each unit decreases as the degradation level increases. Zhu et al. [Reference Zhu, Peng and van Houtum81] considered a multicomponent system with heterogeneous, independently degrading units and significant shared maintenance setup costs. Zhu et al. [Reference Zhu and Xiang82, Reference Zhu, Xiang and Zeng83] formulated a multistage stochastic integer program to optimize the maintenance of a multicomponent system subject to economic dependence. They derived structural results for a simpler two-stage version of the model to enhance tractability. Related to economic dependence is resource-based dependence, in which units are linked through a shared and limited set of resources [Reference Abbou and Makis1, Reference Kurt and Kharoufeh36, Reference Shafiee and Sørensen63]. Olde Keizer et al. [Reference Olde Keizer, Teunter and Veldman51] used a MDP model to jointly optimize maintenance activities and spare parts inventory of a multicomponent system in which the components share identical spares. Maintenance is feasible only if spares are available, thereby inducing resource-based dependence. They obtained optimal replacement and ordering policies that minimize the total long-run average cost per time unit. Similarly, de Pater and Mitici [Reference de Pater and Mitici18] considered multiple repairable components whose maintenance activities are linked via the availability of spares. They integrated remaining useful life (RUL) estimation with a limited inventory of spares to devise a predictive maintenance model subject to resource-based dependence. Wind turbines may be forced to sit idle awaiting the resources needed to restore them to working condition, resulting in economic losses in the form of lost revenues.

Second, stochastic dependence can exist between the units in a multiunit system in at least three ways: via load sharing, dependence of degradation processes due to structural dependencies or the association of degradation processes via a common operating environment. The first type exists when the failure or degradation of one unit influences the lifetimes of other units. For example, if additional power production is required of one wind turbine due to the failure of another, load sharing can lead to accelerated degradation and premature failure of the operational unit [Reference Berrade, Scarf and Cavalcante9, Reference uit het Broek, Teunter and Veldman69, Reference Zhang, Li and Coit76, Reference Zhang, Fouladirad and Barros78, Reference Zhang, Wu, Li and Lee80]. Structural dependence refers to the physical configuration of components within the system, for example, the components are connected in series, parallel, or some other manner [Reference Andersen, Andersen, Kulahci and Nielsen5, Reference Liu, Pandey, Wang and Zhao38, Reference Yousefi, Coit and Zhu75, Reference Zhang, Fouladirad, Barros and Zhang79]. Third, stochastic dependence can be induced by exogenous factors, such as environmental conditions, that simultaneously influence the degradation processes of individual units. Reliability and availability indices, as well as optimal maintenance strategies, for single-unit systems operating in a randomly evolving environment have been well studied [Reference Kharoufeh and Cox31Reference Kharoufeh and Mixon33, Reference Kurt and Kharoufeh35, Reference Zhang, Deng, Liu and Zhang77]. Random shock processes have often been used to model the operating environment and its impact on competing degradation processes [Reference Cha, Finkelstein and Levitin14, Reference Jiang, Feng and Coit28, Reference Rafiee, Feng and Coit55, Reference Rafiee, Feng and Coit56, Reference Yousefi, Coit and Zhu75]. Of particular interest to our work here is a model by Ulukuş et al. [Reference Ulukuş, Kharoufeh and Maillart71], who examined the optimal replacement of a single-unit system subject to environment-driven degradation. Using an MDP model, they established the existence of an optimal degradation-based threshold replacement policy for each environment state. Furthermore, they conjectured conditions for which the thresholds are monotone; however, this conjecture was based only on empirical evidence. Abdul-Malak et al. [Reference Abdul-Malak and Kharoufeh2] extended the model in [Reference Ulukuş, Kharoufeh and Maillart71] to the case of multiple units operating in a shared environment, where stochastic dependence stems from the association of degradation processes via the common environment. They established monotonicity of the value function over the entire state space, although they did not characterize the optimal policy. Instead, numerical approximations were used to illustrate the replacement policies, and for the special case of a single unit, they provided a partial resolution to the conjecture in [Reference Ulukuş, Kharoufeh and Maillart71] and proved conditions needed to ensure that the replacement thresholds are monotone (for each environment state). Our work differs from prior models in that we seek to establish structured replacement policies that consider the following: (1) economic dependence due to shared setup costs, (2) stochastic dependence induced by a common, randomly evolving environment, and (3) the impact of production losses that also depend on the environment state.

1.2. Summary of contributions and results

Our main contributions and results are summarized as follows:

  1. (1) The problem we study has a unique feature: The turbines in the farm are exposed to a common environment that evolves randomly over time. In particular, the degradation processes of the turbines are associated with each other through their common exposure to the environment. Therefore, in addition to economic dependence induced by the sharing of setup costs, there is stochastic dependence among the turbines. We create and analyze an MDP model that incorporates economic and stochastic dependence, both of which significantly influence maintenance strategies.

  2. (2) We establish structural properties that lead to a characterization of the optimal replacement policy. First, we show that insights from the analysis of single-unit systems carry over to our setting. Specifically, for a single turbine, the optimal policy is monotone in its degradation level when the degradation levels of all other turbines are fixed, as seen in Figure 1(a). For a fixed turbine 1 degradation level (level A), there is a turbine 2 degradation threshold (level B) below which it is not optimal to replace and above which it is optimal to replace. Other models assume that it is always optimal to replace a failed system, implying that a replacement threshold always exists. However, in our setting, it is possible that replacement of a failed turbine is not prescribed due to significant shared setup costs. Second, for the special case of two turbines, we provide a novel result that, for a given level of degradation of one turbine, the optimal policy is monotone in the degradation level of the other turbine beyond a certain point. Referring to Figure 1(b), for a fixed turbine 2 degradation level (level D), beyond the point X, there is a threshold degradation level of turbine 1 (level C) below which it is not optimal to replace turbine 2, but above which it is optimal. Similar to the prior case, it may not be optimal to replace turbine 2 when turbine 1 is failed. Third, we show that these thresholds (if they exist) are monotone in the degradation level of turbine 2. That is, in Figure 1(c), when the degradation level of turbine 2 is level G (level H), the optimal threshold for replacing turbine 2 is level F (level E) over the degradation level of turbine 1. Our results show that level E is necessarily less than or equal to level F. As a byproduct, the region where the optimal policy replaces both turbines (the opportunistic region) is completely characterized by a monotone curve (depicted by the dotted curve in Figure 6) in the space of degradation levels of the two turbines.

  3. (3) Our numerical results provide a procedure to assess the value of incorporating dependence and adequately modeling the environment, as well as the consequences of incorrect assumptions on the replacement policies and costs. Our results provide insights as to how the replacement policies depend on the environment and underlying assumptions.

The remainder of the paper is organized as follows. In Section 2, we describe the degradation model and formulate the replacement problem using an MDP model. Section 3 establishes properties of the cost function and the structure of the optimal replacement policy. In Section 4, numerical examples are used to illustrate replacement policies and demonstrate the value of incorporating dependence and the environment. Finally, some concluding remarks are provided in Section 5.

Figure 1. Visualization of the optimal replacement policy of turbine 2. (a) Monotone policy in its own degradation level. (b) Monotone policy in the other turbine’s degradation level. (c) Monotonicity of thresholds.

2. Model and problem formulation

In this section, we first provide a mathematical model of wind turbine degradation due to normal operation and the influence of a randomly evolving environment. Subsequently, we formulate a stochastic optimization problem to minimize the expected total discounted setup, replacement, and downtime costs over an infinite horizon.

2.1. Degradation model

Consider a finite collection $\mathcal{N}:=\{1,\ldots,N\}$ of N wind turbines operating in an offshore wind farm. Critical wind turbine components (e.g., generators, transformers, rotors, and turbine blades) degrade over time due to normal usage and the influence of their ambient operating environment. For example, it is well known that ocean air is corrosive to wind turbine blades due to its higher salt content [Reference Dong, Yuan and Ma22]. In what follows, the phrase “turbine degradation” refers to the degradation of a single, critical component whose replacement requires substantial time, effort, and resources. That is, we do not specify which component but refer more generally to the turbine’s degradation. Each turbine degrades until its cumulative level of degradation first reaches, or exceeds, a critical deterministic threshold ξ (ξ > 0), at which time the turbine is said to be failed. In the problem formulation that follows, we consider discrete decision epochs $m\in \mathbb{Z}_+:=\{0,1,2,\ldots\}$ and define a period as the time between two consecutive decision epochs. Let $X_n(m)$ denote the degradation of turbine $n\in\mathcal{N}$ at decision epoch m with state space $[0,\xi]$. Here, level 0 means a turbine is in as-good-as-new condition. Next, define the vector-valued process $\mathbb{X}=\{\left(X_1(m),\ldots,X_N(m)\right):m\in \mathbb{Z}_+\}$ describing the joint degradation process of all N turbines with continuous state space $\mathcal{X}:=[0,\xi]^N$. At decision epoch m, turbine n is said to be working if $X_n(m)\in [0,\xi)$ and failed if $X_n(m)\ge\xi$. Hereafter, and without loss of generality, we set the failure threshold ξ = 1. Stochastic processes are used extensively in the reliability and maintenance optimization communities to model the temporal and/or spatial evolution of degradation in systems. For a single system, $\{X_n(m):m\in \mathbb{Z}_+\}$ is often assumed to evolve as a discrete-time Markov chain (DTMC) on a finite state space, or when a continuous-time, continuous-state process is more appropriate, degradation can be modeled using gamma, Wiener, or other Lévy processes [Reference Andrawus, Watson and Kishk6, Reference Besnard and Bertling10, Reference Liu, Pandey, Wang and Zhao38, Reference Nielsen and Sørensen48].

For our purposes here, it is appropriate to model not only degradation but also the influence of a randomly varying environment on the evolution of degradation. Wind turbines are often located in proximity to one another within a wind farm; thus, they experience not only similar environmental conditions but also wake effects resulting from changes in wind speed [Reference Ding21]. Although the turbines operate independently, their degradation processes exhibit stochastic dependence due to their exposure to a common environment. Monbet et al. [Reference Monbet, Ailliot and Prevosto44] surveyed stochastic models—many of which are Markovian—that can be used to model wind conditions and the sea state from time series data. Although the environment realistically evolves continuously in time and space, researchers have assumed that it evolves over finitely many discrete states. Representative examples include [Reference Abdul-Malak and Kharoufeh2, Reference Bian, Gebraeel and Kharoufeh11, Reference Kharoufeh and Cox31, Reference Kharoufeh, Finkelstein and Mixon32, Reference Ulukuş, Kharoufeh and Maillart71]. In those specific models, each environment state maps to a unique, positive degradation rate. That is, if the environment occupies state i, then the system degrades linearly at a rate $r(i) \gt 0$. Consequently, in the absence of any type of maintenance intervention, the degradation process evolves as a piecewise linear function of time (almost surely). In contrast to those models, our main theoretical results do not impose such restrictive assumptions but only require that future degradation can be predicted using the current level of degradation and the environment state. The discrete environment states are essentially aggregations of several factors, including wind speed, ambient temperature, relative humidity, wave height, and others. Hence, these states can be viewed as approximations of the true (continuous) environment states over time. For example, if the environment evolves as a diffusion model with a randomly evolving drift coefficient, the distinct environment states can be viewed as the average behavior of the diffusion process within an interval of time. Flory et al. [Reference Flory, Kharoufeh and Gebraeel23] employed Markov chain Monte Carlo methods to model such behavior.

Let J(m) denote the state of the environment at decision epoch m and assume $\mathbb{J}:=\{J(m):m\in\mathbb{Z}_+\}$ evolves as a temporally homogeneous DTMC on the finite state space $\mathcal{L}:=\{1,\ldots,L\}$ with $L\in \mathbb{N}$. The environment states are ordered such that state 1 represents the most favorable condition for maintenance activities and state L represents the least favorable condition. To account for accessibility, resource availability, and maintenance feasibility issues, we can append $\mathcal{L}$ with additional states that reflect conditions under which maintenance cannot be performed (e.g., when wind speeds and/or wave heights are excessive, transport vessels are unavailable and/or crews are unavailable). In such cases, the only feasible action is “do nothing” and wait until the next decision epoch. The one-step transition probability matrix (TPM) of $\mathbb{J}$ is denoted by $\mathbf{P}=[p_{\ell k}]$, where $p_{\ell k}$ is the conditional probability that the environment state is k at decision epoch m + 1, given that it was $\ell$ at epoch m for any $m\in \mathbb{Z}_+$. The number of distinct environment states (L) can be estimated using the well-known Bayesian information criterion [Reference Flory, Kharoufeh and Gebraeel23, Reference Schwarz60]. Anastasiou and Tsekos [Reference Anastasiou and Tsekos4] concluded that six-to-eight distinct states are sufficient to model environmental conditions using a first-order, stationary Markov chain and noted that the stationarity assumption has been employed to model wind speed and wave height in other maritime applications [Reference Bern and Houmb8, Reference MacCallum and Bolland39]. It is well-known that weather conditions exhibit strong seasonality effects, as highlighted and modeled by Byon et al. [Reference Byon and Ding12, Reference Byon, Ntaimo and Ding13]. To account for these effects, one can employ a different DTMC model for different seasons of the year. The transition probabilities can be estimated using proportions data [Reference Jones29, Reference McGuire41] or by mining time series data [Reference Song, Geng, Kusiak and Xu66]. A realization of the joint process $(\mathbb{X},\mathbb{J})$ is an $(N+1)$-dimensional vector $(\textbf{x},\ell)$, where $\textbf{x}=(x_1,\ldots,x_N)\in\mathcal{X}$ and $\ell\in \mathcal{L}$.

We assume that future degradation can be predicted via the current state $(\textbf{x},\ell)$. For each turbine $n\in \mathcal{N}$ and a triplet $(x_n,\ell,t)$, let $f_n:[0,1]\times \mathcal{L} \times \mathbb{R}_{+}\to \mathbb{R}_{+}$ be a generic mapping that predicts the degradation of turbine n after t time units when its current degradation level is xn and the environment state is $\ell$. It can assume any number of functional forms (e.g., linear, piecewise linear, exponential, etc.); however, we impose no such assumption. This mapping satisfies the initial condition $f_n(x_n,\ell,0)=x_n$, and we impose the natural assumption that, for each $n\in \mathcal{N}$, fn is monotone nondecreasing in $(x_n,\ell,t)$, i.e., $f_n(x_n,\ell,t)\le f_n(x^{\prime}_n,\ell^{\prime},t^{\prime})$ for any $x_n\le x^{\prime}_n$, $\ell\le \ell^{\prime}$, and $t\le t^{\prime}$. That is, future degradation is assumed to be higher than the present level of degradation, and because the state space $\mathcal{L}$ is ordered, higher environment states lead to accelerated degradation of the units; however, we impose no additional assumptions on fn. Condition monitoring or sensor data (e.g., from a Supervisory Control and Data Acquisition (SCADA) system) can be used to estimate the evolution of degradation and/or the RUL of components using the techniques developed in [Reference Bian, Gebraeel and Kharoufeh11, Reference Gebraeel, Lawley, Li and Ryan24], for example. Papadopoulos et al. [Reference Papadopoulos, Coit and Ezzat53] suggest that when a dynamic environment is considered, the RUL can be estimated using the models developed by Bian et al. [Reference Bian, Gebraeel and Kharoufeh11].

2.2. Problem formulation

Maples et al. [Reference Maples, Saur, Hand, van de Pietermen and Obdam40] note that the majority of wind turbine O&M costs are due to the replacement of critical components (e.g., generators, transformers, rotors, and turbine blades). Thus, optimally timed replacements that exploit economies of scale can reduce O&M costs for wind farms and the overall cost of wind energy to consumers. Here, we present an MDP model to formulate the problem of optimally replacing wind turbines with the objective of minimizing the expected total discounted, setup, replacement and downtime costs over an infinite horizon.

Periodic inspections, which are assumed to be costless and instantaneous, are performed to reveal the state of each turbine perfectly (i.e., the state is completely observable). Such observations can be obtained, for example, by using sensors attached to critical components and tracking important signals of degradation, such as vibration or acoustic signals [Reference Bian, Gebraeel and Kharoufeh11]. A decision to replace or not replace a unit is made by a decision maker (DM) just after an inspection such that $\mathbb{Z}_+=\{0,1,\ldots\}$ is the set of decision epochs, and the time between any two consecutive inspections is a period in the MDP model. The state of the system is the $(N+1)$-dimensional vector $(\textbf{x},\ell)$ whose state space is $\Gamma:=\mathcal{X} \times \mathcal{L}$. For any decision epoch $m\in \mathbb{Z}_+$, let $a_n\in \{0,1\}$ denote the action taken for turbine $n\in \mathcal{N}$, where $a_n=0$ means “do nothing” and $a_n=1$ means “replace.” Therefore, the N-dimensional vector $\textbf{a}=(a_1,\ldots,a_N)$ describes the actions taken by the DM, and the action space is $\mathcal{A}:=\{0,1\}^N$. Whenever replacement is prescribed, the turbine must be shut down for one full period. Replacement activities commence immediately after the inspection and end just prior to the subsequent inspection. We assume that a period is sufficiently long to ensure that replacements are completed during a single period. Because the average time to complete a major replacement is between 2 to 7 days (see Maples et al. [Reference Maples, Saur, Hand, van de Pietermen and Obdam40]), the time between two consecutive decision epochs is 1 week in our model. However, it is important to note that the length of the period can be adjusted to account for different time scales when the model is used to prescribe the optimal timing of maintenance actions for minor components. In such cases, the period length can be set to 1 day. All replacements are assumed to be perfect and reset the degradation to level 0. For the remainder of the paper, vector inequalities are assumed to hold component-wise. That is, $\textbf{a}\le \textbf{a}^{\prime}$ implies $a_n\le a_n^{\prime}$, and $\textbf{x}\le \textbf{x}^{\prime}$ implies $x_n\le x_n^{\prime}$ for each $n\in \mathcal{N}$.

Let ${\hat{\textbf{x}}}_n(x_n,\ell,a_n)$ be the predicted level of degradation of turbine n at the start of the next period when the current degradation and environment states are xn and $\ell$, respectively, and action an is taken. Then,

(1)\begin{equation} {\hat{\textbf{x}}}_n(x_n,\ell,a_n) = \begin{cases} \min\{1,f_n(x_n,\ell,1)\}, & \text{if}\ a_n=0,\\ 0, & \text{if}\ a_n=1. \end{cases} \end{equation}

Following a replacement, degradation is returned to level 0 (an as-good-as-new condition), and without intervention, the turbine either fails or assumes some higher degradation level $f_n(x_n,\ell,1)$.

Next, we describe the setup and other costs associated with replacements in an offshore wind farm. If at least one unit is replaced, a setup cost cs ($c_s \gt 0$), which comprises significant equipment and crew transfer costs, is incurred. For offshore wind farms, the distance between the base station and the shoreline is usually significant such that helicopters and SOVs and/or JUVs must be used to transport needed equipment and crews to the farm. Additionally, heavy equipment is required to perform replacements; hence, the setup cost cs can be substantial (in excess of £250,000 per day of operation; see [Reference McMorland, Flannigan, Carroll, Collu, McMillan, Leithead and Coraddu42, Reference Shields, Marsh, Stefek, Oteri, Gould, Rouxel, Diaz, Molinero, Moser and Malvik64]). However, this cost can be shared if multiple turbines are replaced in the same period. The shared setup cost induces economic dependence between the turbines. Second, a per-replacement cost cr ($c_r \gt 0$) is incurred for every turbine that is replaced. This cost compromises the procurement cost of a new component, as well as the labor hours needed to complete the replacement. Finally, a downtime cost is incurred for any turbine that either (1) fails in the time interval between two inspections or (2) is taken offline so that a replacement can be performed. This cost corresponds to lost power production due to turbine unavailability [Reference Maples, Saur, Hand, van de Pietermen and Obdam40]. As noted by Wan et al. [Reference Wan, Ela and Orwig72], power output is a function of wind speed and is typically characterized by a monotone increasing, S-shaped curve that is convex at low wind speeds, concave after an inflection point, and flat beyond an upper speed threshold [Reference Hwangbo, Johnson and Ding26]; therefore, we assume the downtime cost rate is a function of the environment state $\ell$. Let $c_d(\ell)$ denote the downtime cost per unit time per turbine when the environment is in state $\ell$. For example, if wind speed instantiates the environment, disjoint intervals of speed may correspond to different regions of the power output curve, as depicted in Figure 2(a). Three distinct wind speed ranges (low, medium, and high) result in distinct regions of power output.

Figure 2. Partition of wind speed into disjoint intervals with distinct downtime cost rates. (a) A typical wind power curve. (b) Downtime cost rate versus environment.

Figure 2(b) illustrates how the wind speed intervals can be mapped to environment states in order to associate a downtime cost rate with each state. Wind speed can be mapped to power production using the well-known method of bins, as prescribed by the International Electrotechnical Commission [27]. In this specific illustration with three environment states, we divide the wind speed range into three intervals of equal length, with the lowest environment state (state 1) corresponding to wind speeds that exceed the cutoff speed; therefore, we assign a positive downtime cost rate for this state. We assume the downtime cost rate $c_d(\ell)$ is strictly increasing in $\ell\in \mathcal{L}$ and bounded from above. That is, $0 \lt c_d(1) \lt \cdots \lt c_d(L) \lt \infty$.

We now describe the degradation dynamics of a particular turbine $n\in \mathcal{N}$ with and without replacement. Figure 3(a) and 3(b) depicts scenarios in which the environment state is fixed at $\ell$ throughout the period. From Figure 3(a), we see that at decision epoch m, the turbine can either be failed (point A) or working (point B).

Figure 3. Degradation evolution of a turbine within a period with and without replacement. (a) A turbine with replacement. (b) A turbine without replacement.

For either case, if replacement is prescribed at time m, the turbine is down for the entire period and is restored to a new condition at decision epoch m + 1 (point C); hence, the cost $c_d(\ell)+c_r$ is incurred in each of the two scenarios of Figure 3(a). Figure 3(b) depicts the same two scenarios in which the turbine is either failed (point A) or working (point B) at time m. Here, if replacement is not prescribed at decision epoch m and the unit is failed, it remains failed at m + 1 so that the incurred cost is $c_d(\ell)$. On the other hand, if the turbine is working at time m (point B) and replacement is not prescribed, two possible scenarios (represented by the dotted lines) may occur. First, the turbine continues degrading and reaches the critical threshold (ξ = 1) before decision epoch m + 1 (point D). In this case, the turbine is down from the crossing time (point D) until the start of the next period (point E). This downtime is denoted by $\tau_n(x_n,\ell)$ and is obtained via the predictive function $f_n(x_n,\ell,t)$. Specifically, let $\tau_n(x_n,\ell)$ denote the predicted within-period downtime if no replacement is performed ($a_n=0$), xn is the degradation at the start of the period, and the environment state is $\ell$. Then,

(2)\begin{equation} \tau_n(x_n,\ell)=\max\left\{0,1-\inf\left\{t\ge 0: f_n(x_n,\ell,t)\ge 1\right\}\right\}. \end{equation}

For this first scenario, the downtime cost $c_d(\ell)\cdot \tau_n(x_n,\ell)$ is incurred for turbine n. For the second scenario, the turbine is working at time m (point B), but its level of degradation does not reach the failure threshold before decision epoch m + 1. Rather, it continues operating for the entire period and evolves to a higher level of degradation at decision epoch m + 1 (point F). In this case, $\tau_n(x_n,\ell)=0$ so that no downtime cost is incurred. Finally, we note that if the turbine is failed at decision epoch m (point A) and replacement is not prescribed, it remains failed at decision epoch m + 1 (point E). Since $f_n(1,\ell,t)=1$ for all $\ell\in \mathcal{L}$ and $t\ge 0$, it follows that $\tau_n(1,\ell)=1$. Therefore, the cost $c_d(\ell)\cdot 1=c_d(\ell)$ is incurred.

For notational convenience, define an indicator function $\mathbb{I}(\textbf{a})$, where $\mathbb{I}(\textbf{a})=1$ if $a_n=1$ for some $n\in \mathcal{N}$ (at least one replacement is performed) and $\mathbb{I}(\textbf{a})=0$ if $a_n=0$ for all $n\in \mathcal{N}$ (no replacements are performed). The one-step immediate cost is then

(3)\begin{align} c(\bf{x},\ell,\bf{a})= \mathbb{I}(\bf{a})c_s + \sum_{n\in \mathcal{N}}a_n\left[c_r +c_d(\ell)\right]+ \sum_{n\in \mathcal{N}}(1-a_n)\,c_d(\ell)\,\tau_n(x_n,\ell). \end{align}

The first term on the RHS of Equation (3) represents the setup cost. The second term is the sum of replacement costs and power production losses incurred for all replaced turbines. Finally, the third term accounts for the downtime costs estimated by the predictive functions for turbines that are not replaced.

All immediate costs are discounted at a fixed rate λ $(0 \lt \lambda \lt 1)$. The objective is to minimize the expected total discounted setup, replacement, and downtime costs over an infinite horizon. Let $\mathcal{P}$ denote the set of all nonanticipative replacement policies. Starting in state $(\textbf{x},\ell)$, the DM solves

(4)\begin{equation} V(\textbf{x},\ell)=\inf_{\pi\in \mathcal{P}}\left\{\mathbb{E}\left[\sum_{m=0}^{\infty}\lambda^m c(\textbf{x}_m,\ell_m,\textbf{a}_{\pi}(\textbf{x}_m,\ell_m)|(\textbf{x},\ell))\right]\right\},\qquad (\textbf{x},\ell)\in \Gamma, \end{equation}

where $\textbf{a}_{\pi}(\textbf{x}_m,\ell_m)$ denotes the action prescribed by policy $\pi\in \mathcal{P}$ in state $(\textbf{x}_m,\ell_m)$ at decision epoch m. Equation (4) is equivalent to

(5)\begin{equation} V(\textbf{x},\ell)= \min_{\textbf{a}\in\mathcal{A}}\left\{c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\textbf{x},\ell,\textbf{a}),k)p_{\ell k}\right\},\qquad (\textbf{x},\ell)\in \Gamma, \end{equation}

where $\hat{\textbf{x}}(\textbf{x},\ell,\textbf{a})=\left({\hat{\textbf{x}}}_1(x_1,\ell,a_1),\ldots,{\hat{\textbf{x}}}_N(x_N,\ell,a_N)\right)$ is the N-dimensional vector of predicted degradation levels at the start of the next period. Note that the state space Γ is Borel-measurable, and the action space $\mathcal{A}$ is finite. Additionally, the immediate costs are strictly positive and bounded, and the problem is discounted. Therefore, by Theorems 6.2 and 6.3 of [Reference Puterman54], there exists an optimal, deterministic stationary replacement policy, and the value iteration (VI) algorithm converges to the optimal value. Next, define the optimal action in state $(\textbf{x},\ell)$ by

(6)\begin{equation} \textbf{a}^*(\textbf{x},\ell):=\min\left\{\textbf{a}^{\prime}\in \mathrm{argmin}_{\textbf{a} \in \mathcal{A}}\left\{c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\textbf{x},\ell,\textbf{a}),k)p_{\ell k}\right\}\right\}, \qquad (\textbf{x},\ell) \in \Gamma. \end{equation}

That is, if there exist multiple optimal actions in state $(\textbf{x},\ell)$, we take the minimum action; hence, Equation (6) has a unique solution. In Section 3, we establish structural properties and provide some practical insights into this important operational problem.

3. Structural results and insights

In this section, we establish some properties of the value function $V(\textbf{x},\ell)$ and the optimal replacement policy of the MDP model presented in Section 2. Before proceeding to the main results, we first provide two lemmas that describe useful properties of the immediate and downtime costs. The proofs of all results are contained in the Appendix.

Lemma 3.1. For each $n\in \mathcal{N}$, $\tau_n(x_n,\ell)$ is monotone nondecreasing in $x_n\in [0,1]$ and $\ell\in \mathcal{L}$.

Lemma 3.1 implies that, for each turbine n, the first crossing time to the failure threshold is shorter starting from higher degradation and environment states. Thus, the resulting downtime within a period is longer. Before proceeding to Lemma 3.3, we first review the notion of discrete concavity, which is more fully elucidated by Murota et al. [Reference Murota45, Reference Murota and Tamura46].

Definition 3.2. Let $X\subseteq \mathbb{Z}$. A function $f:X\to \mathbb{R}$ is said to be discrete concave on its domain X if and only if $X\ne \emptyset$ and

\begin{equation*}f(x-1)+f(x+1) \le 2f(x)\quad\text{for all}\ x\in X.\end{equation*}

Discrete concavity is useful for characterizing structural properties of the value function. Due to the nature of power output of wind turbines, henceforth, we assume that the downtime cost rate $c_d(\ell)$ is discrete concave on its domain $\mathcal{L}$. Furthermore, to simplify notation, let $\delta_n(x_n,\ell):=c_d(\ell)\cdot \tau_n(x_n,\ell)$ for each $n\in \mathcal{N}$. We first examine properties of the immediate cost function, $c(\textbf{x},\ell,\textbf{a})$.

Lemma 3.3. For each $\textbf{a}\in \mathcal{A}$, the immediate cost function $c(\textbf{x},\ell,\textbf{a})$ is

  1. (a) monotone nondecreasing in $(\textbf{x},\ell)\in \Gamma$ and

  2. (b) discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ if $\delta_n(x_n,\ell)$ is discrete concave on $\mathcal{L}$ for each $n\in \mathcal{N}$.

Lemma 3.3(a) asserts that the immediate costs are monotone increasing as the degradation level and environment state increase. That is, it is more costly to operate a system that is more degraded and/or operating in a more detrimental environment. However, Lemma 3.3(b) asserts that the marginal cost increases are diminishing in these two dimensions if $\delta_n(x_n,\ell)$ is discrete concave on its domain. While this assumption may appear restrictive on first glance, it is appropriate in this context due to the nature of power output as a function of wind speed and other relevant conditions. Next, we present structural properties of the value function.

3.1. Value function properties

We now examine important properties of the value function $V(\textbf{x},\ell)$. Proposition 3.5 posits that the monotonicity and discrete concavity of Lemma 3.3 extend to the value function if the transition matrix P possesses the increasing failure rate (IFR) property.

Definition 3.4. Let $\mathbf{P}=[p_{\ell k}]$ be the one-step TPM of a DTMC with state space $\mathcal{L}=\{1,\ldots,L\}$. Then P is said to be IFR if

\begin{equation*}\eta_k(\ell):=\sum_{\ell'=k}^{L}p_{\ell \ell^{\prime}}\end{equation*}

is nondecreasing in $\ell\in \mathcal{L}$ for each $k\in \mathcal{L}$.

Definition 3.4 implies that, starting from a higher environment state, the DTMC is more likely to transition to even higher states. Next, we state the main result related to $V(\textbf{x},\ell)$.

Proposition 3.5. If P is IFR, then $V(\textbf{x},\ell)$ is

  1. (a) monotone nondecreasing in $(\textbf{x},\ell) \in \Gamma$ and

  2. (b) discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ if $\delta_n(x_n,\ell)$ is discrete concave on $\mathcal{L}$ for each $n\in \mathcal{N}$.

Proposition 3.5(a) asserts that if P is IFR, then the cost starting from state $(\textbf{x},\ell)$ is nondecreasing over the entire state space. That is, for a fixed environment state $\ell$, the cost function is monotone nondecreasing in degradation x, and for a fixed level of degradation x, the cost is monotone nondecreasing in $\ell$. However, the costs do not increase without bound. Proposition 3.5(b) asserts that the marginal costs diminish as the environment state increases, a property that follows directly from the discrete concavity of the downtime cost rate function $c_d(\ell)$ on $\mathcal{L}$.

Naturally, one would like to examine the sensitivity of the value function to changes in the model’s cost parameters. For example, how sensitive is the value function to an increase in the setup cost cs in light of the fact that this substantial cost can be shared by performing group replacements? To answer this question, we employ the notion of comparative statics, as described by Milgrom and Shannon [Reference Milgrom and Shannon43]. Let w be any one of the nonnegative cost parameters cs, cr, or $c_d(\ell)$. The functions $c(\textbf{x},\ell,\textbf{a},w)$ and $V(\textbf{x},\ell,w)$ denote, respectively, the immediate cost and value functions associated with parameter w. Proposition 3.6 asserts monotonicity in all three cost parameters.

Proposition 3.6. If $c(\textbf{x},\ell,\textbf{a},w)$ is monotone nondecreasing in w for each $(\textbf{x},\ell)\in \Gamma$ and $\textbf{a}\in \mathcal{A}$, then $V(\textbf{x},\ell,w)$ is monotone nondecreasing in w.

Consider a case in which the DM can reduce the setup cost by decreasing the cost of preparing the JUV and helicopter. Proposition 3.6 asserts that optimally replacing the turbines incurs less cost in this case, that is, $V(\textbf{x},\ell,c_s)\le V(\textbf{x},\ell,c^{\prime}_s)$ for each $(\textbf{x},\ell)\in \Gamma$ and any $c_s\le c^{\prime}_s$. Next, we provide useful structural properties of the optimal wind farm replacement policy.

3.2. Characterizing the optimal replacement policy

Here, we present the main results of this section by establishing structural properties of the optimal replacement policy for a wind farm. For notational convenience, let

\begin{equation*}\mathcal{V}(\textbf{x},\ell):=\{n\in \mathcal{N}:a^*_n(\textbf{x},\ell) = 1\},\end{equation*}

be the set of all turbines for which replacement is optimal in state $(\textbf{x},\ell)$. Similarly, let $\mathcal{N}\setminus \mathcal{V}(\textbf{x},\ell)$ be the set of turbines for which it is optimal to do nothing in state $(\textbf{x},\ell)$.

Theorem 3.7 For each $(\textbf{x},\ell)\in \Gamma$, consider a degradation vector $\textbf{y}\in \mathcal{X}$ such that $y_n\ge x_n$ for $n\in\mathcal{V}(\textbf{x},\ell)$ and $y_n=x_n$ for $n\in\mathcal{N}\setminus \mathcal{V}(\textbf{x},\ell)$. If P is IFR, then $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$.

Theorem 3.7 asserts that, for a given environment state $\ell$, the replacement region extends in each dimension for which it is optimal to replace in degradation level x. The result is elucidated by way of a simple example. Consider a wind farm comprising three turbines (N = 3), and suppose that in state $(\textbf{x},\ell)=(0.5,0.5,0.5,\ell)$, it is optimal to replace turbines 1 and 2 but not to replace turbine 3, that is, $\textbf{a}^*(\textbf{x},\ell)=(1,1,0)$. Figure 4(a) illustrates that these same actions are optimal for any state $(\textbf{y},\ell)$ in the shaded region (extension of the optimal actions in two dimensions).

Figure 4. Extension of optimal actions at degradation level x to higher degradation levels. (a) $\textbf{a}^*(\textbf{x},\ell)=(1,1,0)$ in shaded region. (b) $\textbf{a}^*(\textbf{x},\ell)=(1,1,1)$ in shaded region.

That is, $\textbf{a}^*(\textbf{y},\ell)=(1,1,0)$ for any degradation level $\textbf{y}\in [0.5,1]\times[0.5,1]\times \{0.5\}$. Figure 4(b) illustrates a different scenario in which the state is again $(\textbf{x},\ell)=(0.5,0.5,0.5,\ell)$, but it is optimal to replace all three turbines, that is, $\textbf{a}^*(\textbf{x},\ell)=(1,1,1)$. For this case, we see that action $(1,1,1)$ is optimal for any degradation level $\textbf{y}\in [0.5,1]\times[0.5,1]\times [0.5,1]$, so the optimal actions extend in three dimensions to states with higher levels of degradation.

Now for a given environment state $\ell$, consider a single turbine $n^{\prime}\in \mathcal{N}$ with degradation level $x_{n^{\prime}}$ and fix the degradation levels of all other turbines $n\in \mathcal{N}\setminus \{n^{\prime}\}$. The following corollary characterizes the behavior of the optimal action $a^*_{n^{\prime}}(\textbf{x},\ell)$ for each environment state.

Corollary 3.8. Suppose that P is IFR and let $n^{\prime}\in \mathcal{N}$. Fix $\ell \in \mathcal{L}$ and $x_n\in [0,1]$ for each $n\in \mathcal{N}\setminus \{n^{\prime}\}$. Then the optimal action $a^*_{n^{\prime}}(\textbf{x},\ell)$ is monotone nondecreasing in $x_{n^{\prime}}$.

As a consequence of economic dependence, it is possible that replacement of turbine n ʹ is not optimal for any level of degradation, that is, a degradation-based threshold policy does not exist. However, if reactive replacement is required whenever unit n ʹ is found to be failed, Corollary 3.8 implies the existence of a degradation threshold $x^*_{n^{\prime}}$ such that

(7)\begin{equation} a^*_{n^{\prime}}(\bf{x},\ell)= \begin{cases} 1, & \text{if}\, x_{n^{\prime}}\ge x^*_{n^{\prime}},\\ 0, & \text{if}\, x_{n^{\prime}} \lt x^*_{n^{\prime}}. \end{cases} \end{equation}

Equation (7) mirrors the optimal replacement policy described in Theorem 2 of [Reference Ulukuş, Kharoufeh and Maillart71] for a single degrading system that is influenced by its random environment; however, their model assumes that an immediate, reactive replacement is performed whenever a failure is revealed. Our model shows that, for some turbines, delaying replacement may be optimal due to the economic dependence induced by the shared setup cost (cs) and the turbines’ stochastic dependence induced by their common environment. That is, the threshold for replacing a turbine may depend not only that turbine’s degradation but also on the degradation levels of the other turbines in the wind farm. This interesting feature of our model is more transparent in the case of N = 2 turbines.

3.3. Special case: N = 2

Structural properties of the optimal replacement policy over the entire space Γ are not immediately obvious when a wind farm contains three or more turbines. Fortunately, Theorem 3.7 can be used to establish the dependence of an optimal replacement threshold of a particular turbine on the degradation of the other turbine in a two-turbine wind farm (N = 2). Before formalizing this interesting result, it is helpful to consider a graphical depiction of the two-turbine case in Figure 5. For $\ell$ fixed, the black circles in Figure 5 represent degradation levels $(x_1,x_2)\in [0,1]\times [0,1]$. From Theorem 3.7, we can deduce the following. If the optimal action at point A is $(0,1)$, then $(0,1)$ remains optimal for any point along the line segment AD. Specifically, for a fixed degradation level x 1, if it is optimal to replace turbine 2 at A, then it is optimal to replace turbine 2 for each point along AD. Similarly, if the optimal action at point B is $(1,0)$ (replace turbine 1 but not turbine 2), then $(1,0)$ remains optimal for any point along the line segment BE. Finally, at point C, the optimal action is $(1,1)$, replace both turbines. Then it is also optimal to replace both turbines for every point contained in the shaded region.

Figure 5. Extension of the optimal actions for two turbines in a fixed environment state.

Next, we seek to provide structure to replacement regions over the degradation space $\mathcal{X}$. For a fixed environment state $\ell\in \mathcal{L}$, let $x^*_2(x_1,\ell)$ denote the optimal replacement threshold for turbine 2 when the degradation of turbine 1 is $x_1\in [0,1]$. Similarly, let $x^*_1(x_2,\ell)$ be the optimal replacement threshold for turbine 1 when the degradation of turbine 2 is $x_2\in [0,1]$. When they exist, let

(8)\begin{align} b_1(x_2,\ell)&:=\inf\{x_1\ge 0:\textbf{a}^*(x_1,x_2,\ell)=(1,1)\}, \end{align}
(9)\begin{align} b_2(x_1,\ell)&:=\inf\{x_2\ge 0:\textbf{a}^*(x_1,x_2,\ell)=(1,1)\}. \end{align}

That is, $b_1(x_2,\ell)$ ($b_2(x_1,\ell)$) is the minimum level of turbine 1 (turbine 2) degradation that makes group replacement optimal when the degradation of turbine 2 (turbine 1) is x 2 (x 1). The next proposition asserts that the optimal replacement threshold of one turbine is monotone decreasing in the degradation level of the other turbine beyond these special points $b_1(1,\ell)$ and $b_2(1,\ell)$.

Proposition 3.9. For each $\ell\in \mathcal{L}$, if P is IFR and $\textbf{a}^*(1,1,\ell)=(1,1)$, then

  1. (a) $x^*_2(x_1,\ell)$ is monotone nonincreasing in $x_1\in \{x_1: b_1(1,\ell)\le x_1\le 1\}$ and

  2. (b) $x^*_1(x_2,\ell)$ is monotone nonincreasing in $x_2\in \{x_2: b_2(1,\ell)\le x_2\le 1\}$.

Proposition 3.9 establishes the fact that the replacement threshold of one turbine depends on the degradation state of the other turbine. Figure 6 depicts the region in which it is optimal to replace turbine 2, that is, $\left\{(x_1,x_2):\textbf{a}^*(x_1,x_2,\ell)\in \{(0,1),(1,1)\}\right\}$. The figure illustrates that the replacement threshold for turbine 2 is monotone decreasing in x 1 for any $x_1\ge b_1(1,\ell)$.

Figure 6. Monotone threshold policy for turbine 2 in a wind farm with two turbines.

Figure 6 also highlights the fact that it is advisable for the DM to decrease the optimal threshold of turbine 2 as turbine 1 reaches higher degradation levels. The results in Proposition 3.9 confirm that the replacement thresholds for both turbines are monotone decreasing. In Section 4, we provide some numerical examples illustrating the effects of economic and stochastic dependence on the optimal turbine replacement policies.

4. Numerical illustrations

Here, we illustrate the structure of optimal replacement policies for a two-turbine wind farm using our MDP model. While our main results are valid for any finite number of turbines, we choose to illustrate the policies for N = 2 since they can be easily visualized in the xy-plane. It is worth noting that, although the state space grows exponentially in the number of turbines N, approximate dynamic programming (ADP) and state-space reduction techniques can be employed to solve larger instances, as was done in [Reference Abdul-Malak and Kharoufeh2]. First, we provide numerical examples demonstrating how these policies differ depending on the magnitude of the setup cost (cs) and the prevailing environment state ($\ell$). Second, we assess the importance of incorporating economic and stochastic dependence when evaluating turbine replacement policies. To that end, we consider simpler models that either ignore these features altogether or use fewer environment states than are needed to adequately characterize the environment. We begin by describing a baseline example, which considers a wind farm with two wind turbines. There exists ample evidence to suggest that 6–8 states are sufficient to model environmental conditions [Reference Anastasiou and Tsekos4]; hence, our baseline example contains six distinct environment states.

4.1. A baseline example

Suppose there are two turbines in an offshore wind farm (N = 2), and the turbines are exposed to an environment whose evolution is modeled by a DTMC with state space $\mathcal{L}=\{1,\ldots,6\}$ and TPM

\begin{equation*}\mathbf{P}= \begin{bmatrix} 0.40 & 0.20 & 0.20 & 0.14 & 0.05 & 0.01\\ 0.22 & 0.30 & 0.20 & 0.15 & 0.10 & 0.03\\ 0.20 & 0.14 & 0.30 & 0.22 & 0.11 & 0.03\\ 0.10 & 0.10 & 0.20 & 0.35 & 0.15 & 0.10\\ 0.05 & 0.08 & 0.10 & 0.20 & 0.35 & 0.22\\ 0.03 & 0.05 & 0.10 & 0.15 & 0.40 & 0.27\\ \end{bmatrix}.\end{equation*}

It can be shown that P possesses the IFR property. For this illustration, it is assumed that, for each turbine $n\in \mathcal{N}$, the DM predicts future degradation using the linear predictive function

\begin{equation*}f_n(x_n,\ell,t) = x_n + r_n(\ell)t,\quad t\ge 0,\end{equation*}

where $r_n(\ell)$ is the degradation rate of turbine n in environment state $\ell\in \mathcal{L}$. The form of fn is immaterial as long as it satisfies the nonnegativity and monotonicity assumptions stated in Section 2. It is difficult to obtain proprietary wind farm O&M records to parameterize our model. Therefore, notional setup costs, downtime cost rates, and degradation rates were chosen relative in scale but are not intended to reflect any real data provided to the authors. The linear degradation rates were chosen such that $r_n(i) \lt r_n(j)$ whenever i < j, and their magnitudes are in proportion to the failure threshold ξ = 1. Because the time scale of one period is on the order of 1 week, the rates of degradation are relatively small as compared to the failure threshold of 1. Similarly, the downtime cost rates are selected such that $c_d(i) \lt c_d(j)$ whenever i < j. The setup and replacement costs are $c_s=5$ and $c_r=8$, respectively. Table 1 lists all other notional parameter values.

Table 1. Downtime cost and degradation rates for each environment state.

To implement the VI algorithm of [Reference Puterman54], we discretized the continuous degradation space $\mathcal{X}$ (cf. Kushner and Dupuis [Reference Kushner and Dupuis37]). Specifically, for each $n\in \mathcal{N}=\{1,2\}$, the degradation interval $[0,1]$ was uniformly discretized into 101 states so that $x_n\in \{0.00,0.01,\ldots,0.99,1.00\}$, and we let $\mathcal{X}^{\prime}:=\{0.00,0.01,\ldots,0.99,1.00\}^2$. Then for each $(\textbf{x},\ell)$ in $\mathcal{X}^{\prime}\times \mathcal{L}$, let $v^k(\textbf{x},\ell)$ denote the kth iterate of the VI algorithm with $v^0(\textbf{x},\ell)=0$. The algorithm terminates at iteration k $(k\in \mathbb{N}$) if

\begin{equation*}\big\|v^{k} -v^{k-1}\big\|_{\infty} = \max_{(\textbf{x},\ell)}\left\{\big|v^{k}(\textbf{x},\ell)-v^{k-1}(\textbf{x},\ell)\big|\right\}\le \epsilon_v,\end{equation*}

where $\epsilon_v \gt 0$ is a tolerance level. For each implementation of the VI algorithm, we set $\epsilon_v = 0.01$ and λ = 0.99. The VI algorithm was coded in Python 3.9 and executed on a personal computer with a 3.40 GHz processor and 8 GB of RAM. Next, we examine the sensitivity of the optimal replacement policy to each environment state and the setup cost.

4.2. Baseline example sensitivity

Starting in state $(x_1,x_2,\ell)=(0,0,1)$, the VI algorithm yields an optimal total discounted cost of 3,044.20 for the baseline example. For a fixed environment state, replacement regions are the regions in which at least one turbine is replaced (shaded regions). The opportunistic region is the region in which both turbines are replaced (dark shaded region). Figure 7 depicts the replacement regions in different environment states.

Figure 7. Baseline example optimal replacement policy for each environment state ($c_s=5$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

Figure 7(b)–(f) shows that higher environment states lead to larger replacement regions, as these states imply accelerated degradation and higher power production rates. Hence, in higher environment states, the DM opts to lower the replacement threshold to avoid high production losses caused by turbine failures. However, in Figure 7(a), the largest replacement region is for environment state $\ell=1$ since the production and degradation rates for this state are least among all environment states; hence, the DM prefers to perform the preventive replacements much sooner. Such a scenario may correspond to the overhaul of large, critical components in a wind farm, as major replacements are typically performed in the summer months when environment conditions are milder. Figure 7(a) also illustrates the result in Proposition 3.9—the optimal replacement threshold of one turbine is decreasing in the degradation state of the other turbine when both are in the opportunistic region. However, we also observe nonintuitive behavior for the optimal replacement policy in Figure 7; the optimal replacement threshold of each turbine is not necessarily decreasing over the entire degradation state space. For example, in Figure 7(a), for a fixed degradation state of turbine 2 ($x_2=0.4$), the DM opts to replace turbine 2 at point B but not to replace at points A and C. Thus, by increasing the degradation state of turbine 1, the optimal replacement threshold for turbine 2 decreases first (moving from point A to B) and then increases (moving from point B to C). One possible explanation for this behavior is that, at point C, turbine 2 is closer to the opportunistic region and will reach this region sooner. Thus, it is best for the DM to postpone replacement of turbine 2 until turbine 1 further degrades so that a group replacement can be performed and setup costs can be shared.

In the first instance of the baseline example, the setup cost ($c_s=5$) was relatively low compared to the downtime cost rates. This scenario corresponds to the case when the required resources for replacement are available (e.g., no cost for chartering a JUV and helicopter) and the majority of the setup cost is fueling. For the sake of comparison, we also examined a scenario with a relatively high setup cost corresponding to the case when a helicopter and JUV must be chartered from a spot market, and the total setup cost exceeds the highest capacity of power production for a turbine ($c_s = 80$). Figure 8 shows how the optimal replacement regions shrink with a substantial increase in the setup cost. Figure 8(c)–(f) shows that for some degradation levels, the DM may allow the other turbine to remain failed and postpone replacement. That is, for higher states of degradation, replacing only one turbine is not optimal, and the DM is more inclined to perform group replacement due to the magnitude of the setup cost.

Figure 8. Baseline example optimal replacement policy for each environment state ($c_s=80$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

One would expect the opportunistic region to be larger in higher environment states since the DM opts to perform the group replacement sooner to avoid higher power production losses. However, in Figures 7 and 8, the opportunistic region does not necessarily grow with the environment state. There are two reasons for this nonintuitive behavior. First, if the environment state is 1, then the likelihood of transitioning to higher states is low. Hence, it is expected that, in subsequent periods, the environment remains in state 1 with a higher probability. Second, the downtime cost in environment state 1 is low; hence, the DM prefers to replace both turbines in an environment state that leads to a lower production loss. In the next subsection, we show how models with incorrect assumptions impact the optimal replacement policies.

4.3. Value of correct modeling

Here, our aim is to assess the magnitude of cost increases resulting from incorrect model assumptions. The MDP model incorporates economic dependence via the setup cost cs and stochastic dependence via the association of degradation processes to the turbines’ common environment. Here, we compare the expected total discounted costs obtained by two distinct models and compare those with the costs obtained by the model of Section 2. In the first model, the DM simply ignores both types of dependence and assumes that replacement decisions are made for each turbine independently. In the second model, the DM includes dependence but assumes the environment can be described by a simple two-state DTMC, thereby ignoring any intermediate environment states.

Let $\pi^*$ be the optimal replacement policy obtained by the MDP model of Section 2, and let πi be the optimal replacement policy obtained by Model i, for $i=1,2$. Table 2 summarizes how the models differ with regard to their underlying assumptions.

Table 2. Summary of the three distinct models for the replacement problem.

The full model incorporates both economic and stochastic dependence and all six environment states. After obtaining the optimal policy for each model using the VI algorithm, we simulated the respective costs of the modified policies (π 1 and π 2) on the full model using a large number of sample paths. Along each sample path, the total discounted cost was computed, and these values were compared. The simulation run length, which corresponds to the number of decision epochs, is denoted by M ($M\in \mathbb{N})$. Since the expected one-step costs are bounded, and the cost function is discounted, the simulation run length can be determined a priori to ensure that the total discounted cost is accurate to a fixed constant. The simulation run length M was chosen such that $M\ge \left[\ln((1-\lambda)\epsilon_s/C)/\ln(\lambda)\right]-1$, where C is any valid upper bound on the expected one-step costs and λ is the discount factor. For all numerical examples, M was chosen to correspond to $\epsilon_s = 0.01$, λ = 0.99, and $C=c_s+N[c_r+c_d(L)]$. For the baseline example, $M\ge 1428$, so we chose M = 1500.

To compute the average simulated total discounted cost under each policy, we selected the number of replications (R) to achieve a desired margin of error using the well-known relationship

(10)\begin{equation} R = \left(z_{\alpha/2}\,\hat{\sigma}/\Delta\right)^2, \end{equation}

where $z_{\alpha/2}$ is the critical z-value above which lies area $\alpha/2$ under the standard normal density function, $\hat{\sigma}$ is the estimated standard deviation, and Δ is the desired margin of error. For each of the following models, we chose a 95% confidence level (α = 0.05, $z_{\alpha/2}=1.96$) and $\Delta=1$. The simulation models were coded in the MATLAB R2022a computing environment and executed on a personal computer with an 8-core CPU and 8 GB of RAM. We next describe the modified MDP formulations for Models 1 and 2, along with the corresponding numerical results.

Model 1. The first modified model ignores both economic and stochastic dependence; therefore, the problem can be decomposed into two independent, single-turbine problems. For each decomposed problem, the setup and replacement costs are assessed every time a replacement is performed. Although both turbines are assumed to operate in environments governed by the same transition matrix P, their environments (and degradation processes) evolve independently. Thus, for turbine $n\in \{1,2\}$, the state is $(x_n,\ell_n)\in [0,1]\times \mathcal{L}$, $a_n\in \{0,1\}$ is the action, and the immediate cost function is

\begin{equation*}c(x_n,\ell_n,a_n) = a_n\left[c_s +c_r +c_d(\ell_n)\right]+(1-a_n)c_d(\ell_n)\tau_n(x_n,\ell_n).\end{equation*}

The Bellman equation for the decomposed MDP model n is

\begin{equation*}V(x_n,\ell_n)=\min_{a_n\in \{0,1\}}\left\{c(x_n,\ell_n,a_n)+\lambda \sum_{k_n\in \mathcal{L}}V({\hat{\textbf{x}}}_n(\ell_n,a_n),k_n)p_{\ell_n k_n}\right\},\qquad (x_n,\ell_n)\in [0,1]\times\mathcal{L},\end{equation*}

where

\begin{equation*}{\hat{\textbf{x}}}_n(\ell_n,a_n)= \begin{cases} \min\{1,f_n(x_n,\ell_n,1)\}, & \text{if}\ a_n=0,\\ 0, & \text{if}\ a_n=1. \end{cases}\end{equation*}

Figure 9 illustrates the behavior of the Model 1 optimal replacement policy (π 1) for each environment state. We observe that the optimal replacement threshold of one turbine does not depend on the other’s degradation state in the absence of economic and stochastic dependence. Moreover, the optimal replacement thresholds decrease with the environment state. That is, in environments that induce higher degradation and power production rates, preventive replacements are prescribed at lower levels of degradation for both turbines.

Next, we seek to assess how the Model 1 optimal policy (π 1) performs under the assumptions of the full model (i.e., when economic and stochastic dependence are present). To this end, we apply policy π 1 to a simulation of the full model and compare its resulting simulated total discounted cost. We first performed 5,000 replications of the simulation model (each with $M=1,500$ decision epochs) and obtained a sample standard deviation $\hat{\sigma}\approx256$. By Equation (10), the number of replications needed to ensure $\Delta=1$ is $R=251,763$; hence, we selected $R=252,000$. The policy π 1 was applied in the simulation of the full model, and the simulated total discounted costs for three different initial states are reported in Table 4. The simulated costs are also compared with those obtained using VI on the full model, which includes both economic and stochastic dependence.

Figure 9. Baseline example optimal replacement policy for each environment state ($c_s=80$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

Model 2. Like the full model, this second modification incorporates both economic and stochastic dependence (via shared setup costs and a common environment). However, in this case, the DM lacks sufficient information to assess the number of environment states and the degradation rates associated with each state. Recall that the full model assumes that the environment can transition among six distinct states. In Model 2, the DM uses only two states to describe the environment ($\mathcal{L} = \{1,2\}$) and estimates a degradation rate for each state. Table 3 summarizes the downtime cost rates and degradation rates corresponding to each environment state. Note that the values used for states 1 and 2 in Model 2 correspond to those of the extreme states 1 and 6 from Table 1. For this new two-state model, the DM scales the original transition probabilities, p 11, p 16, p 61, and p 66, to construct the new TPM

\begin{equation*}\widehat{\mathbf{P}}=\begin{bmatrix} 0.98 & 0.02\\ 0.10 & 0.90\\ \end{bmatrix},\end{equation*}

Table 3. Downtime cost and degradation rates for the environment states of Model 2.

where, for example, $\hat{p}_{11}$ was obtained by $\hat{p}_{11}=p_{11}/(p_{11}+p_{16})\approx 0.98$. All other components of this MDP model, including the immediate cost function and the Bellman equation, are identical to the MDP model defined in Section 2. Figure 10 depicts the optimal replacement policy (π 2) obtained by VI algorithm for each of the two environment states. To compare the costs of this policy to the full model, it was necessary to create a mapping between each of the six states in the full model to the two states of the modified environment process. Specifically, the replacement policy for $\ell=1$ (Figure 10(a)) was used for states 1, 2, and 3 in the full model. The replacement policy for $\ell=2$ (Figure 10(b)) was used for states 4, 5, and 6 in the full model.

Figure 10. Model 2 optimal replacement policy for each environment state. (a) Environment state $\ell=1$. (b) Environment state $\ell=2$.

We first performed 5,000 replications of the simulation model (each with $M=1,500$ decision epochs) and obtained a sample standard deviation $\hat{\sigma}\approx 251$. By Equation (10), the number of replications needed to ensure $\Delta=1$ is $R=242,024$; hence, we selected $R=243,000$. As before, we apply policy π 2 to a simulation of the full model and compare its resulting simulated total discounted cost. The simulated total discounted costs for six different initial states are summarized in Table 4 and compared with those obtained using VI on the full model.

Table 4 compares the expected total discounted costs of the full model and the simulated total discounted costs of Models 1 and 2 starting from initial states $(0,0,1)$, $(0,0,2)$, $(0,0,3)$, $(0,0,4)$, $(0,0,5)$, and $(0,0,6)$. This table reveals several interesting results. First, the policies resulting from Model 1, in which both economic and stochastic dependence are ignored, yield average total discounted costs that are just over 8% higher than those generated by the full model, indicating the importance of sharing setup costs and consideration of how the turbines’ degradation processes are related via their shared environment. Second, we note that the policies resulting from Model 2, which includes both types of dependence but fails to adequately characterize the environment process, result in cost increases on the order of 5%. These results provide evidence that accounting for all possible environment scenarios can be important when assessing the lifetime replacement costs of an offshore wind turbine. Finally, we observe a general trend that the total costs are increasing in the initial environment state (assuming both turbines start in an as-good-as-new condition). This observation is in agreement with Proposition 3.5(a).

Table 4. Total discounted costs for six distinct initial states.

4.4. Comparisons with heuristic policies

Here, we assess the performance of the optimal replacement policy obtained from our model with three distinct and common heuristic policies:

  • Reactive replacement policy. Using this policy, the DM replaces the component reactively, that is, immediately following a failure.

  • Age-replacement policy. The classical age-replacement policy prescribes replacement either at failure or at a chosen deterministic age T, whichever occurs first. The replacement age is taken to be the expected lifetime of the unit, which is typically the manufacturer’s mean time-to-failure.

  • Fixed interval policy. This policy performs a replacement every T units, whether a replacement is warranted or not. Here, the deterministic inter-replacement time T is taken to be the manufacturer’s mean time-to-failure.

We compared the total costs generated by the MDP policies with those of the heuristic policies starting from six distinct initial states (corresponding to the six environment states). The results in Table 5 show that the total discounted costs obtained by the MDP model are approximately 17% lower than those obtained using reactive replacement, 14% lower than those obtained using the age-replacement policy, and over 40% lower than those obtained using a fixed interval policy. These results highlight the value of using the optimal policy as compared to simple heuristic policies.

Table 5. Total discounted cost comparison: Optimal policy versus three heuristic policies.

5. Conclusions and discussion

We devised and analyzed an MDP model to optimally prescribe the replacement times of critical components in an offshore wind farm. Our model includes both economic and stochastic dependence between the turbines operating in the farm. Economic dependence is induced by significant setup costs that can be shared by performing opportunistic replacements; stochastic dependence exists in the turbines’ degradation processes due to their common exposure to a randomly varying environment. The main results may have significant practical implications for offshore wind farm operators who seek to reduce turbine downtime and ensure sustained power production capacity. We established the monotonicity of actions for each turbine in the farm (fixing the degradation of all other turbines). Most interesting is the fact that the optimal replacement time of one turbine depends not only on its own level of degradation but also on the degradation levels of the other turbines in the farm. Furthermore, beyond a certain point, the replacement thresholds are monotone. We showed that it may be more beneficial to delay replacements and reduce overall costs by performing group replacements when the setup costs can be shared among multiple turbines. This result is important since the cost of chartering or deploying service operations vessels and/or JUVs is significant. Numerical examples illustrated the structure of replacement policies in the case of two turbines and showed that monotone policies emerge beyond certain degradation thresholds for both turbines. We also demonstrated the value of including dependencies and accounting for the influence of the environment. When compared with three different heuristic policies, it was shown that the optimal policy substantially reduces the total discounted costs.

While the insights gained from the MDP model are valuable, it can be further expanded and improved to incorporate additional contextual information (in the form of constraints) related to the maintenance of offshore wind farms. Several factors influence the feasibility of performing maintenance operations, including the wind speed and wave height, availability of specialized resources (vessels and/or helicopters), and availability of crews to perform the work. Specifically, safety limits for wind speed and wave height dictate conditions for which vessels can safely operate in the sea, and when conditions are unsafe, operations are deemed infeasible. To incorporate this realism into our model, one need only to append an additional state—an indicator variable that assumes the value 0 or 1—indicating if weather conditions are suitable for maintenance operations. When conditions are too harsh, the indicator is assigned a value 0 and maintenance cannot be performed. In this case, the only feasible action is “do nothing” and wait until the next decision epoch. Another critical factor affecting the feasibility of maintenance is the availability of specialized resources (e.g., SOVs, CTVs, JUVs, and helicopters) or work crews. Likewise, these constraints can be included in the MDP model by adding indicator variables to the state vector, which indicate whether a resource or crew are available at each decision epoch. However, the inclusion of these additional components in the state vector increases the dimensionality of the problem to the extent that it may become intractable or the structural results might be lost. Another complication is that the number of states grows exponentially in the number of turbines in the wind farm. To deal with these issues, one can consider ADP techniques, along with state–space reduction strategies, as demonstrated by Abdul-Malak and Kharoufeh [Reference Abdul-Malak and Kharoufeh2].

Although the model presented herein focused on the replacement of major components in an offshore wind turbine, it can be modified to prescribe the timing of other maintenance actions (e.g., minor or medium repairs) for components that experience more frequent failures than major components such as generators and gearboxes. The MDP model employs an assumption that replacement takes one period (up to 1 week) to complete and that all replacements are completed before the next decision epoch. Because the average time needed to repair or replace a minor component is typically shorter than that of a major component, the length of time between decision epochs can be chosen to ensure that the maintenance action is completed within one period. For example, minor repairs or replacements can usually be completed within 1 day, whereas major replacements may require 2–7 days to complete. Thus, the time scale of the MDP model can be modified to account for minor repairs or replacements by setting the period length to 1 day. It is important to note that the per-period costs and the degradation function must be modified in this case. It is expected that the opportunistic region for minor or medium repairs will be larger than the opportunistic region for replacing major components.

Finally, in our framework, the DM’s objective is to minimize the expected total discounted costs, which include the setup and replacement costs, as well as the costs associated with turbine downtime. However, if the objective is to maximize expected profit over a planning horizon, it is necessary to modify the MDP model by incorporating pricing and production rate information into the one-step profit. To this end, let $C_n(\ell)\in \mathbb{R_+}$ be the nominal rate of power production for turbine n when the environment state is $\ell$ and ρm ($\rho_m \gt 0$) the price at decision epoch m. It can be assumed that a turbine produces power at its nominal rate when it is operating. The price can be incorporated by expanding the state to $(\textbf{x},\ell,\rho)$ and using pricing information, for example, from regional transmission operators such as PJM (https://www.pjm.com) to estimate the probability that the price transitions from ρm to $\rho_{m+1}$ in one period. By including the dynamic and stochastic price, the DM may consider maximization of expected profits. During each period, revenue is only accrued for a turbine if it is operational; no revenue is included during downtime. For instance, at the beginning of a period, if turbine n is failed $(x_n=1)$ or is scheduled for replacement $(a_n=1)$, there is no earned revenue because the turbine is down for the entire period. However, if turbine n is operational at the start of period m and no replacement is scheduled, the one-step revenue can be calculated as $C_n(\ell)\rho_m\left[1-\tau_n(x_n,\ell)\right]$, where $\tau_n(x_n,\ell)$ is the predicted downtime during that period. Finally, the profit in each period is determined by subtracting the applicable setup and replacement costs (if there is a need for replacement) from the generated revenues.

Acknowledgements

The authors wish to express their gratitude to two anonymous referees and an Editorial Board Member for their insightful comments and helpful suggestions that have improved this work.

Appendix A.

Proof of Lemma 3.1

Proof. For each $t\ge 0$ and $(\textbf{x},\ell),(\textbf{x}^{\prime},\ell^{\prime}) \in \Gamma$ such that $(\textbf{x},\ell) \le (\textbf{x}^{\prime},\ell^{\prime})$, $f_n(x_n,\ell,t)\le f_n(x^{\prime}_n,\ell^{\prime},t)$. Thus, we have

\begin{equation*}\inf\left\{t\ge 0: f_n(x_n,\ell,t)\ge 1\right\}\ge \inf\left\{t\ge 0: f_n(x_n,\ell^{\prime},t)\ge 1\right\}.\end{equation*}

It follows that

\begin{equation*}1-\inf\left\{t\ge 0: f_n(x_n,\ell,t) \ge 1\right\}\le 1-\inf\left\{t\ge 0: f_n(x^{\prime}_n,\ell^{\prime},t)\ge 1\right\},\end{equation*}

and subsequently,

\begin{equation*}\max\left\{0,1-\inf\left\{t\ge 0: f_n(x_n,\ell,t)\ge 1\right\}\right\}\le \max\left\{0,1-\inf\left\{t\ge 0:f_n(x^{\prime}_n,\ell^{\prime},t)\ge 1\right\}\right\}.\end{equation*}

Proof of Lemma 3.3

Proof. For any given $\textbf{x}\le \textbf{x}^{\prime}$, $\ell\le \ell^{\prime}$ and $n\in \mathcal{N}$, if $a_n=0$, by Lemma 3.1, we have

(A.1)\begin{equation} c_d(\ell)\tau_n(x_n,\ell)\le c_d(\ell^{\prime})\tau_n(x^{\prime}_n,\ell^{\prime}), \end{equation}

and if $a_n =1$,

(A.2)\begin{equation} c_r +c_d(\ell)\le c_r +c_d(\ell^{\prime}), \end{equation}

since $c_d(\ell)$ is strictly increasing in $\ell$. As Equations (A.1) and (A.2) hold for each $n\in \mathcal{N}$, then for any $\textbf{a} \in \mathcal{A}$,

\begin{align} \nonumber c(\bf{x},\ell,\bf{a}) = & \; \mathbb{I}(\bf{a})c_s + \sum_{n\in \mathcal{N}}a_n[c_r +c_d(\ell)]+\sum_{n\in \mathcal{N}}(1-a_n) c_d(\ell)\tau_n(x_n,\ell)\\ \nonumber \end{align}
\begin{align} \leq &\;\mathbb{I}(\textbf{a})c_s + \sum_{n\in \mathcal{N}}a_n[c_r +c_d(\ell^{\prime})]+\sum_{n\in \mathcal{N}}(1-a_n) c_d(\ell^{\prime})\tau_n(x^{\prime}_n,\ell^{\prime}) = c(\textbf{x}^{\prime},\ell^{\prime},\textbf{a}). \end{align}

Lemma 3.3(a) follows immediately. Now, for $\textbf{a}\in \mathcal{A}$ and $\textbf{x}\in \mathcal{X}$, $c(\textbf{x},\ell,\textbf{a})$ is a linear function of $c_d(\ell)$ and $\delta_n(x_n,\ell):=c_d(\ell)\tau_n(x_n(\ell)$, which are discrete concave on $\mathcal{L}$. Thus, Lemma 3.3(b) follows as well.

Proof of Proposition 3.5

Proof. For $(\textbf{x},\ell) \in \Gamma$, denote the mth iterate of the VI algorithm by $v^m(\textbf{x},\ell)$. We prove Proposition 3.5(a) by induction on m. Take $v^0(\textbf{x},\ell) = 0$ for all $(\textbf{x},\ell) \in \Gamma$. Therefore, for any pairs of $(\textbf{x},\ell),(\textbf{x}^{\prime},\ell^{\prime}) \in \Gamma$ such that $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$, by Lemma 3.3, we have

\begin{equation*} v^1(\textbf{x},\ell) = \min_{\textbf{a} \in \mathcal{A}} \left\{c(\textbf{x},\ell,\textbf{a}) +0\right\}\leq \min_{\textbf{a} \in \mathcal{A}} \left\{c(\textbf{x}^{\prime},\ell^{\prime},\textbf{a})+0\right\}=v^1(\textbf{x}^{\prime},\ell^{\prime}) \end{equation*}

since $c(\textbf{x},\ell,\textbf{a})$ is monotone nondecreasing in $(\textbf{x},\ell) \in \Gamma$ for each $\textbf{a} \in \mathcal{A}$. For the induction hypothesis, assume $v^m(\textbf{x},\ell) \leq v^m(\textbf{x}^{\prime},\ell^{\prime})$. From Equation (1), $\hat{\textbf{x}}(\ell,\textbf{a}) \leq \hat{\textbf{x}}^{\prime}(\ell^{\prime},\textbf{a})$ for any given $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$ and $\textbf{a} \in \mathcal{A}$. Since P is IFR, by Lemma 4.7 of [Reference Puterman54], we have

\begin{equation*} \lambda \sum_{k \in \mathcal{L}}v^m(\hat{\textbf{x}}(\ell,\textbf{a}),k)p_{\ell k} \leq \lambda \sum_{k \in \mathcal{L}}v^m\left(\hat{\textbf{x}}^{\prime}(\ell^{\prime},\textbf{a}),k\right)p_{\ell^{\prime}k} \end{equation*}

for any given $(\textbf{x},\ell) \leq (\textbf{x}^{\prime},\ell^{\prime})$ and $\textbf{a} \in \mathcal{A}$. By Lemma 3.3, we add $c(\textbf{x},\ell,\textbf{a})$ and $c(\textbf{x}^{\prime},\ell^{\prime},\textbf{a})$ to the left-hand side and right-hand side of the above inequality, respectively, to obtain

\begin{equation*} c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k \in \mathcal{L}}v^m(\hat{\textbf{x}}(\ell,\textbf{a}),k)p_{\ell k} \leq c(\textbf{x}^{\prime},\ell^{\prime},\textbf{a})+\lambda \sum_{k \in E}v^m\left(\hat{\textbf{x}}^{\prime}(\ell^{\prime},\textbf{a}),k\right)p_{\ell^{\prime}k}. \end{equation*}

Minimizing both sides over $\textbf{a} \in \mathcal{A}$ results in $v^{m+1}(\textbf{x},\ell) \leq v^{m+1}(\textbf{x}^{\prime},\ell^{\prime})$. By Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell)\to V(\textbf{x},\ell)$, as $m\to\infty$, and Proposition 3.5(a) follows. Similarly, we prove Proposition 3.5(b) by induction on m. Take $v^0(\textbf{x},\ell) = 0$ for all $(\textbf{x},\ell) \in \Gamma$. Therefore, for each $(\textbf{x},\ell) \in \Gamma$, we have

\begin{align} \nonumber v^1(\textbf{x},\ell-1) + v^1(\textbf{x},\ell+1)= & \min_{\textbf{a} \in \mathcal{A}}\{c(\textbf{x},\ell-1,\textbf{a})\}+\min_{\textbf{a} \in \mathcal{A}}\{c(\textbf{x},\ell+1,\textbf{a})\} \end{align}
(A.3)\begin{align}\leq & \min_{\textbf{a} \in \mathcal{A}}\{c(\textbf{x},\ell-1,\textbf{a})+c(\textbf{x},\ell+1,\textbf{a})\} \end{align}
(A.4)\begin{align}\leq & \min_{\textbf{a} \in \mathcal{A}}\{2c(\textbf{x},\ell,\textbf{a})\} = 2\min_{\textbf{a} \in \mathcal{A}}\{c(\textbf{x},\ell,\textbf{a})\} = 2v^1(\textbf{x},\ell). \end{align}

Inequalities (A.3) and (A.4) hold by Lemma 3.3(b) and the fact that the min operator preserves the discrete concavity property. For the induction hypothesis, assume $v^m(\textbf{x},\ell)$ is discrete concave on $\mathcal{L}$ for each $\textbf{x} \in \mathcal{X}$. Define $Q(\textbf{x},\ell,\textbf{a})$ as the cost-to-go function in state $(\textbf{x},\ell)$ under action a, that is,

(A.5)\begin{equation} Q(\textbf{x},\ell,\textbf{a}):= c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\ell,\textbf{a}),k)p_{\ell k}. \end{equation}

Then, for $(\textbf{x},\ell)\in \Gamma$, let $Q^m(\textbf{x},\ell,\textbf{a})$ denote the cost-to-go function with respect to $\textbf{a}\in \mathcal{A}$ and the mth iterate of the VI algorithm. That is,

\begin{equation*} Q^m(\textbf{x},\ell,\textbf{a}) = c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k\in \mathcal{L}}v^{m}(\hat{\textbf{x}}(\ell,\textbf{a}),k)p_{\ell k}. \end{equation*}

Since $Q^m(\textbf{x},\ell,\textbf{a})$ is the sum of two discrete concave functions, it is discrete concave on $\mathcal{L}$ for each $\textbf{x}\in \mathcal{X}$ and $\textbf{a}\in \mathcal{A}$. Then, for each $\textbf{x}\in \mathcal{X}$, we have

(A.6)\begin{align} v^{m+1}(\textbf{x},\ell-1)+v^{m+1}(\textbf{x},\ell-1)= &\min_{\textbf{a}\in \mathcal{A}}\{Q^m(\textbf{x},\ell-1,\textbf{a})\}+\min_{\textbf{a} \in \mathcal{A}}\{Q^m(\textbf{x},\ell+1,\textbf{a})\} \nonumber \\ \leq &\min_{\textbf{a} \in \mathcal{A}}\{Q^m(\textbf{x},\ell-1,\textbf{a})+Q^m(\textbf{x},\ell+1,\textbf{a})\}\\ \leq &\min_{\textbf{a} \in \mathcal{A}}\{2Q^m(\textbf{x},\ell,\textbf{a})\} = 2\min_{\textbf{a} \in \mathcal{A}}\{Q^m(\textbf{x},\ell,\textbf{a})\} = 2 v^{m+1}(\textbf{x},\ell). \nonumber \end{align}

Inequality (A.6) holds since the min operator preserves the discrete concavity property. By Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell)\to V(\textbf{x},\ell)$ as $m\to\infty$, and the proof is complete.

Proof of Proposition 3.6

Proof. For any given $(\textbf{x},\ell) \in \Gamma$, denote the mth iterate of the VI algorithm associated with parameter w by $v^m(\textbf{x},\ell,w)$. We prove the proposition by induction on m. Take $v^0(\textbf{x},\ell,w) = 0$ for all w and $(\textbf{x},\ell) \in \Gamma$. For each $(\textbf{x},\ell) \in \Gamma$ and any $w \leq w^{\prime}$, we have

\begin{align*} v^1(\textbf{x},\ell,w) = \min_{\textbf{a} \in \mathcal{A}} \left\{c(\textbf{x},\ell,\textbf{a},w) +0\right\} \leq \min_{\textbf{a} \in \mathcal{A}} \left\{c(\textbf{x},\ell,\textbf{a},w^{\prime})+0\right\}=v^1(\textbf{x},\ell,w^{\prime}), \end{align*}

since $c(\textbf{x},\ell,\textbf{a},w)$ is in monotone nondecreasing in w for each $ (\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$. For the induction hypothesis, assume $v^m(\textbf{x},\ell,w) \leq v^m(\textbf{x},\ell,w^{\prime})$. Then, for each $(\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$, we have

\begin{align*} c(\textbf{x},\ell,\textbf{a},w) +\lambda \sum_{k \in \mathcal{L}}v^{m}(\hat{\textbf{x}}(\ell,\textbf{a}),k,w)p_{\ell k} \leq c(\textbf{x},\ell,\textbf{a},w^{\prime})+\lambda \sum_{k \in \mathcal{L}}v^{m}(\hat{\textbf{x}}(\ell,\textbf{a}),k,w^{\prime})p_{\ell k}. \end{align*}

Minimizing both sides over $\textbf{a} \in \mathcal{A}$ results in $v^{m+1}(\textbf{x},\ell,w) \leq v^{m+1}(\textbf{x},\ell,w^{\prime})$. Finally, by Theorem 6.3 of [Reference Puterman54], $v^m(\textbf{x},\ell,w)\to V(\textbf{x},\ell,w)$, as $m\to\infty$, and the proof is complete.

Proof of Theorem 3.7

Proof. For a given $(\textbf{x},\ell) \in \Gamma$ and $\textbf{a} \in \mathcal{A}$, we define the following sets:

\begin{align*} \mathcal{I}(\bf{x},\ell,\bf{a}) &:= \{n \in \mathcal{N} : a^*_n(\bf{x},\ell) =1,\,a_n= 1\},\\ \mathcal{J}(\bf{x},\ell,\bf{a}) &:= \{n \in \mathcal{N} : a^*_n(\bf{x},\ell) =1,\,a_n = 0\},\\ \mathcal{G}(\bf{x},\ell,\bf{a}) &:= \{n \in \mathcal{N} : a^*_n(\bf{x},\ell) =0,\,a_n=1\},\\ \mathcal{H}(\bf{x},\ell,\bf{a}) &:= \{n \in \mathcal{N} : a^*_n(\bf{x},\ell) =0,\,a_n=0\}. \end{align*}

For example, the set $\mathcal{I}(\textbf{x},\ell,\textbf{a})$ is the collection of all turbines for which the corresponding action is replacement in both $\textbf{a}^*(\textbf{x},\ell)$ and a. For notational convenience, we simply write $\mathcal{I}$, $\mathcal{J}$, $\mathcal{G}$, and $\mathcal{H}$ and suppress the dependence of these sets on $(\textbf{x},\ell,\textbf{a})$. Note that, by the definitions (6) and (A.5),

(A.7)\begin{equation} Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell)) \leq Q(\textbf{x},\ell,\textbf{a}). \end{equation}

Next, we show that $c(\textbf{y},\ell,\textbf{a})-c(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell))\ge c(\textbf{x},\ell,\textbf{a})-c(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$.

(A.8)\begin{align} c(\bf{y},\ell,\bf{a})-c(\bf{y},\ell,\bf{a}^*(\bf{x},\ell))= & \;c_s\big[\mathbb{I}(\bf{a})-\mathbb{I}(\bf{a}^*(\bf{x},\ell))\big]+\sum_{n\in\mathcal{G}}\big(c_r +c_d(\ell)\big)-\sum_{n\in \mathcal{G}}c_d(\ell)\tau_n(y_n,\ell) \nonumber \\ & -\sum_{n\in \mathcal{J}}\big(c_r +c_d(\ell)\big)+ \sum_{n\in \mathcal{J}} c_d(\ell)\tau_n(y_n,\ell) \nonumber\\ = & c_s\big[\mathbb{I}(\bf{a})-\mathbb{I}(\bf{a}^*(\bf{x},\ell))\big]+\sum_{n\in \mathcal{G}}c_r - \sum_{n\in \mathcal{G}}c_d(\ell)(\tau_n(y_n,\ell)-1) \nonumber \\ & -\sum_{n\in \mathcal{J}}c_r+ \sum_{n\in \mathcal{J}} c_d(\ell)(\tau_n(y_n,\ell)-1) \nonumber \\ = & c_s\big[\mathbb{I}(\bf{a})-\mathbb{I}(\bf{a}^*(\bf{x},\ell))\big]+\sum_{n\in \mathcal{G}}c_r - \sum_{n\in \mathcal{G}}c_d(\ell)(\tau_n(x_n,\ell)-1) \end{align}
(A.9)\begin{align} & -\sum_{n\in \mathcal{J}}c_r+ \sum_{n\in \mathcal{J}} c_d(\ell)(\tau_n(y_n,\ell)-1) \nonumber \\ \geq & c_s\big[\mathbb{I}(\textbf{a})-\mathbb{I}(\textbf{a}^*(\textbf{x},\ell))\big]+\sum_{n\in \mathcal{G}}c_r-\sum_{n\in\mathcal{G}}c_d(\ell)(\tau_n(x_n,\ell)-1)\\ & -\sum_{n\in \mathcal{J}}c_r+ \sum_{n\in \mathcal{J}} c_d(\ell)(\tau_n(x_n,\ell)-1) \nonumber \\ = & c(\textbf{x},\ell,\textbf{a})-c(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell)). \nonumber \end{align}

Equality (A.8) holds by the definitions of y and $\mathcal{G}$, and inequality (A.9) holds by Lemma 3.1. Moreover, ${\hat{\textbf{y}}}(\ell,\textbf{a}^*(\textbf{x},\ell)) = \hat{\textbf{x}}(\ell,\textbf{a}^*(\textbf{x},\ell))$ and ${\hat{\textbf{y}}}(\ell,\textbf{a}) \geq \hat{\textbf{x}}(\ell,\textbf{a})$ since for all $n \in \mathcal{N}$

\begin{equation*}\hat{y}_n(\ell,\textbf{a})= \begin{cases} f_n(y_n,\ell), & n\in \mathcal{J},\\ f_i(x_n,\ell), & n\in \mathcal{H},\\ 0, & \text{otherwise}, \end{cases}\end{equation*}
\begin{equation*}\hat{y}_n(\ell,\textbf{a}^*(\textbf{x},\ell)) = \begin{cases} f_n(x_n,\ell), & n\in \mathcal{G}\cup \mathcal{H},\\ 0, & \text{otherwise,} \end{cases}\end{equation*}
\begin{equation*}{\hat{\textbf{x}}}_n(\ell,\textbf{a}) = \begin{cases} f_n(x_n,\ell), & n\in \mathcal{J}\cup \mathcal{H},\\ 0, & \text{otherwise,} \end{cases}\end{equation*}
and
\begin{equation*}{\hat{\textbf{x}}}_n(\ell,\textbf{a}^*(\textbf{x},\ell))= \begin{cases} f_n(x_n,\ell), & n \in \mathcal{G}\cup \mathcal{H},\\ 0, & \text{otherwise,} \end{cases} \end{equation*}

where $f_n(x_n,\ell),\ n\in \mathcal{N}$ is assumed to be monotone increasing in xn. Subsequently, we have

(A.10)\begin{equation} \lambda \sum_{k\in \mathcal{L}}V({\hat{\textbf{y}}}(\ell,\textbf{a}^*(\textbf{x},\ell)),k)p_{\ell k} = \lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\ell,\textbf{a}^*(\textbf{x},\ell)),k)p_{\ell k}, \end{equation}

and by Proposition 3.5(a), we have

(A.11)\begin{equation} \lambda \sum_{k\in \mathcal{L}}V({\hat{\textbf{y}}}(\ell,\textbf{a}),k)p_{\ell k}\geq \lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\ell,\textbf{a}),k)p_{\ell k}, \end{equation}

since ${\hat{\textbf{y}}}(\ell,\textbf{a}) \geq \hat{\textbf{x}}(\ell,\textbf{a})$. Thus, for any given a, we have

\begin{align} \nonumber Q(\bf{y},\ell,\bf{a})-Q(\bf{y},\ell,\bf{a}^*(\bf{x},\ell))= & \; \left[c(\bf{y},\ell,\bf{a})+\lambda \sum_{k\in \mathcal{L}}V({\hat{\bf{y}}}(\ell,\bf{a}),k),k)p_{\ell k}\right] \end{align}
\begin{align} &\nonumber - \left[c(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell))+\lambda \sum_{k\in \mathcal{L}}V({\hat{\textbf{y}}}(\ell,\textbf{a}^*(\textbf{x},\ell)),k)p_{\ell k}\right] \end{align}
(A.12)\begin{align} \geq & \left[c(\textbf{x},\ell,\textbf{a})+\lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\ell,\textbf{a}),k),k)p_{\ell k}\right] \end{align}
\begin{align} &\nonumber - \left[c(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))+\lambda \sum_{k\in \mathcal{L}}V(\hat{\textbf{x}}(\ell,\textbf{a}^*(\textbf{x},\ell)),k)p_{\ell k}\right] \end{align}
\begin{align} = \nonumber & Q(\textbf{x},\ell,\textbf{a})-Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell)) \geq 0. \end{align}

Inequality (A.12) holds by Equations (A.9), (A.10), and (A.11). From the above argument, we conclude

(A.13)\begin{equation} Q(\textbf{y},\ell,\textbf{a})\ge Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell)), \end{equation}

which implies

(A.14)\begin{equation} \bf{a}^*(\bf{x},\ell)\in \mathrm{argmin}_{\bf{a} \in \mathcal{A}}\,Q(\bf{y},\ell,\bf{a}). \end{equation}

Furthermore, by Definition (6),

(A.15)\begin{equation} \bf{a}^*(\bf{y},\ell) \in \mathrm{argmin}_{\bf{a} \in \mathcal{A}}\,Q(\bf{y},\ell,\bf{a}). \end{equation}

Therefore, by Equations (A.14) and (A.15), we have

(A.16)\begin{equation} Q(\textbf{y},\ell,\textbf{a}^*(\textbf{y},\ell)) = Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell)). \end{equation}

Equality (A.16) implies that, if the degradation state is y, taking the optimal action associated with x leads to the minimum cost. In order to show that $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$, it is sufficient to prove that there does not exist any $\textbf{a}^{\prime}\in \mathcal{A}$ such that

(A.17)\begin{equation} \textbf{a}^{\prime} \lt \textbf{a}^*(\textbf{x},\ell)\quad\text{and}\quad Q(\textbf{y},\ell,\textbf{a}^{\prime})=Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell)). \end{equation}

We will prove this assertion by contradiction. Suppose there exists an $\textbf{a}^{\prime}\in \mathcal{A}$ satisfying Equation (A.17), and let us compare $Q(\textbf{x},\ell,\textbf{a}^{\prime})$ and $Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$ under two distinct cases:

Case 1: Suppose $Q(\textbf{x},\ell,\textbf{a}^{\prime}) \leq Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$. In this case, by Definition (6), we conclude $\textbf{a}^*(\textbf{x},\ell) \leq \textbf{a}^{\prime}$, which contradicts the inequality in Equation (A.17).

Case 2: Suppose $Q(\textbf{x},\ell,\textbf{a}^{\prime}) \gt Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell))$. Similar to the arguments above, for the given $\textbf{a}^{\prime} \lt \textbf{a}^*(\textbf{x},\ell)$, by inequality (A.13), we see that

\begin{equation*} Q(\textbf{y},\ell,\textbf{a}^{\prime})-Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell)) \geq Q(\textbf{x},\ell,\textbf{a}^{\prime})-Q(\textbf{x},\ell,\textbf{a}^*(\textbf{x},\ell)) \gt 0. \end{equation*}

That is, $Q(\textbf{y},\ell,\textbf{a}^{\prime}) \gt Q(\textbf{y},\ell,\textbf{a}^*(\textbf{x},\ell))$, which contradicts the equality in Equation (A.17). As a contradiction occurs in both cases, we conclude that there is no $\textbf{a}^{\prime}\in \mathcal{A}$ satisfying the conditions in Equation (A.17); therefore, $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$.

Proof of Corollary 3.8

Proof. For a given $n^{\prime} \in \mathcal{N}$, if $a^*_{n'}(\textbf{x},\ell) = 1$, by Theorem 3.7, $\textbf{a}^*(\textbf{y},\ell)=\textbf{a}^*(\textbf{x},\ell)$ for a y such that $y_{n^{\prime}}\geq x_{n^{\prime}}$ and $y_n=x_n$ for $n \in \mathcal{N} \setminus \{n^{\prime}\}$, that is, $a^*_{n^{\prime}}(\textbf{y},\ell) = 1$. If $a^*_{n^{\prime}}(\textbf{x},\ell) = 0$, then $a^*_{n^{\prime}}(\textbf{y},\ell) \in \{0,1\}$ for any $\textbf{y} \geq \textbf{x}$. Therefore, $a^*_{n^{\prime}}(\textbf{x},\ell)$ is monotone nondecreasing in $x_{n^{\prime}}$ for each xn, $n \in \mathcal{N} \setminus \{n^{\prime}\}$ and $\ell$.

Proof of Proposition 3.9

Proof. For a given $\ell \in \mathcal{L}$, $\textbf{a}^*(1,1,\ell)=(1,1)$, thus $b_1(1,\ell)$ exists, and we have

(A.18)\begin{equation} \textbf{a}^*(x_1,1,\ell)=(1,1), \quad x_1\geq b_1(1,\ell), \end{equation}

and by Definition (8)

(A.19)\begin{equation} \textbf{a}^*(x_1,1,\ell)\neq (1,1), \quad x_1 \lt b_1(1,\ell). \end{equation}

By Corollary 3.8 and Equation (A.18), the replacement threshold $x^*_2(x_1,\ell)$ exists on the degradation space $[b_1(1,\ell),1] \times [0,1]$. For a given $(x_1,x_2)$ in the degradation space $[0,b_1(1,\ell))\times [0,1)$, if $\textbf{a}^*(x_1,x_2,\ell)=(1,1)$, then by Theorem 3.7, $\textbf{a}^*(x_1,1,\ell)=(1,1)$, which contradicts Equation (A.19). Hence, there is no point in space $[0,b_1(1,\ell)) \times [0,1)$, for which the optimal action is to replace both turbines, that is,

\begin{equation*}\textbf{a}^*(x_1,x_2,\ell)\neq (1,1), \qquad (x_1,x_2) \in [0,b_1(1,\ell)) \times [0,1).\end{equation*}

By Theorem 3.7 and Equation (A.18), we have

\begin{equation*}\textbf{a}^*(x_1,x_2,\ell)\neq (0,1),\qquad (x_1,x_2)\in [b_1(1,\ell),1]\times [0,1).\end{equation*}

That is,

(A.20)\begin{equation} \textbf{a}^*(x_1,x_2,\ell)\in \{(0,0),(1,0),(1,1)\},\qquad (x_1,x_2)\in [b_1(1,\ell),1]\times [0,1). \end{equation}

By contradiction, assume that $x^*_2(x_1,\ell)$ is strictly increasing in x 1. Then, there must exist a point $(x_1,x_2) \in [b_1(1,\ell),1) \times [0,1]$ for which $a^*_2(x_1,x_2)=1$, and

(A.21)\begin{equation} a^*_2(x_1+\epsilon,x_2)=0 \end{equation}

for any ϵ > 0. If $a^*_2(x_1,x_2)=1$, then by Equation (A.20), $\textbf{a}^*(x_1,x_2)=(1,1)$, and by Theorem 3.7, $\textbf{a}^*(x_1+\epsilon,x_2)=(1,1)$, which contradicts Equation (A.21).

Value iteration algorithm

References

Abbou, A. & Makis, V. (2019). Group maintenance: A restless bandits approach. INFORMS Journal on Computing 31(4): 719731.CrossRefGoogle Scholar
Abdul-Malak, D.T. & Kharoufeh, J.P. (2018). Optimally replacing multiple systems in a shared environment. Probability in the Engineering and Informational Sciences 32(2): 179206.CrossRefGoogle Scholar
Aflaki, S. & Netessine, S. (2017). Strategic investment in renewable energy sources: The effect of supply intermittency. Manufacturing & Service Operations Management 19(3): 489507.CrossRefGoogle Scholar
Anastasiou, K. & Tsekos, C. (1996). Persistence statistics of marine environmental parameters from Markov theory, Part 1: Analysis in discrete time. Applied Ocean Research 18(4): 187199.CrossRefGoogle Scholar
Andersen, J.F., Andersen, A.R., Kulahci, M., & Nielsen, B.F. (2022). A numerical study of Markov decision process algorithms for multi-component replacement problems. European Journal of Operational Research 299(3): 898909.CrossRefGoogle Scholar
Andrawus, J.A., Watson, J., & Kishk, M. (2007). Modelling system failures to optimise wind turbine maintenance. Wind Engineering 31(6): 503542.CrossRefGoogle Scholar
Arts, J. & Basten, R. (2018). Design of multi-component periodic maintenance programs with single-component models. IISE Transactions 50(7): 606615.CrossRefGoogle Scholar
Bern, T. & Houmb, O. (1984). Simulation of offshore operations by a combined statistical physical model. In Proceedings of the Third International Offshore Mechanics and Arctic Engineering Symposium, New Orleans, LA, American Society of Mechanical Engineers (ASME), .Google Scholar
Berrade, M.D., Scarf, P.A., & Cavalcante, C.A. (2018). Conditional inspection and maintenance of a system with two interacting components. European Journal of Operational Research 268(2): 533544.CrossRefGoogle Scholar
Besnard, F. & Bertling, L. (2010). An approach for condition-based maintenance optimization applied to wind turbine blades. IEEE Transactions on Sustainable Energy 1(2): 7783.CrossRefGoogle Scholar
Bian, L., Gebraeel, N.Z., & Kharoufeh, J.P. (2015). Degradation modeling for real-time estimation of residual lifetimes in dynamic environments. IIE Transactions 47(5): 471486.CrossRefGoogle Scholar
Byon, E. & Ding, Y. (2010). Season-dependent condition-based maintenance for a wind turbine using a partially observed Markov decision process. IEEE Transactions on Power Systems 25(4): 18231834.CrossRefGoogle Scholar
Byon, E., Ntaimo, L., & Ding, Y. (2010). Optimal maintenance strategies for wind turbine systems under stochastic weather conditions. IEEE Transactions on Reliability 59(2): 393404.CrossRefGoogle Scholar
Cha, J.H., Finkelstein, M., & Levitin, G. (2018). Bivariate preventive maintenance of systems with lifetimes dependent on a random shock process. European Journal of Operational Research 266(1): 122134.CrossRefGoogle Scholar
Dalgic, Y., Lazakis, I., Dinwoodie, I., McMillan, D., & Revie, M. (2015). Advanced logistics planning for offshore wind farm operation and maintenance activities. Ocean Engineering 101: 211226.CrossRefGoogle Scholar
Dalgic, Y., Lazakis, I., & Turan, O. (2013). Vessel charter rate estimation for offshore wind O&M activities. In Proceedings of International Maritime Association of Mediterranean IMAM 2013, .Google Scholar
Dalgic, Y., Lazakis, I., Turan, O., & Judah, S. (2015). Investigation of optimum jack-up vessel chartering strategy for offshore wind farm O&M activities. Ocean Engineering 95: 106115.CrossRefGoogle Scholar
de Pater, I. & Mitici, M. (2021). Predictive maintenance for multi-component systems of repairables with remaining-useful-life prognostics and a limited stock of spare components. Reliability Engineering & System Safety 214: .CrossRefGoogle Scholar
Dekker, R., Wildeman, R., & van der Duyn Schouten, F. (1997). A review of multi-component maintenance models with economic dependence. Mathematical Methods of Operations Research 45(3): 411435.CrossRefGoogle Scholar
Department of Energy. (2021). 2021: DOE moves at lightning speed toward clean energy goals. Technical Report, Washington, D.C.: U.S. Department of Energy (DOE).Google Scholar
Ding, Y. (2019). Data Science for Wind Energy, Boca Raton, FL: CRC Press.CrossRefGoogle Scholar
Dong, X.H., Yuan, T.J., & Ma, R.H. (2013). Corrosion mechanism on offshore wind turbine blade in salt fog environment. Design and Manufacture in Mechanical Engineering, Vol. 432. Applied Mechanics and Materials, Stafa-Zurich: Trans Tech Publications Ltd. .Google Scholar
Flory, J.A., Kharoufeh, J.P., & Gebraeel, N.Z. (2014). A switching diffusion model for lifetime estimation in randomly varying environments. IIE Transactions 46(11): 12271241.CrossRefGoogle Scholar
Gebraeel, N.Z., Lawley, M.A., Li, R., & Ryan, J.K. (2005). Residual-life distributions from component degradation signals: A Bayesian approach. IIE Transactions 37(6): 543557.CrossRefGoogle Scholar
Hofmann, M. (2011). A review of decision support models for offshore wind farms with an emphasis on operation and maintenance strategies. Wind Engineering 35(1): 115.CrossRefGoogle Scholar
Hwangbo, H., Johnson, A., & Ding, Y. (2017). A production economics analysis for quantifying the efficiency of wind turbines. Wind Energy 20(9): 15011513.CrossRefGoogle Scholar
IEC. (2022). Wind energy generation systems–Part 12-1: Power performance measurements of electricity producing wind turbines. Technical Report, International Electrotechnical Commission, https://webstore.iec.ch/publication/68499#additionalinfo.Google Scholar
Jiang, L., Feng, Q. & Coit, D.W. (2012). Reliability and maintenance modeling for dependent competing failure processes with shifting failure thresholds. IEEE Transactions on Reliability 61(4): 932948.CrossRefGoogle Scholar
Jones, M. (2005). Estimating Markov transition matrices using proportions data: An application to credit risk. Technical Report, Washington, DC: International Monetary Fund (IMF).Google Scholar
Kök, A.G., Shang, K. & Yücel, S. (2020). Investments in renewable and conventional energy: The role of operational flexibility. Manufacturing & Service Operations Management 22(5): 925941.CrossRefGoogle Scholar
Kharoufeh, J.P. & Cox, S.M. (2005). Stochastic models for degradation-based reliability. IIE Transactions 37(6): 533542.CrossRefGoogle Scholar
Kharoufeh, J.P., Finkelstein, D., & Mixon, D.G. (2006). Availability of periodically inspected systems with Markovian wear and shocks. Journal of Applied Probability 43(2): 303317.CrossRefGoogle Scholar
Kharoufeh, J.P. & Mixon, D.G. (2009). On a Markov-modulated shock and wear process. Naval Research Logistics 56(6): 563576.CrossRefGoogle Scholar
Ko, Y.M. & Byon, E. (2017). Condition-based joint maintenance optimization for a large-scale system with homogeneous units. IISE Transactions 49(5): 493504.CrossRefGoogle Scholar
Kurt, M. & Kharoufeh, J.P. (2010). Monotone optimal replacement policies for a Markovian deteriorating system in a controllable environment. Operations Research Letters 38(4): 273279.CrossRefGoogle Scholar
Kurt, M. & Kharoufeh, J.P. (2010). Optimally maintaining a Markovian deteriorating system with limited imperfect repairs. European Journal of Operational Research 205(2): 368380.CrossRefGoogle Scholar
Kushner, H.J. & Dupuis, P. (2001). Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd edn, New York, NY: Springer Science + Business Media.CrossRefGoogle Scholar
Liu, B., Pandey, M.D., Wang, X., & Zhao, X. (2021). A finite-horizon condition-based maintenance policy for a two-unit system with dependent degradation processes. European Journal of Operational Research 295(2): 705717.CrossRefGoogle Scholar
MacCallum, K.J. & Bolland, J. (1985). The use of transition matrices for weather simulation. Marine Technology and SNAME News 22(3): 238244.CrossRefGoogle Scholar
Maples, B., Saur, G., Hand, M., van de Pietermen, R., & Obdam, T. (2013). Installation, operation, and maintenance strategies to reduce the cost of offshore wind energy. Technical Report, Golden, CO.: National Renewable Energy Laboratory (NREL).Google Scholar
McGuire, T.W. (1969). More on least squares estimation of the transition matrix in a stationary first-order Markov process from sample proportions data. Psychometrika 34(3): 335345.CrossRefGoogle Scholar
McMorland, J., Flannigan, C., Carroll, J., Collu, M., McMillan, D., Leithead, W., & Coraddu, A. (2022). A review of operations and maintenance modelling with considerations for novel wind turbine concepts. Renewable and Sustainable Energy Reviews 165: .CrossRefGoogle Scholar
Milgrom, P. & Shannon, C. (1994). Monotone comparative statics. Econometrica 62(1): 157180.CrossRefGoogle Scholar
Monbet, V., Ailliot, P., & Prevosto, M. (2007). Survey of stochastic models for wind and sea state time series. Probabilistic Engineering Mechanics 22(2): 113126.CrossRefGoogle Scholar
Murota, K. (1998). Discrete convex analysis. Mathematical Programming 83(1): 313371.CrossRefGoogle Scholar
Murota, K. & Tamura, A. (2022). Discrete Fenchel duality for a pair of integrally convex and separable convex functions. Japan Journal of Industrial and Applied Mathematics 39(2): 599630.CrossRefGoogle Scholar
Musial, W., Spitsen, P., Duffy, P., Beiter, P., Marquis, M., Hammond, R., & Shields, M. (2022). Offshore wind market report: 2022 edition. Technical Report, Golden, CO: National Renewable Energy Laboratory (NREL).Google Scholar
Nielsen, J.J. & Sørensen, J.D. (2010). Bayesian networks as a decision tool for O& M of offshore wind turbines. In ASRANet: Integrating Structural Analysis, Risk & Reliability: 5th International ASRANet Conference, Edinburgh, UK, Sutton, London: ASRANet Ltd.Google Scholar
Ohashi, M. & Nishida, T. (1981). Optimal replacement policy for two-unit system. Journal of the Operations Research Society of Japan 24(4): 283295.CrossRefGoogle Scholar
Olde Keizer, M., Flapper, S.D.P., & Teunter, R.H. (2017). Condition-based maintenance policies for systems with multiple dependent components: A review. European Journal of Operational Research 261(2): 405420.CrossRefGoogle Scholar
Olde Keizer, M., Teunter, R.H., & Veldman, J. (2017). Joint condition-based maintenance and inventory optimization for systems with multiple components. European Journal of Operational Research 257(1): 209222.CrossRefGoogle Scholar
Papadopoulos, P., Coit, D.W., & Ezzat, A.A. (2022). Seizing opportunity: Maintenance optimization in offshore wind farms considering accessibility, production, and crew dispatch. IEEE Transactions on Sustainable Energy 13(1): 111121.CrossRefGoogle Scholar
Papadopoulos, P., Coit, D.W., & Ezzat, A.A. (2023). STOCHOS: Stochastic opportunistic maintenance scheduling for offshore wind farms. IISE Transactions, https://www.tandfonline.com/doi/full/10.1080/24725854.2022.2152913.Google Scholar
Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming, Hoboken, NJ: John Wiley & Sons.Google Scholar
Rafiee, K., Feng, Q., & Coit, D.W. (2014). Reliability modeling for dependent competing failure processes with changing degradation rate. IIE Transactions 46(5): 483496.CrossRefGoogle Scholar
Rafiee, K., Feng, Q., & Coit, D.W. (2015). Condition-based maintenance for repairable deteriorating systems subject to a generalized mixed shock model. IEEE Transactions on Reliability 64(4): 11641174.CrossRefGoogle Scholar
Ren, Z., Verma, A.S., Li, Y., Teuwen, J.J., & Jiang, Z. (2021). Offshore wind turbine operations and maintenance: A state-of-the-art review. Renewable and Sustainable Energy Reviews 144: .CrossRefGoogle Scholar
Rinaldi, G., Thies, P.R., & Johanning, L. (2021). Current status and future trends in the operation and maintenance of offshore wind turbines: A review. Energies 14(9): .CrossRefGoogle Scholar
Salari, N. & Makis, V. (2020). Application of Markov renewal theory and semi-Markov decision processes in maintenance modeling and optimization of multi-unit systems. Naval Research Logistics 67(7): 548558.CrossRefGoogle Scholar
Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics 6(2): 461464.CrossRefGoogle Scholar
Seyr, H. & Muskulus, M. (2019). Decision support models for operations and maintenance for offshore wind farms: A review. Applied Sciences 9(2): .CrossRefGoogle Scholar
Shafiee, M. (2015). Maintenance logistics organization for offshore wind energy: Current progress and future perspectives. Renewable Energy 77: 182193.CrossRefGoogle Scholar
Shafiee, M. & Sørensen, J.D. (2019). Maintenance optimization and inspection planning of wind energy assets: Models, methods and strategies. Reliability Engineering & System Safety 192: .CrossRefGoogle Scholar
Shields, M., Marsh, R., Stefek, J., Oteri, F., Gould, R., Rouxel, N., Diaz, K., Molinero, J., Moser, A., & Malvik, C. (2022). The demand for a domestic offshore wind energy supply chain. Technical Report, Golden, CO: National Renewable Energy Laboratory (NREL).CrossRefGoogle Scholar
Short, W., Packey, D.J., & Holt, T. (1995). A manual for the economic evaluation of energy efficiency and renewable energy technologies. Technical Report, Golden, CO: National Renewable Energy Laboratory (NREL).Google Scholar
Song, Z., Geng, X., Kusiak, A., & Xu, C. (2011). Mining Markov chain transition matrix from wind speed time series data. Expert Systems with Applications 38(8): 1022910239.CrossRefGoogle Scholar
Stehly, T., Beiter, P., & Duffy, P. (2020). 2019 Cost of wind energy review. Technical Report, Golden, CO.: National Renewable Energy Laboratory (NREL).CrossRefGoogle Scholar
Tian, Z. & Liao, H. (2011). Condition-based maintenance optimization for multi-component systems using proportional hazards model. Reliability Engineering and System Safety 96(5): 581589.CrossRefGoogle Scholar
uit het Broek, M.A., Teunter, B.J.R.H., & Veldman, J. (2021). Joint condition-based maintenance and load-sharing optimization for two-unit systems with economic dependency. European Journal of Operational Research 295(3): 11191131.CrossRefGoogle Scholar
uit het Broek, M.A., Veldman, J., Fazi, S., & Greijdanus, R. (2019). Evaluating resource sharing for offshore wind farm maintenance: The case of jack-up vessels. Renewable and Sustainable Energy Reviews 109: 619632.CrossRefGoogle Scholar
Ulukuş, M.Y., Kharoufeh, J.P., & Maillart, L.M. (2012). Optimal replacement policies under environment-driven degradation. Probability in the Engineering and Informational Sciences 26(3): 405424.CrossRefGoogle Scholar
Wan, Y.-H., Ela, E., & Orwig, K. (2010). Development of an equivalent wind plant power-curve. Technical Report, Golden, CO.: National Renewable Energy Laboratory (NREL).Google Scholar
Wang, C., Hou, Y., Qin, Z., Peng, C., & Zhou, H. (2015). Dynamic coordinated condition-based maintenance for multiple components with external conditions. IEEE Transactions on Power Delivery 30(5): 23622370.CrossRefGoogle Scholar
Wiser, R. & Bolinger, M. (2017). Wind technologies market. Technical Report DOE/EE-1798, Washington, D.C.: U.S. Department of Energy (DOE).Google Scholar
Yousefi, N., Coit, D.W., & Zhu, X. (2020). Dynamic maintenance policy for systems with repairable components subject to mutually dependent competing failure processes. Computers & Industrial Engineering 143: .CrossRefGoogle Scholar
Zhang, C., Li, Y.-F., & Coit, D.W. (2023). Deep reinforcement learning for dynamic opportunistic maintenance of multi-component systems with load sharing. IEEE Transactions on Reliability 72(3): 863877.Google Scholar
Zhang, N., Deng, Y., Liu, B., & Zhang, J. (2023). Condition-based maintenance for a multi-component system in a dynamic operating environment. Reliability Engineering & System Safety 231: .CrossRefGoogle Scholar
Zhang, N., Fouladirad, M., & Barros, A. (2017). Maintenance analysis of a two-component load-sharing system. Reliability Engineering & System Safety 167: 6774.CrossRefGoogle Scholar
Zhang, N., Fouladirad, M., Barros, A., & Zhang, J. (2020). Condition-based maintenance for a K-out-of-N deteriorating system under periodic inspection with failure dependence. European Journal of Operational Research 287(1): 159167.CrossRefGoogle Scholar
Zhang, Z., Wu, S., Li, B., & Lee, S. (2015). (n, N)-type maintenance policy for multi-component systems with failure interactions. International Journal of Systems Science 46(6): 10511064.CrossRefGoogle Scholar
Zhu, Q., Peng, H., & van Houtum, G. (2015). A condition-based maintenance policy for multi-component systems with a high maintenance setup cost. OR Spectrum 37(4): 10071035.CrossRefGoogle Scholar
Zhu, Z. & Xiang, Y. (2021). Condition-based maintenance for multi-component systems: Modeling, structural properties, and algorithms. IISE Transactions 53(1): 88100.CrossRefGoogle Scholar
Zhu, Z., Xiang, Y., & Zeng, B. (2021). Multicomponent maintenance optimization: A stochastic programming approach. INFORMS Journal on Computing 33(3): 898914.CrossRefGoogle Scholar
Figure 0

Figure 1. Visualization of the optimal replacement policy of turbine 2. (a) Monotone policy in its own degradation level. (b) Monotone policy in the other turbine’s degradation level. (c) Monotonicity of thresholds.

Figure 1

Figure 2. Partition of wind speed into disjoint intervals with distinct downtime cost rates. (a) A typical wind power curve. (b) Downtime cost rate versus environment.

Figure 2

Figure 3. Degradation evolution of a turbine within a period with and without replacement. (a) A turbine with replacement. (b) A turbine without replacement.

Figure 3

Figure 4. Extension of optimal actions at degradation level x to higher degradation levels. (a) $\textbf{a}^*(\textbf{x},\ell)=(1,1,0)$ in shaded region. (b) $\textbf{a}^*(\textbf{x},\ell)=(1,1,1)$ in shaded region.

Figure 4

Figure 5. Extension of the optimal actions for two turbines in a fixed environment state.

Figure 5

Figure 6. Monotone threshold policy for turbine 2 in a wind farm with two turbines.

Figure 6

Table 1. Downtime cost and degradation rates for each environment state.

Figure 7

Figure 7. Baseline example optimal replacement policy for each environment state ($c_s=5$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

Figure 8

Figure 8. Baseline example optimal replacement policy for each environment state ($c_s=80$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

Figure 9

Table 2. Summary of the three distinct models for the replacement problem.

Figure 10

Figure 9. Baseline example optimal replacement policy for each environment state ($c_s=80$): (a) environment state $\ell=1$, (b) environment state $\ell=2$, (c) environment state $\ell=3$, (d) environment state $\ell=4$, (e) environment state $\ell=5$, and (f) environment state $\ell=6$.

Figure 11

Table 3. Downtime cost and degradation rates for the environment states of Model 2.

Figure 12

Figure 10. Model 2 optimal replacement policy for each environment state. (a) Environment state $\ell=1$. (b) Environment state $\ell=2$.

Figure 13

Table 4. Total discounted costs for six distinct initial states.

Figure 14

Table 5. Total discounted cost comparison: Optimal policy versus three heuristic policies.