Social impact evaluation analysis: a systematic qualitative method for assessing engineered products

Abstract

Social impact has been widely discussed by the engineering community, but studies show that there is currently little systematic consideration of the social impact of products in both academia and in industry beyond social impacts on health and safety. While Failure Mode and Effect Analysis (FMEA) is useful for evaluating health and safety risks, new developments are needed to create an FMEA-style evaluation that can be applied to a wide range of social impacts for engineered products. The authors describe necessary modifications to traditional FMEA that transform it into a tool for social impact analysis. The modification of FMEA involves the introduction of positive and negative impacts, the inclusion of discrete and continuous impacts, the consideration of various stakeholder types, and the inclusion of uncertainty in place of detectability. This modified FMEA is referred to in this paper as Social Impact Effects Analysis (SIEA). The paper describes how SIEA is performed and articulates the potential benefits of SIEA.

1. Introduction

Sustainable design encapsulates economic, environmental, and social sustainability. Economic considerations are routinely included in engineering design (Asiedu and Gu, 1998), and environmental considerations are increasingly being considered in design with standards and widely used methodology (Hendrickson et al., 1998; Joshi, 1999). While social impact considerations are being broadly researched (Phillips et al., 2024; Hallstedt et al., 2023; Udoewa, 2022; Richards et al., 2023), research reveals that there is a lack of implementation in engineering-centered academia and industry (Pack et al., 2020; Burleson et al., 2020).

The lack of implementation is not due to a lack of methods. There are many methodologies that are used in social science and global development fields, including 9 methodologies reviewed by Bonilla-Alicea and Fu (Bonilla-Alicea and Fu, 2019) and 18 reviewed by Burleson et al. (Burleson et al., 2020). Some reasons these methods are not widely used in engineering-centered product development are designers not being aware of methods, methods not being widely applicable, and methods being too complicated or time-consuming (Pack et al., 2020; Burleson et al., 2020).

In an intensive literature survey that encompassed multi-disciplinary publications, Rainock et al. categorized the impacts that engineered products can have on society into 11 social impact categories (Rainock et al., 2018). Those categories are:

1. i. Impacts on Conflict & Crime

2. ii. Impacts on Cultural Heritage & Identity

3. iii. Impacts on Education

4. iv. Impacts on Family

5. v. Impacts on Gender

6. vi. Impacts on Health & Safety

7. vii. Impacts on Human Rights

8. viii. Impacts on Paid Work

9. ix. Impacts on Population Change

10. x. Impacts on Social Networks & Communication

11. xi. Impacts on Stratification

Through a systematic review of industrial practice, Pack et. al found that Impacts on Health and Safety are the most widely considered/implemented in engineering practice and that Failure Mode and Effects Analysis (FMEA) is the predominant tool used by engineers for considering such impacts (Pack et al., 2020).

FMEA is a methodical, qualitative method for evaluating primarily the Health and Safety impacts of component failure in engineered systems (Mikulak et al., 2017). When carrying out an FMEA, designers methodically rate product components, their potential failure modes, and possible effects to determine risk associated with the product. As part of the FMEA process, designers list out many possible failure modes for each component in a system and rate each failure mode by its likelihood of occurrence. Designers then list out the negative effects for each failure mode and rate the severity of each effect. Designers also rate each failure mode on how easily each failure can be detected and prevented beforehand. The three ratings (occurrence, severity, and detectability) are used to calculate an overall risk priority number (RPN). The RPNs of each failure mode and effect are used to determine which failure modes need additional design, manufacturing, or quality control efforts to lower risk to acceptable levels.

FMEA was initially created by the United States Military as a means of managing risk and preventing future failures. Later, it was adopted by NASA in the 1960’s, the automotive industry in the 1970’s, and has since expanded in use to many industries, including the medical device industry, the construction industry, the consumer electronics industry, and so forth. Many of these, including the automotive and medical device industries, require the use of FMEA as part of meeting industry standards (Carlson, 2014; Mikulak et al., 2017).

Although FMEA has been widely used and generally accepted, few attempts have been made in archival journals to examine the validity of FMEA. The lack of archival research supporting FMEA, however, does not mean that FMEA is not an accepted and useful method (Shebl et al., 2012; Breiing and Kunz, 2002). FMEA has been validated by industry for decades and its consistent use demonstrates that FMEA is a valued, trusted method (Mikulak et al., 2017).

The simplicity, consistency, and ability to identify factors leading to failures and corresponding corrective actions early in the design process make FMEA popular (Stamatis, 2003). FMEA is also valued because the risks identified by FMEA can be tracked through testing and post-product release as part of Quality Management Systems (QMS) and Risk Management Systems (Carbone and Tippett, 2004; Dudek-Burlikowska, 2011).

While FMEA is valuable for identifying failure modes and preventative actions, it cannot be directly applied to social impact evaluation. In this paper, the authors introduce a new method inspired by FMEA. The new method uses the traditional FMEA structure with essential modifications/extensions that allow any product to be analyzed relative to any of the 11 social impact categories. To avoid confusion with traditional FMEA, the new method is referred to as Social Impacts Effects Analysis (SIEA). In order to aid readability and provide a basis for understanding how SIEA works, the paper is organized by first presenting the limitations of FMEA as it applies to social impact evaluation (Section 2), then a step-by-step process for performing an SIEA is presented in Section 3; it is in this section that new developments are introduced. This is followed by an example SIEA in Section 4. Section 5 discusses the new developments in further detail after the reader is introduced to the method.

2. Limitations of FMEA for social impact evaluation

There are five areas identified by the authors where new developments are needed in order to make FMEA useful for evaluating the full range of social impact categories identified by Rainock et al. (Rainock et al., 2018). They are:

• • Range of Impact Consideration- FMEA, by design, typically analyzes Health & Safety related risks. Health and Safety, however, is only 1 of the 11 Social Impact Categories (Rainock et al., 2018). Furthermore, products with social impacts typically have impacts in many categories (Ottosson et al., 2020). While FMEA in its current form can be used for analyzing risk in other social impact categories, these instances have been rare exceptions and were limited in their consideration. Literature reviewed for this found FMEAs that analyzed social impacts in 8 of 11 categories (see Section 5.1), considering between 1 and 2 impact categories at a time. FMEA’s focus on Health and Safety, as well as the limited range of consideration, is insufficient to fully understand a product’s complete social impact and would limit a design team’s evaluation.

• • Duality of Product Effects- Products can have both positive and negative impacts with large effects on stakeholders (Benoît et al., 2010; Fontes et al., 2018). These impacts can be intentional or unintentional and should be considered in the design of a product (Stevenson et al., 2018; Stringham et al., 2020). However, FMEA typically deals with the occurrence of negative events and not with the positive impacts of products (Ginn et al., 1998). While this is valuable for FMEA’s designated purpose of considering failure modes, it does not help a designer consider the full positive and negative social impacts of their products. Not only does understanding the positive impacts help a designer better understand their product, it also allows a designer to consider how the positive and negative impacts are related, as well as the trade-offs between all impacts.

• • Consideration of Stakeholders- Social impacts are highly dependent on context and the differences between users, communities, and environments (Vanclay, 2002; Fontes, 2016; Epstein and Yuthas, 2014). Social impacts also depend on both the product and the user, which requires different considerations for differing stakeholders (Stevenson et al., 2018). FMEA generally considers a single stakeholder group, with the severity of failure effects being the same for most users in the stakeholder group. In cases where differences in magnitude occur, FMEA typically considers only the worst-case scenario (Stamatis, 2003). Considering only a single stakeholder group limits a designer from fully understanding the social impact of their product across different segments of society.

• • Pattern of Occurrences- The social impact of products can occur both discretely (a single occurrence) and continuously (across extended periods of time). In fact, social sustainability is based on the notion that products can impact individuals and communities over an extended period of time (Graham Trotman/Martinus Nijhoff, 1993; Desa and Koch, 2014). FMEA is designed to deal with the risks of failure events, which are single discrete occurrences (Rose, 2013). These discrete events have varying frequencies and can occur individually or concurrently (Sharma and Srivastava, 2018). While this works well for FMEA’s current usage analyzing failure modes, it would limit a designer from seeing all impacts their product may have. Examining both discrete and continuous occurrences creates a more complete view of a product’s immediate and sustained impact on users.

• • Uncertainty of Impact- FMEA typically only handles uncertainty related to detecting randomly occurring events, as shown in every failure mode being given a detectability score. However, the social impact of products contains many sources of uncertainty, such as uncertainty related to product adoption and use (Cohen et al., 2016), level of impact (Stevenson et al., 2018), verification of impact occurrence, and uncertainty related to secondary effects or impacts (Jacobsen, 2015; Freeman, 2007). Broadening this area of consideration allows for a better understanding of how all the uncertainty related to a product could lead to an impact on people’s lives.

In contrast to traditional FMEA, SIEA encourages the consideration of a wide range of social impact categories, allows for both the negative and positive impacts of products, allows impact magnitude to be different for different user types, allows for impacts that are ongoing and not just one-time events, and considers additional types of uncertainty.

It is important to remember that, like any social impact assessment or design methodology, the success of the proposed SIEA is linked to the people performing the activity. Exploring the social impact of a product works better when those who know the problem deeply, both technically and socially, are involved (Wood et al., 2014). More diverse teams, such as multidisciplinary teams (Van Der Vegt and Bunderson, 2005), and teams that have diversity of gender, race, and ethnicity perform better (Ancona and Caldwell, 1992; Bear and Woolley, 2011). Working with those directly impacted and using a diverse team are recommended as part of performing SIEA.

3. Performing a social impacts effect analysis (SIEA)

In this section, the step-by-step process for performing a Social Impacts Effects Analysis is given. For clarity, a sample SIEA table is shown in Figure 1.

Figure 1. Sample SIEA table.

When choosing whether or not to perform an SIEA, designers should begin by assessing how known or unknown the product’s social impacts are to them. Products with strong expected impacts, newly developed products, and products being deployed in new areas or for new stakeholders are strong candidates for performing an SIEA (Lehmann et al., 2013; Akgün et al., 2007).

The design team should also define stakeholder priorities before beginning the 15 steps below. Understanding stakeholder priorities is crucial to project success and is often best determined through direct communication with stakeholders using a variety of methods (Witmer, 2020). Determining stakeholder priorities at this step also makes assigning impact magnitudes easier and more accurate. Social impacts may have different magnitudes or occurrences for different stakeholder groups. If there are multiple groups of stakeholders with differing priorities, priorities should be defined for each group so impact ratings can be assigned separately for each.

As introduced in this paper, the 15 Steps for completing a SIEA are:

1. 1. Determine Stakeholder Groups: List all stakeholders groups for the product being analyzed. These stakeholders include those who will use the product or be affected by its use. These stakeholders will be attached to each product attribute later in order to identify a product’s impact on each specific stakeholder group. See Column C1 in Figure 1.

2. 2. List Product Attributes: List all attributes of the product at the relevant detail level. An attribute is defined here as a characteristic that describes part of the product’s form or function. For the early stages of design (before the product is defined), designers may list product requirements or expected features. In later stages, the design can be broken down into subsystems, features, or components. The goal of this step is to list many of the possible attributes so that effects and impacts may be assigned to them. See Column C2 in Figure 1.

3. 3. List the Social Impacts of Product Attributes: List the impacts the product has on the stakeholders. See Column C3 in Figure 1. If different impacts exist for different stakeholders, assign each impact to its respective stakeholder group. Notice how Column C3 matches Column C1 with a different impact for each stakeholder group.

4. 4. Categorize Social Impacts: Social impacts can be assigned to one or more of 11 major Social Impact Categories: Population Change, Family, Gender, Education, Paid Work, Stratification, Health and Safety, Human Rights, Social Networks and Communication, Conflict and Crime, and Cultural Heritage and Identity (Rainock et al., 2018). See Column C4 in Figure 1.

5. 5. List the Cause of Each Social Impact: Describe how the product attribute leads to the social impact on stakeholders. Describing this connection will help designers later determine how to adjust the design of the product to improve the social impacts of the product. See Column C5 in Figure 1.

6. 6. Indicate Each Impact as Positive or Negative: Define each social impact as having a positive or negative effect on the stakeholder. See Column C6 in Figure 1. It is possible that a particular impact could be both positive and negative. If this is the case, the impact should be listed twice, once as a positive impact and once as a negative impact.

7. 7. Indicate Each Impact as Continuous or Discrete: Define each social impact as occurring discretely or continuously. This refers to whether a specific impact will affect the user a single time or on an ongoing basis. See Column C7 in Figure 1.

8. 8. Rank the Magnitude of Each Impact: Rate the magnitude of each social impact on a scale of 1–10. This is a measurement of how much the impact affects the life of the stakeholder. Table 1 gives an example scale for rating magnitudes. See Column C8 in Figure 1.

   Table 1. Impact magnitude rating table

   

9. 9. Rank the Occurrence Likelihood of Each Impact: Rate the likelihood of occurrence of each social impact on a scale of 1–10. This is a measurement of the likelihood of a given impact happening. Different ways of considering impact occurrence are given in Section 5.4. Table 2 gives an example scale for rating occurrence. See Column C9 in Figure 1.

   Table 2. Impact occurrence rating table

   

10. 10. Rank the Uncertainty of Each Impact: Rate the uncertainty of each social impact on a scale of 1–10. Uncertainty is a measure of how much uncertainty there is about the occurrence or magnitude of a given impact. Additional information is given in Section 5.5. Table 3 gives an example scale for rating uncertainty. See Column C10 in Figure 1.

    Table 3. Impact uncertainty rating table

    

11. 11. Calculate Impact Priority Numbers:

    The Social Impact Risk Priority Number (IPN) is calculated differently depending on the sign of the impact. For negative impacts, the IPN is calculated by multiplying the Uncertainty minus 11, Magnitude, and Occurrence ratings. The scores range from −1 to −1000, with −1000 being the harmful impact most deserving of design resources. For positive impacts, multiply together 11 minus the Uncertainty, the Magnitude, and the Occurrence ratings. The scores range from 1 to 1000, with 1000 being the most beneficial impact. See Equation 1 and Column C11 in Figure 1.

    Additional Impact Priority Number calculations have been developed by the authors to help designers identify other positive impacts of interest, such as items with high potential for impact but either a low magnitude (Magnitude Priority Number, or MPN), low occurrence (Occurrence Priority Number, or OPN), or high uncertainty (Uncertainty Priority Number, or UPN). These calculations can be ranked with the highest numbers being those with the greatest opportunity to improve the $ IPN $ by improving that category. See Equations 2, 3, and 4. Note that for clarity, the scores for Magnitude, Occurrence and Uncertainty are represented (respectively) as $ M $ , $ O $ , and $ U $ in the equations.

12. 12. Order Positive and Negative Impacts: Ranking impacts allow designers to see which impacts have the highest positive and negative values and therefore see which impacts need action. These actions are meant to change the impact of the design to acceptable levels. In this paper we call these impact management measures, which are discussed in greater detail in Section 5.2. Generally, it is recommended that designers consider all impacts with an $ IPN $ below −343 for product development resources and any impact with an uncertainty score 7 or higher for research resources so that uncertainty can be minimized in the future.

    For negative impacts, large negative numbers indicate which impacts would potentially be the most harmful. The largest negative impacts are those with high magnitude, high occurrence, and low uncertainty. These impacts are most deserving of immediate design resources because the designers are certain that this product attribute will have a large and frequent negative impact. Once the largest potential negative impacts are identified, action can be taken to mitigate those negative impacts.

    For positive impacts, large positive numbers indicate which impacts could be the most beneficial. These impacts have high magnitude, high occurrence, and low uncertainty. These are the most beneficial impacts because the designers can be sure that this attribute will have a large and frequent positive impact. When acting on positive $ IPN $ values, there are several approaches to take. One approach is to assign impact management measures to the impacts with the largest positive $ IPN $ to ensure that these impacts stay high. Another approach is to assign impact management measures to impacts with low positives $ IPN $ in order to improve these impacts. Both approaches have merit and may be used by the design team.

13. 13. Assign Impact Management Measures: Impact Management Measures are specific actions assigned to team members to improve positive impacts or decrease negative impacts. These assigned activities are designed to change either the magnitude, occurrence, or uncertainty of the impact. See Column C12 in Figure 1.

14. 14. Re-score Magnitude, Occurrence, and Uncertainty for Impacts with Management Measures: Adjust Magnitude, Occurrence, and Uncertainty scores as necessary to reflect expected changes as the result of Impact Management Measures. See Columns C13-C16 in Figure 1.

15. 15. Follow Up: Perform follow-up to ensure that Impact Management Measures have the intended effect.

(1) $$ IPN=\left\{\begin{array}{ll}M\ast O\ast \left(U-11\right),& Negative\ Impact\\ {}M\ast O\ast \left(11-U\right),& Positive\ Impact\end{array}\right. $$

(2) $$ MPN=\left({O}^2-{M}^2\right)\ast \left(11-U\right) $$

(3) $$ OPN=\left({M}^2-{O}^2\right)\ast \left(11-U\right) $$

(4) $$ UPN=\left(U-1\right)\ast \left(M\ast O\right) $$

4. Illustration/example

Figure 1 shows several rows of an example SIEA performed by the authors. The exercise examined the social impacts of a modular solar charger produced by Fenix and designed for use in rural Uganda. Due to size constraints, only eight rows of the example SIEA are listed in this paper.

Column C1 lists the stakeholders/users of the solar charger, which in this example are: households in Uganda, children in the households, women in the households, and employed members of the households. Column C2 lists product requirements or features such as mobile payment and mobile locking of the modular solar charger. Column C3 lists the social impacts of the product features, such as increased electricity access for work purposes, educational purposes, and women in the households. Column C4 lists which impact category each impact belongs to, such as Paid Labor or Education. Column C5 lists the impact causes, such as mobile payment availability and lower monthly payment costs, which make the device more accessible. Column C6 lists whether the designers considered each impact as positive or negative. For example, increased electricity access was classified as a positive impact, and the ability of the solar company to lock out users was classified as a negative impact for the user stakeholder group. Columns C6, C7, and C8 are the designer ratings for impact magnitude, likelihood, and uncertainty. Column C11 shows the calculated Impact Priority Number $ IPN $ for each impact. These $ IPN $ s were used by the designers to rank impacts by priority and assign actions to increase positive impacts and decrease negative impacts. Column C12 lists suggested impact management measures such as ensuring customer service assistance is available to help customers who are mistakenly locked out of their solar chargers. Columns C13, C14, and C15 list new rankings for impact magnitude, likelihood, and uncertainty after suggested actions are taken. Column C16 lists the new $ IPN $ scores and Column C17 lists the expected improvement due to the impact management measures.

For reference, completing this SIEA took 3 designers 3 hours to complete and analyzed 42 potential impacts related to 11 product attributes in 7 social impact categories (Education, Family, Gender, Health and Safety, Human Rights, Paid Labor, and Social Networks and Communication). While more time could be spent and more social impact categories could be analyzed, this illustration demonstrates how an SIEA could realistically be used to help designers consider a wide range of impacts from different social impact categories in a timely manner.

This exercise also helped designers see how each of four different stakeholder groups (Households in Uganda, Employed Members of Households, Women in Households, Children in Households) were affected differently by the product. The activity also connected the 42 potential impacts to 11 product attributes. These connections are an important feature of FMEAs that help designers understand how product performance leads to impact, as well as the trade-offs and relationships between product performance and impact for multiple stakeholder groups.

And finally, the calculation of Impact Priority Numbers allowed all 42 potential impacts to be ranked to help determine designer priorities for maximizing positive impact and minimizing negative impact. The four impact priority methods given in Section 3, step 11 (Traditional Impact Priority Number, Magnitude Priority Number, Occurrence Magnitude Number, and Uncertainty Magnitude Number) were used to calculate the highest overall positive and negative impacts as well as high potential targets for improving the magnitude, occurrence, and uncertainty.

Of the impacts shown in Figure 1, the largest positive impacts identified were the impact of increased electricity access due to the cost of the charger. The largest negative impact was electrical shock due to the charger being damaged. The greatest opportunity to increase impact through occurrence was ensuring that the device functions reliably in portable work conditions. The largest opportunity for increasing impact through magnitude was the impact on education by allowing the continual charging of devices in educational settings. The largest opportunity to increase impact by decreasing uncertainty was the impact of increased electricity access for all family members. Identifying the most impactful changes is an important part of project evaluation and creating plans to maximize positive impacts and minimize negative impacts (Watts et al., 2016).

These rankings illustrate how the FMEA approach helps designers not only create a large list of potential impacts but also rank and prioritize which impacts need additional consideration in the design process. Focusing on the highest priority impacts identified increased the overall impacts by both decreasing negative impacts and increasing positive impacts of the product.

5. Highlighting differences between SIEA and FMEA

This section describes the attributes of SIEA in greater detail and how it compares to traditional FMEA. As previously stated, FMEA could not be directly applied to social impact evaluation for five main reasons. Now that SIEA has been introduced and explained, these main reasons can be explained in greater depth. Note that these five changes, as well as additional differences, are summarized in Table 4.

Table 4. FMEA and SIEA comparison

5.1. Social impact consideration range

SIEA aims to help designers consider more relevant social impacts in a wider range of impact categories than methods currently practiced. This method encourages designers to consider more social impacts by systematically listing social impacts for every aspect of the product. Products typically have impacts in multiple social impact categories (Ottosson et al., 2020), which is why SIEA guides designers in considering a broad range of impacts across categories.

As acknowledged in the beginning of the paper, on few occasions FMEA has been used to analyze effects in 8 of the 11 impact categories put forth by Rainock et al (Rainock et al., 2018). The impact categories analyzed include: Conflict & Crime (Rao et al., 2020) (Stefankova et al., 2014), Education (Sutrisno and Lee, 2011), Family (Neisiani et al., 2016), Gender (Ranangen et al., 2018), Health & Safety (Zeng et al., 2010) (Latino and Flood, 2004) (Chiozza and Ponzetti, 2009), Human Rights (H˛abek and Molenda, 2017), Paid Work (Bahrami et al., 2012) (Santos and Cabral, 2008) (Carbone and Tippett, 2004), and Social Networks & Communication (Chandra et al., 2020). FMEA has also been used to analyze environmental impact (Zeng et al., 2010; Vazdani et al., 2017; Roszak et al., 2015). While these studies have occurred, they have been rare exceptions and are only done 1–2 categories at a time. SIEA differs from FMEA in this case because it is designed for situations such as these and is built to provide simultaneous consideration of a wide range of impacts. The example in Section 4 illustrates the broad range of social impacts for an SIEA performed on a modular solar charger, finding impacts related to 7 different social impact categories.

5.2. Product effect duality

SIEA assigns each impact as being either positive or negative. Considering impacts as positive or negative helps designers know which positive impacts need to be protected and which negative impacts need to be mitigated or prevented (Stevenson et al., 2018). Whether an impact is considered positive or negative can vary for stakeholders, as a product may produce positive impacts for one user and negative impacts for another (Rainock et al., 2018). Industrial equipment, for example, may increase profits for its owner but could pose a safety risk to users and create pollution and noise that negatively impacts the community. Often these impacts are paired and lead to trade-offs (Pugel and Walters, 2017).

Additionally, products often have secondary effects where the benefit of a product leads to a change in behavior, which creates additional impacts (Bhamra et al., 2011). For example, farming equipment that improves farm output could improve wages for employees of farmers that own the equipment, or the increase in efficiency could lead to fewer employees or other laborers. Increased production could negatively impact competing farmers. Positive social impacts may produce negative secondary impacts, and negative social impacts may produce positive secondary impacts. Secondary effects are important to consider because they are often overlooked. Unconsidered secondary impacts have been observed as a source of project failure in global development projects (Jacobsen, 2015; Freeman, 2007).

Considering both positive and negative impacts simultaneously offers an improvement to several current risk management methodologies, such as FMEA, that only consider negative impacts (Ginn et al., 1998). The assignment of positive or negative also goes toward calculating the Impact Priority Number, which is used for ranking and prioritizing product impacts. There are differences in how positive and negative impact scores are ranked and the strategies designers use to increase positive impact and decrease negative impacts.

The ultimate goal of SIEA is to consider many of the possible impacts and rank them so that resources can be dedicated to the most important impacts (Xiao et al., 2011; Chen, 2007). After impacts have been ranked by their $ IPN $ , the design team assigns specific actions to improve the impacts and delegates these actions to team members. These corrective actions aim to reduce negative impacts and increase positive impacts by changing the magnitude, occurrence, or uncertainty of selected impacts (Van Leeuwen et al., 2009).

Multiple possible corrective actions can be taken that may have different effectiveness and costs. The design team decides which corrective actions would be the most appropriate and assigns the action to a specific member of the design team (Paciarotti et al., 2014). The right corrective actions in the early stages of design can prevent more costly corrective actions later (Paciarotti et al., 2014). There are several ways to determine how many impacts should be assigned management measures. These ways include using an $ IPN $ threshold, assigning actions to a percentage of impacts with the highest $ IPN $ s, using a screening matrix, or using a graph of $ IPN $ s to find the inflection point for impact priorities (Mikulak et al., 2017; Sankar and Prabhu, 2001; Catelani et al., 2020).

5.3. Stakeholder consideration

Social impacts depend on both the product and the user (Stevenson et al., 2018). Individuals have many differences including differences, in physical attributes, access to resources, cultural upbringing, needs, desires, and priorities. Because social impacts are closely related to stakeholder values, SIEA starts by attempting to identify and understand the priorities of the major relevant stakeholders. This can be done through a variety of methods, such as using personas or including stakeholders in participatory or co-design (Ogawa and Piller, 2006; Wilkinson and De Angeli, 2014; Schuler and Namioka, 1993).

The other main difference is that SIEA handles stakeholder differences by evaluating impact magnitudes separately for each major stakeholder group. This helps design teams consider the most important impacts for each stakeholder group and understand trade-offs between stakeholders. This information helps design teams handle trade-offs better and create a product with desirable impacts (Wong, 2005).

5.4. Occurrence pattern

Several methods used in engineering handle either impacts related to discrete events or the continuous impacts of products (Rose, 2013). For example, FMEA is used to analyze discrete events but not continuous impacts. In contrast, SIEA has been designed to deal with both the discrete and continuous impacts of products. This is important because events that cause impacts can either be discrete, such as a one-time medication that relieves symptoms for a few hours, or continuous, such as the clean water a well supplies to a village 24 hours a day. Additionally, discrete events can create continuous, lasting impacts, such as an injury caused by a product having lasting health effects. Events that continuously provide an impact while in use might also provide little benefit after use of the product has ended, such as an emergency shelter. Some products might not be used continuously, but the availability of use provides an impact, such as a cell phone that provides benefits only while in use, but the ability to use the phone at any time creates a continuous positive impact.

Whether an impact can be thought of as continuous or discrete, all impact causes and impact effects can be described as having an occurrence likelihood. Mathematically, this is a probability or percentage between 0–1 or 0%–00%. Defining occurrence in this way is robust mathematically but is potentially problematic in the context of occurrence ratings that occur in FMEA and other methods. The two main issues with this are that impacts can occur across a broad spectrum of occurrences and that the value judgements associated with assigning an occurrence number vary depending on the individual impacts.

For example, a knee brace could provide stability and improved healing for 19 out of 20 patients, cause irritation in 5 out of 20 patients, and lead to a serious accident by snagging on heavy equipment or moving objects 1 time per 200,000 hours of patient use. Linear and potentially even logarithmic scales may have difficulty providing the right level of resolution across the potential impacts of a product. Furthermore, what qualifies as a low or high occurrence depends on the individual impact. In the previous example, 1 serious injury per 200,000 hours of patient use might be considered a high-occurrence event in need of mitigation, while the 5 in 20 occurrence of irritation might be considered a low-occurrence event that does not need additional design consideration. Some of these issues are resolved by the overall impact score, which accounts for the magnitude and uncertainty of impacts; however, some issues may remain.

There are several approaches to handling these irregularities. The authors present four options that have different strengths and drawbacks that provide flexibility for the best use in different applications or situations.

1. 1. Use a linear occurrence scale related to the mathematical probability of occurrence and a range that encompasses the impacts identified. This approach is straightforward and intuitive mathematically. Potential scaling issues may be resolved with additional impact ranking metrics discussed in Section 3, step 11.

2. 2. Use the mathematical probability of occurrence on a logarithmic scale. This approach handles large ranges of occurrence well and provides good resolution across such large differences. However, this approach is slightly less intuitive and may involve additional calculations.

3. 3. Consider the occurrence in a qualitative manner instead of a strictly mathematical assignment. As discussed previously, whether an impact is considered a low occurrence or high occurrence depends on the impact itself. This approach allows designers the flexibility to make these value judgements. This approach might not be as mathematically rigorous or repeatable, but it may be a more appropriate approach when the effects of product impacts span a very large range. This approach is demonstrated in Table 2 in Section 3.

4. 4. Rank occurrence based on the likelihood that the design team can design the product to achieve the desired level of impact. This approach is applicable early in the product design before mathematical probabilities can be known and helps the design team focus on the areas of design that need the most attention to achieve the desired impact.

5.5. Impact uncertainty

An important aspect of social impact consideration is uncertainty (Leung et al., 2015). There are several types of uncertainty associated with social impacts. For designers, there is uncertainty about the actual impact a product has on the lives of users (Stevenson et al., 2018). There is also uncertainty about the adoption or acceptance of the product (Cohen et al., 2016). Additional uncertainty comes from the potential for secondary impacts or effects (Jacobsen, 2015; Freeman, 2007).

There are several sources for the uncertainty of impacts on the user side as well. There is uncertainty associated with the occurrence of random events. In traditional FMEA, this is referred to as detectability. Increasing the detectability decreases uncertainty by allowing users to foresee a potential failure before the failure and its negative effects occur. There is also uncertainty for the user about whether the use of a product creates an impact. For example, the health benefits of clean water or an improved cookstove may not be immediately apparent to users. This type of uncertainty can affect the probability that an individual will use a product (Au and Enderwick, 2000). This type of uncertainty is also challenging for those who wish to measure or quantify the impact of products.

As designers, it is desirable to reduce all of these uncertainties. Areas of high uncertainty signal the need to use research resources to reduce uncertainty. When it comes to ranking uncertainty for use in an SIEA, there are several possible approaches with varying strengths and weaknesses. These approaches are as follows:

1. 1. Focus on only one type of uncertainty. This approach only considers one source of uncertainty at a time. This is a simple approach that also tailors to the different stages of design. In conceptual stages, designers may rate the uncertainty associated with designing the product, and later stages may then progressively focus on the uncertainty associated with levels of impact, product acceptance, and verifiability of the impact occurring.

2. 2. Consider multiple sources of uncertainty. This approach ranks each type of uncertainty separately. This approach is beneficial in that it considers many types of uncertainty; however, it increases the time required to perform an SIEA and could complicate $ IPN $ rating calculations and ordering.

3. 3. Consider only the largest uncertainty. While several sources of uncertainty exist for all impacts, some may be of greater concern for individual product impacts. Ranking the largest source of uncertainty for each impact lowers the amount of rankings and calculations needed, but it adds complexity as it requires designers to compare different sources of uncertainty mentally.

4. 4. Use a holistic approach. This approach relies on the intuition and experience of users to consider the overall uncertainty from all sources. This approach is not as well defined, but it has the potential to cover the breadth of uncertainty types without requiring the time put into a deep analysis of each type individually.

6. Discussion and concluding remarks

SIEA is built on the aspects of FMEA that are attractive for analyzing the social impact of engineered products, such as the structured methodical approach and qualitative rankings that allow for the comparison of impacts. SIEA differs from the traditional FMEA approach to allow for full consideration of social impacts by a design team. These necessary changes include handling impact differences for multiple stakeholders, evaluating social impacts from a broader range of categories, allowing for continuous and discrete impacts, assigning impacts as either positive or negative, and considering a wide range of the uncertainty associated with the social impact.

In addition to SIEA having the benefit of the methodical and qualitative FMEA approach, using a method similar to current practices may help avoid unnecessary complexity and help increase implementation and adoption (Tajima and Fischer, 2013). SIEA is also an attractive method because the qualitative approach borrowed from FMEA is less costly and thus more likely to be implemented than other social impact assessment methods that require large amounts of data (Henry, 1990).

Future work could include controlled experiments or case studies to determine the performance of SIEA in improving social impact considerations in the product development process. While SIEA cannot follow the same adoption pattern as FMEA (which was created and applied purely in an industry setting), controlled experiments or case studies could provide valuable information on the value of SIEA before industry can rely on this method like they rely on traditional FMEA.

In conclusion, the authors believe that the same methodical approach that has made FMEA successful in many fields (such as product development, risk management, and quality management) could be applied for use as a dedicated social impact analysis method. This method, Social Impact Effects Analysis (SIEA), builds upon FMEA to allow designers to more fully consider the social impacts of their product. Made by combining the desirable features of FMEA with the necessary changes for a focus on social impact evaluation, SIEA provides a method to make social impact analysis more accessible and effective to aid in the development of products with improved social impact.