In this article, we propose a Data Envelopment Analysis (DEA) and Analytic Hierarchy Process (AHP) integrated model to improve the selection process in the acquisition of a weapon system which is the key component to the success of the project
By JaeHun Moon and Seokjoong Kang
In this article,
we propose a Data Envelopment Analysis (DEA) and Analytic Hierarchy Process
(AHP) integrated model to improve the selection process in the acquisition of a
weapon system which is the key component to the success of the project.
In
particular, we applied DEA in the first stage to choose a frontier group among
the candidates in the selection process of the next-generation fighter system
(the 3rd FX) in Korea. Then, by using the Delphi technique, we surveyed
military experts and applied AHP to determine the best choice among the
candidates.
Acquisition of a
Weapon System
Military
products are largely classified as weapon and non-weapon systems. The weapon
system includes all the field weapons (e.g., guided weapons, aircrafts, and
vessels) and all other operating products (e.g., parts, facilities, and
software) that are necessary in a war. The non-weapon system includes all the
elements other than the ones that are classified as the weapon system (e.g.,
equipment, parts, facilities, software, and materials).
The acquisition
is categorized as either a domestic acquisition that purchases internal
products or a foreign acquisition that purchases foreign products. The foreign
acquisition in Korea is further categorized as either foreign military sales
(FMS) or foreign company purchases (FCP). FMS is an acquisition that is
controlled and guaranteed by the US government through a contract between the
US and Korea. FCP, on the other hand, is a direct acquisition from foreign
companies without any government involvement.
Acquisition
Procedure
The two most
common contract methods that follow the national contract law in Korea are the
lowest bidding and negotiation. The lowest bidding method is a basic method,
and the negotiation method is acceptable in a special case according to the
national contract law. Even though the lowest bidding method selects the lowest
bidder, some big projects that are above a certain level also utilize screening
tests to test the capability of the bidder.
Typically, the
procedure for selecting suppliers follows several phases:
1) The
assessment readiness and the assessment of a proposal,
2) The
evaluation of the products,
3) Negotiation,
and
4) The final
selection.
The assessment
readiness for a project consists of writing the request for proposal (RFP), the
announcement of bidding, the formation of the assessment committee, and the
development of an assessment plan including the assessment criteria. One
difference between a purchasing project and an R&D project is that the
former assesses each item for a specific project (e.g., selecting the
next-generation fighter system), and the latter assesses the common items that
are equally applicable to all future projects.
There are
necessary and optional conditions in a purchasing project. The necessary
conditions are essential items for the required operational capability (ROC)
and should be satisfied. The optional conditions are the assessment items other
than the necessary conditions and should satisfy a certain level of
satisfaction (generally, 70%) on the RFP. ROC is used for determining what the
Korean government wants to purchase in terms of the capabilities of the
weapons. Once the ROC is complete, the Ministry of National Defense makes a
budget and initiates the procurement process so that the potential suppliers
can enter the bidding process. The RFP is a solicitation made often through a
bidding process by an agency or company interested in the procurement of a commodity,
service, or valuable asset, to potential suppliers to submit business
proposals. It is submitted early in the procurement cycle either at the
preliminary study or the procurement stage. The RFP presents preliminary
requirements for the commodity or service and may dictate to varying degrees
the exact structure and format of the supplier’s response. Effective RFPs
typically reflect the strategy and short-/long-term business objectives,
providing detailed insight upon which suppliers can offer a matching
perspective.
After assessing
the proposal, the government evaluates the products to select initial products
that satisfy all necessary conditions and meet or exceed the suggested level
requirements for optional conditions. Therefore, usually, multiple products are
selected at this stage. A negotiation process then follows on the basis of
factors such as price, technology, and general conditions.
Then, the
government selects the final product by typically applying either the lowest
cost while satisfying the requirements of the ROC method or the comprehensive
evaluation method. The former is the most common method, but the latter can be
used when the comparison between the suggested products is complex and/or when
the project is so big that it requires many strategic considerations. The RFP
usually includes the determination method in the original announcement of
bidding. Then, a provisional contract is signed by both parties (i.e., the
Government and the suppliers).
In a big
investment case such as the acquisition of a weapon system, the failure of the
project lays a lot of burden on both the buyer and the seller.
However, because of the unique characteristics of the military industry,
neither party acknowledges its mistakes. Therefore, the reasons for failure are
not available to the public and not much has been done to analyze the failures.
However, a
failure analysis is necessary because its learning effects are very helpful for
future project management. The causes of project failures can include errors of
cost analysis, improper setup of the operational concept, unrealistic targets,
and miscommunication between the project manager and the supplier to name a
few. However, the most common cause of the project failure is found in the
phase of selecting suppliers/business partners. The following section discusses
decision-making models that can be used in the selection process.
The Third FX
Project in Korea
Recently, the
Korean government selected F-35A as the next-generation fighter system. The
general selection process is as follows: first, the government investigated
many candidates including F-35A, F-15SE, and Euro-Fighter. After evaluating for
more than two years, the government selected F-15SE as the next-generation
fighter system in August 2013. After this decision, there were considerable
opposition from the press and the national security experts, including the
former Air Force Chief of Staff, partly because F15-SE lacked stealth
capabilities. Then, the Korean government re-negotiated for about five months
and finally, selected F-35A as the next-generation fighter system. This is a
good example that shows why we need a more systematic decision-making process;
in this paper, we try to shed light on such studies. The following section introduces
two of the most widely used decision-making models, which will be incorporated
in our study.
The Concept of
AHP
Analytical
Hierarchy Process (AHP) is a method that improves the inefficiency in the
decision-making process. It is part of a decision support system that
systematically assesses alternatives when there are multiple and complex
objectives and/or the evaluation criteria.
Chart Attribute: A typical AHP model / Source: Palgrave Macmillan
Such criteria can be either
quantitative or qualitative, and AHP can measure these criteria as proportional
scales after quantifying these qualitative criteria. Therefore, the method can
be used in non-standard and complex problems. In fact, Saaty argues that
the AHP is a useful decision-making tool in a complex decision-making situation
with multiple objectives, evaluation criteria, and participating decision
makers. The process determines the priority of the alternatives through a
series of pair-wise comparisons after categorizing the criteria into
multi-hierarchical levels.
Vargas asserts that the theoretical underpinnings of AHP lie with four axioms:
reciprocal comparison, homogeneity, independence, and expectations. The
reciprocal comparison axiom means that decision makers should be able to make
pair-wise comparisons of two factors in the same hierarchy in terms of
preferences and the reciprocal strength of the preferences should exist.
Homogeneity means that the preferences should be expressed by means of a
bounded scale. Independence means that the criteria should not be related to
the properties of the alternatives. Expectations mean that the hierarchical
structure is assumed to be complete.
Because of its
usefulness and simplicity, AHP has been used in various decision-making
situations including marketing, project and risk management,
strategic planning, information technology selection, employee
recruitment, production, and medical and health-care.
Typically, AHP
has five phases:
1) Formation of
hierarchy,
2) Pair-wise
comparison of the factors and alternatives,
3) Estimation of
the weighted values,
4) Consistency
check, and
5) Selection of
the best alternative.
In the first
phase, a decision hierarchy is formed. The goal is at the top level followed by
the evaluation criteria and the alternatives are at the bottom. The
evaluation criteria sometimes have sub-criteria.
In the second
phase, a series of pair-wise comparisons between the factors in the same
hierarchy are implemented to determine the priority of the factors from the
viewpoint of the upper-level hierarchy. Further, in this phase, the pairwise
comparisons between alternatives are performed to obtain the initial priorities
of the alternatives.
In the third
phase, the weighted value for each factor is determined using the eigenvalue
method. The weighted values are subjective, and this relative judgment cannot
guarantee the internal consistency. Therefore, the consistency of the judgment
is evaluated in the following phase.
The consistency
of the decision makers (i.e., whether the judgments made by each decision maker
are logically consistent or not) is evaluated in this phase. If, for example, a
certain decision maker’s judgment is such that alternative A is more important
than alternative B, and alternative B is more important than alternative C,
then it is considered consistent. This can be found out by calculating the
consistency index (CI) and the consistency ratio (CR) for the evaluation
criteria. CI and CR can be calculated by the following formula and considered
to have rational consistency if CR is less than 0.1, acceptable consistency if
it is less than 0.2, and deficient consistency otherwise.
CI= (λmax-n)/n-1
where λmax
denotes the maximum eigenvalue of a pair-wise comparison matrix.
CR=CI/RI
where the random
index (RI) is a CI calculated by a computer simulation based on non-consistent
pair-wise comparisons.
In the final
phase, a comprehensive vector of importance is calculated to determine the
priorities of the alternatives. First, the vector of importance between factors
in each layer is multiplied by the vector of importance on each factor’s
alternative. Then, the rankings of each alternative are determined and the
alternative with the highest ranking is selected as the best alternative.
To learn the
application of DEA-AHP integrated model in the selection process of the South
Korean – Next Generation Fighter System, Download the Paper – LINK
About the Authors:
Currently
serving as a special committee member at Korea Institute for Defense Analyses
(KIDA), Seoul, Republic of Korea. He graduated from the Korea Military Academy
in 1985 and National Defense College in 1992. He received his M.A. from
Graduate School of National Defense Management, Korea National Defense
University, and completed his doctoral course in Management of Technology at
Korea University. During his service in the ROK Army until 2011, he
successively filled various posts, such as Plan Management Instructor at Korea
Army College, Chief of Financial Management of Army Capital Corps & 22nd
Army Division, Vice Director of the Budgeting Control Bureau at the Headquarter
of the Army, and Chief of Financial Management Office/Division at the US-ROK
Combined Forces Command (CFC).
Received his
B.S. and M.S. in Computer Science from Indiana University in 1988 and 1991,
respectively, and his Ph.D. in Electrical Engineering and Computer Science from
University of California, Irvine, in 2003. He worked as a lecturer and research
staff at University of California, Irvine. He also worked as a principle
researcher at Samsung Electronics Co. from 2004 to 2006. Currently, he is an
associate professor at the Department of Management of Technology for Defense
at Korea University, Seoul, Korea. His publications include “A Management
Procedure of Required Operational Capability for Low-cost and High-efficiency”
(2012), “Weapon Systems Development,” and A Study on the Factors Affecting the
Results of Negotiations of Overseas Weapon Purchase” (2010). His research
interests include decision systems for defense acquisition, acquisition
process, and system & software engineering
Publication
Details:
An Integrated
DEA-AHP Model for the Acquisition of a Weapon System: Selection of a Next-Generation
Fighter System in Korea, Journal of
Information and Communication Convergence Engineering. 2015. Jun, 13(2): 97-104
DOI : http://dx.doi.org/10.6109/jicce.2015.13.2.097
Copyright ©
2015, The Korean Institute of Information and Communication Engineering
This is an Open
Access article distributed under the terms of the Creative Commons Attribution
Non-Commercial License (http://creativecommons.org/li-censes/by-nc/3.0/)
