The Role of Enterprise Data Management in Energy and Commodities Industries
Many large integrated oil and natural gas firms, commodity firms, and power utilities rely on an Enterprise Data Management (EDM) system to acquire the data required for their trade and risk management (ETRM/CTRM), enterprise resource planning (ERP), business intelligence (BI), visualization, and accounting and settlement systems. These systems require significant volumes of data. For example, a modern ETRM may require hundreds of discrete data reports containing time series, including energy brokerage data, information from price reporting agencies, and internal operational data about proprietary crack spreads, forward price curves, and more. Power firms must have access to nodal ISO feeds, plant operational data, and
Firms and utilities searching for an EDM system have two major options available to them:
- Build: Firms may build a basic EDM system in-house with their internal IT resources.
- Buy: Firms may license a commercial EDM solution from ZE PowerGroup.
In what follows, best practices from industry experts, thought leaders, and academics will be discussed alongside anecdotes from two firms facing data management dilemmas.
Ultimately, both firms selected a commercial off-the-shelf (COTS) EDM system, ZEMA, to meet their data management needs.
! Energy trade and risk management, business intelligence, and financial and settlement systems must be fed by an EDM system. EDM systems can be built or purchased from a specialized vendor like ZE.
What Is an Enterprise Data Management System?
Data management can be thought of as a shared responsibility between businesses and technology firms to ensure corporate data assets are properly sourced, deployed, integrated, and aligned with corporate needs. Firms without an EDM system effectively have reduced their ownership of an important corporate asset: their data.
A modern EDM solution consists of several major components:
- Data collection tool:
Schedules the collection of information and centralizes data.
- Data quality control tool:
Measures the completeness, timeliness, and accuracy of data stored.
- Data storage tool:
Stores information in an enterprise-level database.
- Metadata management system:
Describes and normalizes data collected into the system.
- Security and entitlement administration functionality:
Manages entitlements and security, and in this way helps manage resources and supports auditing and data compliance.
- Industry-specific analytical engine:
Used for easy data retrieval, business intelligence, and business automation.
- A robust API and workflow engine:
Provides support for enterprise integration and continued application development.
! EDM isn’t just about data collection. A modern EDM system also requires a data quality control system, a storage system, an administration system to provide entitlement control, an industry-specific analytical engine, and software support for integration and future development – all of which is included when you license ZEMA.
Development of Commercial Off-The-Shelf Software
In the mid-90s, a paradigm shift changed the way large software systems were developed, moving away from in-house development and towards component-based software
On the other side, other IT stakeholders tend to have a bias towards building an EDM system because they place a high value on customization, expect departments hosting the system to benefit from the additional budget and personnel maintaining the system, and desire to utilize an in-house skill set.
Managers often face the challenge of sorting out preferences towards building versus the benefits of purchasing a COTS solution. Many industry professionals and academic papers have come to the following general consensus: “Firms should build software to improve core business and differentiate themselves. Firms should buy when improving or automating routine business processes.”
! Buy to standardize, build to compete.
The Case for Building a Modern EDM System In House – Costs and Requirements
Consulting firms offer a more nuanced pros and cons message about EDM software for energy and commodities industries, in line with their commercial interests and client experiences.
Does the Firm Meet the Preconditions Required for In-House EDM Development?
In-house EDM developments require that the following preconditions be met in order to minimize project risks:
- EDM domain and technical knowledge/capabilities are available in house:
Although the firm may have a large, competent IT department and may understand their immediate data requirements, they also need an up-to-date EDM system and industry-specific domain knowledge. Knowledge gaps cause firms to underestimate the true requirements of EDM development, introducing scope-creep, increased costs, and deployment delays. To mitigate these project risks, firms should have managers and
developers with specialized EDM and industry experience.
- EDM development complements the core business competitive advantage:
According to a “buy to standardize, build to compete” philosophy, a firm needs to make a compelling case describing how an in-house EDM software system enhances their competitive advantage enough to justify the costs and risks associated with developing it.
What Is the Initial Development Cost a Firm Is Willing to Undertake to Develop an EDM System?
Once a firm determines that an EDM system is within its core competency and business scope, it must next quantify the opportunity cost, direct cost, and the time-to-market constraints. Many of ZE’s clients formerly attempted to build an EDM software in house.
Drawing from both its clients’ experiences and its own experience producing ZEMA, ZE has developed a breakdown of the costs involved in producing an EDM system. This breakdown is detailed in Table 1.
- Opportunity cost:
There is often a misperception that developing an EDM system is a sunk cost, since in-house developers, managers, and infrastructure are already paid for. There are no free lunches – the benefits of using a firm’s resources to develop an EDM system should be measured against the potentially large opportunity cost of not using those resources.
- Direct cost:
Direct cost savings are often mistakenly cited by firms as the top reason to develop EDM software in house. To fully account for initial development costs, firms should budget for project management and direct administration costs, infrastructure, domain knowledge learning and methodology development, software design and programming, server and licensing costs, testing, documentation, integration with internal systems, and finally, roll-out and user training. Costs do not end with the roll-out; recurring costs are described in the next section. A ZE survey looking at internal development costs and client experiences suggests that a basic EDM software system costs $4,000,000 USD and takes three years to develop, not including ongoing operational costs or future product enhancements. Future enhancements may include a data collection and scheduling engine, the extension of required data feeds, and the development of data quality control and entitlement administration applications, all of which are necessary EDM components. In ZE’s experience, most medium to large energy firms consume anywhere from 100-300 data feeds, and the three year estimated time-to-market development period assumes that an in-house group of experienced data acquisition developers is present. Costs may be reduced by outsourcing, but this poses an increased threat to the management of code quality and the protection of intellectual property.
- Time-to-market constraints:
Decisions to acquire a data management solution are generally time-bound by business requirements. A firm requiring an EDM system usually requires it “soon” (meaning within a year). ZE’s experience, based upon client interviews, suggests that it takes at least two to three years to create a basic data collection and scheduling application, along with a critical entitlement and data quality software. The exact time-to-market depends on variables such as the features required, the number of experienced developers and managers, and the budget. Developing the remaining EDM modules, including the analytical engine, requires additional time. A firm with unrealistic deadlines will force the project manager to balance costs, required features, and time-to-market, resulting in a sub-optimal system.
|Est. Cost||Est. Time-To Market (Years)||Initial Direct Costs|
|$ —||Not Included||Opportunity Cost and Time-To-Market Constraint Costs|
|$ 3,000,000||2.5||Data Collection and Scheduling Application
and Initial Processors
|$ 500,000||0.25||Data Quality Control Application|
|$ 500,000||0.25||Basic Entitlement Administration Application|
|$ —||Not Included||Analytics and Automation Engine|
|$ —||Not Included||Integration API|
|$ 4,000,000||3||Total Initial Direct Costs|
Table 1: Initial Direct Costs
What Is the Ongoing Cost of Maintaining an EDM System?
Recurring operational costs are a significant budgetary component of the overall cost of an in-house EDM system. Table 2 shows an itemized cost breakdown.
- Ongoing software maintenance and licensing costs:
EDM software maintenance is required when the underlying software it relies upon is updated. For example, an ”upstream” operating system update (Windows/Office updates) or framework update (.NET or Java updates) may trigger a corresponding EDM update. A “downstream” software change, such as an ETRM change or even a replacement, may also force an EDM update. A firm typically spends at least $100,000 USD/year maintaining software licenses for items such as server operating systems, developer softwares, and databases.
- Ongoing data maintenance:
Over time, data being collected by a firm may change formats or delivery mechanisms – it may even stop being published. ZE estimates that external market data feeds change about once a year on average – meaning a firm with 300 data feeds needs to possess the ability to reprogram, test, and deploy up to one data feed per day, a task which requires approximately two full-time programmers. Firms should also expend resources to anticipate and detect upcoming data report changes, deploy updates, and minimize service disruption.
- Ongoing data quality monitoring:
Incoming data needs to be monitored on a daily basis to ensure that its quality does not negatively impact downstream analysis or usage. Serious industry lapses from those who skipped data monitoring include daily P&L price swings due to missing data points and hedging financial instruments erroneously based on a poor spreadsheet calculation. With the right software, data quality monitoring can be done with one full-time analyst.
- New data development:
Since a business requires new data sources, the EDM team must analyze and understand new data, develop the data feed, then test and deploy it into the existing EDM framework – all in a timely manner. On average, an experienced developer will complete a new data feed in one week, if the EDM framework is good and the data source isn’t overly complex. The associated costs listed in Table 2 assume that the same pool of programmers are implementing updates to existing data feeds and creating new data feeds.
- Support and training:
EDM user adaptation requires ongoing technical support and training resources – this is especially important in a global firm where users may be in several time zones. Firms who serve geographically diverse users require 24/7 support and enhanced troubleshooting capabilities in order to meet business demands.
|Est. Cost||Annual Recurring Operational Costs|
|$ 100,000||Ongoing Software Maintenance and Licensing Costs|
|$ 150,000||Data Maintenance (300 processsors)|
|$ 50,000||Data Quality Monitoring (300 processsors)|
|$ 50,000||New Data Sources (+5 processors/year)|
|$ 250,000||Application Support and Training|
|$ 600,000||Total Annual Recurring Costs|
Table 2: Annual Recurring Operational Costs
What Is the Cost of Future EDM Development?
Technology, methodology, and hardware continuously improve over time. User input, technology, market events, growth in data availability, external regulation, and deployment scale changes justify a major rebuild cycle once every three to five years. The resources required for a rebuild make this unlikely for any single entity. ZE estimates that a one-time expenditure of $800,000 is required to fund a core team of three to four developers who produce a future iteration of an EDM system in two to five years. This does not account for developing an analytical and automation engine. Nonetheless, the operational experience and user feedback gained from running a basic EDM system for a few years gives a firm insight into improving data collection, quality control, entitlement management, and other critical components of in-house EDM development.
Table 3: Research and Development (R&D) Cost of Future EDM Improvement
|Est. Cost||Est. Time-To Market (Years)||Est. R&D Cost of Future EDM|
|$ 800,000||2||Future EDM Improvement
(3-4 developers and support costs)
Table 3: Research and Development (R&D) Cost of Future EDM Improvement
What Is the Five-Year Total Cost of Ownership (TCO) for an In-House EDM System?
The five-year TCO for the development of an in-house EDM system can be broken down into several large cost components. The initial effort to build a basic data collection and scheduling engine, plus the required data feeds, typically takes three years of careful development and $4,000,000 USD. This does not include developing an industry-specific analytical engine, a robust workflow/automation engine, or an API required for advanced enterprise integration, all of which are critical components of a modern EDM system. The opportunity cost and time-to-market constraints are not included because these factors are firm-specific.
The subsequent two years of operational costs, including data maintenance, application support and training, and software licensing will add up to about $1,200,000 USD. R&D for a future EDM iteration is conservatively estimated at about $800,000 USD, spread out over two years. Again, this doesn’t account for developing an analytical engine, but it does begin to address some of the other components of a modern EDM system.
The total five-year cost of ownership for an in-house EDM system is about $5,400,000 USD. If the firm forgoes R&D and minimizes application support and training, it can save perhaps an additional $1,800,000 USD over two years, but at the expense of long-term application viability and user uptake.
The Case for Buying a Commercial EDM Software – ZEMA
ZE PowerGroup Has a Decade of EDM Software Development Experience
ZE is an experienced software firm that combines energy industry expertise with advanced software development capabilities. ZE started building EDM solutions in 2001 in response to the demands of deregulated North American energy markets. ZEMA is an end-to-end EDM solution designed for collecting data, performing complex analysis, automating business processes, and integrating with any downstream system. ZEMA is now used across the globe by multi-commodity, multi-national clients in the energy and commodities industries. ZEMA’s development as a modern EDM platform is the culmination of the development, R&D, and ongoing efforts of over 200 dedicated staff.
Market Recognition for ZEMA’s Performance and Reliability
ZE has a history of delivering its EDM solution on time and within its clients’ budgetary constraints. ZE’s teams employ a holistic approach to discovering unique client data and integration requirements, then subsequently configure ZEMA’s EDM modules to meet these needs. ZE’s successes have translated into industry recognition: the company has been voted as the top data management house of the year by Energy Risk magazine five years in a row.
ZEMA Reduces Project Risk and Costs
- Opportunity cost:
ZEMA’s well-crafted data management software solution reduces the opportunity costs associated with building an in-house EDM platform.
- Direct cost:
Deploying ZEMA not only provides cost saving opportunities, but also a near-immediate return on investment and enhanced revenue opportunities. These cost savings include reduced integration, data consolidation, data storage, and data cleansing costs. From a business perspective, implementing ZEMA improves data security and regulatory compliance and reduces IT operational expenses without impacting service or incurring risk. Finally, ZEMA reduces the costs of long-term planning and implementation expenses associated with future EDM requirements.
- Licensing ZEMA reduces the time-to-market risks inherent in building an EDM solution:
Simply put, an individual firm would be hard-pressed to develop advanced EDM software of a similar quality, in a shorter time frame, on a smaller budget, and with less domain knowledge at their disposal. Compared with the costs and requirements detailed in the section “Case for Building a Modern EDM System,” implementing ZEMA is a rapid and reliable solution to data management dilemmas, as ZEMA is completely developed and ZE has many years of experiencing implementing the software for clients of all sizes. ZE’s teams work with clients to identify the data they require, install new data feeds, develop adaptors to connect with integration points, and install the EDM system itself. A typical ZEMA build consists of 300 incoming data feeds and connects with two to three major systems, including billing and ETRMs. Implementation times range between one to six months from start to finish, depending on the complexity and scope of the project. Conservatively speaking, this means a ZEMA implementation can be completed in less than a fifth of the time it takes for in-house development of an EDM solution. This translates to huge time and cost savings, as well as a reduction of project delivery risks.
ZEMA Delivers Full-Featured EDM Functionality
ZEMA was developed according to industry best practices; as such, it fulfills all modern EDM requirements.
Clients’ data collection requirements are covered by ZEMA’s Data Manager application, a scheduling engine supported by a library of more than 4,000 data points from all major commodity exchanges, pricing agencies such as Platts and Argus, and other international and regional sources.
- Data feed maintenance:
ZE effectively detects upcoming data feed changes through its broad network of data
vendors and clients. ZE reprograms and deploys fixes, often before the data changes occur, to ensure minimal data disruption.
- New data development:
Organizations often must rapidly procure and develop new data feeds to react to market adjustments. ZE adds hundreds of new feeds per year to support new client requirements. ZEMA data feeds can be quickly built and deployed to capture data in any format and in any granularity. A new ZEMA data feed averages about a week in development time.
- Data quality control requirements are covered by ZEMA’s Data Validation monitoring system.
- Data storage is powered by Oracle or Microsoft SQL server databases.
- Metadata management embedded within the ZEMA database normalizes all incoming data.
- Metadata can be managed either through the Administration Console application or directly through the database itself.
- Security and entitlement management, resource management, auditing, and data compliance are controlled by the Administration Console application.
Industry-Specific Analytical Engine and a Robust API and Workflow Engine
ZEMA Market Analyzer is an energy commodity-specific engine used for data fusion, analytics, business intelligence, and forward curve development. Market Analyzer is the culmination of years of research conducted on behalf of some of the world’s largest energy firms.
ZEMA also features Curve Manager, an automation engine designed to persist Market Analyzer analytics on a scheduled or event-driven basis to downstream systems via integration modules called adaptors. ZEMA integrates seamlessly with major downstream enterprise systems, including internal trade and risk management, business intelligence, financial, and settlement systems. ZEMA does this in three ways:
- ZEMA Curve Manager can automatically feed downstream systems via a library of developed “productized” adaptors. ZE has developed adaptors for all major ETRM systems, SAP settlement systems, and many specialized BI packages, including Spotfire, Matlab, and Tableau. ZE also routinely develops new adaptors for new systems, when necessary.
- ZEMA Data Direct and Curve Portal allow two-way Excel integration, meaning users can log on to ZEMA securely via Excel, connect with a live feed of normalized data or analytics, do business processing via Excel, then upload their results securely back into ZEMA, with full audit records.
- ZEMA’s web service API allows firms to easily design their own integration points.Full Support and Future EDM Development Services
ZE offers an unmatched arsenal of services that allow firms to selectively control aspects of their EDM solution, either in house or in a ZE-hosted environment.
- ZEMA software upgrades:
ZE is constantly improving ZEMA and deploying the latest releases to licensed clients, allowing firms to benefit from cutting-edge advances in technology. ZE’s newest iteration, ZEMA 4, is the result of four years of research and development based on client feedback. ZE will continue to develop features, bug fixes, and updates to its core ZEMA technology to stay ahead in the EDM solution market.
- Application support:
ZE provides 24/7 services to troubleshoot application issues.
- Data integrity support:
ZE pinpoints and repairs any data problems a client experiences, develops tools to improve the quality of data assessed in ZEMA, and advises clients of ways to prevent unexpected data losses. ZE can also directly monitor data quality, upon a client’s request.
- Application training:
ZE provides both on-site and online training services through its branch offices in North America, Asia, and Europe. The company’s global presence means clients benefit from on-site training, regardless of their location.
- Consulting on demand:
ZE analysts have years of experience helping some of the largest global energy firms fulfill their analytical requirements.
ZEMA Improves the Total Cost of Ownership
Case studies of two major ZEMA clients will further reinforce what has been stated above: that licensing ZEMA cuts time-to-market by one-fifth (conservatively) compared to building an in-house EDM platform.
Case Study 1: A Power Firm That Initially Decided to Build an EDM System
This firm has been in operation for over 100 years and is one of the largest utilities in the United States; it has a recent market cap value of over $50 billion in USD.
The firm required an EDM system to collect electricity market data from three transmission jurisdictions, commodity exchange data, and internal operational and forecast data. The firm needed a system that also transformed and integrated aspects of collected data with several downstream systems, including an ETRM system.
Initial Decision to Build an EDM System In House:
The firm had a capable IT department and initially decided to build an EDM solution in house, setting aside a budget of $1,000,000 USD and establishing a take-to-market deadline of one year. One year and $2,000,000 USD later, the firm had not yet developed a full data collection and scheduling application. It had only completed one-third of the required data scrapers. The scheduling engine was not online yet, and the final price tag had been revised upwards to a total of $3,000,000 USD plus an additional year of development. Facing high project delivery risks, uncertain time-to-market, and a loss of confidence in the in-house approach, the firm reversed course and licensed ZEMA. It is important to note that the EDM system the organization had developed in house lacked a data validation system and an advanced analytics and automation engine – all critical components of modern EDM software.
250 individual data feeds, including:
- Electricity market data from ISOs such as CAISO, including high volume five-minute nodal data.
- Commodity exchange data from exchanges such as ICE and NYMEX.
- Weather data from organizations like NOAA and internal weather station feeds.
- Internal operational data, including internal market forecasts, counterparty forecasts, generation units, and gas pipeline transport data.
! ZEMA is a full-featured EDM software, as defined by the industry. Licensing ZEMA reduces project time by one-fifth when compared to in house EDM development.
ZEMA Configuration Selected:
ZEMA Enterprise Solution: Data Manager, Data Validation, Administration Console, Market Analyzer, and Curve Manager.
Licencing ZEMA was significantly more cost effective than the estimated in-house build; the solution was installed and configured within a month. By the third month, all required data feeds were developed, tested, and installed, and by the fourth month, the system was live and fully integrated with the organization’s downstream systems.
Total Cost Savings:
Ideally, the firm would have saved the majority of its $3,000,000 USD budget if it had decided to license a modern EDM solution from the beginning. Maintaining an in-house system is significantly more expensive than purchasing a solution from a commercial vendor, who has the ability to provide both 24/7 support services, data feed maintenance, and new data feed development services.
Case Study 2: Global Integrated Oil/Gas Firm Licenses ZEMA
This global multi-commodity firm is one of the top ten Global 500 firms; it is publicly traded and has a recent market cap value exceeding $200 billion in USD.
As the firm’s data requirements increased, it experienced two points of pain: one technical in nature and the other business-related.
- Business pain point – lack of data entitlement control:
The company decided to acquire an EDM solution as a result of the rapidly increasing cost of data subscriptions (an estimated $2,000,000 USD/year.) The key reasons for its high subscription costs were due to the following:
- Energy price reporting agencies charge high subscription fees and demand that firms report on employees’ data usage. The firm’s lack of entitlement data meant that it was charged a maximum amount based on its global headcount as opposed to its actual data usage.
- Some subscription feeds were licensed multiple times due to the requirements of several separate data management platforms in various business units.
- The firm had no clarity regarding data usuage; therefore, it could not easily determine which subscription feeds were unnecessary.
- Technical pain point – classic data mess problem:
This firm had many data feeds which were managed separately within two commercial data management platforms, plus an in-house spreadsheet-based system requiring manual data input. This effectively created separate data “silos” within the organization. Downstream systems consuming this data included three major ETRMs, an ERP, several BI systems, and user-query tools specific to each silo. The web of integration processes required to connect these data silos with downstream systems was highly complex and difficult to maintain, as each silo had different underlying architectures. The total annual maintenance cost for these systems was estimated at $1.5 million USD/year and growing.
ZEMA was licensed to fulfill the organization’s requirement for a scalable, flexible EDM system to centralize data collection processes, including internal firm-specific data; monitor the quality of data; control data entitlements; provide advanced analytical capabilities; automate tens of thousands of forward curves and analytics; and seamlessly connect with ETRM, ERM, and specialized BI tools.
800 individual data feeds, including:
- International energy price reporting agencies and brokerages, such as Argus, Bentek, Dow Jones, OPIS, Platts, Bloomberg, Reuters, and WSJ.
- North American electricity markets such as AESO, ERCOT, IESO, ISONE, MISO, CAISO MRTU, NBSO, NYISO, and PJM.
- Government agencies and weather organizations like EIA, NOAA, and WSI.
- International commodity exchanges, including CME, ICE, and NYMEX.
ZEMA Configuration Selected:
ZE-Hosted Enterprise deployment model: Data Manager, Data Validation, Administration Console, Market Analyzer, Data Direct Excel, Web Services, and Curve Manager. ZE developers also shipped custom integration adaptors to allow seamless communication between ZEMA as an EDM solution and the organization’s major downstream systems.
The firm set strict resource policies and security guidelines to control the ZEMA system’s integration process. A comprehensive consulting period was used to confirm system requirements, conduct data mapping, plan for data migration, and decommission legacy proprietary databases.
ZEMA was installed and tested sequentially in user acceptance testing (UAT) and production environments. Data feeds were developed, configured, reviewed, and deployed; integration adaptors were developed; and integration outputs were validated against the company’s existing system.
This paved way for the transition from a production environment to go live, followed by the phase out of current systems, user training, and an intensive support period. Finally, the system was fully turned over to the client. The project took eight months to complete. The finalized ZEMA solution collected data from 800 individual feeds and supported 850 users globally. It generated over ten thousand forward curves per day.
This firm’s IT department runs many global systems. As such, it chose to operate and maintain ZEMA on an enterprise-hosted basis. The firm continues to utilize ZE’s 24/7 support, including data feed maintenance services.
Total Cost Savings:
Total savings from entitlement alone are estimated by the firm at $1,000,000 USD annually in subscription costs. EDM platform centralization saved an additional $1,500,000 USD annually in maintenance and system swap-out costs. Industry specific analytics, forward curve and analytics automation, and ongoing EDM improvements continue to provide significant benefits over and above the organization’s previous solution.
In the energy and commodities industries, powerful enterprise platforms require a modern EDM system. A firm without a good EDM platform cannot use its corporate data effectively as an asset, and may lose its competitive advantage in the marketplace. Modern EDM software covers not only data collection, but also validation, administration, analytics, automation, and systems integration functions. Systems like this require proper funding for ongoing support and future development.
The case for “building” an EDM system from scratch hinges on an important business question – is it within a firm’s core business interests to develop EDM software themselves? If yes, then can the firm marshal team managers and developers with the right mix of domain knowledge to follow through on the project? ZE estimates that building a basic in-house EDM system requires a five-year budget of $5,400,000 USD. Initial development comprises three years and more than two-thirds of the budget, while ongoing operational and R&D costs account for the remainder. ZE’s experience indicates that most firms start building an EDM project without realizing the full extent of the time, resources, and expertise required.
The case for “buying” or licensing an EDM system is appropriate if creating EDM software is not an organization’s core business interest. ZEMA is a mature, market-proven platform that complies with best industry practices. Conservatively speaking, ZE’s implementation statistics and client anecdotes prove that a ZEMA project implementation requires only one-fifth of the time required to develop even a basic in-house EDM system.
Most firms do not have the right mix of human capital, financial resources, and time to properly develop an EDM system from scratch. Instead, licensing a full-featured, commercial EDM solution like ZEMA to integrate with existing enterprise platforms is often a smarter choice. From a project risk perspective, commercial EDM software like ZEMA has a well-defined risk profile and timeline. From a cost perspective, an individual firm with little experience in EDM software development would be hard-pressed to develop a system of similar quality, in a shorter time frame, on a smaller budget, and with less domain knowledge at its disposal.
1 P.C. Clements, “From Subroutines to Subsystems: Component-Based Software Development,” Software Engineering Institute, November 1995,
accessed February 1, 2014, http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=29995.
2 I. Ruchkin, “Building Software In house: Too Much Control and Flexibility,” Institute for Software Research, May 9, 2012, accessed February 1, 2014,
“To Build or Not to Build: Building Your Own Data Management System Versus Buying,”
Dataforensics, 2012, accessed February 1, 2014, http://www.dataforensics.net/pdfs/Build%20vs%20Buy.pdf.