This section describes the basic elements of Eagle’s software development process.

Here are a few things that apply to the development process as a whole, which should be kept in mind while reading about each individual element of the process.

• Principles. Every aspect of Eagle’s development process is subject to fundamental principles and objectives discussed elsewhere. Please remember, while reading this discussion of methodology, that these principles always apply to every project and every stage of development.
• Project management. Similarly, every step of the process is also subject to our project management discipline, which is an essential part of our overall development methodology.
• Iterative-ness. Sometimes the development process is referred to as a “spiral”, where the spiral’s circular aspect is the repetition of the steps of the development process and its vertical aspect is progress toward completion of the software application.

The iterative approach is a balance between the rigidity of the “waterfall” method (where each step of development must be fully completed before moving to the next step) and the potential lack of discipline of the “evolving prototypes” method (where a prototype is created quickly with minimal design effort, revised based on observed shortcomings, and then the process is repeated until presumably the final desired result is obtained).

• Descending order of importance. Every step in the development process is more important than the next. Because each step becomes the basis for the next, the earlier you “get it right”, the more time and money you save.

At the very start of a project incorrect assumptions, omissions, or insufficient understanding can be corrected at little or no cost. With each completed step, correction becomes geometrically more costly and time-consuming. Many projects have failed because of simple misunderstandings that could have been corrected easily at the start of the project.

• Rule of necessity. One of the guidelines of the “agile” schools is “do what is necessary, not more, not less.” Because every project is different, it is appropriate to shape the prescribed practices of a methodology to the specific situation at hand.

For instance some projects have teams of 1 or 2 people and cost a few thousand dollars; others may have dozens of developers and cost millions. Some are quick-and-dirty temporary applications, others are in-house systems with limited users, while still others are world-class products with thousands of unrelated users. Some are simple GUI and database applications, others are components of sophisticated mission-critical industrial applications which must never fail.

A description of a methodology is an abstraction. Its application to a specific project should use just those elements of the methodology—no more and no less—that are required by the “rule of necessity” for that project.

• These are just the highlights. This is not a formal description of a methodology and makes no attempt to be “complete” in any way. The purpose of this section is just to share with you in a general way our view of the software development process.

This is the process of discovering the real needs which the new software must meet. The "need" is already understood at some level, of course, because that’s the force that gets the project started. But to be successful, the real needs must be understood in some detail, and implicit assumptions about these needs must be investigated.

A requirements document usually includes the following kinds of information:

• Who are the users? What are their roles and responsibilities? What information or functionality must the system provide in support of the users’ fulfilling their responsibilities?
• What specific usage scenarios (use cases) are envisioned? In support of each scenario what information must be supplied to the software? How does the information and functionality produced by the system relate to each of these scenarios?
• Is the above related to existing business practices? Are we seeking to streamline or improve existing business practices?
• Temporarily setting aside all practical issues, what would be the purely “ideal” picture of business practices, usage scenarios, system behavior and outputs you can envision?
• How much of the “ideal” can be put into practice? What are the approximate ratios of cost and benefit?
• What specific levels of performance are required?
• How will the software be maintained after it is completed? Will business conditions dictate frequent changes and enhancements or will the system remain relatively static? Will future responsibility for the system be in-house or contracted out?
• What types of user support will be needed? Is extensive context-sensitive online help required? Should there be a user manual? What is the expected complexity of user support? Does the system need to provide extensive information about what a user is doing in order for them to be supported?
• How are cost and benefit to be evaluated? What are the guidelines to be used for exploring the benefits of various elements of the “ideal” in relation to their expected cost?
• What kinds of extra flexibility or “over-generalization”—not needed now—should be built-in so future enhancements will be more economical?
• List the specific outputs (reports, screen-based information, exported data, data feeds to other systems) which must be produced.
• List the specific sources of information need to support the creation of the outputs mentioned above (imports, data feeds from other systems, user-inputs, etc.

Definition of the problem domain is a key early step in the development process. As in every other part of the development process there is ample opportunity for unwarranted assumptions, inconsistent interpretations of terms and concepts, and insufficient detail in descriptions of business entities.

It is especially easy to miss the boat here, because most stakeholders are likely to assume they have a good understanding of the problem domain and feel that developers are strangely thick-headed as they insist on detailed and precise definitions of domain terminology and concepts.

Anyone who has been through this process knows, however, that human communication and manual practices successfully and almost invisibly tolerate inconsistencies and all sorts of “special” circumstances. These subtleties in the problem domain must be seen and understood before one attempts to create software in support of these business practices.

The following elements of the description of the problem domain are maintained throughout the development process. It is especially important, however, to take them as far as possible at the very beginning of a project.

• Domain vocabulary. This is essentially a glossary of all terms and concepts relevant to the project. All definitions should be precise and complete. They should include a reference to the source(s) of the information and the date on which they information was obtained. Modifications, clarifications, questions and answers should also be separately noted, along with source and date information.

This document serves a number of purposes.

First, the very act of creating precise, agreed-upon definitions brings into the open many subtleties of the problem domain, leading stakeholders and team members to a deeper understanding of both the requirements and the opportunities inherent in the project.

Second, it creates stability and reliability in project communication. A business vocabulary consists of many interrelated, dependent terms. A common understanding of the nuances of one concept usually has implications for related concepts. Without clear, time-referenced documentation of each term, a dependable understanding of the problem domain will not exist. It’s worth mentioning that stakeholders and team members are often not aware that a common understanding has not been achieved.

Finally, the domain vocabulary serves as a kind of intellectual contract between stakeholders and developers throughout the entire project. As database entities, the object model, and outputs of the system become more distinct, the domain vocabulary remains a point of reference by which the validity of each system element can be judged.

• Object model. A side effect of creating the problem domain vocabulary is the ease with which an initial set of business objects can be identified. Our documentation of the object model initially describes these objects at a fairly high level, indicating their purpose and a preliminary list of services which each object will perform. It also shows the relationship of these objects in terms of usage and hierarchy. Later, during the design phase, this document is supplemented with a more detailed description of object properties, behaviors and relationships.
• Database definition. Closely related to the domain vocabulary and object model is the specification of the database. This document, too, begins at a fairly high level, simply identifying database entities, and their relationship to domain concepts and business objects. As the project unfolds, especially during the external and functional design phases, the database definition becomes extremely precise, with exact definitions for each data element, specification of data types, permitted values and ranges and other constraints, entity relationships, and initial recommendations regarding indexing.

In software development, “risks” are unknowns which could significantly affect the success of a project.

The goal of risk-analysis is to eliminate as many of these unknown factors as early in the development process as possible in order to avoid the waste of time and resources on efforts that are ultimately unsuccessful or ineffective.

By making an explicit effort to evaluate risks early in the project, we can shape the remainder of the project to use technologies, concepts, or tools which have been shown to be appropriate and effective for this project, greatly minimizing the chance of a failed project or one which costs substantially more than originally planned.

The following is a sampling of some of the risks a software project may face. Every project is different, with its own profile of risks—not all of these risks apply to every project.

• Performance. Minimum performance levels are required for specific functions. Need to know whether those levels can be met.
• Reliability. The application requires essentially 100% uptime, no runtime errors, and completely accurate outputs. What needs to be in place in order to insure this level of reliability?
• Fitness for purpose. What is the likelihood that the finished application will not serve the intended purpose? At what points in the project are specific measures needed to guarantee this does not happen?
• Will users like it? Is there a possibility that users will not like the application and will not want to use it?
• Will customers buy it? This risk is not primarily the responsibility of the software developer, but it is one the developer should be concerned with. With respect to what the developers and stakeholders know or should know, are we creating a product that customers will buy? Has the requirements analysis phase adequately addressed this question? What are the attributes of the application that will affect the customer’s desire for the product and their decision to buy (and recommend) the product?
• Will people use it? What are the reasons people may not use the application? What is the ratio of user-effort to user-benefit? How does this affect the requirements and design of the application?
• Reliability of development tools. Are there any doubts about performance or reliability of the development tools for the project? About the database to be used, or third-party software components? Are any new technologies being used whose reliability is unknown? Are we using any data structures or communication protocols that are new or non-standard and may not be compatible with external resources? If the application is reliant on third-party components, what is our fall-back position if the third-party vendor goes out of business or ends support for the product we are using?
• Validity of time and cost estimates. What is our level of confidence in time and cost estimates? How much of a buffer does the budget allow for contingencies? To what extent do the time and cost estimates allow for changes of scope during the project? How do we measure the cost-effectiveness threshold in relation to “go / no-go” decisions on the project?
• Valid closure on requirements. What is our confidence regarding the validity of the application’s requirements document? How do we verify that all stakeholders understand the requirements and agree with them? How do we avoid the problem of stakeholders as a group being wrong about some of the stated requirements? Where are the areas in this project where that might happen?
• Dependence on peer-systems. Does this application depend on any peer-systems for its operation? For example, does it communicate with any third-party systems via web services? Does it interact with in-house systems whose correct operation is essential to this application? What is a justifiable level of confidence in the correct and timely operation of these peer-systems? Do we have fall-back options if and when these systems fail?
• Dependence on infrastructure. What are the critical elements of infrastructure—such as networks, servers, switching, desk top computers, storage systems, etc.—that have unknown or unpredictable reliability? Does this application require any of these elements to have an exceptionally high level of reliability? If so, what is our fall-back position, or what can be done to guarantee the required reliability?
• Functionality of third-party components. Do all third-party hardware or software components required for this application have the required functionality? For example, if a large database is required for the application, and low-latency replication is essential, do we know for a fact that the candidate DBMS actually delivers the functionality and performance levels we expect?
• Validity of algorithms / heuristics. If the success of this application is dependent on proprietary algorithms or heuristic methods, do we know for a fact that these methods are valid and effective? Do we need to objectively test the validity and effectiveness of these methods before proceeding further into the project?
• Conformance to corporate standards and procedures. Are there corporate or organization standards which affect the time, cost and/or approach to development of this application? Are there standards regarding development language, design methodologies, documentation standards, approved third-party tools and components, testing methods?

The third prerequisite for design—after requirements analysis and problem domain definition—is to specify the overall architecture of the application. In this phase we are emphasizing structure of the application from the point of view of high-level responsibilities, and the disposition and dependence of these components in relation to each other and to the hardware / software infrastructure.

The form of the architecture specification will vary greatly depending on, among other things, whether the application is large or small, web-based, database intensive, etc.

In any case, a typical architecture document will address the following areas, as appropriate for the application:

• Physical infrastructure. Description of the network / communications environment in which the application is designed to operate. Specification of security-related issues, including the use of VPN’s, SSL, firewalls.
• Middleware and servers. Specification of required application servers, web servers, database servers, and related components, and server hardware.
• Implementation languages / framework. Specifies, for example, whether application is implemented using .NET, Java or other languages, plus related details such as whether EJB’s are used, and how they be deployed.
• Structure of User-interface, database access. Description of how the application will organize and separate responsibilities for UI, database access, and business logic.
• General structure of application components. High-level diagram showing the disposition of the main functional components of the application in relationship to hardware and middleware components.
• Application-specific development policies. Specification of design and implementation policies for this application with respect to such things as scalability, exception-handling, security, multi-user access, fault tolerance, availability, performance, data types, report implementation, use of database triggers, stored procedures, etc.

The goal of external design is to create a description of all elements of the application which interact with users or external systems.

The documentation and UI prototype which constitute this deliverable make it possible for stakeholders to manually “execute” usage scenarios and to confirm for themselves that the design, to this point, is compatible with their requirements and expectations.

The deliverables for external design normally consist of the following:

• User-interface. Documentation for the entire user interface. This includes menus, all screens and dialogs (or web pages), and any significant warning or error message windows. If the UI is at all complex, a UI navigational diagram is also provided which shows all possible routes through the user-interface and the conditions under which they would be taken.

A UI prototype is also created, which allows users to explore the user-interface, seeing exactly how it will look in the finished application. At this point the prototype consists solely of UI; no functionality or database access is enabled.

• Reports. In business systems, the key importance of reporting is often undervalued. Perhaps we should say: “If this system could endow you with perfect, effortless insight into a specific area of your business, what would you see? What would you know?”

That should be the starting point for report design. Yes, of course there are reports in any system that are utilitarian or mandated by legacy procedures. But all reporting, utilitarian or not, should aim to present exactly the needed information, in the simplest, clearest way possible, and should allow the user maximum flexibility in controlling how the information is selected, sorted and organized for physical presentation.

In other words, all your reports should look great, should be exceptionally informative and easy to understand, and you should be able to easily obtain from the system almost any information ever needed.

• Data imports / exports. Data imports and exports usually exist for one of two reasons. One, to communicate with other systems, either in-house or controlled by business partners. The other, to create data files you can easily manipulate with office tools such as word processors and spreadsheets.

The external specifications should identify each import and export, indicate its purpose, list the information to be exported or imported, and describe user options for selection and sorting.

• Web services interactions. If the system will interact with other systems via web services, the external design documents should describe the nature of the interaction, including its purpose, functional requests, and information exchange.

Once the external design is accepted by stakeholders, the next step is to specify how functionality is to be implemented and how the external elements and business logic relate to each other, to databases, and to external systems.

Here it is important to remember the “rule of necessity”, and to provide sufficient specification, but not more than is needed. It’s crucial to realize that “sufficient” in this case refers to “which information” is provided, not “how much.”

A good example of this distinction is often seen when designing reports. Many reports provide users with interpretations of data. These interpretations transform “raw” data into meaningful values corresponding to domain terminology. It is essential to specify this transformation with complete precision—not in terms of how it is done, but in terms of what is done. This is “sufficient.” It may also be appropriate to state how the transformation is to occur, but the designer must distinguish between the necessity of “what” and the possible convenience of “how.”

Although implementation strategy can vary considerably depending on whether the project is large or small, is an in-house system or a commercial product, and so on, here are a few key points that apply to all projects.

• Plan implementation strategy. There’s no formula for the “best” implementation strategy, but here are some factors to consider. It’s usually a good idea to attack difficult tasks and risks first so any impact on cost, duration or architecture can be known as soon as possible. The approach of creating a small, operational version of the application and then building on that will affect the order in which components are implemented. Other obvious factors are the need to assign tasks based on the skills of team members and the need to level resources in order to minimize elapsed implementation time.
• Create small core application; build on it. Whenever possible, we create a small, operational “core” version of the application, to which additional components can be added as they are completed. By choosing the components for the core application appropriately, we can insure that underlying technologies, architecture, third-party components, development platform and frameworks, middleware, hardware infrastructure, and so on, operate correctly and with expected performance and reliability—and if problems are encountered, we can address them early, minimizing the amount of implemented code that might have to be changed.
• Unit testing. Each developer is responsible for effective unit-testing of his/her components and for initial integrated testing of the component when it is added to the running core application.

The function of the QA specialist is to insure that the gradually developing application satisfies the original project requirements in terms of functionality, reliability, performance. On the basis of the system design specifications, the specialist creates test plans which will reveal any failure to meet requirements.