Introduction

This chapter deals with the issues associated with the organization, management, and high-level planning of the testing process.

A generic approach to organizing the testing process is described, which can be applied to businesses involved in developing and/or testing software. The chapter discusses the implications for the testing process of the size of the business, its prevailing quality culture, and (where appropriate) the maturity of its software development process.

The case studies in Part 2 provide specific examples of how a number of different businesses have customized this generic approach in response to their own particular organizational, management, and testing requirements.

This chapter also considers a number of key requirements that must be considered in implementing a rigorous, effective, and efficient testing process: the need for configuration management and defect tracking and the need to ensure that each of the requirements of the Application Under Test (AUT) has been verified. The issues associated with each of these requirements are discussed and recommendations for supporting them are provided.

The chapter concludes with a discussion of the role of risk-based testing and its use in the management and planning of testing projects.

Overview of the Organization

The British Library (BL) is the national library of the United Kingdom. Its work is supported by the expertise of over 2000 staff members and by links with other organizations worldwide. The Library provides:

1. ▲ Internationally important reading room and enquiry services

2. ▲ The world's leading document supply services

3. ▲ Specialist information services in key subject areas

4. ▲ On-line and Web-based services

5. ▲ Essential services for the library, archives, and information world

6. ▲ Library facilities for the general public.

The Library's outstanding collection, developed over 250 years, of more than 150 million items represents every age of written civilization, every written language, and every aspect of human thought.

BL can be seen as the most complete of all the case studies, embodying aspects of each of the process elements described in Part 1 of this book.

Characteristics of the Testing Requirement

The British Library has a significant reliance on software systems to support its business for the purposes of:

1. ▲ Maintaining very large quantities of information on document holdings

2. ▲ Maintaining extensive lender personal details

3. ▲ Recording details of document borrowings

4. ▲ Maintaining on-line catalogue searching (involving millions of records)

5. ▲ Supporting large-scale document supply.

The majority of these systems need to interoperate in order to inspect or exchange data and/or to invoke operations on other systems. The systems in use are of various ages, based on a variety of implementation technologies, and frequently supported on different operating systems and hardware platforms.

In this book we've talked a lot about programming using ENVY System classes. At the simplest level this involves implementing methods such as loaded or removing on our own application classes. At the most complex level, we've seen code to create checkpoint versions directly out of application editions in the repository. All of this requires manipulating the classes that make up ENVY itself.

This kind of programming can be quite difficult. Although we refer to this as an API, many of these classes and methods are not well documented and have changed over time. The most difficult part of writing ENVY system code is understanding the relationships between components and finding the methods necessary to manipulate these relationships. This is complicated by a few factors. First, the components are not directly related through instance variables; rather, many of the most important relationships are implicit in methods. Further, these methods are not concentrated in the components themselves, but are spread out over three distinct groups of classes: the browsers, the “helper” classes such as EmInterface, and the components themselves.

In the earliest versions of ENVY almost everything was done directly in the browser code. Over time much of the code was moved to the components themselves or to utility classes such as EtTools and EmInterface. This reduced code duplication but the utility classes became very complex, with many methods that manipulate different types of components, making it harder to find any given piece of functionality.

So far, we've seen ENVY as a code organizing tool and as a version control system. These are powerful facilities, but the most important aspects of ENVY are in its support for working in teams. ENVY differs from traditional team programming tools in many respects. Because it is a Smalltalk-based tool it can deal with software components as objects, rather than working with coarse-grained source files. It also breaks away from the check-in/check-out mechanisms to a more concurrent mechanism in which conflicts are resolved through component ownership and through separation of the versioning and releasing operations. This enables developers to work with maximum concurrency while still providing a disciplined process that maintains the consistency of the system.

Team Development with Class Editions

Recall that applications specify a group of class versions that are loaded when the application is loaded, and that these are called the released versions. As we look at team development, this concept becomes more important, and we will see how class and application editions and versions are used to control the team development process.

Before we begin, let's review what it means for a class version to be released:

1. ▪ When an application edition is loaded, the released versions of its classes are automatically loaded.

2. ▪ Once an application is versioned, the list of released class versions cannot be modified.

3. ▪ Most important, the released version is the approved or official edition. If we release a class, we are saying it is stable enough and approved for other developers to use.

In a software project, as in any large project, a significant amount of work is needed to close the project. Teams often defer the hard issues of finally delivering a product until much too late in the process, leading to trouble when the time finally arrives.

This chapter examines the ENVY notion of prerequisites in depth, sees how it can affect packaging, and looks at resolving various prerequisite problems. We will look at various alternatives for the packaging and delivery process and their relative advantages and disadvantages. We'll describe in some detail the packaging and delivery process for both VisualAge and VisualWorks. We'll look at application attachments, an ENVY mechanism for keeping track of external files associated with our program. Finally, we'll address some additional delivery issues and strategies for ensuring that packaging does not become a bottleneck.

Prerequisites Revisited

Before we examine program delivery strategies, it's worth revisiting some of the concepts we've previously seen that have a significant effect on delivery. Regardless of our strategy, knowing what other code our program relies on is important if we are to produce a simple, standalone program. Recall that ENVY provides the notion of application prerequisites to help formalize this information. Some packaging strategies, particularly those using the VisualAge packager, rely heavily on correct prerequisites to determine which components need to be linked and which classes and methods can be discarded. Even if we're not using such a mechanism, ENVY prerequisites can help define a layered architecture with all the dependencies made clear.

This is the chapter to read when you have trouble. We'll discuss dealing with components that won't load, recovering from simple image crashes as well as more complex ENVY crashes, and what you can do to make recovery easier. We'll also discuss more generic project troubleshooting to help you identify potential ENVY-related problems in a project, and suggest some possible remedies. The main topics we won't discuss here are installation and configuration problems, which we covered in Chapter 1.

Component Loading

One of the most common ENVY problems is that component editions will not load. A load may fail for many reasons. Two of the most common reasons are that a component cannot be constructed in the image (for example, the source for a method cannot be compiled or a required superclass is missing), or that loading a component results in the image being inconsistent (for example, a subapplication, class, or method would exist in two different places).

You have three main approaches to correcting a load failure, depending on the situation:

1. ▪ altering an ENVY system setting that affects loading behavior

2. ▪ modifying the component editions being loaded

3. ▪ modifying the image

In the previous chapters we've rapidly covered almost everything you need to do basic development with ENVY. This chapter deals with some of the more complex issues that arise in a larger project. The two primary issues we'll address are configuration management and project organization. ENVY supports sophisticated configuration management that enables us to write portable software that can also exploit platform features. Project organization includes advice on how to structure the ENVY components that make up a project, a discussion of multi-site development, and ways to manage multiple divergent streams. We also discuss some alternative processes to the standard we've been describing so far, explaining how ENVY can be used as part of an “Extreme Programming” project. Finally, we'll add some miscellaneous topics, tricks, and conventions that ease development.

The discussion here is aimed at someone who has a good understanding of the basic ENVY concepts and has used ENVY at least a little. Much of this advice will be most helpful to someone setting up a larger project. New users should begin by reading some of the previous chapters and working with the concepts discussed in those chapters. Toolsmiths will find this discussion important for ways to organize tools for multiple versions or dialects, and will appreciate some of the tips and conventions.

Configuration Management

The distinction between version control and configuration management is not clearly drawn. However, in our context we will identify configuration management as being concerned with variations in our software due to different configuration information.

This book describes ENVY/Developer and its use by three very experienced professional software developers who have used it to build both successful products, including Top Link and VA/Java, and applications for numerous customers. Their unique user-oriented perspectives give developers and managers insight into how to use the product successfully in team development projects.

ENVY/Developer began as Orwell (OOPSLA reference), a research prototype developed in the Object Oriented Research Group, at Carleton University. It was built as part of the Actra project; a multiprocessor embedded Smalltalk, funded by the Defense Research Establishment Ottawa. Orwell addressed the problem of coordinating multiple developers working in the OO RAD environment of Smalltalk.

In designing Orwell, we sought to address the needs of our funding agency DREO, in particular Dr. Brian Barry who would later become CTO, then CEO of Object Technology International (OTI) and was interested in versioning configuration management for Smalltalk. Our goal was to meet the needs of our development team, which meant preserving the incremental nature of the Smalltalk programming environment.

What we developed at Carleton, then quickly re-implemented at OTI, was a strong specific solution for Smalltalk. Little did we know at the time how strong and specific it was to be! It was developed as an internal tool to support building embedded systems, hence the strict rules for compatibility, which were designed to reduce the downstream problems of the packaging or roaming engineer.

Smalltalk was first built as a personal computing environment. Most systems at that time were time-sharing, with character-based terminals. In contrast, Smalltalk provided a high-resolution graphical screen and a mouse to maximize the power available to the one user actually sitting at the machine. Smalltalk was designed so that every component in the system was accessible to the user and could be understood by one person.

This was an outstanding vision, and one that remains both important and ahead of the mainstream even today. However, this emphasis on the personal meant that cooperative development issues were less well represented.

Individuals could be incredibly productive, but the code resided in each developer's image and was not directly visible to other team members. The code could be filed out and exchanged between developers but on an ad hoc basis. The way Smalltalk represented code, and the development practices that Smalltalk encouraged were not a good match for the file-based version control systems of the day. As Smalltalk became more widely used in industry, the need for support of large teams became more critical.

Different schemes were attempted that built relatively lightweight Smalltalk layers on top of existing team programming systems. Finally, in the mid-1980s, Carleton University Professor Dave Thomas and some of his students broke the mold and came up with Orwell, a repository-based system that was tightly integrated with the Smalltalk environment.

The previous four chapters have described ENVY concepts in a tutorial form, helpful for developers getting started with ENVY and advanced developers looking for detailed information and motivation for features. In this chapter we take a different approach, describing the ENVY feature set in a more formal way. This is more appropriate for readers with a background in configuration management seeking to understand the differences between ENVY and other systems. It's also appropriate for users familiar with ENVY who want to understand more about the theoretical motivations behind certain features. This chapter covers most of the material explained in the previous four chapters but in less detail and from a more theoretical perspective.

ENVY Products

Technically, ENVY is a generic term for an entire product line from Object Technology International (OTI). Most commonly, it is used to refer to the team programming environment ENVY/Manager. ENVY/Manager has existed for a variety of different Smalltalk environments. It was originally written for Digitalk's Smalltalk/V line, ported to ParcPlace's VisualWorks (now owned by Cincom Systems), and was then tightly integrated into IBM's VisualAge product line, supporting both the Smalltalk and Java versions. In this chapter we use the term ENVY to refer to the ENVY/Manager environment, as it exists for VisualWorks and VisualAge Smalltalk.

Components

The fundamental concept in ENVY is that of software components. We will begin introducing the different types of components: applications, subapplications, classes, class extensions, and configuration maps.

Smalltalk is a wonderfully open environment. Some or all of the source code is available to developers, and can be changed to do almost anything. From an experimental point of view this is wonderful. We can “burn the disk packs” and invent our own new language and environment. Unfortunately, the people who are paying us may have more prosaic goals in mind.

Our changes may need to be reviewed every time there's an upgrade or a patch to the basic class libraries. Inadequately tested extensions can be more trouble than they're worth, and managers of large projects with many developers are often reluctant to have every developer soup up his or her personal environment.

If we make changes to base classes, they may conflict with third-party products, with other teams in the organization, or even with customers if we're distributing development tools. System changes also scatter code around. If we modify a class in Kernel, then our system's configuration map will have to include our special version of Kernel. Before long we're loading dozens of applications, each containing only a few lines of our code. These all have to be managed by Library Supervisor, complicating maintenance.

On the other hand, changing system code offers very significant advantages. Sometimes the system is wrong. It may have bugs, or be missing functionality that requires changes to existing classes. Power-users like to extend their environments because it makes them more productive.

Before you can start to use ENVY effectively, many questions need answering, even if you already understand Smalltalk development. How do I write code using ENVY? How is that code organized? How do I coordinate with other team members? What's a configuration map?

This chapter sorts out the basic concepts in ENVY, and starts you on the path toward being a productive developer. We start out by discussing development as an individual. In Chapter 3 we move on to team development, and Chapter 4 covers more advanced topics.

The discussion here is aimed at a new user who has at least some experience with Smalltalk development. We'll discuss not only the features but their motivations and best practices, illustrating with examples. The chapter starts out simple, but things will get complicated quickly. Be patient; it'll take some time to become familiar with all of these concepts. More experienced users will find the best practices most interesting, if only to find things they disagree with. Those users looking for a shorter, more formal description of ENVY concepts should see Chapter 5.

Questions and Answers

The example project we'll be using is a simple questionnaire application. There are questions, and there are answers. Questions are strung together into groups, and depending on the answers to one question, the follow-up questions may change. For example, suppose we asked the question, How would you rate this book? If the response was Poor, then we don't need to ask any more questions because the user clearly has bad judgment.

To start with, we'll introduce ENVY/Developer, go over enough of the basic ENVY/Developer architecture to get you started, and describe in some detail how to set up the environment and configure the options available. All readers will want to review the architecture section. The setup sections will be of particular interest to new users and to administrators responsible for the maintenance of an ENVY installation.

Overview

First, let's identify what ENVY/Developer is, where it comes from, and what it's good for. This information will help you understand the architecture and where ENVY's concepts come from.

What Is ENVY/Developer?

ENVY/Developer is a software engineering environment for Smalltalk programming. Specifically, it provides facilities for team programming and delivering significant-sized applications, built on top of the regular Smalltalk development environment. Smalltalk is widely recognized as being very productive for individual developers, but the standard Smalltalk environment was not designed with the idea of collaborative development in mind. ENVY/Developer extends the basic Smalltalk environment to include facilities for team programming and configuration management.

Technically, ENVY is a generic term that refers to a family of products and technologies. These include ENVY/Smalltalk, ENVY/Developer, ENVY/Replicator, and others. However, ENVY/Developer is the most widely used of these products, and it is common usage to use just the term “ENVY” to refer to ENVY/Developer, or even more specifically to ENVY/Manager. Although this is technically incorrect, it is the common usage and is the standard we follow for this book.

The administrator of an ENVY system must look after the many necessary tasks to ensure that the system and the development process run smoothly. This chapter describes some of these tasks as well as tools that make life easier for an administrator. We first describe the tasks associated with managing ENVY users, followed by a discussion of moving code between different images, repositories, and development sites. Administration also includes responsibility for the repository – managing its growth and dealing with system upgrades and the needs of remote developers. Many administrative tasks are repetitive, and we include a section on writing scripts using the ENVY API to automate some of these tasks. This is followed by more abstract discussions on organizing an ENVY project and maximizing code re-use. Finally, you'll find some general tips and tricks for keeping an ENVY system running smoothly.

Obviously, this chapter is primarily of interest to administrators. Power users who want to understand more about the details of ENVY will also find it valuable, and toolsmiths may find the process descriptions and tools useful. Also of interest with respect to administration are Chapter 1, “Getting Started,” which covers installation and setup of an ENVY system, and Chapters 3 and 4, “Team Development” and “Advanced Development,” which address some of the code organization and project management issues that affect the administration of an ENVY system.

While ENVY provides an extensive toolset, tools and enhancements can make life easier for a developer or administrator. This section presents a few such tools, with example source code and explanations of the techniques involved. For full code, including complete source code for these tools, see the book's Web site. The tools we will look at are as follows:

1. ▪ A script manager, which organizes a collection of scripts inside user fields in the repository.

2. ▪ A three-way differences browser, which streamlines the process of merging two divergent streams together.

3. ▪ A simple project management tool to support the layered architecture described in Chapter 4.

4. ▪ A checkpointing facility, which creates versions of a component out of editions without modifying the underlying editions, tremendously simplifying backup, intermediate baselines, and management of multiple repositories. We think this is one of the most useful and important ENVY add-ons we've seen.

5. ▪ An integration of the Refactoring Browser facilities into the standard ENVY browsers.

6. ▪ A way to rename versions in the ENVY repository without creating new editions and versioning them.

7. ▪ Ways of removing source code from ENVY components so that we can deliver libraries to other developers without exposing all of our internal source code.

The material on concepts and on using these tools is appropriate for novice developers and administrators. The material on implementation techniques is very advanced in some areas and is primarily intended for power users and toolsmiths.