Maintenance (which we mean here to include customer support, correcting of defects, and enhancements) not only consumes a large proportion of total system cost, but improving the quality of maintenance practices could also lengthen the useful life of a software system. Thus the economic incentives for maintenance process improvement are strong. In examining software maintenance processes for improvement opportunities, an obvious choice is information flow. Obtaining accurate, up-to-date, and understandable information about the system being maintained is a difficult task because the sources of information are often limited, inaccessible, or unknown. Currently, software maintainers rely on various sources of information about the system they are trying to modify, adapt, or update. These sources range from artifacts of the development process (requirements documents, design documents, documentation within code, test plans and reports, etc.) to user documentation (user manuals and configuration information) to the system itself (both the running system and the source code) to direct access to the system’s original developers and current users. Many of these sources, especially those in the first category, are produced at least in part to facilitate maintenance. A lot of effort is required to both produce and maintain the documentation, but little effort has been made to determine how useful it is for maintenance or how to best package it for effective use during maintenance.
The objective, then, of this research
program is to improve maintainers' access to
relevant information about the systems they are maintaining. Our
approach is empirical, meaning that any
improvements we propose must be derived from empirical evidence concerning how
maintainers currently gain information about the systems they are maintaining.
This project includes studies aimed at gathering empirical knowledge about the
problem, as a necessary prerequisite to empirical validations of process
improvements suggested by this knowledge.
The early work in this program was organized as a
series of empirical studies that aimed at gaining knowledge about how maintainers gather system
information and building on that knowledge to experiment
with changes in how they gather this information. Industrial participation was and continues to be
a key ingredient in the success of this work. This participation is crucial not
only in order to obtain valid and useful results, but also to ensure that the
significant long-term potential of this work is realized.
This line of research began with a small preliminary study using students in a graduate course on system maintenance at UMBC in the fall of 1999. The data from this study gave insight into the range of information sources that are used under different conditions when a maintainer is faced with the task of trying to understand an unfamiliar system. This information fed into the design of a survey, which was distributed to software maintainers in companies and government agencies in 2001-2. The survey addressed what information sources are used for maintenance under different conditions. Maintainers from four different organizations completed the survey and the results have been reported in the literature (see below for references). The major findings include:
· Maintainers overwhelmingly rely on source code as an information source, but some other sources are at least as valuable under some conditions.
· The desire for CASE support in maintenance organizations is limited to support for basic functions such as CM, debugging and editing, managing execution traces, etc., and for tools that are tailored to an application domain.
· The role of human sources of information in a maintenance environment is highly dependent on the organization, its culture, and the history of the system being maintained. Human expertise is always valuable, but which classes of people have the more valuable knowledge varies from situation to situation.
· There is evidence that effectively implemented lessons learned repositories would be of great benefit to maintainers, by decreasing their reliance on human sources of information, but current systems are not effective.
In the fall of 2002, a study was
conducted using students in another graduate software maintenance course at
UMBC. The purpose of this study was to compare the effects of the availability of
different types of information on the effectiveness of a software maintenance
task. Specifically, we tested whether
maintenance teams who have system documentation available to them are more or
less effective (in terms of level of effort, number of defects found in
testing, and amount of rework) than teams with an original developer available
to them or teams with both sources of information available. The data from this study will soon be
published.
Starting
in 2005, our work has focused on the possibility of building effective lessons learned
or experience
repositories for software maintainers, motivated by some of the
results mentioned above. in particular, we are exploring the potential role of
boundary
objects as a facilitator of information
flow for software maintainers.
Boundary objects are documents that serve at least two different
purposes for two different sets of workers performing at least two business
functions in an organization. Our
eventual aim is to build an experience repository
out of software artifacts identified as boundary objects and properly
contextualizing them. Our hypothesis is that we will be able to identify the
documents and other artifacts of the development process that best capture the
experience of the people involved in that process in a way that is useful to
future maintainers, and that we will be able to apply strategies for building
experience, or lessons learned, repositories to effectively deliver this
information to maintainers. Our research plan includes the following:
· identify and contextualize the boundary objects in a software development and maintenance environment;
· gather "war stories" about successful communication mechanisms between developers and maintainers;
· prototype an experience repository with very limited scope within an existing software maintenance environment and evaluate the effect of the lessons learned system on productivity and other factors.
We have to date conducted “war story” interviews with experienced maintainers in three software development and maintenance organizations. Preliminary findings include:
We are currently seeking more organizations with whom to investigate the use of information, particularly documentation, in software maintenance. We are continuing to analyze the data from our “war stories” interviews and synthesizing it into guidance for designing and deploying an experience repository. We are also seeking funding for further work.
If you are an organization that maintains software, either for in-house use or for external customers, and are interested in participating in this research, please contact us at cseaman@umbc.edu or lutters@umbc.edu. There are a variety of ways for a software maintenance organization to participate in this project and to reap benefits in terms of maintenance process improvement. We are interested in tailoring our research objectives to the needs and interests of our industrial partners. Collaborative possibilities include gathering stories from experienced maintenance practitioners about what has and has not worked for them, an exploratory survey or set of interviews aimed at discovering the root problems related to information gathering for maintenance, or the evaluation of proposed improvements (including the development of an experience repository) in the maintenance process to ensure that they produce benefits to the organization. In all cases, we will provide feedback on our findings to your organization in the manner most useful to you, including presentations, seminars, written reports, tutorials, or ongoing consulting on future improvements.
CAREER:
Information Sources for Software Maintenance, Carolyn B. Seaman, proposal
submitted to and funded by the National Science Foundation, July 1999.
The Information Gathering Strategies of Software Maintainers. Carolyn B. Seaman. Proceedings of the International Conference on Software Maintenance, October 2002, pp. 141-9.
The Value of War Stories in
Debunking the Myths of Documentation in Software Maintenance, Wayne G. Lutters
and Carolyn B. Seaman. Information and Software
Technology. [in press]
The early work described here was supported by the National Science Foundation under Grant No. 9984047.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Last updated on January 11, 2007 by cseaman@umbc.edu