Problems associated with programming large systems projects was compared to the tar pits of ancient history where many strong prehistoric animals were caught. The fiercer the struggle, the more entangling the tar, and no beast, no matter how strong has been able to break free. The end results of which is the beast is swallowed up into the tar pit. Large-system programming projects are like such tar pits, where many strong and powerful beasts have thrashed in it. Most projects were completed and are operable but few have been on schedule, within budget and have met the objectives. No one cause seems to be the problem, but the accumulation of various simultaneous and interacting factors. To become a generally usable programming product, a program must be written in a generalized manner. And to become a programming system component, a program must be written so that every input and output conforms in syntax and semantics with precisely defined interfaces. The joy of programming includes : making things happen; making things that are useful to others; making complex objects and watching them work; always learning; and working in a tractable medium. On the other hand, the woes are : one mu st perform perfectly; other people set one's objectives, provide one's resources, and furnish one's information; finding nitty gritty little bugs; debugging the next error is exponentially more difficult than the present one; and the product over which on e has laboured so along appears to be obsolete upon or even before completion!
More software projects have gone over schedule than for all other causes combined. Some major causes are : techniques of estimation are poorly developed; techniques confuse efforts with progress, hiding the assumption that men and months are interchang eable; software managers often lack the courteous stubborness to acknowledge their uncertain estimates; and when schedule slippage is recognized, the natural response is to add more manpower!
There are a few fallacious assumptions. First, often programmers are too optimistic, assuming that all will go well i.e. that each task will take only as it "ought" to take. Second, cost does indeed vary as the product of the number of men and the number of months. Progress does not! Hence, the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. Third, because of programmer optimism the number of bugs expected is usually smaller than the actual, as a result testing is most mis-scheduled part of programming. Fourth, the urgency of the client may govern the schedule more than anything else (even more than the required actual project schedule). Thus, to de-mythologize the man-month, the number of months of a project depends upon its sequential constraints and the maximum number of men depends upon the number of independent subtasks. From these two quantities one can derive schedules using fewer men and more months.
For a large number of minds to be coordinated affects the costs of the effort, for a major part of the cost is communication and correcting the ill effects of miscommunication (system debugging). This suggests that one wants the system to be built by as few minds as possible. But then the problem with the small, sharp team concept is it is too slow for really big systems. To reconcile the two issues, Harlan Mills suggested the use of chief programmer teams where each segment of a large project is tackled by each team. In each team, the chief programmer defines the functional and performance specifications, designs the program, codes it, tests it and write its documentation with another co-programmer. The rest of the teams acts as support personnel t o assist him in his tasks. In this way, conceptual integrity is maintained; there is lesser minds to design the system and lesser variation in conceptual designs. Conceptual integrity is most important in system design. It is better to omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas. If a system is to have conceptual integrity, someone must control the concepts.
There is a tendency for designers to over-design the second system, using all the frills and ideas that were cautiously sidetracked on the first one. Thus, the second is the most dangerous system a man ever designs. But a designer can avoid the the second system effect by being concious of the peculiar hazards of the system, and exert extra self-discipline to avoid functional ornamentation and avoid extrapolation of functions that are obviated by changes in assumptions and purposes.
The manual or a written specification is a necessary tool. It is the external specification of the product. It describes and prescribes every detail of what the user sees. Changes must be quantized and the manual must have dated versions that updates the changes. The style must be precise, full and accurately detailed.
An audit of the tower of Babel project shows why it failed. Firstly, they lack communication - they were unable to talk to each other thus, could not coordinate. And secondly, this la ck of communication led to disputes, bad feelings and group jealousies resulting in clans moving apart from one another. So in a large project, a formal project workbook must be started in the beginning. It is to be used as an official communication med ium between teams. It is a structure imposed on the documents that the project will be producing, it includes objectives, external specifications, interface specifications, technical standards, internal specifications and administrative memoranda. No mat ter how small the project, a manager is wise to begin immediately to formalize at least mini-documents to serve as his data base. The components of the mini-documents are : obejectives, product specifications, time allocations (schedule), money allocations (budget), space allocations, and people allocations (organisation chart).
For most projects, the first system built is often not useable : too slow, too big, too hard to use, or all three. This discard and redesign may be done in one lump or piece-by-piece, but it will be done. Both the actual need and the user's perception of that need will change as program are built, tested and used. Some valid changes in objectsives and in development strategies are inevitable. The techniques for planning a sofware product for change, especially structured programming with careful modu le interface documentation are well known. Also use high-level language, compile-time operations, incorporations of declarations by reference and self-documenting techniques to reduce errors induced by change. Program maintenance consists mainly of chang es that repair design defects, add incremental function, or adapt to changes in the use environment or configuration. Fixing a defect have a 20-50 percent chance of introducing another. After each fix, the entire bank of test cases must be run again to ensure there is no additional errors introduced.
The manager of a project needs to establish a philosophy and set aside resources for the building of common tools, and at the same time to recognize the need for personalized tools. High-level language im proves not only productivity but also debugging; fewer bugs and easier to find. The difficulties of system debugging justifies a thoroughly systematic and planned approach. One should begin system debugging only after the pieces seems to work. Finally, all in all, the tar pit of software engineering will continue for some time due to the infancy of software engineering. But, with time, it will mature and learn to avoid the tar pits or find ways of overcoming the tar pits.
Adapted from "THE MYTHICAL MAN-MONTH" by FREDERICK P. BROOKS, JR.