Waterfall and software

methodology
Software engineering 1
Waterfall

 Originated with construction and manufacturing

 We talked last time about how there are lots of similarities between software and other types of engineering.
 Often direct comparisons or translations are really difficult and end badly.

 Waterfall looks great on paper:

 Gather all the requirements
 Plan to meet the requirements
 Confirm the requirements
 Meet the requirements
Don’t go chasing waterfalls

 The intent of waterfall is to reign in software complexity

 Limit changes after implementation begins by gathering all requirements up front
 Ensure that all requirements map to some code and all code maps to some tests
 Ensure that the client and the software team are on the same page for a solution to a problem

 In practice, it’s hard to gather all requirements up front and client expectations and needs can change
 “boiling the ocean”
 Trying to predict today what you will need 5-10 years from now is difficult even for an individual
Impacts of waterfall

 A 1995 report indicated that waterfall fell short in a lot of its core goals
 31% of all software projects were canceled
 53% are completed under much duress
 16% are “successful”
Nothing stays the same

 Research shows that ~30% of requirements (if you can even get them) change during the development of a
product.

 Technology is undergoing massive change basically all the time. What is cutting edge today might be
completely obsolete by the time a full product is finished.

 Dependencies are also constantly changing: libraries are updated/deprecated, operating systems change,
people join/leave teams, etc.
IBM OS/360

 This is one of the most famous over-budget projects.

 It spurred Fred Brooks to write the now infamous book “The Mythical Man-Month.”

 Because it was so far behind, IBM ended up providing various “stop gap” solutions (BOS, TOS, and DOS),
none of which were a fully featured OS as clients expected.

 The original project was written entirely in assembly for a specific set of IBM mainframe machines. The
hardware worked reliably, but the software was a flop.
Addendum 1 – Change anticipation

 If changes are accounted for in the design phase, they can be controlled by modularization, parameterization,
genericism, etc.

 This would theoretically allow you to apply waterfall to a framework that results in an application.

 This approach has limited success, as significant resources can be spent genericizing components of the
system that are never changed while change requests are constant for a component assumed to be static.

 Going back to business, rigid software leads to opportunity cost.

Addendum 2 - Prototyping

 Combat the “I’ll know it when I see it” client approach by creating a prototype to show the client.

 Problems:
 Non-zero up-front cost
 Client can still change their mind after approving the prototype

 Note that many frameworks now make this approach far more palatable.
 Example: HTML5 >>>>>>> static mock-ups
Software Evolution

 So far, we have primarily talked about the construction of software except for the maintenance phase of
waterfall.

 What if we extended that phase to the logical extreme?

 Present the client with a minimum-viable-product (MVP) and then iteratively refine.

 Software evolution is an idea taken from biology. Possibly you start with a VERY rudimentary system that
over a large number of iterations becomes extremely complex.
Prototypes vs MVP

 The main issue with a lot of the terminology in the text is that it doesn’t account for a lot of the grey area
that happens in practice.

 A prototype and the MVP could certainly be identical, but often the MVP will have a very limited feature set
and a series of known issues or inconveniences.
 Often used by a group of pilot users that provide constant feedback about bugs and high-value features.
Agile development

 Agile development is a concept that includes many different techniques and practices that seek to support
software evolution and minimize project risk due to volatility and technical impediments.

 Some agile tools:

 SCRUM
 Kanban
 Extreme Programming
 Many, many others we will cover throughout the course
To Recap

 The crisis continues – software is still complicated and projects still run over budget in terms of both money
and time.

 Software Engineering is focused on not only technical skill but also communication and other managerial
skills that are often taken for granted.

 The ultimate goal of Software Engineering is to produce fault-free software that satisfies a client’s needs on
time and within budget.
 Even today there is an active community of researchers trying to figure out how to reliably do this.
Software Economics

 Technology is ever changing, which brings with it opportunities to make software safer, more reliable, and
cheaper.

 How do you know when to adopt a new methodology or framework?

 Long-term vs short term pay offs

 Training is not cheap/quick
 Code may have to be re-written
 Proficiency is not given or instant
“Real life” example

 Code generation can save a lot of time, effort, and money when you have a really small team of engineers
and really repetitive functions to write.

 Code generation is not as straight forward as writing code:

 Code has to be generic
 Engineers have to understand the generation strategy
 Engineers have to understand all of the tools used for generation
Life Cycles and Life Cycle Models

 Life cycle models are the theoretical phases or steps that should be followed when building software
 There are many models and the number of phases will vary

 A life cycle is the sequence of actual steps performed to product a specific product

 There can be discrepancies between the model and practice

Software Maintenance and the life cycle

 Historically, the life cycle model was development-then-maintenance

 Recall the slides from last week about waterfall.

 This definition of maintenance is predicated on when development happens (i.e., what phase):
 Faults found the day before release are resolved by classical development.
 Faults found the day after release are resolved by classical maintenance.

 Client changes follow the same pattern. Any change requested after release is maintenance.
Classical Maintenance

 Isn’t really feasible anymore.

 Client requirements and supporting systems are constantly in flux
 Software projects typically span years

 The modern definition of Software Maintenance is defined operationally rather than temporally.
 The process that occurs when a software artifact is modified
because of a problem or because of a need for improvement or
adaptation
The Importance of Maintenance

 There is a misconception that only “bad” software requires maintenance.

 Truly bad software never gets used long enough for meaningful maintenance.

 Good software is long-lived and can undergo decades of maintenance.

Cost of Maintenance

 Maintenance makes up ~90% of the cost of software now.

 Large system vendors write annual maintenance into terms of use and licensing agreements.
 Recurring revenue to pay for additional features and resolve bugs at client expense.
 Allow for smooth revenue realization that supports projection-based staffing techniques.
 Sometimes maintenance agreements are so important that actual licensing fees are entirely waived to get an agreement
finalized.
Cost of maintenance

 The earlier a fault is found, the easier/cheaper it is to resolve.

 Think about the difference between changing a piece of documentation vs changing functionality to account
for the fault.
 Change the code
 Test the code
 Test everything around the code
 Release an update
“Real life” example

 Storing program state in json vs storing program state in a relational database.

 Cost of changing the design on day 1 would have been infinitesimal.

 Cost of changing during year 2 was ~$100k + opportunity cost