Prof.

Hamid Asgari
Hamid.Asgari@kcl.ac.uk

KCL, 13th Dec. 2024

 Lecture 10 Part A - Review

 Project Outputs

 Innovation

 IP Management

 IP Protection

 Dissemination & Exploitation

2
 Lecture 10 Part B - Overview

 Project Success/ Failures

 Project Closure

 Post-Mortem Analysis

 Lessons Learnt

 Closing Remarks and Expectations

3
 1 PROJECT SUCCESS/FAILURE

4 Picture ref.: https://www.projectsmart.co.uk/learning-from-project-failures.php

 Perceptions of Project Success

Success criteria for Project Managers and Customers differ:

 For Project Manager:
 Keep overall costs within the planned budget.
 Deliver the product to the customer at the agreed time.
 Maintain a happy and well-functioning development team [1]
 Deliver product that meets the customer’s expectations.
 Enhance the company image and position.

 For Customer/Users:
 Usability of the developed system [2]; system quality? Meeting their
needs?
 Have longer-term view.

Ref. [1] Sommerville, I. Software Engineering 2010.

[2] Wateridge, 1998
5
 Success Factors (1)

 The factors contributing to the project success:

 Select and use an appropriate Proj. Mgt. methodology: Follow tools
and techniques of the chosen methodology/framework
 Good B&P planning and project execution
 Base the project plans on sound estimates
 Divide up any large project; Clearly well defined project; Meeting the
project’s requirements / objectives that all stakeholders agree
 Customer/stakeholder involvement and consultation
 Seek organisational support providing necessary resources and authority
 Competency of Project Manager; [An important factor]
 Delegate the tasks effectively
 Proper availability and use of resources and skilled project team
 Build a cohesive and committed team with the necessary expertise [1]
 Team motivation, satisfy their professional desires, involve them in
decision-making, create a committed team.
Ref. [1] Kharbanda & Stallworthy, 1986
6
 Success Factors (2)
 Further factors contributing to the project success:

 Follow best practices; do not need to reinvent the wheel

 Make effective communications
 Control - but not prevention - of changes
 Manage risks continuously
 Provide necessary training for project team
 Avoid Scope Creep: Monitor cost, performance and project progress
 Monitor and provide feedback i.e. timely control at each step of the
execution process
 Trouble Shooting: ability to handle unexpected crises and deviations
from plan.
 Meet the agreed budget; Deliver on time; Add value
 Make stakeholders satisfied.

7
 Failure factors (1)

 The factors contributing to the project failure:

 Using the wrong form of organisation (methodology/ framework) for the
project [3].
 Poorly defined scope and objectives
 Failure to identify the key assumptions
 Failing to take the time to estimate and plan the project properly at B&P
phase.
 Lack of details in the project plans
 Poor planning, risk management and control methods
 Not involving user/customer
 Appointing a project manager who proves incapable of bringing the project to
a successful conclusion
 Key staff leaving the project and/or company
 Ineffective project manager and team
 No active support from senior management [2].
Ref: [1]: Survey of 150 IT projects in Philips NV during the 1970s, by Geddes, 1990
[2]: Factors contributing to failure: Ewusi-Mensah, 1997
[3] Reasons for project failure, Meredith & Mantel, 2005
8
 Failure Factors (2)

 Further factors contributing to the project failure:

 Inadequate / excessive planning, monitoring and control
 Poor/lack of effective communication at all levels
 Failure to track project progress
 Constant changes in requirements and failure to track the requirements
 Failure to properly involve the team in decisions
 Failing to learn from past projects [1]
 Little experience with the technology or the application domain
 Lack of proper detailed documentation
 Inaccurate time and effort estimates
 Cultural differences in collaborative projects
 Failure to supply the necessary resources for the project
 Problems in customer organisation [1] (or the company running the
project)

Ref. [1]: Causes of Failure: Barnes & Wearne, 1993

9
 Requirements Capture Problem

“There is little doubt that project requirements are the single biggest
cause of trouble on the software project front. Study after study has
found that, where there is a failure, requirements problems are usually at
the heart of the matter”.
[Robert Glass, publisher of The Software Practitioner]

10
 2 PROJECT CLOSURE

Picture ref.: https://www.prince2primer.com/prince2-project-closure-2/

11
 Project Closing

Acceptance Closure
review review

Deliverables
Project Closure & Invoicing Lesson Learnt
Manage B&P Manage Project

 All projects should come to an end.

 The goal for a good project manager is ensuring that the project is
properly closed.
 Project closing is the last phase of a project:
 When the project outputs are handed over to the stakeholders
 Contractual agreements have properly been met
 Project records are completed and stored for future reference.

12
 Ending Projects

 How does a project end?

 Completed successfully
 Not succeeded / Failed
 Cancelled
 Bankrupt
 Terminated
 Practical reasons
 Banned

13
 Reasons for ending projects

 Successful Projects
 “Successfully” achieved its goals & customer satisfied.

 Failed/Closed Project
 Technological Failure
 Too complex; unable to meet key goal.
 Management Failure
 Too late/expensive; unacceptable quality
 Outdated
 Overtaken by events (such as changes to legislation, product no longer
needed)
 Changed Project Priorities
 Resources could be used better on other work.

14
 Types of Project Termination (1)

Project termination consists of all activities consistent with closing out

the project.
 Natural Termination: the aim of the project has successfully been
completed. Successful Termination.
 Unnatural Termination: when the organisation is no longer to
invest and incur the cost required to complete the project.
 A project is terminated when work on the project is to cease or
has slowed that progress on the project is no longer possible.
 Termination by integration or by addition
 Termination by integration, is the most common way of dealing with
successful projects; The property; equipment, material, personnel and
functions of the project are distributed among the existing elements of
the parent organization. The output of the project becomes a standard
part of the operating systems of the patent, or the client.
 Termination by addition, the project property is simply transferred to an
existing or newly created organisational entity.
15
 Types of Project Termination (2)

 Termination by starvation
 Project ends for a number of reasons such as resources run out,
budget cut, political.
 Termination by extinction
 Termination by top management because the project failed (e.g.
objectives not met, superseded, not profitable).

16
 Project Closure Process/Actions (1)

 The closure process may be initiated by the Senior or Project Manager following
completion of all project technical activities.
 The following two main actions shall be completed:
 Conducting project Closure Debrief/Review for closure and Post-mortem
 Creating a Project Archive
 The PM shall open a separate support project if there are on-going
commitments.
 The PM shall review and update the Risk Register of the project to be closed.
 The PM shall make sure about the material to be transferred to the customer and
those to be retained in the Project Archive.
 Items hired or on loan shall be returned.

17
 Project Closure Process/Actions (2)
 The PM shall ensure Customer Acceptance that can include formal procedure for
accepting project deliverables
 Final review and acceptance by customer/independent evaluators (audit)
 System testing and client approval (Demo)
 The PM shall record the Customer Acceptance/Agreement to closure.
 The PM, Accounts, Department Managers and Quality Manager shall all be
informed of the closure through meeting or by email.
 Project documentation including the contract document, amendments,
agreements, invoices, cost statements, and significant correspondence and also
technical documents need to be maintained for a time period.
 The Commercial Manager (CM) shall check that there are no outstanding
commercial issues which would be likely to require further work by project staff.
 CM shall perform a financial closure.
 PM shall release the project staff and try to allocate them to other works.
 The CM shall approve closure when all requirements have been completed.

18
 Project Closure Review
Verify that all the commitments are fulfilled
and close all activities on the project.
Ensure that all contractual
deliveries toward the Customer are
made and accepted

Ensure all payments, obligations,

penalties are completed

Ensure that all internal and external

work packages are closed

Ensure that all financial elements Note : this review can be

are settled conducted in two phases:
 at the end of main
Ensure that sufficient provisions project activities
are made to cover the warranty  at the contract closure
period / or warranty obligations (end of warranty period)
fulfilled
Ensure that all relevant
documentation is archived

Ensure that project assessment is

done and lessons learnt recorded

Ensure that last Project reporting is

completed

19
 3 POST-MORTEM ANALYSIS

20 Picture ref.: https://programsuccess.wordpress.com/2013/02/01/running-effective-post-mortem-meetings/

 Post-Mortem Audit

 The goal of a post-mortem is a critical analysis of the project in

order to learn and improve, avoiding to repeat the same
mistakes.
 To determine and analyse elements of the project that were
successful or unsuccessful.
 With the participation of the project team, the Project Manager
must collect, analyse and synthesise the strengths and
weaknesses noted during the course of the project.
 Different formats and levels of formality are possible.
 Useful lessons can be learned from successful projects (what
worked, what we could have done better).
 Unsuccessful projects also provide a lot of information.

21
 Content of Project Post-Mortem Analysis
 Project Performance
 Compare Project’s Goals with Achievements
Determine the reasons for deviations from plans
 Management Issues
 Look at the project organisation
 Look at Project Management practices that were effective or created too
much overhead
 Administrative Practices
 Highlight those worked particularly well or not
 Development Issues
 Determine Pros/Cons of Process, Methods, Tools used
 Note major technical problems encountered
 Project Team Performance and Partner Performance.
 With whom you should work and not wok with
 Finally capture all Lessons Learnt and make Recommendations for future
projects.
22
 Post-Mortem Metrics

 A quantitative assessment allows a more precise evaluation of the project

 Data can be used for future projects’ estimations
 Metrics can include the following:

Type of Metric Metrics

Cost Metrics  Planned Efforts and Estimates (e.g. estimated
lines of code)
 Actual Efforts and Achievements e.g. lines of code
 History of changes to requirements, … and code
Schedule Metrics  Original Schedule
 Final Schedule
 History of schedule slippage events
Quality Metrics  Errors at each stage of development

23
 Post-Mortem Results Structure

 The outputs of a post-mortem audit should be captured in a

document.
 The document can be used to capture and disseminate the lessons
learned and used as a reference for future similar project.
 A post-mortem report can be organized as follows:
 Project overview: information about the project, to give the context
 Key accomplishments: The good & what worked well
 Key problem areas: the worst factors that team encountered
 Lessons learnt (B&P and Execution Phases).
project_post-mortem-template.docx

24
 Release of Project’s Staff

 Transition to new activities can be disruptive for the team

(consider, e.g. a project lasting for years).
 Two important aspects:
 Ensuring proper recognition of experience gained in the
project and results obtained.
 Ensuring proper tasks are assigned to the team members.
 Ensuring personnel appraisals.

25
 4 EVALUATIONS &
LESSONS LEARNT
 Based on my own experience

26 Picture Ref.: https://www.eschoolnews.com/2018/12/19/the-most-important-lessons-we-learned-this-year/

 B&P Phase: Evaluation Criteria for EC Project Proposals

1. Scientific and/or technological (S&T) excellence (topics addressed by Score:

the CFP) (Threshold 3/5;
 Soundness of concept, and quality of objectives Weight 1)
 Progress beyond the state-of-the-art
 Quality and effectiveness of the S/T methodology and associated work plan
2. Quality and efficiency of the implementation and the management Score:
 Appropriateness of the management structure and procedures (Threshold 3/5;
 Quality and relevant experience of the individual participants Weight 1)
 Quality of the consortium as a whole
 Appropriateness of the allocation and justification of the resources to be
committed (budget, staff, equipment)
3. Potential impact through the development, diss. and use of project Score:
results (Threshold 3/5;
 Contribution, at the national/EC and/or international level, to the expected Weight 1)
impacts listed in the work programme under relevant topic/activity
 Appropriateness of measures for the dissemination and/or exploitation of
project results, and management of intellectual property.
Remarks: Overall score:
Threshold for each criteria 3; Total threshold 10 (Threshold
Typically need to score 13 to have a chance of winning 10/15)

27
 B&P Phase – Evaluation results on a number of Proposal Submitted

Criteria %Proposals %Proposals

(Neutral + (Positive)
Negative)
Progress beyond State of the Art 70 30
Appropriateness of allocation & justification of 65 35
resources
Quality & effectiveness of Scientific &Technological 64 36
methodology and work plan
Appropriateness of dissemination & exploitation, & 53 47
IP Management
Contribution to the expected impacts in the 40 60
customer’s work programme
Soundness of Concept & Quality of Objectives 33 67
Quality of Consortium as a whole 33 67
Appropriateness of Management structure & 13 87
procedures
Quality & Relevant experience of individual 13 87
participants
28
 B&P Phase - Recommendations in Submitting Proposals

 Scientific/Technological Excellence
 Consider advances in technology/science, need to consider previous/current
projects results
 Provide proper references, especially in the literature review section
 Specify field trials or equivalent demonstration/evaluation method used
 Specify success criteria that is viewed positively.

 Quality & Efficiency of implementation and management

 Sufficient resources need to be allocated to project management – e.g. allocate
between 7 to 10% of total resources
 Name WP leaders that is viewed positively
 Include technical quality management (e.g. review process)
 End user participation/engagement in consortium/project is important
 Consider risks (technical, management, etc.) risks

 Potential impact through development, dissemination and use of project results

 Identify the benefits for society in general
 Advisory groups are generally viewed positively
 Evaluators like to see how the project outcomes continue after the project ends –
an exploitation plan
 Results available as open source is favoured (if using public funding)
 Lack of involvement in standardisation bodies is a problem
 Key deliverables marked as private/restricted is viewed negatively when using
29
public funding.
 B&P Phase: Lessons Learnt
 Overall the project must not appear to be expensive for the work proposed.
 The system integration must properly be described.
 The description of the work plan much include sufficient detail.
 The planned deliverables must include technical aspects as they may be
inputs to the following WPs.
 The deliverables should be linked to the milestones defined.
 Allocation of resources to WP must be justified.
 The role of participants in a WP must be explained.
 The proposal may need to provide information about the field trials.
 The costs of equipment and the costs of subcontracting must sufficiently be
justified.
 The consortium must be well balanced.
 The proposal must allocate sufficient resource budget to project
management. Any insufficient funding would jeopardise the project
coordination and hamper the work.
 Provide sufficient detail about specific costs, such as travel, hardware and
software etc.

30
Project Execution: Lessons Learnt from a successful project (1)

 Allow start up lead time for recruitment for university take-ins

 Energy & eager to learn is more important than to have the correct level of
experience for a research candidate at the beginning
 Certification (Technical) landscape was not adequately mature to support
the full objectives of this study at this time
 Study would have benefited from greater focus on experience from other
domain certification (technical) approaches
 It would be more beneficial to identify when draft documents are required to
allow adequate time for review & achieving the right level of quality
 It would have been beneficial to have an integrated / shared working
environment, covering both documents & software
 Good project management is important to effectively manage even a small
sponsored Collaborative R&D project.

31
Project Execution: Lessons Learnt from a successful project (2)

 It might be worth for sponsor considering providing independent technical

review of the project to confirm that it is in line with the UK strategic
direction & maintain a high standard of deliverables
 Project managed to adequately re-focus based on challenges with lack of
certification standardisation (moved from review of to push to support
standardisation in this area)
 Slow upfront start from university (due to funding constraints) reduced the
impact the university could have on the direction of the study
 More active review of the Project Management Plan to ensure that any
changes to the WPs are reflected in this document as the research
progresses
 Due to the high level nature of the architecture, it was difficult to provide
any precise security requirements & the methodology used was not bested
suited to this environment.

32
Project Execution: Lessons Learnt from a successful project (3)

 The phasing of the security element was potentially too early within the
project schedule to gain best benefit
 The approach of advisory board is useful but to make it more effective,
would need to be funded for peers & may benefit from a different mix
(wider cross-section to be considered)
 Consideration could be given to generating some of the project documents
as publically available to support subsequent research and knowledge in
this area
 The scope of this feasibility study was very wide and was a challenge to
know the exact elements to be explored
 Trade off between ambition & being able to deliver a meaningful
outcome
 Good level of interaction and support from the Evaluator/Customer
 As the detailed technical direction is managed by the project, this may not
result in the outcomes expected by all parties
 This has the benefit of being flexible & adaptable but may not provide high level
results.

33
 Root cause of Issues identified in some projects that caused project
overrun or even failure.

 The effort/time to capture customer needs in bid phase underestimated and risks are not
considered properly.
 Underestimated complexity of customer requirements causing lack of maturity for the solution.
 Underestimate the gap between technical knowledge/skills and in devising the technical
approach/ solution in B&P phase.
 Deficiency in aligning the objectives with stakeholders ecosystem.
 Risks identified by BM/PM was not checked and reviewed by stakeholders and management
(justifying and convincing them).
 Assumptions made but not documented.
 Unrealistic pressures on cost (over-estimated and under-estimated cost).
 Underestimated complexity to manage in bid phase vs. engineering practices in maturing the
solution.
 Deficiency in anticipation with regard to project staffing and complexity needs.
 Deficiency in collaborative behaviour and alignment of objectives at all design/engineering levels.
 Lack of engineering/integration impact and little time spent in B&P phase generating shortcuts.
 IT environment and supporting tools should support collaborative working (Webex, wiki, etc.)
 Off the shelf products not sold as they are since the company intend to achieve full compliance.

34
 Lessons Learnt - Engineering Weaknesses
Rule 4
Rule 2

Rule 5

Scattered Engineering Teams Lost and Discarded Knowledge

Rule 1 Rule 3

Over Engineering

Generation of Useless
Wishful Thinking
Information

35
 Closing Remarks - Industry Expectation (1)
 The company expect that new graduates/recruits demonstrate
on the job performance:
 Be focused – listen and understand customers’ needs; problem space
 In doing the work pay attention to the details but not get lost
 Challenge the status quo and ask questions for making improvements
 Be innovative for creating value, identify opportunities, foster emergence and
implementation of new ideas (Invention Disclosures, Patents)
 Perform through teaming, work is conducted in a multi-cultural environment,
share info., value collective achievements
 Develop proper plans for work activities to be conducted
 Initiate solutions, make recommendations and make sure to complete the
work
 You will be accountable for the quality and correctness of proposed solution
 Prepare and present solutions at different meeting
 Write structured technical reports & present technical issues in a
comprehensive manner.
36
 Closing Remarks - Industry Expectation (2)

 The company expect that new graduates/recruits demonstrate

commitment to professional development:
 Develop yourself and others; identify your strong points and development
needs
 Enhance your professional knowledge, follow and understand leading-edge
technological developments
 Develop your skills and knowledge, try to have at least one speciality in a
subject area
 To become a recognised authority in own speciality and then become a
lead contributor.

37
 Have a Successful Submission of
Your Project Plan

GOOD LUCK FOR THE EXAM IN JAN.

Hamid Asgari
38