DevOps

HTTPS://WWW.YOUTUBE.COM/WATCH?V=_I94 -
TJLOVG
 DevOps is an Integrated Approach to Software
Development
• Why DevOps?

Ops

Dev
 The Problem is in the Silos

Figure 1: The Silo Based Approach to Agile Software Development

• The 4 main silos:

• Business
• Development
• QA
• Operations
 Business, Development, QA & Operations
• As part of the transition to agile methodology, organisations have made the
effort to bridge the divide between business and software development by
establishing roles such as the Product Owner (PO) and the Business Analyst
(BA).
• Although the “silo mentality” still exists in many organisations, the PO and BA
are pivotal in ensuring that the business imperative of the software
development process is upheld.
• The PO’s and BA’s enable collaboration between the “business silo” and the
development and quality assurance (QA) silos.
• However, a glaring omission from this collaborative imperative advocated in
the Agile Manifesto is the lack of clarity on the role of the operations staff, the
people who are responsible for the technical alignment of the newly
developed application into the “organisational ecosystem” or the
organisational infrastructure.
• This divide between the various stakeholders in the software development
process is illustrated in Figure 1.
 The Silo Mentality…not that bad☺
• Business is logically separate from development although the PO’s and
BA’s manage to ensure that the Wall 1 divide is circumvented by
conveying business requirements to the development team in the form
of the system specifications document that contributes towards the
Product Backlog.
• There is also a close working relationship between the development
teams and QA.
• In many instances the Scrum development team members are
responsible for running unit tests thereby conflating the roles of
developer and tester.
• The formal reviews and inspections are conducted by PO’s, BA’s,
developers and testers who collectively perform the QA function.
 The Elephant in the Room!
• No one want to talk about Operations/Infrastructure!!!
 The Glaring Ommision is the Elephant in the Room
• The glaring omission is the lack of focus on the operations phase where the
system is deployed onto a live production environment.
• Once a new system is commissioned by the business division, the functional
requirements are conveyed to the development teams by the PO’s and BA’s.
• The development teams develop the system iteratively and add functionality in
an incremental manner using Scrum oriented methods.
• As each increment is endorsed by QA, the expression “done” is used to
indicate the completion of a task.
• Once all increments have been completed and the required functionality is
achieved to the satisfaction of QA, the newly developed application is “tossed
over the wall” to the operations people who are responsible for the build
engineering phase of the application’s development lifecycle.
• The build engineering phase is traditionally regarded as a separate “silo” in the
process of getting the application into a “live”/ production environment.
 The Operations Phase
• Empirical evidence suggests that there is not much support for the operations phase
in the SDLC and once the application traverses Wall 2 (illustrated in Figure 1), a
“bottleneck” situation is created because of the lack of resources available to the
operations staff.
• However, the bigger impediment to productivity is the lack of collaboration between
the development team and the operations team when it comes to tackling system
integration problems. … this is were DevOps has a major focus!!
• Once a system is handed over to the operations staff, the Scrum team is disbanded
and allocated to other projects.
• The operations staff are then saddled with the task of fixing operational errors
typically linked to the network and security infrastructure of the organisation.
• The operational error fixes are compounded by the sparse documentation that is
generated as well as the non-availability of members of the development team.
• In order to address the impasse between development and operations, an incursion
into realms of software build engineering is necessitated.
 The Need for a Collaborative Environment
• According to Aiello (2017) software development has to have a complete lifecycle
approach and it is misleading to speak about a software development methodology
that functions in isolation of the context in which the software is developed.
• The context from an organisational perspective includes BA’s, QA, project managers,
security managers, developers, testers, operations engineers and end users.
• In order to achieve this all-inclusive environment, a possible strategy is to pack each
sprint with functional requirements as well as a continuous integration (CI) and
continuous delivery (CD) imperative.
• This viewpoint is supported by Black (2017) who is of the opinion that currently,
software development occurs in an environment where there are many “moving
parts” and in order to account for the various influences on the development
process, an agile scrum based approach is required together with a DevOps culture to
ensure that the business value planned for a software system is achieved and
delivered in a short space of time.
• The main benefit of adopting a DevOps approach is that it will enable the breaking
down of the traditional silos that impedes software development productivity and
enables a lifecycle approach that extends from inception to release of the system on
a production server.
 Unpacking “Operations”
• The activity of Operations is all about Build Engineering (BE)
• According to Aiello and Sachs (2016, p. 91) BE is the discipline of
efficiently converting source code into binary executables that is in a
state of readiness for deployment to the underlying technology
infrastructure.
• This is normally achieved by running scripts that are created using
technologies such as Ant, Maven or Make so that the process is
repeatable and quick.
• There is currently a trend for the development team to run these scripts
that are then deployed to a test environment, and not the actual
production/live environment. The Build Engineer performs the
operations task of deploying the application to the production
environment.
 The Complexities of BE
• However, as Aiello and Sachs (p. 93) point out, this process can become quite
complicated and the Build Engineer is required to make an intervention on the
development side to rectify incompatibilities or bugs that were not identified
during QA.
• Quite often this entails an incursion into development technologies such as the
.Net, Java or Cobol platforms.
• The complexity of the task is exacerbated by poorly written source code or
when the build activity is undertaken as a “big bang” deployment rather than
an incremental one (Aiello, 2017).
• Aiello and Sachs (2016, p. 97) make the salient point that the Build Engineers
need to engage with the developers early in the development lifecycle so that
there is convenient availability of deep knowledge of the system from a
developmental and technical perspective.
• Organisations that adopt a DevOps strategy tend to curate an environment
that enables easy collaboration between the various software development
stakeholders.
Understanding the Development-Deployment Pipeline – Fig 2
 CI
• The first phase is the actual development undertaken by the computer programmers.
• The main deliverable from this phase is the contribution of incremental functionality towards the development of
a system.
• The evolving system is referred to as the baseline system and developers are required to contribute towards the
incremental enhancement of the baseline system by adding functionality that is the output from sprint cycles.
• According to Frankel (2017) developers are encouraged to “commit” code towards the baseline as often as
possible. This activity of committing code towards the baseline is referred to as continuous integration (CI).
• The management and co-ordination of the “commits” towards the baseline system is done via automation servers
and tools.
• The tools that are popularly used currently for this process in South Africa are GitHub (an ex open source
software/current Microsoft version control tool); GitHub is the most popular version control system out there,
with some 57 million developers on board. It’s also loved by the open-source community – as public repositories
are free.
• and Bitbucket that has leanings towards the open source community…and unlimited private repositories for free
(for up to five users)
• A significant observation with regards to software development teams in South Africa is that the practice of “code
commits” to the baseline system has only been recently identified as an area of urgency. Previously the code
commits usually took place at the end of each day. The current imperative is to commit code as often as possible
during the course of each day.
 The Problem with CI
• Aiello and Sachs (2016, p. 133) warn against too many builds in a short space of time
because it creates a destabilising environment for the developers.
• The code baseline is constantly monitored by the CI server for any changes to the
baseline code.
• If a change is detected, the code merge tests are immediately executed and feedback
is provided to the developers.
• This practice ensures that “code merge” defects are detected early which can then
be handled on a continual basis rather than a “big bang approach” where all defects
are handled at the end of the sprint, at which point the system has acquired “…a level
of complexity that is not easily tamed” (Aiello, 2017).
• The testing environment that detects build related issues linked to merge conflicts or
possibly conflicts with the different environments in which the application will be
installed is referred to as the CI server e.g. the java based open source server named
Jenkins.
• All errors identified by the CI server are sent via an error report to the members of
the development team.
 Continuous Delivery
• The 2nd phase entails the delivery of the systems that have undergone a
successful build and has been released for QA testing as well as staging
tests.
• This is referred to as the continuous delivery phase were the code
baseline is in a state of readiness to be deployed to the production server
at any time.
 Continuous Deployment
• The final phase entails deployment of the system onto the organisation’s production
servers.
• This phase is referred to as the Deployment phase and is usually not a continuous
process.
• The immediate release of newly developed systems via automated processes may be
impractical for pragmatic reasons.
• The main reason is the imperative to release new features to the end user base in a
controlled manner thereby ensuring that end users are not overwhelmed with
changes to the system in short time intervals.
• Another reason could be business related and business managers decide on the most
opportune moment to release new features based on business competitiveness
factors.
• The Practice in SA --the deployment cycle used by the 4 main South African banking
institutions is to deploy feature changes to their banking systems a maximum of
twice every month, on a Sunday evening.
From Dev to Ops in a Scrum Way – Fig 3
 The Problem with “Done”
• The first principle based on the Agile Manifesto is an attachment of the
highest priority level to the imperative to “...satisfy the customer through
early and continuous delivery of valuable software” (Fowler & Highsmith,
2001).
• However, the focus on continuous delivery seems to have a blurred
interpretation within the confines of software development
nomenclature.
• This assertion is based on the empirical evidence which suggests that the
agile-based term “done” alludes to the activity of development and
internal testing within the software development environment…not the
production environment!!!
 The Problem with a Fragmented View of Software
• The activity of development is focused more on the software system and the
pre-defined requirements tests.
• The system is not viewed as a product or a “business commodity” that has
relevance to a set of end users as well as a strong organisational context where
it contributes to the attainment of business value.
• The “disconnect” between software development and the attainment of value
from a software product has been addressed by invoking techniques such as
Continuous Integration (CI) as illustrated in figures 2 and 3.
• The practice of CI is endorsed as a core activity of agile software development
and is documented by the Agile Alliance as an initiative to ensure that
software that is tagged as “done” should be available for immediate release
into a production environment.
• However, the complexities of implementing CI and achieving a state whereby
software produced from the agile development phase may be tagged as
“immediately releasable” are not easily attained as explained in an article by
Martin Fowler (see Fowler, 2006),
 The Main Problem
• The problem of the lack of continuity between development and deployment
has been the subject of current interventions that entail the use of tool-based
sport for CI.
• The current availability of sophisticated tools from the open source community
(such as Bitbucket and GitHub from Microsoft) and vendor based tools (such as
Team Foundation Server) has made the strategy of CI a feasible option because
of the minimal overhead that is incurred to ensure that developers engage in
the practice of updating the code base on a regular basis.
• The problem is two-fold.
• CI is restricted to the development environment which is not an accurate reflection of the
production/live environment. CI ensures that the functional requirements are met and
then the system is handed over for delivery and deployment to an environment that is not
identical to the development environment.
• CI is not a formal part of the Scrum “ceremony” and it needs to be incorporated as a
Scrum method. This will ensure alignment with the agile principles that espouse CI and
Continuous Delivery.
 Integrating Scrum with Operations = DevOps
• Scrum methodology does not make direct reference to any form of CI as
an integral part of the methodology.
• The pivotal role played by CI cannot be ignored by the methodological
component of Scrum software development practice.
• The incorporation of CI practice into the Scrum ceremony may be seen as
part of the evolutionary trajectory of agile software development
methodology where software process improvement techniques have to
incorporate aspects from the delivery and deployment phase. Aligned to
the afore-mentioned imperative to adjust software process models so
that CI is incorporated as part of the development process, the de facto
model representing the Scrum-oriented software process has been
modified to incorporate activities that enhance a DevOps approach to
software development.
 A Comment from Bob!
• In order to mitigate this problem Aiello (2017) makes the suggestion that “…as
a Build Engineer, I would like to sit with the developers so that I can learn how
the system works and I am plugged into the flow of the development effort”.
• The point being made is that Build Engineering planning should be
incorporated early and continually in the development lifecycle.
• From an agile perspective, this could occur at the Sprint planning phase where
the BE (serving in the capacity as a representative from the Operations team)
is provided with details of the development platform so that the testing and
production environment could be configured to enable compatibility
between development and operations. A proposed solution that
integrates BE into the Scrum into the Scrum development cycle is
illustrated using a cross-functional flowchart in Figure 4
A Scrum Development Operations Model (SDOM)
 A Discourse on the Scrum DevOps Model
• The Sprint Planning meeting incorporates the Build Engineer (BE) who uses this
opportunity to engage with the development team and establish familiarity with the
functional specifications of the Sprint cycle.
• The BE is also responsible for configuring the development machines, the integration
server, the test server as well as a staging server were the product increment may be
tested in an environment that is as close to the production environment as possible.
• According to Aiello (2017) the BE may opt for adoption of a strategy referred to as
containerisation where the production environment is simulated on development
and test machines inside “containers” that provide a development space that is
independent of the underlying operating system.
• In this way the development team is able to develop in a truly production-like
environment thereby mitigating the complexities of incompatibilities between the
development, testing and deployment environments.
• Aiello (2017) made mention of the current technological stack that enables this
strategy and suggested that the optimal tools to enable continuous integration using
the strategy of containerisation are the open source tool named Docker.
 A Revamped Standup
• The Scrum stand up meetings should include a discussion on the status of the baseline code for the
evolving system.
• The development team should report on the frequency of their “code commits” to the baseline
system which resides on the integration/version control server. Ideally, there should at least be a daily
build that is “triggered” by the “code commits”.
• However, Aiello and Sachs (2016, p. 125) caution about the undue complexity that may be added to
the development overhead when there are too many builds to contend with.
• The reason for this concern is that the entire development team has to update changes to their local
development workspace so that they always have a current version of the baseline system.
• If there are too many commits and builds that occur then the intention to always keep a current
version of the system on the local development machine can become quite a distraction from the
actual task of software development.
• The benefits of engaging in a practice of continuous integration (CI) is that the baseline system is
always kept in a stable state and there is minimal effort to integrate new code.
• The dilemma is resolved by getting team members to make a commitment to the CI practice with the
undertaking that during the initial stages of development, code commits to the baseline system may
be infrequent. However, as the system approaches the final stages of completion, there should be
frequent code commits thereby ensuring that the final integration and testing phases do not have to
deal with the complexity that may be introduced by lack of adherence to a CI strategy.
 QA
• The testing suite should include unit tests that consist of cyclomatic tests so that there is optimal testing of
branch and looping logic.
• The test server that invokes pre-defined functionality tests (test driven development (TDD)) should be customised
to include different stages of development.
• During the initial development stages, the pre-defined TDD strategy will not be feasible because of incomplete
coding.
• However, as McConnell (1996) points out, the use of a strategy that entails smoke testing and stubs/place holders
for incomplete functionality will ensure that the baseline code is always in a stable state.
• The smoke tests are not as complex as TDD tests, but they ensure that the evolving system demonstrates basic
functionality and is always in a state where it runs and produces some form of output. The invocation of a smoke
testing and TDD strategy enables the developers to do an internal verification that a user story or a task has been
completed.
• This is crucial from a workflow perspective. If the team is using a Kanban Board to track the development
progress, then the internal verification that the task has been successfully completed will enable the task to be
labelled as “done” or “verified” from a workflow perspective, thereby freeing up the number of tasks that fall
into the verification swim lane of the Kanban Board.
• The mechanism of the Kanban Board is that there is a pre-defined limit to the number of tasks that can be placed
in a specific swim lane. However, the Scrum Board technique does not have such a restriction. Based on the
evidence of the empirical data, the strategy of restricting the number of items in any specific swim lane is
regarded as a good strategy because it provides a quick indication of a “bottleneck” situation that needs to be
resolved before any further development in the sprint can be undertaken.
 BE
• The role of the Build Engineers is to provide a testing environment that is
relevant and as close to the production environment as possible. Aiello and
Sachs (2016, p. 129) make reference to a “pre-flight” build where the
development and integration platforms are similar to the production
environment.
• This strategy may be viewed as a “right shift” where the BE provides resources
for the developers early in the development lifecycle so that the build quality
of the evolving system can be verified before it is handed over to the
operations team.
• Aiello (2017) makes the point that setting up of the test environment is a
complicated process because the test server has to be set up so that it
provides the runtime dependencies that developers were using in their local
development machines.
• Hence the involvement of the Build Engineer as an additional role player is
crucial because it ensures that there is a degree of compatibility between the
development, test and production environments.
 Product Increment – not release!!
• The Product Increment is the immediate output of a Sprint cycle. The phase
traditionally referred to as “done” is shifted to the right of the Scrum process
because once a product increment has been completed, the quality of the
increment has to be verified and validated prior to the allocation of a “done”
status.
• This verification and validation process first occurs internally between the PO,
the Scrum development team and the Build Engineer during a Product
Increment Review session. The presence of the BE is required to sort out
issues that deal with the testing environment.
• The documentation requirements for the product increment is also deliberated
upon during this phase of the Scrum development cycle. Plans for the
maintenance of the system should be incorporated into deliberations at this
stage and the documentation requirements to support maintenance activity
should be identified and created by the development team.
 Deployment Package
• The BE is then responsible for developing a deployment package that is
tested in a production-like environment. Once more a containerised
version of the deployment package should be made available for Quality
Assurance (QA) and user acceptance testing (UAT).
 Sprint Review = DevOps +QA
• The Sprint Review phase has undergone a “right shift” due to the added layer of quality
checks (product increment review) and build engineering activity to ensure that the system is
in a deployable state.
• The Sprint Review phase is essentially a showcase of the system’s functionality, usability and
performance.
• This phase also presents an ideal opportunity for the end user to interact with the evolving
system thereby providing the development team and the Product owner with an opportunity
to obtain feedback regarding the system’s functionality and usability.
• This phase is a vital inclusion in the Scrum development cycle because one of the problems
identified from the study’s empirical evidence is that the end users do not have ample
opportunity to interact with the system and provide feedback that the developers could use
to improve the usability of the system.
• This phase serves the purpose of re-establishing the close working relationship between the
development team and representatives from the end user group. This phase is also an ideal
opportunity to engage the BE on issues related to the system’s performance because
performance testing can be done in a production-like environment.
 Now we are “Done”
• The final phase of the Scrum sprint cycle is the assessment from the
stakeholders that the product increment is done and can be made
available for the build package that may be subjected to integration tests
in a staging environment.
• Once more, a containerised version of the “done” portion of the system
is maintained and used as a baseline system onto which new elements of
functionality are added during the remaining sprint cycles until the full
system has been developed.
• The containerised “done” version of the system is also in an immediately
releasable state.
 Assumptions Made by the Model
• Scrum based software development practice is widespread within South African
commercial organisations. Aligned to this knowledge, the proposed model is a Scrum-
based model for software development.
• There is an imperative for the software development process to demonstrate and
deliver business value. This is achieved in the current model by proposing a strategy
of continuous integration so that a stable baseline version of the system is
maintained in a stable state that may be “showcased” at the end of each sprint cycle.
• The role players at the showcase event consists of the stakeholder(s) from business,
the development team, quality assurance, the end user domain and the BE. The
showcasing exercise is an opportunity for the business representatives to obtain early
affirmation of the business value and make decisions regarding the early release of
functionality thereby enhancing the prospect of business competitive advantage. The
“luxury” of having the system in a deployable state has been achieved by adopting a
DevOps strategy consisting of continuous integration, continuous delivery and
containerisation.
• The technical details regarding the adoption the afore-mentioned approaches has
been obtained after consultation with experts in the domain of operations and build
engineering.
 Assumptions
• There is a need to adopt a DevOps approach in order to reduce the
“disconnect” between development and operations. This call for the
DevOps approach has been interpreted as a symptom of the need to
create a better collaborative environment for software development.
• The model proposed in Figure 4 achieves this imperative by providing a
visual guideline that outlines the roles played by the constellation of
stakeholders at various stages of development. This collaborative
environment has a strong resonance with organisational culture that is
classified as Developmental
 Virtualisation & Containerisation
• Because of differences between development and production machines, to achieve
consistency, developers use virtualization technologies to recreate production environments
on development machines.
• It involves encapsulating or packaging up software code and all its dependencies so that it
can run uniformly and consistently on any infrastructure. The technology is quickly maturing,
resulting in measurable benefits for developers and operations teams as well as overall
software infrastructure.
• With traditional methods, code is developed in a specific computing environment which,
when transferred to a new location, often results in bugs and errors.
• For example, when a developer transfers code from a desktop computer to a virtual machine (VM) or
from a Linux to a Windows operating system.
• Containerization eliminates this problem by bundling the application code together with the related
configuration files, libraries, and dependencies required for it to run.
• This single package of software or “container” is abstracted away from the host operating system, and
hence, it stands alone and becomes portable—able to run across any platform or cloud, free of issues.
• Put simply, containerization allows applications to be “written once and run anywhere.” This
portability is important in terms of the development process and vendor compatibility.
 Docking the container
• Containers encapsulate an application as a single executable package of
software that bundles application code together with all of the related
configuration files, libraries, and dependencies required for it to run.
Containerized applications are “isolated” in that they do not bundle in a
copy of the operating system.
• Instead, an open source runtime engine (such as the Docker runtime
engine) is installed on the host’s operating system and becomes the
conduit for containers to share an operating system with other
containers on the same computing system.
OC – THE Competing Values Framework

Figure xxx: Level of Agility Classified according to Organisational Culture