Final Placement Report

JAGANNATH H
H00453021
Abstract
This report presents an overview of my placement experience as a DevOps
Engineer at Touchlab Limited. It outlines the key projects undertaken during the
period, focusing on the development, automation, and deployment of workflows,
testing strategies, and hardware integration. The primary objectives included
setting up CI/CD pipelines, ensuring code quality through branch protection
rules, and enabling automated testing with hardware-in-the-loop methods. The
impacts of these contributions included improved efficiency, enhanced code
reliability, and structured development processes.

Project Overview
During my placement, I worked on several projects aimed at improving the
development and deployment processes within Touchlab Limited. The primary
focus areas included:
1. Establishing efficient GitHub workflows and maintaining repositories.
2. Developing and integrating automated testing mechanisms.
3. Implementing hardware-in-the-loop (HIL) testing to ensure software-
hardware integration.
4. Streamlining the management of repositories and branch rules to support
team collaboration and maintain code quality.

By leveraging tools such as GitHub Actions, Docker, and ROS frameworks, I

contributed to building an infrastructure that contributed to both development
and operational needs. This report documents the detailed activities, their
impacts, and the lessons learned.

Main Activities
1. GitHub CI Workflow and Cleanup

1.1 Create and Refine GitHub Workflows

To enhance the development lifecycle, I designed and implemented GitHub

workflows for the repositories. Key tasks included:

 Repository Creation: Established new repositories to industry-standard

branch naming conventions and default configurations to maintain
consistency and ease of navigation.
  Branch Rules:
o Applied branch protection rules to guard the integrity of the main
branches by mandating pull requests and ensuring passing status
checks prior to merge.

o Standardized branch naming conventions to clearly reflect their

purpose, such as feature/feature-name for active development and
deprecated/branch-name for inactive or outdated branches.

 Commit Guidelines: Introduced structured commit message templates to

promote clarity and descriptiveness in commit logs. This practice also
encouraged increased, frequent commits to simplify tracking and
debugging.

 Pull Request Protocols: Developed standardized pull request (PR)

templates that included sections for detailed descriptions, issue references,
and testing instructions. These templates ensured thorough reviews and
streamlined the process for contributors and reviewers alike.

1.2 Repository Cleanup and Well-structured Documentation

This initiative was designed to enhance the maintainability and user-friendliness

of GitHub repositories. By streamlining the repository structure and providing
clear documentation, these steps made better collaboration and usability.

Branch Cleanup
 Objective: Remove redundancy from the repository.

 Approach: A thorough review was conducted to identify branches that were

either merged into the main branch or no longer in active use. These
branches were archived to prevent confusion.

 Outcome: Active branches became more visible, making navigation

straightforward for contributors and maintaining a cleaner repository
structure.

README Updates

 Approach: README files were rewritten to include:

o Clear instructions for setting up the repository locally.
o Detailed guidelines on repository usage.
o An overview of the project's goals and key features.
  Outcome: The README files serve as a guide, reducing time for new
contributors while also improving the understanding of each repository's
purpose for casual browsers.

1.3 Enforcement and Communication of Branch Rules

This section focuses on making a disciplined workflow and ensuring that best
practices are consistently followed.

Branch Protection Rules

 Objective: Prevent accidental errors and unauthorized changes to critical

branches.

 Approach:
o Implemented mandatory status checks for branches, ensuring that all
tests and checks pass before changes can be merged.
o Required pull request reviews before merging, maintaining a layer of
quality workflow.

 Outcome: These measures ensured a secure and quality-driven

development process, protecting critical code from unverified changes.

Team Communication

 Objective: Ensure that all team members are aware of and follow the
established rules.

 Approach:
o Detailed the rules in internal documentation using tools like wikis.
o Organized regular team discussions to clarify rules, gather feedback.
o Provided examples and practical demonstrations for better
understanding.

2. Code Testing and Automation

2.1 Automated Tests for touchlab_sensor_driver

To ensure reliable development processes, an automated CI pipeline was

established for the touchlab_sensor_driver package, focusing on build automation,
testing, and code quality. The key components of this effort included:
  Pipeline Triggers:
o The CI pipeline was configured to execute under multiple scenarios:
 Code Pushes: Automatically triggered upon new commits to
the repository.
 Pull Requests: Ensured tests ran as part of the review process,
enabling early detection of potential issues.
 Manual Execution: Provided flexibility for developers to run
tests on-demand.

 Testing Framework:
o Leveraged colcon tools, a powerful build system designed for ROS,
to compile and test the touchlab_sensor_driver package.
o Ensured that test execution produced logs, capturing key details to
aid in troubleshooting and debugging during failures.
o Established procedures to maintain a high degree of test coverage,
implementing the identification of regressions.

 Linting and Code Quality:

o Integrated ament_cpplint, a ROS-compatible linter, to enforce
consistent coding standards.
o Set up workflows to fail if critical linting issues were detected, helping
to maintain clean, maintainable code.
o Provided detailed feedback through linting reports, empowering
developers to resolve issues quickly and efficiently.

2.2 Automated Tests for tl_startup

This section describes the implementation of automated testing workflows for the
tl_startup repository. The purpose of these workflows was to ensure the quality
and reliability of the code through continuous integration (CI) processes.

Key Features:

1. Linting with flake8:

a. flake8 is a static analysis tool used to enforce coding standards by
identifying potential bugs, stylistic errors, and deviations from
Python's PEP 8 guidelines.
b. The linting ensures that developers adhere to consistent coding
practices, reducing errors in the codebase.
2. Formatting with black:
a. black is a code formatter that reformats Python code for improved
readability and consistency across the team.
b. This tool helps eliminate arguments about code style and allows
developers to focus on the functionality.
3. Workflow Configuration:
 a. The workflows were designed to allow flexibility during the
development process. Even if minor linting issues were detected, the
pipeline was configured to proceed.

2.3 Learning ROS GitHub Actions

This section highlights focused learning and practical application of GitHub

Actions within the Robot Operating System (ROS) ecosystem. The subsection
comprises two main aspects:

1. Workflow Setup:
a. I gained an understanding of how to configure GitHub Actions
specifically for ROS environments. This involves crafting workflows to
automate key tasks, such as building software, running tests, and
ensuring that the code integrates well with the ROS framework. The
workflows likely included triggers (like code pushes or pull requests)
and specific job definitions to carry out ROS-compatible actions.

b. The configuration process required knowledge of ROS-specific tools

like colcon for building and testing, and ensuring compatibility with
ROS package structures and dependencies.

2. Practical Implementation:
a. Beyond theoretical learning, I applied this knowledge to actual ROS
repositories, reinforcing my understanding with hands-on experience.
This step is crucial as it bridges the gap between academic
understanding and real-world application.

b. By implementing these workflows in live repositories, I ensured that

the systems i worked on were both functional and aligned with
industry standards for ROS development.
2.4 Create Project Repositories

I developed a framework for setting up and managing repositories across multiple

projects:

 Repository Structure: Designed a structure for all master repositories to

ensure consistency and clarity. Each repository included key directories,
such as src, tests, and docs, and followed a template.

 Documentation:
o Recorded and maintained repository structures in a guide for easy
reference.
o Ensured every repository included a comprehensive README file
detailing the project’s purpose, usage instructions, and contribution
guidelines.

2.5 Automated Tests for zmq_comms

To ensure reliability and maintainability in the zmq_comms package, a specialized

Continuous Integration (CI) pipeline was established with the following
components:

 Build and Test Process:

o Environment Consistency: Leveraged Docker containers to create
uniform environments across different stages of development and
testing. This minimized discrepancies caused by variations in local or
server configurations.

o Automation: Employed colcon, a build tool widely used in ROS-based

projects, to automate the building and testing of the package. The
automation ensured that builds and tests were reproducible and free
from manual intervention.

o Test Execution: Tests were designed to validate the core

functionalities of the zmq_comms package, ensuring that the
communication protocols and data handling adhered to specified
requirements.
2.6 Linter Actions

Focused on enforcing coding standards by integrating comprehensive linting

actions across several critical repositories. This initiative targeted three
repositories:

1. zmq_comms: A repository central to communication protocols.

2. touchlab_sensor_driver: A repository crucial for managing sensor

interactions.

3. tl_startup: A repository responsible for initializing system configurations

and services.

Objectives of Linter Actions

 Consistency: Standardize code formatting and style across multiple
repositories to enhance readability and maintainability.
 Error Reduction: Identify and address coding errors or inefficiencies early
in the development process.
 Scalability: Implement a framework that can be easily extended to
additional repositories in the future.

Implementation Details

1. Tool Selection:
a. Utilized tools such as ament_cpplint for C++ repositories and flake8
for Python-based repositories.
b. These tools were chosen for their compatibility with the respective
codebases and their ability to enforce robust coding standards.

2. CI Integration:
a. Incorporated linter tools into existing CI pipelines using GitHub
Actions.
b. Configured workflows to automatically run linting checks on code
pushes, pull requests, and manual triggers.

3. Documentation and Communication:

 a. Clearly documented linting rules, configuration files, and best
practices to ensure developer awareness.
b. Conducted team sessions to introduce and demonstrate the benefits of
these standards.

Benefits and Impacts

 Enhanced Code Quality: By systematically addressing linting errors, the
codebases across these repositories now adhere to industry best practices.
 Developer Efficiency: Automated linting eliminated the need for manual
code reviews focused solely on style and formatting.
 Unified Standards: A consistent coding style across repositories simplified
onboarding for new developers and improved team productivity.

3. Testing with Hardware-in-the-Loop (HIL)

3.1 Fingertip Sensor Setup

I configured a hardware environment for Hardware-in-the-Loop (HIL) testing.

This setup integrated physical fingertip sensors with software workflows to test
and validate the performance of critical systems. Key components and processes
involved:

 Fingertip Readout Board: Developed and connected fingertip readout

boards for real-time data capture from sensors.

 Arduino Masterboard: Integrated an Arduino-based masterboard to act as

the central interface for sensor communication and control. Configured the
masterboard to handle sensor signals efficiently and ensured compatibility
with the testing framework.
 Touchlab C++ Driver: Built, compiled, and tested the Touchlab C++
driver for interaction between software and hardware components.
Leveraged CI workflows and Google Test (GTest) to validate driver
functionality.

 C++ Driver Implementation: Ensured the driver was capable of

interpreting sensor data accurately, enabling effective hardware feedback
loops within the HIL framework.

 Touchlab C++ Driver Testing: Done extensive automated tests to

guarantee the reliability of fingertip sensor integration.
3.2 GitHub Runners

1. Self-Hosted Runners:
a. These are custom GitHub runners hosted on infrastructure
maintained by the organization (in this case, Touchlab Limited)
instead of relying on GitHub's cloud-hosted runners.

b. By using self-hosted runners, the goal is to:

i. Enhance Performance: Custom hardware and software
configurations allow for optimized builds and testing tailored to
specific project needs.
ii. Reduce Dependency: This approach minimizes reliance on
GitHub-hosted infrastructure, which can sometimes be limited
depending on usage.

2. Environment Isolation:
i. Security: Each build and test is performed in an isolated
environment, protecting sensitive data and ensuring that
different jobs don't interfere with each other.
ii. Consistency: Using containers ensures that builds are
reproducible across different systems, as the environment
remains consistent.
3. Advantages of This Approach:
a. The use of self-hosted runners is particularly beneficial in scenarios
requiring custom hardware or software dependencies.
b. It provides flexibility to scale infrastructure based on project demands
and enables better control over the CI/CD processes.

3.3 Documented Procedures for Authentication and Usage in CI/CD

Pipelines:

This section involves outlining steps and best practices to ensure secure access
and operation within Continuous Integration and Continuous Deployment (CI/CD)
workflows. Specifically, it addresses the following key areas:

1. Authentication Mechanisms:
a. GitHub Tokens: The usage of Personal Access Tokens (PATs) or
GitHub-provided tokens ensures that pipelines can securely access
private repositories and fetch dependencies. This reduces the risk of
unauthorized access while maintaining operational integrity.
 b. Scoped Tokens: Tokens are configured with minimal necessary
permissions, adhering to the principle of least privilege to limit
potential damage in case of a security breach.
2. Container Registry Access:
a. Procedures for authenticating CI/CD workflows with the GitHub
Container Registry are detailed. This includes steps to configure
pipeline access and pull necessary Docker images required for builds
or deployment.
3. Documentation:
a. Clear, centralized documentation on setting up and managing these
credentials ensures that team members can easily replicate or
troubleshoot the authentication process in their own workflows.

3.4 GitHub Tokens in CI/CD

In CI/CD pipelines, authentication is a crucial step when accessing resources such

as private repositories or container registries. GitHub tokens offer a secure way
to handle this authentication without exposing sensitive credentials.

1. Container Registry Access:

a. Pipelines often need to pull or push Docker images. By using GitHub
tokens with the GitHub Container Registry, permissions are managed
securely, ensuring that only authorized workflows can access these
images.
b. Tokens help authenticate without storing permanent credentials,
reducing the risk of unauthorized access.
2. Private Repository Dependencies for ROS Packages:
a. When working with ROS (Robot Operating System) projects,
dependencies might be hosted in private GitHub repositories. By
leveraging tokens, CI workflows can clone these repositories during
build or test phases.
b. This approach maintains security and ensures that external users or
unauthorized processes cannot access sensitive code.

3.5 Build and Compile Touchlab C++ Driver

The Touchlab C++ driver was a critical component in enabling communication

between the software and the integrated hardware systems. This section
highlights the process undertaken to build, compile, and test the driver to ensure
its robustness and compatibility with various hardware setups.

Build Process:
  A fully automated build process was established using Continuous
Integration (CI) workflows, ensuring consistency across all development
stages.
 Dockerized environments were leveraged to replicate uniform build
conditions across different machines. This approach minimized
discrepancies due to environmental variations, ensuring consistent results.
 The CI workflow was configured to trigger builds on multiple events,
including code commits, pull requests, and scheduled intervals. This
ensured timely validation of new changes and early detection of issues.

Integration of Google Tests (GTests):

 Google Test framework (GTest) was integrated into the build process to
validate the functionality, performance, and edge-case handling of the C++
driver.
 Unit tests were developed to cover core functionalities such as data parsing,
error handling, and communication protocols with the sensor arrays.
 Stress tests were designed to evaluate the driver’s performance under high-
frequency data input scenarios, ensuring stability during prolonged use.

Automated Validation:
 The CI pipeline not only executed builds but also ran all relevant GTests
automatically, with results being logged and reported. Any failures were
flagged, and detailed logs were generated to aid in quick debugging.
 Test artifacts, including logs and performance reports, were stored as CI
outputs for future reference and analysis.

Results / Impacts
1. Improved Efficiency: CI/CD pipelines reduced manual efforts in testing
and deployment, enabling faster development cycles.
2. Enhanced Code Quality: Branch protection rules, automated linting, and
testing ensured robust codebases.
3. Streamlined Collaboration: Consistent commit practices and detailed PR
templates facilitated better team collaboration.
4. Effective Hardware Integration: HIL testing enabled seamless
interaction between software and hardware components.

Conclusion
My placement at Touchlab Limited as a DevOps Engineer provided invaluable
learning opportunities. By implementing scalable CI/CD pipelines, automating
testing workflows, and integrating hardware-in-the-loop systems, I contributed
significantly to the organization’s development practices. This experience
enhanced my technical expertise and understanding of DevOps principles,
preparing me for future challenges in the field.