Senior Software Engineer (Data Pipeline) - Archipelo
Senior Software Engineer (Data Pipeline) - Archipelo
Oct 1, 2024
About Us
Archipelo is a software development lifecycle (SDLC) data insights platform—that increases software
security, compliance, and developer productivity. Archipelo provides a historical record of all coding events
to proactively identify security and compliance risks—at the earliest stages of the software development
lifecycle. This approach ensures secure coding practices, identifies potential threats, maintains compliance,
mitigates AI-related risks, and enhances developer productivity, enabling the delivery of safe and reliable
software. The company is funded by top investors in Silicon Valley, including the first investors of Google,
Twitter, Zoom, LinkedIn, and Uber. Our team has backgrounds from NASA, LinkedIn, Facebook, Amazon,
AWS, Cisco, MIT, Harvard, Stanford, and Berkeley. Our company is based in San Francisco, California, but
our team is all remote and globally distributed.
Description
We seek a Senior Software Engineer to lead technology development on the frontier of security, code
context, and software delivery. A successful applicant is one capable of building software using a variety of
technologies. You will help our team design, test, and rapidly iterate on robust systems for event storage
and processing. You will help create Archipelo products that empower organizations and developers with
the best tools to enhance their software security and developer productivity—including APIs, browser
extensions, IDE integrations, bots, and webhooks across services.
Stack
● Our backend systems are built with Go and Kotlin/JVM
● Our frontend is built using TypeScript & Svelte
● Our cloud automation is built using Terraform
● Our data is primarily stored in PostgreSQL
● Our search engine is powered by OpenSearch
● Our services run on Google’s Cloud Platform
Responsibilities
● Design and develop systems for event retention, processing, and querying
● Implement event retention best practices
● Experiment in ways that accelerate prototyping and maximize resource utilization
● Ensure event processing pipelines work quickly, reliably and predictably
● Manage our data pipeline, including scheduling, dataflow programming, SQL, and data labeling
● Orchestrate the operation of clusters of commodity machines
● Review code, mentor other engineers, and support other teams that rely on the event store
● Attract, recruit, and retain top engineering and scientific talent
Minimum Qualifications
● Familiarity with backend development in Java, Scala or Kotlin
● Good SQL knowledge and understanding
● Experience with cloud computing – AWS, Azure or GCP
● Basic knowledge of IO and memory profiling on Linux/UNIX
● Familiarity with service delivery in a Linux environment
● Minimum 8+ years of professional software engineering experience
● Proven expertise and leadership as a world-class senior software engineer
Preferred Qualifications
● PhD or Master’s degree in computer science/engineering, mathematics, physics, or related field
● 10+ years of professional software engineering and data engineering experience
● Proficient with distributed systems and the coordination of high-volume independent commodity
machines into complete, functional systems to handle diverse workloads
● Expertise with SQL and Postgres
● Expertise in event pipeline development in Java, Kotlin, Scala or Python
● Experience with machine learning and NLP
● Experience with immutable data
● Expertise with machine learning frameworks (like Keras or PyTorch)
● Advanced understanding of optimization and profiling on Linux
● Advanced working knowledge of information retrieval and search technologies
● Expertise with configuration and use of open-source search systems to query and understand data
● Experience with the following technologies:
● ElasticSearch, Solr and Lucene
● Machine learning infrastructure
● Kubernetes, Docker, Terraform
● Deep learning, GNNs
● CircleCI, GitHub Actions, Jenkins
What We Offer
You have the opportunity to join an early-stage startup and have significant ownership of technology
development. You will work at the highest level and collaborate with world-class colleagues, advisors and
technical experts. Our team is growing rapidly and we hope you'll grow with us, too.
2
● Competitive salary & equity packages
● Unlimited vacation and sick leave
● Strong remote work culture and esprit de corps
Diversity Commitment: We are focused on building a diverse and inclusive team. We welcome people of all
backgrounds, experiences, abilities, and perspectives and are an equal opportunity employer. We do not
discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital
status, veteran status, or disability status. Learn more at archipelo.com.