Open Positions at Trinity

Linux Senior Software Engineer

PRIMARY ROLE:

Trinity Cyber is looking for a senior software engineer to join their engineering team to help build out a highly customized x86 Linux based network security system. Ideal candidates will be able to work remotely with the network analysis and operations team to generate feature requests, design a conceptual software architecture, and implement that architecture.

WHAT YOU WILL DO:

Trinity Cyber's solution is a system comprised of many components including the core application and surrounding supporting tools. You should be comfortable being the primary developer for one or more major subsystems within the core application as well as its interaction with the other tools. You will be expected to be able to take a concept from whiteboard through design to implementation including unit and integration testing, code review, deployment to development and test systems, and integration with supporting tools. The software you develop will be "mission critical" and running 24x7 with SLAs in place depending on it operating correctly.

QUALIFICATIONS AND EXPERIENCE:

An ideal candidate should have general knowledge of software development under Linux (CentOS 7) to include:

    • Basic systems administration: package management, shell scripting, SELinux basics, common service configuration (sshd, httpd, named, dhcpd, etc.), iptables
    • Knowledge of compiler, debugger, and testing tool chains - gcc/clang/icc, gdb, CMake, autotools, lcov, gcov, Google Test, etc.
    • Continuous integration/continuous deployment systems - Jenkins, Bamboo, etc.
    • Static and dynamic code quality/code analysis tools - CodeSonar, cppcheck, sanitizers
    • Skill with at least one command line text editor (vim, emacs, etc.)

 An ideal candidate would also have some combination of the following knowledge and experience: 

  1. Experience writing high throughput and/or low latency user space applications in C++ (11/14 or newer preferred)
    • Linux networking stack and/or custom user space packet processing (DPDK or similar) experience highly desired
      • Related development experience such as high frequency trading, game engine development, or high performance compute are also applicable
    • Performance tuning on Linux - full system (vTune, perf, gprof, etc.) and micro-benchmarking approaches
    • Throughput/Latency tradeoffs
    • Familiarity with basic function of major kernel subsystems (network stack, memory management, process/thread management, etc.); especially interaction with user space applications
    • Ability to separate 'fast path'/'data plane' from 'slow path'/'control plane' functionality
    • Experience with "lockless" data structures; including CPU architecture level interactions
  1. Experience working on larger systems
    •  2S and 4S Intel architectures including NUMA constraints
    • Applications with large memory footprint and impacts on cache efficiency for high throughput/low latency code
    • Hardware/software and user space/OS kernel interactions - BIOS tuning, cstates, process groups and affinity, PCI CPU affinity
    • Process and thread pinning 
  1. Understanding of network protocols:
    •  Basics - Ethernet/IPv4/IPv6/TCP/UDP
    • HTTP (1.X)
    • WebSockets and 2.X/SPDY experience welcome
    • DNS
    • SMTP
    • SSL/TLS
    • Routing protocols
    • BGP/RIP/OSPF/etc.
    • Telnet
    • FTP 
  1. Familiarity with issues surrounding application layer proxying of network protocols
    • Transparent intercepting proxy vs. explicit proxy
    • L2 vs L3 vs L7 transparency and performance/reliability impacts of proxies on various protocols
    • Experience with Linux "tproxy" or equivalent features (either standard stack or custom stack) 
  1. Experience with Tier 1 ISP grade high performance multi-node x86 network appliance development desirable
    • 10Gbps+ throughput application layer proxies
    • 100,000s+ connections/s and concurrent connections
    • Liveness detection, node failover, load balancing, and state distribution in multi-rack level systems
    • Multi-tenant environments

Bachelor’s Degree in Information Technology or related field and approximately ten years of industry experience is preferred.

In lieu of a degree, a combination of education and related years of work experience that provides the necessary skills and knowledge to perform the essential job functions would be acceptable.

All candidates must possess the highest level of personal integrity, value team success over individual achievement, have the ability to contribute significantly to extending a culture of collaboration, both internally and externally, in order to maintain the superior reputation of Trinity Cyber - and enjoy having fun.