Open Positions at Trinity

LINUX SENIOR SOFTWARE ENGINEER

Primary Role:

Trinity Cyber is looking for a senior software engineer to join their engineering team to help build out a highly customized x86 Linux-based network security system. Ideal candidates will be able to work remotely with the network analysis and operations team to generate feature requests, design conceptual software architecture, and implement that architecture.

What You Do Will Include:

Qualifications and Experience:

An ideal candidate should have general knowledge of software development under Linux (CentOS 7) to include:

• Basic systems administration: package management, shell scripting, SELinux basics, common service configuration (sshd, httpd, named, dhcpd, etc.), iptables
• Knowledge of compiler, debugger, and testing tool chains - gcc/clang/icc, gdb, CMake, autotools, lcov, gcov, Google Test, etc.
• Continuous integration/continuous deployment systems - Jenkins, Bamboo, etc.
• Static and dynamic code quality/code analysis tools - CodeSonar, cppcheck, sanitizers
• Skill with at least one command line text editor (vim, emacs, etc.)

An ideal candidate would also have some combination of the following knowledge and experience:

1. Experience writing high throughput and/or low latency user-space applications in C++ (11/14 or newer preferred)

• • Linux networking stack and/or custom userspace packet processing (DPDK or similar) experience highly desired
    • Related development experience such as high-frequency trading, game engine development, or high-performance computing are also applicable
  • Performance tuning on Linux - full system (vTune, perf, gprof, etc.) and micro-benchmarking approach
  • Throughput/Latency tradeoffs
  • Familiarity with the basic function of major kernel subsystems (network stack, memory management, process/thread management, etc.); especially interaction with userspace applications
  • Ability to separate 'fast path'/'data plane' from 'slow path'/'control plane' functionality
  • Experience with "lockless" data structures; including CPU architecture level interactions

2. Experience working on larger systems

• • 2S and 4S Intel architectures including NUMA constraints
  • Applications with large memory footprint and impacts on cache efficiency for high throughput/low latency code
  • Hardware/software and user space/OS kernel interactions - BIOS tuning, cstates, process groups, and affinity, PCI CPU affinity
  • Process and thread pinning

3. Understanding of network protocols:

• Basics - Ethernet/IPv4/IPv6/TCP/UDP
• HTTP (1.X
• WebSockets and 2.X/SPDY experience welcome
• DNS
• SMTP
• SSL/TLS
• Routing protocols
• BGP/RIP/OSPF/etc.
• Telnet
• FTP

4. Familiarity with issues surrounding application layer proxying of network protocols 

• Transparent intercepting proxy vs. explicit proxy
• >L2 vs L3 vs L7 transparency and performance/reliability impacts of proxies on various protocols
• Experience with Linux "tproxy" or equivalent features (either standard stack or custom stack)

5. Experience with Tier 1 ISP grade high-performance multi-node x86 network appliance development desirable

• 10Gbps+ throughput application layer proxies
• 100,000s+ connections/s and concurrent connections
• Liveness detection, node failover, load balancing, and state distribution in multi-rack level systems
• Multi-tenant environments

Bachelor’s Degree in Information Technology or related field and approximately ten years of industry experience is preferred.

In lieu of a degree, a combination of education and related years of work experience that provides the necessary skills and knowledge to perform the essential job functions would be acceptable.

All candidates must possess the highest level of personal integrity, value team success over individual achievement, have the ability to contribute significantly to extending a culture of collaboration, both internally and externally, in order to maintain the superior reputation of Trinity Cyber - and enjoy having fun.

SENIOR NETWORK ENGINEER

Primary Role:

Trinity Cyber is looking for a senior network engineer to join their engineering team in working remotely to support the development and operation of a man-in-the-middle network threat prevention service. This position has three major functions:

1. Provide network support for our client services to include creating new delivery methods, client onboarding, and expansion into the SASE and Tier 3 ISP spaces.
2. Support our existing internal corporate network and dev/test environments.
3. Modernizing our internal network to incorporate SDN/SDWAN (possibly positioning it as a ‘shared underlay' for both internal use and client services).

What You Will Do Includes:

As a highly skilled generalist in the networking field, you will:

• Support our existing corporate networks across our headquarters and five data centers comprised mainly of Juniper switches and Palo Alto firewalls.
• Modernize our corporate network shifting from OSPF over VLANs (physical leaf/spine) to VXLAN/EVPN with an eye towards an SDN environment.
• Convert our statically routed single ISP edges to BGP peered with provider agnostic IP space and then maintain/expand that connectivity to include additional peers and partner providers.
• Support onboarding clients to our service: evaluate their network edge architecture, develop a proposed integration with our service, and oversee the staging/testing/implementation of the design from a consulting role to the client’s existing MSP/networking team.
• Maintain documentation in a system of truth and implement all internal configuration changes via full automation from that system (whenever practicable)

Our environment includes a public-facing corporate network with full internet access, a closed development and operations network with limited internet connectivity, and a variety of “one off” networks for malware analysis and testing.

Client connectivity to our service is customized to each client’s needs but generally falls into one of a few categories: layer 2 point to point circuits delivered by a partner service provider, layer 2 over layer 3 tunnel (Ethernet over VPLS/EVPN over GRE), or layer 3 tunnel (IPSec GRE) directly. We are looking to add endpoint VPN (OpenVPN style) support in the near future. We have clients using both hairpins and direct internet access through us.

Qualifications and Experience:

An ideal candidate would possess the following combination of skill sets:

• Bachelor’s Degree in Information Technology or related field and approximately ten years of industry experience is preferred.
• In lieu of a degree, a combination of education and related years of work experience that provides the necessary skills and knowledge to perform the essential job functions would be acceptable.
• The position requires a creative, can-do, security-first attitude.
• Must be process-oriented and have the ability to think through and clearly describe in detail the steps required to achieve an outcome.
• MUST be able to program in Python to automate network management.
  • Familiarity with multi-developer environments, version control (git), etc. would be beneficial.
• Need to have real software development experience - not just "a little scripting to make the job easier".
• You must prefer Linux/BSD to Windows - CLI as the primary environment.
• We desire someone with strong skills regarding:
  • Juniper switching/routing; including experience with fully automated device configuration
    • L2: QinQ/service provider style configs, VLAN translation, quality of service on multi-destination traffic.
    • L3: OSPF, eBGP, anycast/ECMP.
    • Other: L2 over L3, VXLAN, Virtual Chassis, MC-LAG, Leaf/Spine.
  • Management: PyEZ, JTI, SNMP traps, SLAX, ZTP, event scripts, ISSU/NSSU.
    • Palo Alto Networks firewalls; including experience with highly automated device configuration.
    • Basics: L2/L3/loopback/aggregate interfaces and subinterfaces, static route monitoring, security policies, NAT rules.
    • Tunneling/Routing/Tenancy: IPsec tunnels, GRE tunnels, PBF, multiple virtual routers, multiple vsys including inter-vsys traffic flows.
    • SSL decryption broker (transparent chains).
    • Global Protect agent and Panorama experience would be a plus.
  • Physical infrastructure (data center) experience.
    • Racking/cabling, cable management, optic troubleshooting, heat management.

The preferred candidate would also have some of the following knowledge and experience:

• Mix of ISP, MS(S)P, and Data Center experience; campus/wifi and use of public cloud are not our focus.
  • Implementation/maintenance/support (not just use) of public or private cloud infrastructure.
• Experience with virtualized deployments of Juniper/PA.
• Some knowledge of other vendors and device types (Cisco, Arista, Fortinet, F5, A10, Ubiquity, etc.).
  • Ability to apply knowledge regardless of a vendor with minimal effort.
• Experience with SDN on both the networking and server (OVS or similar) sides.
  • SDWAN experience would be a bonus.
• Experience with Puppet, NetBox, Windows DHCP, Windows NPS, and RSA.

All candidates must possess the highest level of personal integrity, value team success over individual achievement, have the ability to contribute significantly to extending a culture of collaboration, both internally and externally, in order to maintain the superior reputation of Trinity Cyber - and enjoy having fun.