Best practice

• our operations management utilises what we deem best practice. that is, we either have implemented or are striving to achieve the following in everything we implement and manage:
• provisioning of new (or re-purposing of existing) infrastructure is either automated or documented transparently.
• bootstrapping and configuration of infrastructure is automated transparently.
• routine maintenance of infrastructure is automated transparently.
• extraordinary maintenance of infrastructure is automated or documented transparently.
• output/logs from infrastructure are aggregated and available transparently for analysis.
• issues with infrastructure are transparently visible, obvious and trigger alerting.

Technologies and platforms

• to realise these objectives, some of the technologies and tools we utilise include:
• ansible for instance configuration and maintenance.
• cloud-config for instance bootstrapping.
• custom orchestration: rubberneck utilises cloud-config like maintenance configurations.
• docker for applications with complex dependencies that benefit from being shared and run by an expanded audience.
• github actions with self-hosted runners for ci build, test, release, deployment.
• letsencrypt/certbot for automated cert acqisition and renewal.
• nginx for cert serving, reverse proxying, load balancing.
• mongodb, pgsql for application state.
• prometheus, promtool. loki, grafana for log aggregation and dashboarding.
• route53, cloudflare for dns management, load balancing, health/uptime monitoring.
• ssh, gpg, wireguard and ed25519/curve25519 are the underpinnings of our security infrastructure.
• terraform for instance provisioning and bootstrapping.

Requirements

• a successful candidate would likely have experience, skills and aptitudes that include:
• given our mission, a strong, personal motivation and conviction to contribute in the transaction privacy space is an essential aspect.
• written and spoken english proficiency or the ability to interact with written and spoken english effortlessly using technology.
• championing technology, tools and practices that facilitate the mission. understanding when, why and how to use and champion what.
• an ability to rapidly prototype and demonstrate the benefits and pitfalls of solution proposals. any aptitude for dashboarding tools like react, vue, grafana or others is useful.
• networking
• a practical understanding of networks, dns, tls, firewalls, port forwarding, reverse proxying, traffic shaping and packet dropping
• distributed source control
• git and github/gitlab and the use of their associated tooling and configuration should be second nature. a github/gitlab profile that demonstrates a history of best practice is always impressive. if privacy or other concerns prevent this, then some other way of conveying this experience is helpful.
• a bias toward transparency is essential.
• our intention is to make everything we do discoverable, understandable and repeatable by others. ie:
• see my patch implementation at this url: …
• is infinitely preferable to:
• i fixed all the things and they work great now.
• an appreciation of security concerns including (an aptitude to learn quickly is fine):
• what aspects of an implementation or configuration should be deemed secrets?
• what aspects of an implementation or configuration can benefit from being transparent/public and why?
• a practical understanding of gpg, ssh, wireguard, rsa, ed25519 and curve25519 and when to use which is essential.
• being adept at spotting vulnerabilities and weaknesses.
• being able to read bash will make it easier to understand what’s going on. we use a lot of it in both documentation and implementation.
• understanding when to containerise or componentise a thing and being able to articulate the justifications for either.
• understanding of checksums and hashing.
• understanding what type of updates should happen automatically and what type should be more considered or deliberate.
• weekly status report (more frequently when there are critical issues in the pipeline) including:
• a written summary (internal, notion) of what was achieved, what is planned and any blockers.
• two zoom meetings (immediate infrastructure team, wider engineering department), articulating the same, with webcam on.
• out-of-hours reachability and availability for mission-critical or emergency issue resolutions.
• being relaxed about admitting mistakes or problems with a view to sharing learnings and improving the overall sittuation.

Good to Have

• some skillsets and other considerations that will set you apart, include:
• being available during pst, cst or est hours is a bonus as eet and aest are currently covered.
• being willing and able to hold peers accountable for the responsibilities above.
• knowing how to use cryptocurrencies, create and manage accounts/keys/wallets, instigate and analyse transactions and other extrinsics or smart contracts.
• experience running a complete/full blockchain node (for any chain but, substrate or ethereum are especially useful, understanding tokenomics, validation or staking dynamics, more so).
• understanding distributed and decentralised systems, incentivisation, ipfs, tor, etc…
• rust, go, javascript and python (or any async capable language) skills.
• cryptography primitives and/or blockchain at the protocol or white-paper level.