The Development & Operations (DevOps) Engineer is responsible for the oversight, maintenance, improvement, and development of Burning Man's server and development operations infrastructure (hardware, operating system and software, cloud based, and network) and various services (backups, DNS, email, cloud services etc). This involves both the performing system administration duties and providing support to other departments.
The deadline to apply for this role is 5:00 pm Pacific on Sunday, April 17, 2022.
DUTIES & ESSENTIAL JOB FUNCTIONS
Administration and Maintenance (80%)
Server / Network
- Responsible for provision, deployment, optimization, and maintenance of load-balanced and scalable EC2 instances and databases in AWS
- Responsible for ensuring proper monitoring, scale, OS upgrades and installation of necessary patches and resiliency across all environments from test to production
- Responsible for systems administration for all network servers and instances
- Manage platform provisioning, standardized configuration management and code deployment by gathering use cases from engineering to support CI/CD and integrated testing solutions
- Responsible for integrated logging and reporting solutions for end to end application scaling and troubleshooting
- Support engineering by creating, modifying, maintaining scalable deployment strategies and infrastructure that follows best practices. creating maintaining standardized development environments from production to containerized local environments.Ensure that systems are safe and secure against cybersecurity threats
- Monitor health of web services and be first line of defense to make changes / updates
- Configure, administer, maintain, test and operate all backup systems
- Supervise ongoing request-driven administrative tasks, such as server account creation, and email alias and list requests
Documentation
- Keep architecture, network diagrams and server documentation up to date
- Analyze log files when required to investigate and recommend solutions to issues
- Audit and improve performance and security on all servers on a regular basis
- Perform root cause analysis for production errors
- Working with software developers and software engineers to ensure that development follows established processes and works as intended
- Create and document capacity planning strategies including data collection and analysis for capacity building and planning
DNS
- Manage domain names and DNS in our third party DNS provider
- Implementation, installation and updates of SSL certificate
On Playa and Volunteer Support (20%)
- Manage and support the playa Technical Operations Team with technical solutions and issues as necessary.
- Recruit, train and maintain a pool of outside volunteers to help with sysadmin tasks as needed
- Collaborate with outside sysadmin volunteers, leverage their contribution and coordinate their help with diverse tasks as needed
OTHER FUNCTIONS AND RESPONSIBILITIES
- Attend Department and Organization meetings as required
- Contribute to writing post event debriefs, Embers and afterburn reports
- Provide hosting services and ISP usage reports and forecasts to be used for budget planning
- Identify risk areas and suggest potential solutions.
- Communicate with vendor hosting services when needed to support technical initiatives.
- Perform other duties as assigned.
TEACHING AND LEARNING
- Documentation - Assists in the creation, publishing, and maintenance of process documentation related to their assigned duties.
- Knowledge sharing – Helps acculturate and educate junior staff and volunteers in Burning Man culture and business processes.
- Training development - Collaborates with the education team to develop training materials in their area of expertise, as requested.
- Learning – Pursues ongoing learning in their area of expertise, working to stay current on emergent knowledge, skills, and processes.
QUALIFICATIONS
REQUIRED
- 5+ years experience in DevOps role
- Linux expert (Debian)
- Fluent in shell scripting
- Ability to manage servers with Ansible
- Expert level knowledge of cloud computing (Amazon AWS, EC2, others) be ready to document and set best practices
- Expert in building and scaling Amazon instances (horsepower, load balancing, etc.) to serve WordPress sites (on demand scaling)
- Able to support code deployment procedures to servers via Github (theme/plugin code deployment)
- Able to configure and understand DNS and various records (CNAME, etc)
- Expert in Nginx and Apache configuration and maintenance
- Expert in mySql databases, and DB replication services
- Experience deploying Django / Ruby / php / Python applications
- Strong communication and documentation skills
- Detail oriented, reliable and dependable
- Dedicated to providing uptime and quality service
- Comfortable juggling tasks and priorities
- Maintain high standards of confidentiality
- Able to meet deadlines and honor commitments
- Proven experience managing complex projects
PREFERRED
- Knowledge and experience within the Burning Man Project, organization and community and able to work in Nevada around the Burning Man event
- Very motivated by and interested in current and emerging web technology trends
- Leadership experience managing the resolution of production support issues in a high profile and high-risk environment
- Ability to supervise and mentor others to foster a cooperative and collaborative work environment
Note: If you have gained the necessary experience to succeed in this role and have acquired this experience in other ways, we encourage you to apply and share all relevant experience in your application.
REQUIRED LICENSES/CERTIFICATIONS
Good driving record.
ABOUT THIS ROLE
- This is a regular, full-time, eligible for benefits role, with preference given to candidates located in proximity to one of Burning Man’s operations (San Francisco, CA, Reno, NV, or Gerlach, NV.) Working hours will be based on Pacific Time.
- Ability to adjust schedule seasonally to work at the Burning Man event held in Nevada will be required.
- Burning Man Project celebrates the importance of creating a diverse environment and enthusiastically encourages Black, Indigenous, Peoples of Color, LGBTQIA+, Immigrant, women, and people with disabilities, of all ages, to apply.
- Burning Man Project provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.
- Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records
- Notice for Remote Opportunities: Burning Man Project is currently able to hire remote employees living/working in the following states: AZ, CA, CO, FL, ID, LA, MA, MI, MT, NC, NM, NV, NY, OR, TX, WA, WI. Working hours for all positions are based in Pacific Time.
PHYSICAL REQUIREMENTS/WORK ENVIRONMENT
- Off-playa: Work is primarily sedentary in nature, no special demands are required.
- On-playa: Work is primarily sedentary but will also include the need to find and meet with department representatives who may not be in the vicinity therefore this could include walking or using some form of transportation. Some of the work will be in outside weather conditions and will be exposed to fumes or airborne particles as well as possible extremes in temperature