Staff Software Engineer, Data Infrastructure

Slack is looking for an experienced Software Engineer to join our Data Infrastructure team. You will build and integrate scalable backend systems, platforms and tools that power our production Data Warehouse and help our partners implement, deploy and analyze data assets with a high level of autonomy and limited friction. You will play a meaningful role in scaling our data infrastructure as well as making our user experience interacting with data more efficient, pleasant and productive.

As Slack’s data grows (along with the number of customers, features and employees), the goal of the Data Infrastructure team is to provide the best blend of the tools and technologies that will strengthen the efficiency and dependability of the way we make decisions. You will design and integrate systems that power processing of our large and diverse datasets to allow our partners to focus on their strengths: data applications, data modeling, data analysis, search or machine learning.

Slack's API and web backend is built on PHP/Hack, and our backend services are written in Java and Go. Our data infrastructure is built on Apache Sqoop, Airflow, Hadoop, Hive, Presto, Spark, Kafka, Pinot, Flink and MySQL/Vitess.

What you will be doing

• You’ll own technical strategy for broad or complex requirements with insightful and forward-looking approaches that go beyond the direct team and solve large open-ended problems. 
• You’ll participate in the strategic development of methods, techniques, and evaluation criteria for projects and programs. This will include assessment of build vs buy decisions at every stage, backed by proof of concepts, benchmarking, etc. 
• Build and scale data infrastructure that powers batch and real-time data processing of hundreds of billions of records daily.
• Provide visibility into the health of our data infrastructure (comprehensive view of data flow, resources usage, data lineage, etc).
• Automate and handle life-cycle of the systems and platforms that process our data.
• Evolve maturity of our monitoring systems and processes to improve visibility and failures detection in our infrastructure.
• Streamline the intake of the raw data into our Data Warehouse, propose new age solutions for data ingestion and collaborate with key stakeholders to execute on it 
• Provide implementations to expose actionable data to internal and external partners.

What you should have

• 7+  years of experience working with data technologies that power analytics (e.g. Apache Sqoop, Airflow, Hadoop, Hive, Spark, Presto, Kafka, Pinot, Flink, etc or similar technologies).
• You have experience provisioning and maintaining large big data clusters. You have worked on scalability and reliability challenges for the same.
• You have a deep understanding of data persistence (relational, key/value, document, columnar, graph).
• You are skilled at crafting and building robust backend data services (distributed systems, concurrency models, microservices).
• You have a strong dedication to code quality, automation and operational excellence: unit/integration tests, scripts, workflows.
• You posses expertise in object-oriented and/or functional programming languages (e.g. Go, Java/Scala, Python).
• You can lead technical architecture discussions and help drive technical decisions within your team.
• You are a strong communicator. Explaining complex technical concepts to designers, support, and other engineers is no problem for you.
• You have a Bachelor's degree in Computer Science, Engineering or a related field, or equivalent training, fellowship, or work experience.

Slack is an Equal Opportunity Employer and participant in the U.S. Federal E-Verify program. Women, minorities, individuals with disabilities and protected veterans are encouraged to apply. Slack will consider qualified applicants with criminal histories in a manner consistent with the San Francisco Fair Chance Ordinance.

Slack is registered as an employer in many, but not all, states. If you are not located in or able to work from a state where Slack is registered, you will not be eligible for employment.

Slack has transformed business communication. It’s the leading channel-based messaging platform, used by millions to align their teams, unify their systems, and drive their businesses forward. Only Slack offers a secure, enterprise-grade environment that can scale with the largest companies in the world. It is a new layer of the business technology stack where people can work together more effectively, connect all their other software tools and services, and find the information they need to do their best work. Slack is where work happens.

Ensuring a diverse and inclusive workplace where we learn from each other is core to Slack’s values. We welcome people of different backgrounds, experiences, abilities and perspectives. We are an equal opportunity employer and a pleasant and supportive place to work.

Come do the best work of your life here at Slack.