Pluralsight
Senior Data Engineer
Senior Data Engineer
Job Description:
Join Pluralsight's Data Engineering team within the Data Services Organization. We are on a mission to build and maintain an infrastructure capable of handling high-volume, business-critical data. This role is crucial in unifying data sources across our multiple products and domains, with a focus on both streaming and batch data processing. Your Role:
As a Senior Data Engineer, you'll lead our team in implementing best practices with modern data tools. You'll primarily work with technologies such as Snowflake, Kafka, Materialize, and dbt, focusing on crafting robust data solutions. While these will be your main tools, we are also open to exploring new horizons in streaming data processing. Technologies like Spark, Flink, and Scala may become part of our toolkit as we discover innovative use cases and expand our capabilities. Your role involves applying your expertise in data transformations and leveraging your knowledge of streaming platforms. Who You Are:
- 5+ years of experience in data engineering, with expertise in both batch and streaming data.
- Strong foundation in SQL development on RDBMS (Snowflake and Postgres preferred).
- Functional knowledge of Kafka and familiarity with other streaming technologies (Flink, ksqlDB, Databricks, etc.).
- Proficient in dbt for data transformations and building data pipelines.
- Hands-on experience with scripting languages (Python, BASH, Scala is a plus).
- Experience with dimensional data modeling and understanding of data workflow diagrams.
- Familiar with source control and deployment workflows for ETL processes.
- Experience in metadata management and data quality.
- Knowledge of software engineering best practices, including CI/CD, monitoring, and alerting for production systems.
- Development and management of both streaming and batch data pipelines.
- Ensuring high-quality, reliable data delivery, with a focus on streaming data platforms and dbt transformations.
- Supporting and evolving the data environment to maintain speed and availability.
- Leading data cataloging and management efforts.
- Overseeing production performance tuning and resource management.
- Proven capability to manipulate and analyze complex, high-volume data from a variety of sources.
- Strong experience in designing and building end-to-end data models and pipelines, including both streaming and batch processing.
- Knowledge of data management fundamentals and data storage principles.
Join our mission to democratize technology skills. At Pluralsight, we’re dedicated to our values and creating an inclusive environment where diverse viewpoints are celebrated. We are an equal opportunity employer and value each member's unique contributions.
Bring yourself. Pluralsight is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability, age or protected veteran status. Pluralsight will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please visit the bottom of our website to learn how to request an accommodation.
#LI-JM2
#LI-Remote