Principal or Senior Solutions Architect (Data Engineering)

Principal or Senior Solutions Architect (Data Engineering)

This job is no longer open

phData works exclusively in the realm of data and machine learning. By partnering with the biggest cloud data platforms (Snowflake, Azure, AWS, Cloudera), we’re able to help the world’s largest companies solve their toughest challenges.

Our work is challenging and our standards are high, but we invest heavily in our employees, starting with a 2-4 week bootcamp to ensure you’ll be successful. Plus, you’ll get to work with the brightest minds in the industry and the latest data platforms on the market. And, because the data and ML industry is changing rapidly, you will always have the opportunity to learn - whether that’s a new technology, diving deeper into your preferred stack, or picking up an entirely new skill set.

Even though we're growing extremely fast, we maintain a remote-first, exciting work environment. We hire top performers and allow them the autonomy to deliver results. Our award-winning workplace fosters learning, creativity, teamwork. Most importantly, our team has the option to work from the conveniences of their home or work from our state of art Bangalore office located in the heart of the city.

  • 2022 Snowflake Partner of the Year
  • 2022 Best Places to Work
  • Best Places to Work (2017, 2018, 2019, 2020, 2021)
  • 5000 Fastest Growing US Companies (2019, 2020, 2021)
  • Minneapolis/St. Paul Business Journal’s Fast 50 (2021)
  • Snowflake Elite, Snowpark Accelerated & Snowflake Partner of the Year (2020 & 2021)

 

 

Principal Or Senior Solution Architect Core Competencies

A Principal OR Senior Solution Architect at phData is responsible for leading the architecture, design, development, and validation of large scale complex data integration and data pipeline workloads. This includes utilizing technologies and data services from cloud providers such as AWS/Azure and Cloud Native platforms such as Databricks (Datalake or Lake House) or Snowflake Cloud Data Platform. They will also play a key role in data integration, modeling, governance, and security tasks, as well as phData internal tools. The Senior Solution Architect will work closely with architects and other technical stakeholders to understand customer data needs and implement appropriate solutions. The ultimate goal of this role is to enable customers to extract value from their data and bring data solutions to life through data development projects.

 

Must Have Technical Delivery Skills

  1. Strong working experience with Hadoop & Apache Spark (on Prem or Cloud version) including designing, developing, managing and optimizing activities for production grade data applications.
  2. Deep technical expertise building end to end data pipeline for medium to large scale data sets from a variety of sources (structure and/semi-structure/unstructured) to the data platform like Hadoop Platform or Cloud Native Data Platforms.
  3. Should have working experience managing more than one technical projects at any given time.
  4. Have strong working knowledge on how to architect medium/large scale complex data requirements and translate that architecture into a working solution.
  5. Working exposure on defining data governance policies and procedures, including data security and access controls.
  6. Good understanding on the Cloud storage services (be it AWS S3 or Azure ADLS Gen2 or GCP Buckets) and know-how with respect to small/large dataset with different data formats.
  7. Very strong understanding and working knowledge on SQL (standard/analytical/advance) alongside traditional/conventional data warehousing design patterns. Also good knowledge on best practices around SQL and how to enforce them in an enterprise environment.
  8. Solid understanding of the data validation process using some kind of utilities or automation.
  9. Good working knowledge with bash scripting or python scripting to enable automation in the unix platform.
  10. Hands-on experience troubleshooting, optimizing, and enhancing data pipelines and bringing improvements in the production environment.
  11. Strong knowledge and working experience  with one of the version control systems (Ex. GitHub or GitLab or Bitbucket or Code Build) and  continuous integration and deployment procedure patterns in data engineering space.
  12. Good ability to produce architectural/design documents, best practice documents,  data integration diagrams and artifacts related to data design.
  13. Must have working experience with one of the data engineering orchestration tools (like Apache Airflow or Apache Oozie or any other commercial tool).
  14. Staying up-to-date with industry trends and technologies related to data management and architecture.

Nice To have Skills (In Past 2 Years)

  1. End to End Data Migration Experience from Legacy (Oracle or SQL Server or DB2 or Netezza etc) to Snowflake
  2. Data Transformation tool like dbt (dbt cli or dbt cloud)
  3. Data Integration Cloud Tool life Fivetran
  4. Low Code No Code ETL Tool like Matillion

Behavioral Requirement

  1. Must be curious and hungry to learn.
  2. Be ready to learn quickly (in a very structured & methodological manner)  and adapt to new technologies or new types of tools as and when required.
  3. Demonstrated ability to work independently as well as with team and customer/client stakeholders
  4. Good communication skills (verbal and written) - one of the most important skills when working with phData as a consulting and service organization.
  5. Good and collaborative team player with in-house team as well as hybrid team with members from client, vendor, and in-house resources.
  6. Given the project execution dimension and deliverables, a strong sense of time management is required. (For example, schedule variance, effort variance, cost variance, and so on.).
  7. The ability to guide and drive a project team during unforeseen circumstances or when risk becomes an issue and the team is racing against deadlines.
  8. A keen attention to detail is required, whether it is for requirement documentation, code review, architectural review, or any other task that may or may not have an impact on project deliverables.
  9. Understanding how a team works and the art of delegation, as well as how to get a team to deliver value based on project or organizational goals, are essential.

Team Management Skills

  1. Leading and motivating a team of data engineers to achieve project and organizational goals.
  2. Providing guidance and support to immediate team members to help them develop their skills and careers.
  3. Setting performance expectations and conducting performance evaluations for team members.
  4. Identifying and addressing conflicts or problems within the team, and facilitating resolution.
  5. Supporting and promoting a positive and inclusive team culture.
  6. Ensure the team members are learning and staying up-to-date with new technologies and making sure all of them are aligned with the organization's larger objectives.

 Qualifications & Other Requirements

  1. BE/BTech in computer science or MCA or equivalent degree with sound industry experience (15+ years)
  2. A minimum of 5+ years experience in developing production/enterprise grade big data (like Cloudera, Hortonworks, HDInsight, Hadoop/Spark Cluster) and elementary working knowledge with cloud native data  engineering solutions (AWS Data/Storage Service, Azure Data/Storage Services, GCP Data/Storage Services) or 2 years working experience with Snowflake or Databricks technologies.
  3. Good Programming or Scripting Language Experience (Python or Java or Scala). Must have developed a small or mid side applications or data product with complete SDLC cycle.
  4. Good awareness on how a cloud-based system works (be it AWS or Azure or GCP) including basic and common features like storage, security and data services.

 

Perks and Benefits:

  • Medical Insurance for Self & Family
  • Medical Insurance for Parents
  • Term Life & Personal Accident
  • Wellness Allowance
  • Broadband Reimbursement
  • Professional Development Allowance
  • Reimbursement of Skill Upgrade Certifications
  • Certification Bonus
This job is no longer open
Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.