Currently Open - PySpark/Spark Developer

 

The Leader In Cloud Applications & Big Data

KPI Partners provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data, business intelligence, and data discovery.

KPI works with both corporate technology departments and corporate business units to develop value-added decision support solutions, not just new technology deployments.

Title: Spark Developer

Locations: 
Fremont, CA, Boston, MA, Dallas, TX (USA)
Hyderabad and Bengaluru (India)

Experience: 4-8 years

Education/Qualification: Bachelor’s degree in Computer Science or related field, or equivalent education and experience

Requirements:

  • Must be a Subject Matter Expert in Spark.
  • Proficiency with Big Data processing technologies (Hadoop, Spark, AWS)
  • Experience in building data pipelines and analysis tools using Python, PySpark, Scala
  • Create Scala/Spark jobs for data transformation and aggregation
  • Produce unit tests for Spark transformations and helper methods
  • Write Scaladoc-style documentation with all code
  • Design data processing pipelines
  • Good to have experience with Hadoop / AWS.
  • Good to have Spark certification
  • Preferred to have Java background.
  • Passionate about learning new technologies
  • Ability to learn new concepts and software quickly
  • Analytical approach to problem-solving; ability to use technology to solve business problems
  • Familiarity with database-centric applications
  • Ability to communicate effectively in both a technical and non-technical manner
  • Ability to work in a fast-paced environment

Skills:

  • Spark / Scala
  • PySprak
  • Python
  • Amazon Glue
  • AWS