Data Engineer – Palantir & PySpark Developer

Role: Data Engineer – Palantir & PySpark Developer

Key Responsibilities

• Design and develop data pipelines in Palantir Foundry using Code Repositories and Pipeline Builder
• Build scalable data transformations using PySpark (Spark SQL, DataFrames, RDDs)
• Ingest, clean, and process large datasets from multiple structured and unstructured sources
• Implement data modelling (bronze/silver/gold layers) for analytics-ready datasets
• Optimize Spark jobs for performance, cost, and scalability
• Develop reusable components and maintain data lineage and governance in Palantir
• Collaborate with stakeholders, analysts, and data scientists to deliver business insights
• Ensure data quality, validation, and monitoring across pipelines
• Work with CI/CD pipelines and version control (Git) for deployment
• Key Skills
• Technologies: Palantir Foundry, PySpark, Apache Spark, SQL
• Programming: Python
• Data Engineering: ETL/ELT, Data Modelling, Data Warehousing
• Cloud Platforms: AWS / Azure / GCP (whichever applies)
• Tools: Git, Airflow (if used), Docker (optional)
• Concepts: Distributed Computing, Data Governance, Data Quality

Regards,

Reshu Seth

Recruitment Consultant

Renaissance InfoSystem

M: +61 478 487 026

E: reshu@reninfo.com.au

W: http://www.reninfo.com.au