Python/Spark/AI Developer
Full-time
Recruit-It
Role purpose & context:
Our client is looking for a Python/Spark/AI Developer to replat form legacy T-SQL onto Spark/Delta Lake pipelines and build the APIs and AI features around them, writing type-safe, spec-first Python.
Key roles & responsibilities:
Must-have technical skills / experience:
Preferred / nice-to-have technical skills:
Seniority and experience:
Required qualifications:
Why you’ll love working the company
They believe in taking care of their team and creating an environment where you can thrive. As part of the company, you’ll enjoy:
Our client is looking for a Python/Spark/AI Developer to replat form legacy T-SQL onto Spark/Delta Lake pipelines and build the APIs and AI features around them, writing type-safe, spec-first Python.
Key roles & responsibilities:
- Build Spark/PySpark pipelines on Delta Lake, replat forming T-SQL into Spark SQL.
- Write type-hinted, tested Python with pytest suites and CI lint/type-check gates.
- Build REST APIs (Fast API) for pipeline jobs, with auth, idempotency, and status semantics.
- Validate migrated pipelines against the legacy system with parity evidence.
- Work spec-first against design docs/ADRs, documenting changes as you go.
Must-have technical skills / experience:
- Python (3.12) as a primary language — modern idiomatic Python: type-hinted code that passes strict static analysis (pyright/mypy), Pydantic models, ABC-based provider patterns, packaging with modern tooling (uv or Poetry), Click or similar CLI frameworks.
- Apache Spark / PySpark — production experience building data pipelines on OSS Spark (not only a managed vendor platform): Data Frame API and Spark SQL, partitioning and performance tuning, understanding of driver/executor architecture and Spark Connect.
- Delta Lake or an equivalent Lakehouse table format (Iceberg/Hudi) — MERGE INTO, schema evolution, time travel, idempotent write patterns.
- SQL — strong, dialect-portable — able to read legacy T-SQL (stored-procedure-era logic) and re-express its semantics faithfully in Spark SQL; comfortable reasoning about hashing, surrogate keys, and deduplication logic in set-based terms.
- Automated testing with pytest — fixtures, markers, tiered suites (unit/integration/e2e); test-first habits and comfort being held to parity/regression evidence.
- REST API development — Fast API or equivalent: request validation, auth (HMAC or similar signed-request schemes), idempotency, job-status semantics.
- Docker-based development — working daily against a Compose stack (Spark cluster, object storage, metastore, databases).
- Git + CI discipline — GitHub flow, PR-driven work with lint (ruff), type-check, and test gates on every change.
Preferred / nice-to-have technical skills:
- LLM integration engineering — building provider-neutral AI features behind an abstraction: prompt construction, structured output validation, local/self-hosted inference (Ollama, vLLM, llama.cpp, LM Studio) as well as hosted APIs; evaluation and guard-railing of model output used in data workflows.
- Data governance / privacy engineering — PII tokenization and hashing schemes, k-anonymity concepts, re-identification risk, POPIA/GDPR-adjacent data handling; multi-tenant isolation awareness.
- Lakehouse platform components — Hive Metastore, Trino, Apache Ranger; how catalogs, views, and row/column policies compose into a governed query surface.
- Azure data estate familiarity — ADLS Gen2, Synapse/ADF concepts (the legacy being replaced), Azure Key Vault, Service Bus.
- Observability instrumentation — structlog/structured logging, Open Telemetry metrics and traces, Open Lineage.
- Migration/parity experience — replat forming pipelines with byte/cell-level output comparison against a legacy system.
- Certifications — Databricks Certified Developer for Apache Spark or equivalent Spark credential.
Seniority and experience:
- – Intermediate-to-senior. 4+ years professional Python development, with 2+ years building Spark (or comparable distributed) data pipelines in production. Must work autonomously against written design/requirements docs and ADRs — the codebase is heavily spec-driven, and changes are expected to arrive with tests and documentation, not just code.
Required qualifications:
- Bachelor’s degree in Computer Science, Engineering, or equivalent experience.
- Spark credential advantageous; proven record shipping production Python/Spark pipelines and working independently in a hybrid/remote team.
Why you’ll love working the company
They believe in taking care of their team and creating an environment where you can thrive. As part of the company, you’ll enjoy:
- Flexible Working Arrangements : Whether you are a night owl or an early bird, they offer hybrid and remote options to suit your lifestyle
- Comprehensive Benefits : From a wellness program to home office reimbursements and continuous learning opportunities, they have got you covered.
- Team Culture: Fun team-building activities, regular socials, and a supportive, inclusive culture that values transparency, accountability, and work-life balance.
- Performance Incentives : Competitive salaries, ESOP, and recognition for your hard work.
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Python/Spark/AI Developer in South Africa vacancy
R 800,000 pa
...looking for an opportunity to build modern, AI-powered products from end to end? This... ..., combining React, TypeScript and Python with cloud technologies and emerging AI... ...professionals with the right roles. Do you know a developer or technology specialist looking for a...- ...agentic workflows. Own the full lifecycle of AI features: data collection and evaluation,... ...powered systems in production. ~ Strong Python skills, with hands-on experience using... ...with streaming data pipelines (Kafka, Spark) or feature stores. Contributions to open...
- ...AI Automation & Systems Developer - improve operations. Relevant Qualification 2-4 Years experience in Software Development, Systems Development, Automation. Exp building applications and workflow solutions. Desigh, build and maintain busiens applications and...
- ...environments is looking for an exceptional Senior AI Engineer to join them You will have... ...into production-grade solutions If Python is your engineering foundation, LangChain... ...that we can successfully place the right developers with the right companies in the right...
R 1,000,000 pa
Ready to take your AI engineering expertise to the next level?... ...for performance and cost, and develop scalable machine learning solutions... ...experience ~ Strong Python skills, with hands-on experience... ...streaming data pipelines (Kafka, Spark) or feature stores....- ...retailer, is looking for a highly talented Senior Software Engineer (Python) to join our team. We are a young, dynamic, hyper-growth... ..., you will be someone whose expertise directly translates into developing robust software in the e-commerce domain. You will have the opportunity...
R 700,000 pa
We are looking for an Intermediate AI Engineer to join our IDEA (Insights, Data & Enterprise... .... Responsibilities: Assist in developing Generative AI and LLM-based solutions .... ...and integrate AI solutions using Python, APIs, and cloud technologies . Support...- ...senior-level position ~ Experience in writing robust, efficient production code ~ Proficient in a strongly typed language such as Python ~ Experience with development in a Linux environment ~ Experience with SQL database systems ~ Open to diversifying language...
- ...KEY RESPONSIBILITIES: Design and build AI/ML products that drive real business outcomes from ideation to production Work closely with business teams (Revenue, Studios, Finance, Marketing, Operations) to uncover high-value use cases Run discovery sessions to understand...
- Fully Remote Working Requirements: Experience in data architecture, data engineering, and AI/ML deployments. Strong experience with OCI Data Science, Oracle AI Services, or equivalent enterprise cloud data platforms. Hands-on OCI experience is mandatory.
- WERE HIRING: FRONT-END DEVELOPER Location: South Africa IgniteByte Digital is looking for a talented Front-end Developer to join... ...tests Collaborate with designers, back-end developers and QA Use AI-assisted tools such as GitHub Copilot or Cursor, with proper...
- Duties: Leadership, GIS & Process Optimisation Investment Analysis, Due Diligence & Technical Oversight Commercial Negotiations & Transaction Execution Permitting, Licensing & Environmental Compliance Strategic Business Development & Pipeline Growth ...
- ...Location: Cape Town Key Responsibilities Develop and maintain application software using C. Design and implement new software features. Debug, test, and optimise existing code bases. Produce technical design documentation for new functionality. Collaborate...
- ...experience ~ Experience with APIs, debugging and code reviews ~ Experience mentoring developers ~ Comfortable using AI-assisted development tools Advantageous: Python / Django / Django REST Framework Vue / JavaScript / TypeScript PostgreSQL Linux,...
R 900,000 pa
...are looking for a highly experienced Senior AI Software Engineer to help design, build,... ...services and AI-enabled applications. Develop secure, observable, and production-ready distributed... ...one modern backend language such as: ~ Python ~ TypeScript / Node.js ~ Java ~ Go...R 20,000 - 30,000 per month
...Developer (Telecommunications) Location: Woodmead Salary: R20 000 to R30 000 p.m. Our client in the Telecommunications industry is looking for a Developer. Key responsibilities Integrations (socket, Web Services (SOAP and RESTFUL), Methods) Web Portals...- ...landscape. If you're passionate about architecture, mentoring developers, improving delivery practices, and building a high-performance... ...Scalable Enterprise Architecture Security-Conscious Engineering AI-Assisted Development Workflows What We're Looking For ~8+...
R 2,000 - 6,000 per month
...Role Description We are seeking a highly skilled Web Developer based in LATAM to design, develop, and maintain web applications for... ...Angular. ~ Experience with back-end technologies such as Node.js, Python, PHP, or Ruby on Rails. ~ Knowledge of database management...- ...Certification - advantageous Well Rounded Developer with 8+ years' solid development... ...Stack Development Experience directing AI coding agents (e.g. Claude Code, Codex,... ...Advantages: Java NodeJS Typescript Python PostgreSQL MySQL Advantageous...
R 2,800 - 3,200 per month
...Exciting Remote Opportunity! We're Hiring an AI Operations & Systems Architect! Are you an operational builder and AI systems... ...standardize qualification gates between sales/setup/processing, and develop internal condition-clearing copilots. ~ Lender Matching & Deal...- Our client in the Telecommunications Industry is looking to hire an Integrated Developer. Duties and Responsibilities • Developing C# .NET solutions for the organization. • Creating in-house applications using the .NET framework. • Debugging and maintaining written...
- ...This is a remote position. We are seeking an experienced Intermediate Java Developer with a strong hands-on development background and exposure to integration, databases, testing, deployment and troubleshooting. The successful candidate will be responsible...
- ...certification and learner-records system for a public-sector client in the education and training sector. You will be the senior developer on the system, working on-site with the client's team and transferring skills and knowledge to their in-house developers. Location...
R 1.4 pa
...looking for a Player-coach technical Lead AI Engineer who owns delivery of major AI... ...systems to production ~ Strong production Python ~ Deep experience in at least two of:... .... Do you have a friend who is a developer or technology specialist? We pay cash for...- ...Customer Service Agent – AI & Technology We are an e-commerce company looking for a Customer Service Agent – AI & Technology to join our team. This role combines day-to-day customer support with opportunities to work on AI, automation, and customer service technology...
R 840,000 pa
...environments. This is a great opportunity for a developer who is ready to move beyond the basics,... ...Development Experience directing AI coding agents (e.g. Claude Code, Codex,... ...: Java NodeJS Typescript Python Advantageous Skills: AWS Solid understanding...R 650 ph
...scraping, LLM integrations, and agentic AI systems. This role is ideal for a hands-on... ...enjoys solving complex data challenges and developing intelligent automation solutions that... ...Qualifications & Experience Strong experience with Python and/or Node.js/TypeScript. Advanced...R 750,000 pa
...are looking for an Intermediate Full Stack Developer to join a growing engineering team... ...and API-driven applications with emerging AI capabilities. This role is ideal for a... ...object-oriented backend languages (e.g. Java, Python, Go, etc.) Frontend Angular...- ...Quick CV Drop Off Role: SeniorJava Developer Duration: 12 Months Location: South Africa Are you a Software Developer who... ...established enterprise platforms and the development of new products, AI capabilities and proof-of-concepts . AI is a genuine part...
R 600,000 pa
...Monitor system performance, availability, backups, patching, and disaster recovery processes. Develop and maintain automation solutions using Bash, PowerShell, and Python. Manage IT incidents, troubleshoot technical issues, and ensure timely resolution of service...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Python/Spark/AI Developer. Be the first to apply!
