Data Enginner
iDbase Software
Job Description Starburst Data Engineer
Role Purpose
We are seeking a Starburst Data Engineer to support and maintain an enterprise Starburst platform and assist with the migration of workloads from Apache Hive to Starburst.
The role focuses on production support, troubleshooting, SQL query optimisation, platform stability, and helping users successfully run analytical workloads on Starburst. Strong SQL skills and Knowledge of Presto, Starburst or Trino are essential.
Key Responsibilities
Assist with the migration of data workloads from Apache Hive to Starburst (Trino)
Investigate and resolve migration issues while providing support for production stability and operational continuity.
Monitor platform health, availability, query performance, and resource usage.
Investigate and resolve platform, connectivity, access, and SQL query issues.
Rewrite and optimize SQL queries for Starburst compatibility, maintainability, and query performance.
Optimize Starburst query performance through effective use of partitioning, file formats, caching, and related tuning techniques.
Troubleshoot slow or failed queries and recommend improvements.
Assess existing Hive schemas, queries, and pipelines for migration readiness.
Work with engineering, architecture, infrastructure, and business teams to resolve issues and improve platform performance.
Support platform upgrades, configuration changes, deployments, and operational improvements.
Maintain technical documentation, troubleshooting guides, and support procedures.
Participate in incident resolution and root-cause analysis.
Promote automation, monitoring, security, and operational best practices.
Required Skills
Advanced SQL skills, including query optimization, complex joins, and window functions.
Understanding of partitioning, bucketing, indexing, and related data optimization strategies.
Strong debugging, troubleshooting, and performance tuning skills.
Experience with Apache Hive and distributed data processing environments.
Good to have Knowledge of Presto, Starburst or Trino.
Experience working with big data ecosystems such as HDFS, S3, and data lake architectures.
Knowledge of modern table for formats such as Parquet, ORC.
Knowledge of analytical file formats such as Parquet
Good to have knowledge of modern table formats such as Iceberg and Delta
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
Preferred Skills
Experience supporting large-scale data platform modernization or migration initiatives.
Familiarity with performance benchmarking and workload optimization in distributed query environments.
Exposure to data governance, metadata management, and secure data access practices.
Ability to create clear technical documentation and communicate effectively with cross-functional teams.
Professional Traits
Self-starter with strong ownership and accountability.
Strong support and customer-service mindset.
Disciplined, reliable, and highly committed to delivering quality work.
Strong team player with excellent collaboration and communication skills.
Quick learner with the ability to adapt to new technologies and environments.
Proactive approach to troubleshooting and problem-solving.
Able to work independently as well as in cross-functional teams.
Focused on continuous improvement, automation, and operational excellence.
Strong analytical and platform support mindset.
- ...Harness your expertise to shape transformative data solutions within a dynamic, forward-looking organization dedicated to innovation in the insurance and management sectors. This role offers a strategic platform to influence enterprise-wide data architecture, drive modernization...
- You design how data, knowledge and systems connect, so that the AI solutions Centrax Digital delivers have reliable, governed information to work from. ABOUT CENTRAX DIGITAL Centrax Digital is an AI-first consulting and digital engineering firm operating from Bryanston...
- About the job Data Scientist Role Purpose Derive business value from data through systematic analysis and interpretation. Proactively source data from multiple suppliers and conduct advanced statistical and analytical work to extract actionable insights....
- About the job BI Data Scientist Role Purpose As a data scientist, you will play a key role in leveraging data to drive insights, inform decision-making, and enhance our life and non-life insurance operations. You will work closely with cross-functional teams...
- ...Data & Analytics Lead (Governance & Strategy) Sandton R 1 300 000 - R 1 500 000 We're looking for a Data & Analytics Lead to drive data strategy, governance, and analytics across the business. This is a strategic role focused on enabling trusted, well-managed...
- ...Job Description The Technology Data Architect designs and governs enterprise-grade Azure data platforms, delivering secure, scalable, and high-performance solutions aligned to business and regulatory needs. The role drives data modernization, analytics, automation,...
- ...part of this exciting growth journey, to reset our future and shape our destiny as a proudly African group. Job Summary The AI & Data FinOps Lead is responsible for establishing, leading and scaling a new AI FinOps capability within Absa's Chief Data & Applied AI...
R 110,000 per month
About the job Senior Azure Data Engineer Role Overview Help us migrate a major bank to Azure Data Factory and Fabric, building a modern data platform in Sandton. You'll be key to a critical, high-visibility project. What We're Looking For - 5+ years of...- ...Job Description The Delivery Manager is accountable for the end-to-end delivery of strategic Data & Analytics initiatives within the Data & Analytics portfolio. This role ensures delivery execution is aligned to business priorities, architectural principles, regulatory...
- ...Group is seeking an experienced Enterprise Architect to join a strategic architecture engagement, supporting the development of OT/IT Data Integration and Business Building Architecture. The role will work across Enterprise, Data, Integration, Security and Business...
- ...Role Purpose We are building a talent pool for future Product Management & Data Quality Specialist (Assistant Buyer) opportunities. This role is ideal for detail-oriented individuals passionate about products, data accuracy, and process improvement. You will play a...
R 35,000 per month
...Job Description THE OPPORTUNITY THAT AWAITS YOU A leading simulation science and data technology consultancy, based in the Northern Suburbs of Johannesburg, is seeking high-calibre Graduate Analysts to join their team of analytical problem-solvers. This is...- ...Key Responsibilities ~ Serve as the central intake point for ad-hoc data and reporting requests across the wider group and various departments. ~ Locate, extract, and assimilate data from disparate internal systems, ensuring it is checked, transformed, and structured...
- ...Ready to architect the future of data? If you're passionate about designing world-class data platforms, driving cloud transformation, and turning complex business requirements into powerful data solutions, then this opportunity could be your next big move! We're on...
- ...growth journey, to reset our future and shape our destiny as a proudly African group. Job Summary To support the Head: Credit Data and Regulatory Control, Credit Risk in establishing a dedicated oversight and control function in credit risk regulatory reporting to...
- ...responsibilities: Load, review and submit daily payment batches through banking platforms. Create EFT files from approved source data and upload them to bank portals. Validate beneficiary details, amounts, currencies, value dates and remittance references....
- ...delivery issues. Respond to routine customer enquiries and direct operational issues to the right person. Assist with scheduling, data entry, filing and general office administration. Check documents and system entries for accuracy and completeness. Help...
- A leading financial services organisation is seeking a Data Scientist to join its high-performing analytics team. This role is ideal for an individual who enjoys uncovering insights from complex datasets, building predictive models, and leveraging advanced analytical techniques...
- ...Monitor financial performance and analyse variances vs budget/forecast. Maintaining monthly Fixed Asset Register Interpret financial data and present reports to management. Provide detailed financial and cost analysis reports. Process Accounts Payable activities,...
- ...Mid-Level Data Scientist / Data Science Architect / Data Science Developer | Bryanston, Centurion or Gqeberha | Permanent Turn complex data into intelligent solutions that solve real business problems. This is an opportunity to combine data science, machine learning...
- ...multiple brand/s and categories simultaneously. Direct P&L responsibility and margin optimization. Driving profitable growth through portfolio mix management. Strong analytical skills in financial and marketing data. Proficiency with MS Office and IT systems.
- ...regression suites, online evaluation and human review. - Engineer for safety: prompt injection defence, output filtering, sensitive data redaction and escalation paths. - Deploy and operate AI workloads in production, including monitoring, drift detection and incident...
R 10,000 - 15,000 per month
...monitoring the Market Analysts KPI's.Duties & ResponsibilitiesUpdate and maintain reports according to information in the CRMAudit CRM data and reports to ensure accuracy, completeness and consistencyMaintain organised and up to date records of all relevant dataMonitor...- ...departments / Business units during auditing (BBBEE verification - DTI and DMR / etc.) Identify opportunities for improvement of master data quality Participate in improvement projects as required Key requirements: ~ Degree (B.com) or National Diploma (Business...
- A leading firm in financial analytics is seeking a sharp, detail-oriented Graduate or Junior Data Scientist to join their dynamic team in Johannesburg. This is a great opportunity to apply your skills to real-world financial challenges like fraud detection, client churn...
- ...Key Responsibilities: Design and build data pipelines, models, and analytical frameworks Conduct exploratory and predictive analysis to uncover trends and opportunities Apply scientific methods to test hypotheses and validate insights Collaborate with engineers...
- ...Role Purpose: ~ The Master Data Manager is responsible for optimizing and managing product and supplier master data in compliance with standard operating procedures, data governance rules, anti-corruption and finance control requirements. This role ensures the accuracy...
- ...Compliance Technology WatersEdge Solutions is partnering with a global, AI-powered technology business to recruit an experienced Senior Data Engineer . This is an opportunity to join a growing engineering team working at the intersection of data, software engineering, AI,...
- ...designs that fit each client's existing platforms, on CAIF-5, the Centrax AI delivery framework. You make the trade-offs across models, data, integration, security, cost and operations explicit, and you work within the Ground and Embed pillars of CAIF-5. WHAT YOU WILL DO -...
- The Payroll Administrator will be responsible for administering and processing payroll input by capturing and verifying payroll data, maintaining leave records, ensuring data accuracy, executing net salary runs, issuing payslips, and preparing statutory documents. Key...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Enginner. Be the first to apply!
