Founded in 2002, GoQuo Sdn. Bhd. is one of the travel technology pioneer in providing ancillary solutions to major airlines and online travel agents in Asia. Our clients include Malindo Air, Nok Air and more and we are expanding our client base to other regions. GoQuo strives to become a world class travel technology provider.
We are looking for a Big Data Engineer who will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Duties and Responsibilities
• Create and maintain optimal data pipeline architecture for data science team to build and optimize our data product into an innovative industry leader.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google’s Big Data technologies.
• Experience building and optimizing Big Data data pipelines, architectures and data sets.
• Experience with big data tools: Hadoop, MapReduce, HFS, Spark, Kafka, etc.
• Experience with data pipeline and workflow management tools: Dataflow DataProc, Dataprep, etc.
• Experience with Google cloud services: Compute Engine, Container Engine, Cloud SQL, BigQuery, Bigtable etc.
• Experience with stream-processing systems: Storm, Spark-Streaming, etc.
• Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.