Big Data Engineer Job Description
We are looking to hire a talented big data engineer to develop and manage our company’s Big Data solutions.
As a big data engineer, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with development teams, build cloud platforms, and maintain the production system.
Big Data Engineer Responsibilities:
- Developing and maintaining Hadoop systems
- Analysing and determining the company’s big data needs
- Coordinating with the software development teams to finalize the scope of the system
- Developing and delivering big data solutions
- Developing company applications by building cloud platforms
- Managing loading from disparate data sets
- Developing and maintaining production systems
- Creating data pipeline architecture
- Planning and optimizing data delivery
- Using SQL and AWS to load data from various data sources
Big Data Engineer Requirements:
- Degree in computer engineering or computer science.
- Previous experience as a big data engineer.
- In-depth knowledge of Hadoop, Spark, and similar frameworks.
- Knowledge of scripting languages including Java, C++, Linux, Ruby, PHP, Python, and R.
- Knowledge of NoSQL and RDBMS databases including Redis and MongoDB.
- Familiarity with Mesos, AWS, and Docker tools.
- Good communication skills.