Job Description

Cloudera Admin:

• Design and automate the deployment of Cloudera CDP components (Data Lake, Data Hubs, Data Services) using Terraform.
• Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation
• Collaborate with data engineering teams to integrate CDP with existing Big Data workflows.
• Write and maintain automation scripts using Shell and Python.
• Manage infrastructure and configuration using YAML and JSON.
• Configuring and tuning Cloudera services for performance and security.
• Ensure secure, scalable, and cost-effective deployments on cloud platforms (AWS, Azure, or GCP).
• An understanding of all the Hadoop daemons along with their roles and responsibilities in the cluster.
• Should be able to troubleshoot issues in Cloudera services and fix those.
• Adding and removing nodes in the cluster.
• Rebalancing nodes in the cluster.
• Employ security using an authentication and authorization system such as Kerberos.
• Knowledge of changes required for migrating to Cloudera cloud version CDP
• Knowledge of Cloudera data services (CDW, CDE, CDF, CAI)
• Design Native Cloud Application Architectures and optimize applications for AWS
• Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53
• Must have in depth AWS development experience (Containerization Glue, Docker, Amazon EKS, Lambda, EC2, S3, Amazon DocumentDB, PostgreSQL)
• Strong knowledge of DevOps and CI/CD pipeline (GitHub, Jenkins)
• Scripting capability and the ability to develop AWS environments as code
• Hands-on AWS experience with at least 1 implementation (preferred in an Enterprise scale environment)
• Experience with core AWS platform architecture, including areas such as: Organizations, Account Design, VPC, Subnet, segmentation strategies.
• Environment and application automation
• CloudFormation and third-party automation approach/strategy
• AWS Cost Management and Optimization

Job requirements

• Extensive experience with Cloudera cloud data platform (CDP) and CDP Services and Big data knowledge.
• Proficiency in Terraform for infrastructure as code (IaC).
• Strong hands-on experience with Cloudera CDP and Hadoop ecosystem (Hive, Impala, HDFS, etc.)
• Experience with GitHub Actions or similar CI/CD tools (e.g., Jenkins, GitLab CI).
• Solid scripting skills in Shell and Python.
• Extensive experience in designing, provisioning, deploying and configurtion of the Cloudera cluster based on customer's need
• Extensive experience in AWS service EC2, VPC, ELB, S3, RDS, Lambda, Route 53 etc and should be able to design and deploy cloudera cluster on AWS cloud
• Strong knowledge of data processing using Cloudera services
• Good understanding of CI/CD concepts, version control, and DevOps best practices.

BIGDATA-HADOOP
Key responsibilities
Cloudera Admin:

• Design and automate the deployment of Cloudera CDP components (Data Lake, Data Hubs, Data Services) using Terraform.
• Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation
• Collaborate with data engineering teams to integrate CDP with existing Big Data workflows.
• Write and maintain automation scripts using Shell and Python.
• Manage infrastructure and configuration using YAML and JSON.
• Configuring and tuning Cloudera services for performance and security.
• Ensure secure, scalable, and cost-effective deployments on cloud platforms (AWS, Azure, or GCP).
• An understanding of all the Hadoop daemons along with their roles and responsibilities in the cluster.
• Should be able to troubleshoot issues in Cloudera services and fix those.
• Adding and removing nodes in the cluster.
• Rebalancing nodes in the cluster.
• Employ security using an authentication and authorization system such as Kerberos.
• Knowledge of changes required for migrating to Cloudera cloud version CDP
• Knowledge of Cloudera data services (CDW, CDE, CDF, CAI)
• Design Native Cloud Application Architectures and optimize applications for AWS
• Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53
• Must have in depth AWS development experience (Containerization Glue, Docker, Amazon EKS, Lambda, EC2, S3, Amazon DocumentDB, PostgreSQL)
• Strong knowledge of DevOps and CI/CD pipeline (GitHub, Jenkins)
• Scripting capability and the ability to develop AWS environments as code
• Hands-on AWS experience with at least 1 implementation (preferred in an Enterprise scale environment)
• Experience with core AWS platform architecture, including areas such as: Organizations, Account Design, VPC, Subnet, segmentation strategies.
• Environment and application automation
• CloudFormation and third-party automation approach/strategy
• AWS Cost Management and Optimization

Cloudera Admin:

• Design the cluster, deciding on the number of nodes based on the estimated amount of data the cluster is going to serve.
• Deployment and upgrade of Cloudera Hadoop distribution.
• Migrating the cluster to new Cloudera cloud platform
• Configuring and tuning Cloudera services for performance and security.
• An understanding of all the Hadoop daemons along with their roles and responsibilities in the cluster.
• Should be able to troubleshoot issues in Cloudera services and fix those.
• Adding and removing nodes in the cluster.
• Rebalancing nodes in the cluster.
• Employ security using an authentication and authorization system such as Kerberos.
• Knowledge of changes required for migrating to Cloudera cloud version CDP
• Knowledge of Cloudera data services ( CDF , Atlas, CML cloudera machine learning)

AWS infra / services :

• Architecting/operating innovative solutions built on AWS
• Design and Implement AWS architectures and environments
• Play a key role in migrating and transforming legacy solutions to the cloud
• Design Native Cloud Application Architectures and optimize applications for AWS
• Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53
• 'Hands-on keyboard' AWS implementation experience across a broad range of AWS services.
• Must have in depth AWS development experience (Containerization Glue, Docker, Amazon EKS, Lambda, EC2, S3, Amazon DocumentDB, PostgreSQL)
• Strong knowledge of DevOps and CI/CD pipeline (GitHub, Jenkins)
• Scripting capability and the ability to develop AWS environments as code
• Hands-on AWS experience with at least 1 implementation (preferred in an Enterprise scale environment)
• Experience with core AWS platform architecture, including areas such as: Organizations, Account Design, VPC, Subnet, segmentation strategies.
• Backup and Disaster Recovery approach and design
• Environment and application automation
• CloudFormation and third-party automation approach/strategy
• AWS Cost Management and Optimization
• Experienced in developing Web Services with Python, JSON
• Familiar with Encryption, Logging, and Privacy/Security Protocols (e.g. TLS 1.2, ELK stack)
• Good knowledge of REST/SOAP/JSON web service API implementation.
• Presentation skills with a high degree of comfort speaking with executives, IT Management, and developers
• Demonstrated ability to adapt to new technologies and learn quickly

Job requirements

• Extensive experience with Cloudera products like Cloudera distribution Hadoop (CDH) and Cloudera cloud data platform (CDP)
• Extensive experience in designing, provisioning, deploying and configurtion of the Cloudera cluster based on customer's need
• Extensive experience in AWS service EC2, VPC, ELB, S3, RDS, Lambda, Route 53 etc and should be able to design and deploy cloudera cluster on AWS cloud
• Strong knowledge of data processing using Cloudera services
• Strong knowledge of scurity implementation using Sentry, Ranger etc
• Strong in troubleshooting and fixing issues in intergartion of Cloudera services along with AWS services
• Good in understanding of performance tuning of the cloudera services like virtual warehouses and Spark for best user experience
• Python /Unix scripting knowledge

Skills Required