Excellent understanding /knowledge on HADOOP(Gen-1 and Gen-2) and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Resource Manager (YARN), Node Manager and Aplication Master. 

Expert in understanding the data and designing/implementing the enterprise platforms like Hadoop data lake and huge Data warehouses.

Have over 2 years of experience as Hadoop Architect with very good exposure on Hadoop Technologies like HDFS, YARN, MapReduce, Sqoop, Flume, HBase, Hive, Presto, Oozie and Spark.

Reliance Techservices, Oct 2015 - Present

• Understanding the data nature from different sources and designing the injection processes for HDFS
• Developed Spark scripts by using PySpark shell commands as per the requirement.
  
• Developed pig scripts to transform the data into structured format and automated through Oozie coordinators.
  
• Collected data using SPARK STREAMING from Source in near-real-time and performs necessary Transformations and Aggregation on the fly to build the common learner data model and persists the data in HDFS.
  
• Explored the usage of SPARK for improving the performance and optimization of the existing algorithms in HADOOP using SPARK CONTRXT, SPARK SQL and SPARK YARN.
  
• Worked with SPARK STREAMING to ingest data into Spark Engine. 
• Developed Spark Code using PySpark and SPARK-SQL/STREAMING for faster testing and processing of data.
• Involved in converting Hive/SQL queries into Spark Transformations using SPARK RDDs and SCALA.
• Worked on the SPARK SQL and Spark Streaming modules of SPARK and used Scala and PySpark to write code for all Spark use cases.
• Extensive experience in using THE MOM WITH ACTIVE MQ, APACHE STORM, APACHE SPARK & KAFKA MAVEN AND ZOOKEEPER. 
  

Accenture,  July 2013 - Sep 2015

• Worked extensively in creating MAPREDUCE jobs to power data for search and aggregation. Designed a data warehouse using HIVE.
  
• Importing and exporting data into HDFS and HIVE using SQOOP.
  
• Used Bash Shell Scripting, SQOOP, AVRO, Hive, Pig, Java, Map Reduce daily to develop ETL, batch processing, and data storage functionality.
  
• Used PIG to do data transformations, event joins and some PRE-AGGREGATIONS before storing the data on the HDFS.
  
• Exploited HADOOP MYSQL-Connector to store Map Reduce results in RDBMS.
  
• Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
  
• Worked on loading all tables from the reference source database schema through SQOOP. 
•  Worked on designed, coded and configured server side J2EE components like JSP, AWS and JAVA.
  
• Collected data from different databases (i.e. Oracle, MYSQL) to HADOOP.
  
• Used OOZIE and Z00KEEPER for workflow scheduling and monitoring.
  
• Worked on Designing and Developing ETL Workflows using Java for processing data in HDFS/HBASE using OOZIE.
  
• Experienced in managing and reviewing HADOOP log files.
  
• Involved in loading and transforming large sets of structured, semi structured and unstructured data from relational databases into HDFS using SQOOP imports.
  
• Working on extracting files from MYSQL through SQOOP and placed in HDFS and processed.
  
• Supported Map Reduce Programs those running on the cluster.
  
• Cluster coordination services through ZOOKEEPER.
  
• Involved in loading data from UNIX file system to HDFS.
  
• Created several Hive tables, loaded with data and wrote Hive Queries in order to run internally in Map Reduce.
  
• Developed Simple to complex Map Reduce Jobs using HIVE and PIG.
  
• Involved in Analyzing system failures, identifying root causes, and recommended course of actions.
  
• Used MAPREDUCE to process the large data sets over a cluster of computers using parallel processing. 

Adroitent, May 2011 - June 2013

SQL/JAVA DEVELOPER

Responsibilities:

• Developed physical data models and created DDL scripts to create database schema and database object.
  
• Wrote user requirement documents based on functional specification.
  
• Created new tables, written stored procedures, triggers for Application Developers and some user defined functions. Created SQL scripts for tuning and scheduling.
  
• Involved in performing data conversions from flat files into a normalized database structure.
  
• Developed source to target specifications for Data Transformation Services.
  
• Developed functions, views and triggers for automation.
  
• Extensively used Joins and sub-Queries to simplify complex queries involving multiple tables and also optimized the procedures and triggers to be used in production.
  
• Performance Tuning in SQL Server 2000 using SQL Profiler and Data Loading.
  
• Installing SQL Server Client side utilities and tools for all the front-end developers/programmers.
  
•  Involved in performance tuning to optimize SQL queries using query analyzer.
  
• Created indexes, Constraints and rules on database objects for optimization.
  
• Creation/ Maintenance of Indexes for various fast and efficient reporting processes.
  
• Monitored the Database Growth and Space Requirement. Handling Users/Logins/User Rights.
  
• Managing historical data from various heterogeneous data sources (i.e. Excel, Access).
  
• Designed the front-end user interface using JSP, Servlets, jQuery, AJAX, JavaScript, CSS.
  
• Strong understanding of JavaScript, its quirks, and workarounds.
  
• Proficient understanding of cross-browser compatibility issues and ways to work around such issues.
  
• Worked with a variety of issues involving multithreading, server connectivity and user interface.
  
• Design of application components using Java Collections and providing concurrent database access using multithreading.
  

B.Tech CSE 2005 - 2009 from Nova College of Engineering

INTERMEDIATE MPC 2003 -05 from Sri Chathanya Educational Institute

SSC 2002 - 03 from St.joseph High School