Cross-Platform Music Synchronization: A Monthly Automated Pipeline

The “Monthly Music Sync” project is designed with the goal of synchronizing a user's liked songs between YouTube Music and Spotify platforms using Python and relevant APIs. The ELT pipeline is built using Python, AWS S3, YouTube Data API, Spotify API, and integrates with third-party libraries such as spotipy and youtube_dl. The project retrieves music data from YouTube Music and Spotify using their respective APIs and processes this data by extracting metadata and searching for corresponding tracks on Spotify. The processed data, specifically YouTube video IDs, are stored in an AWS S3 bucket for tracking synchronized songs.
The pipeline is automated to run monthly via Apache Airflow, ensuring periodic synchronization of liked songs.
Future enhancements include further integration with Apache Airflow for automation, development of a React App for a user-friendly interface, and enhanced error handling mechanisms.

python/libraries: spotipy, youtube_dl, Pandas, plotly.
tech/tools: Python, AWS S3, YouTube Data API, Spotify API, Apache Airflow, React (for future enhancements)



Spotify User's "Liked Songs" Data exploration & visualization

With the aim of retrieving the user's music data via Python's REST API and staging it to the Postgres server, the ELT pipeline has been designed using a combination of Python, Postgres database, Dagster for automation, docker for containerization, and Power BI for visualization. The data is transformed in a number of ways using pandas to produce useful insights that can be used to build PowerBI dashboards and visualizations. With Dagster, the pipeline is automated to run every month. The project's ultimate objective is to discover the user's monthly music preferences by analyzing particular characteristics, such as the recently released "audio-features" by Spotify.

python/libraries: Pandas, plotly, urllib, pandasql, request.
tech/tools: Dagster, Docker containers, Power BI



A responsive "Travel website" clone developed using the "mobile-first" design approach

Developed using web development essentials such as HTML, PostCSS and JavaScript (Babel & Webpack) and Node.js.
The website was developed as per the "mobile-first" design approach such that the devices downloads only the data that is appropriate for the current device, thereby providing faster loading/reloading time.



A WordPress powered interactive Univeristy website

An interactive “University” website clone developed using using PHP, JavaScript, WordPress theming & the WordPress REST API.

Created “like” feature, “Live/on the fly” search functionality, WordPress custom field Post types using JavaScript and JQuery which was then connected to the backend using JSON and WordPress Rest APIs and created raw JSON data.
libraries: Gulp, JQuery, Bootstrap, WordPress RestAPI, Font Awesome
tech/tools: HTML, SASS, JavaScript, PHP, WordPress, Git


About me.

Data Engineer experienced in working across different stages of data pipeline, including acquisition, integration, ODS and real-time data marts. Adept in working quickly and efficiently in close collaboration with analytic, engineering, and other stake holders.

Currently I'm seeking to continue my career with a vibrant company that will allow me to be part of a highly collaborative team, where I can get an opportunity to develop my technical skills and work on projects meeting challenging client requirements.

Here is my downloadable resume.




  • MS SQL Server 2016/2019
  • ETL Pipelines
  • Web Development
  • Python
  • JavaScript
  • SSIS/SSRS/SSAS
  • Visual Studio 2016/2016
  • PostgreSQL
  • SQL/T-SQL
  • Git
  • HTML/CSS
  • Airflow
  • Docker

Elements

Text

This isboldand this isstrong. This isitalicand this isemphasized. This issuperscripttext and this issubscripttext. This isunderlinedand this is code:for (;;) { ... }. Finally,this is a link.

Heading Level 2

Heading Level 3

Heading Level 4

Heading Level 5
Heading Level 6

Blockquote

Fringilla nisl. Donec accumsan interdum nisi, quis tincidunt felis sagittis eget tempus euismod. Vestibulum ante ipsum primis in faucibus vestibulum. Blandit adipiscing eu felis iaculis volutpat ac adipiscing accumsan faucibus. Vestibulum ante ipsum primis in faucibus lorem ipsum dolor sit amet nullam adipiscing eu felis.

Preformatted

i = 0;
while (!deck.isInOrder()) {
    print 'Iteration ' + i;
    deck.shuffle();
    i++;
}

print 'It took ' + i + ' iterations to sort the deck.';

Lists

Unordered

  • Dolor pulvinar etiam.
  • Sagittis adipiscing.
  • Felis enim feugiat.

Alternate

  • Dolor pulvinar etiam.
  • Sagittis adipiscing.
  • Felis enim feugiat.

Ordered

  1. Dolor pulvinar etiam.
  2. Etiam vel felis viverra.
  3. Felis enim feugiat.
  4. Dolor pulvinar etiam.
  5. Etiam vel felis lorem.
  6. Felis enim et feugiat.

Actions

Table

Default

Name Description Price
Item One Ante turpis integer aliquet porttitor. 29.99
Item Two Vis ac commodo adipiscing arcu aliquet. 19.99
Item Three Morbi faucibus arcu accumsan lorem. 29.99
Item Four Vitae integer tempus condimentum. 19.99
Item Five Ante turpis integer aliquet porttitor. 29.99
100.00

Alternate

Name Description Price
Item One Ante turpis integer aliquet porttitor. 29.99
Item Two Vis ac commodo adipiscing arcu aliquet. 19.99
Item Three Morbi faucibus arcu accumsan lorem. 29.99
Item Four Vitae integer tempus condimentum. 19.99
Item Five Ante turpis integer aliquet porttitor. 29.99
100.00

Buttons

  • Disabled
  • Disabled

Form