Sensory Music (Under Active Development)
Since 2021, I've been constantly learning and improving my knowledge and skills with machine learning.
This is one of the first major projects that stems from my desire to learn more about this topic and
engage in it in a greater way. Furthermore, it's a project that would potentially benefit me and other
musicians after its competition.
The Process
I learned as much as I could on machine learning, which includes computer vision. Whether those resources
were YouTube videos or textbooks, I put a lot of effort into understanding the fundamental concepts (including
the maths behind it (convolutions, matrix multiplications, etc.))
After learning the ins and outs of machine learning (or at least to the best of my abilities), I learned the
PyTorch machine learning library. Through it, I am able to create and train my own models (or use other models
that other people have trained) for my own programs and projects.
High-Level Overview of the Functionality
Sensory Music, which is a computer vision object detection project that is able to detect symbols and other details
on a music sheet and convert it into a digital copy along with a MIDI file to play the music without an instrument.
I was responsible for the machine learning models to be used for computer vision and my friend was responsible for
implementing the completely trained computer vision code into an executable file for Android and a wider range of
devices in the future.
Skills and Tools Used
Google Colab (Free Version)
Machine Learning Libraries
PyTorch
YOLO-V5
Knowledge of Machine Learning