“Final Bar Line Theatrhythm” is a rhythm game by Square Enix featuring numerous songs from their Final Fantasy series. I wanted to try breaking the game using code.
First, I connected to my PlayStation 5 via remote play to be able to use a computer to control the games via Python. To do so, I used the win32api to mimic keyboard inputs. But not before I screenshot the play area and used the CV2 library to recognise the music note elements within the game. Each image also has a set of those keyboard inputs mapped, so everything it sees a red dot will mimic pressing the cross button. This project was a great time that let me play with new libraries, but the project was very heavily limited by my internet connection speeds meaning the precision of reacting to the screen was very tough to keep in time with the music.