There isn’t a file deletion option in the CON explorer. Strictly speaking, all the files that make up a CON are required, so you shouldn’t ever want to delete anything, album art included. Obviously the game can survive without album art, but we didn’t anticipate people actually wanting to do that on purpose.
The easiest workaround I can suggest is to replace the album art with a file containing the default art, or anything else you’d like to show instead.
You are about to leave Rhythm Gaming World and access the external songs DB. While no song or file sold for commercially available videogames is hosted there and only fan generated content is allowed, Rhythm Gaming World has no direct control over the database and no files are hosted here. Alright, we had to made you aware of this, now you can visit the database!