Songcache Modification Tool
- This topic has 33 replies, 4 voices, and was last updated 2 years, 8 months ago by
.
-
Posts
-
Updated to v0.6 alpha, mostly because I had a need to modify Track Numbers.
Here’s the changelog:
20181231 v0.6 alpha
- Version number label on UI for easy identification
- Error message cleanup
- Added support for trackNumber ("track" "num" "number" "tracknum" "tracknumber")
- Additional song attribute aliases: see SCRIPTING-HOWTO.txt
Well here’s an interesting bug…
At this point, I’m convinced that a soundcache file output by the tool is essentially “read only”. You’ll see all of your changes in RB3 (yay!) but if you add new DLC/customs afterward, it will rescan them every time.
The trick, then, is to keep a copy of the “native” songcache to add new customs to, then generate a new modified songcache with the tool to do the fancy tricks. It’s another task to complete before you get to playing, though. Ugh. But manageable if your collection doesn’t change that often.
The technical issue, I suspect (after eliminating other theories through testing), is that the songcache file is allocated with extra space to allow it to grow over time. As stands, our tools (like Nemo’s CON Creator/Explorer) will shrink down all that extra space to just the size of the cache itself. This leaves it readable, but impaired for writing/modifying. Unfortunately, the X360 library that we’re using doesn’t have an obvious way to tell it, “oh, and leave space for the special sauce”.
I’ll keep working on it for now, but there comes a point where an “intellectual exercise” isn’t worth the frustration of poking at a (mostly) black box. If anyone has had experience with adding extra STFS blocks to a file in a CON, please drop me a line.
An update to the previous bug report:
I’ve confirmed that if the rewritten cache exceeds 0xAA (170) blocks in size, it will be considered “read-only” by RB3. This threshold is documented to cause another tree level to be used in the hash table, but I’m not handling it capably (apparently). Most likely I’m not passing the right parameters to the X360 library API and it’s just assuming what it thinks are “sane” defaults.
This means that for a smaller library, your songcache would be fine (i.e. “rewritable”) and will expand when adding new customs (without rescanning), which is why I hadn’t run into this issue previously. But with a billion customs at our disposal, who has a library that small?
Will put up an update soon, including small fixes and sanity checks that arose while debugging this. At least at this point I’ve generated test files for the exceptional-size case, so hopefully a robust and verified v1.0 release won’t be far behind.
A small v0.7 update…
Apparently a few songs have album art intact in the file, but for some reason don’t display in the game. Since these are primarily from movie soundtracks, it seems logical that HMX didn’t want to license the cover art separately from the audio, so they just punted.
So now these songs are automatically fixed when writing out a new cache:
- Promised Land (Vesuvius, free)
- Charlene (Stephen and the Colberts, free)
- Accidentally in Love (Counting Crows, Lego)
- Ghostbusters (Ray Parker Jr., Lego)
It didn’t seem to be at all useful to fix these within the scripting engine itself, since this is an extremely specific case, so it’s just part of normal operations after v0.7.
(NOTE: Caches are still writing as “read only” after growing past a certain size — perhaps in the 1.4MB range. If anyone can send a knowledgeable X360 or STFS coder my way to help me out, I’d appreciate it.)
Want to know which of your songs aren’t wipe-proof?
; Print out info for non-wipeproof customs
if id contains 10746 then
println artist ", "" song "" (" shortname ")"
end
I’ll add this script to the download next time I update, but you can just paste it to a text file yourself for now.
I’m running into a problem when trying to parse a cache file I think may be corrupt.
Yeah, I know, I fed it broken bits and I’m shocked and surprised that it broke? Actually not really, but hear me out.
I’m chasing an odd bug with some of the latest entries in my fork of the Rock Band Harmonies Project. I’ve narrowed the bug down to two upgrades. If I include the upgrade for Song A only, all is well. If I include the upgrades for Song A and Song B, Song B is fine, but the game crashes when trying to play Song A. I was hoping the cache tool might shed some light on exactly what is in the cache file that might be causing the game to choke. Unfortunately, when I feed either cache file to the script, I get an “out of memory” error. More specifically, I see this in the program’s main window:
Found 628 non-empty packages
Found 573 non-empty store packages
Expecting 1487 songs. Ready? Fight!
Loaded all 1487 songs.
Loading footer...And I see this in the .NET error window:
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at CacheTools.Cache.ReadFromFile(String filename, Boolean isWiiFile)
at CacheTools.Form1.buttonLoad_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
************** Loaded Assemblies **************
mscorlib
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4110.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v4.0.30319/mscorlib.dll
CacheTools
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/Skunk/Documents/RB3/RB3%20Cache%20Tools/CacheTools.exe
System.Windows.Forms
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4110.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Windows.Forms/v4.0_4.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
System
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4110.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System/v4.0_4.0.0.0__b77a5c561934e089/System.dll
System.Drawing
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Drawing/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
System.Configuration
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Configuration/v4.0_4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
System.Core
Assembly Version: 4.0.0.0
Win32 Version: 4.8.4110.0 built by: NET48REL1LAST_B
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Core/v4.0_4.0.0.0__b77a5c561934e089/System.Core.dll
System.Xml
Assembly Version: 4.0.0.0
Win32 Version: 4.8.3761.0 built by: NET48REL1
CodeBase: file:///C:/Windows/Microsoft.Net/assembly/GAC_MSIL/System.Xml/v4.0_4.0.0.0__b77a5c561934e089/System.Xml.dll
************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.
For example:
When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.I can provide the cache files themselves if that would help. That’s assuming you’re willing to chase something that people with good cache files will likely never see. If not, that’s totally fine.
I can provide the cache files themselves if that would help. That’s assuming you’re willing to chase something that people with good cache files will likely never see. If not, that’s totally fine.
Sure, I’ll take some of that action. Get me a link to the caches in question and I’ll give it a whirl.
Updated to v0.8 today.
@FujiSkunk provided an additional working cache that helped uncover the actual problem: The Bob Marley RB3 pack (the one that contains “Kaya”) also includes a license file for additional wardrobe swag (namely “male_premium_hendrix_woodstock”). Thus far, this is the ONLY package we’ve found that has such extra license information, and the original spec I coded from was mistaken on what was contained in this chunk.
So that’s sorted out. Quickie fix, and I also reenabled the config to catch the exception and dump it to the text results area instead of halting the program altogether.
Thanks again to Fuji!
Now one might wonder: if there’s a license for “male_premium_hendrix_woodstock” in that pack, could we also fake other license entries in the cache to add other wardrobe unlocks?
UPDATE: 20200224 v0.9 alpha
– Added “import”, “export”, “macro” commands (Thanks to hentaiphd for idea and collab)
– Added “verbose load” mode: hold down Shift key before cache loads to see lots of info (SLOW-ASS ON BIG CACHES!)Two new scripts are included to fix TU4 export licenses (“Dani California” and “Black Hole Sun”) and EU DLC (“Hier Kommt Alex” and “Rock ‘n’ Roll Star”).
Drop me a line if you test these out: as of this time, their correctness is entirely theoretical!
hey, just like pksage’s song cache on his website
Sent from my Galaxy5 using Customs Creators Collective mobile app
hey, just like pksage’s song cache on his website
I don’t think I quite know what you’re referring to: maybe the RB3 Cache Exporter tool? This is quite different than that.
Hey so I’ve recently tried to use this tool once I got a hold of CON files for Hier Kommt Alex and Rock ‘n’ Roll Star, but they still do not show up in my library even after running the EU fix multiple times. I’ve made sure that I’m on TU4 and not TU5. I’ve also made sure to delete the original cache and replace it with the new one from my flash drive. No luck. Any help?
Hey so I’ve recently tried to use this tool once I got a hold of CON files for Hier Kommt Alex and Rock ‘n’ Roll Star, but they still do not show up in my library even after running the EU fix multiple times. […]
How big is your songcache file to begin with? There is the known issue where after it reaches a certain size it’s considered “read-only”, and the game will force a rebuild every time. While it will retain the modified data for most songs, some of those title/region exclusion issues might be enforced on the rebuild.
So in order to check, reduce the number of CONs your console can see, making sure that the European region titles are available (maybe on a separate small USB stick), and then build a fresh songcache. If the EU titles are then available, this is the bug you’re seeing.
And again apologies for this bug: it’s the one thing that prevents this utility from being truly useful, and I’ve yet to get in touch with an “expert” on building and hashing CON files to help me figure out what the issue is (SOMEONE PLEASE FIND ME AN EXPERT?!). My personal solution is to build my first cache with ONLY the CONs that will be modified, so that it’s under the size limit for modification, then let the console build the rest after adding the rest of the unmodified songs.
(It’s either that, or I should change this utility to trawl through CONs and rip them apart to modify their internal .dta assets…)
- You must be logged in to reply to this topic.