[Gigakoops]

This was discussed at length…

 

How is it any better if the song gets put on playtesting, and that instrument is not playtested? The song still goes out. You’re assuming that because it gets put in playtest, that every instrument gets playtested on every difficulty to the satisfaction of every potential player. That has not been supported by the past few months.

 

Established authors have proven their ability to chart songs accurately. If they happen to put out a song with a mistake, and someone catches it, then they can use the Bug Tracker and it will be fixed.

 

I realize that playtesting has been rather slow these past few weeks. I’m just saying, having the chance for it to be playtested is better than not letting it be playtested outside of the admins. Sorry if I come off as a bit sour over this, but I do prefer releasing things right the first time instead of releasing something that may have a bug in it. I kinda get that from authoring for the RBN, where you have to get it right or else.

[TrojanNemo]

You’re missing the point, again. Just because it gets put into playtesting, does not guarantee any potential bugs will be addressed, or even the song will get playtested. Perhaps once we have established a clear playtesting structure with assigned playtesting, that’d be different.

 

The second point you need to grasp, is that we are NOT RBN. We can issue fixes. RBN can’t. That’s the difference. You assume that because songs don’t go in playtesting, they will have bugs. Our evidence for the authors who are considered Established Authors suggests otherwise. Songs came out from RBN that were not nearly 100% quality. Established Authors are those who release songs with exceptional quality. While it may not be 100%, the past few months have shown that almost nobody has found problems in their songs. And once again, if they do find a problem, we can fix it!

 

The alternative is turning our process into the slow monster that was RBN, which is its main undoing. We will not do that.

 

And when we say it can go without playtesting, it means without playtesting. We (the admins) don’t playtest songs outside of what’s on playtesting. Individual authors are encouraged to playtest their own material locally, which is what most of us do.

 

P.S. You will notice I opted not to include myself in the list of “Established Authors”

[Nightmare Lyra]

Uuuh, how is just having the possibility of public playtests along with admin tests going to make “everything into a bloathed RBN monster where nothing gets done”? Also no testing at all is never good, you should always have a second person look over the chart, or else you might do like me and release a custom with a broken pitch on vocals and nothing is done before someone sends a message about it, if they ever do.

[Farottone]

As someone who takes realistic charts seriously, it kinda scares me that letting charts skip public playtest is an option. I’d prefer we release things at as high a quality as possible instead of “good enough”.

 

The discussion is very articulate and I think “I prefer things to be perfect instead of being just good” is overwhelmingly reductive. ” src=”/wp-content/uploads/invision_emoticons/default_SA_smile.gif”> Of course we all would prefer things to be perfect, it’s the price of perfection that needs to be taken into account.

 

I think the discussion we had can’t possibly be summarized here: it was too long and with too many aspects of the issue to be reported briefly. We can of course discuss specific issues (like the fact that we have released not perfect songs that breezed past playtesting) but I want to touch on one issue you raised: “good enough”.

 

If you take a look at any blog, forum, reddit page, whatever you will find that people are considering C3 releases as the golden standard. Now, no song in the top 10 of the most downloaded C3 songs has ever entered playtesting. Less than a handful of the top 50 most downloaded songs has ever entered playtesting. That means that people consider the work we all do as a reference for customs work based on songs never pushed through playtesting. So, there’s not much to worry about that. What you call or consider “good enough” is actually the golden standard for people playing the songs. If we want to discuss how we’ve come to the conclusion of not requiring playtesting for all songs (which is not new, by the way, it’s how we started) it’s important we start from the fact that people consider our, meaning of all C3 authors, work to be very good.

[Farottone]

The discussion has been tense and long, as you can imagine (not specifically on this issue though) because we care so much about this. keep asking things. ” src=”/wp-content/uploads/invision_emoticons/default_SA_biggrin.gif”> Carry on, keep the questions coming. ” src=”/wp-content/uploads/invision_emoticons/default_SA_biggrin.gif”>

[Gigakoops]

All I’m saying is, it can’t hurt to put all the songs in public playtest. Even if no one tests it, if they’re scheduled for release, and you admin folk feel it’s good enough for release, then I say go on ahead. But I feel having the option for it to be tested before release by others is a good thing. I’m not saying we should bar them from release until they’ve been tested on everything, just saying all songs should be treated equally.

[pksage]

I think we can comfortably say — based on the evidence, if nothing else — that most people on the admin team have different opinions about song quality. This was a big part of our discussion.

 

I used to be a stickler for perfection, but as farottone has mentioned, that desire is simply not reflected in our demographic. Rock Band is basically dead, and we’re here partying in the barren ashes. I think the way to keep people interested is to make more customs, even if they’re at 80% quality, rather than releasing just a few at 100% quality. If we were talking about 40% or 50% quality, it’d be a different matter, but the “good enough” threshold is…well…good enough.

 

Again, not all of us feel this way, and that divide is how we came to the current compromise. It’s not perfect, but a good compromise means that no one is entirely happy.

 

By the way, we do strongly stress that established authors get at least one more pair of eyes on every song they release, and we will not hesitate to strip “established author” status from someone who releases several songs with big bugs. The established authors on the admin team are not exempt from this rule.

 

post-Gigakoops-edit: The issue of transparency was also discussed at length. The counter-arguments are that 1) this way even other authors can have some of the releases be a surprise, which is nice, and 2) sometimes the cost of high volume is not being able to respond to playtesting at all. If a playtest thread gets bogged down in minor fixes, it gets pointlessly delayed and no one wins. Think of it like PR in RBN; you don’t fail someone for minor issues. Our threshold for “minor issues” is just a little different.

[Farottone]

All I’m saying is, it can’t hurt to put all the songs in public playtest.

 

It actually can. ” src=”/wp-content/uploads/invision_emoticons/default_SA_smile.gif”>

 

Even if no one tests it, if they’re scheduled for release, and you admin folk feel it’s good enough for release, then I say go on ahead. But I feel having the option for it to be tested before release by others is a good thing. I’m not saying we should bar them from release until they’ve been tested on everything, just saying all songs should be treated equally.

 

Full disclosure, I’m on the “playtesting optional and not mandatory” camp.

 

It serves no purpose to have authors put songs through testing and then not testing them. None whatsoever. If the song is not tested it’s as if it was never put through testing. And no, not all songs are created equal. I’m not gonna name names, but songs from author xxx, yyy or zzz I don’t care at all to see them before release, I know the author has spent countless hours testing and polishing them and has a work ethic that ensures the song is as good as it can reasonably be. Because “skipping playtesting” is not like skipping school: it’s not something naughty, it’s something comparatively unnecessary in the opinion of part of your group for selected authors who do spend a lot of time caring for their songs.

 

Now, why do I say it can actually hurt having songs in playtesting? Because the time it takes to fix minor issues that only a playtester will notice is better spent authoring new songs that all our public can enjoy. More time for fixing stuff means less time to author. This is MY point of view, mind you. There are others in the group who will chime in and disagree on this point, and they will offer their reasons. I believe both our points of view are valid but I evidently support mine. The fact that we disagree doesn’t mean we undermine what we say, it means our minds are still healthy. ” src=”/wp-content/uploads/invision_emoticons/default_SA_smile.gif”>

 

One thing though I don’t believe to be subjective: playtesting has to be structured and equal for all songs. If it isn’t, it doesn’t serve the purpose of letting us release songs of constant quality. If it isn’t, in MY opinion, it’s far better to have an accurate “sanity check” (as pksage and espher cleverly refer to that) for songs from non established authors. Because you may have not noticed that, but the songs that make it to Friday sometimes have required us to work until 3 in the morning to tweak things around, and it’s something none of us can do on a long term basis, hence the long discussion.

[Gigakoops]

I personally feel that time fixing songs up is equally as important as time spent authoring new material. But then again, I seem to be in the minority here. I just thought I’d bring up something I happen to disagree with. But I see you guys are stone set on keeping things this way for now. I’ll put up with it, but that doesn’t mean I have to like it ” src=”/wp-content/uploads/invision_emoticons/default_SA_cheeky.001.gif” />

 

I understand where you guys come from when you say that the average audience wants more songs, and that trying to prime up each one to perfection would bog that down. I just personally prefer quality over quantity, and that doesn’t seem to be the average mindset (at least according to that poll we had on the forums discussing what people prefer in a custom).

[Farottone]

I just personally prefer quality over quantity, and that doesn’t seem to be the average mindset (at least according to that poll we had on the forums discussing what people prefer in a custom).

 

But this is a different discussion altogether. ” src=”/wp-content/uploads/invision_emoticons/default_SA_smile.gif”> Why are you linking playtesting with quality/perfection? You even said “Even if no one tests it, if they’re scheduled for release, and you admin folk feel it’s good enough for release, then I say go on ahead”. A song noone tests is as good as a song never put through playtesting. Why do you think a song being put through playtesting is perfect or it’s ineherently better than a song not put through playtesting?

 

Don’t get me wrong, I respect the “quality over quantity” point of view very much. I just don’t see it here because the playtesting we have is not assurance of quality. You say you’re “just saying all songs should be treated equally”, but then again they are not. Because we can’t pay playtesters and because we don’t have a business-like structure, all songs through playtest are treated not equally. Some are tested on all instruments, some are not tested at all.

[espher]

Full disclosure: I’m in the “every song should enter testing, even if it doesn’t require corrections” camp. Farottone and I are on opposite sides of the fence. I’m in the camp of far more structure and mandatory QC, as the original proposal I put forward for feedback would indicate. ” src=”/wp-content/uploads/invision_emoticons/default_SA_cheeky.001.gif”>

 

Trust me when I say we discussed this issue at great, occasionally heated length, and the system pksage put forward is the system we settled on for the current exercise. It’s a good compromise between our viewpoints, and it still gives authors the option to put their songs in test if they feel they need it, while authors who perhaps have their own internal/peer testing and have established a precedent for releasing quality charts are given the benefit of the doubt in good faith. Note that if someone abuses that and puts shoddy charts out even once I will aggressively petition to get that “Established” tag rolled right on back. ” src=”/wp-content/uploads/invision_emoticons/default_SA_wink.gif”>

 

If I ever get back around to authoring every single one of my songs will be in playtest for several weeks before release. ” src=”/wp-content/uploads/invision_emoticons/default_SA_smile.gif”>

[ws54]

I’ve said this previously, “all songs should go through the same play-testing procedure”.

 

It is helpful to find someone passionate about the songs. Since releases cover such a wide range of tastes it seems logical to me to try to gather testers from a little bit larger pool. Farrotone and I have similar tastes in music and it seems like an obvious choice to have me review at least a lot of his Bass and Guitar charts. I for one would have given all of the Queen songs a thorough test. I was planning on authoring Love of My Life myself at some point and I might have duplicated effort and might have wasted my time. But I suspected that song was on the his Queen list this time. Can’t wait to play it. I hope it is good ” src=”/wp-content/uploads/invision_emoticons/default_SA_biggrin.gif” />

 

Re: Established Authors, I don’t think my drum charts will ever reach that level. They will probably never be good enough the first time (my next song does not have drums.)

[Farottone]

I for one would have given all of the Queen songs a thorough test.

 

Now you’ve got me curious though. ” src=”/wp-content/uploads/invision_emoticons/default_SA_biggrin.gif”> Why didn’t you test the song I put through playtesting?

[espher]

Since releases cover such a wide range of tastes it seems logical to me to try to gather testers from a little bit larger pool.

 

The tricky thing is finding the bodies to do this. For the initial round of (unstructured) playtesting recruitment, about 95% of the people picked up listed at least two of rock, classic rock, hard rock, prog rock, or metal as their genres of choice, and certain genres (including my tracks) got little or no testing. Perhaps another round would go better as awareness/volume picks up, especially if we had a more rigid testing process, but people generally don’t test things they don’t care about it. ” src=”/wp-content/uploads/invision_emoticons/default_SA_cheeky.001.gif”>

[pksage]

On that topic (and the topic of the OP), I’m torn about the idea of assigned playtesting. I suspect many people would see that as TOO structured, or even like a job, and we don’t want C3 to turn into that. On the other hand, though, it would make sure that every song gets playtests when it needs them — there’s several songs in the playtest forum that are just kind of languishing at the moment.

 

Thoughts? How would you guys react if we said “all authors must do X playtest(s) of others’ songs for each of their own songs, and they have to be on specific songs we assign”? Or the same idea, but only with dedicated playtesters and not authors (assuming we can actually find any)?

[Author]

Posts