Scenario Testing

[K K Rossokolski]

In the previous thread, I got stuck into the H2H process for approving scenarios. I emphasise That this is NOT a go at those hardy souls who have tried over the years to make it work. Bloody hard to plough the paddock if the mule is dead. I am surprised they managed to turn any sod at all.

Pertinent points
1. By my count, three CS scens have passed testing since 2007.
2. My observation of the site over a long period clearly shows the top designers do not use the H2H system.
3. The system demands an absurdly high level of results. I re-emphasise we are testing games, not wonder drugs or rocket ships to land on the Sun.
4. The system rewards testers in a niggardly way. People put a lot of effort into testing FOR THE CLUB....they should be well rewarded.
5. No provision exists for comment as opposed to testing.
6. The program has limited capacity.

No point in gobbing on without ideas to fix it.
These are mine: :stir: :soap:
1. Scenarios are acceptable with three scores of 6.66 or greater, two of 7 or greater, or one of 7.5 or greater.
2. Any test scen scoring 3.33 or less is withdrawn and sent back to the designer's drawing board
3. Any scen in the program not being tested after three months is withdrawn.
4. The program be open to increasing test slots to meet demand.
5. Members may make a comments only evaluation, which could point out obvious errors or deficiencies.
Rewards
1. Any scen under official test is a ladder game, and win/loss/draw points are awarded as per the SM scale.
2. Both test players are awarded, for completed games, bonus points as follows;
SM1...20
SM2...35
SM3...55
SM4...80
SM5...110
Size is important, so big turn games give additional bonuses as follows
30-39 turns ....25
40-49 turns.....50
50-74 turns.....100
75 or more.......200

Any member making constructive comments gets 25 bonus points. These need to be a bit more profound than , say, "Rubbish" "Great". OC H2H Coy will decide if comments are worth a bonus.

[Chuck10mtn]

Hey KK,
Thanks for the good ideas. About all the games that I've played for the last 6-10 months have all been test games. I hope they don't put me on the no game turned in list.
I find it better to play a game that might be lousy but has never been played before, that way both sides don't know what to expect.
I hope your idea of bigger pts for testing will pay-off and get more people to test the games. I would also encourge the testers to send what you think about games as you play them. I have a bad habit of forgeting things after a time. Just some thoughts from somebody who tests game for people.

Chuck

[Larry Reese]

I back this wholeheartedly. While I've tested only a few of my scenarios they have generally been well received and folks have had a lot of fun. None of these (to my recollection) have gone through H2H, though I would have liked that. Now, for example, I have a monster in test with three players who have offered really valuable input and for at least on set of player (Jason and I) the scenario has turned out to be a real seat of the pants thrill.

LR

[Mr Grumpy]

In the previous thread, I got stuck into the H2H process for approving scenarios. I emphasise That this is NOT a go at those hardy souls who have tried over the years to make it work. Bloody hard to plough the paddock if the mule is dead. I am surprised they managed to turn any sod at all.

Thank you for appreciating the hours of work i put in keeping the H2H area up to date and answering the many emails i receive asking for help and advice. :)

Pertinent points
1. By my count, three CS scens have passed testing since 2007.

Indeed, but this is a CS issue, both CM and OpS areas have had many more scenarios approved over this period.

2. My observation of the site over a long period clearly shows the top designers do not use the H2H system.

You may be correct, i have no idea who the top CS designers are, but we have top designers using the H2H in the other areas.

3. The system demands an absurdly high level of results. I re-emphasise we are testing games, not wonder drugs or rocket ships to land on the Sun.

Well the approval score has not been a issue on the other ladders and i notice mwest recently approved two scenarios in quick time at the CS area without reporting any issues, then again he was prepared to put some leg work in posting for testers and following up feedback, just goes to show how easy the system can work if you put your mind to it.

4. The system rewards testers in a niggardly way. People put a lot of effort into testing FOR THE CLUB....they should be well rewarded.

Well how much testers should be rewarded is a subject that has been discussed many times, testing is supposed to be done for the benefit of the whole community and not for advancement up the ladders, so while the club rewards testers a bonus, it will never be enough for some people.

5. No provision exists for comment as opposed to testing.

Well other ladders use the H2H sub forum for commenting on a scenario without testing it.

6. The program has limited capacity.

I have had no complaints about there not being enough room on the testing page, i have had designers complain they cannot upload new scenarios, when i check the test page i normally find they have already uploaded their quota and despite not having got their existing scenarios through testing they want to upload more, the answer to this is have me remove the 'deadwood' scenarios or get the existing ones tested to make room for the new material.

No point in gobbing on without ideas to fix it.
These are mine: :stir: :soap:
1. Scenarios are acceptable with three scores of 6.66 or greater, two of 7 or greater, or one of 7.5 or greater.

The 8 or more score works just fine, this fact is based on results all over the H2H not just CS.

2. Any test scen scoring 3.33 or less is withdrawn and sent back to the designer's drawing board

Well there has been occasions when for strange reasons testers give good scenarios very low scores, this feature is not needed as a scenario with constant low scores will be removed when i do a spring clean of the H2H areas.

3. Any scen in the program not being tested after three months is withdrawn.

That's a tough one as some very good scenarios go through 'rounds' of testing and there can be periods that a scenario is on the test page while the scenario is updated, under this rule Big Bully would be toast now. I do regular sweeps to clear out deadwood scenarios so they do not become a issue, soon this will be done automatically.

4. The program be open to increasing test slots to meet demand.

No problems reported on this anywhere in the H2H.

5. Members may make a comments only evaluation, which could point out obvious errors or deficiencies.

Not really sure what you mean?

Any member making constructive comments gets 25 bonus points. These need to be a bit more profound than , say, "Rubbish" "Great". OC H2H Coy will decide if comments are worth a bonus.

Any tester who only puts 'Rubbish' or 'Great' in a report will not receive any testing points.

In making these comments bear in mind i oversee the whole of the H2H and not just CS, we are having good success on other ladders along the lines that mwest has done at CS, the system will not test scenarios automatically it requires some legwork from the designer, it is very noticeable that when i see a designer posting for testers, responding quickly to feedback and talking to myself about the best way to make the H2H area work for them, they achieve excellent results, so i base my comments on my experience that the H2H can and does work if you really want it to.

I am the first to acknowledge that the present H2H system is not perfect in many ways and myself along with the site webmaster Raz have given the H2H a complete revamp based on issues raised by designers/testers from all over the H2H, unfortunately this new H2H cannot be launched until we move to the new site, until then i am afraid we will have to make the best of the old version.

Thank you for your feedback, i always welcome players opinions and when i think they raise a good point i make a note of them and make a change, i incorporated 30 such changes into the new H2H, so at least i can not be accused of not listening. :)

[Huib Versloot]

I think the H2h area is not that bad, although the requirements to pass are silly. Now when I test and I think a scn is worthy to pass, I give it 8 on all fronts, just to show I think it is ready to pass. I may have rated the same scn lower if it was just a ladder game.

One of the reasons I don't use H2H is because once scns pass, they're stored in a different table than other scns in the regular depot. That table is not accessable if you want to update the scn, while the regular table is, so it is redundant in the worst possible way.

H2H can still be a good way to advertize new scns.

[Mr Grumpy]

I think the H2h area is not that bad, although the requirements to pass are silly.

OK you are of course intitled to your opinion. :)

One of the reasons I don't use H2H is because once scns pass, they're stored in a different table than other scns in the regular depot. That table is not accessable if you want to update the scn, while the regular table is, so it is redundant in the worst possible way.

H2H can still be a good way to advertize new scns.

You can update the files at the approved list, but only by contacting myself so this is not a huge issue, the original idea was that the players wanting to obtain the H2H files would use the approved scenario list (where they could be sure they were approved) rather than wade through the host of scenarios in the depot, the entry in the depot was purely to report the H2H game against and it was not envisaged that the files would also be maintained here. :chin:

[Mr. Guberman]

ummm...what is H2H for?

either a scenario swims or it sinks...

Excellent work out there and much labor which is much appreciated.

...what is H2H for?

I'll leave it at that.

Cheers...and much thanks and kudos to all.

Curt

[K K Rossokolski]

Referring to post #4

WOW!!
I spent a working lifetime in the Navy, too much of it fighting the military bureacracy, but I see now I was up against amateurs.

I am happy for my three little efforts currently in H2H to be considered "toast" as you so eloquently put it, and removed at your earliest convenience. You might find a way to recompense any testers with a bonus point or two, if the rules can allow such a thing.
I shan't bother you again.

[Mr. Guberman]

Eeeeeesy...Rod...easy...

c'mon getalong lil' dogie, c'mon....

PM me...
cheers
Curt

[Mr Grumpy]

Referring to post #4

WOW!!
I spent a working lifetime in the Navy, too much of it fighting the military bureacracy, but I see now I was up against amateurs.

I am happy for my three little efforts currently in H2H to be considered "toast" as you so eloquently put it, and removed at your earliest convenience. You might find a way to recompense any testers with a bonus point or two, if the rules can allow such a thing.
I shan't bother you again.

You should read my post more carefully, under your proposal in point 3 of your ideas to fix the H2H you stated "3. Any scen in the program not being tested after three months is withdrawn." so under the rules you want brought in your Big Bully scenario would have been removed (toast), i was just pointing that out to you, sorry you did not get the obvious point i was was making. ;)

By coincidence i recently contacted you about this scenario to try to find out if you were intending to push it on, i know you read my PM but you still have not got round to replying to it??