A maths question, chance vs skill.

[agedhorse]

3 hours ago, Downunderwonder said:

The test was set to challenge bass players to pick which was tubes and which was solid state emulating tubes. I think it's pretty straightforward! It wasn't asking which was preferred, only which was which.

 

The hypothesis being tested was ''tube emulation is close but no cigar, bass players know this and can tell the difference''. Further, ''here is our chance to prove it''.

 

Since it was such a simple test the binary choice enforces an allowance for a coin toss answer affecting the results so strongly.

 

On further reflection, the responses that were wrong had to be from folks that ''had an inkling'' but were wrong. I don't think anyone would have given an actual coin toss answer but even if they did they are also ''don't really knows'' split 50/50. So mathematically we know the ''can't tell'' group comes to 50% of the total being 2x 25% who answered wrong.

 

The key to it is I think it is just as likely to be picking correctly as incorrectly if you don't have the capability to properly discern A from B.

 

One of these days I might have a crack myself. I would make it a lot harder by including an unknown number of tube devices to pick out of the samples.

 

If I do I will need a real statistician to interpret the results back to the likelihood any one basschatter can correctly identify a recording is from a solid state or tube amp.

What were the numbers when comparing tube to tube and solid state to solid state... ie. controls? Were they statistically 50-50 or were they skewed as well?

[Downunderwonder]

50 minutes ago, agedhorse said:

What were the numbers when comparing tube to tube and solid state to solid state... ie. controls? Were they statistically 50-50 or were they skewed as well?

I am not sure what you mean. I thought I had described it well enough already but another go...

 

Relying on memory, the samples were set up as level and eq matched as could be. Aim was to test detection of signature SS modeling against real thing.

 

Sets of rerecorded tracks. Each set had iirc three levels of drive dialled in so 'clean' SS vs 'clean' tubes, 'mild O/D' v mild tubes, fuller O/D SS v fuller O/D tubes. Listen as many times as you like before plumping for which set belonged to SS and which was the tubes.

 

I was surprised by the low result. 75% 'correct'. I thought they were very close but no cigar and figured most would pick it same.

 

Even more surprised was I by the enthusiastic interpretation of the tester that it meant 75% could tell real tubes from modeling effects.

[agedhorse]

I would not have been surprised if under a control test (same unit to same unit) players "discovered" a 25% difference from where pure guessing would result.

 

Without control testing, you don't know if your original listening test is valid. Failing a control test is not uncommon, just shows that there is a flaw somewhere in the methodology.