https://mattersoftesting.blog.gov.uk/improving-testing-standards-by-sharing-test-quality-data/

Improving testing standards by sharing test quality data

computer screen showing test quality information homepage

Everyone who uses the MOT testing service can now access test quality information (TQI). You’ll find useful statistics about how many tests your garage has done, how long your tests take and what your testers tend to fail vehicles on. You’ll be able to compare all this information against the national average.

Authorised Examiners (AEs) and site managers will be able to use this to monitor and improve the standard of testing at their sites.

What information is available?

group b vehicles screenshot
What test quality information looks like on the MOT testing service

You’ll be able to see statistics about your Vehicle Testing Station (VTS) including the:

  • number of MOT tests carried out
  • average age of vehicles tested
  • average test time
  • percentage of vehicles failed

You’ll be able to compare all your results against the national average. There'll also be a breakdown of tester’s failures by component category and the national failure rate for those components.

You’ll only find the information for the classes of vehicle that your VTS tests. So if your VTS only test classes 1 and 2 vehicles then you’ll only see information for those classes. If you test all classes of vehicles then you’ll see information for all classes.

The information displayed only includes the results of initial MOT tests and doesn't include any re-tests or cancelled tests. MOT tests that have been recorded as a pass with rectification (PRS) will be classed as failing the initial test.

You’ll be able to download a copy of all this data.

How AEs can use this information

TQI will help you manage the quality of testing at your facility. How you use it will depend on how you manage the site, its size and number of staff, etc.

You should review this information regularly and look into any unusual differences in the data; documenting the outcome of any investigation you’ve undertaken and what action you took. An example could be if a tester’s fail rate for a component category is 20% and the national failure rate is 8%.

In this case, you could organise quality control checks on failed tests where the tester has failed components within that category. You could check the tester’s understanding of the pass/fail criteria then record your findings.

It’s important to show that you’re reviewing, understanding and acting on the information found on the system.

How DVSA will use this information

DVSA’s vehicle examiners will look at this information before they visit a VTS. During a site assessment, they'll look for evidence of any action taken by AEs where they find unusual differences in the data.

As there could be valid reasons why a testers fail rate is unusually high or low vehicle examiners will look at all the facts before taking any action.

Your feedback

We've received some feedback asking why the component fail rate is higher when shown in TQI than when it was shown in the MOT comp as a Scheme Performance Report (SPR).

This is because TQI uses the number of failed initial tests as part of the calculation, whereas SPR used the total number of initial tests (passes and fails) to calculate the rate.

We've also received feedback that some of you can’t effectively use TQI to compare testers fail rates as your testers carry out differing numbers of tests per month.

We're planning to review how this information is presented as we want to make sure it benefits users and contributes to improving the quality of the MOT test.

Please keep sending us your comments and suggestions about the MOT testing service, as it's only with your feedback that we can introduce things to the service that you want to see.

Let us know in the comments below or send your feedback about the MOT testing service to MOT@vosa.gsi.gov.uk.

This article was updated on 29/11/2016 to include your feedback on TQI. 

55 comments

  1. Comment by Martin posted on

    Test results are area specific, town cars get a high percentage of failures on brakes due to lack of use and where I am its mainly suspension and lights due to the poor state of the roads.
    Coastal vehicles its corrosion.
    I have tested in the south and north so I know from experience.
    percentages mean absolutely nothing in the real world

    • Replies to Martin>

      Comment by darren posted on

      totally agree with you on that one

  2. Comment by Mohan Dadral posted on

    Mohan Dadral,

    I think everyone needs to carry on their normal practise, on the other hand I can see VOSA needs to have stats.
    I can see also this is important as Europe/Brussells are trying to get us to do MOTs once every two years, DESPITE BREXIT, Lets do what we do, do it well. The stats will always be there. The stats will always be used in different ways.Famous quote- Lies! Lies! And STATISTICS! If you are doing your job correctly then whatever anyone else has to say, Ie Vosa, Then we need to show we are working to to MOT scheme regs to the letter & therefore we know we are in the right. Lets be careful out there guys

    Thanks
    Mo

  3. Comment by Kevin from Hull posted on

    As long as human beings do this job almost every car tested will get a different result from each tester that looks at it.

  4. Comment by Alex posted on

    80% fail rate on bulbs against 50% ave, If they don't work I fail them...

  5. Comment by Mike parsons posted on

    I only test vehicles which have just been serviced , we always service vehicles first and replace any worn parts which might fail the M O T .
    What is my national average going to tell the D V S A ?

  6. Comment by Jones posted on

    I was once asked why by vehicle examiner why I was failing a lot if coil springs and bulbs!!!i actually looked at him in disbelief and asked if I he really wanted an answer or was it a joke question!if my statistics show I am failing high amount of bulbs or whatever I don't want my judgment questioned over statistics!in all honesty I feel the mot scheme is becoming a bit of a farce!

  7. Comment by David posted on

    I work for the Royal Mail and all our vehicles have a full service before the mot test,where all mot items are checked,so my failure rate is very low compared to the national average.

  8. Comment by Sam posted on

    Some one sat at a desk again making more work for us keep it simple and it workes make. It hard and it dont in my mind a pass is a pass a fail is a fail i dont cair about statistics

    • Replies to Sam>

      Comment by darren posted on

      totally agree with you on that one

  9. Comment by Shahid posted on

    Hi how do you acess this info i cant seem to acess it

    • Replies to Shahid>

      Comment by Thomas (DVSA) posted on

      Hi Shahid,

      Testers can find this on their profile page in the 'Related' section
      AEDMs and AEDs can find information for their organisation on their AE details page in the 'Related' section
      Site Managers can find information for their Vehicle Testing Station on their VTS details page in the 'Related' section

      Regards

      Thomas

      • Replies to Thomas (DVSA)>

        Comment by Piston doctor posted on

        I have been testing for over 15 years now I still test every car in the same routine all different cars and all in different time I have days were nothing fail and days with nothing passing it never bothers me the national average means nothing because how many category do u have ? Age/ miles/ where it is made just look at how recalls are worked out

  10. Comment by Stephen posted on

    Not bad. But according to the % my NT has a 100% failure rate on Number plates and Vin numbers.
    Does the failure % include PRS's

    • Replies to Stephen>

      Comment by Matters of Testing posted on

      Hi Stephen,

      When a tester records a test where the outcome is PRS a VT30 refusal notice and a VT20 pass are produced.

      The VT30 refusal is treated as the initial test and is used in the test quality information calculations.

      The VT20 pass is treated as a retest and not included in the test quality information calculations.

      If you email mot@vosa.gsi.gov.uk with the VTS number, testers username and when this happened we can look into this for you.

      Regards

      Thomas

    • Replies to Stephen>

      Comment by Tony S posted on

      Yes, something is not right, I looked at mine and it shows that I have failed a few for road wheels.
      I've never failed road wheels, only advised a couple, so the new system must be picking up advisory's and showing them against our fail rates!!!

      • Replies to Tony S>

        Comment by Matters of Testing posted on

        Hi Tony,

        The development team is aware of this issue and are currently working to fix this.

        Regards

        Thomas

    • Replies to Stephen>

      Comment by Piston doctor posted on

      I have a 60%fail rate my colleague have a 100% pass rate why he services his first and I do all the ones that comes through the door so how would that work ?

  11. Comment by Andy posted on

    I thought we did the tests to make sure that the vehicle are in a road worthy condition not to get your averages the same as everyone else. It does not mater to me if all the vehicle I do in one day pass or fail, I'm not going to compromise my test standards to get my averages right.

    • Replies to Andy>

      Comment by darren posted on

      totally agree,we have no idea what condition the car is like what comes through our doors

      • Replies to darren>

        Comment by Peter Hedges posted on

        I agree with you

  12. Comment by Hank posted on

    So going back to like the old system was when you printed out your testers logs ,does this mean that the old system was better than the new one ?? ( think we all know the answer to that ) it's just cheaper for them and more expensive for the testing station these days !!

  13. Comment by Keith posted on

    That was a good read.....

  14. Comment by conrad posted on

    how will this work in a station like mine who only test we are not looking to fail as other testing staions do just to get the easy jobs like tyres etc we have a high through put and most of our cars come ready for test as most customers have the car serviced before the test .

  15. Comment by Craig posted on

    My concern with this item is the section about if a tester has a high rate in one area you can check back and take action. If the tests are carried out to correct standards all the time shouldn't it be more of a case to look st the stations or testers that are not failing any of these items as it indicates they are being missed

    • Replies to Craig>

      Comment by Tony S posted on

      Quite agree, good comment

  16. Comment by Stephen posted on

    So when can start using this new thing'y,

  17. Comment by Neil posted on

    As a tester, I like to use this to judge whether I am testing correctly. On the old sustem it showed I was failing a high number of road wheels. This made me look at the testing manual to re-aprrase
    How I tested.

  18. Comment by Julian NT QC SM posted on

    Averages are important as they show you who is doing what , it is always perceived to be big brother etc watching however this is another tool . There is no pressure to be an average it is just useful information every test station has a different set of customers and vehicles , what matters is the job is done correctly . If you are a multi tester station it is interesting over the course of time to compare testers , on a single tester station it is also interesting to compare to national average where no comparison within the testing station is possible .
    It will not affect my testing standards as I endeavour to follow guidance from the manual as well as discussing with other testers on site if necessary , it will still be my test and my decision .

  19. Comment by Andy posted on

    TBH i never have nor ever will give a flying monkey's under carriage about statistics & never check percentages;& why should i ?
    All i want to do is carry out a full,fair & as far as possible to the book MOT test!
    Every car i test is carried out on that vehicle's merits,stuff statistics!
    What would DVSA rather me do? Fiddle figures to suit them over actual road safety?Rubbish!

    • Replies to Andy>

      Comment by Tony S posted on

      Quite agree again, good comment

    • Replies to Andy>

      Comment by A.DANTAS posted on

      WELL DONE PHILIP; I ALWAYS DO THE SAME KEEP UP THE GOOD WORK REGARDSA.DANTAS

  20. Comment by Stu I posted on

    What will change if a vehicle test is carried out to the correct standard the pass or fail that is what creates the stats or are we expected to put stats first so as not to be penalised

    • Replies to Stu I>

      Comment by Andy posted on

      Sound like statistics should matter more than actual safety.

  21. Comment by Robert Walker posted on

    Consideration has to be given the type of car like small new-ish 2 door, eg. C1s' , MX5s', can be examined and logged in 30 minutes. On the other hand, 5 doors, tail gates, 7 seaters all extend the time. It took the two of us 2 hours to test and log the failures on a Rolls Royce. But one Rolls Royce every blue moon number crunched into an mean average doesnt mean anything apart from bad for the books.

  22. Comment by Jb posted on

    What good is an average percentage ? If a car fails its test it fails because it's not road worthy and should not affect your decision just because you averages are low, the mot is not a number crunching exercise its there for the safety of the road users and not government targets !!!!

    • Replies to Jb>

      Comment by darren posted on

      well said totally agree

  23. Comment by Colin.Hilbourne posted on

    We were told to test as presented to give a true picture of the car. We also will Mot a car prior ro carying out servicing , so that we can adjust what service we do for what money the customer has
    I got ask why i failed "position lights" alot , my answer was if they didnt work i failed them? We allow an hour for a test? If it takes less great, if it take a more time, that normally means there is alot of failure/advise items to do.

  24. Comment by Jon smith posted on

    I never ever lost a smart card should I have lost a couple just to keep up with the rest of the country to keep statistics correct at my vts it's a complete joke if a scrapheap comes in it can take 5 times as long to test and enter results as a nice well looked after car what does it matter statistics can be fiddled if I do 4 tests every 2 mins then one at four hours the average is useless

  25. Comment by G posted on

    This was on the old system

    • Replies to G>

      Comment by Bob Barrie NT/QC/SM posted on

      its been missed on new system good to have it back good to compare against national averages

  26. Comment by Peter Hedges posted on

    I just test what comes through the gate and to the standard set .
    As for states they mean nothing as a lot of the time many testers will not bother with things like washer bottle empty they just fill it.

    • Replies to Peter Hedges>

      Comment by darren posted on

      very true i dont really bother about the national average because no one can predict what comes through the door

  27. Comment by Michael Cleary posted on

    One size doesnt fit all as the saying goes as in some of the above comments some testers never come across corrosion as they work at main agents doing fairly new cars some of us come across it more often so the averages should be taken with a pinch of salt, A.Es and Q,Cs,should only be concernd with carrying out tests to the correct standards,Garages that have sales targets & bonuses related to MOT testing should be looked at more closely.

  28. Comment by Stephen posted on

    Funny a lot people say national average, well surely that would depend on where you live/test.
    Roads vary from area to area or council to council, some of us might fail 25% on suspension and head light aims because of poorly maintained roads. If you live near the coast corrosion could play a major part too. We shouldn't judge ourselves on an average pass/fail. Just do what we're doing now.

  29. Comment by John posted on

    About time this was reintroduced,it was a good way of monitoring my testers results and dealing with anything that was unusually high or low compared to the National average.
    I always printed off the results and went through them with my testers.
    I think the new system is eventually becoming as good if not better than the old system..

  30. Comment by Peter posted on

    Why are testers not able to view this. Seems like more "big brother" to me. Would be nice if testers can see how they compare to others when disputes over productivity occur, work load sharing & more importantly, wages. We don't put safety boots on at our cost, buy tools inc pens just so we can get a small percentage of what we earn our employer only to be told "your going too slow" without back up to prove facts. Under the old system testers could print out performance reports & show employers ourselves. They couldn't disagree with our comments then as it was in black and white for ALL to see. Now it's just a case of testers taking the word or the employer and lumping it!

    • Replies to Peter>

      Comment by Matters of Testing posted on

      Hi Peter,

      All MOT testing service users will be able view this data not just Authorised Examiners and site managers. You'll also be able to download the data as a spreadsheet and print it off if you wish to do that.

      Regards

      Thomas

      • Replies to Matters of Testing>

        Comment by Peter posted on

        Oh, OK. Many thanks. Didn't read it fully obviously. Ooops!

  31. Comment by Rick Ager posted on

    Rick Much easier to find training opportunities good

  32. Comment by el posted on

    liked this option on the old system also good to see where improvements can be made and where things need looking at .

  33. Comment by Stephen posted on

    Just like the old system. That's good new's then, now I as a site manager/QC can see how long my NT's are taking.

    • Replies to Stephen>

      Comment by Philip Marrinan NT / QC posted on

      I am an NT and QC at a busy test station, why should it make any difference how long a tester takes to do a test, all that should matter is that it is done to standard and properly.No tester should be pressurized to do tests fast as that is when problems arise

      • Replies to Philip Marrinan NT / QC>

        Comment by Otis posted on

        Spot on Phillip. I would also like to enforce the comments regarding failure rates. It depends on what comes through the door to which we have no control. Whilst some stats may give an indication that an NT may beveither struggling or has a high failure rate they can be misleading .

      • Replies to Philip Marrinan NT / QC>

        Comment by Andy posted on

        Bang on Philip; Quality of test far out weighs speed of test & anyone who thinks vice- versa should be shut down!