The new approach to risk rating using MOT data from both testers and garages has been live for over 6 months now. So far, we think it’s going well.
But, like previous versions of risk rating the new approach has caused testers and garages some concern. I thought I would explain what risk rating means to us, what it should mean to you, and what we’re doing to improve how it works.
Making the most of what we’ve got
One of the main reasons for the risk ratings is working smart and working efficient. Like all organisations, we only have a finite number of staff and a finite amount of time. We can’t be everywhere at once.
That means we need to think of new and innovative ways to help us target the right garages and testers. This allows us to focus less of our energy on good garages and testers, and to focus our efforts to improve the overall quality of MOT testing.
That’s really the main point of using data to assess risk. It isn’t used to make decisions about a garage or tester’s performance. It’s a tool we use to point us in the right direction.
Risk, not performance or discipline
As I said, this is about rating risk, not performance. For us red and amber shows the testers or garages that might be worth giving attention to. But before we plan some sort of enforcement check – we will also review the data to check an enforcement check is the right thing to do.
And when we do the enforcement check, if we don’t find any serious problems it will improve the risk score.
Unlike disciplinary actions, like formal or final warnings, risk rating is not absolute. It does not mean that we have found anything wrong or indeed you are doing anything wrong. It is simply a tool and an indicator for us to help plan our work.
What should you do if you’re rated red?
The first thing to say is that there’s no need to panic. Turning red or amber does not mean we’ll come racing down to the garage to see what is going on. Instead, we’ll monitor that tester or VTS to see if they stay high risk month after month. If they do then yes, a visit may be needed.
If you’re red or amber, you should do the same. Monitor your risk rating and dig through your test logs and test quality information to see what could be causing it. You should also speak to your peers and your manager to try and understand why you’re on a higher rating. If you haven’t got time for this you may want to consider employing a consultant or third party to help.
Making the system work better
Risk rating will evolve and improve as we better understand how to use the data effectively. The work we have been doing over the years has enabled MOT to be more dynamic, giving us the ability to adapt much more quickly than when we relied mainly on-site assessments.
For example, we knew we had a problem with low volume testers fluctuating on the risk rating due to the fact they test infrequently. So, we have tweaked the system to improve this. We have also been out speaking to our vehicle examiners, helping them to understand when to mark an area of the site review unsatisfactory and when not to.
Looking to the future, we have more ideas on the table to further improve risk rating for you and us. Again using MOT data, we are exploring the idea of predicting the result of the test for the vehicle being tested. This isn’t going to happen imminently – we’re just testing the theory at the moment.
But if the results show we can predict the result with a high level of accuracy, we might be able to couple this with other metrics to help risk rating. Obviously this will need to be trialled and tested before we start using it, and it would simply be an indicator that something is different that might require deeper investigation.
And finally…
It’s easy for me to say don’t worry about risk rating when I’m writing this from behind my desk (well, actually on a train to Nottingham) but you seriously don’t need to. Just having a red risk rating doesn’t mean we take action. We will only take action if we find something wrong.
If you believe you are doing things correctly and have used all the tools available to you to establish this, then please carry on doing the great job you do helping us keep Britain’s roads some of the safest in the world.
We'll be launching a guide on how to manage your garage soon.
If you have a question about your risk rating score, or want to appeal your score, you can contact our customer service centre on 0300 123 9000.
134 comments
Comment by Hooters posted on
For varying opinions, have a look at the MoT Forum.
Just search MTAETA which should get you there.
Comment by Lee Heywood posted on
What exactly is the risk that's so often mentioned. My station was rated red yet both the testers were at the time, green. Also, has it been considered that dishonest testers are going to fix the rag which in turn gives them more breathing space to carry in being dishonest.
Thanks Lee.
Comment by Tony S posted on
Hi All
So I've been testing for over 30yrs now without any glitches, until mid last year, when a new manager came in, (3 years into my job).
He didn't like my advisory's.
Long story short, I leave mid February.
Always had a green RAG to this point. Average 160 tests a month.
Only 1/2 a month in Feb, test numbers were down.
1st March I'm in the red.
Fortunately an old friend gave me some work, so my RAG didn't matter.
Had I gone to a main dealer, it may well have mattered!
Did 14 tests in March and I'm back in the green.
My test standards have always been the same throughout!
So why should it matter how many tests you do or don't do, how many pass or fail, how new or old the car is, how many days you work or not.
I get how it works, but it don't make any sense!
I get that we shouldn't worry about being in the red, but some of the AE's and AEDM's do and don't want Testers that are in the red.
Comment by john burvill posted on
I am in the green my station is in the green but my N T is in amber , we have looked at his test logs and he is just below in all failure rates by 1 to 2%
but the nat/ average of amount of vehicles tested over three months is 146
he has tested 509 vehicles has that put him in the amber.
Comment by Julia (DVSA) posted on
Hi John
Risk rating looks at a number of different aspects of testing and it's the combination of these that lead to the RAG rating. Your tester is a high volume tester but this alone would not cause the amber rating. The amber rating means that something is different from the average. That does not necessarily mean anything is wrong, it could be right to be different, for example, older vehicles often go hand in hand with higher failure rates and longer test times. Being amber is nothing to worry about if you are happy that testing is being done correctly.
Comment by john burvill posted on
Hi Julia; thank you for your reply, we will now look at times and failure for older vehicle against newer vehicles , but could I ask another question ?
I have just spent twelve thousand pounds on a new exhaust emissions mot test machine and I was wondering when the mot test price increase is going ahead due to the fact we have not had a increase in mot prices for over six years .
Comment by gbr1802 posted on
RUBBISH high volume of tests should have no effect on risk rating. Just because its not average doesnt mean anything is at risk here
Comment by Julia (DVSA) posted on
Hi there
A high volume of tests doesn't indicate you are doing anything wrong.
Comment by TC posted on
Hi just to clear one point on wipers self park not working .Wipers stop were you put them pass or advice .Manuel says automatically stop in position obscuring view . I would say that is when you turn them off !
Comment by Scott posted on
Hello TC
Its only a fail if they cannot switch off in the off screen position and cause a significant obstruction to driver view of the road. If you can switch them off manually in the off screen position using the wiper control this would be satisfactory for a pass
Comment by Steve posted on
looking at the updates to the manual. diesel emission test not completed due to high level of smoke .
so where is the change in the MTS ?
Also if you open the manual and look at the list of updates when you get to the part that mentions AXLES MAYBE SOMEONE SHOULD OF READ THEASE THINGS FIRST
Comment by richard posted on
I did report the Axle one via here, but I've no idea if anyone has picked up on it yet as its still the same.
Comment by Chris (DVSA) posted on
Hi Richard,
Thanks for reporting this to us. We're now aware of the issue and are working to correct it.
Comment by richard posted on
How long should it or does it take to change something that we rely on ? How do the technical wheels work here ? It would be interesting perhaps having an article on how things work on in the background, maybe it'll stop the same questions getting asked all the time or the frustration of not getting a reply back from an answered question.
Comment by Julia (DVSA) posted on
Hi Steve
This is the path to follow. Then select the relevant year of vehicle
1. Noise, emissions and leaks > Exhaust emissions > Compression ignition > select year.
The change log is currently being addressed and fixed.
Comment by steve posted on
thanks for the reply but the wording in the MTS does not tally up with the manual for the emissions
thanks
Comment by Simon (DVSA) posted on
Hi Steve
As testers usually refer to aborting the tests, the wording to select is
''Smoke levels significantly in excess of specified limits, emissions test aborted''
We thought this would be simpler to read. When selected the wording on the test summary and MOT documents is
''Exhaust emissions test not completed because smoke levels are significantly in excess of the specified limit values''
We may look to change this going forward.
There are other reasons for rejection which are worded differently to what is printed on the certificate, which was done for the same reasons as described.
Comment by Steve posted on
After 40 years in the motor trade including testing ,all the new so called initiatives like RAG I threw in the towel and got out and I knew of 4 outher testers that have done the same in 1 year .so carry on DVSA the system now is broken
Comment by Andrew Smith posted on
I got points from Green to red right away 2 officers came an question me so hard didn’t know what to say couple off defects missed they like to pick on tested try an find something maybe get higher bonus not fair i do my job probably always been in the green never got a site visit now I lost sleep over this
Comment by Julia (DVSA) posted on
Hi Andrew
You don’t need to lose sleep over this. You’ll only usually get a site visit if we are concerned with how the garage or tester is performing. As it says in the blog post:
‘Instead, we’ll monitor that tester or VTS to see if they stay high risk month after month. If they do then yes, a visit may be needed.’
DVSA staff who visit garages are not on bonuses.
Comment by mark posted on
no not on bonuses but like too look for faults
Comment by Wayne posted on
The mot scheme has in my opinion lost its way. Too many things to keep up to date with and the constant changes to the inspection manual is a joke! If something works leave it alone don’t try and change it. And one last thing please bring back the old fashion refresher course we’d all learn and take it in better with taking to fellow mot testers!
Comment by Gordon McFarlane posted on
I used to work for this lot DVSA The Tester risk rating is a "Blunt Instrument"
I have been every colour under the sun It is not a predictor of non compliance it is just an excuse for poor service . Near to where I MOT we know that a garage is not compliant and guess what its green as are its testers. I also do some consultancy nobody thinks it has any value if you try to find out why you are not green no one can tell you
Comment by Sean McCarthy posted on
Hi,
Could you give us an idea of how you are treating the variance to site and national averages for a given category. Treatment 1 or 2?
I ask because of the 2 following potential situations
Example A
National average is 50% Tester average is 40%
Treatment 1 - variation is 10 i.e. 50 - 40 = 10
Treatment 2 - variation is 20% i.e. 10 is 20% of 50
Example B
National average is 4% Tester average is 2%
Treatment 1 - variation is 2% i.e. 4 - 2 = 2
Treatment 2 - variation is 50% i.e. 2 is 50% of 4
Treatment 1 gives a very different result to Treatment 2 in each of the examples. Should we be looking at a 20% variation of 10% or a 50% variation of 2%?
It would really help us to identify which areas to concentrate on for a tester.
It would also really help if you could give us some idea of what a Red or Amber variance would be. I understand there are other factors that make up a RAG score such as average length of test, average age of vehicle, number of test etc but if you can give us an indication we could at least begin to concentrate on what we need to i.e. the Red elements
Thanks
Sean
Comment by lee posted on
Who RobbWebb ?? he appears to be answering on behalf of DVSA
Comment by Julia (DVSA) posted on
Hi Lee
Rob works for DVSA - sorry this wasn't made clear. Any future replies will come from him with (DVSA) after his name.
Comment by Bob posted on
Glad i am not testing now first started in 1988 to much hassle moving the goalposts too much should have kept the refresher courses could talk to real people that is in the trade including VE that know what they are talking about & not someone that has just left collage looking at computer screens and dont have a clue trying to keep them selves in a job and only see targets & it will be fun when the govt will eventually privatize the dvsa
Comment by jooles posted on
and again you are not listening ,how many more negative comments do you need to hear about this rag rating ,why you cannot admit you have got this wrong is beyond me and thousands of testers,we do a thankless job as it is we do not need to be persecuted by your organisation as well ,,
Comment by An old man posted on
Over 40 years a m.o.t. Tester . Have never wanted to become a IT wizard. Just test a vehicle correctly . Have sent email with mistakes in manual and testing advice . No reply mistakes still there. Time to retire.
Comment by 1802gbr posted on
same problem here at one time the manual was like the bible but not any more sick of sending messages with no acknowdgement or reply usually messages dont even get onto here
Comment by Rob Galvin posted on
A car should be tested based only by its own merit, I believe this system is putting testers in anxious situations where their decision making is being effected by their rag rating.
I.E "I haven't failed enough of this particular item for a while, I'd better fail this one rather than advise it to increase my average".
I employ 12 testers, this is my biggest concern.
Comment by Chris posted on
Or potentially the opposite "ive failed too many off this so i better pass this one"
Comment by Bill posted on
Would it not be possible to colour code the data from the Test Quality Information and Test Logs reports to give the Testers and Managers an idea where there are abnormal values. Normal value data could be green and abnormal values could be red. This at least be an indicator for anyone trying to analyse the data.
Comment by Bryan posted on
Off topic but can somebody please tell me why we still have to fail number plates for incorrect spacing when the system doesn't even recognise a space and we know that as soon as they leave the test centre they will put their show plates back on as nobody will say a word about them until next year's mot?
Comment by Tony S posted on
Well my guess is it's 'because we are, were, told to'. (as in, on a 5 year refresher course)
I guess DVLA don't keep a check on these cars with ANPR and confiscate the plate like they say they will when you buy the plate.
Also the police don't bother too much nowadays.
So it's all down to us Testers for now.
Once the correct plate is on you could advise 'correct plate fitted at time of retest'.
Comment by mark posted on
Iwonder if the dvsa are reading all these comments and scrap this rag idea once and for all because it isnt working just stressing good testers who will just give up eventually because its not worth the hassle
Comment by W Morgan posted on
Sadly all this is achieving is more confusion and stress for the genuinely good testers who eventually will simply leave the testing scheme and those who were not so great and possibly not doing things correctly will remain as they didn't care to begin with.
Comment by Kevin posted on
We have just got 40 points after a visit.how long is it before we will go back to green
Comment by robwebb posted on
Hi Kevin,
It's not possible to give a concrete time scale.
The key is to keep testing to the right standards, and over time the score should improve.
Rob
Comment by Kevin posted on
Thank you
Comment by NC posted on
"Predicting MOT results" what a brilliant idea. We could log a car on, DVSA would tell us the result, we would print off the relevant certificate, give it to the customer and 10 minutes later log the next car on. Just think how many MOT's we could do in a day! All the costs of equipment and calibrations etc etc would be saved, in fact testing lanes could be done away with. This would solve all the arguments over "Risk Assessment" because if we pass or fail according to DVSA's instruction we cannot do wrong! Site visits could be done in minutes because all that would need to be checked would be a desk a chair and a computer which would alleviate considerably the work load on the Examiners. The savings and the problems this would solve are phenomenal how come nobody has thought of this before!!
Comment by robwebb posted on
Hi NC,
As mentioned by other blog posts, there are weaknesses in using national average failure rates. So using rates based on the actual vehicles that have been presented can be helpful. Prediction therefore isn't intended to replace the test itself (!) - its just a way of, in the background, comparing the results that a tester gets for, say, their last 100 tests with what one may expect. It will inevitably have some imperfections, but will better reflect the real world. As noted elsewhere - for testers the key is to use the standards set out in the manual and test to the correct standards.
Rob
Comment by AVM posted on
Yes, we have slipped into amber or red but your reason does not always show in performance/test logs. It's not always obvious as we do not know the criteria you use
Comment by AVM posted on
Can DVSA give advice on their criteria for analysis of performance reports
ie what percentage figure should we use for defects above or below the National Average ?
We often have a risk score changing on a monthly basis, but there seems to be no perceptible change to the scores ect?
Comment by TOM posted on
As a contracted mot from the motor trade , my risk score has to be green else I cannot work. I may only carry out 10-40 tests a month as I have a second job as a mechanical engineer where MOTs arent required on plant equipment.
Some information on how the risk score is calculated would be beneficial to ensure that I don't affect my rated and loose work due to low numbers
Comment by robwebb posted on
Hi Tom,
What we don't want is testers to become a slave to the score. The key thing is to test to the right standards - doing the test properly. As mentioned the score looks at a number of factors - test time, failure rate etc - but will continue to evolve and develop to best ensure that those that do the job well have a score that reflects that.
Rob
Comment by MartiN posted on
Oh dear all your research obviously does not take into account human nature.
Those that care will manipulate the figures they enter and those that do not care will not bother thereby negating any improvement you are trying to achieve.
Comment by Kevin Davies posted on
Not telling the station manager wen someone has been graded amber or red is ridiculous the manager has a role in managing the the vts and he is not being provided with the information required to do so. Hiding behind data protection is a poor excuse and can easily be achieved there is no personal data involved in the ratings.. The review of the the TQI may not be obvious as to why they are rated red then a system of guess work is introduced with which encourages testers to try and manipulate the system to try and remedy what they believe may the reason for their rating. I do believe that the majority of stations and testers want to provide a high quality if service both in the testing and to their customers. The secrecy in both why the ratings are given and who is graded amber or red does not support this in my view.
Comment by Dave Smith posted on
Totally agree with the ability to see you Testers Status is an absolute must. How can we as AEDM's help monitor this situation if we cant see the info. ' closing the stable door once the horse has bolted ' comes to mind?
Comment by Will posted on
All this does is slowly but persistently put pressure on good people trying to earn a living. Over a period of time it gradually sucks every bit of fun and enjoyment out of the job. What next? League tables? A vehicle is a pass or a fail. In your obsession to somehow drive up standards, you are creating stress and worry for honest folk doing a good job. You are treating everyone as if they were dishonest and although you wouldn’t pertain to this - that is how it makes people feel. You will simply drive good people from the industry. Leave good alone.
Comment by db posted on
hit the nail on the head there will.
Comment by Nasir posted on
Hi everyone I am MOT TESTER MORE then 3 years if you follow these
1. Take your time to check the car correctly A to Z
2. Check all the safety of the vehicle UNDER
3. Don’t listen to anyone just do the right thing
4.last but but less don’t try to failed the car for no reasons to make money
Always read the MOT manual and keep your self update about the new roles ...
Comment by merc123 posted on
we will see i have 110 points comeing off this month it will be interesting what my rag score will be next month our will it stay the same as for hireing out side consults do we get brownie points for this as there not cheap i think we should as were are being proactive too correct any problems we have before they cause us too have a problem as answer too this last question would be nice
Comment by Julia (DVSA) posted on
Hi there
The site risk score is based on the site review that the VE does, together with the scores from all the testers. The site review looks at the effectiveness of the quality system of the garage - so how good they are at proactively managing test quality. There is no necessity to employ a consultant and no specific 'brownie points' for doing so - but if a good consultant is employed and does a good job of helping the garage manage things, the the garage should score well.
Comment by Tony posted on
So by analysis of the data, manipulation of my test quality, I can drag myself back to amber. Mmmm, difficult one that.
Comment by Andy posted on
I think that this RAG score system is going to cause more problems when testers start to micro analyze there test logs and alter the way they record tests .This is another work place stress that in my view is unfair.
Comment by Nga posted on
Guess this keeps some"analysts" in employment....
Comment by Robert walker posted on
At the end if the day you go through the proper procedure follow the correct pass or fail criteria and your in the red what more can you do think the system is totally flawed in my opinion
Comment by Nfp AE posted on
You are missing a vital piece of information.... we have no control on what vehicle is presented ... and therefore the rating does not reflect the tester but the vehicles presented of which we have no control... this is not a fair reflection of our skilled MOT testers and AE’s
Comment by Julia (DVSA) posted on
Hi there
You are right - and that is why we plan to add more sophistication over time. One of those areas is to look to move away from average fail rates to something based on the actual vehicles presented. It will still be imperfect - but any approach will be; but if it can improve how we deploy resource then that will be a good thing.
Comment by Paul posted on
Looking to the future, we have more ideas on the table to further improve risk rating for you and us. Again using MOT data, we are exploring the idea of predicting the result of the test for the vehicle being tested. This isn’t going to happen imminently – we’re just testing the theory at the moment.
Before you know it, you will be arrested for the crime you will commit next year. Oh, hang on, that film has already been done hasn't it? Was it the Minority report?
And with the way we are going you could be fined for passing wind shortly.
It’s a year since the major changes to the MoT Testing Service (MTS) and I would like to be able to quote statistics sometime in the future. If possible, I would like everyone connected with testing to vote please; you all know how important our ‘feedback’ is! Hopefully the results will be used to further improve things.
Comment by Julia (DVSA) posted on
Hi Paul
The blog covers what we use risk scores for - which is about targeting effort, not taking direct action.
As mentioned elsewhere the next stage is to look at the comparison of failure rate across a sample of the actual vehicles tested. This work should better handle the criticisms mentioned elsewhere in this blog about use of average fail rates.
Comment by MH posted on
I am now rated amber, as far as I can tell it is because lots of bikes are presented for test with the rear reflector incorrectly positioned, should I just overlook this obvious minor fail to regain my green rating?
Comment by Julia (DVSA) posted on
Hi there
No. We're absolutely not advising testers to do anything they don't think is right for the sake of improving their risk rating score. If a vehicle should fail, please act in line with the manual and your judgement as an MOT tester. We can then use information about kinds of faults disproportionately found on vehicles to improve our risk rating scores.
Comment by Sm posted on
This traffic light system does cause questions. I have 3 testers, one in red, 2 in amber but garage shows as green.
When looking at quality checks we are always over high on testing old vehicles. The system does not cater for country mot stations to dealership stations. Stations out in the country deal with vehicles of age 10 years more, apologies to dealerships they deal with first time tests and vehicles under 10 year old?
Comment by Julia (DVSA) posted on
Hi there
As mentioned elsewhere the next stage is to look at the comparison of failure rate across a sample of the actual vehicles tested. This is exactly the sort of scenario that this should better handle.
Comment by Andy posted on
My rating is always amber and my testing information is so far away from the national average. considering most of my tests at a mercedes benz dealer garage are 3-5 year old mercedes with low mileage and sales cars that have already been prepped to be sold on day of sale.
Comment by Julia (DVSA) posted on
Hi Andy
As mentioned elsewhere the next stage is to look at the comparison of failure rate across a sample of the actual vehicles tested. This is exactly the sort of scenario that this should better handle.
Comment by Mike posted on
I contacted area managers and all the way up to Bristol headquarters to find out what a red flag was about.
Nobody is prepared to tell me, you are collecting data on us using it against us and alienating us testers in the process.
If we ask you why you should tell us
How else will the ridiculous scheme improve!!!!
Comment by Julia (DVSA) posted on
Hi Mike
Apologies that nobody has got back to you. This blog aims to explain what we use the blog for - which is risk rating to help DVSA target its resource. If you have a specific query, drop a note to the MOT mailbox referencing this blog post and we will get back to you.
Comment by Paul posted on
You know what, i cant really believe how this job has changed and what direction it is going in....i mean who actually thinks up these ideas. Is there really that little work out there in the ‘IT’ world , that they dream up ludicrous ideas that instil minor paranoia in testers to ensure they are ‘performing correctly’.
Most of us chose this profession because we are hands on, problem solving people...we like being helpful and it gives us a sense of purpose. NOW it seems we should have an ‘above average’ IT skill to use the damn system, look over our shoulder incase we press the wrong button or god forbid veer from the national average, and damn to the 25yrs plus experience in mechanics....cos the computer says NO..
We all dread where this will end end up and hark back to the days when we just looked and assessed vehicles for safety...advised the customer what needed or not to be done...and could carry out the job we actually trained for.?..
Comment by Julia (DVSA) posted on
Hi Paul
The aim of the blog was to try and explain risk rating a little more for those who want to know. Of course none of that takes a way the focus that testers must have on testing vehicles - that's the important thing here, and the IT is there to support that. But - there are challenges with MOT test quality that we know exist - so we do need to direct our enforcement effort to support those testers that need it and, where necessary, take enforcement action.
If you're a good tester and can stand by the quality of your testing, you should have nothing to worry about. Any action we take will always be on the basis of an actual, in-person site visit and inspection. We never make any decisions solely on the basis of risk ratings, as these are just a tool to help us work our where to look.
Comment by Julian posted on
The RAG is a waste of everyone’s time. My fail rate was 50% in my last employment, RAG was green, both for me and the VTS . My QC always mentioned it , but followed it up with , “ don’t worry as long as you can stand by your tests “ . I always test to the same standard, by the book . Never saw anyone from DVSA , so green is good , even i you fail 50%
Comment by Julia (DVSA) posted on
Hi Julian
I disagree that the RAG ratings are a waste of time, but it's basically true that, so long as you're confident you're doing everything right and can stand by your tests, you shouldn't have anything to worry about. Our aim is that an honest, professional, conscientious tester who works to a high standard will always be rated green. This is what we're working towards, but it will be a process of improvement and refinement.
Comment by Martin posted on
Rubbish !
Comment by John posted on
I can't believe that you are developing a system to predict the result of a test. You won't need testing stations anymore as your system should know whether it's a pass or fail and send the owner a certificate. I would love to know how it will determine that a particular vehicle has been bodged by it's owner or a tyre damaged by a pot hole regardless of make or age. This will just have us all on red because the computer predicted a fail and we passed it or vice versa. Perhaps you should develop a computer controlled robot and rule us all out.
Comment by Julia (DVSA) posted on
Hi John
This is still under development. Essentially this wouldn't be expected to be 'correct' on every vehicle - but over, say, 100 vehicles tested one could expect, say, 75% match. This therefore would mean we could get away from average failure rates and get to expecting a failure rate to be within x% predicted over a period. As you say - the actual test will be the actual test - the IT can't really know what happens on the day.
Comment by Clive posted on
There appears in the face of it to be a significant cause for concern if the risk has changed. This is due to the limited vehicles tested and the heightened risk.The announcement of a "Consultant role" seems at odds given that a consultant is unlikely to test many vehicles as they are by virtue a consultant.. can we please give due consideration for those act as QC at several establishments.
Comment by Julia (DVSA) posted on
Hi Clive
The consultant role isn't intended for people to test vehicles. It's intended for consultants brought in by the garage to help monitor standards. This can be an external person hired by your garage or it can be someone who works for your garage.
Comment by Clive posted on
Hi Julia
So this is a role for an external provider or a third party in a garage who does not test . So what qualifications , knowledge a
nd/or experience are we looking for in someone who takes up or is contracted to the role?
Sorry if this sounds a bit Janet and John
Comment by Julia (DVSA) posted on
Hi Clive
AEs and AEDMs already have access to these functions but a garage can assign the role to anyone.
These would be people with knowledge of the MOT scheme to review data we make available to you. It’s not necessary for a garage to employ a consultant.
The consultant can view:
• test quality information
• test logs
• tester annual assessment certificates
• site review outcomes
They cannot carry out any other actions, such as buying slots, starting tests or adding and removing roles.
Comment by Adam posted on
What a load of woolly cobblers. Have the courage of your convictions, be honest and say it in a realistic way. The whole object should be to detect non compliant testers and VTS, The RAG rating should not be in the general public/Trade domain it’s just an indicator to direct your resources. The result of your follow up is what matters but has guys have pointed out the score might stop someone getting a job, that’s wrong.
Comment by dave bs posted on
I do believe the risk rating system is not helping testers in an amber or red rating get employment when needed (I am fortunate only time I went amber was for 2 weeks off at Christmas and will be again for August) perhaps it should be hidden with a system that sends a message to a tester if there records look out of place
Comment by stevie m posted on
Julia, looking at some of the comments from testers that have been on holiday going to amber i can see their concern its the one point i disagree with. maybe you can implement some sort of box to tick if testers are going on holiday or on the sick for say longer than a week in that way their testing rating wouldnt be affected.
Comment by Olivia (DVSA) posted on
Thanks Stevie, we'll pass this feedback onto the team.
Comment by W A posted on
When computerisation started we were all told it would save a lot of and less paperwork Totally not the case it gets worse every month so if we come out of the eu and stop following their directives will we get a more sensible system to follow
Comment by James perry posted on
Completely agree and if there lossing testers
It clearly says there is a problem with the way things are being done being a tester for the last 20 yrs and passing every test they put to you I am considering hanging up my gloves too becoming way to much hard work
Comment by James posted on
Predict MOT Result! How? no way will you ever get accurate data. If you do please predict the lottery numbers for me!
Comment by Leigh posted on
Chris, I understand how the risk rating works and understand why DVSA are using the data to work more efficiently, however you say that because a Tester might be RED or AMBER not to worry. Authorisations as part of their recruitment process are quite rightly asking potential candidates if they have had any previous disciplinary with DVSA and what is their RAG rating. Authorisations are nervous of employing a 'RED' Tester as this will affect the VTS RAG rating potentially meaning more DVSA visits, so this way of risk scoring is effectively hindering good Testers securing a job. Testers do not have to disclose their RAG score but yet again a refusal to give the information again affects the candidates chance of securing that employment. Equally the scoring system drives the wrong behaviours as some Testers try to manipulate the figures to get back into the GREEN E.g. to PRS or not to PRS, which goes against everything the data is designed to achieve. Just my thoughts 🙂
Comment by James perry posted on
? to true
Comment by Andy Dowden posted on
Hi every year I take out our insurance policy they always ask how we are on the traffic light system be for giving us our quote so it seems to effect our premiums as well
Andy
Comment by Julia (DVSA) posted on
Hi Leigh
This is a really good point and something we're very aware of. We're always looking into ways of improving how the risk rating system work. Thanks for your constructive feedback.
Comment by James McEwan posted on
I have employed a tester from another garage. When I log him on my system he has advised me he is showing amber. How does he progress back to green? All my other testers are green and so is the VTS .
Comment by Julia (DVSA) posted on
Hi James
It's a case of looking at why this might have happened. Looking at test logs, event history and TQI reports may help pinpoint this.
Comment by Julia (DVSA) posted on
Hi James
The tester may have been amber due to circumstances beyond his control so we advise him to look at his score after a month of working alongside the testers at his new VTS – that would give a better comparison.
Comment by con posted on
I have have been red from the start because of points i got acting on advise from the help line (my first points in over 20 years some use the advise was) unfortunately for me i will be red for the next 3 1/2 years its also put my site in the red so we are stuck with it .
so how will the new system affect me as nothing will change so i could always be a target for being in the red to long
Comment by Julia (DVSA) posted on
Hi Con
A number of things could put you in the red but you really do need to take a look at things like test logs, TQI reports and event history to see why this is the case. It's unlikely you'll be in the red this length of time.
Comment by con posted on
the risk is part of disciplinary i have 500 points from bad advise from the help line all my figures are close to the national average apart from number of tests as i do about 170 a month but i have 500 points so i am basically a target for the next 3 1/2 years
Comment by Julia (DVSA) posted on
Hi Con
If you want to appeal your rating please contact the DVSA Customer Support Centre on 0300 123 9000 and they will look into this for you.
Comment by Mike Williams posted on
Con, you got 500 hundred points due to bad advice from the contact centre. That's enough for immediate knock off. Why didn't you appeal the decision at the time. What was the advice.
Comment by Con posted on
Hi Mike Long story short got me to log on a vehicle not on site to issue a replacement as original tester used wrong details.
They sort of addmited it happened by letting me keep testing but since they issue no incident numbers I could not prove it was advise from them.
Think my previous 20 years clean record helped keep me testing
Comment by Abzzz posted on
So what about VTSs that have garages use them a lot of the time the cars are serviced and checked before so fly through so that tester will have a low failure rate and goes amber because he’s not failing enough cars and majority of the failure are PRSd
Comment by Julia (DVSA) posted on
Hi Abzzz
Failure rate is just one of the factors at play. But inevitably a risk score is not perfect.
Comment by kerry posted on
well said con 🙂
Comment by Paul k posted on
Being red dont mean nothing if ur doing nothing wrong i was red for8 yrs loved it enjoy
Comment by Andrew Trigg posted on
I think this is an excellent idea, however, are DVSA HGV/PSV Testers risk rated in the same way, similar to OCRS? Doing this might help to improve relations between the DVSA and PSV/HGV Operators.
Comment by Roger Peatman posted on
Proof reading would be a good idea too !!
It gets rid of annoying mistakes.
Comment by Paul Beoojesm posted on
There you have A I is now running the scheme
Comment by Terry Hill posted on
Information well received. It is a big plus to have a better understanding of how you view the risk rating results against how us the testers see it on the shop floor.
Comment by stevie m posted on
agree100% terry
Comment by Roy posted on
Agree.
Comment by stewart duthie posted on
I do 50 mots one month and fail 14 test for faults,then check my risk score ,to find i am high on country average of mot fail criteria
I like a laugh,get real here,what do you think i should do pass every car that comes in the door to be as good as some fast fit score,dont think so.
I dont beleive in a minuite that fast fits,and a good 30 to 40 % of your garage testers are not failing more,either that or they are telling customer ,yip your car failed and not even tested to heap it on on jobs.
The system still has many loopholes, and there are still john waynes out there working on cars that shouldnt be.
Comment by Julia (DVSA) posted on
Hi Stewart
We're absolutely not advising testers to pass vehicles that shouldn't pass to improve their risk score. If you're doing everything right, and testing to the correct standard, you have nothing to worry about. As we said in the blog post, we don't take disciplinary action on the basis of risk rating scores. All it means is that we're more likely to come by for an inspection if the risk score remains on red for an extended period. If we then find nothing wrong, we will use that to improve how we calculate the scores.
This is all about trying to focus more of our time and effort on the garages and testers that really need it.
Comment by Andy Forbes-Buckingham posted on
I think it would very helpful to know what the concerns were, it simply says VTS RAG RED which has the effect of making us worry, or trying to correct an issue that may not exist
Comment by Julia (DVSA) posted on
Hi Andy
As Chris says in the blog post, you don't need to worry but just take a look through your test logs and test quality information to see what could be causing it. Ratings do change over time for many testers.
Comment by Mark posted on
I check this all the time and I can't see why I go between amber and red the only thing I can see is I do above average because I'm the only tester in our garage and I take about 55 minutes over a test but that doesn't mean I'm doing it wrong just a bit slow I didn't realise mot testing was a race now
Comment by Abdul hayat posted on
I keep bouncing in and out of amber to green because I’m not failing enough I do look at test logs and test quality and everything has percentages that is wrong because for example areas by the sea are going to have a higher failure for corrosion than a area that is far from the sea air So how does this work
Comment by Julia (DVSA) posted on
Hi Abdul
This is all part of the process of improving MOT risk ratings. The more information we have, the more we can take into consideration when calculating the risk ratings. At the moment, it's still early days, but as we work with this system for longer, the better it will get.
Comment by Neil Bennett posted on
Hi Julia,
"to see what COULD be causing it" and what could be causing it at the coal face and what is actually causing it COULD be two totally different things. you already know why it is amber or red, why not inform the tester of your decision and area for improvement.
Comment by Rod Bennett posted on
Would it be possible to include another advisory in the n list of non component advisories? A comment on no print out due to low emissions would be useful.
Thanks
Comment by Craig posted on
On new emission machine you will always have a print out on emission even if there is no reading. Need to start measuring the revs. And it will give u a print out
Craig
Comment by Jim posted on
Take two weeks holiday your number of tests done is a lot different to previous month highlights an issue. Following month not on holiday number of tests done is a lot different to previous month highlights an issue. Brilliant system
Comment by Truth posted on
If you have been on holiday for a week that makes you a risk, as you will be rusty and probably still thinking about the sunshine when you return. It’s actually quite clever really.
Comment by John posted on
Are you for real, people tend to be worse before holiday
Comment by Richard posted on
"we are exploring the idea of predicting the result of the test for the vehicle being tested."
Why on earth would you want to do this, I've read why you want to try, but again why? I can't see this being benifical to any one.
Comment by FJW posted on
I can't see the reason for this either, other than to use it to go to DfT for staffing/funding requests. You can't predict how people look after their vehicles, particularly if they change ownership. I think many people would rather you invest in staffing ATF's.
As others have said, a slow month or a holiday changes your risk status, so what do you discuss with managers? We can't all fit into the 'national average' failure rates for all things. We test a variety of vehicles from new to old, and even 3 year old cars fail due to poor maintenance, and indeed the state of the roads. So is a good garage one that doesn't deviate from national averages? If i'm above average for say suspension components because the roads here are horrific and the average motorist sees maintenance (and even the MOT) as an anathema, does that make me a risk or am I enhancing road safety and preventing accidents?
Use of algorithms only gives an indication of risk, it doesn't replace good intelligence and ground work. Reliance solely on what a computer tells you just turns you into a drone.
Comment by Julia (DVSA) posted on
Hi there
We agree. Along with all the other factors, national averages are a starting point, and make the risk rating less good than it could be. As noted we are looking at improving the score to be based on the results one could expect based on the actual vehicles a tester has tested - so becoming a more realistic score.
The score just points us to garages of interest. We also still use the same intelligence and investigation approach we always have - this supplements it.
Comment by FJW posted on
I can’t see any benefit to this either, other than DVSA using it to go to DfT to ask for more money/staff. You can’t predict how owners will maintain their cars. I suspect the HGV/PCV trade would rather you invested in staffing ATF’s.
As others have said, have a slow month or have a bit of time off and your rating changes. So what are you meant to discuss with managers or trawl through your stats to discover? We can’t all be around the national average for all areas of the test. If I’m above average for let’s say suspension components, that doesn’t make me a risk necessarily, it could be because the roads are horrendous in my local area or vehicles aren’t maintained well. Most vehicle owners see maintenance (and indeed the MOT) as an anathema. Many 3 year old vehicles that come off fleets have been very poorly maintained. So, if I’m outside the national average, am I considered a risk or helping to improve road safety by finding these defects?
Algorithms are only so helpful and are no replacement for good intelligence and groundwork. Blindly following what is displayed on a computer screen just makes you a drone.
Comment by Julia (DVSA) posted on
Hi there
We'll never take disciplinary action on the basis of risk rating scores and they aren't intended to be a true measure of performance. What will happen is we'll investigate further on the basis of risk rating scores.
If we find evidence of serious misconduct, we'll take strong discplinary action. If we find evidence that a tester or station needs to improve their practices, we'll work with them to make that happen.
If we find that a tester or station is doing everything right, and there's a perfectly reasonable explanation for any results we see outside the national average, we'll take that into consideration to improve our risk rating system.
Comment by Roy Peverall posted on
Will I always be amber
Comment by Olivia (DVSA) posted on
Hi Roy, you need to contact our customer service team to discuss your individual rating, on 0300 123 9000.
Comment by Julia (DVSA) posted on
Hi Roy
If you look into your testing history and test quality information, you should be able to work out what's bringing your risk rating score down. That will help you to improve your score.
Comment by KW posted on
Your update does not mention any way of making contact with yourselves, or whomever it is that produces the risk ratings in order to understand why a particular rating has been given, or to amend it if it's in error.
Your comment " If you believe you are doing things correctly and have used all the tools available to you to establish this, then please carry on" does not really help.
You've also stated elsewhere that the ratings are not shared with managers, but you encourage testers to ask why their ratings are what they are with peers & managers.