He's certainly changed that now. He was hired by John Denham, Skills Secretary, to lead Labour's drive to improve science and research - effectively becoming the joint-second highest official at the newly-formed Department for Innovation, Universities and Skills (DIUS).
It followed a career in academia spanning more than 30 years. A former professor of statistics at Nottingham University, he has held posts at Oxford and University College London. He's a past president of the Royal Statistical Society and was elected a Fellow of the Royal Society in 2001. Before joining DUIS last September, Professor Smith had already been close to the Labour administration.
In 2003/04 he led an inquiry into mathematics education for teenagers and in 2006 completed a report for the Home Office on the issue of public trust in crime statistics.
The maths inquiry indicated that he was willing to speak his mind. At the time of its publication, he said the country had "shot itself in both feet" by failing to reform A-levels properly. In an echo of comments made this week, he also said reforms demoralised the less able and failed to stretch the very brightest.
He hits out at 'schizophrenic' diplomas, education for the 'masses' at the expense of the most able, and ministers' exaggerated claims for science. The government should focus on getting GCSEs and A-levels right before it introduces "schizophrenic" new diplomas, warned Professor Smith this week. He also suggested ministers were educating "the masses" at the expense of equipping the most able.
Professor Smith, second only to the permanent secretary at the Department for Innovation, Universities and Skills (DIUS), warned that ministerial rhetoric was exaggerating improvements in science education and that "insidious" health and safety legislation was stifling scientific curiosity in schools.
The Government's former maths tsar, who joined DIUS in September 2008, suggested golden hellos for trainee teachers would be better spent on higher salaries for teachers of certain subjects.
On the new work-related diplomas, Professor Smith, said: "In core subjects like maths and physics we already have a shortage of qualified teacher cover. Are we wise in adding different bits of curricular offerings, each of which will require additional teacher input?
"Are we thinking in a joined-up way when we plan curriculum developments and new programmes, whether we have the teacher power, planning and recruitment? Might we not be better getting GCSEs and A-levels right first?"
He described the new science diploma as a "slightly schizophrenic" concept trying to challenge A-levels and offer work-based learning.
He also said universities were saying "we won't touch" the Government's new A* A-level grade because they felt it favoured independent schools.
In the annual Tribal education lecture, he warned: "We have a tension in the education system. We are educating everybody - the masses - for citizenship, for (mathematical) competences and functionality.
"Higher education and the innovation and high tech industries of the future involve those at the end of the spectrum who are capable of achieving and aspiring to more professional levels of mathematics.
"There are still serious questions in the system about whether we have really cracked that balance."
Schools Secretary Ed Balls defended all three academic diplomas: "The proposed content of the science diploma is significant and ambitious, and through the consultation we will want to consider whether all the benefits can be realised through a single science diploma."