Have we improved public understanding of qualifications with a vocational purpose?

In reality we have to recognise that the public and government treat academic qualifications as ‘high risk’ and whenever there is a problem with marking or grading the spotlight shines intensely on such failings. However a qualification in construction or IT engineering may be just as high risk for the students who want to achieve recognition of their performance. The difference in passing or failing their award could be, in its own way, just as life-changing as it is for anyone doing a GCSE in French or a GCE in Economics.

Section 128 (1) (d) of the Apprenticeships, Skills, Children and Learning Act 2009 stresses the importance of Ofqual’s objective to increase ‘awareness’ of the qualifications system. It’s probably true to say that most people have an approximate idea of the form and function of GCSEs and GCEs and that there are some leading ‘examination boards’ who run the awards. The Annual Report catalogues 300 GCEs and 800 GCSEs offered by three organisations (in England).

The level of public perception and understanding and confidence in the 17,000 ‘other’ qualifications offered by 170+ AOs must make Ofqual’s promotion of awareness and understanding of vocational qualifications more difficult. There are awards, certificates and diplomas at each level in the QCF as well as entry level qualifications and higher level qualifications. There are over 9,000 other qualifications on the Qualifications Register which didn’t even register any achievement. The complexity of such a system is challenging and we it would be wrong to be complacent about taking steps to resolve this.

The cost of qualifications

The cost of the qualifications system increases, recently generating critical headlines in the press. The costs of GCSEs and GCEs are easily available (an average of £27 for a GCE, for example). However, Ofqual remarks, rather drily, that ‘we do not currently collect or collate fees for other qualifications, and analysis of their trends over time is complex’. This means we do not know how much we are paying for the vast range of other qualifications and we don’t know whether the qualifications represent value for money. Ofqual will want to see this situation changed and AOs need to be prepared to provide information and evidence to support the income they make from public examinations.

Should we design a different model?

Finally, we are regulating the system we have and perhaps not the system the country needs. At some point it might be useful to discuss a new model for vocational qualifications to take us forward into the future. Comments would be welcome -

The day started with a thought-provoking session on “User involvement in research: making sense of a radical new development (action research)” by Peter Beresford of Brunel University.  He looked at the different types of user research including: involvement of service users through consultation; collaborative or partnership approaches (where the researcher works with users to develop the research framework); and user-controlled research (where the users define the agenda, commission and / or conduct the research themselves).  He suggested that a different method is needed to evaluate the quality of user-involved research beyond the classic hierarchy used for instance by the NHS where values of neutrality, distance from the research, reproducibility of results etc. all rate highly.  User involvement could be seen as undermining these values despite the richness and insight that they can bring to a research project.

Thomas Haigh from the University of Wisconsin provided an overview of the use of techniques from historical research applied to the LIS domain.  This proved to be particularly interesting when considering the history of the profession and the services they provided.  Thomas gave a fascinating insight into the way in which schools of librarianship and information science departments drew together and how there is now a strand that can be seen as a part of Informatics.

He contrasted the United States where, for example, health informatics and chemical informatics have pooled their common experience and concepts resulting in the emergence of the iSchools, and Europe where informatics is derived from computer science and where LIS departments are becoming more closely allied with computing.

Mike Thelwell provided an introduction to webometrics and demonstrated some of the exciting work at Wolverhampton University to develop tools for harvesting data from the internet to create metrics such as connections between websites, and to analyse the sense of the content of websites, especially user generated content in comments pages, for instance.

Nick Moore’s paper rounded off a day of excellent sessions on research methods.  He gave a masterful insight into the history of policy making in the UK and the development of information policies around the world.  This highlighted the work of the Policy Studies Institute and provoked a reflection on how the lessons he has learned in a long and distinguished career could be applied to future research.  He ended on a forward-looking note by asking participants to identify what they thought were the important research issues coming up.  Ideas converged around the following themes:

• Who owns information?  The current models of publication and intermediation of information are breaking down.  User access to the publishing media means that there is an explosion of user generated content (UGC).  Do the current economic models still apply or will they break down as they have in the recording and digital video sectors?  How are intellectual property rights going to be managed – indeed are they still relevant?
• Privacy research about personal data and the way it is handled by information systems.  Changing attitudes to privacy and issues such as the “right to be forgotten” are emerging research themes.
• Evidence-based policy is becoming topical again.  Many remember the late 1990s when evidence-based policymaking was popular.  However Nick Moore did point out the tendency of established interests to pressurise researchers into policy based evidence-making (i.e. doing research to provide the evidence that supports a pre-determined conclusion).

Overall an interesting and worthwhile day which demonstrated the diversity of research and research approaches that can be applied to the LIS sector.

In earlier blogs I highlighted the dangers in allowing major global companies to acquire a dominant position with regard to the qualifications market. Last month in the TES, John Wood (chair of the Independent Schools Association) has argued that ‘aggressive commercialism’ has seriously undermined the integrity of the examinations system. I have much sympathy to his call to stop awarding organisations endorsing/publishing textbooks and running seminars designed to help teachers ‘play the system’.

Some people argued that the entry into the market of such major players might bring much needed investment and innovation into what had been described as a “19^th century cottage industry trying to deliver 21^st century expectations” (QCA, 2003). That may be so but it also brings with it dangers. Back then regulatory principles were based on the goals of social regulation. However, as the dual challenges of commercialism and globalisation have grown the issue of economic regulation has come to the fore, designed to address issues such as value for money and competition.

Yet, Ofqual have only begun to scratch the surface when it comes to economic regulation and the problem is that the supply of qualifications does not function like a market for utilities or for consumer goods. One critical feature of the qualifications ‘quasi-market’ is that the income earned by AOs comes mainly from the taxpayer through the fees paid for examination entry. This is very different from a market where informed customers are paying personally for products and services.

Qualifications are now so focused on ‘accountability’ measures and the functioning of league tables for schools. The use of qualifications for this purpose is entirely legitimate but questions have to be asked when the whole system is distorted in order to focus almost exclusively on this end. The purpose of a qualification as a ‘personal goal’ for a young person and a ‘public mark’ of their knowledge and competence seem to have been forgotten.

It is astonishing and perhaps depressing that 100 years after the recommendation to establish an ‘Examinations Council’ as the first qualifications regulator, the same issues of lack of clarity, uncertainty over standards and the question of the degree of control by the state remain major concerns. The high stakes nature of qualifications, their dominant role in education and training, the explosion in the number of awarding organisations and the rapidly increasing cost to the tax payer all make the effective regulation of qualifications an on-going matter of public debate.

It is clear that the year ahead is going to offer many challenges to the qualifications sector. Aspire Squared and our associates would like to wish everyone success and happiness in 2012.

Information management

• Introduction to metadata
• Metadata and digital repositories
• Developing a thesaurus
• Developing a file plan for records management

David Haynes the tutor for these events is author of Metadata for Information Management and Retrieval and brings his extensive practical experience of implementing information systems and advising clients on information retrieval issues to these courses.

For other courses in information management see www.buckleyowen.com

Research methods for service providers

Our recently launched series of research methods for service providers is primarily directed at library and information services and are available as on-site courses.  We can also tailor these workshops for other service sectors, such as education, social care, and professional bodies.  The following courses are now available:

• Evaluating service impact
• Using social media for research

David Haynes, David Streatfield and Noeleen Schenk are all experienced research practitioners who have developed and used a variety of social research techniques in the course of funded research, consultancy assignments and for public authorities.  See Eventbrite for details of the public courses.

Skills and Qualifications

Aspire Squared and its associates will be planning workshops in the skills and qualifications sector.  As well as public subscription courses, these can also be made available as bespoke sessions delivered on-site for individual organisations or groups of organisations.

• Risk Management and Awarding Organisations
• Meeting the new regulatory requirements
• Self-Evaluation for Awarding Organisations

If you are interested in any of our on-site training courses, or would like to discuss ways in which we can tailor our courses to your specific requirements contact us.

I have to confess that if I were faced with the General Conditions of Recognition (Ofqual, 2011) for the first time I would find myself knocked back. The challenge of regulation can indeed be daunting. The basic regulatory documentation runs to hundreds of pages.

Is it true that however that regulation inhibits innovation? The argument Glenys Stacey (CEO of Ofqual) made in her speech to the Associate of Learning Providers (ALP) on 21^st June is that Ofqual’s job is to ‘remove barriers to innovation for awarding organisations’. She flags up the establishment of the Innovation Advisory Group to oversee a research exercise to ‘identify drivers and barriers to innovation’.

As an example of innovation she highlighted the practice of ‘garra rufa skin exfoliation’, whereby tiny minnow-like fish are used to nibble dead skin from someone’s feet.  This may be an innovation in beauty therapy practice but I don’t think it’s an innovation driven by regulation. Anyway, animal welfare considerations and the potential risk to humans of cross-infection have led to the practice being banned in many countries.   Although I would question the example, I welcome the message in the speech to use regulation as a way of promoting innovation. 

It’s encouraging to see that Ofqual is going back to first principles with its innovation ‘stock-take’ exercise to explore what innovation might really mean for the awarding sector. Although innovation is flagged up as one of Ofqual’s statutory duties it needs to find a way of defining what constitutes successful innovation in awarding.

Regulation can be seen as encouraging innovation by creating a level playing field for large and small operators, so that the industry is not dominated by a few, large companies. Within a healthy and competitive market place regulation can help drive forward efficiency and cost savings. Can regulation stimulate innovation in the favour of an awarding organisation’s customers with more streamlined processes, a higher quality of support services or more valid and reliable assessment systems or would changes like these have happened anyway or in spite of regulation?

Interestingly, there is no mention of innovation in the fifty pages of frequently asked questions (FAQs) about Ofqual’s approach to regulation (published by Ofqual in July this year) so perhaps there is little appetite for innovation in the currently challenging business climate.  Yet challenging times can offer opportunities to do things differently and to innovate rather than ‘battening down the hatches’.

Do you think regulation is the enemy of innovation or are there ways in which regulation can stimulate innovation and drive forward progress in the awarding sector?  I’d welcome your comments.

For the full 8-page report see Social_Networks_Haynes_ 2011.

Risks

The results of the surveys showed that while individual users and potential users of social network services were concerned about protection of personal data, workplace respondents were primarily concerned with risk to the organisation.  Workplace respondents identified the following risks:

• Reputation risk to the organisation 
• Liability for the actions of people posting on the site
• Accidental disclosure of information that could lead to loss of intellectual property
• Security breaches by exposing the organisation to malware
• Non-compliance with the Data Protection Act and other regulations
• Time wasting during work

Individual users (and potential users) identified the following personal risks:

• Harassment (e.g. stalking)
• Identity theft and fraud
• Abuse of personal data by advertisers (e.g. spamming)
• Loss of privacy (where personal data is shared beyond the original intended audience)

Protecting personal data

A variety of measures was considered for protecting personal data on social networks and indeed several respondents suggested that more than one regulatory method would be needed:

• Educating users or providing guidelines on use of social networks were the most frequently mentioned precaution. 
• Monitoring and moderation of social network sites
• Technical measures such as software filters, or time-limited access
• Service providers taking greater responsibility for data security
• Personal responsibility for what personal data they reveal
• Social network providers should take responsibility for protecting privacy

Data Protection Act

Legislation is a major part of the regulatory landscape.  Many respondents felt that the UK’s Data Protection Act (DPA) was ineffective or only partially effective for protecting personal data on social networks.  One expert suggested in an interview that the Section 30 (domestic use) and Section 36 (freedom of the press) exemptions excluded social networks from the provisions of the Act.  Other concerns were about inability to enforce the Act where the social network service is outside the EU.  Ignorance about the provisions of the Act among users was also seen as a limiting factor.

In contrast, some respondents felt that the Data Protection Act was an effective tool for protecting personal data on social networks, especially in light of the Information Commissioner’s recently increased powers of enforcement.

Further work is proposed to look at how the legislation is applied in the workplace and to compare it to other means of regulating access to personal data.

David Haynes

August 2011

This new regulatory approach moves the regulatory focus from the accreditation of qualifications to the recognition of awarding organisations. The change of focus is likely to mean that awarding organisations will need to review all areas of governance, management and operations to ensure that they are ‘fit for purpose’ and meet the Ofqual requirements.

Ofqual staff will be visiting each awarding organisation to ensure that its new requirements are understood fully but they will not be able to give advice to AOs as they seek to respond to the new requirements.

It is perhaps unfortunate that the first of the regulator’s responses to frequently asked questions from AOs are to do with governance and the suitability for continuing recognition of an awarding organisation. When you get a question from an AO along the lines of ‘are some criminal convictions more serious than others’ or ‘what must we do if we have a Senior Officer with an unspent conviction’ you wonder whether you should read any further!

‘Conflicts of interest’ is another area which caused concern for AOs and the problem here is that an answer to the predicament of an awarding organisation does depend on the exact circumstances in which they find themselves. Sometimes Ofqual’ only available response is to refer the questioner back to the ‘Conditions and Criteria’.

Ofqual state that the ‘conditions and criteria’ are intended to specify WHAT awarding organisations must achieve not HOW they should achieve it. Many of the answers to the frequently asked questions posed by AOs include the response ‘it is for the awarding organisation to decide’ or ‘an awarding organisation must make a judgement’. The right answers will all depend on the circumstances in which the staff of the awarding organisation find themselves.

Some key terms are not defined by Ofqual. In the section on resources for example there is the reference to the ‘efficient’ development, delivery and award of qualifications. AOs ask Ofqual for its interpretation of ‘efficient’ and in the response Ofqual falls back on the term ‘being given its ordinary meaning in English’ – undertaken without wasted resources, effort, time or money  - if that is helpful!

The document on the Statutory Regulation of Qualifications published by QCA in 2000 was the first attempt at making regulation work for qualifications and the organisations which offer them. Often the decision making process was more ‘art than science’ and relied on staff at QCA getting to know and appreciate the ambitions and dedication of the awarding body as it did the awarding  body appreciating the regulatory framework. The new approach by Ofqual is very different but will raise many of the same issues encountered previously.

A number of the questions received from AOs were not about the ‘conditions and criteria’ but about the approach to regulation. ‘Taking Regulatory Action’ was published by Ofqual on 22 June 2011 and sets out how the regulator intends to apply its powers. The new publication creates a whole new set of uncertainties about how Ofqual intends to regulate and what the impact will be on awarding organisations which are struggling to survive the new wave of regulation.

One of the things I welcome is the requirement to continue with mathematics and English if a learner does not achieve a decent GCSE in those subjects by the age of 16. However, the nature of the maths and English qualifications must be relevant to the needs of those learners and enable them to progress. The worst thing that could happen is for young people who have struggled with GCSEs to have to sit the same qualifications again and again.

It does perplex me –as much as it astonishes most employers – that a school can take eleven years of a young person’s life between the ages of 5 and 16 and still not enable them to achieve acceptable levels of literacy ad numeracy.  The CBI has identified the lack of these skills as one of the most critical barriers to business success.

In the Executive summary of the Wolf Report one of the main issues identified is the ‘diet of low-level vocational qualifications, most of which have little or no labour market value’. Among 16 – 19 year olds, the Review estimates that at least 350,000 get little or no benefit from the post-16 education system (Wolf Report, 2011).

Chris Jones of City & Guilds gets it right when he stresses that whilst dead-end courses need to be cut ‘it is important that we differentiate between these and high-quality craft or vocational qualifications which are valuable to both students and to employers, and will play a big role in rebalancing and growing our economy’ (Chris Jones in a letter to The Telegraph, Tuesday 17^th May 2011).

The Register of Regulated Qualifications lists 798 GCSEs and GCE ‘A Level’ qualifications. A search of the Register for vocational qualifications – NVQs, Vocationally Related Qualifications and those accredited within the Qualifications and Credit Framework – produces a list of almost 9,000. Many of these will be the low level awards criticised in the Wolf Report and should never have been included in the Register.

Mr Gove is looking for qualifications to incorporate external assessment, provide progression opportunities, be recognised by employers and HE, offer rigour breadth and depth, and constitute an appropriate size. The question at the top of my mind is how Ofqual is going to ‘identify the best vocational qualifications’ if it regulates at awarding body level and not qualification level. What criteria is Ofqual going to use to get rid of dud VQs and confirm those which measure up to the requirements listed by the Secretary of State?

The new arrangements for regulation have just been published by Ofqual – ‘The General Conditions of Regulation’. Awarding bodies, and others, will be scrutinising the balance between the requirements for the awarding body (Part 1) and the requirements for the ‘regulated qualification’ (Part 2). Let’s see how they react. With the background of the Wolf Report we can then assess whether the new Conditions are going to move us from rhetoric to reality.

If you use social networks, you trade personal information in return for access to services.  Personal data from your social network profile may be available to:

• all Internet users;
• other subscribers to the service;
• your designated direct contacts (or friends); or
• premium users who pay for privileged access to personal data (a feature used by recruitment agencies and large employers)

When you sign up to a service, the consequences of revealing personal information is not likely to be top of your agenda.  Often, it seems there are few controls over how your personal information might be used or who has access to it.

Some attempts have been made to regulate access to personal data on the Internet.  The Information Commissioner’s Office in the UK operating under the mandate of the Data Protection Act attempts to regulate the gathering, use (processing) and access to personal data, as well as its eventual disposal. However if the service provider is not in the UK enforcing the Act is irrelevant.  Even schemes such as the Safe Harbour are inadequately policed and afford little protection to users[1].

Territorial limitation of national legislation makes regulation of the Internet difficult. This is compounded by the multitude of access points, the global reach of services, and the discrepancies between national approaches to data protection.

Multi-national regulation such as the European Union’s Data Protection Directive is essentially territorial and faces many of the same limitations of national legislation.  Even international rules such as the OECD Guidelines on the Protection of Privacy and Transborder Flows of Personal Data have not been implemented fully by its members. 

If we think of the Internet as a place, perhaps we can begin to consider ways of developing more effective regulation.  We need to explore ways in which the Internet environment behaves and is managed.  Lawrence Lessig in his book Code 2.0 predicts that the Internet will become more tightly regulated as it evolves to reflect government and commercial values.  He suggests that local regulation of behaviour on the Internet will become more prevalent.   Jonathan Zittrain on the other hand suggests that the Internet will effectively be regulated by the community of people that use it.  Reflecting the values of the user community certain types of behaviour will become unacceptable and offenders will be excluded (by technology for instance) or ostracised.  If that begins to happen, then perhaps we can truly say that the Internet is a place with its own laws and regulation.

Sources:

Data Protection Act 1998. C29.

European Parliament. Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data.

Lessig, L. Code version 2.0. 2006.

OECD Guidelines on the Protection of Privacy and Transborder Flows of Personal Data, 1980. Paris:  OECD Publishing, 2002.

US Department of Commerce. US-EU Safe Harbor Framework.

 Zittrain, J. The Future of the Internet. London: Penguin Books, 2009

1. Resource description
2. Information retrieval
3. Management of information
4. Rights management, ownership and authenticity
5. Interoperability and e-commerce

These purposes were based on an understanding of the main ways in which metadata was being a few years ago and anticipating the rise in e-commerce and e-government. 

At that time most of the discussion about Web 2.0 was focused on the semantic web.  Since then the discussion has moved on to social networking, with the rise of services such as FaceBook, Twitter and LinkedIn. This calls for a re-think of the purposes of metadata and the following modified list represents my current thinking:

1. Resource description and identification – the form and existence of information resources.  Includes identifiers and used in applications such as catalogues, RFID codes, book and article numbering (DOIs and ISBNs).  Authenticity of items may also be established with this type of metadata
2. Retrieval and Dissemination – the most familiar purpose of metadata to make a resource discoverable.  Typically includes indexing and categorisation (classification) data.  Standards such as Dublin Core focus primarily on this purpose as does a lot of the metadata used by institutional repositories.  Can also be used proactively to distribute or disseminate information to target audiences.
3. Preservation and Retention – any material or information resource in a collection whether it be physical or electronic needs to be preserved and retained in a controlled way if it is to be available and usable by its intended audience.  Some metadata standards such as PREMIS are all about preservation of electronic resources, while in other standards, specific data elements are used to deal with preservation issues.
4. Users and agents – this is metadata about users (e.g. audience), and managing the usage of a resource –e.g. loan of materials by a library (circulation control).  This can also be used for determining who has access to a resource – where security or confidentiality are considerations
5. Ownership and rights – ownership of intellectual property rights is important for commercial systems such as iTunes where electronic resources are made available for a fee.  This provides a way of ensuring that all elements in the supply chain get paid.  It is also used for capturing rights statements such as intellectual property rights

I have removed the interoperability purpose because this is more about a way of doing things than a purpose.  Management of information is too general and is covered by the more specific areas such as preservation, usage and ownership.

Ian Beeson (2006) in Metadata Overload advocates four different levels of analysis: content, composition, meaning and use of documents.  I have chosen to describe metadata in terms of its use, because I believe that reflects the experience of most people who come across metadata.  I think that his analysis is sound, although I would take issue with his distinction between human use and machine or system use of metadata.  Inasmuch as machines are proxies for people, it should be possible to analyse machine use of metadata in the same framework as the purposes of metadata that I have described above.

Sources

Beeson, I. (2006) Metadata Overload. Higher Education Academy

Gilliland-Swetland, A. (1998) Defining Metadata (in Baca, M (Editor) Introduction to Metadata: pathways to digital information) ISBN 0-89236-533-1. Getty Information Institute)

Haynes, D. (2004) Metadata for Information Management and Retrieval. Facet Publishing