Reports

The Toolbox Steering Group sees ongoing evaluation as an important part of a continuous improvement process.  Some of these commissioned evaluations have focussed on the the management process, others on product usage.  The reports are available for download below:

2007 Flexible Learning Toolboxes and learning objects evaluation

This evaluation is aimed at providing qualitative feedback on the impact of Flexible Learning Toolboxes (Toolboxes) on e-learning in vocational education and training (VET), as well as making recommendations to inform future Toolbox and learning objects development with a view of improving the uptake and usage of student e-learning through the use of Toolboxes and the Toolbox repository.

The research was conducted throughout 2007 with a focus on the teachers’ experiences with Toolboxes throughout 2006 and anticipated use in 2007. The evaluation was restricted to institutes of Technical and Further Education (TAFE).

Download the '2007 Flexible Learning Toolboxes and learning objects evaluation' report in PDF (147KB)

2006 Toolboxes and Learning Objects: Usage Trends in TAFE

In 2007, statistical data was collected from across Australia, evaluating the usage of Toolboxes and learning objects in 2006. This report examines the use of Flexible Learning Toolboxes and the Toolbox Repository through a survey of 2224 VET practitioners. The results indicate an overwhelmingly positive attitude towards the value of Toolboxes with more than 50 TAFE institutes using the resources. The attitude of respondents to the value of Toolboxes and the Learning Object Repository was overwhelmingly positive, highlighting the value these resources have for teacher/trainers and learners.

Download the 2006 Toolboxes and learning objects: Usage trends in TAFE report in PDF (421KB)

Trials of Learning Objects (2006)

The purpose of the Trials of Learning Objects (Series 7) was to understand how teachers/trainers were accessing and using the Series 7 learning objects with learners. The Project worked with VTE Practitioners in the re-use and sharing of learning objects (learning objects) to explore the degree to which the current infrastructure supported this form of use and to investigate the factors impeding and supporting the development of effective learning settings using these technologies.

The project involved a number of VTE teachers and provided appropriate technical support to enable them to create online learning environments using learning objects as the fundamental building block.  The project explored how learning objects were used to create quality learning settings and sought to discover how best these opportunities might be provided to other teachers in the Australian VTE sector.

Under the guidance and mentorship of Project personnel, 3 trials were conducted among VTE students in discrete units in Queensland, Tasmania and South Australia.  This report describes the conduct of the project and discusses the outcomes that were observed through a detailed review of all stages.

View the full report of the Trails of Learning Objects in Word (3.47MB)

View the full report of the Trails of Learning Objects in PDF (2.5MB)

National Audit of Flexible Learning Toolboxes (2005)

This Audit examines the uptake and use of Toolboxes in 2004 and 2005, and expected use in 2006 to show:

Monitoring of Toolbox usage will enable continuous improvement of resource development, expanding the VTE system’s knowledge and practice in this area.

This Final Report looks at the findings of the 2005 Audit on Toolbox Usage.  An online survey was conducted to ascertain the use of Toolboxes within the various sectors of the VTE community (both public and private organisations).  Information was sought on teacher and learner numbers, ways in which Toolboxes were being accessed and used within organisations, and information about the technical environments within the responding organisations.  Feedback was also sought on respondent’s satisfaction with Toolboxes and the impact Toolboxes have had on learning outcomes and e-learning take-up within organisations.

View the full report of the National Audit of Flexible Learning Toolboxes in Word (1.95MB)

View the full report of the National Audit of Flexible Learning Toolboxes in PDF (1MB)

Indigenous Australian Delivery Trials of Flexible Learning Toolboxes (November 2005)

The Indigenous Australian delivery trials of Flexible Learning Toolboxes (Toolboxes) were conducted in a range of geographical locations, diverse educational settings and with students of different ages, cultural backgrounds and language groups. The four locations were selected to gain insight into e-learning with Toolboxes in widely different Indigenous training contexts.

View the full report of the Indigenous Australian Delivery Trials is available in PDF (848KB) or Word (692KB).

Back to top ^

Research initiatives of the Quality E-learning Resources project (June 2005)

This presentation provides an overview of 2005 research activities being conducted by the Quality e-Learning Resources project (internally) and in collaboration with TAFE partners. Research activities include investigating SCORM 2004 and Toolboxes and the pedagogy of Learning objects.

Research initiatives of the Quality E-learning Resources project (shockwave in html - opens in new window - 585KB)

Back to top ^

Developing Quality Online Materials: The flexible learning Toolbox experience (December 2004)

Since 1999, the Flexible Learning Toolbox initiative has been the model for a nationally co-ordinated approach in the development of online materials. The large scale and high profile of the Toolbox initiative provides an exemplar for the study of practice in the management and development of quality e-learning resources. A continuous improvement cycle has seen each series of Toolbox development build on the knowledge and understandings of previous Toolbox development and usage.

In October 2003, the Steering Group for the Resources for Teaching, Learning and Assessment Program requested that the learnings from the Toolbox experience (1999 to 2004) be documented and made available for the VET system.

As a result, this document has been developed to capture the knowledge, experiences, tools and techniques that have arisen from the Flexible Learning Toolbox initiative.

Compressed version Developing Quality Online Materials Report (Zipped Word file - 2.04MB) - Recommended faster download

Developing Quality Online Materials Report (Word - 4.47MB)

Back to top ^

Enterprise Trial of Toolboxes Report (December 2004)

In 2004, the Flexible Learning Toolbox Project conducted an Enterprise Trial of Toolboxes. It aimed to provide a number of businesses with an opportunity to implement a Toolbox within a training program under supported conditions, and to capture their experience. Feedback from the Trial is now helping to improve e-learning professional development and support for industry and businesses.

Enterprise Trial of Toolboxes Report (Word - 1MB)

View case studies on each of the businesses involved in the trials:

Back to top ^

Series 2 Rework Report (November 2004)

A summary of the changes and improvements made to create the updated (Version 2) Series 2 Toolboxes.

Series 2 Rework Improvements Report (Word - 74 KB)

Back to top ^

National Audit of Flexible Learning Toolboxes (September 2004)

This paper is a report of providers’ use of national Flexible Learning Toolboxes in 2003 and providers’ intentions for 2004. It outlines the story of Toolboxes in terms of their usage, uptake, impact on flexible learning in general and the distinct benefits of Toolbox development for Australia’s providers of vocational education and training (VET).

The purpose of the report is to identify:

National Audit of Toolboxes Report (Word - 2.16 MB)

National Audit of Toolboxes Report (PDF - 741 KB)

Back to top ^

Evaluation of the Usage of National Flexible Learning Toolboxes (Series 3) (2003)

This document is a final report on an evaluation of usage of Toolbox Series 3 online learning products.

The objective of the evaluation was to gather and analyse patterns of acceptance and usage of Toolbox Series 3 through a survey of purchasers, and a focused field test. The evaluation looks at  usage patterns, teacher and student acceptance, student learning, technical and organisational issues, and makes comparisons with Series 2 Toolboxes.

The report aims:

Usage Report of Flexible Learning Toolboxes - Series 3 (Word - 1.9 MB)

Back to top ^

Toolbox Champions : The Human Face of Flexible Learning Toolboxes (2002)

This Evaluation of the Toolbox Champion Services aims to provide feedback and advice to the National Online Product Steering Group on the progress of the Toolbox Champions initiative in 2002, and to inform decisions on the enhancement of future Toolbox Champion Services.

Evaluation of the Toolbox Champion Services 2002 (Word - 754 KB)

Back to top ^

Evaluation of Equity Toolboxes (2002)

An evaluation of development processes for the first group of Equity Toolboxes is presented in the report Evaluation of Equity Toolboxes 2002. The report describes development issues and solutions, and makes recommendations regarding online products for learners in the following groups: indigenous learners; learners with a disability; and learners with literacy & numeracy needs. The author of the report is Phoebe Palmieri.

Evaluation of Equity Toolboxes 2002 (Word - 313 KB)

Back to top ^

Response to the Evaluation of Equity Toolbox Development (2002)

A response to the recommendations in the Evaluation of Equity Toolboxes 2002 Report.

This response was prepared by the Online Product Steering Group and National Project Managers, indicating follow on action and implications for future practice.

Response to Evaluation of Equity Toolboxes 2002 (Word - 68 KB)

Back to top ^

Adapting Online Resources for Indigenous Learners (2002)

This report describes the experience of teams who adapted two Horticulture Toolboxes for Indigenous learners as part of the Equity Toolbox initiative 2001-2.  It addresses a range of issues of interest to VET practitioners (including learning design for Indigenous ownership, consultation protocols, and cultural sensitivities concerning acknowledgement and permission).

Adapting Online Resources for Indigenous Learners (Word - 169 KB)

Back to top ^

Evaluation of the Usage of National Flexible Learning Toolboxes (Series 2) - Executive Summary and Recommendations (2001)

Access Testing Centre conducted an evaluation of the usage of the Series 2 Toolboxes.

Evaluation of the usage of the Series 2 Toolboxes (Word - 41 KB)

Back to top ^

Implications of Toolbox Series 2 Usage Evaluation for 2002 (2001)

The project team and Steering Group have prepared a response to the Evaluation Report.

Response to the Evaluation of the Series 2 Toolboxes (Word - 40 KB)

Back to top ^

An Evaluation of Series 1 Toolbox Usage - Report Summary (2001)

This project was undertaken to determine the impact of Series 1 Toolboxes since they became available in the market place in 1998 and 1999, the years in which they were developed. Information was obtained from over 130 individuals and organisations out of approximately 390 purchasers who have between them acquired over 890 Series 1 Toolboxes for trial and use.

Evaluation of Series 1 Toolbox Usage - Report Summary (RTF - 116 KB)

Evaluation of Series 1 Toolbox Usage - Report Summary (PDF - 33 KB)

Back to top ^

Series 2 Development (Abridged) (2000)

This evaluation was conducted by Learning Network Queensland. Download the Evaluation Report. The report recommended a continuation of the project management processes used for this series, and the use of formative evaluation through a mentoring program in future developments.

Abridged Series 2 Evaluation Report (Word - 74 KB)

Back to top ^

Series 1 Development (1999)

This evaluation was conducted by Queensland Open Learning Network. Download the Evaluation Report. The recommendations from this report were very influential in shaping Toolboxes from Series 2 onwards.

Series 1 Evaluation Report (PDF - 387 KB)

Back to top ^

These documents are available in Portable Document Format (PDF) or Microsoft Word.

Get Adobe Acrobat ReaderIf you do not already have the Acrobat Reader to read PDF's you can download it for free from Adobe.

Back to top ^