The Toolbox Steering Group sees ongoing evaluation as an important part of a continuous improvement process. Some of these commissioned evaluations have focussed on the the management process, others on product usage. The reports are available for download below:
- 2007 Flexible Learning Toolboxes and learning objects evaluation
- 2006 Toolboxes and Learning Objects: Usage Trends in TAFE
- Trials of Learning Objects (2006)
- National Audit of Flexible Learning Toolboxes (2005)
- Indigenous Australian Delivery Trials of Flexible Learning Toolboxes (November 2005)
- Research initiatives of the Quality E-learning Resources project (June 2005)
- Developing Quality Online Materials: The flexible learning
- Enterprise Trial of Toolboxes Report (December 2004)
- Series 2 Rework Report (November 2004)
- National Audit of Flexible Learning Toolboxes (September 2004)
- Evaluation of the Usage of National Flexible Learning Toolboxes (2003)
- Toolbox Champions: The Human Face of Flexible Learning Toolboxes (2002)
- Evaluation of Equity Toolboxes (2002)
- Response to the Evaluation of Equity Toolbox Development (2002)
- Adapting Online Resources for Indigenous Learners (2002)
- Evaluation of the Usage of National Flexible Learning Toolboxes (2001)
- Implications of Toolbox Series 2 Usage Evaluation for 2002 (2001)
- An Evaluation of Series 1 Toolbox Usage - Report Summary (2001)
- Series 2 Development (Abridged) (2000)
- Series 1 Development (1999)
This evaluation is aimed at providing qualitative feedback on the impact of Flexible Learning Toolboxes (Toolboxes) on e-learning in vocational education and training (VET), as well as making recommendations to inform future Toolbox and learning objects development with a view of improving the uptake and usage of student e-learning through the use of Toolboxes and the Toolbox repository.
The research was conducted throughout 2007 with a focus on the teachers’ experiences with Toolboxes throughout 2006 and anticipated use in 2007. The evaluation was restricted to institutes of Technical and Further Education (TAFE).
In 2007, statistical data was collected from across Australia, evaluating the usage of Toolboxes and learning objects in 2006. This report examines the use of Flexible Learning Toolboxes and the Toolbox Repository through a survey of 2224 VET practitioners. The results indicate an overwhelmingly positive attitude towards the value of Toolboxes with more than 50 TAFE institutes using the resources. The attitude of respondents to the value of Toolboxes and the Learning Object Repository was overwhelmingly positive, highlighting the value these resources have for teacher/trainers and learners.
The purpose of the Trials of Learning Objects (Series 7) was to understand how teachers/trainers were accessing and using the Series 7 learning objects with learners. The Project worked with VTE Practitioners in the re-use and sharing of learning objects (learning objects) to explore the degree to which the current infrastructure supported this form of use and to investigate the factors impeding and supporting the development of effective learning settings using these technologies.
The project involved a number of VTE teachers and provided appropriate technical support to enable them to create online learning environments using learning objects as the fundamental building block. The project explored how learning objects were used to create quality learning settings and sought to discover how best these opportunities might be provided to other teachers in the Australian VTE sector.
Under the guidance and mentorship of Project personnel, 3 trials were conducted among VTE students in discrete units in Queensland, Tasmania and South Australia. This report describes the conduct of the project and discusses the outcomes that were observed through a detailed review of all stages.
This Audit examines the uptake and use of Toolboxes in 2004 and 2005, and expected use in 2006 to show:
- how Toolboxes have contributed to the sustainable uptake (new as well as sustaining existing uptake) of quality e-learning within the VTE system
- how Toolboxes have contributed to the Framework goal of providing ‘a flexible VTE system which meets diverse client needs and helps them succeed in a global environment’.
Monitoring of Toolbox usage will enable continuous improvement of resource development, expanding the VTE system’s knowledge and practice in this area.
This Final Report looks at the findings of the 2005 Audit on Toolbox Usage. An online survey was conducted to ascertain the use of Toolboxes within the various sectors of the VTE community (both public and private organisations). Information was sought on teacher and learner numbers, ways in which Toolboxes were being accessed and used within organisations, and information about the technical environments within the responding organisations. Feedback was also sought on respondent’s satisfaction with Toolboxes and the impact Toolboxes have had on learning outcomes and e-learning take-up within organisations.
The Indigenous Australian delivery trials of Flexible Learning Toolboxes (Toolboxes) were conducted in a range of geographical locations, diverse educational settings and with students of different ages, cultural backgrounds and language groups. The four locations were selected to gain insight into e-learning with Toolboxes in widely different Indigenous training contexts.
This presentation provides an overview of 2005 research activities being conducted by the Quality e-Learning Resources project (internally) and in collaboration with TAFE partners. Research activities include investigating SCORM 2004 and Toolboxes and the pedagogy of Learning objects.
Research initiatives of the Quality E-learning Resources project (shockwave in html - opens in new window - 585KB)
Since 1999, the Flexible Learning Toolbox initiative has been the model for a nationally co-ordinated approach in the development of online materials. The large scale and high profile of the Toolbox initiative provides an exemplar for the study of practice in the management and development of quality e-learning resources. A continuous improvement cycle has seen each series of Toolbox development build on the knowledge and understandings of previous Toolbox development and usage.
In October 2003, the Steering Group for the Resources for Teaching, Learning and Assessment Program requested that the learnings from the Toolbox experience (1999 to 2004) be documented and made available for the VET system.
As a result, this document has been developed to capture the knowledge, experiences, tools and techniques that have arisen from the Flexible Learning Toolbox initiative.
Compressed version Developing Quality Online Materials Report (Zipped Word file - 2.04MB) - Recommended faster download
Developing Quality Online Materials Report (Word - 4.47MB)
In 2004, the Flexible Learning Toolbox Project conducted an Enterprise Trial of Toolboxes. It aimed to provide a number of businesses with an opportunity to implement a Toolbox within a training program under supported conditions, and to capture their experience. Feedback from the Trial is now helping to improve e-learning professional development and support for industry and businesses.
Enterprise Trial of Toolboxes Report (Word - 1MB)
View case studies on each of the businesses involved in the trials:
- Hayman Island Resort (Queensland) (Word - 305KB)
- Hungry Jack's RTO (Tasmania) (Word - 303KB)
- Hungry Jack's RTO (South Australia) (Word - 305KB)
- Interwork Limited (South Australia) (Word - 308KB)
- Maxima Training Services (South Australia) (Word - 298KB)
- Territory Insurance Office (TIO) (Northern Territory) (Word - 306KB)
A summary of the changes and improvements made to create the updated (Version 2) Series 2 Toolboxes.
Series 2 Rework Improvements Report (Word - 74 KB)
This paper is a report of providers’ use of national Flexible Learning Toolboxes in 2003 and providers’ intentions for 2004. It outlines the story of Toolboxes in terms of their usage, uptake, impact on flexible learning in general and the distinct benefits of Toolbox development for Australia’s providers of vocational education and training (VET).
The purpose of the report is to identify:
- the extent of Toolbox use across Australia;
- the range of usage modes and the delivery conditions of these Toolboxes;
- the attitudes of providers to Toolboxes as training resources; and
- the impact that national funding of Toolbox development has had on the take-up and sustainability of e-learning in the VET sector.
National Audit of Toolboxes Report (Word - 2.16 MB)
National Audit of Toolboxes Report (PDF - 741 KB)
This document is a final report on an evaluation of usage of Toolbox Series 3 online learning products.
The objective of the evaluation was to gather and analyse patterns of acceptance and usage of Toolbox Series 3 through a survey of purchasers, and a focused field test. The evaluation looks at usage patterns, teacher and student acceptance, student learning, technical and organisational issues, and makes comparisons with Series 2 Toolboxes.
The report aims:
- to provide strategic advice to key stakeholders
- to determine improvements for the future development of Toolbox products
- to describe the implementation and deployment into training organisations.
Usage Report of Flexible Learning Toolboxes - Series 3 (Word - 1.9 MB)
This Evaluation of the Toolbox Champion Services aims to provide feedback and advice to the National Online Product Steering Group on the progress of the Toolbox Champions initiative in 2002, and to inform decisions on the enhancement of future Toolbox Champion Services.
Evaluation of the Toolbox Champion Services 2002 (Word - 754 KB)
An evaluation of development processes for the first group of Equity Toolboxes is presented in the report Evaluation of Equity Toolboxes 2002. The report describes development issues and solutions, and makes recommendations regarding online products for learners in the following groups: indigenous learners; learners with a disability; and learners with literacy & numeracy needs. The author of the report is Phoebe Palmieri.
Evaluation of Equity Toolboxes 2002 (Word - 313 KB)
A response to the recommendations in the Evaluation of Equity Toolboxes 2002 Report.
This response was prepared by the Online Product Steering Group and National Project Managers, indicating follow on action and implications for future practice.
Response to Evaluation of Equity Toolboxes 2002 (Word - 68 KB)
This report describes the experience of teams who adapted two Horticulture Toolboxes for Indigenous learners as part of the Equity Toolbox initiative 2001-2. It addresses a range of issues of interest to VET practitioners (including learning design for Indigenous ownership, consultation protocols, and cultural sensitivities concerning acknowledgement and permission).
Adapting Online Resources for Indigenous Learners (Word - 169 KB)
Evaluation of the Usage of National Flexible Learning Toolboxes (Series 2) - Executive Summary and Recommendations (2001)
Access Testing Centre conducted an evaluation of the usage of the Series 2 Toolboxes.
Evaluation of the usage of the Series 2 Toolboxes (Word - 41 KB)
The project team and Steering Group have prepared a response to the Evaluation Report.
Response to the Evaluation of the Series 2 Toolboxes (Word - 40 KB)
This project was undertaken to determine the impact of Series 1 Toolboxes since they became available in the market place in 1998 and 1999, the years in which they were developed. Information was obtained from over 130 individuals and organisations out of approximately 390 purchasers who have between them acquired over 890 Series 1 Toolboxes for trial and use.
Evaluation of Series 1 Toolbox Usage - Report Summary (RTF - 116 KB)
Evaluation of Series 1 Toolbox Usage - Report Summary (PDF - 33 KB)
This evaluation was conducted by Learning Network Queensland. Download the Evaluation Report. The report recommended a continuation of the project management processes used for this series, and the use of formative evaluation through a mentoring program in future developments.
Abridged Series 2 Evaluation Report (Word - 74 KB)
This evaluation was conducted by Queensland Open Learning Network. Download the Evaluation Report. The recommendations from this report were very influential in shaping Toolboxes from Series 2 onwards.
Series 1 Evaluation Report (PDF - 387 KB)
These documents are available in Portable Document Format (PDF) or Microsoft Word.
If you do not already have the Acrobat Reader to read PDF's you can download it for free from Adobe.