eLearning Design Lab

The e-Learning Design Lab Strategic Plan Legacy Document August of 2006

NOTE: The intent of the “Legacy Document" was to serve as a resource that captures the essence of the responses to the survey questions and the subsequent planning sessions. The legacy document served to inform the planning process. Only those sections in the original document that have been incorporated into the strategic plan have been deleted.

Visibility of e-Learning Design Lab (eDL)

The lab has achieved a level of visibility on campus that is reflected by the amount of contacts the lab receives from groups and individuals seeking consultation on the development of online instruction or who want to interact on topics of mutual interest. Nationally, visibility has evolved from the products that have been disseminated, articles published, professional presentations, and the conferences and seminars sponsored by the lab early in its history. As decisions are made regarding future directions and core participants in the lab, the initiatives to communicate what the lab does and the contributions that it makes will need to be revisited and commitments made to these activities. Central to creating the conditions that make such activities warranted and effective is the refinement of what we are, clarification of our foci for R&D, the identification of appropriate funding agencies, and an expansion of collaborators.

Intellectual Property: Because the eDL was one of the first units on campus to become involved in the development of online instruction and e-learning in the form of intellectual property, the lab stimulated the need for policies and/or became the first unit to experience the technology transfer process through the Kansas University Center for Research. While there were and continue to be concerns related to the process, the outcome was selected intellectual properties were licensed by KUCR as a start-up company. The lab was also one of the first e-learning units on campus to go through invention disclosure. Those individuals involved with the lab who go engage in these processes are now more experienced and the lab is better positioned as new opportunities emerge related to technology transfer.
External Proposals: The lab has submitted over 22 proposals to federal and state agencies with ten being funded by ten different organizations. Working with different project officers from different agencies adds to the knowledge base which also builds the credentials of the lab. Early in its history the lab was funded largely by the Office of Special Education Programs (OSEP). Today the lab only has one award from this agency. The eDL has engaged in collaborative activities with several private companies. Of the four relationships that were established, three are continuing at the present time.
Consultation: Over its history, the lab has evolved as one of a small number of units on campus where academic departments and/or research units seek consultation on the design, development and distribution of e-learning programs. This has contributed to the visibility of the lab on campus. The lab has not sought to become a service unit on campus; however, by virtue of the work that is does, there is an assumption on the part of many that it does offer services. Other than consultation and acting as a source of information, the lab has only undertaken a small number of collaborative on-campus production products. They have been on a contractual basis.
Conferences: During the first three years of the lab’s history, conferences were sponsored on campus. They attracted a national audience and gave visibility to online instruction on campus. Subsequent seminars were sponsored by the lab as well as brown bag sessions. In recent years, these have become less necessary due to the many conferences that are held nationally. There may now be a need for such a role that relates to bringing together individuals specifically interested in interdisciplinary R&D related to e-learning.
Content/Production Processes: The lab may be unique as a unit in an academic setting that has experience in moving from the development of large amounts of content through the production of e-learning products designed to go to scale. The process that the lab has both developed and validated represents a number of significant products.

Suggested Initiatives for Enhancing the Visibility of the Lab

Ensure that we have full documentation on the products we produce and that we take the creation of tools through the complete process of readying them for external use and protected for technology transfer.
Establish more direct relationships with researchers affiliated with the Information and Telecommunication Technology Center (ITTC) and Center for Research on Learning (CRL) to determine opportunities for collaboration and/or products that might be enhanced by dissemination through the lab. The GIST by Keith Lenz has been cited as an example along with the SIM products of the CRL.
Reinstate our seminars and brown bag luncheon sessions. Consider topical conferences that appeal to others nationally with similar research and development interests.
Increase our efforts in publishing manuscripts that document the processes, products, research, issues of concern to the lab and tools developed by the lab.
Arrange for project officers from agencies such as NSF, DARPA, IES, OSEP, and FIPSE to visit campus and/or to participate in events that we might sponsor on campus.
Develop our web site to more accurately and effectively communicate what the lab has done or is doing.
Develop and make public the lab research agenda.
As participants, faculty can be more proactive in communicating their affiliations with the lab.

Strengths of the eDL:

The legacy of work related to online instruction at the postsecondary and staff development levels, including the focus on asynchronous instruction, tool design, content generation and management processes, and the collaborative relationships extend across departments and research units including Education, Engineering, Computer Science, Design, the ITTC, and the CRL. The lab has a record of work and a history of relationships with an R&D focus along with a demonstrated ability to be competitive for external funds within and external to the University. This history provides a context for assessing its strengths

Collaborative Projects: The lab has a history of state educational agencies, other institutions, academic departments and research units within the university, that have approached the lab for consultation. This has occurred over the years even though the eDL does not advertise in ways that imply it is seeking work. The outcome has been a continuing dialogue with many units and specific collaborative projects with some.
Culture of the eDL: The building of a culture that has resulted in GRAs being attracted to the lab for opportunities to become engaged in research and development experiences. The outcome has been these students have stayed with the lab and become the primary talent pool for the production work and significant contributors to design and researchable ideas.
Commitment of the Core Group: The commitment of different individuals to the evolution of the lab during their tenure as participants and their continued interest and participation in the lab, even though they the have left the institution for professional opportunities, assumed other responsibilities, or in the case of students, graduated.
Early Outreach Activities: The early outreach activities of the lab that were initiated and implemented in the area of e-learning including the hosting of national conferences, campus lectures and seminars, brown bag luncheon sessions, and the participation of faculty on committees at all levels.
ITTC and CRL Partnership: The success in building a collaborative effort of a sustainable nature across two research units. The seed money provided by the ITTC and the CRL has been effectively used in strategic ways to maintain the initiative and to secure grants and contracts. This can serve as an example of a successful internal partnership and investment. While the future of the lab is dependent on new directions, expanded participation, and initiatives, it has been successful on a per participant basis in generating external funds to support work within its mission.
Interdisciplinary Collaboration: The ITTC-CRL partnership represents an example of interdisciplinary collaboration that is central to the institutional commitment to research and consistent with the emerging funding policies and practices of external funding agencies. The partnership has provided a level of credibility that has been helpful to the evolvement of the lab. Both sponsors have been most facilitating when needed, while not imposing direction or structure on the lab.
Talent Base: The talent base of the lab, including the faculty participants, staff and students, is a significant strength of the lab. The lab has been fortunate to have attracted talented students whose expertise and skills will serve them well in changes that may evolve from the strategic planning process.
Tenure of the Professional Staff: The tenure of the professional staff offers a particular strength. They have been with the lab since its beginning and are familiar with all aspects of the lab. Both staff members serve on the management team and will be a major asset in transitioning to new leadership and/or mission direction.
Diversity of Staff and Students: The culture that has been achieved within the life of the lab has resulted in relationships among all participants including students that contribute to a team process essential to the kind of work that is done. There is diversity among members of the lab across disciplines, personal backgrounds, ethnicity, career ambitions, areas of expertise and gender that combine to create a productive environment. The culture is enhanced by the quality and physical design of the space in which the lab is housed.
Products, Tools and Knowledge Contributions of the eDL: In looking to the future, it is important to note what the eDL has achieved in the form of products. These products, tools, dissertations and theses need to be inventoried and described as part of the strategic planning process as they represent the work of the lab. They also serve as indicators of our “Core Competencies“ that under gird our mission. As directions are set for the future, these core competencies will likely be applicable but not sufficient. For example, we have not developed a core competency in the area of simulations even though some work on IKME was in this area.
Tools: The eDL has produced a number of different tools that have been used in the production of e-learning resources. While the tools are in the form of authorizing systems and have gone through a number of versions that have potential for future development, they have not been disseminated in a marketable form. They continue to be modified depending on client need.
Content: The content reflected in the many modules and courses that have been developed by the eDL represents a unique body of content structured for easy conversion to learning objects. The RLO project will result in a model for disaggregating the modules into a repository of learning objects and a tool for reconfiguring objects as instructional units applicable to staff development and/or pre-service. The model will generalize to other bodies of content as well as to the generation on learning objects. This is an example of how residual products can form the basis for subsequent work.
e-Book: The e-book design and authoring system that was created for release of an e-book in February of 2006, represents another example of building on a legacy designed to create new systems and/or tools. The e-book authoring system can be applied to content in a wide array of disciplines. The design draws from the online instructional designs of the lab, adds new features and modifies prior features to result in an e-book design that is different from what is typical of the e-book market.
Dissemination: The extranet model that was developed during the Online Academy and employed in making subsequent products available nationally has been well tested and proved to be efficient and effective. The model, along with the procedures and processes, comprise a set of experiences that represent an important capacity. Over 180 institutions of higher education have entered into formal agreements with the eDL. We continue to receive inquiries on accessing those products. A similar model has been implemented with state educational agencies.
Manuscripts: The lab has made an investment in publishing manuscripts that describe the validated processes that have been designed and employed by the lab on topics such as content generation and management, instructional design, beta testing, needs assessment and dissemination. This effort, combined with the over 50 online instructional modules/courses that have been developed and disseminated, contribute to building the knowledge base as well as a national identity for the lab.
Dissertations and Theses: A significant factor in the long term contribution of the lab to the knowledge base are the theses and dissertations completed by students who have either worked in the eDL, been advised by faculty affiliated with the lab or supported in some manner by the lab. To date, 4 theses and 10 dissertations have been completed. These individuals are likely to continue their research and teaching in areas related to e-learning and their contributions to the literature will continue impart due to the influence of the eDL on their work. We may want to consider kuscholarworks.ku.edu as a repository for eDL publications.

Weaknesses of the eDL:

Just as strengths are largely defined by perspectives, the same is true of weaknesses. Building a strength in a small organization means that something else does not receive attention sufficient to also make it a strength. That is the case faced by the eDL. What may now be perceived as a weakness in the future could become a strength if it is viewed as central to the future direction of the lab and efforts are invested in the process of making it a strength.

Postsecondary Emphasis: While most of the content central to the products and tools developed by the lab has been related to teacher education, the mission of the lab has been focused more on postsecondary education or adult learners in general. However, the funding opportunities that have been available and pursued have been heavily oriented toward teacher education. When the lab sought and successfully obtained funding from non-teacher education sources such as the U.S. Army, it has not been as diligent as it could or should have been in generalizing the resulting tools and processes to populations such as teachers. In the future all projects should intersect as a refinement of existing e-learning technology or, when focusing on new technologies, develop a plan to determine the generalization of results and designs to other populations of learners.
Lab Oriented Projects: There has been a tendency to focus more on projects central to the eDL than creating conditions that encourage individual participants to obtain support for their own work that is aligned with the mission of the lab. This may be part of the maturing process of an R&D unit or an oversight, but the lab has not grown as much through the individual efforts of participants as through collective efforts. This may have a limited effect on broadening collaboration. If more individual efforts are encouraged and supported, those efforts might bring a wider array of collaborators to the lab.
Few Flexible Research Dollars: There has been a lack of sufficient resources for the lab to move into areas of research for which there is currently no source of support. This has the effect of diminishing opportunities for innovations, new discoveries and more basic research on factors contributing to effective e-learning.
Lack of Planning Time: The daily activities of the lab, in terms of the time that participants have to contribute the lab, have centered on carrying out funded projects or seeking support for research and development. That has resulted in less time being devoted to long term planning. In a field that is changing as quickly as e-learning, that can mitigate against the generation of new ideas and engagement in innovative initiatives. Every effort should be made to capitalize on this strategic planning activity and to establish a practice of devoting more time to the generation of new ideas and innovative activities.

Opportunities for the eDL:

Central to the strategic plan are decisions regarding the various opportunities the lab should pursue. Some R&D opportunities are systemic to the evolution of a new research area such as e-learning. These are also emerging opportunities that are defined by funding agencies that become known but that may not necessarily be appropriate. However, because they are presumed to be important by the agency warrants their review.

New Tools and Design: Develop new versions and generations of tools and designs for postsecondary and staff development levels that represent a natural extension of the work that has systematically been conducted by the eDL. Create new designs for the online teaching process while remaining committed to rich and validated content.
Lessons Learned: Take advantage of what we have done and expedite our efforts to disseminate lessons learned. Based on what has been learned from the models we have developed, tested and implemented, move that work into other designs to study the effects of a wider use of media in e-learning instructional environments.
External R&D Participants: Increase the focus of the lab on tool development and research building on the skills and expertise of colleagues from Engineering and Computer Science. Expand the participation of colleagues from these fields as direct participants and individual investigators in the lab and as collaborators in the instructional development arena.

Suggested Initiatives for Expanding the Focus of the eDL

Develop a closer working relationship with researchers in the CRL and the ITTC to determine where interests intersect. Particular attention could be given to not only new technologies and a greater emphasis on learning research, but to how e-learning might contribute to the dissemination of research results as well.
Consider creating a mechanism within the lab for increasing the emphasis on the lower ranges of the life span (e.g., early child hood or school age). Such a mechanism might be a division or focus area. Attention could be given to assistive technologies, new tools, instructional designs and/or environments. The emphasis could be on teachers, caregivers or other professions in addition to learners.
Assess lessons learned for areas of research that build from prior work. An example could relate to evaluation, design/development processes, beta testing models and student performance. This is an area where scales can be developed and made available for use. This could result in a contribution to the field, additional identity for the lab, and increased capacity for related work. It could also be source of researchable questions for graduate students looking for thesis and dissertation topics.
The development of relationships that result in an expanded set of affiliated researchers vested in the work of the lab should be a priority for the future. What drives the selection process needs to be systemic to the mission of the lab and built from or result in collegial relationships with participating faculty and researchers. There is already a group of faculty and researchers who have had considerable association with faculty affiliated with the lab and who have had some involvement with the lab.
There are a number of academic departments, research units, and entities within the university that are engaged in offering faculty support in the design and development of online instruction such as the Division of Continuing Education and the Center for Excellence in Teaching whose missions are complimented by the work of the eDL. There may be merit in the lab developing relationships that are more closely aligned with these organizations. This might result in an assessment of the need for e-learning tools, online instructional systems and or research. Additionally, there might be some role for the lab in the creation of graduate certificate programs offered by academic departments.
Relationships with the field might be enhanced by developing a more complete web site including all publications of participating faculty, technical reports of the lab, theses and dissertations of graduate students affiliated with the lab and the positions of students who have gone on to careers in the field.
Formalizing collaborative relationship with researchers from other institutions, schools and industry settings for purposes of research and development might warrant consideration. A beginning might be to establish more formal relationships with GRAs once they graduate and move on to careers in other settings.

Potential threats to the success of eDL :

This should be a continuing topic of consideration throughout the planning process. As ideas surface and are explored they will likely be examined in the context of the promise they offer the lab as well as the challenges that the lab might face in addressing them. Thus, revisiting this topic informally throughout the process will have merit.

Intellectual Property Issues: The unknowns involved in intellectual property rights related to e-learning in academic settings.
Commercial Growth: The growing industry of commercial Learning Management Systems (LMS) such as the Blackboard will increasingly make available tools with more features for instructors and trainers to use in creating online instruction. Such systems were not available when the lab was established. Their influence needs to be examined in the context of where the lab should be focusing its research and development energies.
Importance of Future Decisions: Failure to make the right decisions in the future (e.g., recruiting the right new colleagues, pursuing the right funding opportunities, reaching down deep in restructuring the lab, and/or over-looking what has been achieved and offers a good foundation upon which to build for the future).
Resources: The inability, as a self supporting unit, to respond to emerging opportunities and priorities of funding agencies and to diversify sources of funding. The latter needs to include sources such as private foundations, industry contracts and revenue streams from technology transfer.

Measuring the impact of the eDL :

In contrast to academic departments, the lab is a unit that is largely free to set its own mission and determine the nature of the work that it carries out. Participants are invited to affiliate with the lab and the lab is self-supporting through grants and contracts. These conditions allow the eDL to be responsive to changing conditions. For these reasons, monitoring the landscape to determine factors that may influence the operations of the lab is an important process. The key is to focus on those factors that most directly impact the mission of the lab and/or call for modifications in its mission. The most important conditions may be circumstances on campus or on the national scene. These are difficult to predict, but the task must be continuously revisited.

Positive Factors :

e-Learning and Teacher Education: E-learning continues to gain popularity as a vehicle for the delivery of staff development across disciplines and particularly in education. The module, e-book and the reusable learning object tools, combined with the large amount of content that has been developed by the lab, places the eDL in an excellent position to be responsive to this trend. We also are beginning to work in areas like tracking systems that hold great promise for professional development and in-service programs in terms of tracking progress of students, reporting back to instructors, students, an possibly to accountability systems related to accreditation. It may be timely to revisit arrangements with the Department of Special Education, School of Education and the Division of Continuing Education to create a model for the offering of courses and the tailoring of new courses to different populations. In education, states are beginning to become assertive in establishing alternative teacher education preparation models and utilizing online delivery systems to meet the increasing need for teaching personnel. Negotiating more control of the TeachSpecialEd.com program for delivery through the University of Kansas might be an appropriate move at this time.
Incentives for Faculty: The eDL model depends greatly on the energy and talent of faculty who invest their efforts above the normal expectations for their roles. While there are scholarly benefits and their teaching is influenced by the benefits derived from participation in the lab, there is need for an incentive system. The incentives should result in residual support for the lab to lessen the dependency on grants and contracts and, where appropriate, should personally benefit the individual contributors in some manner. University policies allow for such benefits. Because of the products and tools that were created by the lab, more attention should be given to working through the system to fully achieve technology transfer and the achievement of marketing those products with commercial value. This takes time but it may be a key factor in the future in retaining involvement of participants in the lab. This is not unlike what occurs in the sciences. There are several options including start-up companies, licensure arrangements with the commercial sector, partnerships with specific industries, and the creation of not-for-profit vehicles.
Engagement of Students: The academic setting of the University offers a number of positive factors that influence the future of the lab. One is clearly the opportunity to engage talented students in the work of the lab. We cannot underestimate the significance of these resources. They bring ideas, solutions, energy, commitment and excitement to the work of the lab. They are also committed to seeking out emerging trends, to understand them, to gain knowledge about them, and be part of any opportunity to better prepare for their careers. This would not be as true if the lab was a commercial enterprise. We need to think more about this resource from the context of how it influences what we do.
Changes in Academic Leadership: With a new provost and several new deans in roles of leadership, the institution is changing. They each come from different institutions and some of their home institutions have been very successful in the area of online instruction. They may bring new perspectives and commitments to how the University should be investing in e-learning opportunities. This could change the climate in which the lab functions.
Undergraduate Opportunities: There are also emerging opportunities on campus that might be explored. For example, some of the required undergraduate courses might be placed online. This could allow students options and reduce the need for hiring more instructors at the last moment in the fall when enrollments go up and a need arises for more sections of the required courses. Additionally, students who are heavily committed to work or to athletics might benefit from having access to more online options in order to gain flexibility in their schedules. This could be explored with some departments. Possibly some units might contribute since their students would benefit.

Negative Factors:

Lack of Flexible Resources: The lack of discretionary revenue that allows for new ideas to be explored without having to have such work funded by grants and contracts is a negative. This is also related to the time demands on the participants and the need to find ways that makes affiliation with the lab more appealing. It is important for participants to be public about their affiliations with the lab and to align their projects in some manner with the lab to expand the culture and to contribute to the synergism that is so important among students, staff and faculty to carry out the mission of the lab.
Commercial Competition: Companies are expanding their investments in the creation of new e-learning products. They have major resources to do this and they will be introducing new product regularly. This combined with the boutique industries that are very specialized result in a climate that makes it more difficult to impact. However, the lab has an advantage in that we are somewhat of a boutique industry that gives far more attention to content than commercial firms and we have the opportunity to focus precisely on technologies that enhance learning without also being concerned with broader markets and infrastructures. We can form partnerships that might result in revenues without having to maintain the full range of support.
Graduate Program Needs: The lack of a strong and comprehensive graduate program in instructional technology is also a negative. While we are successful in attracting quality GRAs, our needs would be better served if the graduate program was stronger and was also generating visibility by virtue of research and the preparation of leaders for the field. Currently, the lab is doing much of what an academic department ought to be doing. Somehow the eDL should seek to become more integral to enhance the SOE in pursing the development of a strong program.

Suggested Factors in Measuring the Success of the eDL.

Products: The impact and use of the products that emerge from the work of the lab plus the publications of those affiliated with the lab, copy rights and patents, presentations and sponsored events are examples of measure.
Dissertation and Theses: The dissertations, projects and theses that are completed by students affiliated with the lab and that use resources of the lab are added examples.
Funded Projects: The contracts and grants that the lab submits and which are awarded represent measures. This includes the varied agencies, the number of students supported, and indirect costs generated.
Influencing Policies: Because the lab addresses a relatively new area of research and development, the extent to which the lab plays a role in helping to frame and or implement institutional policy in this area is a contribution.
Inquiry Contacts: The number of contacts and consultations combined with actual cooperative projects that result from campus inquiries represents a measure of the lab’s success to achieve some level of visibility even though we do not aspire to be a service organization.

Accommodating the Research Interests of Participating Researchers:

The discussion on research interests resulted in the framing of preliminary research themes. They were viewed as work in progress warranting revisiting once the core group of researchers had been expanded... The first theme centered on how the lab might accommodate the interests of researchers in a manner that facilitates the work of participants while at the same time building unity within the lab. The second theme centered on some research foci that might serve as a framework for research groups. An additional topic, not related to themes or topics for groups, was addressed relating to the concept of the lab creating a model for transforming research papers and results to an e-learning model for dissemination to higher education students and practicing professionals. We have made some initiatives on this in the past and the time might be right to seek some support from KUCR to build the capacity to do this. It was agreed that a brief paper will be developed with a follow-up meeting at KUCR.

Theme I - Accommodating Research Interests: Other research organizations in academic settings are faced with the challenge of creating and maintaining an organizational structure that meets the needs of researchers who hold their primary affiliations in academic departments but come together in a research unit. In this environment they can benefit from a research infrastructure and from aligning with colleagues who have similar research interests. This is particularly true in the sciences and in fields such as engineering. For example, the areas on campus such as the biosciences and engineering for labs and/or research, are grouped under the umbrella of a center. The ITTC has a large number of researchers that have formed a series of labs within the structure of the ITTC. Over time the configuration of labs change to form a better fit with the research interests of the participants, funding priorities and collaboration opportunities. The ITTC becomes the umbrella organization, but the work of the ITTC is a result of the productivity of the individual labs within the ITTC. During its formative years the eDL has been the primary unit and the work has been structured within this area without a more diverse structure. This has largely been due to the evolutionary process and the fact that most participants have in some manner been contributors to most projects. We are now at a stage when interests vary and where there is a wider array of research questions needing to be pursued. In essence we may have reached a level of readiness for considering the structuring of divisions, work groups or research groups headed by researchers with the cluster of groups constituting the eDL. There appears to be consensus to further explore a model that would result in an operational structure centering on research led by researchers. In moving to such a relationship with academic departments, a model needs to be considered that might allow for joint appointments and further the respective academic and research missions. Two individuals were mentioned as potential resources people including Victor Frost and Ron Borchardt.
Theme II Potential Research Foci for Research Groups: The discussion on themes was first directed to the Parameter Construct (figure 1) with the idea that this might be a framework that would be useful in structuring groups. It soon became evident that it was created for a larger purpose as it was aligned with the mission of the eDL. While the foci that identify the work of groups would need to fit within the parameters of the construct, the “R&D Foci“ dimension of the contract does not necessarily translate to the identification of research groups at any particular time for the lab. These groups are broader in their implications that might ultimately be defined as a research group. However, the Principles and Target “Problem“ Learners dimensions of the construct would apply. Thus, the construct remains a viable tool and should be considered in framing research groups as the cells of the construct do not automatically translate into research groups.

Work Group Topics:

After considerable discussion n the theme concept a preliminary list of work group topics emerged warranting further attention. In some cases it appeared that a group might be a collaborative effort and in other cases it might be more related to an interest of an individual. Both are equally legitimate as leadership options for a group.

Preliminary Group Topics included:

Universal Design for Learners with a focus broader than K-12 and one that might offer particular insights as to features that will enhance the engagement of postsecondary students across disciplines.
Scalability of e-learning models is applicable to professional development and traditional postsecondary instruction. This could include tool designs that focus of effectiveness and efficiency beyond face-to-face models. Scenarios, case studies, multimedia vignettes and consensus builders were ideas that surfaced. Don shared experiences where reforms are so labor intensive and costly that they fail to achieve their potential because current models do not allow for the timeliness of implementation at scale.
Evaluation models that build from an empirical base and which are targeted at e-learning in the larger context. The context could include student outcomes and preferences, design, content, target audience, etc. with an aim to impacting over all quality.
Management and presentation in electronic form of large data sets where communication to varying audiences if important. Today in all levels of education there is a focus on data driven decision making. And often the emphasis on community, state and/or national comparisons become important for practitioners, policy makers and consumers to understand. This is also true in areas beyond education as well. This might build from the work of Jim Miller in visualization.
Embedded Tracking Designs are becoming important at all levels as a basis for improving education and training. This is true at the K-12 level, postsecondary level and for professional development and training. This theme might be a stand-alone or it might be merged with evaluation.