This framework has been developed as a guide for LITRE project PI’s to use in designing their assessment and for organizing and submitting project reports. The framework is presented as a report submission guide, but the Introduction and Method sections should be modified for planning purposes and for preparation of the project assessment plan.

The framework is intended to a) make sure required information is provided,
b) improve the LITRE Assessment Committee’s ability to synthesize and learn from results of individual projects and summarize them into a comprehensive annual report, and c) provide guidelines and resources for a well-developed assessment plan.

Prepared by the LITRE Assessment Committee: Members: Geetanjali Soni (Chair), Hannah Gracz, Percy Hooper, Jacqueline Hughes-Oliver, Susan Katz, Stan North Martin, Brad Mehlenbacher, Donna Petherbridge, Dianne Raubenheimer, Joni Spurlin, Traci Temple, Nancy Whelchel, Carrie Zelna.

Sections of LITRE Project Report

ABSTRACT (a page or less)
The abstract summarizes the findings of the report and includes purpose, scope, methods, major results, and conclusions.

INTRODUCTION The introduction explains the project as well as provides relevant background information. Program goals, project objectives and student learning outcomes are included in this section.

• DESCRIPTION: Describe the research project, its general purpose and significance.

• HISTORY: Include a history of the project including milestones, prior results, and origin.

• LITERATURE REVIEW OR REVIEW OF THE STATE OF THE ART (whichever is more relevant to your discipline):
  Literature Review: Synthesize previous research on the topic under study. The purpose of the literature review is to convey what knowledge and ideas have been established on a topic, what their strengths and weaknesses are, and how the existing research leads, links, or relates to your project.
  Review of State of the Art: This summarizes answers to the following questions (as relevant): How have others attempted to solve this problem before? What methods are currently and/or publicly in use to address the problem? What are the most recent developments in this area? What previous solutions are currently documented in the US Patent and Trademark Office?

• BACKGROUND: Include a description of the population, including students (number, demographics, etc.), information about the Principal Investigator and other participants, discipline(s) involved, the course(s) and setting, as well as a description of the technology used.

• LITRE GOALS, PROJECT OBJECTIVES AND STUDENT LEARNING OUTCOMES

LITRE Goal(s): Describe your project in terms of the four areas of student learning identified in the LITRE plan (problem solving, empirical inquiry, research from sources, performance). Which (one or more) of these areas does your project focus on?
Project Objective(s): The project objectives are what you, as the PI, want to accomplish through this project. List the specific objectives of the project.
Student Learning Outcome(s): These outcomes are what you hope the students will achieve as a result of the project. Student learning outcomes relate to what the students know (cognitive), ways students think (affective/attitudinal), or what students do (behavioral, performance, psychomotor) as a result of the activity. These are tangible, specific, concrete, measurable, and achievable in the specified time period.

Student Learning Outcomes could be derived from the Project Objectives.

See Appendix 2 for sample LITRE goals, Projective Objectives, and Student Learning Outcomes.

 

METHOD (Project Design methods and Data Collection methods)

• PROJECT DESIGN AND PROCESS:  This section details the methodology of the project. Enough detail should be provided to allow replication of the project. Include details of:

• Pedagogical approach or teaching method (lecture/presentation of materials, discussion, project based, etc.)

• Technology used and purpose of use (planning class activities, seeking information, analyzing information, creating new information, allowing access to new situations, simulations, etc.)

• Type of study (case study, comparative study, experimental design, etc.) Explain the steps of the process.

• Other variables included in the study (effects of student characteristics such as learning style, age, gender, motivation, etc.)

• DATA COLLECTION & ANALYSIS (EVALUATION/ASSESSMENT OF STUDENT LEARNING): This section includes details about assessment of student learning. How was the data collected (survey, student project grades, student-generated content/artifacts, observations, etc.)?  How did you measure the Student Learning Outcomes and Project Objectives listed above?  Please indicate which measures you used for each Student Learning Outcome. Measures of both direct evidence of student learning, through evaluation by the PI (capstone projects, student created artifacts, performance on exams, national test scores, portfolios, etc.), and indirect evidence (surveys, self- reporting, focus groups, graduation rates, etc.) should be included. Include all instruments, tools, and measures such as rubrics, tests, and survey used. Also include details about data analysis methods and procedures.

• LITRE REQUESTED DATA: We request that you reflect on how the project affects you and other faculty on the project.  Below are some areas to consider when addressing this question.

Twice a semester (mid-term and end), please enter your responses into the LITRE online log.
Pedagogy:       How did the technology affect the way courses were taught?
                        What were found to be useful and non-useful techniques/strategies?
                        What did you find necessary to change about your teaching/pedagogy.
                        What were the unintended/unanticipated effects? Please include both positive and negative experiences.

Workload:      How did the use of technology affect your workload?

Content:          How did the content of related courses change due to the use of technology?  

Technology:    What are students using the technology for and how successful is the technology in meeting its purpose? What are the strengths or weaknesses of using this particular technology?
How satisfied were you with the use of the technology?

Training:         What training did you need and how effective was this training?  Are there other improvements needed in faculty development activities?

Support:          Did you feel satisfied with the technical support?

• TIMELINE: Plan should include timelines for the whole project including the assessment plan (project plan/project implementation, assessment plan submission, assessment implementation, report writing, etc.). Final report should indicate the duration of the study and main milestones.

• RESOURCES: Summarize resources involved with the program (money, people, time, admin support, etc.)

 


RESULTS
Report the main results of the project. Explain your actual findings. Answer the question, "What did I find out?" Summarize analyses and findings in text and use figures and/or tables to present results effectively. Do not include raw data or discuss or interpret results here. (Report that in the next section.)

Summarize your findings for each Student Learning Outcome.

Results from the LITRE Requested Log will be entered online.

DISCUSSION
Provide an interpretation of your results and support your conclusions using evidence from your project. The significance of findings should be clearly described. Focus is on your project.

SUMMARY OF SIGNIFICANT FINDINGS:  Summarize activities, accomplishments, and important findings over the grant period.  Discuss findings about student learning outcomes and whether your Program Objectives were met. Include strengths/weaknesses and challenges.

• CONCLUSIONS: Include discussion of questions such as: What have you learned about student learning and technology? Did technology help meet your student learning outcomes? How did technology impact and interact with other variables or dimensions of the learning environment (that your study focused on)? How did technology change the learning environment? Which tools were found to be most useful?

• Potential Applications for Others on Campus: How might others on campus apply what you've learned to their teaching and learning?  Include:

• Specific lessons you have learned about student learning and technology.

• Specific lessons you have learned about teaching with technology. This could include reflections on 1) what you found necessary to change about your teaching/pedagogy and 2) what the unintended/unanticipated effects of the project were. (Please include both positive and negative experiences.)

• Specific strategies that you found worked well in the technology-enriched environments you taught in. These are intended to help others as they design their own learning/teaching environments.

• LITRE REQUESTED DATA:  Summarize your reflections on how technology affected your teaching. (Questions in the LITRE Log.)

 

PUBLICATIONS
List any publications or presentations you or your colleagues on the project have made or will make in the near future.




APPENDIX 1
Definition of “Learning”. :Student learning at NC State: Four ways of knowing and doing.

1. Problem solving
Definition. A way of knowing and doing that involves an initial state and an end state and some difficulty in moving from the initial state to the end state and consists of overcoming the difficulty and attaining the end state.
Common ways of knowing and doing. (1) Finding or being given a problem; (2) defining the problem;
(3) determining various possible solutions, ways of attaining end state; (4) evaluating the solutions; (5) applying the optimal solution to the problem.
Variations. (1) designing a product; (2) applying specialized knowledge; (3) finding sources that provide
pertinent information.
Question for technology education: What kinds of technology and technology education do we provide students to best enable them to solve problems in ways appropriate to their fields of study?
2. Empirical inquiry
Definition. A way of knowing and doing that consists of answering questions by drawing conclusions from
systematic inquiry based on empirical data.
Common ways of knowing and doing. (1) Asking or being given a research question; (2) designing or being given a research method for answering the question; (3) applying deliberate and thorough observational skills; (4) organizing, summarizing, and presenting data; (5) coming to conclusions based on the data.
Variations. (1) hypothetical empirical inquiry; (2) descriptive empirical inquiry; (3) prospective empirical inquiry.
Question for technology education: What kinds of technology and technology education do we provide students to
best enable them to engage in empirical inquiry appropriate to their fields of study?
3. Research from sources
Definition. A way of knowing and doing that involves academic investigation using primarily library and Internet resources, the “research paper.”
Common ways of knowing and doing. (1) Asking or being given a research question; (2) locating relevant primary and
secondary sources to answer the question; (3) critically evaluating the sources; (4) marshalling the evidence to
answer the research question.
Variations. Varies according to discipline. Doing the research is not an end in itself, rather a means of learning
about and entering into the conversation of the discipline.
Question for technology education: What kinds of technology and technology education do we provide students to
best enable them to engage in academic research appropriate to their disciplines?
4. Performance
Definition: A way of knowing and doing that results in an artifact which has value in and of itself but, more
importantly in terms of learning, as direct evidence of the performance, a student’s ability to successfully apply
art and technique.
Common ways of knowing and doing. Depends on the discipline.
Variations. Depends on the discipline.
Question for technology education: What kinds of technology and technology education do we provide students to
best enable them to engage in the modes of performance appropriate to their disciplines?




Appendix 2: Examples of LITRE Goals, Project Objectives, and Student Learning Outcomes

LITRE Goal	Problem solving
Project Objective	To develop course materials around real engineering problems in which student and faculty use Excel and VBA in integrated lab and lecture lessons.
Student Learning Outcome	Students will be able to use Excel and VBA to solve realistic engineering problems in different disciplines.
LITRE Goal	Performance (Improve student writing)
Project Objective	Cut down on the amount of time spent reading student drafts and providing low-level comments
Student Learning Outcome	Students will recognize and address needs of various audiences and purposes. Students will write with minimal mechanical/grammatical errors.
LITRE Goal	Performance
Project Objective	I want students to develop a love for poetry. Student Learning
Student Learning Outcome	Students will be able to compare and contrast ballad and sonnets. Students will create their own sonnets following expected conventions.
LITRE Goal	Research from sources (Researching language for thinking, tools for scholarly primitives)
Project Objective	Expose students to computing technologies for teaching scholarly primitives
Student Learning Outcome	Students will be able to categorize technology/tools for their discipline
LITRE Goal	Performance (Use LMS to increase communication skills through discussion boards)
Project Objective	Develop well-defined discussion threads to enable students to demonstrate their knowledge in discipline
Student Learning Outcome	Students will be able to show increased levels of ability to make arguments in the field
LITRE Goal	Empirical Inquiry (New software will facilitate processing, productivity, and uniformity of the data across campus)
Project Objective	Incorporate technology to develop teaching methods that: a) improve general student learning, b) provide common platform for faculty in numerous departments, c) facilitate communication and, d) accommodate students’ individual learning styles.
Student Learning Outcome	Students will show an increase in productivity by performing their own data analysis.
LITRE Goal	Problem solving (Research from Sources, Empirical Inquiry, Performance)
Project Objective	Develop learning modules to help students: a) understand the complexities of choosing among several QSAR models b) recognize that careful choice is a crucial part of scientific discovery and c)critique choices that other researchers make.
Student Learning Outcome	Students will be able to define different assessment measures and be able to compute them. Students will be able to follow “faulty” choices to the next step, i.e. implementation. Students will make written and verbal presentations of paper reviews.