Overview
Analysis is the process of examining the evidence collected in Stage 3 to determine what the results show, whether the Minimum Criteria for Success were met, what patterns are present, and what the findings mean.
A strong analysis goes beyond reporting numbers. It connects the results to the outcome and assessment method, identifies strengths and areas for improvement, considers relevant patterns or differences, and provides the foundation for decisions in Stage 5.
Regardless of the type of outcome, analysis should answer the following questions:
What were the results?
Was the Minimum Criteria for Success met?
What do the results indicate about achievement of the outcome?
What strengths or areas for improvement are evident?
Are there meaningful patterns, differences, or trends?
Are there limitations that affect how the results should be interpreted?
A complete analysis demonstrates alignment among:
Outcome → Assessment Method → Minimum Criteria for Success → Results → Analysis
Major Steps
Analyzing assessment results involves the following major steps:
Prepare and organize the evidence
Analyze the results using the method established in Stage 2
Compare results with the Minimum Criteria for Success
Examine patterns, components, and relevant differences
Interpret what the findings mean
Document limitations and contextual factors
Prepare the findings for use in improvement
Prepare and Organize the Evidence
Before beginning analysis, confirm that the evidence collected in Stage 3 is complete and organized.
Review:
The number of students, respondents, records, artifacts, transactions, or other cases included
Whether the intended population or sample was represented
Whether any data are missing or incomplete
Whether the planned assessment method was followed
Whether scoring, coding, or data fields are consistent
Whether any changes were made to the assessment method or collection process
Document any significant departures from the original assessment strategy because they may affect how the results should be interpreted.
Analyze the Evidence
The analytical approach should match the type of evidence collected and the assessment method established in Stage 2.
Rubrics and Other Scored Student Work
Assignments, projects, presentations, portfolios, performances, and other open-ended student work are often evaluated using a rubric or scoring guide.
When multiple reviewers are involved:
Use a common rubric or scoring guide.
Provide reviewers with clear definitions for each performance level.
Conduct a calibration session before scoring, when appropriate.
Have reviewers score common examples and discuss differences in interpretation.
Establish a consistent process for resolving substantial scoring differences.
Apply the agreed-upon scoring process consistently to the remaining artifacts.
Reviewers may evaluate student work together or independently, depending on the nature of the assessment and available resources. Regardless of the approach, the goal is to support consistent interpretation and application of the evaluation criteria.
When reporting rubric results, examine both overall performance and performance on individual rubric dimensions when those dimensions provide useful information for improvement.
Exams, Embedded Questions, and Other Selected-Response Measures
For exams, quizzes, embedded questions, or other selected-response measures, analyze the questions that directly address the outcome being assessed.
Results may be examined using:
Percentage of students meeting the established performance level
Mean or median scores
Percentage of items answered correctly
Performance on specific questions
Performance on groups of questions aligned with particular knowledge or skill areas
When an assessment contains questions addressing multiple outcomes or competencies, analyze the relevant questions or groups of questions separately rather than relying only on the overall exam score.
Surveys, Interviews, and Other Indirect Measures
For surveys, interviews, focus groups, and other indirect measures, analyze the evidence in relation to the Program Outcome being assessed.
Survey results may be summarized using:
Frequencies
Percentages
Means or medians
Response distributions
Comparisons across items
Trends over time
For open-ended responses, interviews, or focus groups, identify recurring themes, patterns, strengths, concerns, or suggestions that relate directly to the outcome.
When response rates are available, report them and consider whether participation levels affect the interpretation of the findings.
Program Performance Data
Program performance data may include graduation, retention, enrollment, employment, licensure, time-to-degree, excess-hours, participation, or other institutional indicators.
Analysis may include:
Current performance relative to the Minimum Criteria for Success
Change from a prior year or assessment cycle
Multi-year trends
Differences among relevant groups
Comparison with established internal or external benchmarks, when appropriate
Avoid interpreting a single increase or decrease as a trend when only limited data are available.
Administrative and Operational Data
Administrative assessment may involve processing times, service utilization, completion rates, compliance rates, customer satisfaction, error rates, financial measures, or other operational indicators.
Analysis should focus on whether the intended result was achieved and what the evidence indicates about the effectiveness, efficiency, quality, or impact of the service or process.
Where appropriate, examine:
Change over time
Volume and workload
Processing or turnaround time
Completion or resolution rates
Error or compliance rates
Participation or utilization
Stakeholder satisfaction
Differences by service type, location, period, or other relevant categories
The analysis should focus on the outcome being assessed rather than simply describing activities completed by the unit.
Report the Results
The Results section should provide a clear and objective summary of the evidence collected.
At minimum, report:
The number of students, respondents, artifacts, records, transactions, or other cases assessed
The major findings
The performance measure identified in the assessment method
The Minimum Criteria for Success
Whether the Minimum Criteria for Success were met, partially met, or not met, as applicable
Where appropriate, include counts as well as percentages so readers can understand the size of the population or sample represented.
Example: Student Learning
Minimum Criteria for Success:
At least 80% of assessed students will earn a score of 3 or higher on a 4-point rubric.
Results:
Of the 40 students assessed, 34 (85%) earned a score of 3 or higher. The Minimum Criteria for Success were met.
Example: Program Outcome
Minimum Criteria for Success:
At least 85% of graduating students responding to the exit survey will indicate that they are satisfied or very satisfied with academic advising.
Results:
Of the 120 graduating students who responded to the survey, 108 (90%) indicated that they were satisfied or very satisfied with academic advising. The Minimum Criteria for Success were met.
Example: Administrative Outcome
Minimum Criteria for Success:
At least 90% of requests will be processed within five business days.
Results:
Of the 500 requests processed during the assessment period, 420 (84%) were completed within five business days. The Minimum Criteria for Success were not met.
Examine Results Beyond the Overall Score
An overall result may show whether the criterion was met, but it may not explain where performance was strongest or where improvement is needed.
When appropriate, examine results at a more detailed level.
Examples include:
Individual rubric dimensions
Individual exam questions or groups of questions
Survey items
Specific services or processes
Relevant student or stakeholder groups
Instructional modalities
Locations
Course sections
Time periods
Other meaningful categories
Example
An overall written communication score may meet the established criterion while individual rubric dimensions show different levels of performance:
Argument Development: 3.4
Use of Evidence: 3.2
Organization: 3.1
Disciplinary Writing Conventions: 2.6
The overall criterion may have been met, but the lower score for disciplinary writing conventions identifies a specific area for further examination and potential improvement.
Disaggregation should be used when it provides information that is meaningful and actionable. Avoid creating subgroup comparisons when the number of cases is too small to support reasonable interpretation.
Compare Results Over Time
When prior results are available, compare current findings with previous assessment cycles or reporting periods.
Consider:
Has performance improved, declined, or remained relatively stable?
Has the Minimum Criteria for Success been met consistently?
Are previously identified areas for improvement showing progress?
Are new patterns emerging?
A single year of data provides evidence about current performance. Multiple assessment cycles provide stronger evidence about patterns and trends over time.
When the outcome or assessment method is new, state that the results establish a baseline for future comparison.
Interpret the Results
The Analysis section should explain what the findings mean rather than repeat the Results section.
A strong analysis:
Identifies important strengths
Identifies areas in which performance did not meet expectations
Explains meaningful patterns in the evidence
Considers differences across measures or components
Compares current and prior results when available
Considers contextual factors that may help explain the findings
Identifies issues that warrant further investigation
Provides a clear basis for decisions or improvement actions
Results vs. Analysis
Results answer:
What happened?
Analysis answers:
What does it mean?
Example
Results:
Of the 40 students assessed, 34 (85%) earned a score of 3 or higher on the writing rubric, exceeding the 80% Minimum Criteria for Success. Performance was strongest in argument development and use of evidence and lowest in disciplinary writing conventions.
Analysis:
The findings indicate that students are generally demonstrating the expected level of written communication. Although the overall criterion was met, lower performance in disciplinary writing conventions suggests that students may need additional opportunities to practice discipline-specific formatting, citation, or writing expectations. This area should be considered when determining improvement actions.
Consider Limitations and Context
Assessment findings should be interpreted in light of the conditions under which the evidence was collected.
Relevant considerations may include:
Small sample sizes
Low survey response rates
Missing or incomplete data
Changes in the assessment instrument or rubric
Changes in curriculum, services, staffing, or processes
Differences in the population being assessed
Unusual circumstances during the assessment period
Changes in data definitions or collection procedures
Limitations do not invalidate assessment results, but they should be acknowledged when they affect the strength or interpretation of the conclusions.
Avoid making causal claims unless the assessment design provides evidence sufficient to support them. For example, an increase in graduation rates following a curricular change does not by itself demonstrate that the curricular change caused the improvement.
Before Moving to Stage 5
Before developing improvement strategies, confirm that:
The Results clearly report what was found.
The number of cases assessed is reported.
The Results are compared with the Minimum Criteria for Success.
It is clear whether the criterion was met.
Relevant component-level or disaggregated findings have been examined.
Current results are compared with prior results when available.
The Analysis interprets the findings rather than simply repeating them.
Strengths and areas for improvement have been identified.
Relevant limitations or contextual factors have been acknowledged.
The findings provide enough information to support decisions about improvement.
Stage 5 will use this analysis to determine what should be maintained, changed, strengthened, or investigated further.