|
High
School Program Evaluation The WIRE (Western Institute for Research and Evaluation) evaluation conducted for Junior Achievement this year focused on assessing the impact of four High School Programs: JA Economics, JA Company Program, JA Success Skills, and GLOBE. The most important findings relate to student learning, critical thinking, quality of life, and teacher and consultant perceptions.
Summary of Program Impact on Student Learning
WIRE found that students understand and retain more economic concepts after participating in Junior Achievement. Figure 1 illustrates that the changes in student learning performance on the ORTs are educationally meaningful.
Figure 1. Student Performance on the ORTs.
GLOBE is not included on the chart because only two classes were completed, severely limiting the student sample size.
Programs Impact Learning of All Types of Students
While there were some differences between the amount of learning gained by various ethnic groups in every program, across programs, all ethnicities achieved higher ORT scores after participating in the programs.
The JA programs seem to do exceptionally well at improving the economics knowledge of at-risk students. The gains made by certain groups of at-risk students in the JA programs exceed the gains made by at-risk students in many other educational settings. The presence of a business role model does much to increase the meaningfulness and credibility of the programs.
Summary of Program Impact on Student Critical Thinking
The alternative assessment results show that JA High School Programs help students use higher levels of thinking, such as synthesizing and integrating information in a way that allows them to make solid business decisions.
Figure 2. Student Performance on the
Alternative Assessment Exercises
Summary of Program Impact on Students’ Quality Of Life
WIRE also found that JA programs have an impact on students’ quality of life (QOL) in the following categories: Workforce Readiness, Attitudes Toward School, Citizenship and Deportment, and Ability to Make Major Life Decisions. Most of the changes were small and not statistically significant. This is due in part to the finding that many students already held positive attitudes toward work and school.
However, for the majority of students, across all programs, JA is having an influence in their lives. This impact appears to be greatest for the Company and Success Skills Programs. Both of these programs present very practical content using hands-on activities that are engaging and easily grasped by most students. The approach used by these two programs may be helping students to develop real-life skills which they are confident can be used to make major life decisions.
Summary of Perceived Impact
According to
Teachers and Consultants
Teachers and consultants agree that in all programs students are gaining basic business knowledge. Teachers and consultants affiliated with the Economics and Company Programs also felt that JA students are obtaining economics knowledge. Success Skills teachers and consultants cited job skills, such as interviewing and résumé writing, as being topics that students have mastered after participating in JA.
Across all programs, teachers and consultants agree that JA students are better equipped to think critically, solve problems and take on different perspectives than are students who have not participated in JA. They also reported several areas in which they saw a difference between JA students and other teenagers. These included:
-
Getting a job
-
Succeeding at a job
-
Solving problems
-
Acting professionally
-
Making major life decisions
-
Respecting authority
-
Handling major life events
-
Taking on leadership roles
-
Resolving conflicts in an effective manner
-
Improving self esteem
-
Building confidence
Within programs, teachers and consultants rated the impact of JA with almost universal accord. However between programs, there were some conspicuous differences. Teachers and consultants in the Economics Program consistently gave a lower rating of the impact of JA on students’ quality of life. Conversely, consultants and teachers of the Company and GLOBE Programs gave consistently higher ratings of impact. Both Company and GLOBE Programs require students to assume leadership roles.
Summary of Program Effectiveness
Overall, all stakeholder groups (students, teachers, and consultants) saw value in the JA High School Programs. Teachers were more positive about the effectiveness of the programs than consultants and both teachers and consultants rated the Economics and Company Programs highly. However, teachers seem more impressed with the Success Skills and GLOBE Programs than do consultants.
Not only were students benefiting from the program, but 80% of the teachers also indicated that they learned from participating in the programs (this was true across programs). Consultants indicated that participating in JA had positively shifted their perceptions of teenagers. Again, this was true across programs.
While stakeholders were positive about the programs they also saw room for improving the various programs’ effectiveness. Specific suggestions for each program were noted for the JA Curriculum team.
High School Programs Evaluation Results
The WIRE (Western Institute for Research and Evaluation) evaluation conducted for Junior Achievement this year focused on assessing the impact of four High School Programs: JA Economics, JA Company Program, JA Success Skills, and GLOBE. The most important findings relate to student learning, critical thinking, quality of life, and teacher and consultant perceptions.
Summary of Program Impact on Student Learning
WIRE found that students understand and retain more economic concepts after participating in Junior Achievement. Figure 1 illustrates that the changes in student learning performance on the ORTs are educationally meaningful.
Figure 1. Student Performance on the ORTs.
GLOBE is not included on the chart because only two classes were completed, severely limiting the student sample size.
Programs Impact Learning of All Types of Students
While there were some differences between the amount of learning gained by various ethnic groups in every program, across programs, all ethnicities achieved higher ORT scores after participating in the programs.
The JA programs seem to do exceptionally well at improving the economics knowledge of at-risk students. The gains made by certain groups of at-risk students in the JA programs exceed the gains made by at-risk students in many other educational settings. The presence of a business role model does much to increase the meaningfulness and credibility of the programs.
Summary of Program Impact on Student Critical Thinking
The alternative assessment results show that JA High School Programs help students use higher levels of thinking, such as synthesizing and integrating information in a way that allows them to make solid business decisions.
Figure 2. Student Performance on the
Alternative Assessment Exercises
Summary of Program Impact on Students’ Quality Of Life
WIRE also found that JA programs have an impact on students’ quality of life (QOL) in the following categories: Workforce Readiness, Attitudes Toward School, Citizenship and Deportment, and Ability to Make Major Life Decisions. Most of the changes were small and not statistically significant. This is due in part to the finding that many students already held positive attitudes toward work and school.
However, for the majority of students, across all programs, JA is having an influence in their lives. This impact appears to be greatest for the Company and Success Skills Programs. Both of these programs present very practical content using hands-on activities that are engaging and easily grasped by most students. The approach used by these two programs may be helping students to develop real-life skills which they are confident can be used to make major life decisions.
Summary of Perceived Impact
According to Teachers and Consultants
Teachers and consultants agree that in all programs students are gaining basic business knowledge. Teachers and consultants affiliated with the Economics and Company Programs also felt that JA students are obtaining economics knowledge. Success Skills teachers and consultants cited job skills, such as interviewing and résumé writing, as being topics that students have mastered after participating in JA.
Across all programs, teachers and consultants agree that JA students are better equipped to think critically, solve problems and take on different perspectives than are students who have not participated in JA. They also reported several areas in which they saw a difference between JA students and other teenagers. These included:
-
Getting a job
-
Succeeding at a job
-
Solving problems
-
Acting professionally
-
Making major life decisions
-
Respecting authority
-
Handling major life events
-
Taking on leadership roles
-
Resolving conflicts in an effective manner
-
Improving self esteem
-
Building confidence
Within programs, teachers and consultants rated the impact of JA with almost universal accord. However between programs, there were some conspicuous differences. Teachers and consultants in the Economics Program consistently gave a lower rating of the impact of JA on students’ quality of life. Conversely, consultants and teachers of the Company and GLOBE Programs gave consistently higher ratings of impact. Both Company and GLOBE Programs require students to assume leadership roles.
Summary of Program Effectiveness
Overall, all stakeholder groups (students, teachers, and consultants) saw value in the JA High School Programs. Teachers were more positive about the effectiveness of the programs than consultants and both teachers and consultants rated the Economics and Company Programs highly. However, teachers seem more impressed with the Success Skills and GLOBE Programs than do consultants.
Not only were students benefiting from the program, but 80% of the teachers also indicated that they learned from participating in the programs (this was true across programs). Consultants indicated that participating in JA had positively shifted their perceptions of teenagers. Again, this was true across programs.
While stakeholders were positive about the programs they also saw room for improving the various programs’ effectiveness. Specific suggestions for each program were noted for the JA Curriculum team.
| 
