Learning Assessment

Institution name:

Valencia College

Designated contact person/Title:

Dr. Joyce Romano, Vice President for Student Affairs

Contact telephone:

407-582-3402

Institution address:

190 S. Orange Ave

City, State, Zip:

Orlando, Florida 32801-3204

Website:

www.valenciacollege.edu

President's name:

Dr. Sanford Shugart

Number of years current president has held the position:

11Years

President's Assistant:

Barbara Halstead

392

24%

1262

76%

Barbara Jenkins

Deputy Superintendent

K12 Partnership – We have a formal Interinstitution Articulation Agreement with OCPS that names a coordinating group that meets at least annually and delineates our partnership in dual enrollment, career pathways, entry assessment, preparation for college, scholarships, faculty and staff development, and college transition programs with middle school and high school students.

 Pamela Tapley

Assistant Superintendent of Secondary Curriculum and Instruction

K12 Partnership – We have a formal Interinstitution Articulation Agreement with SDOC that names a coordinating group that meets at least annually and delineates our partnership in dual enrollment, career pathways, entry assessment, preparation for college, scholarships, faculty and staff development and college transition programs with middle school and high school students.

Dr. Tony Waldrop

Provost and Vice President

Academic Affairs

Direct Connect is a unique 2+2 partnership that guarantees admission of all Valencia graduates into the University of Central Florida as juniors. Valencia and UCF staff has built pathways for early advising, transcript sharing, and early admission to ensure a smooth transition to the university. UCF also provides its bachelors programs on two Valencia campuses. Shared buildings, libraries and joint philanthropy are other features of this unique partnership which currently 60,000 Valencia students have joined.  

Dr. Falecia D. Williams

West Campus President

Advisory Committees comprised of local business and industry representatives, faculty, and staff serve as a cooperative and collaborative entity for examining workforce trends, curricular content, and student learning outcomes.  Advisory Committees are a significant partnership for engaging the interdependence economic development and academic training of a quality workforce. 

Dr. Falecia D. Williams

West Campus President

Employers provide onsite workplace learning experiences to Associate in Arts and Associate in Science students pursuing a degree within the designated industry sector.  They work collaboratively with faculty supervisors to establish and evaluate learning and performance objectives that  support the program outcomes of the degree in which the student is enrolled.  Students are also provided a formal opportunity to network with potential employers.

Dr. Falecia D. Williams

West Campus President

Corporations and businesses provide supplemental funding, clinical sites, and use of their facilities and resources via signed affiliation agreements.

The State of Florida has a statewide articulation agreement between community colleges and the eleven State universities that guarantees any student who graduates with an A.A. Degree will be accepted to one of the eleven state universities.  In addition, Valencia has partnered with private institutions to expand student options for seamless transition. Some of the agreements are created with an Articulated Degree Pathway. They are designed for students to transfer to a particular public or private university as a junior to complete a four-year bachelor degree in a specific major.

Barry University

Al McCullough, Exec. Interim Dir.
ACE Enrollment Management

2+2 partnership

Belhaven College

 Alexis Fields, Campus Dir.

Dr. Richard Harris, AVP for Adult Studies

2+2 partnership

Capella University

Ricardo Grant

Account Manager

2+2 partnership

Valencia has been on a journey to become a more learning-centered college since 1994. We have advanced our strategic planning through alignment with national initiatives to improve student success (i.e. student persistence, course & degree completion).  A broad, deep, and continuing conversation has resulted to transform our organizational culture and improve student completion of courses and degrees. Through the ACE/Kellogg Project (1995), Valencia began engaging faculty/staff in new conversations about learning, common purpose, and educational issues. A culture of programmatic development emerged to improve student learning and completion.

One such institutional effort was the College Prep Task Force through which procedural, curricular and learning support changes were made to improve students' completion of developmental education. This reform work continued in the College Level Achievement Initiative.  Faculty reviewed and aligned the curriculum in conjunction with targeted high school teachers through a summer faculty development program, Destinations (which has continued annually). Completion of the developmental education sequence in two years in reading, writing and mathematics improved from the 30-40's% in 1991 to over 70% in reading and writing and nearly 60% in math (2008).

Valencia's study of student learning and success through Title III programs created the institutional momentum and design of LifeMap (our developmental advising model), Teaching and Learning Academy (faculty induction process), core competencies of a Valencia graduate (Think, Value, Communicate and Act), engaged teaching practice (through learning strategies, inclusive classrooms, action research, “connection and direction”).

As a Vanguard Learning College (League for Innovation), Valencia adopted the two fundamental learning-centered questions; “How will this impact student learning? and How will we know?” These questions established a community of learning leaders and the now ingrained practice of collaboration focused on student success.

Valencia developed a proposal for the Achieving the Dream (AtD) initiative which focused on closing gaps in student achievement through data-driven decision making (2004). In a collaborative planning process, we developed a “model of innovation” and identified strategies that were “ripe, effective, and scalable”:  Supplemental Learning (SL), Student Success Skills (SLS), and Learning Communities (LinC).  Faculty and staff were invited  to pilot, review data, and make plans for expansion. Valencia was awarded the first annual Leah Meyer Austin Award for our progress in “closing the gaps” in six “front door” courses (2009).

This led the College Learning Council to adopt the Foundations of Excellence self-study in order to continue the focus on the “front door” (2007-08).  The self-study led us to explore nine foundational dimensions (Philosophy, Organization, Learning, Campus Culture, Transitions, All Students, Diversity, Roles and Purposes, and Improvement) to facilitate conversations, study, and recommendations for a coherent first-year experience.

The Developmental Education Initiative (2009) continues scaling up SL and LinC strategies and also building an accelerated, fully realigned developmental education program.  A fully integrated Developmental Education Learning Community will lead entering students to complete 21 college level credits in their first year (2011-12).

Together these efforts illustrate a “long obedience in the same direction” towards improved student completion.

Valencia's learning-centered history includes engaging in large scale, data-supported, institutional efforts aimed at improved student learning, completion and success.  Four such efforts include the annual Program Viability, Growth & Success meetings  in which faculty, staff and administrators gather on a yearly basis to consider career and technical programs in light of established targets for program growth and student success (more on this work will follow in section 2); the development, implementation and annual review of  Valencia's Strategic Plan; our efforts to close achievement gaps through the Achieving the Dream and Developmental Education Initiatives; and our work with the Foundations of Excellence process.

The Strategic Plan

Valencia's Strategic Plan for 2008-2013 has four Goals: 1) Build Pathways, 2) Learning Assured, 3) Invest in Each Other, and 4) Partner with the Community.  Progress on each Goal, and the targeted Objectives for each, is evaluated on a yearly basis with the help of nearly 200 faculty, staff and administrators engaged primarily in two activities: 

• The College Planning Council convenes Strategic Goal Teams  for each of the four goals in the Strategic Plan.  Each goal team monitors progress toward the goal, collects and interprets data, monitors related strategic issues, makes recommendations to the annual Big Meeting, and prepares an annual report on progress toward the goal
  .
• The Big Meeting is an important component both of the College's Institutional Effectiveness effort and of the annual evaluation of progress toward the goals and objectives in our Strategic Plan. The meeting provides the opportunity to identify and make recommendations to the President about any missing pieces of the work to be done to achieve the strategic goals and objectives, to make recommendations regarding how the work might best be sequenced and coordinated, and to recommend any changes the group may deem advisable to the 2008-13 strategic goals and objectives and/or the evaluation plan. 
• Annual presentations are made to the District Board of Trustees  Strategic Plan Goal Reports on the progress made on each goal.

Achieving the Dream and The Development Education Initiative                               

Achieving the Dream (AtD) is a multiyear national initiative to help more community college students succeed. The initiative is particularly concerned about student groups that have traditionally faced significant barriers to success, including students of color and low-income students. AtD works on multiple fronts, including efforts at community colleges and in research, public engagement and public policy. It emphasizes the use of data to drive change. Building of the work of AtD, the Developmental Education Initiative (DEI)  is a national effort to increase the number of students who complete their developmental education courses successfully and move on to college-level courses.  DEI at Valencia will expand the reach of the AtD strategies to an ever great number of students.

Central to our work on both initiatives has been the creation of a College Data Team. The Data Team collects and structures college-wide reflection on student success rates, persistence from fall to spring and fall to fall, time to degree and time to complete prep mandates. For both DEI and AtD strategies, we monitor the achievement gaps between ethnic groups in six target courses: MAT0012C (Pre-Algebra), MAT0024C (Beginning Algebra), MAT1033C (Intermediate Algebra), ENC1101 (English Composition I), POS2041 (US Government) and MAC1105 (College Algebra).  We also track success rates and achievement gaps in Developmental Reading I (REA 007C), Developmental Reading II (REA 0017C), Developmental Writing I (ENC 0015C), and Developmental Writing II (0025C).     Development Education Report

In addition to traditional forms of data collection and analysis, we use qualitative methods to study the impact of our strategies. For example, we conducted a semester long study of our Student Success course. We documented faculty and student responses to the content of the course, and then analyzed those responses for specific themes. Based on these results, changes were made to the curriculum of Student Success and to an important feature of the course, the Student Success portfolio. 

Foundations of Excellence

At the heart of the Foundations of Excellence (FoE) process is the work done by the nine dimension teams – one team for each of the nine foundational dimensions (Philosophy, Organization, Learning, Campus Culture, Transitions, All Students, Diversity, Roles and Purposes, Improvement).  Each dimension has a series of performance indicators on which the dimension team members must evaluate the College using institutional data, publications, the student FoE survey, the faculty and staff FoE survey, and focus group data.  This data, combined with reflection and discussion allows the dimension teams to evaluate the College's performance, assign the College a grade, and make recommendations for improvement.  The result of this process allowed each dimension team to produce a report.  

An analysis of the results by the 180 person taskforce revealed the existence of six global themes – themes that transcend the conclusions of any one team.  The six global themes include the following: New Students Transitioning Into Valencia, Coordinated New Student Experience, Learning, Data Collection and Dissemination, Faculty and Professional Development, and Communication.  Work is well underway on the implementation of the taskforce recommendations -  Foundations of Excellence Report.

As result of this work A New Student Experience Committee has been established to promote the creation of a coordinated New Student Experience.  Part of this work has been focused on the student experience in the  top 10 highest enrolled courses for new students at Valencia.

The primary source utilized by Valencia to verify the appropriateness of degrees, certificates, and credentials earned through its academic programs with regard to the regional labor market demand is the Regional Targeted Occupations List.  The List is derived from recommendations made by the Florida Workforce Estimating Conference as authorized under F.S.§445.011. The conference meets semi-annually to establish a Statewide Occupational Demand List and provides recommendations to Workforce Florida, Inc. for establishing Regional Targeted Occupations Lists. Workforce Florida, Inc. approves the Lists.

The Workforce Estimating Conference develops Florida's official information set for use by the state planning and budgeting offices to ascertain the personnel needs of current, new, and emerging industries.  The conference applies quantitative and qualitative research methods to create short-term and long-term forecasts of employment demand for jobs by occupation and industry, entry and average wage forecasts among those occupations, and estimates of the supply of trained and qualified individuals available or potentially available for employment in those occupations. Emphasis is placed upon occupations and industries which yield short-term and long-term employment with High-Skills/High-Wage levels.  Data is generated through surveys conducted as part of Florida's internet-based job matching and labor market information system as well as local surveys conducted by economic development agencies and others. Based upon its review and analysis of such survey data, the conference makes recommendations semi-annually to Workforce Florida, Inc. on additions and deletions to the lists of locally targeted occupations and industries.  This work of the conference results in the annual publication of Regional Targeted Occupations Lists by Workforce Florida, Inc.

Each year Valencia Deans and Faculty complete a comparative analysis of its degrees and certificate programs and the Regional Targeted Occupations List.  Florida's curriculum frameworks for A.S. and technical certificate programs include a designation of correlated occupations.  For each degree and certificate program, the college compares the correlated occupations to the occupations identified on the Regional Targeted Occupations List and assigns an institutionally derived indicator for each program's level of compatibility with regional workforce demands. Programs that have a correlated occupation appearing on the Regional Targeted Occupations List with a High Skill/High Wage indicator are given the highest rating.  Programs that do not have a correlated occupation appearing on the Regional Targeted Occupations List are reviewed more deeply to project short-term and long-term program viability.

Valencia Deans and Faculty review the viability of A.S. and technical certificate programs annually as part of its institutional model for continuous improvement.  The process includes a detailed review of student learning outcomes, curriculum, and program performance measures such as completion, placement, and transfer to the baccalaureate.  Data obtained from Florida Education and Training Placement Information Program is used to gauge the level employment of students who complete certificates and degrees.  Based upon performance, each program completes an action plan that guides practice and curriculum for the purpose of improving the student learning experience.

Valencia Workforce Services provides online job listings but not formal placement services or formal collection of employee performance data.

Valencia utilizes the Regional Targeted Occupations List and other labor market data sources to evaluate program performance and inform models of enhancement.  Labor Market Statistics (LMS), from the State of Florida Agency for Workforce Innovation, and the Analyst, provided by Economic Modeling Specialists, Inc. (EMSI), are additional data sources regularly used in such analysis.  The Labor Market Statistics includes a collection of economic, demographic, and labor market data related to employment and wages, economic indicators, labor force, and population. The Analyst is a web-based tool that comprises local employment data, compatibility analyses for local training programs, and re-employment scenarios.  Annually, Valencia examines and compares this collect of data sources to inform our work in determining program viability and modes for continuous program improvement.

Valencia holds a Program Viability and Success Forum annually as well as specialized periodic reviews of all A.S. degrees and certificates based upon the  Five-Year Program Review Schedule.  Such efforts examine and determine program effectiveness, efficiency, and methods of improvement.  Evaluation teams include internal staff such as senior administrators, deans, program directors, faculty, and advisors as well as external representatives with background and expertise in the respective curricular areas.  Review criteria include faculty profiles, full-time to part-time ratios, enrollment data, student yield, program stability, placement rates, scheduling alternatives, advising, articulation agreements, learning resources support, Advisory Committee efforts and program learning outcomes. Specific recommendations for improvement are linked to a Plan of Action. 

The Plan of Action is reviewed periodically until the recommendations for improvement have been implemented and evaluated. Comprehensive documentation of this process is provided through an annually published handbook titled The State of Workforce Education at Valencia, and  Program Review Reports are published every five years. Examples demonstrate systemic programmatic review for the purpose of continuous improvement in accomplishing the institutional mission:   Summary of 2009-10 Program Success Strategies; five-year A.S. program review ( Business Administration); selected A.S. program performance dashboards ( Graphic and Interactive Design).

The cycle provides for planning, assessment, and evaluation for continuous improvement of programs and addresses state, national and professional accountability measures, as well as regional accreditation core requirements and comprehensive standards based on the Principles of Accreditation, Foundations for Quality Enhancement, Commission on Colleges, Southern Association of Colleges and Schools (SACS). Each plan is approved by the Campus President and obtains additional feedback through the college governance process.  Recommended changes are prioritized for funding through the budget planning process to ensure appropriate scale and sustainability.  Outcomes from these systemic program reviews have yielded changes in program content and sequencing, staffing levels, faculty development, and learning support services.  The effectiveness of programmatic changes are assessed and evaluated during the next Program Viability and Success Forum as well as the five-year program review. Successful changes are documented for best practices. 

While Central Florida is best known for its tourism industry, it is also the center of the simulation industry; a world leader in the science of photonics and the field of financial software; and an emerging hub for digital media technologies and bioscience research.

Valencia learning-centered work includes strong faculty development in support of the assessment of learning in the classroom (Outcomes Based Practice).  For some years, assessment efforts focused on the shared (interdisciplinary) contribution to the 4 Competencies of Valencia Graduate identified by faculty (Think, Value, Communicate and Act).  While these competencies had broad appeal and are now deeply rooted in the institutional culture, we discovered their broad applicability created difficulties for measurability/assess-ability. Since 2007, we have engaged in a culture shift in our assessment efforts by working to develop and approve Program Learning Outcomes for each program of study.  Formally published in our College Catalog as a promise to our students, the outcomes serve as a starting point for ensuring the alignment of all courses and co-curricular activities with program outcomes.

Plans for the assessment of program outcomes are designed, approved, and implemented by faculty with the support of Valencia's College Learning Council, the Learning Assessment Committee, the Offices of Learning Assessment, Institutional Research and Faculty Development. The development of yearly assessment plans is guided by a common  Program Learning Outcomes Assessment Template and an  approval process flow-chart  to ensure broad –based program faculty involvement in the process.  The assessment plans rely heavily on embedded assessments and the collection of student artifacts at points in a student's progress toward the completion of a degree or certificate.  The nature of embedded assessment, especially in larger programs like General Education, require sustained faculty engagement with each other to balance program-wide consistency for assessment purposes with academic freedom in individual classrooms. Student artifacts are collected based on random sampling of students within the targeted courses – the samples are stratified based on Campus, mode of instructional delivery, and the contract status of the instructor.  Institutionally, we consider assessment activities at the program level as primarily a tool to promote faculty dialog about the learning goals they have for students.  To date, the majority of program level assessment activities have yielded agreements concerning common expectations (rubrics, etc.) and improvements to the assessment process.  The “validity and reliability” of the assessment results is reflected in the ability of our faculty to build consensus about collective action to improve student learning. We are one or two cycles away from meaningful / actionable evidence of student learning at the program level in the majority of programs.

Faculty efforts in this area are recognized by our District Board of Trustees of a Faculty Compensation Plan that includes an institutional effectiveness component:

By May 13, 2012,  all academic programs will have implemented an assessment plan that has been approved by the Learning Assessment Committee (LAC)….In order for the faculty, collectively, to be eligible to receive the Institutional Effectiveness (IE) component that is in addition to their normal salaries in Fall 2012, 90% of all of the academic programs must have developed faculty approved improvement plans.  These improvement plans will be based on the learning assessment data compiled from each academic program's assessment plan.

Learning Outcomes Assessment data is used by program faculty to improve learning and curricular alignment. An example is from a report recently produced by English Faculty after assessing Information Literacy in English Composition paper on Assessment Day 2011 (Papers were randomly selected from all sections of ENC1101).  Assessment Day is an annual event for faculty to gather with their colleagues across campuses to assess student artifacts and discuss ways of improving either the assessment process or student learning.

 As a result of the findings from the assessment of student artifacts, the thirty-three faculty participants voted to work on improving instruction of the Program Learning Outcomes for General Education under the Information Literacy heading. Specifically, faculty agreed to address two objectives during the upcoming year: To teach students to properly integrate source materials into an essay and to document sources within the essay. An official vote to approve the upcoming 2011-2012 improvement plan will be sent to the eligible ENC1101 voter list to obtain consensus among at least 2/3 of the English faculty.

The improvement plan will include having each campus English Coordinator work with their respective faculty to develop a plan to address the two objectives and a campus-based improvement plan to increase students' ability to properly integrate source materials into an essay and document sources within the essay. The four campus coordinators will work together toward an overall improvement plan for the college's English department. During Assessment Day 2012, the English faculty will conduct the same assessment work as completed during the May 6, 2011 meeting and compare the results of the 2012 student artifacts assessment data with that of the 2011 data to determine if any improvements to the student work have been demonstrated.

Work on the articulation and assessment of Program Learning Outcomes has moved Valencia to create much stronger structural alignment between the Offices of Learning Assessment, Institutional Research, Faculty Development, and the Teaching / Learning Academy – all of these departments now report to the AVP for Assessment & Institutional Effectiveness within Academic Affairs.  Valencia's experience with faculty development and the assessment of learning outcomes at the classroom, course and program levels has situated us as leader within the State College System in the State of Florida.  For three years in a row Valencia has sponsored a State Assessment Meeting for the 28 Colleges in the State System – in June 2011 160 Faculty, Program Chairs and Administrators from 22 of the 28 State Colleges were in attendance.  The center piece of the meeting for the past three years has been an intensive  5 hour workshop  on the articulation and assessment of program learning outcomes.

Program wide assessment activities have also revealed challenges with helping adjunct faculty to participate in the process.  As a result of these concerns, a college-wide professional staff member has been hired in Faculty Development to focus on the needs and engagement of Valencia adjunct faculty.