Evaluating Critical Thinking Skills

Like many of you, I wear multiple professional hats. Critical thinking skills are at the nexus of all my roles. The importance of improving critical thinking transcends disciplines, even though the contexts and applications vary. As a sociologist, I see the how the deficit of critical thinking skills has a negative impact on society. As an evaluator, I find that these skills are frequent targets for NSF projects across disciplines.

Identifying important skills and implementing strategies to improve them is only one part of a grant proposal. An equally challenging issue is finding appropriate assessments.

Through the years I have learned some useful tips in selecting an instrument to best complement your evaluation needs. You should select an assessment that:

• 1. Aligns with the skills that are important to your project.
  2. Is transparent about both the questions and the rubric for assessing those questions.
  3. Provides insight and/or training as to how you can improve the skills.
  4. Provides flexible comparison groups for you (e.g., pre and post, or national user norms).
  5. Provides reports that are easy to read and use.
  6. Demonstrates validity, reliability, and cultural fairness.

As we struggled to find assessment options that could meet these standards ourselves, we developed and refined the Critical-thinking Assessment Test (CAT). If you are seeking to improve students’ critical thinking skills, you may want to consider this instrument.

The Critical-thinking Assessment Test (CAT)

This NSF-funded instrument is the product of 20 years’ extensive development, testing, and refinement with faculty and students from over 350 institutions and over 40 NSF projects. One innovation of this assessment is its integration of short-answer essay questions based on real-world situations. It provides quantitative and qualitative data on the skills that faculty believe are most important for their students to have 10 years after graduating.

Skills Assessed by the CAT:

Evaluating Information

• • Separate factual information from inferences.
  • Interpret numerical relationships in graphs.
  • Understand the limitations of correlational data.
  • Evaluate evidence and identify inappropriate conclusions.

  Creative Thinking

  • • Identify alternative interpretations for data and observations.
    • Identify new information that might support or contradict a hypothesis.
    • Explain how new information can change a problem.

    Learning and Problem Solving

    • • Separate relevant from irrelevant information.
      • Integrate information to solve problems.
      • Learn and apply new information.
      • Use mathematical skills to solve real-world problems.

      Communication

      • • Communicate ideas effectively.

        Our team truly enjoys working with evaluators and PIs to help them assess these skills and provide evidence of their success. Some NSF projects and courses have made gains in critical thinking equivalent to those gained in an entire four-year college experience. Our partner institutions have experienced positive outcomes, growth, and learning from working with the CAT.

        You can find more information about the CAT here, or you can contact me with any questions you have.

        References:

        About the Authors

        

        Ada Haynes

        Professor of Sociology and Co-Director of the Center for Assessment and Improvement of Learning

        Ada Haynes is a professor of sociology and co-director of the Center for Assessment and Improvement of Learning (CAIL) at Tennessee Technological University. She is also an independent evaluator on NSF STEM grants focusing on improving student learning and faculty development from multiple NSF divisions, including ATE. Over the past 20 years she has worked on assessing and improving critical thinking, including serving as the co-PI on three NSF grants to develop, refine, and disseminate the Critical-thinking Assessment Test (CAT) and to establish the CAIL center.

        

        Except where noted, all content on this website is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

        Related Blog Posts

        stock image of icons mapping out plan from idea to execution" width="1000" height="500" />

        Steps for Communicating Evaluation Design to PIs

        As ATE evaluators, our team engages with many new ATE PIs. To equip new clients for success, we initiate a prescriptive process.

        

        Strategic Data Collection: A Critical Element to Telling a Great Story

        We, external evaluators, understand that telling the whole story of our client’s efforts is built on the foundation of quality analysis that.

        

        Using a Utilization-Focused Lens to Improve Data Collection

        From my experience evaluating different types of programs, including NSF-funded Advanced Technological Education (ATE) projects, I have learned firsthand the practical usefulness.

        How Can We Help?
        Contact Us Anytime

        Address 1903 West Michigan Ave.
        Kalamazoo, MI 49008-5237 Phone (269) 387-5920 Email info@evalu-ate.org Stay Connected

        

        

        
        Copyright © EvaluATE 2024

        EvaluATE is supported by the National Science Foundation under grant number 2332143. Any opinions, findings, and conclusions or recommendations expressed on this site are those of the authors and do not necessarily reflect the views of the National Science Foundation.