Assess Better – Put down the Cookie (Cutter). Individualizing the assessment process
By Dr. Megan Miller, BCBA-D
The Feb theme for the 2018 Do Better Movement is Assess Better. We started this month with a video activity related to identifying skills to assess with an early learner. The Do Better community identified quite a few skills they would assess further. However, there was a general theme of relying on commercially available assessments for this process. In our webinar for Assess Better, we discuss in more detail the next steps we took in the assessment process for this learner.
The professional development activity for this month was a 3-hour webinar covering the topic of Assessment for learners diagnosed with autism. The webinar focused on describing a multitude of commercially available assessments for learners with autism, providing examples of ways these assessments can be used together to develop appropriate intervention, discussion of assessment in general and different resources to use for assessing skills that do not related to commercially available assessments, and reviewing some video examples. Currently, behavior analytic training programs both at universities and through supervised fieldwork seem to focus too much on training individuals to complete 1 or 2 specific assessments instead of training on the assessment process. Behavior analysts successfully developed very effective interventions for learners well before the existence of any commercially available assessments and it is critical to our field that we continue to develop skills related to this process. This webinar is available for FREE until March 13th, 2018 and can be accessed in our Slack Group #assessbetter channel and/or on Navigation Behavioral Consulting’s Facebook page found here. After the 13th, the Webinar will be available on our website: www.navigateaba.com/ce-courses
For our assess better blog, we are talking about how to put down the Cookie Cutter and develop individualized assessments for our learners. I could probably write a whole book on this topic but for the sake of time, I have narrowed down the focus to a few key areas: observation, learn your population, familiarize yourself with resources available to you based on this population, and an example document of how we make use of the resources available to us.
When conducting assessments, it is critical that observation occurs PRIOR to determining which formalized assessments will be used with the learner and to help you pinpoint what may need to be assessed further. It is near impossible to know that the proper assessment process is being conducted with a learner if you have never seen the skillset of that learner. Observation can be done by viewing a few quick video clips of the learner when feasible or by observing the learner’s natural interactions when conducting the intake. During the observation, it is important to look for:
- What does this learner do that is not at the same skill level as his peers? à Skills that seem to be completely absent for the learner
- What does this learner do that is not at the same rate as his peers? à Maybe the skill is present but not occurring at the same frequency as we would see in a peer
- What does this learner do that might be interfering with learning from the natural environment?
- What does this learner do that is supporting learning from the natural environment?
- What overall strengths does this learner have?
- What preferences does this learner have? à What is the learner allocating his responding to during the observation and what does he gaze at or seem drawn to throughout the observation
If you are newer to the field, you may start out with just looking for the items listed above and anything else trained by your supervisor. However, the more learners you encounter, the more you will develop your own list of things you are attending to and assessing with your learners. This is another critical element of the assessment process. DO NOT BECOME DOGMATIC! Whatever you learn about assessment now, constantly reflect how you can do it better and what else you can learn about and experience to improve the effectiveness and efficiency of the assessment process.
Observation should not just be limited to the learner you are working with either. You should observe those who support the learner to see what skill sets they may have and how they are already helping or potentially interfering with the learning process. If you are working with a population that is newer to you or your learner is performing at a level that you have less experience with, you should also observe peers. Peers can refer to any number of things, for example:
- Toddlers when working with 0-3 population
- Students who are around the same age as the learner
- Co-workers when working in a business
- Adults performing the same skill when working in group homes or residential settings
Learn Your Population
I can’t state this enough! As behavior analysts, we know HOW to teach…to truly be an expert and Do Better in our careers, it is critical to learn the population you are serving. For some people, this may mean learning as much as you possibly can about Autism. For others, this may mean learning about the culture and overall behaviors of the white collar business world. Knowing your population will depend entirely on what your job consists of and again it doesn’t happen over night. However, the more you can learn about the observable and measurable behaviors related to the populations you are serving and the resources available for those populations, the more effective you will be at individualizing the assessment process and developing individualized interventions.
My main example for this is autism. While I understand the science of behavior analysis and could apply it with any population, I have chosen autism as my focus population. When I first graduated with my masters (too many years ago to count), I spent countless hours scouring the books and resources available for autism. Some of my mentors did not support this and often reminded me that I was a behavior analyst and not an autism specialist. However, I knew that to best serve my clients, I needed to be familiar with research and resources from all fields related to this population. I read everything with a critical eye and focused heavily on how to apply behavior analytic techniques to the resources I was encountering. We will talk about this more in May when we talk about Evidence-based practice!
I built a solid understanding of Autism in those early years by soaking up all those resources but it wasn’t until I started working at The Florida State University as a research assistant that I realized some huge deficits that I had! Even though I was working with children with autism, I had never once really dug in deep with assessing and learning about the core diagnostic domains. I was a behavior analyst, what did I care about the DSM-V. Boy was I wrong! The ADOS and other resources available to me at FSU that included operational definitions for these symptoms made me realize I was missing out on assessing so many key skills with my learners with autism. Furthermore, conducting these assessments with little ones who were 14-23 months old made me realize I had NO CLUE what typically developing children did at those ages! I immediately committed to doing better and learning more about typical child development and ensuring I disseminated information to my supervisees regarding appropriately assessing these skill areas for children with autism. **I would also like to note here another key feature in learning your population or in general doing better as a behavior analyst is to be ready to take on any opportunity available to you that is in an area that you aren’t as familiar with and could learn from. I had my PhD and took on a job that someone with an undergraduate degree could do, but I had the opportunity to work with one of the leading researchers in autism diagnosis and treatment. There was no way I was passing that up!
If you are newer to the field and/or just thinking about this concept for the first time, some easy things to do to learn about your population are:
- Conduct Google scholar searches for your population to see what research, especially literature reviews exist specific to the populations you are working with and read those articles
- Do the same with Amazon or Google book searches and commit to reading at least 1-2 books per year that would help improve your understanding of the population. Reading the descriptions and reviews can help you know if a book will be helpful.
- Attend conferences or webinars about the population
- Attend community events about the population
- Review assessment materials for the population à age and diagnosis, especially standardized assessments
Familiarize yourself with Resources
I already discussed this a little bit above. Familiarizing yourself with resources related to the population will consist of learning the population as a first step and the suggestions made above. It is important to note that familiarizing yourself with resources doesn’t mean you must buy every single book out there and memorize it. However, the more books, curriculums, assessments, etc that you can get your hands on and learn about, the easier it will be to develop individualized assessment and interventions for your learners. A lot of communities have resource libraries or networks of providers where a behavior analyst could review materials without purchasing everything. Familiarizing yourself with resources also doesn’t mean you should use everything completely as written. Many of the resources I use I pinpoint what exactly about the resource is helpful for this specific learner and I incorporate that within the assessment and intervention process. Sometimes the resource just has a few key points that I keep in mind when developing my assessment and sometimes it has worksheets that help me not re-invent the wheel for teaching a certain skill but I still have to put that worksheet within a behavior analytic framework.
One of the easiest ways to familiarize yourself with resources is to go to the book stores at conferences and look through the books there. Again, you don’t have to buy each one but you can get a good idea as to whether the resource would be helpful for the work you are doing. There are tons of books out there now-a-days so it is important to look through the books with a critical eye and think about how the book would help improve your assessment and/or intervention process.
There are many resources that behavior analysts often overlook that could help improve the assessment process. Resources that fall into the standardized assessment arena and general checklists that have been made available by government agencies and non-profits make up a good number of these resources. In fact, they are so often overlooked, I don’t even think I listed them on the resource I made! Dr. Gina Green has presented on this topic in great detail so if you get a chance to see her talk about it, I highly recommend it. We also discuss this a bit in the webinar from this month so check it out for more details!
Resources We Use
As a supplement to the webinar that we did, I created a table that describes different assessments, curriculums, or other resources that we have found helpful when working with children diagnosed with autism. Not everything listed on this table is “evidence-based” but we will discuss that more in May! Check out the table HERE and let us know what thoughts you have about items to add, which of these you’ve found helpful, etc
Go Forth and Assess!!
I imagine many people reading this blog are students or recently certified analysts. One of the most common questions I get when supervising is “how will I ever learn how to do this assessment thing?” The best way, is to just start doing it! Take initiative and work with your supervisors or mentors to start conducting whatever assessments are used. This doesn’t mean your results will be used for the child or you will be paid for this but the best way to learn is by getting in there and just doing it! This typically works best if you:
- Get your hands on assessments relevant to the population you serve and start reading them, take note of what skills are assessed, and how they are assessed
- Watch people conduct assessments, take notes, and ask questions
- Review the results of the assessments and see if you can pinpoint what skills would be targeted first and what other skills may need to be assessed further
- Observe learners during initial intakes and see if you can pinpoint what skills need to be assessed further
- Observe learners, watch YouTube Videos, read examples of learners and see if you can develop your own ideas for skills to assess and how to assess them based on what you know about the population and typical development WITHOUT referencing a specific assessment. If you can do this, it shows you are really developing behavior analytic skills related to assessment. If you can’t do this, it helps you identify gaps in your knowledge base and you can seek out resources to fill those gaps!
In conclusion, we need to be most concerned about learning in general how to Assess Skills not which assessment to use or how to conduct a specific commercially available assessment. Pretty much anyone can pick up a commercially available assessment, conduct it, and pick some skills to work on. We need to go beyond this. We should know as behavior analysts how to break skills down and arrange the environment to promote successful acquisition of those skills. If you aren’t getting sufficient practice with that in your training, seek it out and/or practice it on your own. Just observe someone doing something and see if you can pinpoint what is being done well, what could be done better, what data would be needed to determine how best to teach the skill, and how the target skill can be broken down for teaching. When we learn as much as we can about what skills we should even be looking for in the first place based on the learners we are serving, it can make this process a lot more effective. Sometimes this they may be as simple as basing it on the goals and the needs of the family and the learner and other times it may rely heavily on your ability to dive in and learn about those skills and then apply what you’ve learned to the learner you are assessing.