The interviews provided a rich and meaningful insight into the participants’ understanding and experiences of procedural fidelity as it relates to the implementation of PT in school settings. The authors used template thematic analysis to assist them in addressing the study aims. However, the authors must acknowledge that this thematic analysis goes beyond the initial study aims to include valuable information gathered regarding how to achieve high procedural fidelity and shield against fidelity errors in PT. Participants’ views on the matter have been conceptualized into three overarching themes summarized in Table 2 and depicted in relation to their subthemes in the thematic map in Fig. 1 below.

Table 2 Summary of themesFig. 1The Complexity of Procedural Fidelity in PT

This theme portrays participants’ views on the meaning of procedural fidelity and fidelity errors in PT. It highlights the complexity of procedural fidelity in PT and the numerous different errors that could occur within each component of the PT system. The theme also displays the idiosyncrasies with regard to participants’ perspectives on fidelity errors and their impact.

“I Don’t Know Which is the Most Common”

Participants described a wide range of fidelity errors observed while training others to implement PT and the difficulty in determining the single most common error. Pinpoint, chart, and decide were the components that were emphasized when exploring errors observed in practice across interviews. Participants highlighted that teachers often struggled to pinpoint the behavior they wanted to measure or to select the most appropriate skill to pinpoint. With regard to the chart component, participants talked about intervention agents regularly having difficulty with more advanced elements of charting, such as “charting the record floor? […] not charting the data by minute.” Record floors are symbols used on the SCC to denote the duration of a timing period and represent the lowest non-zero result of an assessment (White & Neely, 2012). Another common error included intervention agents “charting after the fact,” meaning that they chart data post-session, which in turn prevents in-session decision-making. Failing to change the instructional or practice approach when learners are not progressing was most commonly observed within the decide component.

There were some differing views among participants with regard to the frequency of certain errors on the chart component. Some participants believed incorrectly charting data, i.e., plotting corrects and incorrects on the SCC, was one of the most common errors, while most participants believed that it was one of the least common and stated that when intervention agents do chart, they chart quite well.

“I think people, whoever all the trainers out there are, they get the people to accurately plot the data, in my experience. Unfortunately, that's too much of a focus sometimes.”

Similarly, participants spoke about a range of errors that occur on a less frequent basis. Some participants stated that forgetting to measure incorrect responses or learning opportunities occurred but was less common. Another mentioned that they had occasionally observed timing periods that did not match the pinpoint, e.g., timing for too long.

Omission Errors Occur More Than Commission Errors

When interviews were analyzed for errors of omission and commission, participants most frequently mentioned omission errors. Most omission errors highlighted were in relation to the chart and decide components. Numerous participants highlighted that the SCC is often omitted completely and that more “advanced elements” of the chart are often omitted, including record floors and minimum celeration lines. Minimum celeration lines are used as a means of setting goals and are drawn on the chart to indicate the minimum rate of progression that the learner needs to make to meet their pinpoint frequency aim in a timely manner. Similarly, participants mentioned that the decide component is often completely omitted.

“Not making changes according to the data. So even the data is kind of showing that you're kind of getting either a very flat line or minimal celeration, just kind of sticking with what you're doing and not changing anything up whatsoever.”

Although less frequent, commission errors were also discussed. They included adding “non-behavioral verbs” to pinpoints such as “know, understand, or be able to,” as well as calculating the percentage of correct responses and the use of verbal feedback or coaching during timed practice. Making decisions too often was a commission error mentioned by several participants. One participant described it as “they [intervention agents] make so many decisions rapidly we can't assess the effects of an intervention because it’s like change, change, change, change.”

Decisions, Decisions: Errors on Critical Components Can Have a Detrimental Impact

When talking about fidelity errors, participants without guidance regularly referred to the impact that such an error would have on learners and particularly emphasized emotional impact.

“It just hurts my heart seeing a child or a learner of any age on the same skill for weeks, and they’re not doing any better. Like, talk about deflating motivation.”

Most participants believed errors in the decide component were the most detrimental. Specifically, failing to change the instructional or practice approach when learners are not progressing was highlighted as the most damaging error in this component.

“The children themselves get to a point where they’re like, oh, do you know what? I just can’t do this. I’m not gonna do it anymore. I’m not getting any better, you know. So and so is on this pack (level), and I’m not getting anywhere, or I’m really bored of this now.”

Participants emphasized that failing to change the instructional or practice approach when learners are not progressing also impacts the intervention agents and trainers. One participant remarked, “Whoever is implementing the program also may […] kind of, you know, not believe in it as much.” Participants also considered the absence of a session routine and starting practice with skills before accurate responding was achieved as detrimental to the effectiveness of the system.

Participants reported that they believed the least detrimental errors were those associated with specific or more advanced charting conventions. One participant emphasized this by saying, “I don’t really know when charting zero if it makes a difference if you use a question mark, divide by two below the counting time, or an upside-down question mark.” Another said, “If I would be confident that the employee would be like consistent in always charting the data by minute, but then I wouldn’t see like a record floor. I would not be super concerned.” One participant believed omitting the chart completely was not overly damaging, whereas most stated that the chart was necessary, but some variation and errors in charting conventions could be tolerated. Participants also reported that variation in the amount of coaching used during frequency-building timed practice was one of the least damaging errors they have seen in practice.

Errors Have a Ripple Effect

Multiple participants perceived pinpoint errors as having a damaging impact and a knock-on effect on all other components. One participant described this as “if you pinpoint the wrong thing, you can’t be sure […] if the behavior change you’re getting is what you need.” Pinpoint was not the only component on which errors were perceived to cause a ripple effect. Participants reported that errors on the chart component led to errors on the other components of PT, as it affects one’s ability to analyze the data. Errors on the SCC could lead the intervention agent to believe that a learner’s performance is better than it is and to continue with an instructional approach when a change is warranted or vice versa. Participants highlighted that errors with regard to setting pinpoint frequency aims can have a ripple effect, impacting the sustainability of the PT system. Setting frequency aims too high may prevent both teachers and students from contacting reinforcement. One participant said, “They won’t really see the effects that they want, […] and they’ll go, do you know what, I’ve been doing this for 3 weeks, it’s not working, I give up”. Without a strong reinforcement contingency early on, both teachers and learners may become frustrated, which may lead to more fidelity errors or even complete disengagement with PT.

Maintaining Fidelity Without Losing Flexibility

Participants acknowledged that the components and principles of the PT system were fundamental but believed that some flexibility in their application could be tolerated. In some cases, flexibility was emphasized as essential for the successful implementation of PT. Flexibility was conceptualized as the adaptability of a component, intervention agent, or trainer in PT. Flexibility is discussed as a spectrum from flexible to inflexible.

Precision Does Not Mean Without Flexibility

Participants reported that some components of the PT system are flexible and more tolerant of fidelity errors than others. Participants discussed the flexibility of all components of the PT system, apart from try, try again. The majority of participants discussed elements of the chart component within which flexibility in implementation could be tolerated. The regularity with which one charts data was highlighted as a flexible element. How often one charts can vary since there is a family of charts from which to choose. Although the daily per minute SCC is the most used, intervention agents have the option to use weekly, monthly, or yearly SCC.

“Chart weekly and then make decisions on a weekly basis. At least you’re still regularly making decisions.”

Some charting conventions were also highlighted as being flexible, with one participant summarizing the level of flexibility they believed should be tolerated as “the conventions can be a little looser […] As long as everybody is clear about what everything means, I’m cool.”

Several participants believed that there is flexibility regarding how pinpoint frequency aims are set. Some participants reported that it did not matter whether frequency aims were set based on research or determined by measuring a fluent performer’s rate of responding. Two participants highlighted arrange instruction or practice as a flexible component, specifically, for how long one engages in practice and how often practice sessions are conducted. One interviewee highlighted this by saying, “We came to a compromise, that they do it 3 days a week […], but one of those sessions should ideally be on a Friday […], and that seems to work quite well.” Another participant highlighted aspects of the decide component that offered flexibility. They noted that if intervention agents make.

“If your change in an intervention is ineffective, that’s not really a problem, right? You just change it again. It tells you pretty quick, so that probably wouldn’t greatly impact if you’re looking at your data frequently enough.”

Others discussed the essentiality of flexibility within some components. Within arrange instruction or practice, flexibility was discussed when implementing frequency-building. In particular, one participant highlighted the flexibility necessary to implement performance feedback since it should be individualized for each learner and dependent on the intervention agent. In other words, every intervention agent has a slightly different approach to providing feedback in line with their personality.

Although participants emphasized the flexibility of the system, they also reported that some components are inflexible and require higher levels of fidelity. Participants believed that the decide component, specifically continuous data review, was the most inflexible component and must be implemented to the highest level of fidelity. Participants believe that intervention agents should review the learning picture as they chart.

Procedural Fidelity Requires Flexible Intervention Agents

While conveying the message that being a flexible intervention agent is important, participants discussed how inflexibility can often present itself in practice. Participants described inflexible intervention agents as procedurally rigid, and as a result, the PT system gets “all kind of rote feeling.” Participants hypothesized one reason intervention agents become inflexible is that following a set procedure is simply easier.

“The biggest implementation problem is getting folks to move away from a procedure […] People like black and white procedures.”

Another reason for the inflexibility in implementing PT may be a lack of conceptual understanding. This presents as intervention agents lacking versatility when making data-based decisions and making changes to instructional and practice approaches.

“Just very rigidly sticking to… basically not making decisions based on the data, just sticking to very rigid kind of fluency building.”

Participants believed that a strong conceptual understanding may prevent fidelity errors and promote flexibility in intervention agents. Participants identified key concepts, including rate, fluency, agility, “growth is nonlinear,” and sequencing as essential for intervention agents to understand. Such conceptual knowledge may develop their flexibility. As one participant stated, “If they could really learn those concepts, I think the whole thing would make a lot more sense to people. And when things make more sense to people, they don’t need like 85 steps.”

Participants also discussed how a lack of conceptual understanding can lead to more fidelity errors and rigidity, such as PT being used where it is not suitable, including efforts to increase fluency before a learner demonstrates accuracy. Additionally, a lack of conceptual understanding and flexibility can result in a negative implementation experience for the intervention agent.

“It gets, it’s very tedious. It feels tedious as a trainer, and it feels tedious as a learner. I think that doesn’t work.”

Another participant stressed the importance of flexibility, emphasizing the importance of being responsive to the learner as opposed to following a rigid procedure, stating, “It’s not about the procedure; it’s about the child.”

Flexible Trainers Make Compromises: Seeing the Bigger Learning Picture!

This subtheme represents a common thread among participants regarding the flexibility required to provide effective training in PT and promote high procedural fidelity in practice. Central to those perspectives was the view of intervention agents as learners. In seeing intervention agents as learners, participants highlighted the importance of compromise. Flexible trainers understand fidelity errors as learning opportunities that are part of the intervention agent’s implementation journey and accept that when PT is implemented in practice, it may not look like what the trainer initially envisioned or as described in textbooks or journal articles. Overall, the subtheme encompasses participants’ perspectives that trainers should view intervention agents, such as teachers, in the same way teachers view their learners.

“Just remember that the principles of behavior analysis and the system of PT work for the people you’re training just as much as they work for the kids.”

Participants believed that in their role as trainers, it is important to tolerate some fidelity errors to get intervention agents invested in the system. Participants suggested that in order to be a flexible trainer, one should acknowledge their biases.

“You can be married to a science, but you also have to realize when people don’t like it.”

“Within our science, we’ve got to not come across as telling anybody this is a better way of doing things, and we know better than you.”

Participants made reference to implementation being a social process, not solely a scientific process, even if the practice being implemented is scientific. The trainer merely telling an intervention agent to implement a procedure will not evoke implementation behavior. The participants brought attention to the social hierarchy between trainers and intervention agents and emphasized the need to avoid inflexible and perhaps authoritative approaches to implementation.

As trainers, participants recognized that errors may have benefits. Although this may sound counterintuitive, participants believed that allowing intervention agents to implement PT to a lower level of fidelity may be advantageous. Specifically, they suggested that if trainers are flexible and accept the flexibility of some components, intervention agents, such as teachers, may be more enthusiastic about using PT in their classrooms.

“It won’t be as effective. It will still be very effective […] they find it easier. And actually, the practice and the timings do happen.”

“They (students) had their goals and so on, but they were not using the chart at all, and that was a lot better than not doing it.”

Participants believed that tolerating some errors in implementation was “better than nothing” but recognized its shortcomings, stating “it was missing like a lot at the same time.”

Participants suggested that intervention agents may be more enthusiastic if trainers were flexible regarding certain fidelity errors, as this could reduce the perceived response effort associated with PT. Participants regularly referred to the high response effort involved in implementing PT as a teacher. As one participant pointed out, “teachers are overworked,” and the presentation of a high-response effort task such as engaging in training and implementing PT can lead teachers to ask, “Why do I have to do more?”. Participants emphasized the importance of trainers finding “common ground” with teachers and showing them how easy it is to fit into PT what they are already doing.

“A lot of people will ask me ‘how do I start’ […] I go – pick one task, just pick one task that lends itself quite easily to doing this and see how you get on.”

Participants believed that intervention agents viewed some components of the PT system as having higher response effort than others and that this may be where trainers need to be most flexible. They suggested a variety of strategies that could be used to address this. Participants highlighted that intervention agents believe the chart is high response effort and suggested that “digitally charting” could be an alternative to address this. Digitally charting involves using digital software to automatically plot the data on a virtual SCC once the user enters the number of correct and incorrect responses and the count time.

Data review and decision-making analysis were named by participants as other elements that teachers found demanding. One participant explained, “For people that haven’t trained in behavior analysis or psychology, looking at data is something very, very overwhelming.” Participants expressed that intervention agents often believe that they must “carry a pile of charts home every night.” However, participants identified that making decisions in the moment and having a system for decision-making was crucial to the sustainability of PT and was an effective way to reduce this perceived response effort.

“This is not your evening activity. This is what you do right on the spot.”

A few participants also suggested that having a learner become their own intervention agent or chart manager can reduce the response effort required by teachers. When the learner is the intervention agent, they can run their own timings, chart their own data, review data, and make decisions. This becomes a “self-ownership procedure” for learners, and teachers can take on a supervisory role by supporting students as they need. Participants also stressed that in order to reduce response effort for teachers, trainers should use accessible language, or “plain English,” as opposed to PT jargon as originally intended by Lindsley (1991).

“It’s about using that language that’s familiar to them.”

It’s a System

The third theme comprises participants’ views of PT as a system and how it fits within other structures. Throughout the interviews, participants highlighted that the underlying principles are at the core, and the five components are built to form the PT system. Participants placed a lot of emphasis on the point that PT is more than an intervention or teaching methodology and that it is a series of components that must work in unison.

“Everything goes so hand in hand, […] every step is important.”

Participants stressed this point further by stating that although it is a flexible system, no component of the system could be eliminated entirely as each component builds on the previous. Participants recognized that often there are misconceptions that PT is an intervention for a specific problem or specific learner, that it is a flashcard procedure or frequency-building in isolation. Participants highlighted that these misconceptions lead to errors and more misconceptions, which can become a vicious cycle and are detrimental to the perception of PT.

“The word gets out that Precision Teaching isn’t all that; it doesn’t work that well.”

Participants differentiated PT from a specific intervention by emphasizing the bigger learning picture, that PT is a system to promote meaningful change in an individual’s life, teaching learners how to learn and grow.

“There’s a whole method to teaching learners to grow. They’re learning to the point of automaticity […] It’s not just about charting things and turning timers on. It’s not just mechanics. […] there’s this big picture here.”

Another element that participants mentioned that differentiates PT from an intervention is how it can be used with a variety of skills and used in conjunction with other interventions.

…Within a System

PT is one of many systems that are nested within a classroom system, school system, and the wider education system. Participants discussed how other systems and their structures can facilitate or act as a barrier to the implementation of PT with fidelity in practice. Participants described how classroom management issues may be a barrier to the implementation of PT in practice. However, strong classroom management systems and establishing strong session routines were identified by participants as a means to decrease the likelihood of other fidelity errors occurring.

“So, there’s not good session routines in a classroom. Like the classroom is inefficient. It’s kind of unwieldy, like no… there isn’t this set way that the children are proceeding. So, it kind of gets looking kind of chaotic.”

Participants highlighted the wider school environment as another system in which PT fits. Specifically, participants stressed the importance of a supportive school system to ensure its adoption in classrooms. If PT is not embedded within the school system, there may not be resources, including rooms and/or staff, to ensure effective implementation.

“(If) it doesn’t get embedded, you know, teaching assistants are coming to us frustrated because they’re not getting the time to run those sessions, and you know they’ve turned up and the rooms occupied.”

It was also believed that the hierarchy of intervention agents in the school system may not support implementing PT as intended. Participants stated that if teaching assistants implement PT sessions in schools, they may not have the authority to make changes to instruction approaches and may be required to bring this to the attention of the classroom teacher.

“So, the systems sometimes don’t support some of the procedural aspects of a Precision Teaching model.”

Participants also highlighted where PT fits within the wider education system. Participants reported clashes between the PT system and the education system with regard to curriculum pressures. Participants emphasized the importance of the time dedicated to practicing material within PT and achieving fluency in every skill before moving on. They also discussed how this contrasts with the education system model, which is focused on covering the curriculum.

“We have a big pressure […] to reach some results, so we can test. […] They look at the end of the program, and so mostly to accuracy and not to frequency.”

Participants also reported that the traditional training model offered through the education system does not facilitate the teaching of a large system such as PT, that is, “There’s not a lot you can train in half a day without that kind of background knowledge.” However, they suggested that if used in conjunction with in-class coaching, the traditional training model may be effective at training intervention agents to implement PT. This in-class coaching model fits within the participants’ discussion on the importance of an implementation support system. Participants stated that continued supervision or the chart parent concept within PT is vital to the sustainability of PT in practice. A chart parent refers to the person who has trained you to use the PT system and the SCC and becomes a mentor who supports you in implementing PT throughout the course of your PT experience.

“Oftentimes people will make a ton of errors because they don’t have that… that frequent coaching and a chart parent to nurture their, their skillset and repertoire.”