← All Webinars | L.A.B.S. #1

1
00:00:07.500 --> 00:00:10.400
Hello everyone. Thank you so much for joining us today.

2
00:00:10.400 --> 00:00:13.700
We're gonna give everyone

3
00:00:13.700 --> 00:00:16.200
a few minutes to get started. So we'll start at

4
00:00:16.200 --> 00:00:19.200
about three minutes after the hour. But in

5
00:00:19.200 --> 00:00:21.500
the meantime, I'm gonna throw a poll up there.

6
00:00:23.100 --> 00:00:24.000
a little welcome poll

7
00:00:25.200 --> 00:00:26.300
for everyone to

8
00:00:27.900 --> 00:00:29.500
everyone to answer

9
00:00:31.500 --> 00:00:34.200
While we're waiting, we're curious to know how many of

10
00:00:34.200 --> 00:00:37.300
you here today have yourself earned a

11
00:00:37.300 --> 00:00:40.700
certification or more and whether you

12
00:00:40.700 --> 00:00:41.700
have to maintain it.

13
00:00:56.400 --> 00:00:58.100
Let's see. Oh, we already got a few.

14
00:00:59.600 --> 00:01:00.200
answers

15
00:01:03.900 --> 00:01:05.200
so for everyone has

16
00:01:07.200 --> 00:01:09.000
earned a certification nice.

17
00:01:18.900 --> 00:01:21.500
wow, if you certifications that do not require any kind

18
00:01:21.500 --> 00:01:22.500
of Maintenance, that sounds

19
00:01:25.900 --> 00:01:26.600
It's not a bad deal.

20
00:01:31.200 --> 00:01:34.400
All right, everybody. We'll get started in about one minute. So just a little

21
00:01:34.400 --> 00:01:37.500
bit more time to answer our little Icebreaker pole

22
00:01:37.500 --> 00:01:37.600
here.

23
00:02:21.900 --> 00:02:24.300
Wow, so far. Everybody has a certification.

24
00:02:32.300 --> 00:02:34.700
All right. I'm gonna end this whole.

25
00:02:36.600 --> 00:02:39.700
And share the results with you guys. So 100% of

26
00:02:39.700 --> 00:02:40.700
you in attendance today.

27
00:02:41.500 --> 00:02:45.100
Have one or more certifications very impressive crowd and

28
00:02:44.100 --> 00:02:48.000
of that about 33% Do

29
00:02:47.100 --> 00:02:51.300
not require any any kind of recertification, which

30
00:02:51.300 --> 00:02:54.100
is fascinating and some of

31
00:02:54.100 --> 00:02:54.300
them.

32
00:02:56.400 --> 00:02:59.400
At least one have require certification and some of them three or

33
00:02:59.400 --> 00:02:59.500
more.

34
00:03:01.200 --> 00:03:04.200
Very interesting. Well, thank you all for indulging me

35
00:03:04.200 --> 00:03:08.500
in in the entry poll. So once

36
00:03:08.500 --> 00:03:12.000
again, hello everybody. Thank you so much for joining us today. We're

37
00:03:11.500 --> 00:03:14.300
delighted that you're here with us for our very first

38
00:03:14.300 --> 00:03:17.500
live webinar in our learning and assessment broadcast

39
00:03:17.500 --> 00:03:18.300
series.

40
00:03:19.100 --> 00:03:22.200
My name is Sandy Hogue. I'm the business manager at

41
00:03:22.200 --> 00:03:25.400
its and I'm going to be your host and moderator today

42
00:03:25.400 --> 00:03:29.000
during our session. We are really really

43
00:03:28.400 --> 00:03:31.800
excited today to talk to you about the evolution

44
00:03:31.800 --> 00:03:34.400
of recertification and what

45
00:03:34.400 --> 00:03:37.300
longitudinal assessment brings to the table.

46
00:03:38.700 --> 00:03:41.400
Just to set the stage. Today's presentation should last

47
00:03:41.400 --> 00:03:44.500
about 45 minutes, and we've saved time for

48
00:03:44.500 --> 00:03:45.900
Q&A at the end.

49
00:03:47.300 --> 00:03:50.500
So before we dive in I do want to go over a few

50
00:03:50.500 --> 00:03:51.500
housekeeping items.

51
00:03:54.300 --> 00:03:57.400
We will be saving some as I mentioned. We'll be saving some dedicated

52
00:03:57.400 --> 00:04:00.700
time for questions and answers at the end of the presentation.

53
00:04:00.700 --> 00:04:03.400
But please do submit your questions as

54
00:04:03.400 --> 00:04:06.600
you think of them throughout the presentation using the Q&A feature.

55
00:04:06.600 --> 00:04:09.600
There's a there's a little button at the bottom of your Zoom

56
00:04:09.600 --> 00:04:12.500
window where you can submit questions using the

57
00:04:12.500 --> 00:04:15.300
Q&A feature. You can actually also upload questions there.

58
00:04:15.300 --> 00:04:18.200
So if someone asks something that you're also curious about, you know,

59
00:04:18.200 --> 00:04:21.800
feel free to to like it give it an upvote and I'll

60
00:04:21.800 --> 00:04:24.800
kind of dish out the questions at the end based on

61
00:04:24.800 --> 00:04:26.800
sort of the most popular questions.

62
00:04:27.700 --> 00:04:30.000
So I'll be coming back in at the end and I'll be

63
00:04:30.400 --> 00:04:32.400
directing questions to our panelists.

64
00:04:33.300 --> 00:04:36.300
There is also a chat function in Zoom. You should

65
00:04:36.300 --> 00:04:39.100
see it at the bottom of your window and you know, we definitely want to hear from you

66
00:04:39.100 --> 00:04:42.300
today. So don't be shy feel free to use that chat

67
00:04:42.300 --> 00:04:45.800
button to share your thoughts with the panelists directly or with other

68
00:04:45.800 --> 00:04:48.400
participants. Just remember to keep it

69
00:04:48.400 --> 00:04:49.000
on topic.

70
00:04:50.300 --> 00:04:53.400
This webinar will be recorded. So we'll share the link to

71
00:04:53.400 --> 00:04:57.100
access that after the webinar has ended and there

72
00:04:56.100 --> 00:04:59.300
will also be a survey at the end. So keep

73
00:04:59.300 --> 00:05:02.500
an eye out for that. It should appear in your Zoom window. So just don't

74
00:05:02.500 --> 00:05:05.300
quick click away too quickly, you know, we really love

75
00:05:05.300 --> 00:05:08.400
to hear your feedback about today's webinar as

76
00:05:08.400 --> 00:05:11.400
well as suggestions for future topics. We are all ears for

77
00:05:11.400 --> 00:05:11.500
that.

78
00:05:13.200 --> 00:05:16.700
Okay. So housekeeping is over one last

79
00:05:16.700 --> 00:05:19.500
thing if you are not familiar with its

80
00:05:20.100 --> 00:05:23.600
we are a well welcome. First of all Hello. Nice

81
00:05:23.600 --> 00:05:24.000
to meet you.

82
00:05:24.900 --> 00:05:27.700
We are a technology company established 25

83
00:05:27.700 --> 00:05:30.700
years ago with the vision of bringing test delivery online

84
00:05:30.700 --> 00:05:33.800
something that at the time was unheard

85
00:05:33.800 --> 00:05:36.200
of during you know over the

86
00:05:36.200 --> 00:05:39.200
last 25 years. We've been at the Forefront of innovation in

87
00:05:39.200 --> 00:05:42.600
the industry working really closely with our partners to develop a

88
00:05:42.600 --> 00:05:45.200
suite of software and services that not only

89
00:05:45.200 --> 00:05:48.100
support assessment programs, but also help them evolve.

90
00:05:49.100 --> 00:05:52.500
Our mission is to drive the assessment industry forward through

91
00:05:52.500 --> 00:05:55.300
partnership technology and investment.

92
00:05:56.100 --> 00:06:00.000
Now this is a good segue into our discussion today because we're

93
00:05:59.500 --> 00:06:02.400
often working with our partners to you know,

94
00:06:02.400 --> 00:06:05.700
take a step back and see how the market is evolving

95
00:06:05.700 --> 00:06:08.700
and help bridge that gap between the old

96
00:06:08.700 --> 00:06:11.700
way of doing things to to the new modern way

97
00:06:11.700 --> 00:06:12.100
of doing things.

98
00:06:13.100 --> 00:06:16.300
And today we'll be looking at recertification through that

99
00:06:16.300 --> 00:06:19.500
lens sharing what we've learned along the way about the

100
00:06:19.500 --> 00:06:20.400
old and the new.

101
00:06:21.300 --> 00:06:24.800
So without further Ado, I'm going to bring in our panelists for

102
00:06:24.800 --> 00:06:27.500
today Ryan and Elizabeth are going to introduce themselves and

103
00:06:27.500 --> 00:06:31.400
I'll disappear into the background until the

104
00:06:31.400 --> 00:06:33.600
very end. So Ryan and Elizabeth take it away.

105
00:06:35.900 --> 00:06:37.100
Thanks, Andy.

106
00:06:38.700 --> 00:06:39.100
Hi everybody.

107
00:06:40.600 --> 00:06:43.200
Hi, I'm I'm Elizabeth Pitta. I'm a

108
00:06:43.200 --> 00:06:46.900
senior product manager here at its and I've

109
00:06:46.900 --> 00:06:49.900
been working with Ryan on our continuous certification

110
00:06:49.900 --> 00:06:52.500
solution for the past three years

111
00:06:52.500 --> 00:06:55.600
four years. It's been a while. It's like

112
00:06:55.600 --> 00:06:59.600
a while. It's a journey and and

113
00:06:58.600 --> 00:07:01.200
I'm very excited to be here today to

114
00:07:01.200 --> 00:07:01.800
talk to you all about it.

115
00:07:02.700 --> 00:07:05.900
And my name is Ryan Howard. I'm the director of medical assessments

116
00:07:05.900 --> 00:07:09.300
here at ICS. I've been part of the implementation

117
00:07:08.300 --> 00:07:12.300
team with a number of its's solutions.

118
00:07:11.300 --> 00:07:15.200
And so I'll be bringing those experiences,

119
00:07:14.200 --> 00:07:17.400
you know from that journey into this webinar.

120
00:07:17.400 --> 00:07:20.300
So without further Ado I'm going

121
00:07:20.300 --> 00:07:21.300
to get us started.

122
00:07:22.500 --> 00:07:26.300
All right. So our goal today is to discuss its

123
00:07:25.300 --> 00:07:28.500
is continuous certification product Journey,

124
00:07:28.500 --> 00:07:32.000
you know from inception to where it is today and the

125
00:07:31.300 --> 00:07:34.900
hopes that you know, even if you aren't using our solution that

126
00:07:34.900 --> 00:07:37.500
you're presented with what we believe

127
00:07:37.500 --> 00:07:41.300
is a modern flexible reliable approach

128
00:07:40.300 --> 00:07:42.300
to recertification.

129
00:07:43.100 --> 00:07:46.700
Before we dive into the fun details of how to modernize the

130
00:07:46.700 --> 00:07:49.100
approach to research. I want to spend just a

131
00:07:49.100 --> 00:07:52.800
couple of minutes discussing, you know, how recertification examinations

132
00:07:52.800 --> 00:07:55.100
have traditionally been structured so that

133
00:07:55.100 --> 00:07:57.200
we can better understand the journey to where we are now.

134
00:07:58.200 --> 00:08:01.400
I will say that this explanation is coming from the

135
00:08:01.400 --> 00:08:04.500
lens of the medical space since that is where you

136
00:08:04.500 --> 00:08:07.400
know, its got started with our continuous certification solution.

137
00:08:07.400 --> 00:08:10.400
But the solution is relevant is

138
00:08:10.400 --> 00:08:14.000
impactful for all spaces and we encourage all spaces

139
00:08:13.200 --> 00:08:17.200
to, you know, ask questions and stay present.

140
00:08:19.100 --> 00:08:22.400
Traditionally after a high stakes initial certification, you

141
00:08:22.400 --> 00:08:25.600
know candidates were required to maintain their

142
00:08:25.600 --> 00:08:28.800
certification by completing sometimes up to

143
00:08:28.800 --> 00:08:31.900
three components, you know, one of which banking continual

144
00:08:31.900 --> 00:08:34.400
education credits or CE by

145
00:08:34.400 --> 00:08:38.800
performing things like trainings or attending webinars performing activities

146
00:08:38.800 --> 00:08:41.400
also annual or by annual

147
00:08:41.400 --> 00:08:44.200
quality improvement activities Often by conducting, you

148
00:08:44.200 --> 00:08:47.200
know, peer reviews of Imaging and other

149
00:08:47.200 --> 00:08:47.700
procedures.

150
00:08:48.700 --> 00:08:51.800
And a high stakes assessment every five and

151
00:08:51.800 --> 00:08:54.500
sometimes even 10 years where the

152
00:08:54.500 --> 00:08:57.700
candidate has to travel to a testing center and essentially answer

153
00:08:57.700 --> 00:09:00.600
the recertification examination again.

154
00:09:01.400 --> 00:09:04.400
So I'm going to paint a little picture for you for what this experience

155
00:09:04.400 --> 00:09:07.500
feels like say I'm a doctor. I was

156
00:09:07.500 --> 00:09:10.300
certified, you know, nine-ish years ago. Now, I've

157
00:09:10.300 --> 00:09:13.900
got my cozy Medical Practice on the beaches of the Carolinas and

158
00:09:13.900 --> 00:09:17.200
I'm due for my recertification examination. I'm

159
00:09:16.200 --> 00:09:19.200
notified that for my research I have

160
00:09:19.200 --> 00:09:20.700
to travel to Chicago.

161
00:09:21.700 --> 00:09:23.200
sit for a five-hour examination

162
00:09:24.200 --> 00:09:25.300
not excited about it.

163
00:09:26.200 --> 00:09:29.400
Now I've got to take time off work. I have to reschedule my patients. I have

164
00:09:29.400 --> 00:09:32.200
to find after school care for my kids. I have to spend thousands of

165
00:09:32.200 --> 00:09:35.800
dollars on airfare accommodations the test

166
00:09:35.800 --> 00:09:38.300
itself. I have to study and prepare for the test

167
00:09:38.300 --> 00:09:41.500
after my 10-hour workday and juggling between sports and ballet

168
00:09:41.500 --> 00:09:44.500
recitals for the kids, you know, you can really see that

169
00:09:44.500 --> 00:09:47.400
these high stakes exams were

170
00:09:47.400 --> 00:09:50.400
taxing on both the certifying boards and the

171
00:09:50.400 --> 00:09:52.500
candidates and they really had some major drawbacks.

172
00:09:53.400 --> 00:09:56.900
You know with a point in time assessment, you're really

173
00:09:56.900 --> 00:09:59.400
only proving what you know at

174
00:09:59.400 --> 00:10:02.000
that time and based off a feedback, you know

175
00:10:02.600 --> 00:10:05.400
candidates weren't retaining most of the knowledge due

176
00:10:05.400 --> 00:10:08.300
to you know, they're cramming for the test and honestly, you know,

177
00:10:08.300 --> 00:10:11.200
forgetting immediately after their recertification and not having

178
00:10:11.200 --> 00:10:12.900
to address it again for another 10 years.

179
00:10:14.700 --> 00:10:18.000
So a number of years ago in the medical space,

180
00:10:17.200 --> 00:10:20.600
they started conducting, you know research and took

181
00:10:20.600 --> 00:10:23.400
it upon themselves to really rethink what

182
00:10:23.400 --> 00:10:26.300
recertification should look like. You know, what should remain what

183
00:10:26.300 --> 00:10:29.400
should be new what should change in most cases

184
00:10:29.400 --> 00:10:32.200
the the CE and the QI activities provided really

185
00:10:32.200 --> 00:10:35.400
measurable benefits with little burden on candidates. So in many

186
00:10:35.400 --> 00:10:39.200
instances programs chose to maintain these

187
00:10:38.200 --> 00:10:41.700
pieces of research, but revamp

188
00:10:41.700 --> 00:10:43.900
what was clearly needed for the assessment piece.

189
00:10:44.800 --> 00:10:47.400
So there's really a big recognition that

190
00:10:47.400 --> 00:10:50.500
assessment is important to recertification. But

191
00:10:50.500 --> 00:10:53.300
each program, you know has its own

192
00:10:53.300 --> 00:10:56.600
idea of how to provide, you know value to their candidates further

193
00:10:56.600 --> 00:10:59.600
their knowledge less than the burden while still achieving

194
00:10:59.600 --> 00:11:03.500
their assessment requirements integrating longitudinal

195
00:11:02.500 --> 00:11:05.300
assessment into that solution was

196
00:11:05.300 --> 00:11:08.400
thought to be a really good way to maintain the

197
00:11:08.400 --> 00:11:11.400
assessment requirement, but also to ensure a candidate

198
00:11:11.400 --> 00:11:14.400
knowledge growth growth knowledge and growth with a

199
00:11:14.400 --> 00:11:17.600
more continuous and less burdensome approach

200
00:11:17.600 --> 00:11:18.600
to assessment.

201
00:11:20.100 --> 00:11:23.400
So about five years ago, you know, we at its you know,

202
00:11:23.400 --> 00:11:26.500
we're also thinking about how to you know enhance assessment

203
00:11:26.500 --> 00:11:29.500
and ways to provide value not to just

204
00:11:29.500 --> 00:11:32.800
candidates but Industries as a whole so that assessment

205
00:11:32.800 --> 00:11:35.200
wasn't just a way to you know, check a box

206
00:11:35.200 --> 00:11:38.500
on a requirement, but it could also be a mechanism

207
00:11:38.500 --> 00:11:41.400
for growth and learning, you know, as you know,

208
00:11:41.400 --> 00:11:44.500
we at its were developing this product medical boards were

209
00:11:44.500 --> 00:11:47.500
coming to us with their own thoughts and ideas

210
00:11:47.500 --> 00:11:51.200
of how they wanted to change recertification for their candidates, you

211
00:11:50.200 --> 00:11:53.700
know from the American Board of surgery with

212
00:11:53.700 --> 00:11:56.500
their continuous certification product where candidates were

213
00:11:56.500 --> 00:11:59.600
presented with rationales after answering questions

214
00:11:59.600 --> 00:12:03.200
along with the ability to read attempt incorrect

215
00:12:02.200 --> 00:12:05.800
questions after completing, you know their assessment

216
00:12:05.800 --> 00:12:07.200
and reviewing the learning material.

217
00:12:08.100 --> 00:12:11.300
To the national board of osteopathic medical examiners in their Catalyst product

218
00:12:11.300 --> 00:12:14.700
where they integrated learning material and longitudinal assessment

219
00:12:14.700 --> 00:12:17.400
into the mix to provoke to promote content in

220
00:12:17.400 --> 00:12:20.500
more digestible quarterly deliveries instead of the once

221
00:12:20.500 --> 00:12:21.800
in five year once in 10 year.

222
00:12:22.500 --> 00:12:25.100
And the American Board of medical studies with their certainly product.

223
00:12:25.100 --> 00:12:28.900
They also took a longitudinal approach to recertification, but

224
00:12:28.900 --> 00:12:31.600
with adaptive targeted item deliveries, so

225
00:12:31.600 --> 00:12:34.200
that candidates were getting questions relevant to

226
00:12:34.200 --> 00:12:37.300
their specialty and adaptive based off of performance.

227
00:12:38.600 --> 00:12:41.500
So as you can see, you know, one thing, you

228
00:12:41.500 --> 00:12:44.500
know is really clear each board needed something a

229
00:12:44.500 --> 00:12:48.300
little bit different and that's the approach that we took, you

230
00:12:47.300 --> 00:12:50.600
know with developing this continuous certification

231
00:12:50.600 --> 00:12:53.000
product Elizabeth is going to

232
00:12:53.200 --> 00:12:56.400
go into greater detail on the foundations and the key

233
00:12:56.400 --> 00:12:59.100
features of the product, but it was clear that

234
00:12:59.100 --> 00:13:02.900
the system needed to be, you know, mobile capable to accommodate,

235
00:13:02.900 --> 00:13:05.100
you know, the on the go of the modern world

236
00:13:05.100 --> 00:13:08.500
incorporate opportunities for learning so they can really

237
00:13:08.500 --> 00:13:11.200
get a deeper understanding of the content in the

238
00:13:11.200 --> 00:13:14.400
assessment and flexible to be able to support, you

239
00:13:14.400 --> 00:13:17.300
know, multiple Industries, you know, all of them needing something

240
00:13:17.300 --> 00:13:20.900
a little bit different, you know, I am coming from this, you

241
00:13:20.900 --> 00:13:23.500
know with my history of the medical space, but you

242
00:13:23.500 --> 00:13:26.500
know, as I said, you know, this is a product that you

243
00:13:26.500 --> 00:13:30.300
know, we believe, you know, we'll revolutionize, you know recertification and

244
00:13:29.300 --> 00:13:32.400
not just the medical space but the it space

245
00:13:32.400 --> 00:13:33.700
and every other space in between

246
00:13:36.500 --> 00:13:37.400
What that opacity of Elizabeth?

247
00:13:38.500 --> 00:13:41.300
Ryan so like Ryan was

248
00:13:41.300 --> 00:13:44.300
saying, you know, this has been a product Journey. So when

249
00:13:44.300 --> 00:13:47.700
we took a step back we had when

250
00:13:47.700 --> 00:13:50.800
we had these boards come to us and say, you know, I we want

251
00:13:50.800 --> 00:13:54.400
to change the way that we're doing recertification itss traditionally

252
00:13:53.400 --> 00:13:56.700
done a lot of high-stakes certification.

253
00:13:58.100 --> 00:14:01.200
Where we're candidates are going to a test center,

254
00:14:01.200 --> 00:14:05.100
you know, they're doing this point in time exam things

255
00:14:04.100 --> 00:14:08.700
are very secure and highly managed and

256
00:14:07.700 --> 00:14:10.300
this was kind of a bit of a turn for

257
00:14:10.300 --> 00:14:13.500
us. So we had to take a step back and actually look at what we

258
00:14:13.500 --> 00:14:14.700
were trying to provide and

259
00:14:15.700 --> 00:14:18.800
kind of going along those lines. We did some

260
00:14:18.800 --> 00:14:21.700
research and kind of landed on the adult learning

261
00:14:21.700 --> 00:14:25.200
theory as a foundation for how we

262
00:14:25.200 --> 00:14:28.200
were going to move forward with this new product. So there's a lot

263
00:14:28.200 --> 00:14:31.400
of research surrounding the study of adult learning but the

264
00:14:31.400 --> 00:14:35.300
adult learning theory is kind of emerged as one of the top theories

265
00:14:34.300 --> 00:14:37.800
studied by Scholars, and it

266
00:14:37.800 --> 00:14:40.700
looks at adult learning and how that differs from primary

267
00:14:40.700 --> 00:14:43.600
learning that you do in primary education in

268
00:14:43.600 --> 00:14:44.100
childhood.

269
00:14:45.400 --> 00:14:49.200
And so when we started doing our brainstorming we

270
00:14:48.200 --> 00:14:51.200
looked at the the adult learning theory and

271
00:14:51.200 --> 00:14:53.300
it's five assumptions for how adults learn.

272
00:14:53.900 --> 00:14:56.100
So each one of these assumptions speak to

273
00:14:56.100 --> 00:14:59.500
General ways in which adults perceive learning and

274
00:14:59.500 --> 00:15:02.600
how they prefer to take in information. The first

275
00:15:02.600 --> 00:15:05.200
one is the self-concept which refers to

276
00:15:05.200 --> 00:15:08.200
the idea that there is a difference between how children learn in

277
00:15:08.200 --> 00:15:11.600
primary education where they're passive Learners who take

278
00:15:11.600 --> 00:15:14.200
in information that is taught to them as

279
00:15:14.200 --> 00:15:17.500
opposed to adults who are more in control of the

280
00:15:17.500 --> 00:15:20.700
information that they they're learning. They seek

281
00:15:20.700 --> 00:15:24.500
out different forms of communication and

282
00:15:23.500 --> 00:15:26.400
and learning and they

283
00:15:26.400 --> 00:15:29.200
really want more control over what they want

284
00:15:29.200 --> 00:15:32.400
to learn when they want to learn it and how

285
00:15:32.400 --> 00:15:33.300
they want to learn it.

286
00:15:34.200 --> 00:15:36.800
So they're really seeking that autonomy over their education.

287
00:15:37.700 --> 00:15:41.300
The second piece learning from experience. I'm sure

288
00:15:41.300 --> 00:15:44.500
it's most of you know, as you grow you have many life experiences.

289
00:15:44.500 --> 00:15:47.400
So adults really have that wealth of

290
00:15:47.400 --> 00:15:50.500
Life Experiences different than than children

291
00:15:50.500 --> 00:15:53.400
and that those experiences can be

292
00:15:53.400 --> 00:15:56.900
used to inform and enrich their learning. They've lived

293
00:15:56.900 --> 00:15:59.200
life and often had to like learn

294
00:15:59.200 --> 00:16:02.500
on the job use real-world situations.

295
00:16:02.500 --> 00:16:05.400
And whether that's the Practical experience

296
00:16:05.400 --> 00:16:08.200
of of the topic that's being

297
00:16:08.200 --> 00:16:11.800
tested or whether it's bringing a different perspective to

298
00:16:11.800 --> 00:16:14.300
new material that's being learned drawing from

299
00:16:14.300 --> 00:16:17.200
that knowledge is a really powerful factor for learning new

300
00:16:17.200 --> 00:16:20.600
material and retaining it because it gives some context to

301
00:16:20.600 --> 00:16:21.400
what they're actually learning.

302
00:16:22.900 --> 00:16:25.200
The third assumption the Readiness to learn

303
00:16:25.200 --> 00:16:28.300
is the idea that adults learn best when the subject

304
00:16:28.300 --> 00:16:29.300
matters important to them.

305
00:16:29.900 --> 00:16:32.800
So the more relevant a topic is to an

306
00:16:32.800 --> 00:16:35.200
individual the more they're willing to invest time and

307
00:16:35.200 --> 00:16:37.200
effort into the learning of that subject.

308
00:16:38.800 --> 00:16:41.700
The fourth one listed on here is the orientation to

309
00:16:41.700 --> 00:16:44.400
learning and this is an assumption that depending on

310
00:16:44.400 --> 00:16:47.300
the research paper that you're looking at is sometimes split into two

311
00:16:47.300 --> 00:16:50.300
separate assumptions just depends

312
00:16:50.300 --> 00:16:53.400
on on who the author is but it boils down

313
00:16:53.400 --> 00:16:55.400
to two basic points. And the first one is

314
00:16:56.400 --> 00:16:59.500
When learning adults want to know why they should obtain this

315
00:16:59.500 --> 00:17:02.400
knowledge. What makes it useful what makes it worth their time.

316
00:17:02.400 --> 00:17:05.400
And then secondly, they want to know how they

317
00:17:05.400 --> 00:17:08.800
can immediately apply this to their lives. So how

318
00:17:08.800 --> 00:17:11.500
can this information solve existing problems that

319
00:17:11.500 --> 00:17:14.400
they've seen how will it benefit their life today

320
00:17:14.400 --> 00:17:17.400
tomorrow next week next month. It's not

321
00:17:17.400 --> 00:17:20.400
the theoretical application of

322
00:17:20.400 --> 00:17:23.700
this knowledge sometime in the future. So it's not algebra

323
00:17:23.700 --> 00:17:24.600
that you're going to use one day.

324
00:17:25.600 --> 00:17:28.400
I've used. Yeah algebra so

325
00:17:28.400 --> 00:17:31.800
many times in my daily life. It's definitely that more immediate

326
00:17:31.800 --> 00:17:33.400
application of that knowledge.

327
00:17:34.500 --> 00:17:37.800
So adults will generally gravitate towards learning opportunities that

328
00:17:37.800 --> 00:17:40.800
are task oriented life focused

329
00:17:40.800 --> 00:17:43.000
problem solving Centric.

330
00:17:44.500 --> 00:17:47.500
This one on here is the motivation to learn which is

331
00:17:47.500 --> 00:17:50.300
similar to the first one where as children. We learn because

332
00:17:50.300 --> 00:17:54.000
of external factors such as parents and teachers. However, as

333
00:17:53.500 --> 00:17:56.300
adults we want to learn for our own

334
00:17:56.300 --> 00:17:59.300
reasons, so it for example could be progress in work

335
00:17:59.300 --> 00:18:02.400
or to boost self-esteem. It's a it's

336
00:18:02.400 --> 00:18:05.500
an internal motivation that drives learning and

337
00:18:05.500 --> 00:18:08.800
the retention of knowledge as opposed to an arbitrary

338
00:18:08.800 --> 00:18:11.500
means to an end like passing a class in order to advance to

339
00:18:11.500 --> 00:18:11.900
the next level.

340
00:18:13.900 --> 00:18:16.300
He brought me some PTSD but

341
00:18:16.300 --> 00:18:17.200
I mentioning algebra.

342
00:18:19.700 --> 00:18:21.300
So applicable it is.

343
00:18:24.400 --> 00:18:28.200
Though, there we go. So when we

344
00:18:28.200 --> 00:18:31.100
started building our recertification solution, we wanted to

345
00:18:31.100 --> 00:18:34.400
take those five assumptions of adult learning theory and translate them

346
00:18:34.400 --> 00:18:37.600
into some practical features. That would make this platform really useful.

347
00:18:38.400 --> 00:18:41.300
And since our our part

348
00:18:41.300 --> 00:18:45.100
of the recertification process is the assessment portion where

349
00:18:44.100 --> 00:18:47.900
we test to make sure all the candidates have the knowledge necessary

350
00:18:47.900 --> 00:18:50.500
to be recertified. One of the first key

351
00:18:50.500 --> 00:18:53.300
points was to embrace the longitudinal assessment

352
00:18:53.300 --> 00:18:56.600
model that Ryan spoke about where we're delivering a smaller number

353
00:18:56.600 --> 00:18:59.100
of questions over a longer period of time.

354
00:19:00.200 --> 00:19:03.400
As opposed to all the questions all at once at a

355
00:19:03.400 --> 00:19:03.800
point in time.

356
00:19:04.700 --> 00:19:07.200
So for us that meant that we have

357
00:19:07.200 --> 00:19:10.700
we have two different time intervals that we look at for the

358
00:19:10.700 --> 00:19:11.400
space repetition.

359
00:19:12.200 --> 00:19:15.600
The smaller period of time in which candidates have

360
00:19:15.600 --> 00:19:19.400
to answer a set of questions. We call

361
00:19:19.400 --> 00:19:20.200
it a segment.

362
00:19:21.500 --> 00:19:24.000
In the larger interval of

363
00:19:24.100 --> 00:19:27.800
time over where their ultimately being a assessed over

364
00:19:27.800 --> 00:19:31.100
the course of the entire delivery of

365
00:19:31.100 --> 00:19:34.500
questions. We call that a cycle and together these

366
00:19:34.500 --> 00:19:37.400
two intervals help to break down learning into

367
00:19:37.400 --> 00:19:40.600
immediate opportunities that build into longtime knowledge

368
00:19:40.600 --> 00:19:41.100
retention.

369
00:19:42.300 --> 00:19:45.200
So we built the system to be configurable for what those time intervals would

370
00:19:45.200 --> 00:19:48.400
look like for some of our clients a segment is

371
00:19:48.400 --> 00:19:51.300
quarterly. They have the candidate has 10 to 20

372
00:19:51.300 --> 00:19:53.400
questions that they have to answer by the end of a quarter.

373
00:19:54.100 --> 00:19:57.300
and then their certification performance is judged over

374
00:19:57.300 --> 00:20:00.800
an overall longer period of time so sometimes

375
00:20:00.800 --> 00:20:03.400
like Ryan mentioned that's as long as five six ten

376
00:20:03.400 --> 00:20:06.200
years and that's to give the candidate time to

377
00:20:06.200 --> 00:20:09.400
demonstrate their knowledge acquisition and retention so we

378
00:20:09.400 --> 00:20:12.400
can we can see the performance from day one to

379
00:20:12.400 --> 00:20:15.200
year 10 and see any

380
00:20:16.600 --> 00:20:18.800
areas of concern or also areas of improvement

381
00:20:20.300 --> 00:20:23.400
So for other clients though, we also do this for a

382
00:20:23.400 --> 00:20:26.200
much shorter period of time we have some clients that are

383
00:20:26.200 --> 00:20:29.500
doing a short as a few weeks for a segment and then

384
00:20:29.500 --> 00:20:32.400
there are summative decision about whether the candidates met.

385
00:20:32.400 --> 00:20:36.500
The requirements for recertification is also much shorter sometimes

386
00:20:35.500 --> 00:20:38.700
on a yearly basis where each year

387
00:20:38.700 --> 00:20:42.100
the candidates assessment essentially resets and

388
00:20:41.100 --> 00:20:44.300
they have a blank slate to start over again.

389
00:20:45.100 --> 00:20:48.300
So it's really down to kind of what makes the most sense for both

390
00:20:48.300 --> 00:20:51.700
admins or test creators as well as the candidates

391
00:20:51.700 --> 00:20:54.300
because the goal of reimagining this

392
00:20:54.300 --> 00:20:58.000
certificate recertification is to make it as unburden some

393
00:20:57.200 --> 00:20:59.100
as possible for everyone.

394
00:21:00.800 --> 00:21:03.500
One of the next key pieces to the

395
00:21:03.500 --> 00:21:07.100
platform was to tie in the orientation

396
00:21:06.100 --> 00:21:09.400
of learning concept and give

397
00:21:09.400 --> 00:21:12.600
candidates practical resources that are available before

398
00:21:12.600 --> 00:21:16.000
doing their assessment immediately after answering

399
00:21:15.200 --> 00:21:18.500
a question during the assessment and then

400
00:21:18.500 --> 00:21:21.800
also have those learning resources available

401
00:21:21.800 --> 00:21:24.400
after answering all the questions and that

402
00:21:24.400 --> 00:21:27.600
could be like days and months and years after they've seen

403
00:21:27.600 --> 00:21:28.100
the material.

404
00:21:28.900 --> 00:21:31.400
And that's to give that candidate many

405
00:21:31.400 --> 00:21:34.300
opportunities to go back and study up on an area that

406
00:21:34.300 --> 00:21:37.200
they may need to work on or delve deeper into a particular

407
00:21:37.200 --> 00:21:40.400
topic and we really did this to encourage

408
00:21:40.400 --> 00:21:43.500
the the learning assessment to run

409
00:21:43.500 --> 00:21:46.700
in parallel to encourage the

410
00:21:46.700 --> 00:21:48.500
learning to run in parallel with the assessment.

411
00:21:49.600 --> 00:21:52.100
We also wanted to make sure that this material could be

412
00:21:52.100 --> 00:21:56.200
updated at any time because as I

413
00:21:55.200 --> 00:21:58.500
think we all know Industries and the

414
00:21:58.500 --> 00:22:01.800
world changes and evolves every year

415
00:22:01.800 --> 00:22:04.600
and since we have many

416
00:22:04.600 --> 00:22:07.300
medical boards as clients, you know,

417
00:22:07.300 --> 00:22:09.800
a lot of medical information evolves over time.

418
00:22:10.500 --> 00:22:13.300
One of the more interesting things that we've

419
00:22:13.300 --> 00:22:16.700
seen over the past two to three years was how

420
00:22:16.700 --> 00:22:20.000
boards use this platform to

421
00:22:19.100 --> 00:22:22.700
get information out about probably the

422
00:22:22.700 --> 00:22:25.700
most emerging medical topic of the time covid

423
00:22:25.700 --> 00:22:28.500
and how to get the information out on covid

424
00:22:28.500 --> 00:22:31.900
best practices. So as I

425
00:22:31.900 --> 00:22:35.600
as I'm sure you've seen the protocols

426
00:22:34.600 --> 00:22:38.300
regarding covid care have evolved significantly

427
00:22:37.300 --> 00:22:41.200
over the last three years and it

428
00:22:40.200 --> 00:22:44.000
gave us a really unique opportunity to update

429
00:22:43.600 --> 00:22:46.300
to look at the learning material and have

430
00:22:46.300 --> 00:22:48.900
that be something that could be updated.

431
00:22:50.500 --> 00:22:53.400
Really on like a monthly basis as things evolved and

432
00:22:53.400 --> 00:22:57.300
changed. It also gave us the opportunity to create

433
00:22:56.300 --> 00:22:59.500
test content around an

434
00:22:59.500 --> 00:23:02.800
emerging topic so that it could be inserted into

435
00:23:02.800 --> 00:23:05.500
the candidates recertification Journey when

436
00:23:05.500 --> 00:23:08.900
it was most relevant rather than like years later when

437
00:23:08.900 --> 00:23:12.000
the topic of covid hopefully is

438
00:23:11.200 --> 00:23:15.000
more stale and not as relevant. So

439
00:23:14.300 --> 00:23:17.400
that was that's been a very interesting factor that

440
00:23:17.400 --> 00:23:20.200
we've seen with being able to update those

441
00:23:20.200 --> 00:23:21.400
resources at any time.

442
00:23:22.400 --> 00:23:25.300
And I think that's a really good point about, you know, being able to

443
00:23:25.300 --> 00:23:28.300
you know, having the system support, you know,

444
00:23:28.300 --> 00:23:31.700
immediate updating of you know, learning resources, obviously, you

445
00:23:31.700 --> 00:23:34.200
know, we wouldn't want to you know, impact test delivery and

446
00:23:34.200 --> 00:23:37.700
update content, you know throughout, you know, the content's journey,

447
00:23:37.700 --> 00:23:40.000
but you know disassociating, you know,

448
00:23:40.400 --> 00:23:43.400
the the learning opportunities, you know, providing Journal articles,

449
00:23:43.400 --> 00:23:46.800
you know URLs study guides summaries key

450
00:23:46.800 --> 00:23:49.500
learning points associating that to the

451
00:23:49.500 --> 00:23:52.600
items themselves and being able to publish that you know,

452
00:23:52.600 --> 00:23:55.700
or push it out to test takers, you know independently of

453
00:23:55.700 --> 00:23:58.900
test delivery I think is a really powerful and

454
00:23:58.900 --> 00:24:01.800
like you said Elizabeth, you know, helpful feature, you

455
00:24:01.800 --> 00:24:04.700
know, when things like, you know an epidemic, you know

456
00:24:04.700 --> 00:24:07.400
occur. Hopefully, we will never have to experience that again,

457
00:24:07.400 --> 00:24:10.700
but you know, the world's changing quite

458
00:24:10.700 --> 00:24:14.000
quickly so, you know having a system that

459
00:24:13.400 --> 00:24:16.200
you know can adapt to that change I

460
00:24:16.200 --> 00:24:17.000
think is really powerful.

461
00:24:18.400 --> 00:24:19.100
Yeah, absolutely.

462
00:24:20.400 --> 00:24:21.900
next slide excellent

463
00:24:26.400 --> 00:24:29.200
So the to incorporate the

464
00:24:29.200 --> 00:24:33.100
adult learning theory assumptions of self-concept where

465
00:24:32.100 --> 00:24:35.400
the candidates want more autonomy over what

466
00:24:35.400 --> 00:24:38.600
and where and how they're learning and to

467
00:24:38.600 --> 00:24:41.500
support also that Readiness of Readiness to learn assumption

468
00:24:41.500 --> 00:24:44.200
where candidates find Value in the material that's

469
00:24:44.200 --> 00:24:47.300
important and relevant to them. We really wanted

470
00:24:47.300 --> 00:24:51.000
to ensure that the platform was giving candidates that personalized learning

471
00:24:50.000 --> 00:24:51.400
approach.

472
00:24:52.100 --> 00:24:55.100
So in order to do this, we built out a few key features that we have

473
00:24:55.100 --> 00:24:58.600
listed here. And the first is a configurable algorithm

474
00:24:58.600 --> 00:25:01.500
that incorporates but a

475
00:25:01.500 --> 00:25:04.900
lot of data points including candidate performance and candidate

476
00:25:04.900 --> 00:25:07.400
personal feedback on on their

477
00:25:07.400 --> 00:25:10.100
performance and we use all of those

478
00:25:10.100 --> 00:25:13.200
data points to deliver test contents. That

479
00:25:13.200 --> 00:25:16.000
is the most relevant to the individual candidate.

480
00:25:16.700 --> 00:25:19.100
So, like I said, we use

481
00:25:19.100 --> 00:25:22.100
an approach that incorporates a lot of different data points

482
00:25:22.100 --> 00:25:25.300
as well as waiting on each one of those data points

483
00:25:25.300 --> 00:25:29.200
that can all be configured so that you know,

484
00:25:29.200 --> 00:25:32.800
if a client decided they wanted to deliver more

485
00:25:32.800 --> 00:25:35.200
test questions and put the emphasis on

486
00:25:35.200 --> 00:25:36.000
ones that were

487
00:25:37.700 --> 00:25:40.600
That the candidate got wrong, but they thought that they

488
00:25:40.600 --> 00:25:43.900
might have gotten right and they were very confident in their answer. We

489
00:25:43.900 --> 00:25:46.100
can put a higher weight on those so that more of those

490
00:25:46.100 --> 00:25:50.200
types of questions are selected for future future segments

491
00:25:49.200 --> 00:25:53.500
the candidates taking questions and it

492
00:25:53.500 --> 00:25:53.600
also

493
00:25:54.600 --> 00:25:57.400
What that means is if we're if we have a

494
00:25:57.400 --> 00:26:00.500
large enough pool of questions for any client

495
00:26:00.500 --> 00:26:03.200
that no two candidates will really ever get the

496
00:26:03.200 --> 00:26:06.700
exact same test and that factors

497
00:26:06.700 --> 00:26:10.000
even like specialty certifications or having multiple

498
00:26:09.800 --> 00:26:12.400
certifications can be built

499
00:26:12.400 --> 00:26:15.300
in so that all of that is taken into account

500
00:26:15.300 --> 00:26:18.700
to give a more relevant test question experience

501
00:26:18.700 --> 00:26:21.200
and you know

502
00:26:21.200 --> 00:26:25.900
it we've seen that be very useful especially

503
00:26:24.900 --> 00:26:27.900
since this is we're

504
00:26:27.900 --> 00:26:30.200
taking a different approach with the way that we're doing security for

505
00:26:30.200 --> 00:26:30.900
these exams.

506
00:26:31.600 --> 00:26:34.500
And having you know two candidates who

507
00:26:34.500 --> 00:26:38.000
have maybe even the same credentials and the

508
00:26:37.100 --> 00:26:40.300
same specialty certificates won't have the

509
00:26:40.300 --> 00:26:43.200
same questions on their form just because of of the

510
00:26:43.200 --> 00:26:46.200
way that they have answered and their performance in previous

511
00:26:46.200 --> 00:26:49.200
questions. So it really is a nice way to

512
00:26:49.200 --> 00:26:52.800
build insecurity just by having the

513
00:26:52.800 --> 00:26:55.600
the factor that no two

514
00:26:55.600 --> 00:26:56.500
tests are the same.

515
00:26:57.800 --> 00:27:00.100
So another feature that we

516
00:27:00.100 --> 00:27:03.800
built in was to customize the test experience in a

517
00:27:03.800 --> 00:27:06.500
way to enforce learning over time through the repetition

518
00:27:06.500 --> 00:27:09.400
of ideas material over the

519
00:27:09.400 --> 00:27:12.600
course of the entire cycle and we have two different

520
00:27:12.600 --> 00:27:16.700
mechanisms that we do this with we developed

521
00:27:15.700 --> 00:27:18.100
the product to give clients kind of

522
00:27:18.100 --> 00:27:21.100
multiple ways to do this, but the two that I'm going to

523
00:27:21.100 --> 00:27:24.200
talk about right now, we do retrys or re

524
00:27:24.200 --> 00:27:27.500
attempts and clones and they're very

525
00:27:27.500 --> 00:27:30.800
similar in that both take into account the

526
00:27:30.800 --> 00:27:34.600
candidates performance on previous questions with retrys,

527
00:27:33.600 --> 00:27:36.400
we present the exact same question to

528
00:27:36.400 --> 00:27:39.600
the candidate at a different interval to kind

529
00:27:39.600 --> 00:27:42.700
of reinforce the concept of what they're testing over

530
00:27:42.700 --> 00:27:45.400
time and then clones we

531
00:27:45.400 --> 00:27:48.300
present the same concept or area of information, but it

532
00:27:48.300 --> 00:27:51.300
is actually a different question also presented at

533
00:27:51.300 --> 00:27:54.700
a different interval and that's to kind of ensure that the knowledge that

534
00:27:54.700 --> 00:27:57.500
they had gained is retained

535
00:27:57.500 --> 00:27:57.600
over.

536
00:27:57.800 --> 00:27:58.100
time

537
00:27:58.800 --> 00:28:01.000
and we use these in conjunction. We have

538
00:28:01.300 --> 00:28:04.100
some clients that use both of them. We have someone that use one or the other

539
00:28:04.100 --> 00:28:07.300
but it's been very interesting to

540
00:28:07.300 --> 00:28:10.500
see the statistics that come about for candidates who

541
00:28:10.500 --> 00:28:13.300
see a question attempt it get it

542
00:28:13.300 --> 00:28:17.000
wrong and then have one of these other opportunities to either

543
00:28:16.300 --> 00:28:19.300
see the same question again, or to see a question on

544
00:28:19.300 --> 00:28:22.700
the same topic again and the improvement over time that

545
00:28:22.700 --> 00:28:25.200
having that repetition provides.

546
00:28:26.800 --> 00:28:29.300
And then finally the final piece

547
00:28:29.300 --> 00:28:32.200
that we wanted to incorporate into the solution was a way

548
00:28:32.200 --> 00:28:35.400
for candidates to know exactly where they stand within the recertification process

549
00:28:35.400 --> 00:28:38.900
and with their progress and performance at all times. And

550
00:28:38.900 --> 00:28:41.300
so to do that we built in a dynamic

551
00:28:41.300 --> 00:28:44.600
dashboard that is configurable configurable by

552
00:28:44.600 --> 00:28:47.100
client and it can show all types of

553
00:28:47.100 --> 00:28:50.600
performance indications such as progress within their

554
00:28:50.600 --> 00:28:53.100
segment progress within their

555
00:28:53.100 --> 00:28:56.400
entire cycle the percentage correct in

556
00:28:56.400 --> 00:28:59.600
any given topic area. The number of

557
00:28:59.600 --> 00:29:02.300
incorrect questions, the number of correct questions,

558
00:29:02.300 --> 00:29:05.600
we even have the comparison to

559
00:29:05.600 --> 00:29:08.300
their peers or a cut score so that

560
00:29:08.300 --> 00:29:11.600
they can see how they're doing in comparison to

561
00:29:11.600 --> 00:29:14.300
others that are following the

562
00:29:14.300 --> 00:29:17.300
same recertification process or a cut

563
00:29:17.300 --> 00:29:20.300
score where they know. I need to achieve 80% to get

564
00:29:20.300 --> 00:29:23.600
this recertification and I'm sitting

565
00:29:23.600 --> 00:29:26.300
right now at 85% So that's great. I'm above the cut score.

566
00:29:27.700 --> 00:29:31.600
And we even have a

567
00:29:30.600 --> 00:29:33.500
widget to help the candidate plan

568
00:29:33.500 --> 00:29:36.400
their assessment strategy and that we we

569
00:29:36.400 --> 00:29:39.500
built in to help them organize the time that they're spending per week

570
00:29:39.500 --> 00:29:41.200
on their assessment progress.

571
00:29:42.700 --> 00:29:45.900
And all of this combines to give the candidates insight

572
00:29:45.900 --> 00:29:48.200
into their performance and and really help them understand

573
00:29:48.200 --> 00:29:52.000
the areas where they may need Improvement or where

574
00:29:51.300 --> 00:29:55.300
they're doing very well and I think

575
00:29:55.300 --> 00:29:58.800
I mentioned it before but we've we've seen how this data increases

576
00:29:58.800 --> 00:30:01.400
the motivation to continue learning and growing and

577
00:30:01.400 --> 00:30:03.700
how performance is improved over time.

578
00:30:04.800 --> 00:30:08.000
And and that dashboard we have over 250

579
00:30:07.300 --> 00:30:11.000
metrics that we can configure to

580
00:30:10.200 --> 00:30:13.200
kind of display in the various dashboard widgets and

581
00:30:13.200 --> 00:30:16.300
work with we work with each client when we're bringing them on

582
00:30:16.300 --> 00:30:19.400
to make sure that we're giving candidates the right information that's

583
00:30:19.400 --> 00:30:22.400
important and relevant to them on their recertification Journey.

584
00:30:23.900 --> 00:30:26.800
It is really interesting to see how you know these boards

585
00:30:26.800 --> 00:30:29.300
or you know clients are using

586
00:30:29.300 --> 00:30:33.200
the dashboards. You know, I feel like no no

587
00:30:32.200 --> 00:30:35.400
1 dashboard is the same or no

588
00:30:35.400 --> 00:30:38.100
two dashboards are the same and you know,

589
00:30:38.100 --> 00:30:41.500
it really gets to you know, now that we're delivering, you know

590
00:30:41.500 --> 00:30:45.200
questions over an extended period of time what type

591
00:30:44.200 --> 00:30:47.300
of information do you want to convey about

592
00:30:47.300 --> 00:30:51.000
progress and performance, you know to your candidates some programs

593
00:30:50.300 --> 00:30:54.300
don't want to show you know, any performance metrics,

594
00:30:53.300 --> 00:30:56.600
you know, no correctness because they want to consume,

595
00:30:56.600 --> 00:30:59.700
you know, the data internally, you know process it

596
00:30:59.700 --> 00:31:02.400
key changes, you know, all of that, you know

597
00:31:02.400 --> 00:31:06.200
kind of stuff and then you know present performance and

598
00:31:06.200 --> 00:31:09.400
you know a score report, whereas some you know want to

599
00:31:09.400 --> 00:31:12.500
show you like this example we have on the screen here. You know,

600
00:31:12.500 --> 00:31:16.200
how are you doing in this particular quarter in

601
00:31:15.200 --> 00:31:18.800
addition to how this quarter is contributing

602
00:31:18.800 --> 00:31:21.400
to all the quarters in the past or

603
00:31:21.400 --> 00:31:23.900
showing cumulative score. So it's been

604
00:31:23.900 --> 00:31:26.300
really fascinating to see you know,

605
00:31:26.300 --> 00:31:29.200
how programs are using these dashboards.

606
00:31:30.500 --> 00:31:33.200
Yeah, thank you Ryan. Um, you also reminded me

607
00:31:33.200 --> 00:31:36.300
I skipped over the third point in that first bullet, which is

608
00:31:36.300 --> 00:31:40.100
that we also provided a support

609
00:31:39.100 --> 00:31:42.700
for candidates designing their own assessments. And

610
00:31:42.700 --> 00:31:44.200
this is all this was done.

611
00:31:45.100 --> 00:31:48.400
Within limits, so we're not saying that a candidate can say I only

612
00:31:48.400 --> 00:31:51.700
want questions in this one area and nothing else our clients

613
00:31:51.700 --> 00:31:54.400
say, all right. These are the parameters. These

614
00:31:54.400 --> 00:31:57.900
are the the margins that we want our test

615
00:31:57.900 --> 00:32:00.900
questions to be delivered within so, you know,

616
00:32:00.900 --> 00:32:03.400
there's these four content areas and

617
00:32:03.400 --> 00:32:07.300
within those four content areas between 25

618
00:32:06.300 --> 00:32:09.600
and 40% of the the questions

619
00:32:09.600 --> 00:32:12.400
could be come from any content area. So we

620
00:32:12.400 --> 00:32:15.300
designed within the system away for candidates

621
00:32:15.300 --> 00:32:19.500
once a year to go in and play with

622
00:32:19.500 --> 00:32:22.800
the percentages so that if there was an area that's more useful or

623
00:32:22.800 --> 00:32:25.600
relevant to their actual practice that area

624
00:32:25.600 --> 00:32:28.700
could be dialed up or if there's something that maybe is

625
00:32:28.700 --> 00:32:31.400
not as useful in their practice or something.

626
00:32:31.400 --> 00:32:34.300
That's just not in their interest we could dial that down to

627
00:32:34.300 --> 00:32:37.600
the minimum percentage and that goes into the

628
00:32:37.600 --> 00:32:41.600
test the algorithm for test questions selection again,

629
00:32:40.600 --> 00:32:44.200
so that it's a very personalized individualized.

630
00:32:45.100 --> 00:32:45.900
coach to the test

631
00:32:50.400 --> 00:32:53.300
All right. So, you know, what did we I know we we threw a

632
00:32:53.300 --> 00:32:56.400
lot of you know information out there into The Ether,

633
00:32:56.400 --> 00:32:59.400
you know one thing is clear, you know, a modern

634
00:32:59.400 --> 00:33:02.700
approach to recertification, you know needs to

635
00:33:02.700 --> 00:33:05.500
be flexible and adaptable every program

636
00:33:05.500 --> 00:33:09.000
every board every it certification, you

637
00:33:08.300 --> 00:33:11.500
know, they don't need or necessarily even want the

638
00:33:11.500 --> 00:33:14.400
same thing. So, you know, our product provides that

639
00:33:14.400 --> 00:33:17.200
flexibility and I think it's important, you know for

640
00:33:17.200 --> 00:33:20.600
any product to you know, be flexible and adaptable to you

641
00:33:20.600 --> 00:33:23.700
know, an ever-changing, you know industry or you

642
00:33:23.700 --> 00:33:26.800
know world when adopting, you

643
00:33:26.800 --> 00:33:29.200
know, a recertification solution or even in

644
00:33:29.200 --> 00:33:32.000
your you know, beginning concepts of it. You know,

645
00:33:32.100 --> 00:33:35.400
I I think something to take away is, you know,

646
00:33:35.400 --> 00:33:38.300
don't get overwhelmed by all of you know,

647
00:33:38.300 --> 00:33:41.200
the options. There's a lot of things you can do. Do you

648
00:33:41.200 --> 00:33:44.600
need to do them all probably not so really take

649
00:33:44.600 --> 00:33:47.700
some time and think about what you're really trying

650
00:33:47.700 --> 00:33:50.200
to achieve and select features based off.

651
00:33:50.400 --> 00:33:53.400
Maybe even start small, you know start with something simple

652
00:33:53.400 --> 00:33:54.500
and you can grow it from there.

653
00:33:55.700 --> 00:33:58.000
um, you know, each of our implementations of the

654
00:33:58.700 --> 00:34:01.300
continuous certification product has been its own

655
00:34:01.300 --> 00:34:05.100
unique journey and you know all a little bit different from

656
00:34:04.100 --> 00:34:07.100
one another but you know now that

657
00:34:07.100 --> 00:34:10.400
we're getting the product out there it's been in use for over four

658
00:34:10.400 --> 00:34:13.200
years, you know, we finally get a chance

659
00:34:13.200 --> 00:34:16.100
to dive into the data to see you know,

660
00:34:16.100 --> 00:34:19.200
how is this changing industry is how is this

661
00:34:19.200 --> 00:34:22.100
being perceived and you know the data kind of

662
00:34:22.100 --> 00:34:25.800
speaks for itself, you know candidates like and appreciate the

663
00:34:25.800 --> 00:34:29.100
longitudinal approach to recertification and

664
00:34:28.100 --> 00:34:32.100
they're actually excited to come back and

665
00:34:31.100 --> 00:34:34.400
when there are questions are available to answer them imagine

666
00:34:34.400 --> 00:34:37.600
that someone wanting you know to to get

667
00:34:37.600 --> 00:34:40.100
their next quarters questions not just to get it

668
00:34:40.100 --> 00:34:43.600
over with but you know to figure out like what concepts are

669
00:34:43.600 --> 00:34:46.500
they going to be, you know, improving on or

670
00:34:46.500 --> 00:34:48.500
you know being introduced to

671
00:34:49.800 --> 00:34:52.900
You know, for example the award net the

672
00:34:52.900 --> 00:34:55.800
now award-winning certainlink product has a 98%

673
00:34:55.800 --> 00:34:58.100
user satisfaction rating, which I think

674
00:34:58.100 --> 00:35:01.800
is pretty outstanding for any type of assessment the data

675
00:35:01.800 --> 00:35:04.600
also shows that with periodic item delivery

676
00:35:04.600 --> 00:35:07.400
with allowing for you know, learning and

677
00:35:07.400 --> 00:35:10.600
assessment to run in parallel candidates are retaining knowledge,

678
00:35:10.600 --> 00:35:13.900
you know, for example, one of our programs saw that

679
00:35:13.900 --> 00:35:16.400
60% of candidates did better

680
00:35:16.400 --> 00:35:19.600
on their re attempt after taking advantage of

681
00:35:19.600 --> 00:35:22.700
learning content in the platform answering a

682
00:35:22.700 --> 00:35:25.500
question incorrectly. And then using what

683
00:35:25.500 --> 00:35:28.800
was provided in the platform the resources the rationales

684
00:35:28.800 --> 00:35:31.600
the study guides Etc and when they were delivered, you

685
00:35:31.600 --> 00:35:35.100
know that retry or clone later on 60% did

686
00:35:34.100 --> 00:35:38.200
better. That's the person that gained

687
00:35:37.200 --> 00:35:40.900
the skill that they wouldn't have gained in the

688
00:35:40.900 --> 00:35:41.800
traditional approach.

689
00:35:42.900 --> 00:35:45.100
So when our professionals, you know,

690
00:35:45.100 --> 00:35:48.600
whether they be doctors or software developers are able

691
00:35:48.600 --> 00:35:51.300
to continuously grow their knowledge through

692
00:35:51.300 --> 00:35:54.500
assessment and remain up to date with constantly evolving

693
00:35:54.500 --> 00:35:57.500
Concepts and Technology. That's you know,

694
00:35:57.500 --> 00:36:00.100
I can only speak for myself. That's really a world that you know,

695
00:36:00.100 --> 00:36:00.900
I want to be in

696
00:36:04.200 --> 00:36:04.600
and with that

697
00:36:05.500 --> 00:36:08.200
our most important slide we want to thank you

698
00:36:08.200 --> 00:36:11.700
all for attending and these are our dogs Elizabeth's dognia

699
00:36:11.700 --> 00:36:12.900
and my dog Payson.

700
00:36:15.800 --> 00:36:18.000
Thank you so much Ryan and Elizabeth. You have

701
00:36:18.200 --> 00:36:21.200
some very cute dogs. I have to say they are

702
00:36:21.200 --> 00:36:25.400
very very photogenic. Yes. Photogenic

703
00:36:24.400 --> 00:36:27.700
is the word. All right.

704
00:36:27.700 --> 00:36:30.400
Well, thank you. Thank you so much. That was just a wealth

705
00:36:30.400 --> 00:36:33.500
of information. We are going to switch gears and

706
00:36:33.500 --> 00:36:36.700
get into the Q&A portion. We have several questions

707
00:36:36.700 --> 00:36:39.400
in the chat. So I'm just

708
00:36:39.400 --> 00:36:43.500
gonna sort of Usher them over to you all sorting by

709
00:36:43.500 --> 00:36:47.500
the most upvotes at any one time. So first of

710
00:36:47.500 --> 00:36:50.800
all, we're going to start with question about

711
00:36:50.800 --> 00:36:53.500
does anyone outside of the medical space using

712
00:36:53.500 --> 00:36:55.100
this type of product currently.

713
00:36:56.400 --> 00:36:58.800
Yeah, I'll take it's a great question. You know we did.

714
00:36:59.600 --> 00:37:02.200
Speak a lot about the medical space and you

715
00:37:02.200 --> 00:37:05.200
know the medical space, you know did kind of

716
00:37:05.200 --> 00:37:08.200
revolutionize this type of assessment. They you know

717
00:37:08.200 --> 00:37:11.500
did push this out there to a lot of their candidates but

718
00:37:11.500 --> 00:37:14.200
as of yet we we are actually

719
00:37:14.200 --> 00:37:17.500
having conversations with some of our it Partners

720
00:37:17.500 --> 00:37:20.400
in you know, releasing this

721
00:37:20.400 --> 00:37:23.800
type of platform out to them. We do

722
00:37:23.800 --> 00:37:26.800
not have any implementations at the moment, but there are conversations going

723
00:37:26.800 --> 00:37:29.000
and I'm really excited for this type of

724
00:37:29.300 --> 00:37:31.300
product to break into the it space.

725
00:37:32.400 --> 00:37:35.200
And I'll just piggyback on that to say that I think.

726
00:37:35.700 --> 00:37:38.300
the the medical space was one of the first ones to

727
00:37:38.300 --> 00:37:42.700
to kind of adopt this because they had the need for recertification and

728
00:37:41.700 --> 00:37:44.400
I know not all Industries have

729
00:37:44.400 --> 00:37:47.400
that as we saw from the original polls, you

730
00:37:47.400 --> 00:37:50.200
know, some people can get a certification and then there's no requirement for

731
00:37:50.200 --> 00:37:53.300
the recertification and I think as

732
00:37:54.500 --> 00:37:57.200
As it's becoming more widely adopted I

733
00:37:57.200 --> 00:38:00.700
think other Industries are seeing the benefit in recertification

734
00:38:00.700 --> 00:38:03.900
both financially and you

735
00:38:03.900 --> 00:38:07.000
know for knowledge retention because as

736
00:38:06.600 --> 00:38:09.500
we spoke about things do change so

737
00:38:09.500 --> 00:38:12.000
as more Industries kind of

738
00:38:12.200 --> 00:38:15.700
move towards the need for recertification. I see more adopting

739
00:38:15.700 --> 00:38:16.900
this this kind of platform.

740
00:38:18.500 --> 00:38:20.300
Yeah, that makes a lot of sense to me.

741
00:38:21.700 --> 00:38:24.100
All right. Next question. Are you seeing a

742
00:38:24.100 --> 00:38:27.700
shift from being forced to to learn to

743
00:38:27.700 --> 00:38:29.600
self-driven learning?

744
00:38:31.300 --> 00:38:34.700
That go ahead right? That's a

745
00:38:34.700 --> 00:38:37.400
fun question. So we actually you know,

746
00:38:37.400 --> 00:38:41.500
some boards and clients are taking advantage of this

747
00:38:41.500 --> 00:38:44.600
where Elizabeth brought it up a little bit about, you

748
00:38:44.600 --> 00:38:47.400
know, providing candidates the ability to

749
00:38:47.400 --> 00:38:50.500
you know, partially or completely designed their

750
00:38:50.500 --> 00:38:53.500
own assessment, you know, where the boards can

751
00:38:53.500 --> 00:38:56.700
say, you know, 40% of the assessment is, you

752
00:38:56.700 --> 00:38:59.200
know, your core questions. These are required you have to take

753
00:38:59.200 --> 00:39:03.100
these but then incorporating elective content

754
00:39:02.100 --> 00:39:05.500
areas, you know into the design where the candidate

755
00:39:05.500 --> 00:39:08.500
whether it's based off of you know, there's some specialty

756
00:39:08.500 --> 00:39:11.800
specialty or just you know, a keen interest,

757
00:39:11.800 --> 00:39:14.500
you know can add questions, you know for a

758
00:39:14.500 --> 00:39:17.900
Content area, you know to their assessment so actually empowering

759
00:39:17.900 --> 00:39:20.300
them to contribute to you

760
00:39:20.300 --> 00:39:24.100
know, what questions are going to get even without that, you

761
00:39:23.100 --> 00:39:26.500
know, we're finding that the metrics

762
00:39:26.500 --> 00:39:29.100
of you know, using the rationale as you

763
00:39:29.100 --> 00:39:31.000
know, we're tracking every click on any

764
00:39:31.200 --> 00:39:35.500
In the platform when we're providing those URLs

765
00:39:34.500 --> 00:39:38.300
for candidates to go out and you

766
00:39:38.300 --> 00:39:42.800
know learn if they choose to do so, you know, they're really utilizing

767
00:39:41.800 --> 00:39:44.600
those features and we're starting

768
00:39:44.600 --> 00:39:47.500
to, you know, dive into those metrics now, so it's a

769
00:39:47.500 --> 00:39:47.700
great question.

770
00:39:48.700 --> 00:39:51.300
and not to bring up the pandemic yet again, but

771
00:39:51.300 --> 00:39:54.400
I think it actually provided a huge shift

772
00:39:54.400 --> 00:39:57.600
in kind of self-driven learning because we're seeing

773
00:39:58.600 --> 00:40:01.300
a lot more online learning, you know,

774
00:40:01.300 --> 00:40:05.000
everyone had to move online and that inherently brings

775
00:40:04.400 --> 00:40:07.300
in more the need for

776
00:40:07.300 --> 00:40:10.300
more self-driven learning. So I I think it's

777
00:40:10.300 --> 00:40:11.600
only going to increase

778
00:40:14.000 --> 00:40:14.200
great.

779
00:40:15.800 --> 00:40:15.900
All right.

780
00:40:16.900 --> 00:40:19.400
So next up has there been

781
00:40:19.400 --> 00:40:23.100
any research to your knowledge on retention

782
00:40:22.100 --> 00:40:25.900
rates for summative assessment versus longitudinal

783
00:40:25.900 --> 00:40:28.300
assessment. I assume that means

784
00:40:28.300 --> 00:40:31.700
you know, how how well you're retaining candidates

785
00:40:31.700 --> 00:40:34.400
when they take a summative assessment

786
00:40:34.400 --> 00:40:35.900
versus the longitudinal

787
00:40:37.300 --> 00:40:40.300
Yeah, so we at some of our partners have shared,

788
00:40:40.300 --> 00:40:43.500
you know this information with us where you know

789
00:40:43.500 --> 00:40:46.500
one example is a program that you know is still

790
00:40:46.500 --> 00:40:49.600
you know, there's still piloting the longitudinal approach,

791
00:40:49.600 --> 00:40:52.500
you know, so they're offering both the you know, summative traditional

792
00:40:52.500 --> 00:40:55.600
research exam or you can

793
00:40:55.600 --> 00:40:59.200
do you know, the longitudinal approach. So they're finding

794
00:40:59.200 --> 00:41:02.700
that the candidates that are going into

795
00:41:02.700 --> 00:41:05.600
you know, the longitudinal approach. They really

796
00:41:05.600 --> 00:41:08.300
want to stay there. They're big, you know, their scores are higher than the

797
00:41:08.300 --> 00:41:11.600
summative, you know, they really want to maintain there and they're speaking

798
00:41:11.600 --> 00:41:14.700
to their peers about it. So, you know, we're finding

799
00:41:14.700 --> 00:41:17.700
that programs that allow for you

800
00:41:17.700 --> 00:41:20.800
to register throughout the year for you

801
00:41:20.800 --> 00:41:23.700
know, the launch student approach that you know, you may

802
00:41:23.700 --> 00:41:26.400
have start the year with 3,000 candidates, but you're ending the

803
00:41:26.400 --> 00:41:29.400
year with 5500 because people are

804
00:41:29.400 --> 00:41:32.000
starting to you know, get the word out and you know

805
00:41:32.400 --> 00:41:34.500
take advantage of this new approach.

806
00:41:35.700 --> 00:41:38.000
Interesting so that kind of speaks to the retention as well

807
00:41:38.200 --> 00:41:41.100
as just to the popularity. Yeah, and I think we

808
00:41:41.100 --> 00:41:44.100
are we have a lot of anecdotal evidence right now. I think

809
00:41:44.100 --> 00:41:47.400
the psychometric analysis is just beginning

810
00:41:47.400 --> 00:41:50.300
to happen because you know, we're we're kind of coming up on

811
00:41:50.300 --> 00:41:53.400
that 10 year mark of this starting to

812
00:41:53.400 --> 00:41:56.400
be a thing. So I think we're finally getting some some good

813
00:41:56.400 --> 00:41:59.500
data. So I I would imagine we'll have some data driven

814
00:41:59.500 --> 00:42:02.200
metrics as well.

815
00:42:02.200 --> 00:42:05.500
Yeah. I'm really excited for the next year. So, you

816
00:42:05.500 --> 00:42:08.000
know, I think you hit the nail on the head Elizabeth with you know,

817
00:42:08.800 --> 00:42:11.100
at least you and I we have been very focused, you know

818
00:42:11.100 --> 00:42:14.200
on you know, the delivery of the product and making sure that

819
00:42:14.200 --> 00:42:17.100
it's you know, achieving the needs, you know

820
00:42:17.100 --> 00:42:20.400
of our partners of the industry. I can't wait to

821
00:42:20.400 --> 00:42:23.500
start diving into you know the data to

822
00:42:23.500 --> 00:42:26.400
start to, you know, enhance the platform on you know

823
00:42:26.400 --> 00:42:29.900
that I've driven feedback instead of just like you

824
00:42:29.900 --> 00:42:30.500
said anecdotal.

825
00:42:32.300 --> 00:42:35.700
Well, we look forward to that data as well part two

826
00:42:35.700 --> 00:42:38.300
of the webinar. Yes tune in

827
00:42:38.300 --> 00:42:38.400
later.

828
00:42:40.100 --> 00:42:43.300
All right. Here's a here's a juicy one. What advice do you

829
00:42:43.300 --> 00:42:46.900
all have based on your interactions with other organizations

830
00:42:46.900 --> 00:42:49.500
test sponsors for an organization that is

831
00:42:49.500 --> 00:42:53.600
considering This Modern approach to recertification. You

832
00:42:52.600 --> 00:42:55.300
know, the question ask for

833
00:42:55.300 --> 00:42:58.200
that seems a little overwhelming to go from CE credit which is

834
00:42:58.200 --> 00:43:01.300
continuing education credit requirements to

835
00:43:01.300 --> 00:43:03.300
a longitudinal assessment platform.

836
00:43:04.500 --> 00:43:07.100
What do you have to say does seem overwhelming?

837
00:43:08.400 --> 00:43:11.800
I would say from some of some of

838
00:43:11.800 --> 00:43:15.100
our clients who maybe not necessarily went strictly

839
00:43:14.100 --> 00:43:18.400
from CE credits to now incorporating an

840
00:43:17.400 --> 00:43:20.300
assessment into that. But from the ones

841
00:43:20.300 --> 00:43:23.800
that we had that already had an assessment. It was

842
00:43:23.800 --> 00:43:26.600
definitely a lot easier. I think you know

843
00:43:26.600 --> 00:43:29.600
for someone coming from a program where

844
00:43:29.600 --> 00:43:32.500
there was no assessment component. It

845
00:43:32.500 --> 00:43:35.800
might be a bit more difficult just because the amount

846
00:43:35.800 --> 00:43:39.500
of content that you would have to generate for test questions and perhaps

847
00:43:39.500 --> 00:43:42.300
for learning resources and things like that is quite

848
00:43:42.300 --> 00:43:45.600
significant, but that being said a

849
00:43:45.600 --> 00:43:48.800
lot of the programs start off with with very small

850
00:43:48.800 --> 00:43:51.400
numbers of questions per quarter. I mean even something like

851
00:43:51.400 --> 00:43:54.200
five questions per quarter, so that's saying, you know,

852
00:43:54.200 --> 00:43:57.400
you have to come up with content 20 questions for for a

853
00:43:57.400 --> 00:44:00.900
year. It's a little less overwhelming when

854
00:44:00.900 --> 00:44:03.000
you can break it down into that smaller.

855
00:44:04.700 --> 00:44:06.000
smaller bit of information

856
00:44:09.500 --> 00:44:12.500
Yeah, that seems more manageable for you know, once once it

857
00:44:12.500 --> 00:44:15.200
gets up and running. It seems more manageable for it. Yeah, and there's

858
00:44:15.200 --> 00:44:18.200
there's definitely front load of work, you know generating all

859
00:44:18.200 --> 00:44:21.200
that content is a lot of work and but once that is

860
00:44:21.200 --> 00:44:24.100
in there and then you're just doing kind of

861
00:44:24.100 --> 00:44:27.700
a maintenance of you know, new questions new topics

862
00:44:27.700 --> 00:44:30.300
new things like that. It actually becomes a

863
00:44:30.300 --> 00:44:30.300
lot.

864
00:44:31.300 --> 00:44:34.500
Like almost I don't want to say like a no maintenance, but

865
00:44:34.500 --> 00:44:37.400
it's a lot easier to run. Mm-hmm.

866
00:44:38.500 --> 00:44:39.300
That's encouraging.

867
00:44:41.100 --> 00:44:44.400
All right question from Gina. Do you know of any boards that

868
00:44:44.400 --> 00:44:47.900
offer CME for their longitudinal assessment?

869
00:44:51.200 --> 00:44:54.200
So I guess give credit. Yeah, that's a good question. I

870
00:44:54.200 --> 00:44:57.200
don't know if the top of my head. Typically there's a

871
00:44:57.200 --> 00:45:00.800
there's a CE requirement and then launch to know requirement, but

872
00:45:00.800 --> 00:45:03.100
I would not be surprised. You know

873
00:45:03.100 --> 00:45:07.700
with what I know in the space that you know, the pure participation

874
00:45:06.700 --> 00:45:09.700
in, you know, a Lunchable assessment

875
00:45:09.700 --> 00:45:13.300
product, you know would you know achieve those

876
00:45:13.300 --> 00:45:15.400
credits or contribute to those credits?

877
00:45:19.500 --> 00:45:19.700
awesome

878
00:45:20.700 --> 00:45:25.100
All right. So here's a question about retakes. It

879
00:45:24.100 --> 00:45:27.400
sounds like are there any attempt limits on the

880
00:45:27.400 --> 00:45:30.200
individual items as far as retakes go

881
00:45:30.200 --> 00:45:33.400
or can candidates retake failed

882
00:45:33.400 --> 00:45:36.700
items after some time like what are the tell us

883
00:45:36.700 --> 00:45:38.700
a little bit more about any kind of limitations there?

884
00:45:39.900 --> 00:45:42.200
So one of the the key things that we always

885
00:45:42.200 --> 00:45:45.400
say about this platform is it's configurable. So we have

886
00:45:45.400 --> 00:45:48.300
a lot of configuration that can

887
00:45:48.300 --> 00:45:51.000
be done and you know, one of those for the

888
00:45:51.500 --> 00:45:54.200
re attempts we can put limits on how many times an item is

889
00:45:54.200 --> 00:45:57.800
allowed to be reattempted. So, you know, if it's something

890
00:45:57.800 --> 00:46:00.600
where there's four different options and

891
00:46:00.600 --> 00:46:03.500
you say you get four times before this

892
00:46:03.500 --> 00:46:06.300
is no longer reattempted, you know,

893
00:46:06.300 --> 00:46:09.500
that might not be the best for like a psychometricians in

894
00:46:09.500 --> 00:46:12.300
their scores. So yeah, we do have limits we get you could

895
00:46:12.300 --> 00:46:15.300
say can only be attempted once it can be very attempted twice.

896
00:46:15.300 --> 00:46:18.500
We also have configuration for when that

897
00:46:18.500 --> 00:46:21.500
can be redelivered so we can say, you know, don't show

898
00:46:21.500 --> 00:46:24.500
it for three segments so that

899
00:46:24.500 --> 00:46:27.300
you know, there's time before you're shown that

900
00:46:27.300 --> 00:46:28.200
same question again.

901
00:46:30.200 --> 00:46:33.600
Great sounds like a new t-shirt that says it's configurable

902
00:46:33.600 --> 00:46:34.400
we do.

903
00:46:34.600 --> 00:46:35.100
We do.

904
00:46:37.200 --> 00:46:40.400
All right. Okay. So comment. This seems powerful

905
00:46:40.400 --> 00:46:43.400
is the need is the big question is the need for

906
00:46:43.400 --> 00:46:46.500
the original credential still there or do

907
00:46:46.500 --> 00:46:50.300
you see that going away? So the initial certification when you

908
00:46:50.300 --> 00:46:54.100
see in your crystal ball good question is again it

909
00:46:53.100 --> 00:46:56.500
really it really depends on you

910
00:46:56.500 --> 00:46:59.500
know, what, you know your program it wants

911
00:46:59.500 --> 00:47:03.200
to do, you know, I feel like it would be a hard sell for you

912
00:47:02.200 --> 00:47:05.300
know, many a medical programs to

913
00:47:05.300 --> 00:47:08.700
eliminate that you know high stakes initial certification.

914
00:47:08.700 --> 00:47:12.300
You know, there's a lot that goes into that. Whereas I

915
00:47:11.300 --> 00:47:14.400
feel like there would be more flexibility, you

916
00:47:14.400 --> 00:47:17.400
know on you know in it space where you know,

917
00:47:17.400 --> 00:47:20.300
this platform can be the achievements and

918
00:47:20.300 --> 00:47:23.700
the maintenance of you know your certification so

919
00:47:23.700 --> 00:47:27.400
I don't see initial certification going

920
00:47:26.400 --> 00:47:29.600
away for everybody, but this would

921
00:47:29.600 --> 00:47:32.600
be a very valid and useful way

922
00:47:32.600 --> 00:47:33.100
to you know,

923
00:47:34.800 --> 00:47:37.500
obtain and retain a certification

924
00:47:39.300 --> 00:47:41.000
great Elizabeth anything to add there?

925
00:47:42.400 --> 00:47:45.400
No, I think Ryan hit it on the head. I I don't think

926
00:47:45.400 --> 00:47:47.700
that anything where Maybe.

927
00:47:48.800 --> 00:47:53.300
Physical safety is such

928
00:47:51.300 --> 00:47:54.400
as the medical boards or

929
00:47:54.400 --> 00:47:58.300
maybe some of the on-training exams for for certain

930
00:47:57.300 --> 00:48:01.000
industries would ever

931
00:48:00.600 --> 00:48:03.400
switch to this but I could see

932
00:48:03.400 --> 00:48:07.600
for sure it or some other other Industries

933
00:48:07.600 --> 00:48:10.500
where you need the knowledge,

934
00:48:10.500 --> 00:48:13.700
but don't necessarily need to demonstrate it

935
00:48:13.700 --> 00:48:16.300
in the first place. You just need to have that knowledge

936
00:48:16.300 --> 00:48:19.000
all along in order to you know,

937
00:48:19.200 --> 00:48:22.200
get the value of that certification. Yeah, I could see that happening. I

938
00:48:22.200 --> 00:48:25.200
don't think it's anytime soon, but we could see that

939
00:48:25.200 --> 00:48:27.300
right anything's possible I guess.

940
00:48:28.600 --> 00:48:29.300
Never Say Never

941
00:48:30.600 --> 00:48:33.800
All right. Here's a question. Are these assessments Norm or

942
00:48:33.800 --> 00:48:35.300
Criterion reference?

943
00:48:40.900 --> 00:48:41.900
Ryan do you want to take that?

944
00:48:43.600 --> 00:48:44.300
regards to Norms Maybe

945
00:48:45.100 --> 00:48:48.800
Well, I I'm struggling with the term Criterion reference.

946
00:48:48.800 --> 00:48:51.300
So sorry if I answer this incorrectly but you know

947
00:48:51.300 --> 00:48:55.300
Elizabeth kind of mentioned before that, you

948
00:48:54.300 --> 00:48:58.100
know, the platform supports providing a

949
00:48:58.100 --> 00:49:01.900
dynamic comparison to others in my

950
00:49:01.900 --> 00:49:04.500
group, you know, as other people are answering the same questions as me.

951
00:49:04.500 --> 00:49:07.600
I'm being compared against them, you know it getting that

952
00:49:07.600 --> 00:49:10.000
you know competition out and you know,

953
00:49:10.200 --> 00:49:13.600
the competitiveness of people but then we also support

954
00:49:13.600 --> 00:49:16.700
more of the static or

955
00:49:16.700 --> 00:49:19.800
cut score performance Norms where you could

956
00:49:19.800 --> 00:49:22.300
display both of them you could display one of them but

957
00:49:22.300 --> 00:49:25.300
you know for these performance metrics if

958
00:49:25.300 --> 00:49:28.800
you know that you must achieve 70% you

959
00:49:28.800 --> 00:49:31.600
know, we can display that throughout the platform at an

960
00:49:31.600 --> 00:49:34.100
assessment level at a Content area level at an item

961
00:49:34.100 --> 00:49:37.700
level, you know saying, you know, the performance requirement

962
00:49:37.700 --> 00:49:41.000
is 70% and here's where you are in,

963
00:49:40.500 --> 00:49:43.300
you know your performance. Like are you

964
00:49:43.300 --> 00:49:44.700
ahead are you below if you're below?

965
00:49:45.100 --> 00:49:48.100
Where's your risk? You know, what am I at risk of

966
00:49:48.100 --> 00:49:51.600
never achieving 70% you know, those are the things the platform,

967
00:49:51.600 --> 00:49:54.300
you know is able to share in the

968
00:49:54.300 --> 00:49:56.200
dashboard. Mm-hmm, and I think

969
00:49:57.300 --> 00:49:58.900
I think maybe

970
00:50:00.200 --> 00:50:03.800
the reason we were a little struggling with this is that

971
00:50:03.800 --> 00:50:06.600
we actually don't make those decisions for programs.

972
00:50:06.600 --> 00:50:09.100
So we're there to capture the data. We're there to

973
00:50:09.100 --> 00:50:12.500
display metrics, but it's

974
00:50:12.500 --> 00:50:15.500
ultimately the clients that decide what the

975
00:50:15.500 --> 00:50:18.500
performance requirement is

976
00:50:18.500 --> 00:50:21.200
for the recertification. I think most of most of the

977
00:50:21.200 --> 00:50:24.300
clients that we currently have are doing it based off

978
00:50:24.300 --> 00:50:27.500
of a cut score but that's not to say that we you

979
00:50:27.500 --> 00:50:30.300
couldn't use a different metric to to decide if

980
00:50:30.300 --> 00:50:32.000
they've qualified for recertification.

981
00:50:33.800 --> 00:50:36.500
And what I'm hearing too is that if there is some kind of a norm

982
00:50:36.500 --> 00:50:39.300
based on your industry, right? Some of I don't know the sort

983
00:50:39.300 --> 00:50:42.300
of the title of norms, but if there was something you could just load that

984
00:50:42.300 --> 00:50:45.500
into the system and it can be it can reference so

985
00:50:47.300 --> 00:50:48.000
Great, okay.

986
00:50:49.400 --> 00:50:51.800
So sounds great a couple more left.

987
00:50:53.300 --> 00:50:56.200
Is the time limit set at the item level or

988
00:50:56.200 --> 00:50:59.200
at the segment sort of quarter assessment level.

989
00:50:59.200 --> 00:51:02.600
Where is the timing coming into play here? So quote

990
00:51:02.600 --> 00:51:04.100
Elizabeth. It's configurable.

991
00:51:05.100 --> 00:51:08.300
So the system supports again. It depends

992
00:51:08.300 --> 00:51:11.300
on you know, what your your goal is for the platform,

993
00:51:11.300 --> 00:51:14.500
but we do support item level timing no

994
00:51:14.500 --> 00:51:18.200
item timing and it's gonna have unlimited timing timing can

995
00:51:17.200 --> 00:51:20.700
adapt to ADA accommodations

996
00:51:20.700 --> 00:51:23.100
or time Factor, you know, because we could, you know

997
00:51:23.100 --> 00:51:26.200
double time a limited time time and a half Etc.

998
00:51:26.200 --> 00:51:29.300
But then also applying timing at you

999
00:51:29.300 --> 00:51:32.200
know, the assessment level, you know, you have a quarter

1000
00:51:32.200 --> 00:51:35.700
or you have a month to answer these questions or

1001
00:51:35.700 --> 00:51:38.700
perhaps, you know, once you start answering you

1002
00:51:38.700 --> 00:51:41.500
might have a quarter to answer the questions,

1003
00:51:41.500 --> 00:51:44.500
but once you actually start answering you have you know, 90 days

1004
00:51:44.500 --> 00:51:47.100
after that point to complete so, you

1005
00:51:47.100 --> 00:51:50.600
know, there's lots of different ways that you could apply timing and scheduling

1006
00:51:50.600 --> 00:51:53.100
to your assessment. It really just goes to you know, what

1007
00:51:53.100 --> 00:51:53.500
are your goals?

1008
00:51:55.400 --> 00:51:55.800
Great.

1009
00:51:56.900 --> 00:51:59.800
All right, two more questions. First one

1010
00:51:59.800 --> 00:52:02.500
doesn't take a lot of people to generate content for

1011
00:52:02.500 --> 00:52:04.100
this such as larger teams.

1012
00:52:04.900 --> 00:52:07.100
Think we spoke about this a little bit earlier. But is there

1013
00:52:07.100 --> 00:52:08.600
anything you want to elaborate on Ryan Elizabeth?

1014
00:52:09.500 --> 00:52:13.000
I just said, you know, the industry knows writing good

1015
00:52:12.600 --> 00:52:15.400
items is time consuming it's

1016
00:52:15.400 --> 00:52:19.800
expensive as far as money resources. So, you

1017
00:52:18.800 --> 00:52:21.300
know, I was kind of getting at that when

1018
00:52:21.300 --> 00:52:24.400
I had mentioned, you know, if you're adopting a platform like this,

1019
00:52:24.400 --> 00:52:28.000
you don't have to start with a 2000 item

1020
00:52:27.600 --> 00:52:30.200
item pool. We have clients that

1021
00:52:30.200 --> 00:52:33.300
do have 6,500 items in their Dynamic.

1022
00:52:33.300 --> 00:52:36.300
I don't pull but then we also have clients that

1023
00:52:36.300 --> 00:52:39.500
are delivering at 10 questions 15 questions

1024
00:52:39.500 --> 00:52:43.000
a quarter and all they can provide us with are

1025
00:52:42.100 --> 00:52:45.700
those 10 or 15 questions quarterly. That's fine.

1026
00:52:45.700 --> 00:52:47.300
You know, it's

1027
00:52:48.400 --> 00:52:51.500
I think the most important part is you're spending the time to

1028
00:52:51.500 --> 00:52:54.900
write good material and not

1029
00:52:54.900 --> 00:52:57.300
getting overwhelmed with oh, you know, I need

1030
00:52:57.300 --> 00:53:00.200
I need to have three years worth of questions. You don't need

1031
00:53:00.200 --> 00:53:03.400
to have that. Mm-hmm. And I think the other

1032
00:53:03.400 --> 00:53:06.300
advantage of platform like this is that it extends the

1033
00:53:06.300 --> 00:53:09.300
life cycle of an item and it extends

1034
00:53:09.300 --> 00:53:12.500
the life cycle of resource content that

1035
00:53:12.500 --> 00:53:15.100
you've written because it can be used as a

1036
00:53:15.100 --> 00:53:18.800
resource for one set of questions, but it

1037
00:53:18.800 --> 00:53:21.600
doesn't have to be questioned specific, you know a topic

1038
00:53:21.600 --> 00:53:24.800
area. You can have these resources reused and

1039
00:53:24.800 --> 00:53:27.500
the items, you know, when it's when it's reached

1040
00:53:27.500 --> 00:53:30.900
its retirement age and it's no longer appropriate for for

1041
00:53:30.900 --> 00:53:34.700
an exam where you know, you really trying to test

1042
00:53:33.700 --> 00:53:36.400
knowledge it can

1043
00:53:36.400 --> 00:53:39.500
it can be recycled into you know, practice exams things

1044
00:53:39.500 --> 00:53:42.700
that can also be delivered through this this platform.

1045
00:53:45.100 --> 00:53:49.500
Awesome. All right last question of the day what item

1046
00:53:49.500 --> 00:53:51.000
types are supported in this platform.

1047
00:53:52.400 --> 00:53:55.400
We do see you know, traditional multiple choice

1048
00:53:55.400 --> 00:53:58.800
multiple choice multiple select the platform

1049
00:53:58.800 --> 00:54:01.300
will support any item type that you know,

1050
00:54:01.300 --> 00:54:04.400
any of our assessment products will support from drag and

1051
00:54:04.400 --> 00:54:07.800
drop to you know, find and replace your anything. We

1052
00:54:07.800 --> 00:54:11.700
we are finding a growth in article-based

1053
00:54:10.700 --> 00:54:14.400
questions. So they're still traditional multiple

1054
00:54:13.400 --> 00:54:16.800
choice or you know, multiple choice multiple select

1055
00:54:16.800 --> 00:54:20.000
items, but they're Associated to some sort

1056
00:54:19.100 --> 00:54:22.100
of URL or resource where you know, the

1057
00:54:22.100 --> 00:54:25.700
candidate needs to go out read this resource

1058
00:54:25.700 --> 00:54:28.100
watch this video and then they

1059
00:54:28.100 --> 00:54:32.300
have questions Associated to it. It's very similar to you know,

1060
00:54:32.300 --> 00:54:35.200
like reading reading passage item types. But

1061
00:54:35.200 --> 00:54:38.800
instead of the platform itself hosting the

1062
00:54:38.800 --> 00:54:41.000
reading passage, it's you know, a medical

1063
00:54:41.400 --> 00:54:44.800
journal or you know, it's a YouTube video on SQL development

1064
00:54:44.800 --> 00:54:47.000
or you know, something like that that you have

1065
00:54:47.200 --> 00:54:50.300
to go out process and then the system will deliver

1066
00:54:50.300 --> 00:54:51.800
questions to you.

1067
00:54:52.400 --> 00:54:53.400
based off that

1068
00:54:54.300 --> 00:54:57.700
That's great and speaking of you mentioned videos. I assume

1069
00:54:57.700 --> 00:55:00.200
that for sort of medical type exams. There's probably

1070
00:55:00.200 --> 00:55:03.800
a high number of images and and video

1071
00:55:03.800 --> 00:55:04.500
content that you see.

1072
00:55:05.300 --> 00:55:08.100
Yeah, really? Yeah, we have

1073
00:55:08.100 --> 00:55:11.300
we also have we have a media viewer that

1074
00:55:11.300 --> 00:55:14.300
we've built in specifically for kind of

1075
00:55:14.300 --> 00:55:17.200
the medical industry where it's very high resolution you can

1076
00:55:18.400 --> 00:55:21.200
Go down into into detail into CT scans and things

1077
00:55:21.200 --> 00:55:24.800
like that. But the the Media Player supports a variety

1078
00:55:24.800 --> 00:55:26.600
of of media types.

1079
00:55:28.600 --> 00:55:28.900
awesome

1080
00:55:30.500 --> 00:55:33.300
All right, everybody. Well, that was the last

1081
00:55:33.300 --> 00:55:36.300
question that we have today. Once again, thank you so much

1082
00:55:36.300 --> 00:55:39.800
for attending our webinar today. We hope you found it relevant and

1083
00:55:39.800 --> 00:55:42.200
valuable. Once again, keep an

1084
00:55:42.200 --> 00:55:45.100
eye out for the Post webinar survey in your browser. Once the session

1085
00:55:45.100 --> 00:55:48.300
ends. We really would love to hear your feedback and you

1086
00:55:48.300 --> 00:55:51.500
know, feel free to connect with us on social media to keep the conversation

1087
00:55:51.500 --> 00:55:54.300
going and don't hesitate to reach out to us

1088
00:55:54.300 --> 00:55:57.300
directly via email to share what you'd like to see in a

1089
00:55:57.300 --> 00:55:57.800
future webinar.

1090
00:55:58.500 --> 00:56:00.500
All right. Take care of everyone. Thanks for joining.

1091
00:56:02.100 --> 00:56:02.700
Thanks everybody. Thanks.