[0:17] Excused

[0:17] all

[0:19] excused kelty

[0:22] excused pascal

[0:24] here

[0:25] vice chair to tone

[0:27] I am here

[0:30] adam chair here

[0:32] OK

[0:35] first up we've got4

[0:38] departments here to talk about

[0:39] to continue our conversations

[0:43] about Ai and so we're eager

[0:46] to get to them in the meantime

[0:49] we have invited the officeice

[0:51] offormation Technology to come

[0:54] forward and do a quick

[0:59] presentation on the reorg

[1:00] that has happened in the

[1:00] department

[1:04] so they're gonna present for you

[1:06] know if you can keep it to 10

[1:08] minutes that would be great. We

[1:09] may have a couple of questions

[1:11] but I'm gonna limit us to to5

[1:14] minutes so we can get into

[1:15] what's next the meat of what

[1:17] we're going to talk about is

[1:19] gonna be about45 minutes for

[1:21] hick puff and then I would ask

[1:24] CDLE to hopefully stick to 20

[1:26] minutes and then

[1:29] CDpS if they're in the room

[1:31] we'll try to do 20 minutes as

[1:34] well. and 10 will be an

[1:37] executive session assuming that

[1:39] senator Ball joins us or rep

[1:41] Kelty joins us. they've both

[1:43] RSVpd yes. so with that we will

[1:46] go ahead and get started.

[1:46] director Tunberg

[1:54] good morning. thank you for

[1:57] having us. Can somebody make

[1:58] page a presenter please.

[2:02] Thank you

[2:06] thank you senator Marchman as

[2:08] you said we are here to briefly

[2:11] provide an update. we just

[2:14] crossed our 90 days since the

[2:16] second part of our very large

[2:17] reorganization and reset

[2:22] ' s very well thank you

[2:26] great.ext slide please

[2:30] we've provided these materials

[2:31] to you so I'm going to go

[2:34] through them as you said very

[2:35] briefly happy to follow up on

[2:37] any questions. The first thing I

[2:38] want to start with is the

[2:39] grounding principles of our

[2:42] reset and our reorg that

[2:44] really guide the way we are

[2:46] working as a whiteT moving

[2:47] forward. there are a couple I

[2:49] want to flag for you. The first

[2:51] is starting with our users we're

[2:54] in service to Coloradans the

[2:56] taxpayers who need us to deliver

[2:57] value through technology

[3:00] Additionally we are focused on

[3:02] outcomes over process. we

[3:04] want to be delivering measurable

[3:05] value as the outcome

[3:10] thank you.

[3:14] umpresentative Pascal requested

[3:15] that we provide a brief update

[3:19] on the work in flight at the

[3:21] point at which we did the sort

[3:22] of phase two of our

[3:26] reorganization we had this

[3:28] was in the end of May more than

[3:31] 200 in flight projects or

[3:36] products or programs of those

[3:38] 197 continue

[3:42] uninterrupted. we have

[3:45] successfully completed25 an

[3:48] additional 16 are in close out.

[3:49] we have achieved the objectives

[3:51] and are either transitioning

[3:54] them to a long term operations

[3:56] and maintenance, finishing them

[3:56] out in some other way

[3:59] Additionally 10 have been

[4:02] strategically retired, canceled

[4:05] or combined with others in

[4:07] addition to not seeing

[4:10] interruption in our workloads we

[4:13] are seeing greater velocity,

[4:15] lower costs and

[4:20] faster time to completion as a

[4:20] result of the reorganization

[4:26] we as part of our reorganization

[4:28] we divided our org into three

[4:31] primary delivery functions

[4:33] supported by security and

[4:34] infrastructure additionally

[4:36] supported by our enterprise

[4:38] policy governance andans group

[4:39] and supported by our

[4:40] administration arm

[4:45] to talk through some of the

[4:47] value and outcomes we are seeing

[4:49] in each of those. In our

[4:50] delivery to Coloradan's layer

[4:53] last week we launched the mobile

[4:55] driver's license within my

[4:58] Colorado. this enables the 2

[4:59] million Coloradans who have a

[5:03] myolorado account to use their

[5:06] account to transit through TSA

[5:08] as well as gain access to

[5:10] federal buildings in a privacy

[5:10] preserving secure way

[5:15] Additionally we launchedignin

[5:17] with myolorado with our summer

[5:21] EBT program. this yielded a70%

[5:24] reduction in the time it took

[5:26] families to access their summer

[5:29] EBT benefit. This seems somewhat

[5:30] funny cause we went from a

[5:31] little bit more than 2 minutes

[5:34] to 25 seconds but this is the

[5:36] way that digital services should

[5:39] be delivered frictionless, easy,

[5:40] secure privacy preserving.

[5:43] sure

[5:44] rapascal

[5:48] thank you madam chair. So can

[5:50] you tell what is the summer EBT

[5:51] program and how is that

[5:53] different from like regular

[5:55] ongoing activity with that

[5:57] program Directorymberg thank you

[5:59] for the question. The summer EBT

[6:01] programme is a program that

[6:03] enabled when kids can't get free

[6:05] and reduced breakfast or lunch

[6:07] at school. it provides a cash

[6:09] benefit that they can use to

[6:10] ensure that they have proper

[6:10] nutrition and feeding throughout

[6:11] the year.

[6:12] the summer

[6:19] repascal and I'm also curious

[6:21] about the mobile driver's

[6:24] license I know that we already

[6:25] could get our driver's license

[6:27] through myolorado very useful

[6:29] when I lost my driver's license

[6:31] but how is this different? Is is

[6:33] being able to get through TSA

[6:35] with this a new thing or what's

[6:37] differentrectorunberg great

[6:39] question. yes. so the visual

[6:41] version the digital driver's

[6:42] license that you can use within

[6:44] Colorado is not always accepted

[6:44] in other

[6:47] states or by federal partners

[6:50] like TSA it was not you couldn't

[6:52] use it to travel which is a

[6:53] primary thing people need their

[6:55] I for so the mobile driver's

[6:57] license is a new technology

[6:58] standard. It was available

[6:59] within the Apple wallllet

[7:01] Samsung wallet, Google wallet

[7:04] but now we have a nonig tech

[7:05] choice for Coloradan's really

[7:06] meeting Coloradans where they

[7:07] are

[7:09] OK and I do see we have

[7:11] additional questions. You only

[7:12] have4 minutes left so I'm gonna

[7:14] give you your4 minutes back you

[7:16] figure out what you need to do

[7:17] with those and then we will get

[7:20] to questions. Fantastic. So in

[7:22] addition we on the delivery to

[7:25] agency side are primary

[7:26] customers are agencies who we

[7:27] deliver to and then on to

[7:30] Coloradans we are seeing

[7:31] significant direct agency

[7:34] savings we have reduced overhead

[7:36] in OitT operations costs for

[7:36] example

[7:41] CdPS is we project a $1.2

[7:43] million savings in fiscal year

[7:45] 27 by having a dedicated

[7:47] staffing model and reducing the

[7:48] overhead. You can look through

[7:50] these later but we see this

[7:51] consistently across all of our

[7:52] agencies

[7:56] on a delivery to state employees

[7:58] significantly reduced the manual

[7:59] effort of key functions like

[8:02] human resources or constituent

[8:03] services at the governor's

[8:06] office on security initiatives

[8:08] we over the last few months have

[8:11] had an uptick in external

[8:13] international actors attempting

[8:15] to attack our infrastructure.

[8:17] The Iranian water system in

[8:20] attack jill Frazier ourisso

[8:22] together with our security team

[8:22] provided crucial

[8:25] intelligence that supported that

[8:27] response. Additionally we're

[8:29] seeing North Korean threat

[8:31] actors attempting to gain

[8:33] employment and access our

[8:34] systems

[8:37] drafted and released new HR

[8:38] processes that are keeping North

[8:39] Korean actors out of our

[8:42] essential systems more there on

[8:43] platforms and infrastructure

[8:46] we've already achieved $6

[8:49] million in platform savings via

[8:50] direct negotiations with our

[8:51] vendors we are deeply grateful

[8:53] for them.dditionally we've

[8:55] enrolled in GovRamp which

[8:58] reduces duplicating assessments

[9:00] for the services we now leverage

[9:01] clear third party risk

[9:03] assessments and continuous

[9:04] monitoring

[9:07] reducing the cost increasing the

[9:08] security of our platforms and

[9:08] infrastructure.

[9:12] on hiring we have received and

[9:15] processed nearly 3000

[9:17] applications for the various

[9:19] roles we have posted we've

[9:21] hired88 new individuals

[9:23] including 1f returning OIT

[9:25] employees

[9:30] arepod hiring this is our

[9:31] dedicated agency support

[9:37] or72% complete. we've hired55

[9:41] of76 targeted roles and e of our

[9:42] agencies have a

[9:47] full staffing so really making a

[9:48] haste on that ensuring we're

[9:49] providing value to our agencies.

[9:53] We just completed our 6th month

[9:56] of monthly employee checkin

[9:56] survey

[10:00] one of the questions we ask our

[10:02] employees is how are they

[10:05] feeling about the change.

[10:07] this is about the reorg and we

[10:08] ask are you on board?

[10:10] are you cautious to have

[10:12] questions? are you stuck?

[10:15] starting in March we saw that

[10:17] the cautious group really led

[10:22] the way in July our numbers

[10:24] declined this is right after we

[10:25] announced the layoff as well as

[10:27] fiscal end of year a lot of work

[10:29] and a lot of friction but in

[10:31] August a couple of weeks ago we

[10:33] really see these numbers switch.

[10:36] we see for the first time a

[10:37] really significant increase in

[10:38] those people who are saying

[10:40] they are on board this change.

[10:42] they are seeing value and

[10:43] benefit and those who are

[10:44] cautious to climb

[10:47] stuck remains the same and we

[10:49] will continue to help those

[10:51] folks move along. Last week we

[10:53] launched organizational roadmaps

[10:54] which I have shared with you.

[10:55] it's the first time in my

[10:56] knowledge OIT has had public

[10:59] roadmaps and a public technology

[11:01] strategy. I welcome you to look

[11:02] at those at your leisure.

[11:04] I believe that is it

[11:10] and40 seconds left brilliant way

[11:12] to go. I'm excited to see

[11:13] where those trends go. are you

[11:17] going to continue to do those

[11:19] that employee change readiness

[11:21] survey. Yes, thank you for the

[11:23] question and we'll in addition

[11:24] to continuing it each month

[11:25] we'll continue to share the data

[11:28] with you. Obviously it is a one

[11:30] point in time data collection

[11:33] not a trend yet but we hope we

[11:34] continue. Same great. I just

[11:35] want to acknow

[11:38] led ge that we have the rest of

[11:38] our committee here

[11:43] we have well most we have

[11:46] representative Kelty and senator

[11:49] Ball here. we have

[11:51] representative to tone with her

[11:52] hand uppresentative to tone

[11:57] thank you very much. thank

[11:58] you for the presentation.

[12:02] I think some of them might

[12:07] colorado things are cool I

[12:08] these are kinds of things that

[12:10] you know we had never heard that

[12:11] you were working on before and

[12:12] I'm kind of

[12:15] wondering why you were

[12:17] prioritizing these kinds of

[12:19] more vanity projects

[12:22] then focusing on some of the

[12:25] other issues that seem to be

[12:27] more pressing at this time as

[12:29] you pointed out especially the

[12:31] cybersecurity threats being that

[12:32] we have a pretty

[12:35] difficult situation with our

[12:37] cybersecurity in addition

[12:40] I have heard through the

[12:44] grapevine that there are many

[12:46] employees that are at their

[12:47] breaking point

[12:51] and they're being overworked and

[12:53] maybe that's why you're

[12:55] getting such great performance

[12:57] and cost reductions because

[12:58] you're working people to the

[12:58] bone

[13:03] and I'm worried about those

[13:04] employees who are valuable

[13:05] assets to the organization

[13:09] not wanting to be with the

[13:10] organization anymore because

[13:10] they're being

[13:11] broken

[13:16] and overworked. So how tell me

[13:16] about that and and

[13:19] tell me what you're gonna do

[13:20] about it

[13:24] OK so I'm just going to take a

[13:26] pause. We have had some

[13:30] conversations with OlLS and

[13:31] Ed Decheco to try to understand

[13:35] what the authority of the JTC is

[13:39] in in talking to a number of

[13:41] these employees we do think

[13:43] there are a couple of nuggets of

[13:45] wisdom that we'd like to add to

[13:47] the quarterly report that we

[13:49] would either run a bill or talk

[13:50] to you about but

[13:55] in terms of statutory authority

[13:59] OIT is called out in

[14:03] 2437.51057 that they are the

[14:05] ones in charge of all the Hr

[14:08] stuff so we are going to be

[14:12] focused on the IT stuff but

[14:14] we are not going to be able to

[14:18] oversee the HR things. That said

[14:20] just one second please

[14:23] I would I'm gonna give you

[14:25] the floor to respond to whatever

[14:27] you would like to relate to that

[14:29] question but knowing that we are

[14:31] not going to be in a position to

[14:33] take any kind of a role because

[14:35] statutorily we don't hire fire

[14:37] we don't get involved in the Hr

[14:40] of of things so I'm gonna go to

[14:41] directorunonberg and then I'll

[14:42] go back toresentative Taonene.

[14:45] thank you madam chair and thank

[14:47] youpresentative Taonene for the

[14:49] question. As you pointed

[14:51] outpresentative to tone are

[14:52] people and employees and

[14:54] colleagues at OIT are our

[14:56] greatest asset. They are the the

[14:58] mechanism by which we deliver

[15:00] value to agency partners to

[15:01] state employees and to

[15:04] Coloradans as I pointed out

[15:06] we are seeing more and more

[15:09] colleagues be excited and

[15:11] invigorated and have positive

[15:12] feedback regarding the reorgan

[15:14] ization

[15:17] I know that it is a change.

[15:20] change is hard and we are doing

[15:22] a lot of work to support

[15:23] employees through the change

[15:25] that looks like significant

[15:27] amounts of training more than

[15:30] we've ever offered very clear

[15:33] performance expectations so that

[15:34] people know what they're

[15:36] expected to do and how they're

[15:38] supposed to show up in this

[15:40] model. I appreciate again

[15:43] changes hard and I do not

[15:44] minimize that

[15:47] and we're seeing people excited

[15:48] and the feedback is strong.

[15:53] very good and we are out of time

[15:55] but I am going to have rep to

[15:58] tone you can ask one last

[15:59] question and we'll see if we

[16:01] can't get like a really quick

[16:02] response and then we're going to

[16:04] move on to the business at hand

[16:04] today.

[16:07] we'rep tommy my first question

[16:09] my first question wasn't

[16:11] answered so I would like that

[16:13] answer but I think that the fact

[16:17] that you know employment was

[16:18] brought up by Misshomberg

[16:22] in the presentation and the

[16:24] reorganization and loss of

[16:24] employment

[16:28] is something that was not

[16:31] given anybody was given a heads

[16:33] up about and that affects how

[16:36] the organization functions and

[16:38] that reflects on our oversight

[16:41] duty of how the work is getting

[16:42] done and how the taxpayers money

[16:44] is being spent so I think that

[16:44] it is very relevant

[16:46] to what's going on

[16:49] but but that's what I would like

[16:53] to know and why why when I asked

[16:55] for questions that the

[16:56] questions are not being answered

[16:59] in a timely manner of've sent

[17:02] several emails and why is the

[17:02] legislative liaison

[17:06] not responding to my questions

[17:06] when I

[17:10] ask her questions on text. there

[17:11] there's there's a

[17:15] of a way of trying to

[17:17] get information to not get into

[17:19] my hands so it makes me very

[17:21] suspicious of what's going on

[17:24] with OIT over the last

[17:26] several months prior to you

[17:28] getting into power here and with

[17:31] you there and that is a very big

[17:32] concern of mine

[17:35] and I wanna know what kind of

[17:36] transparency you're going to

[17:37] give to us when we ask for

[17:38] questions.

[17:43] and I will just kind of share

[17:45] what what the conversation that

[17:47] we had you guys are we're

[17:49] going to be done with the

[17:50] conversation at this point, but

[17:51] we are going to be adding to the

[17:53] quarterly update just as a heads

[17:57] up. the number and cost and

[18:01] project number and costs and

[18:03] projects by contractor versus

[18:07] employee in terms of how you

[18:08] employ folks and then

[18:11] adding all that information

[18:13] aboutervicehub just the projects

[18:15] that are ongoing so I think that

[18:16] will actually address a lot of

[18:19] it and I'm hopeful that that is

[18:21] going to keep us in our lane in

[18:24] terms of ledge liaison we

[18:26] need to make sure that we stick

[18:29] to the formal channels of

[18:31] communicating especially

[18:32] during interim. I know it's a

[18:34] really busy time. I'm struggling

[18:36] myself to keep up with text

[18:38] messages so I would just say

[18:38] that a really

[18:43] good norm committee team is

[18:47] if we could stick to our

[18:49] official forms of communication

[18:51] through email. and please

[18:53] feel free to copy any and all of

[18:56] us on that so with that we're

[18:57] gonna move along and we'll

[19:01] welcome hickpuff to the table.

[19:02] Come on up. thank you.

[19:45] Wonderful thank you. yes

[19:45] please go ahead.

[19:50] marchman my name is Ralph Cho.

[19:50] I'm blessed to be the chief

[19:52] operations officer for

[19:53] healthcare policy and finance.

[19:56] I'm also ecstatic to be one of

[19:57] the senior executive team

[19:59] members in charge of the AI

[20:01] governance over at Hickpuff. I I

[20:02] will tell you for someone who

[20:03] has been in the business world

[20:04] long enough to remember when

[20:05] there weren't PCs

[20:09] the the advent of artificial

[20:11] intelligence and what it can be

[20:13] for our organization to help the

[20:15] efficiency, the effectiveness,

[20:17] the delivery of services to our

[20:19] members the interactions with

[20:21] our providers are all very very

[20:23] exciting things but they come

[20:25] with a real price in making sure

[20:26] that those things are done

[20:29] prudently, intelligently

[20:32] today we've got a number of

[20:32] folks we're going to shuttle in

[20:34] and out a number of folks we

[20:35] brought our greatest cast of

[20:36] characters

[20:39] with us so you know my job was

[20:41] to to really to just introduce

[20:42] introduce the theme and the

[20:45] enthusiasm that we have as we

[20:46] all know is one of the largest

[20:48] spends within the state wherever

[20:50] we can find ways to make things

[20:51] better, faster, smarter, more

[20:54] intelligent, more effective we

[20:55] need to do that but we need to

[20:56] do it with guidance and

[20:58] I'm gonna ask Shaland to

[20:59] introduce himself since he's

[21:01] going to cover our governance

[21:02] processes to give you a

[21:03] foundation and then we have I

[21:05] know a number of questions that

[21:06] you all have asked across a

[21:08] number of systems and we will be

[21:09] shuttling the folks through to

[21:11] be able to answer those specific

[21:12] questions on the specific

[21:14] questions and at some point here

[21:17] I will defer one of these

[21:19] seats or my seat Nina Mac is

[21:21] in the in the building and

[21:23] you you know part of this is

[21:25] CBMS and so don't don't mind us

[21:26] while we shift and move seat

[21:26] s

[21:30] sounds great and just for

[21:33] timekeeping um45 minutes so

[21:34] however you guys want to maybe

[21:36] this first one tell me how many

[21:38] m or whatever you want to do but

[21:39] if you could keep time, I just

[21:41] got45 minutes on my clock. Very

[21:43] good. Who would like to go? Very

[21:44] good. thank you.

[21:47] thank you madam Chair members of

[21:48] the committee. I'm Shaon Weiner.

[21:50] I'm the department's deputycFO

[21:52] overseeing compliance and

[21:52] enterprise risk.

[21:55] and I want to follow on with

[21:58] what Ralph said Ai is a brand

[21:59] new thing. there's a lot of

[22:01] potential in it but hickpuff

[22:03] wanted to make sure that when we

[22:05] approach this we did it right.

[22:06] we don't want to do Ai just for

[22:09] the sake of Ai we want to do Ai

[22:11] right and so we started from a

[22:12] compliance mindset

[22:14] we were one of the first

[22:17] agencies here or almost anywhere

[22:19] at some multi state conferences

[22:21] we've been asked to tell them

[22:22] how we approach governance

[22:23] because we were some of the

[22:25] first ones out of the gate with

[22:27] a strong governance structure

[22:29] and that governance structure

[22:31] uses a couple components. The

[22:33] first off we built an Ai

[22:34] business governance committee.

[22:35] so not only do we have the

[22:37] technology governance that we

[22:39] get with our partners at OIT but

[22:41] we said we don't want to just

[22:42] know what technologies are allow

[22:43] ed we want to know what

[22:46] technologies we should use and

[22:46] how we should use them.

[22:51] and so before any Ai tool rolls

[22:52] out we did have some that were

[22:53] in prior to this process that

[22:55] had gone through OIT approval

[22:57] but for the anything coming out

[22:59] now it goes through a governance

[23:01] process where we have a

[23:03] crossfunctional team. We have

[23:04] legal representatives we have

[23:05] privacy representatives we have

[23:07] technology representatives we

[23:08] have bias and ethics

[23:09] representatives we have program

[23:11] representatives and we want to

[23:12] make sure that we're coming at

[23:14] it from every perspective before

[23:15] we do that. We also have

[23:16] executive leadership myself,

[23:17] Ralph

[23:19] ar ish are on that committee

[23:20] as well

[23:23] Now this committee doesn't

[23:25] bypass other rules. We didn't

[23:26] want to set a separate process

[23:29] in place this focuses on is the

[23:31] Ai the right thing? So we still

[23:32] go through our normal OiT

[23:33] approvals we still get our

[23:34] support from our IT partners we

[23:36] work very cooperatively with

[23:37] them when we have our initial

[23:39] meetings with requesters they're

[23:40] sitting in the room right next

[23:42] to us asking questions too to

[23:43] make sure that we all have the

[23:44] answers we need.

[23:46] we also make sure we do budget

[23:47] approval just like we would for

[23:49] any other technology.hings have

[23:51] to be procured they go through

[23:53] the standard procurement process

[23:54] we get cybersecurity reviews, we

[23:56] get privacy reviews we get all

[23:57] of those things. So the bottom

[23:59] line of this we bring in the

[24:02] experts we need to make good

[24:03] decisions about the use of the

[24:04] tools that we have.

[24:08] on the other side of that we

[24:09] also want to make sure that we

[24:11] have the people there because Ai

[24:12] tools are just that they're

[24:14] tools they help people do things

[24:15] more efficiently. they help

[24:18] people do better things but they

[24:20] never replace people. the people

[24:23] are what do the work and so we

[24:24] designed this as a support

[24:26] structure to help those people.

[24:28] anything that comes out of Ai

[24:30] has to be reviewed whether it's

[24:30] a direct

[24:33] piece of output from a

[24:35] generative Ai we expect that the

[24:36] person getting it they're

[24:37] ultimately responsible for what

[24:39] it says and if it says something

[24:41] wrong they're held to

[24:41] accountable to it the same way

[24:43] as if they'd written something

[24:45] wrong on their own so they're

[24:47] responsible for that. on the

[24:48] other side sometimes we have

[24:49] systems that are using these

[24:51] processes we want to make sure

[24:52] that we have auditing in there

[24:53] that we're sampling that we're

[24:54] testing these things to make

[24:54] sure that the outputs are right

[24:59] we also make sure that we

[25:01] have privacy and security

[25:03] compliance. It does not get PHI

[25:05] unless it has a HIPA BA just

[25:06] like any other system out there.

[25:09] we make sure that we have those

[25:10] things in place before anyone

[25:12] can use it for that will

[25:13] restrict access we'll do things

[25:14] like that to make sure that we

[25:15] don't have those things happen.

[25:18] we also want to make sure that

[25:20] if there is any unapproved

[25:22] unsecure anything like that we

[25:24] have a strict prohibition

[25:24] against those things we have

[25:28] prohibited tools because they

[25:29] automatically join meetings

[25:31] without the person present. We

[25:32] didn't think that was

[25:33] appropriate and so we prohibited

[25:34] those and we've worked with our

[25:35] partners at OIT to lock that

[25:36] down so that people can't

[25:38] download those things. so we've

[25:40] taken those steps to make sure

[25:41] that we're doing it right and

[25:43] that's the biggest part that we

[25:44] have with our governance

[25:45] structure and so with that I'll

[25:48] hand it over to antoinette for

[25:50] one of the first case studies of

[25:50] one of the tools that we have

[25:51] approved.

[25:53] yes you may begin

[25:56] right

[26:00] good morning madam chairirre

[26:02] representatives and senators I'm

[26:04] Antontte Taranto, the chief

[26:05] customer officer for the

[26:06] department

[26:09] I don't know if this is close

[26:10] enough

[26:15] so I often think about

[26:16] government

[26:16] collecting taxes

[26:19] closer

[26:23] thank you. let's do that. OK. So

[26:25] I often think about government

[26:27] collecting taxes in two ways

[26:31] time and dollars. So my job is

[26:32] to shrink the time

[26:35] and medicaid is complicated.

[26:37] Healthir Colorado has more than

[26:39] 30 different benefit programs

[26:41] with services delivered through

[26:42] multiple partners and processes.

[26:45] A member may have a simple

[26:46] question

[26:50] and not knowing where to start

[26:53] what terminology to use or what

[26:55] information applies to them is

[26:56] is difficult

[26:58] so we added a conversational

[26:59] assistant to the Healthir

[27:01] Colorado mobile app. It's

[27:03] available 24/7 and members can

[27:05] answer questions in everyday

[27:06] language.

[27:09] the role of AI is intentionally

[27:12] narrow. It uses more than70

[27:13] approved healthalthir

[27:16] Coloradoours to generate plain

[27:17] language responses.

[27:21] and importantly this is just an

[27:23] informational tool. it does not

[27:25] determine eligibility, approve

[27:26] or deny services or coverage or

[27:27] make clinical decisions.

[27:31] and we regularly review

[27:35] the questions members are asking

[27:36] in addition to the answers being

[27:36] generated.

[27:41] and we're on slide two great

[27:45] so now that members are using

[27:46] it we're starting to see what

[27:46] this can tell us.

[27:49] Members have asked roughly

[27:53] 10,000 questions satisfaction is

[27:56] about72% and we're seeing89%

[27:59] accuracy. Most sessions are just

[28:00] one question.

[28:03] and to me that last piece really

[28:05] matters. Success isn't getting

[28:07] someone to sit there and have a

[28:08] long conversation about

[28:08] Medicaid.

[28:11] it's helping them get what they

[28:12] need and get on with their day.

[28:16] we're also learning from what

[28:16] they're asking

[28:19] The assistant is able to answer

[28:21] 62% of questions without

[28:22] directing the member elsewhere.

[28:27] for the remaining 38% we

[28:28] intentionally designed

[28:31] Ai to guide the member to the

[28:32] appropriate partner.

[28:37] and 30 of that 38% are going to

[28:39] counties for eligibilitylated

[28:41] needs which I think we all

[28:43] probably knew that but now we

[28:45] have information that confirms

[28:49] So technology is doing more than

[28:51] answering those questions it's

[28:52] showing us where the friction

[28:52] is.

[28:55] and lastly

[28:58] reiterating what some of what

[28:59] Shain has said

[29:03] before each use we tell members

[29:05] to not enter their protected

[29:07] health information or their

[29:09] personal information. If they do

[29:11] there are guardrails in place

[29:12] designed to remove that

[29:12] information.

[29:14] and that's all I have for you.

[29:14] thank you.

[29:18] very good rett Pasco.

[29:23] thank you madam chairir. so

[29:24] my question is about where you

[29:28] said89% accuracy so I first of

[29:29] all I want to double check is

[29:30] this

[29:32] the scenario where you've got

[29:33] someone who works for the

[29:35] department that's going to

[29:38] review it before it goes to the

[29:41] member or is it automatically

[29:43] going straight to the member and

[29:44] then

[29:46] what is the process for

[29:50] dealing with the other

[29:54] they can't do math 11% Miss

[29:54] Toronto.

[29:57] thank you madam chair andres

[30:00] representative. yes, I was

[30:02] hoping you'd ask that question.

[30:05] so we designed the Ai for that

[30:09] 11% to refer people to our

[30:11] member contact center. so it's

[30:13] not technically an inaccurate

[30:14] answer but when it doesn't know

[30:17] the answer it refers to our

[30:20] contact center. so what we've

[30:23] discovered in that 11% are

[30:24] times where

[30:27] people are asking for things we

[30:29] didn't train it for. for example

[30:31] people ask for explanation of

[30:35] benefits. or it's asked

[30:38] members often ask for pharmacy

[30:38] bin numbers

[30:42] things that we don't have in

[30:45] Medicaid. so we have to then we

[30:46] are learning from those

[30:48] questions and then we go back in

[30:49] and we train the Ai to say

[30:51] Medicaid doesn't have a

[30:52] pharmacybI number you use your

[30:55] state ID which consists of so

[30:57] those are the errors and again

[30:59] back to when we do identify

[31:01] those errors it is automatically

[31:04] referring folks to our member

[31:04] contact center.

[31:11] Yes, hett Pasco were you talking

[31:12] about one particular one of

[31:13] their programs or were you just

[31:15] talking in general about their

[31:16] errors. I just wanted to

[31:16] clarify.

[31:21] sorry sorry I think here

[31:22] we're talking about maybe this

[31:23] is what we should clarify also

[31:27] this is where members of are

[31:29] going to the website and asking

[31:33] questions and I I understand

[31:36] that the areas where the AI

[31:37] doesn't know the answer is that

[31:39] how we're defining it?rrect OK.

[31:41] correct and then you can go

[31:43] ahead and follow up. I know you

[31:45] had another question that was

[31:46] was trying to

[31:47] so

[31:50] is there any way that you guys

[31:53] are looking for like as they

[31:55] call it hallucinations like they

[31:57] give a very confident wrong

[31:59] answer and are we looking for

[32:02] that and and how and what's

[32:05] going on thereto yes madam

[32:09] chairirpresentative, yes I

[32:11] personally am looking at every

[32:13] single question and response. I

[32:15] am not seeing hallucinations

[32:17] where things are just flat out

[32:22] wrong we have trained it to

[32:23] specifically look at our

[32:28] documents the se0 and

[32:31] those guard rails are very tight

[32:34] so when it's not sure and it

[32:37] can't find it in that library

[32:38] it's saying you know I don't

[32:41] know thisact our staff at the

[32:42] member contact center.

[32:44] yeah

[32:45] pascal

[32:50] sorry

[32:53] so just to follow up, you

[32:55] guys have your own information

[32:57] set and it's your own sandbox

[32:59] it's not going out on the

[33:01] internet and finding random

[33:02] stuff or anything like that and

[33:04] if it can't find it in that set

[33:06] of data then it just says I

[33:08] don't know go to a human

[33:10] basically yes exactly

[33:13] Perfect. OK.

[33:17] is there any way

[33:19] I know we're giving a warning

[33:21] that says don't enter

[33:23] HIPparotected kind of stuff but

[33:28] since it isIPparotected data

[33:31] that people could be feeding in

[33:34] how are we separating that? I

[33:35] know I heard you say you're

[33:37] looking at every response that

[33:39] comes in. you would be the one.

[33:43] are there non clinical staff are

[33:44] are there folk like that would

[33:44] probably be

[33:45] you you're probably a non

[33:47] clinical staff. I'm not sure

[33:51] who's looking at HIPAda. Is that

[33:53] problematic or is there any way

[33:55] just to reject and identify

[33:57] hiAda like do you know what I

[33:59] mean? Do you understand what I'm

[34:01] saying? yes madam chair thank

[34:05] you. so I have yet to see hey

[34:06] I have this rash

[34:12] what do I do? what I am

[34:14] seeing are things like is this

[34:15] covered is that covered and

[34:18] those are in my mind general

[34:19] questions. now you can connect

[34:21] dots saying oh maybe this person

[34:24] needs chiropractic care but

[34:25] they're asking are these things

[34:27] covered? I have yet to see

[34:30] someone coming into it saying

[34:32] hey I have this going on what do

[34:36] I do? we also have our

[34:36] nurse advice line as

[34:39] one of the go to's

[34:43] for the responses if we suspect

[34:45] people are asking or wanting

[34:48] health information or you know

[34:48] clinical guidance

[34:55] thank you madam chairir. and one

[34:57] additional thing with that

[35:01] the system in which this exists

[35:03] already has a HIPA BA in place

[35:05] so we have our legal protections

[35:07] in place in addition to the

[35:09] functional protections. What's a

[35:11] BA Oh sorry a business associate

[35:16] agreement it's the forms it's

[35:17] the addition to the contract

[35:19] that allows us to provide PHI to

[35:22] a contractor. Got you and that

[35:22] is legal from C

[35:25] M S too. that was another

[35:27] question I had that's fine. I'm

[35:29] sure they're fine with sending

[35:31] that data. OK.

[35:33] any other questions

[35:38] OK. we might have saved time on

[35:41] your guys' so very good. thank

[35:43] you very much. We will move on

[35:44] to the next segment.

[35:48] good morning madamir and members

[35:49] of the committee. I'mco Libloom.

[35:51] I'm the provider relations and

[35:52] fiscal agent operations

[35:52] divisionreor

[35:55] at the department of

[35:56] healthalthcare policylicy and

[35:59] financing. thank you for the

[36:01] opportunity to discuss our use

[36:03] of artificial intelligence

[36:04] within the Colorado Medicaid

[36:07] provider call center. I want

[36:08] to begin by providing some

[36:10] contents about what the

[36:13] technology is, why we use it,

[36:15] and what it does not do. the

[36:16] department contracts with Optum

[36:17] to operate our provider contact

[36:19] center and as part of that

[36:20] service we use the core Ai

[36:22] platform to provide an enhanced

[36:22] conversational selfservi

[36:26] option for medicaid providers

[36:26] who contact call center.

[36:30] Our goal in implementing this

[36:31] technology was pretty pretty

[36:32] straightforward which was to

[36:33] improve

[36:35] service delivery to

[36:37] providers. provider's contact

[36:39] us with a range of of questions

[36:41] and many of those are routine

[36:42] inquiries where information the

[36:43] provider needs already exists

[36:46] when that within hickpuff's

[36:49] Medicaid system core AI allows

[36:50] us to to make that information

[36:52] more readily available through

[36:54] an automated conversational

[36:55] interaction while preserving

[36:57] live customer service resources

[36:59] for inquiries that require

[37:02] additional assistance. So

[37:02] when a provider calls the

[37:03] technology uses artificial

[37:03] intelligence

[37:05] to recognize the intent of the

[37:07] provider's request and determine

[37:08] the appropriate predefined

[37:10] conversational flow. the

[37:12] system can query Hickpuff's

[37:13] existing medicaid data and

[37:15] provide information responsive

[37:16] to the provider's inquiry

[37:18] and when an inquiry cannot be

[37:19] appropriately resolved through

[37:21] selfservice it's transferred to

[37:22] a live customer service agent.

[37:25] an important distinction is that

[37:27] artificial intelligence and

[37:29] automated decision making are

[37:31] not synonymous. Cory eyes and an

[37:33] ai enabled service interface and

[37:36] it's not a medicaid decision

[37:36] making system.

[37:39] I cannot approve or deny a

[37:41] claim it can't change or adjust

[37:42] a claim. it cannot determine

[37:44] payment amounts, approve or deny

[37:45] provider enrollment determine

[37:47] provider status or make member

[37:49] eligibility determinations.

[37:51] Those decisions continue to be

[37:52] made through Hickpuff's

[37:54] established medicaid systems

[37:55] and business processes.

[37:59] Core Ai can retrieve and

[38:01] communicate information about a

[38:02] decision that has already been

[38:04] made. for example it may provide

[38:05] a provider with information

[38:07] about the status of a claim

[38:09] but communicating the outcome of

[38:10] a decision made by Hickpuff

[38:12] system is very very different

[38:12] than making that decision

[38:13] itself.

[38:17] also want to distinguish between

[38:19] Hickpuff's implementation for

[38:22] some of the degenerative Ai

[38:23] technologies that we've received

[38:25] significant public attention

[38:26] core Ai technology platform is

[38:28] capable of supporting generative

[38:29] Ai however the department's

[38:31] implementation of the provider

[38:33] call center does not use

[38:35] generative Ai or large

[38:36] language modules

[38:39] The system does not

[38:40] independently form formulate

[38:41] answers or interpret medicaid

[38:43] policy.stead it uses intent

[38:45] recognition follows

[38:46] predefined conversational flows

[38:48] and retrieves information from

[38:50] HiPuff's existing system. That

[38:51] distinction is also important

[38:54] when considering the risk of Ai

[38:55] hallucinations which you

[38:57] talking about in your last

[38:59] conversation our tool is not

[39:00] independently generating

[39:01] information in response to

[39:03] provider questions it's

[39:04] responding within a fine

[39:06] conversational processes and

[39:06] retrieving information directly

[39:09] from hickpuff's medicaid system.

[39:13] it does not create or infer

[39:14] information beyond what is

[39:15] available through those

[39:17] processes and systems. data

[39:19] production protection is another

[39:21] important consideration whenever

[39:23] technology interacts with

[39:24] Medicaid information

[39:25] the platform does not retain

[39:29] provider or medicaid data beyond

[39:31] the active transaction limited

[39:32] operational logs are maintained

[39:34] for up to 90 days for security

[39:37] audit troubleshooting and

[39:39] support purposes and the data

[39:40] associated with the tools

[39:41] maintained within the

[39:42] continental United States.

[39:45] Finally the human component

[39:46] remains an important part of our

[39:48] service model. Coreyi is

[39:49] intended to enhance customer

[39:51] service not eliminate or replace

[39:53] human assistance routine

[39:54] inquiries can be handled

[39:57] efficiently through 24x7

[39:59] selfservice allowing live

[40:00] representatives to focus their

[40:01] time and expertise on

[40:03] interactions that require

[40:05] additional research or

[40:07] assistance and when an ur

[40:08] inquiry cannot be appropriately

[40:09] resolved through the automated

[40:11] interaction it's transferred to

[40:12] a live representative and

[40:15] handled through the department's

[40:16] established processes.

[40:17] Ultimately we view this as

[40:19] targeted and deliberately

[40:21] limited use of artificial

[40:24] intelligence. we're using AI

[40:25] for what it can do well in in

[40:28] this environment which is

[40:29] understand the nature of the

[40:30] provider's request to guide the

[40:31] interaction and help providers

[40:33] quickly and efficiently access

[40:35] information that already exists

[40:36] within our systems. we have

[40:38] not delegated Medicaid decision

[40:40] making authority to artificial

[40:41] intelligence and that

[40:42] distinction between using

[40:45] Ai to improve access to

[40:47] information and using Ai to make

[40:49] consequential decisions is

[40:51] fundamental to how we

[40:52] implemented this

[40:57] and and you'll see up on the

[40:58] slide just for

[41:01] comparison purposes our core

[41:03] AI product is about 2700

[41:05] interactions per month and we

[41:07] take about m probably 1f to

[41:09] 16,000 live live provider calls

[41:13] so with that if there's any

[41:14] questions I'd be happy to

[41:14] to answer them

[41:19] right and just so I understand,

[41:21] do is someone else gonna be

[41:23] presenting about the Olivia

[41:25] programme. OK, we only have

[41:28] about4 minutes left. I do see we

[41:31] have some questions. let's go

[41:32] ahead and take the questions

[41:36] but we'll try to be quick with

[41:38] them. OK senatorball

[41:43] thank you madamir and thanks for

[41:44] the presentation

[41:47] the slide has improved customer

[41:49] experience how do you measure

[41:51] that and is that improved over

[41:55] measurements when people talk to

[41:56] actual people

[42:00] thank you for the question

[42:04] Lindblom thank you for the

[42:04] question.

[42:09] so we do have another

[42:10] piece of new technology that we

[42:11] implemented with with our

[42:13] call center implementation and

[42:15] that was a survey tool so every

[42:16] provider is able to

[42:20] provide feedback on on that

[42:22] transaction whether or not it

[42:23] was through core Ai or from

[42:27] talking to a a a person on

[42:29] the phone. we started off

[42:33] with about a score of4.5 out of

[42:35] five when we first went live and

[42:37] in May of of 2025 and we're up

[42:41] to about a4.8 average score so

[42:43] we we have seen you know the

[42:47] satisfaction of the providers

[42:48] go up in the in the last year

[42:51] and a half and it also gives

[42:52] them a chance if they don't need

[42:54] to talk to a a a person just

[42:55] very very much like it would be

[42:57] if you or I called our bank and

[43:00] needed to know what is our

[43:02] account balance we didn't want

[43:03] to talk to somebody we could get

[43:04] that information through

[43:04] automated means

[43:06] Thank you

[43:09] a repelty

[43:20] Oh unmuting would be great

[43:23] sorry about that well for me

[43:26] I guess I wanna understand

[43:28] as far as you si utilizing cori

[43:30] and and and everything you know

[43:32] are you for as far as the IT

[43:33] infrastructure are you using the

[43:35] current IT infrastructure this

[43:36] is or or my understanding this

[43:39] is a third party tool that

[43:43] you're using and if it's you

[43:44] do you know if there

[43:44] if they

[43:48] re using in cloud or or how are

[43:50] they using it IT wise and how

[43:51] involved I really want to know

[43:53] really how involved was our IT

[43:55] security and involved in

[43:57] usingoreAii in in your

[43:58] decision making and and and that

[44:01] director Lindblom

[44:05] thank you for the question so

[44:07] the way the technology works it

[44:09] is it is a third party tool that

[44:12] Optum uses and it uses APIs to

[44:13] to connect with our medicaid

[44:16] management information system so

[44:19] it will make a call for example

[44:21] if a provider puts in a

[44:22] transaction control number for a

[44:23] claim that they want status on

[44:25] it would make a call to our MmIs

[44:27] and then the status of that that

[44:28] claim would be

[44:30] would be returned.

[44:33] in terms of the second part of

[44:34] the question is that something

[44:35] that

[44:37] you might be able to to weigh

[44:41] in on IT security I think

[44:43] andrep Kelty would you like to

[44:45] just restate your questions so I

[44:46] don't add my own

[44:49] interpretation so that our IT

[44:50] folks can answer that.

[44:53] Sure I wanted to know how

[44:54] involved they were in the

[44:56] selection of Kitoreiii and the

[44:58] and the safety of it with the

[44:59] information that's gonna be

[45:00] utilized in or being sent and

[45:02] driven into it I cause I know

[45:05] that they're not a US company so

[45:07] that bothers me a little bit

[45:08] but I just wanna know how

[45:10] closely our own IT people were

[45:12] involved in choosing it and and

[45:14] and how do they feel with as far

[45:16] as the security of that that

[45:16] it's not a US company

[45:21] thanks repkelty thank you Rep

[45:22] Kelty

[45:26] so OIT was involved in the

[45:27] approve the review and approval

[45:29] process. so they were in

[45:31] there the selection was actually

[45:32] through our contractor our

[45:33] contractor chose their

[45:35] subcontractor submitted that to

[45:37] us to request to be able to use

[45:39] this. We then ran it through the

[45:42] process with OIT to get OIT to

[45:45] review and approve it from

[45:46] their through their Ai process

[45:47] as well as through the security

[45:51] and privacy aspects of

[45:52] their standard review process.

[45:52] so

[45:53] it followed their standard

[45:54] review process.

[45:57] that's helpful and did you say

[45:58] that the contractor is Optum

[46:02] provider call center contractors

[46:06] optimum yes. soopptum chose core

[46:07] Ai OK.

[46:10] mm hm

[46:11] yeah

[46:12] OK

[46:13] thank you

[46:20] very good. let's move to

[46:23] Olivia and we we took oh gosh

[46:24] I'm so bad rett Pascal keep

[46:25] going.

[46:29] so repelty I'm interested to

[46:30] know what country

[46:32] uhoreaii is from.

[46:38] repelty do you know

[46:40] I believe it's from India.

[46:43] did she say India?

[46:47] good yeah India that we're

[46:48] getting some nods here at the

[46:49] table.

[46:51] OK.

[46:55] OK. thank you and thank you

[46:59] go ahead please thank you good

[47:01] morning committee. My name is

[47:02] Ashleyennzo and I'm the director

[47:03] of fraud waste and abuse at

[47:04] Colorado Medicaid.

[47:06] I'm here to talk to you guys

[47:08] about hickpuff's new analytic

[47:09] tool that we're utilizing to

[47:11] identify and detect fraud waste

[47:13] and abuse patterns in claims and

[47:14] billing information that's

[47:14] submitted by providers.

[47:17] hickpuff has contracted with

[47:19] Oliviaanalytics through standard

[47:21] procurement efforts to acquire

[47:23] licenses to their trademarked

[47:26] FWA finder FWA being fraud waste

[47:26] and abuse.

[47:29] FWAiner is a service as a

[47:31] software or aA platform that

[47:33] enables state medicaid agencies

[47:35] to identify and investigate

[47:36] providers for suspected program

[47:37] integrity concerns

[47:41] the tool serves as a deployment

[47:42] pipeline and reporting platform

[47:44] for statistical models that are

[47:46] developed by internal staff as

[47:46] well as allevia staff.

[47:49] those statistical models are

[47:51] used to assign risk and generate

[47:53] leads for potential fraud

[47:55] investigation and to identify

[47:56] where there are anomalies in

[47:59] billing among peer groups. The

[48:01] tool only runs algorithms that

[48:03] are developed by human analytics

[48:04] staff and are in no way

[48:05] developed by generative

[48:07] artificial intelligence there

[48:09] are nollMs this is solely our

[48:11] team developing these models

[48:13] within the tool or the Olivia

[48:14] staff putting those models

[48:14] within the tool.

[48:18] the platform has over 150

[48:19] analytic models and hick puff

[48:21] analytics staff have verified

[48:23] approximately75 of those models

[48:25] as accurate to run against our

[48:26] medicaid data.

[48:29] to give you a sense of kind of

[48:30] the analytic models that we are

[48:31] running we have a model

[48:33] that's for impossible hours

[48:35] which examines providers billing

[48:36] more than 24 hours in a day.

[48:39] we have a utilization outlier

[48:41] model which examines per member

[48:42] per month paid amounts to

[48:43] providers that measures them

[48:45] against their peers we have

[48:47] spike models that detect

[48:49] providers with large sudden

[48:50] increases in their billing

[48:53] and then the network analysis

[48:55] model which flags providers who

[48:56] may have connections to known

[48:56] risky providers.

[49:00] the tool stores the analytic

[49:02] models and runs them monthly to

[49:04] generate the leads that we then

[49:06] look at or the focal points.

[49:07] So do you want to make it

[49:09] crystal clear that the tool

[49:11] takes absolutely no action

[49:12] whatsoever. These leads come out

[49:15] in a statistical mathematical

[49:17] sense we then have our data

[49:18] analysts as well as our

[49:19] investigation and case managers

[49:21] who take a look at those leads

[49:23] to then determine if there's

[49:24] some additional action that

[49:24] needs to be taken

[49:27] Humans manually review all of

[49:29] the data and the leads that come

[49:30] out of the tool they manually

[49:32] open cases for further

[49:33] investigation we manually

[49:35] collect additional documentation

[49:39] and we manually initiate adom's

[49:41] audits and so forth. The

[49:43] platform runs on Olivia's Aws

[49:45] cloud environment which is soC2

[49:47] type 2 and high trusts certified

[49:48] and includes all national

[49:49] institute of Standards and

[49:52] Technology controls the tool has

[49:53] received OIT approval for use

[49:57] after reviewing the security

[49:58] and data pros protocols prior to

[49:59] go live. There's also a

[50:01] multifactor authentication for

[50:02] the users to log in.

[50:05] The tool connects to Hickpuff's

[50:07] data warehouse which has also

[50:09] been approved by OIT for proper

[50:10] security measures and controls

[50:13] and then the FWAiner receives

[50:15] monthly copies of flat files of

[50:16] data from the warehouse that are

[50:17] used in the models

[50:20] so just to go over real quick

[50:23] who utilizes the tool. The tool

[50:25] is only used by fraud waste and

[50:27] abuse analytics staff. it is not

[50:28] used by members there is no

[50:29] logins for providers other

[50:31] agencies have no access to the

[50:33] tool in fact a lot of our

[50:35] other offices do not have access

[50:36] to the tool it's solely for our

[50:37] division.

[50:39] the tools not used by any

[50:41] external parties and it does not

[50:43] connect to the internet other

[50:44] than the cloudbased platform

[50:45] where we log in so there's again

[50:48] no no large language models

[50:49] there's no generative Ai nothing

[50:50] that is used within the tool.

[50:53] and just a little bit as to

[50:55] why we're using the tool we have

[50:57] statistical analysts that have

[50:58] been creating models like this

[51:01] for decades at heckpuff. The

[51:03] problem is that the tool we

[51:04] didn't have a tool that could

[51:05] store all of these models so

[51:06] they would create things in R or

[51:09] in python or in sequel be able

[51:11] to manually run those models on

[51:13] occasion or on a regular basis

[51:14] and it would be very difficult

[51:16] to cross compare models. The

[51:17] tool allows us to store

[51:18] everything within one place so

[51:20] it's automatically running on a

[51:20] monthly basis. we can still

[51:24] code within the tool we can code

[51:25] externally from the tool and

[51:27] deploy these models within the

[51:29] tool or we can use predefined

[51:31] models that have come from

[51:32] analysts at the Olivia

[51:35] company

[51:39] we are utilizing this tool to

[51:41] again just help to identify more

[51:43] and more areas where we may

[51:44] need to to put in some

[51:45] additional effort. Antoinette

[51:47] said that she's responsible for

[51:49] saving time. I'm responsible for

[51:51] finding money. and so this

[51:53] tool is helping us to really

[51:54] actually hone in on the bad

[51:55] actors that we may need to look

[51:56] at which is actually

[51:57] advantageous for our providers

[51:59] because we no longer have to

[52:00] look at the group of the

[52:01] providers as a whole to say

[52:03] who's doing what bad y'all are

[52:05] in trouble. now we can say these

[52:06] guys look a little bit different

[52:07] than the

[52:07] rest of your peer group and so

[52:09] we're gonna pull them out and do

[52:10] a little bit more investigation

[52:11] without impacting other

[52:12] operations.

[52:19] I will pause there you ok we are

[52:21] going to now move to quick

[52:24] questions on this and then we're

[52:25] gonna welcome the CBMS crew to

[52:28] come up I do seehett Pascal

[52:29] has her hand up but I saw rep to

[52:32] tone first so we'll do rep to

[52:33] tone rep pascal and then

[52:36] senatorball rep to tone vice

[52:36] chair to tone thank you madam

[52:37] chair

[52:38] thank you madam chairir.

[52:42] compared to the models that you

[52:44] have been previously running

[52:47] individually how does the

[52:47] accuracy compare to this model

[52:54] and have you noticed any marked

[52:55] increase in savings of time

[52:59] using this model compared to the

[53:01] old model because if you are

[53:02] having more you know less

[53:03] accuracy with this model and you

[53:06] have to follow back up or follow

[53:09] leads that are no good then you

[53:11] may not be saving time so just

[53:12] curious about

[53:15] that.rectordurienzo thank you

[53:17] for the question. so as in

[53:19] response to the accuracy of the

[53:20] models I don't have metrics

[53:20] because again these are

[53:21] analytics staff that are still

[53:23] coding within the tools so their

[53:25] models are the ones that are

[53:26] being deployed within the tool

[53:29] unless we're using a predefined

[53:30] model which they're also

[53:32] verifying as accurate. So I

[53:33] think that the predefined models

[53:35] as well as the ability to code

[53:36] within the tools save

[53:37] administrative time from us

[53:39] having to use everything

[53:40] external to the tool and then

[53:40] deploy it

[53:43] but the the tool is

[53:45] definitely allowed us to look

[53:47] at things a lot quicker and on

[53:49] slide two you'll actually see a

[53:50] little bit of what we've been

[53:54] looking at within the metrics

[53:56] so we've helped to generate82

[53:57] new leads with the tool for

[53:59] review and prioritization of

[54:00] actions and just to give you

[54:01] guys a little bit of a a

[54:03] breakdown as to what we're

[54:05] seeing so in particular

[54:06] groups we're actually utilizing

[54:08] the tool to identify prepayment

[54:10] efforts where providers could be

[54:10] placed on prepayment

[54:13] to monitor their billing and

[54:15] help correct any actions we

[54:17] have4 providers of a certain

[54:18] provider type that are on

[54:19] prepayment review right now that

[54:20] we've already generated about

[54:23] $200,000 in savings in six weeks

[54:25] just by having those very

[54:27] specific providers on prepayment

[54:29] review in some other areas we're

[54:30] utilizing the tool to look at

[54:32] post p analytics to determine

[54:34] who we may need to audit next.

[54:36] We right now have a risk pool of

[54:38] approximately 12 providers in a

[54:40] couple provider types that is

[54:40] around

[54:41] I can give you the exact number

[54:43] we'll say tens of millions of

[54:44] dollars wow

[54:47] nice work rett Paskell and then

[54:48] Senator Ball.

[54:51] thank you madam chairir. I was

[54:53] just curious for each of these

[54:54] three tools how long have they

[54:54] been deployed?

[54:57] I'll go first since my mic's

[55:01] onsio thank you the tool went

[55:02] live September of last year for

[55:05] the livia FWAiner and we

[55:06] received CMS certification in

[55:06] May of this year.

[55:13] What about core Aii core Ai was

[55:14] implemented in May 2025.

[55:16] and then what about the member

[55:20] three months 3 months ago.ice.

[55:22] did you get all that

[55:24] Very good

[55:25] Senator Ball

[55:31] thank you madam chairir. for

[55:34] Olivia of the82 new leads how

[55:37] many of those were actually

[55:38] fraud waste and abuse

[55:42] director Durizio. I would have

[55:44] to get the the breakdown. I mean

[55:45] they're they're all classified

[55:46] as potential fraud waste and

[55:48] abuse.raud comes with an element

[55:51] of criminal intent and so those

[55:52] cases are typically less than

[55:55] the waste and the abuse cases

[55:56] we can't comment on cases

[55:58] that may have been referred for

[55:59] investigation because they may

[56:00] be with law enforcement so we

[56:01] don't want to impede their their

[56:04] work but they're all considered

[56:06] fraud waste and abuse potential

[56:06] issues

[56:08] very good

[56:13] thank you so much. we

[56:15] appreciate you all. very

[56:17] good. we're gonna move on and

[56:20] get CBmS up, ir say if I may I

[56:23] did phone a friend core Ai is

[56:25] based in California. Oh OKok,

[56:26] very good. thank you.

[56:31] pascal yeah

[56:34] might have used some Ai to try

[56:37] to see what the story was there

[56:41] but yeah it's yeah it's it's it

[56:43] seems like it has a significant

[56:46] presence in India

[56:48] includingiyderabad. I always

[56:48] have trouble with that name

[56:52] but yeah it looks like it's

[56:53] not actually established there

[56:56] I have

[56:57] I have a question

[57:00] here we go very good. Miss mac

[57:21] and the floor is yours whenever

[57:22] you are ready.

[57:24] you so much madam Chair.

[57:29] right my name is Nina Mac. I'm

[57:30] the chief product officer for

[57:31] Kobe's the Colorado

[57:32] benefitnefits eligibility and

[57:34] enrollment systemstems which

[57:37] includes CBMS and peak and

[57:38] I'm here to talk about the two

[57:42] Ai uses in CBmS the first is

[57:44] the use of hyperscience Icr Ocr

[57:45] so that's intelligent character

[57:47] recognition and optical

[57:50] character recognition this is

[57:51] used to help with document

[57:54] processing. so right now when

[57:55] folks have to apply report a

[57:56] change

[57:59] or renew their benefits they

[58:00] often need to send in paperwork

[58:01] they can do so digitally but

[58:02] several people still choose to

[58:05] send in paperwork through the

[58:07] mail in paper format and when

[58:09] that happens then caseworkers

[58:10] need to process that

[58:11] documentation manually enter the

[58:13] data into CBmS and then that can

[58:15] obviously be time consuming and

[58:16] then you run the risk of fat

[58:18] fingering and making mistakes

[58:19] when you're transcribing from a

[58:21] document into into the system so

[58:24] hypersciences ICr Ocr solution

[58:25] extracts the data from the

[58:26] documents the

[58:29] documents get scanned in then it

[58:30] gets sent to hyperscience Icr

[58:32] where it will extract the data

[58:35] convert handwriting into text

[58:36] and then upload that data into

[58:39] CBMS the accuracy of that

[58:41] process is over 99.8% accurate.

[58:43] it is very good at when it

[58:45] recognizes text, it recognizes

[58:47] it accurately. and we are

[58:50] using this currently for about8

[58:50] types of documents that can be

[58:51] submitted

[58:53] next slide

[58:59] and all anytime data is

[59:01] extracted it is available in

[59:03] CBMS screens so that workers can

[59:04] review it they can see which

[59:06] fields were changed by ICr they

[59:08] can correct errors they can also

[59:10] entirely revert the ICr data

[59:12] changes back to the prior

[59:13] state if they see any issues

[59:14] with the data entry

[59:18] if there is handwritten data and

[59:20] hyperscience rcr it does not

[59:21] cross the threshold for

[59:22] confidence that it can read it

[59:23] correctly it will flag it for

[59:25] human review rather than trying

[59:26] to attempt to extract that data

[59:27] and upload it to CBmS.

[59:31] and there is the ability for

[59:33] humans to be able to say whether

[59:35] or not it was accurate in and

[59:37] that and that helps it train and

[59:38] get better in recognizing the

[59:39] text over time

[59:43] so in the past 12 months

[59:45] over 200,000 documents have gone

[59:47] through ICr so that workers

[59:48] did not need to complete the

[59:50] manual data entry for those

[59:52] and a couple of the key things

[59:53] besides the time savings with

[59:55] that is also some of the

[59:57] impacts for for for the clients

[1:00:00] that we're trying to serve. So

[1:00:00] for example when a

[1:00:02] redetermination or renewal for

[1:00:02] medicaid comes in

[1:00:06] it can it will check that

[1:00:08] there's a signature on that page

[1:00:09] and as long as there's a

[1:00:11] signature it will mark in CBmS

[1:00:13] that the redetermination has

[1:00:15] been received which will then

[1:00:16] start that process so that the

[1:00:17] case cannot be closed for lack

[1:00:19] of returning the paperwork

[1:00:21] otherwise if you had to wait

[1:00:23] for a person to enter in that

[1:00:24] data then there is a period

[1:00:25] between when the document was

[1:00:26] submitted and when somebody

[1:00:27] processes it where there could

[1:00:29] be you know a chance for then

[1:00:31] that case to get closed instead

[1:00:32] of it automatically being marked

[1:00:33] so if the person did what they

[1:00:33] needed to

[1:00:35] do then their case can't be

[1:00:37] affected and then similarly

[1:00:39] for snap if a client reports

[1:00:41] that they have no changes then

[1:00:42] their benefits can automatically

[1:00:43] continue rather than wait for a

[1:00:44] worker to process that

[1:00:44] paperwork.

[1:00:47] I'll pause there before the

[1:00:49] second use case. Oh I appreciate

[1:00:51] that. How did you know? I'm

[1:00:53] excited that that will hopefully

[1:00:55] that redetermination renewal

[1:00:57] as well as the snap is just

[1:00:59] automatic. I hope that really

[1:01:01] helps people stick to their

[1:01:02] benefits that's a very

[1:01:04] compassionate IT use I would say

[1:01:04] repelty

[1:01:10] I was gonna mention something

[1:01:12] about the the India

[1:01:13] connection withoreiii.

[1:01:15] that

[1:01:18] I do that now or at the end and

[1:01:21] let's do it. let's just do it

[1:01:23] now. we're good. and then you

[1:01:25] come right back just hold that

[1:01:27] thought Miss mac OK thank you I

[1:01:28] just I just want to pick

[1:01:29] something up although coreyii

[1:01:31] has the headquarters in Orlando

[1:01:35] how in Florida but the research

[1:01:37] and the development which is you

[1:01:39] know when we're working with

[1:01:41] government entities and that the

[1:01:42] the security of where the actual

[1:01:43] data is coming out of I don't

[1:01:44] care what they have on officeice

[1:01:45] or they

[1:01:47] print their paper whatever that

[1:01:48] is I care about where the where

[1:01:49] the research and development

[1:01:51] where were the core of the the

[1:01:53] crux of the software's coming

[1:01:55] from and that is out of India. I

[1:01:56] just want to put that out there

[1:01:57] because I don't want us to think

[1:02:00] this is a US they have a base in

[1:02:03] in in in in in the United States

[1:02:04] but the but the research and

[1:02:07] development of of the actual

[1:02:09] software comes out of India so I

[1:02:11] just want to make that clear to

[1:02:12] everyone when they're going

[1:02:13] forward and and looking at the

[1:02:14] security and where our data you

[1:02:14] know

[1:02:16] possibly could be going and

[1:02:18] they're expanding that area of

[1:02:20] their company in India. I just

[1:02:20] want to put that out there

[1:02:23] yeah you know and I appreciate

[1:02:25] it and I would say that data

[1:02:26] ownership

[1:02:29] is a concern all around and I

[1:02:31] think that what your statement

[1:02:34] is is a reflection of that as

[1:02:36] well. So I do appreciate you

[1:02:37] mentioning that. I do see the

[1:02:39] director of hip puff nodding his

[1:02:43] head so noted thank you rep

[1:02:44] Kelty thank you thank you

[1:02:46] Miss Mac.

[1:02:53] Thank you madam Chair. The

[1:02:55] second use case is a tool that's

[1:02:57] called panagrame Are Natural

[1:02:59] Dialogue Assistant. this is

[1:03:01] currently state facing work this

[1:03:03] is only is being used by state

[1:03:05] workers currently so when

[1:03:08] somebody has a question about

[1:03:10] policy wants to know how they

[1:03:12] should if they have a question

[1:03:13] they want to know a citation

[1:03:15] they can use this panda chatbot

[1:03:17] instead of needing to look

[1:03:19] through hundreds thousands of

[1:03:20] pages of documents and try to

[1:03:20] sift through and find the

[1:03:21] appropriate answer

[1:03:24] it provides answers in plain

[1:03:26] text along with the citation

[1:03:29] to where it is and it is

[1:03:29] specifically trained on

[1:03:31] documents that are uploaded by

[1:03:34] CdHS programme area subject

[1:03:35] matter experts so a specific set

[1:03:37] of documents the question can

[1:03:39] be typed in they get the answer,

[1:03:40] they get the citation

[1:03:43] the

[1:03:45] this is currently in a pilot

[1:03:47] phase so as I mentioned only a

[1:03:49] handful of state program staff

[1:03:50] have access to this this is so

[1:03:51] that they can test the kind of

[1:03:53] answers that come back before

[1:03:55] releasing it to county workers

[1:03:57] again they're given those

[1:03:57] references so that they can

[1:03:59] verify the accuracy. there's

[1:04:01] also a feedback mechanism for

[1:04:02] them to report if the response

[1:04:05] was unsatisfactory and they

[1:04:07] can only access those documents

[1:04:08] that have been specifically

[1:04:10] approved by CDHS for it to use

[1:04:10] and so if you ask a

[1:04:14] question like what is your

[1:04:15] favorite ice cream? it will not

[1:04:17] return an answer. it will say I

[1:04:18] cannot provide that I cannot

[1:04:19] answer that question. so it is

[1:04:21] very limited in terms of the

[1:04:23] questions it is willing and able

[1:04:25] to answer and currently that

[1:04:26] testing in the pilot phase has

[1:04:27] been rather limited just

[1:04:29] because there are so many other

[1:04:30] priorities when this was first

[1:04:31] implemented a whole lot of

[1:04:33] things happened right afterwards

[1:04:35] and so making this be

[1:04:36] exactly what it needs to be

[1:04:37] before we turn it on for other

[1:04:38] workers hasn't been the first

[1:04:40] priority and now as we're

[1:04:40] looking at replacing CBmS we are

[1:04:41] looking at

[1:04:43] implementing this a bit more

[1:04:45] holistically. and looking at

[1:04:46] making sure that we have the

[1:04:47] right tools and and the right

[1:04:49] use cases because it's not just

[1:04:50] the county workers who need this

[1:04:51] but potentially call center

[1:04:53] agents and other folks and so we

[1:04:54] want to make sure we have the

[1:04:55] right tool in place to support

[1:04:56] all folks and that we have the

[1:04:58] right plan in place to pilot and

[1:04:59] get the the testing data that we

[1:05:00] need before rolling it out more

[1:05:01] widely.

[1:05:03] and really nice work on that

[1:05:07] acronym too it's really cute but

[1:05:10] oh gosh I would say this is

[1:05:14] this is so cool and I really

[1:05:17] see this as something that

[1:05:18] would be really helpful for

[1:05:21] every agency I bet and so I'm

[1:05:23] wondering if we can't do this

[1:05:25] really well in hick puff and

[1:05:29] then or CDHS both that's right

[1:05:30] it's a combined and

[1:05:33] and then see if we can't

[1:05:35] implement this elsewhere. I mean

[1:05:37] gosh legislators would even

[1:05:39] enjoy something like that but ok

[1:05:41] I see senator Ball has his

[1:05:43] hand up. I think no I'm sorry

[1:05:45] rep Kelty I can't see things

[1:05:46] apologiesrep Kelty

[1:05:49] oh no I just forgot to to take

[1:05:51] my hand out it's an old hand

[1:05:55] it's an old hand OKok no worries

[1:05:56] no worries

[1:06:01] yeah rett escal. thank you madam

[1:06:02] chairir. this is I guess more

[1:06:04] of a process question for the

[1:06:07] board committee it I am

[1:06:09] very interested in finding out

[1:06:12] the readiness ofcBMS for work

[1:06:14] requirements starting in

[1:06:18] January2027 and I'm wondering do

[1:06:19] we have that on the agenda

[1:06:20] somewhere in the future

[1:06:25] Miss mac do you want I know we

[1:06:27] don't have it on the agenda yet

[1:06:29] but I know it will be coming. so

[1:06:31] do you have any idea when we

[1:06:34] will be doing that is there I

[1:06:36] guess the way this usually goes

[1:06:38] please director make yourself

[1:06:41] make your way to the table.

[1:06:43] usually we get our IT requests

[1:06:45] budget requests if it's

[1:06:47] under500,000 it wouldn't come to

[1:06:49] us I'm thinking I don't know if

[1:06:50] the work requirements are going

[1:06:52] to require an IT project but

[1:06:57] yeah so at some point it would

[1:06:59] be yeah it would be good to get

[1:07:01] an update on that project even

[1:07:03] if it's not a new or a

[1:07:06] continuing appropriated ask. so

[1:07:07] chair marksman I would tell you

[1:07:09] that this is already scheduled

[1:07:10] in the next meeting or two at

[1:07:12] the Medicaid Commission. So

[1:07:13] there'll be an overview of

[1:07:15] what's going on with work

[1:07:16] requirements there certainly

[1:07:17] that may likely generate

[1:07:21] inquiries from the team here.

[1:07:22] this is applying mostly existing

[1:07:26] technologies so it's more about

[1:07:27] them looking at the policy and

[1:07:28] the implementation

[1:07:31] as I had opportunity to talk to

[1:07:32] the commission

[1:07:35] what was the commission lastst

[1:07:36] week

[1:07:37] you know one of the things that

[1:07:38] we're making sure that we're

[1:07:39] doing is we're continuing to

[1:07:40] keep that North Star up how do

[1:07:41] we keep everyone

[1:07:43] is entitled to be covered

[1:07:45] covered starting with reminding

[1:07:47] people that there is an appeals

[1:07:48] process that retains their

[1:07:50] benefits during any time where

[1:07:51] they believe that they have not

[1:07:51] been

[1:07:53] treated appropriately by changes

[1:07:55] in process changes in technology

[1:07:57] or other things like that so

[1:07:58] we're we're making sure that

[1:07:59] we're very clear in that kind of

[1:08:01] safety net that says we can't

[1:08:02] just take it away from you and

[1:08:05] there you are. OK. that sounds

[1:08:09] good. while you're here mean

[1:08:11] I so and I'm also assuming we

[1:08:12] will get an update with CBms

[1:08:13] just our quarterly update we

[1:08:14] usually get that so

[1:08:17] Miss Mac will you be bringing

[1:08:19] information to us at our

[1:08:20] quarterly update.

[1:08:23] thank you madam chairir. we are

[1:08:24] always happy to update you

[1:08:26] whenever you want. we have

[1:08:27] submitted both in writing and

[1:08:29] then when you ask happy to come

[1:08:31] here before you and speak to it

[1:08:33] as well. OK, very good. we're

[1:08:35] trying to get to a model where

[1:08:37] we're doing more face to face so

[1:08:39] that

[1:08:43] we get so many emails and things

[1:08:45] to read that often we don't get

[1:08:46] to it but if you're sitting

[1:08:49] right in front of us we do so

[1:08:50] thank you for that. We will

[1:08:50] likely be adding

[1:08:53] you to an upcoming quarterly

[1:08:54] update on the CBmS to get that

[1:08:55] question answered

[1:08:59] I was gonna ask you do for

[1:09:01] all of the systems we heard

[1:09:06] about today are our clients are

[1:09:08] members are our folks who are

[1:09:10] using these systems not the ones

[1:09:12] that are just internal but

[1:09:15] the others do they know that Ai

[1:09:17] is being used iss there some

[1:09:19] sort of a message so that they

[1:09:21] know that Ai is being used

[1:09:24] core Ai or some other public

[1:09:25] fa

[1:09:26] cing Ai

[1:09:30] so Cha thanks for your thank you

[1:09:31] for your question. my

[1:09:32] recollection is that we have not

[1:09:37] predisclosed to laution here if

[1:09:39] you go beyond here you will be

[1:09:40] interacting with artificial

[1:09:41] intelligence we know that

[1:09:43] there's a number of these that

[1:09:47] are are are general questions

[1:09:48] that are being managed. I know

[1:09:51] with the member direct with the

[1:09:54] member interactions there are

[1:09:55] questions that they are

[1:09:57] specifically told like around

[1:09:59] PhI that they should not put in

[1:10:00] because of those things

[1:10:02] Antoinette you disclosure on

[1:10:02] yours

[1:10:05] toronto yes thank you madam

[1:10:09] chairir yes every time they

[1:10:11] open up the chatbot we remind

[1:10:13] them so if they open and close

[1:10:15] it and open it again we remind

[1:10:17] them that they are using Ai and

[1:10:21] they consent to it and we

[1:10:23] also remind them to not put

[1:10:25] protected personal information

[1:10:26] in it.

[1:10:33] very good. right. oh yes one

[1:10:34] other piece I did not share

[1:10:37] earlier we did some extensive

[1:10:41] member testing with probably

[1:10:44] 20 to 30 members prior to launch

[1:10:44] as well.

[1:10:49] on der ful Well good luck and we

[1:10:51] will see you for our next update

[1:10:52] and we'll just be in touch when

[1:10:54] that will be so very good. thank

[1:10:55] you

[1:11:09] do that. OK, I would like to

[1:11:12] welcome our next step which is

[1:11:13] CDLE

[1:11:50] OK.

[1:11:53] I believe that we have deputy

[1:11:55] executive director and chief

[1:11:59] communications officer Havend

[1:12:02] and then we have COO Helen

[1:12:03] Thurman.

[1:12:05] Wonder who is going to kick us

[1:12:08] off? thank you madam chairir.

[1:12:09] I can get us started this

[1:12:11] morning again Chehavin deputy

[1:12:13] executive director for the

[1:12:14] department of laborbor and

[1:12:16] Employment. today we will be

[1:12:17] presenting our experience

[1:12:21] utilizing Ai with some of our

[1:12:24] external applications in the

[1:12:26] really in the job seeker and

[1:12:27] unemployment insurance claimant

[1:12:30] space but we also do have an

[1:12:33] update on a new project Cocomp

[1:12:35] which is replacing the division

[1:12:38] of workers' compensations legacy

[1:12:39] system and moving it into a

[1:12:44] modernized environment and

[1:12:45] so with that I am going to turn

[1:12:47] it over to Miss Thurmond to talk

[1:12:48] about familymi shield.

[1:12:51] very good M Thurman

[1:12:53] you need to hit that button and

[1:12:55] it should turnres up better

[1:12:56] that's it you

[1:12:59] my name is Helen Thurman. I'm

[1:13:00] the chief operations officer. I

[1:13:02] am also responsible for the

[1:13:03] technology strategy and

[1:13:05] innovation for the department.

[1:13:07] I'd like to start off today with

[1:13:10] the family shield so this is a

[1:13:11] product that we put in place

[1:13:14] back in 2024 when the family

[1:13:17] division was first getting going

[1:13:20] and we are receiving our claims

[1:13:21] and and

[1:13:23] employees are seeking their

[1:13:25] benefits for the first time and

[1:13:28] so we needed a way to detect

[1:13:32] fraudulent claims. so this

[1:13:35] particular product today has

[1:13:38] saved our claims investigators

[1:13:41] from an hour of work that they

[1:13:42] would have spent looking at each

[1:13:45] claim documentation down to 1f

[1:13:46] minutes and what the

[1:13:49] what this technology does is the

[1:13:51] AI that is being used here is

[1:13:53] basically looking at the data

[1:13:55] it's classifying the data as

[1:13:57] whether it's an employee seeking

[1:14:01] benefit or an employer data or a

[1:14:03] healthcare provider data so it

[1:14:05] classifies information. it also

[1:14:08] is looking at anomalies. are we

[1:14:11] seeing odd behaviors are we

[1:14:13] seeing duplicate use of emails

[1:14:14] that we have seen before for

[1:14:15] another claim and so this

[1:14:19] is a way for our human

[1:14:21] investigators to save time

[1:14:23] looking at the claims that are

[1:14:24] coming through.

[1:14:27] so at this time this is solely

[1:14:30] used for the detection and it

[1:14:34] just flags suspicious claims and

[1:14:37] a human a an actual investigator

[1:14:39] will look at all the flagged

[1:14:41] records to determine whether it

[1:14:42] truly is suspicious or not

[1:14:46] currently as I mentioned

[1:14:49] we are looking at about 1000

[1:14:51] records a month and we are

[1:14:53] saving from an hour of work per

[1:14:55] record down to 1f minutes.

[1:15:03] Very good. so it's an internal

[1:15:05] tool that you use to kind of

[1:15:06] figure out kind of what we just

[1:15:09] heard the other that CBmS and

[1:15:12] hickpuff are using to find that

[1:15:14] kind of fraud waste abuse issues

[1:15:15] and then a person's gonna look

[1:15:18] at it and take that case and

[1:15:19] follow up. it's not that this

[1:15:20] perfect.

[1:15:21] thank you. Reppascal

[1:15:25] thank you madam chairir. So in

[1:15:27] addition to saving time is there

[1:15:31] any change on like how much

[1:15:32] fraud waste fraud and abuse is

[1:15:35] being found. Has that changed?

[1:15:36] Oops Miss Berman

[1:15:41] thank you chair. madam

[1:15:42] chairirs

[1:15:43] thank you for the question

[1:15:45] Representative Pascal. so you're

[1:15:47] asking if there's something else

[1:15:49] that we we're finding from this

[1:15:50] I'm sorry I didn't quite catch

[1:15:53] the rettpascal thank you madam

[1:15:55] chair. So I'm asking is the

[1:15:57] fraud rate found changing due to

[1:15:58] the use of this tool. so we have

[1:16:01] thank you for the question

[1:16:05] p re s ent ative Pascal. so we

[1:16:07] have found about 95% of the

[1:16:08] fraud that's coming through

[1:16:11] initially we probably have paid

[1:16:14] out some fraud but at this point

[1:16:16] we are 95% confident that we're

[1:16:16] catching the fraud.

[1:16:24] rettpascal she's got another

[1:16:26] question how do you come up with

[1:16:27] that confidence level

[1:16:28] though when we

[1:16:30] yes go ahead M Thman

[1:16:33] thank you madam chairir.

[1:16:35] sopresentative Pascal so

[1:16:36] basically what we do is we

[1:16:39] compare the actual flagged items

[1:16:41] with real fraudulent cases and

[1:16:44] we track the metric of how

[1:16:46] accurate are we? Is it truly a

[1:16:47] fraud case or is it

[1:16:50] something that we was

[1:16:53] detected and erroneously by the

[1:16:54] the

[1:16:54] the algorithm

[1:16:59] OK so 95% of what's being

[1:17:01] flagged looks like it's really

[1:17:02] fraud. Is that where the number

[1:17:03] comes fromermon

[1:17:07] thank you madam Chair. that's

[1:17:08] correct. OK, thank you.

[1:17:11] good questions. Great. thank you

[1:17:12] for that.

[1:17:15] what's next

[1:17:19] So the next product that we are

[1:17:21] going to share with you about is

[1:17:22] an interviewer

[1:17:25] eightfold interviewer that we

[1:17:27] use on the connecting Colorado

[1:17:30] platform you may already know

[1:17:31] that the connecting Colorado is

[1:17:34] our labor exchange platform and

[1:17:37] we rolled that out last year and

[1:17:39] we have seen great success using

[1:17:42] AI in terms of how eightfold the

[1:17:45] vendor is allowing the job

[1:17:49] seeker to find or locate jobs

[1:17:50] that seem to match their

[1:17:52] resumes. They had this other

[1:17:52] technology and interviewer

[1:17:55] basically it's a practice

[1:17:56] platform so

[1:17:59] a job seeker has a choice

[1:18:01] they're they're not required to

[1:18:03] use this but basically this is a

[1:18:05] tool for them to practice

[1:18:07] interviewing based on the job

[1:18:08] that they're looking at, it will

[1:18:13] be rulespecific mock interview

[1:18:16] format where the AI acts as an

[1:18:18] interviewer and will ask

[1:18:19] appropriate questions that

[1:18:23] pertain to that job and so the

[1:18:25] job seeker is able to now

[1:18:27] practice what that interview

[1:18:29] experience would be like and

[1:18:30] given that out in the industry

[1:18:31] more and more companies are

[1:18:35] actually using AI as a tool for

[1:18:37] interviewing process for

[1:18:38] screening process. this is a

[1:18:39] great tool for

[1:18:41] a job seeker to get practice,

[1:18:43] get comfortable in answering

[1:18:46] questions and interacting with a

[1:18:49] tool. So basically it

[1:18:51] summarizes for the interviewer

[1:18:55] how you did and and allows them

[1:18:57] to practice again and again and

[1:18:58] allows them to focus on the

[1:18:59] areas where they were weak in

[1:19:01] responding to specific

[1:19:03] questions. Again this is

[1:19:05] entirely optional the job seeker

[1:19:07] does not have to use this tool

[1:19:08] it is just an option that is

[1:19:12] we are making available to them

[1:19:14] as a pilot. Currently we are

[1:19:16] scheduled to go live this Friday

[1:19:19] and this pilot will run for six

[1:19:23] months or so we will end by June

[1:19:26] of 2027 and up to 10,000

[1:19:28] interviews will be allowed and

[1:19:31] after that point we will collect

[1:19:33] feedback from the users or

[1:19:34] rather we will be collecting

[1:19:35] feedback throughout the whole

[1:19:37] process but at that point we

[1:19:38] will assess all the feedback to

[1:19:39] determine whether this is

[1:19:41] an effective tool whether it's

[1:19:43] something that our job seekers

[1:19:44] are finding beneficial.

[1:19:49] very good. thank you.

[1:19:51] what

[1:19:56] is what is available now to job

[1:20:00] seekers versus this new tool.

[1:20:01] can you tell me what is

[1:20:03] available at this time

[1:20:04] because I know people are

[1:20:07] getting pegged to particular

[1:20:11] jobs but I'm just curious how

[1:20:13] this fits into what's already

[1:20:15] there. if you could explain that

[1:20:16] please. thank you madam Chair.

[1:20:16] so

[1:20:20] basically right now what we have

[1:20:23] is a labor exchange platform

[1:20:25] where a job seeker can look for

[1:20:27] a particular job based on their

[1:20:29] skills and what they're

[1:20:31] interested in and the platform

[1:20:33] will call up the list of jobs

[1:20:35] that are posted available from

[1:20:38] variety of employers. they can

[1:20:40] also go to our workforce centers

[1:20:42] and get actual training if they

[1:20:45] would like to do training

[1:20:46] with a person. so that is an

[1:20:46] available

[1:20:50] option for them but they would

[1:20:51] have to actually call make an

[1:20:55] appointment and interact with

[1:20:57] somebody at an actual workforce

[1:20:59] center. The other things that

[1:21:00] are available are approved

[1:21:03] providers of training they can

[1:21:05] do online training via that

[1:21:07] option so if they have a

[1:21:09] specific topic that they feel

[1:21:10] they would like to upskill on or

[1:21:14] basically refresh their skill

[1:21:15] set they can do that kind of

[1:21:17] training as well online or in

[1:21:19] person depending on what the

[1:21:22] trainer is providing but this is

[1:21:25] entirely new in that you can

[1:21:26] actually do it whenever you want

[1:21:29] at your home and and do it over

[1:21:31] and over again and practice with

[1:21:32] an Ai tool.

[1:21:34] that's fantastic. I am just

[1:21:35] going to put a little pin in

[1:21:37] this and we will have to talk

[1:21:39] about this when CDLE comes

[1:21:40] before us but

[1:21:43] I'm curious about the vetting

[1:21:45] that we do for the for the

[1:21:48] employers who are a part of our

[1:21:49] system. I've gotten a number of

[1:21:51] complaints that CDLE is actually

[1:21:54] pointing people to like the geo

[1:21:57] prison and the ice facility in

[1:22:00] Hudson as well as instead of

[1:22:02] posting local jobs they're

[1:22:04] seeing a lot of like Amazon and

[1:22:06] you know like really big jobs

[1:22:07] that is not really related to

[1:22:09] this. I wanted to put that on

[1:22:10] your radar

[1:22:13] so that when you come back we

[1:22:14] can address that and I'll make a

[1:22:18] note of it as well. but I

[1:22:21] think that exhausts my questions

[1:22:23] about the Ai interviewer do you

[1:22:26] have any OK very good. thank you

[1:22:27] so much. right

[1:22:29] what's next

[1:22:33] madam chairir we're gonna

[1:22:35] move into our unemployment

[1:22:37] insurance virtual agent solution

[1:22:41] so in the division of Ui we

[1:22:43] although we have pretty steady

[1:22:46] continued claims and initial

[1:22:48] claims we always obviously have

[1:22:49] a claimant population who wants

[1:22:51] more detailed information on

[1:22:54] their claim status. a couple

[1:22:57] of years ago we implemented a

[1:23:00] solution that provides a chatbot

[1:23:00] function

[1:23:05] or more recently an

[1:23:06] interactive voice response

[1:23:09] system when authenticated gives

[1:23:11] the claimant more detailed claim

[1:23:13] information like last payment

[1:23:17] date claim status appeals

[1:23:19] issues not the full breadth of

[1:23:21] claim status but more detailed

[1:23:23] than if they were to just

[1:23:25] utilize the chatbot on its own

[1:23:29] and so there's really two

[1:23:30] systems working together

[1:23:30] currently 3

[1:23:35] one is the the chatbot that does

[1:23:36] not provide detailed claim

[1:23:38] information. it's a

[1:23:41] conversational gen ai solution

[1:23:43] the other is the phone system

[1:23:45] that when authenticated does

[1:23:47] provide more detail and that

[1:23:50] detail is delivered via an API

[1:23:54] on the Ui claims application

[1:23:57] within the system. When a

[1:23:59] claimant interacts with

[1:24:00] theivVR system.

[1:24:03] they are asked for their

[1:24:07] claimant ID their date of

[1:24:11] birth and no social security

[1:24:13] number is exchanged it then

[1:24:15] authenticates with again the

[1:24:18] main mauai plus system to

[1:24:21] pull up that detailed claim

[1:24:23] information. Again the chatbot

[1:24:25] via the website does not allow

[1:24:27] that more detailed

[1:24:30] information delivery. so we

[1:24:30] do

[1:24:33] believe that since implementing

[1:24:35] these tools we've been able to

[1:24:37] provide greater selfservice

[1:24:39] to claimants so when they are

[1:24:43] able to authenticate about50% of

[1:24:45] claimants are doing so. so it

[1:24:48] has reduced dependency on our

[1:24:49] actual staff in the call center

[1:24:53] we are receiving on average

[1:24:55] several 1000 calls into the call

[1:24:57] center a day because of staffing

[1:24:58] we're not able to get to those

[1:25:00] and so this has allowed us to

[1:25:00] provide that service

[1:25:03] to those claimants who

[1:25:05] otherwise were not able to speak

[1:25:06] with an agent.

[1:25:10] we plan on and we'll get to

[1:25:12] this in the next slide.

[1:25:15] expanding this you may have

[1:25:16] seen in the materials previously

[1:25:18] submitted that there were three

[1:25:21] vendors we actually have since

[1:25:23] we submitted this documentation

[1:25:25] terminated the relationship

[1:25:27] with that vendor and I'll get

[1:25:29] into some of the challenges that

[1:25:31] were presented and where we hope

[1:25:32] to go from here

[1:25:35] but I'll pause there about

[1:25:36] current state

[1:25:39] Oh doctorctorhaen can you

[1:25:43] clarify I see the unemployment

[1:25:45] insurance has google is a

[1:25:47] developer bought copy and wao

[1:25:51] twillo twillolio is that what

[1:25:52] you're talking about one of

[1:25:53] those is gone. All three of

[1:25:55] those are gone. all three of

[1:25:56] those are gone. OK will you go

[1:25:57] ahead and explain or are you

[1:25:59] getting into it later we'll go

[1:26:01] to repa to get into it now after

[1:26:04] represent let's do thathettscal

[1:26:05] thank you madam chairir. I just

[1:26:06] wanted to follow

[1:26:09] up on what you said about this

[1:26:11] getting deeper information on

[1:26:13] the claim is a phone system is

[1:26:14] that like a you know push eight

[1:26:17] to find out this push 9 that

[1:26:18] kind of system madam

[1:26:21] chairirrectorhavan yes it is

[1:26:24] oh it is uhivVR but when

[1:26:25] authenticated with a CId or a

[1:26:28] claimant ID can provide a more

[1:26:28] detailed claim information.

[1:26:31] and then do they have a way to

[1:26:32] exit out of

[1:26:35] that if they need to actually

[1:26:37] talk to a human. Yes madam

[1:26:39] chairir they do and and in

[1:26:42] both the chatbot through the

[1:26:43] website and at the end of

[1:26:45] theivVR they do have an

[1:26:46] opportunity either online to

[1:26:48] thumbs up or thumbs down the

[1:26:51] response was it helpful or not

[1:26:53] and likewise at the end of the

[1:26:55] voice system have the

[1:26:56] opportunity to provide feedback

[1:26:59] or speak to an agent. OK, great.

[1:27:00] Why don't you move into our

[1:27:01] vendor conversation. OK thank

[1:27:02] you madam chairir.

[1:27:05] so they said when we were

[1:27:07] preparing for this presentation

[1:27:10] we were currently engaged

[1:27:13] with a vendor who had multiple

[1:27:16] sub vendors and that

[1:27:19] relationship provided

[1:27:21] while it helped us move toward

[1:27:23] the goal of a more seamless

[1:27:25] system it actually presented

[1:27:27] more challenges and so we

[1:27:29] mutually terminated that

[1:27:30] agreement with the vendor within

[1:27:31] the last several weeks and are

[1:27:32] using that

[1:27:35] experience to take a step back

[1:27:38] and our RFP we do believe

[1:27:41] that a single vendor solution is

[1:27:42] preferred I will just say

[1:27:45] that even in that scenario

[1:27:48] there was no exchange of

[1:27:50] between the three vendors in

[1:27:53] EPIi so all of that again

[1:27:55] through the API interface was

[1:27:57] pulling from the main Ui claims

[1:28:00] system that a claimant interacts

[1:28:01] with that their data is stored

[1:28:02] within but as

[1:28:05] I said we were finding some

[1:28:08] challenges in a more seamless

[1:28:09] integration and development with

[1:28:10] multiple vendors.

[1:28:17] we we still are looking for

[1:28:19] 11 solutions that that does

[1:28:22] integrate the chatbot and the

[1:28:25] phone system so that an agent

[1:28:29] has a kind of an end to end line

[1:28:31] of sight into the claim status,

[1:28:32] the interactions with the

[1:28:33] claimant, the full claim history

[1:28:37] and so that is the the ideal

[1:28:39] next step future state for

[1:28:40] the solution will we will be

[1:28:40] exploring next

[1:28:43] makes sense

[1:28:47] OK, very good. Next

[1:28:54] think it's back to me fors some

[1:28:57] discontinued or yeah so we're

[1:28:58] actually going to talk a little

[1:29:00] bit about workers' comp and the

[1:29:02] first product that was on our

[1:29:05] list was thisscovery navigator

[1:29:09] which we were planning to do and

[1:29:10] discovered in the process that

[1:29:11] it wasn't meeting our

[1:29:13] expectations. so this team the

[1:29:16] dying team it's the division

[1:29:18] ofend medical examination team.

[1:29:20] they normally handle about 3000

[1:29:20] medical pack

[1:29:23] et s a year and handling that

[1:29:27] many documents per case which

[1:29:28] often ends up being hundreds of

[1:29:33] pages of records humans were

[1:29:35] basically having to do duplicate

[1:29:37] the records and it was taking

[1:29:39] too much time so the intent here

[1:29:41] was to use technology to help us

[1:29:43] to classify the documents to

[1:29:45] group them and to duplicate them

[1:29:46] and make sure that they were

[1:29:47] actually a

[1:29:51] complete set of documents per

[1:29:53] case. as I mentioned this

[1:29:55] particular product did not meet

[1:29:57] our expectations and so we have

[1:29:59] decided to end this and we are

[1:30:01] pursuing or looking at a

[1:30:05] different solution and that is

[1:30:06] that next slide

[1:30:09] so we're calling it the medical

[1:30:12] packet consolidation and this is

[1:30:15] something that is currently in

[1:30:17] our on our roadmap but has not

[1:30:19] been scheduled yet because the

[1:30:21] codecom platform, this is a

[1:30:24] modernization of an old legacy

[1:30:25] platform that the workers' comp

[1:30:28] has used and that system just

[1:30:30] went live in July with its first

[1:30:33] release. there are three other

[1:30:34] releases planned core

[1:30:36] functionality that is deemed

[1:30:36] higher prior

[1:30:39] ity for that division and so at

[1:30:42] this time the medical packet

[1:30:42] consolidation

[1:30:46] work is slated for at best end

[1:30:48] of next year if we get to it at

[1:30:51] that point. again it would

[1:30:53] not replace a human actually

[1:30:55] reviewing the packets that have

[1:30:58] been put together by Ai it would

[1:30:59] just look at the flags that are

[1:31:01] potentially duplicates of

[1:31:03] records and so it would save a

[1:31:05] lot of time for the the

[1:31:06] specialists in the dying team.

[1:31:13] are there any questions about

[1:31:14] this particular technology

[1:31:19] I just want to make sure I'm

[1:31:21] looking at the right one. I was

[1:31:22] looking at

[1:31:23] Discoverynavigatorplacement and

[1:31:26] chatbot Cocom on the Ai it's on

[1:31:27] the Ai inventory spreadsheet

[1:31:29] which you may not have in front

[1:31:31] of you. So I I can go ahead and

[1:31:33] start talking about the chapa

[1:31:35] too and maybe we can handle all

[1:31:37] three together that sounds good.

[1:31:38] Great thank you

[1:31:41] madam chairir. so the chatbot is

[1:31:42] wait one second please repasal

[1:31:45] oh I apologize

[1:31:50] yeah who's oh it's rep to tone

[1:31:53] and then repelty rep to tone

[1:31:54] vice chair to tone thank you

[1:31:59] thank you madam chair. So I

[1:32:02] see that your using claud in

[1:32:02] this application

[1:32:07] how do you make sure that you're

[1:32:08] not

[1:32:13] having any of the information be

[1:32:16] used for training or be saved

[1:32:18] in some way that could

[1:32:20] compromise somebody's

[1:32:20] information that could be

[1:32:21] protected

[1:32:25] M Thurmon thank you madam Chair.

[1:32:27] thank you for the question

[1:32:29] Representative Taonene. So

[1:32:31] basically all these documents

[1:32:34] are in a closed environment and

[1:32:35] these are only the documents

[1:32:38] that have been submitted by the

[1:32:40] actual claimants and so there is

[1:32:44] no possibility of injecting any

[1:32:47] kind of vulnerabilities or

[1:32:49] prompts I think maybe

[1:32:50] potentially you're thinking of

[1:32:52] so it's only looking at what

[1:32:56] is already within our database

[1:32:57] of the documents that have been

[1:32:58] submitted

[1:33:03] I shared to

[1:33:05] tone go ahead thank you man and

[1:33:07] I my question really wasn't

[1:33:10] answered what I was asking was

[1:33:10] about

[1:33:16] laude being used as a part of

[1:33:16] that back end

[1:33:19] or oh so that was the other

[1:33:21] one that was on the other side

[1:33:23] that was the others because you

[1:33:24] changed the slide back because

[1:33:25] you were on the other side said

[1:33:26] lauded

[1:33:29] and aws and that was the one I

[1:33:31] was asking the question about. I

[1:33:33] saw that too. where was that

[1:33:35] clad was it's the next slide

[1:33:38] slide

[1:33:40] yeah

[1:33:42] yeah AWsex tract with claude

[1:33:47] access through amazon bedrock

[1:33:49] was the question was asking

[1:33:51] yeah that was what I was asking

[1:33:54] is if it's using laude is it is

[1:33:55] that data being used for

[1:33:57] training for claude and is any

[1:33:59] of the information being

[1:33:59] protected and how do you ensure

[1:34:01] that it's being protected if

[1:34:02] it's personally identified

[1:34:02] information.

[1:34:05] gottcha. I think that I think we

[1:34:07] get that question. Who would

[1:34:08] like to feel that Miss Thurman?

[1:34:11] thank you madam chairir.

[1:34:13] thank you for that question

[1:34:14] Representative Taonene. May I

[1:34:16] get back to you with what our

[1:34:17] current processes for ensuring

[1:34:18] that the

[1:34:22] data is not being used for

[1:34:23] cloud. I'm pretty sure it's not

[1:34:25] but I'd like to come back to you

[1:34:26] with a definitive answer.

[1:34:29] we would appreciate that is

[1:34:31] there any way that you could do

[1:34:33] that within the next like5

[1:34:34] business days. Absolutely,

[1:34:36] yesonder. and if you'll just put

[1:34:39] it in writing to our JTC staff

[1:34:41] JTC staff can send it out to all

[1:34:44] of us if that's OK. awesome.

[1:34:44] thank you so much.

[1:34:49] senator I'm sorry rebelunty

[1:34:53] makeman and and actually

[1:34:56] rep to tone asked the the main

[1:34:57] question that I had for that and

[1:34:59] I I have to apologize I'm I'm

[1:35:02] doing this via my my cellphone

[1:35:05] so I can't really see the the

[1:35:07] room other and it's the size of

[1:35:09] a dime so I I don't know who's

[1:35:10] at the table who's not at the

[1:35:12] table but I did have a question

[1:35:13] about the fraud detection. I

[1:35:14] don't know if that person is

[1:35:15] still there or not. I can't

[1:35:16] tell. So

[1:35:20] but the fraud detection you know

[1:35:20] there was a

[1:35:23] program in here about the fraud

[1:35:25] detection. it was the Ai form

[1:35:27] data the family shield yeah

[1:35:31] we've got Miss Thurman here

[1:35:32] who can respond to that

[1:35:34] OK I probably like cause I can't

[1:35:35] see a thing so I'm kind of going

[1:35:37] blind here butries I I'm very

[1:35:39] much in debt you know the IT and

[1:35:40] thet security and stuff like

[1:35:41] that and that's just been kind

[1:35:43] of rattling around in my brain

[1:35:45] right now but you know as we

[1:35:46] know that the fraud detection

[1:35:49] really is a continuous loop, you

[1:35:50] know what I mean rather than

[1:35:52] like a finished product finished

[1:35:53] long r them so and theyment the

[1:35:55] logarithm that they're utilizing

[1:35:58] and I just I wanna know what

[1:36:01] that what does that looking for

[1:36:02] currently right now and what

[1:36:05] is your, your change rate as far

[1:36:07] as keeping up with the the the

[1:36:09] hackers or the anti detection

[1:36:11] the anti anti detectors you know

[1:36:12] what I mean like the ones who

[1:36:13] are trying to constantly go

[1:36:14] around it. what is the the

[1:36:15] change rate that you have for

[1:36:21] that and because with the

[1:36:22] hackers attackers were you know

[1:36:23] whatever you want to call them

[1:36:24] you know they're we're trying to

[1:36:25] keep them from exploiting the

[1:36:27] data right to to get around

[1:36:29] through you know either

[1:36:31] adversarial adaption or

[1:36:32] otherwise you know they don't

[1:36:33] call it

[1:36:35] what model model model

[1:36:35] invasion I believe model

[1:36:37] invasion attacks so what are we

[1:36:39] doing to prevent that and how

[1:36:41] often is that longer than being

[1:36:44] changed to prevent them from

[1:36:44] getting in and getting the data.

[1:36:48] Miss Thurmon thank you madam

[1:36:49] chairir

[1:36:53] thank youpresentative Pascal

[1:36:57] was it is yes no worries so I

[1:36:59] can tell you right that at this

[1:37:03] point we are using 166 active

[1:37:06] indicators to determine whether

[1:37:09] a claim is fraudulent or not and

[1:37:11] of that a lot of it is

[1:37:13] statistical analysis and we have

[1:37:15] certain ones that we absolutely

[1:37:17] look for like duplicate email

[1:37:19] addresses or whether the bank

[1:37:20] account information has is on a

[1:37:20] sanction

[1:37:24] list for AFAFacC and and we look

[1:37:25] to see whether the employer's

[1:37:28] FEIN number is identical to the

[1:37:29] claimant's social security

[1:37:31] number but your question is how

[1:37:34] do we keep up with the changes

[1:37:35] that are occurring in the

[1:37:37] cybersecurity space and for that

[1:37:40] particular rate of change, I

[1:37:41] would like to get back to you

[1:37:43] with the after consulting

[1:37:45] with the team and provide you

[1:37:46] with a fuller response.

[1:37:49] thank you. I appreciate that and

[1:37:50] could you do that within the

[1:37:50] five days?

[1:37:53] OK, I appreciate you thank

[1:37:55] youonder thank you so much and

[1:37:57] yeah that'll be helpful.

[1:38:01] awesome do you want to get into

[1:38:02] the chatbot?

[1:38:07] so this is again for the coca

[1:38:09] platform and the intent here was

[1:38:12] to actually provide our

[1:38:15] claims and and our health

[1:38:17] providers and employers with

[1:38:19] another option for getting

[1:38:21] answers closer to real time

[1:38:24] versus getting on a phone and

[1:38:25] having to talk with somebody

[1:38:27] although that option is still

[1:38:28] available and and I might add

[1:38:31] that this particular division is

[1:38:32] exceptional when it comes to

[1:38:32] responding to

[1:38:35] their phone calls, they get up

[1:38:37] to about 60 calls a day and most

[1:38:39] of the calls are answered within

[1:38:42] 20 seconds or shorter so it's

[1:38:44] not that the folks who are

[1:38:45] calling in are having to wait a

[1:38:47] long time that is not what we're

[1:38:49] trying to solve for here but we

[1:38:53] did want the actual claimant or

[1:38:54] whether it's a provider or

[1:38:55] employer to be able to look up

[1:38:57] the status of their current

[1:39:00] claim or if they needed to find

[1:39:02] another document a way to get to

[1:39:03] that information faster

[1:39:06] and be able to self serve. so

[1:39:08] this chatbot is currently not

[1:39:10] scheduled. it is something that

[1:39:12] is on our roadmap and we would

[1:39:14] like to have this developed

[1:39:17] after all of our core platform

[1:39:19] functionalities are in place and

[1:39:22] at best we're guessing at this

[1:39:23] time that we may be able to get

[1:39:25] to it by the end of 2027.

[1:39:30] Very good

[1:39:33] what's next

[1:39:38] OK I believe I have the next one

[1:39:39] again. this is the snow

[1:39:43] snowflake cortex so this is

[1:39:45] actually for not just one

[1:39:47] division but we are in the

[1:39:50] process of developing a data

[1:39:54] lake for all of CDE and at some

[1:39:55] point in the future we will have

[1:39:57] most of our critical division's

[1:40:00] data on this platform and the

[1:40:02] intent here is to leverage the

[1:40:04] built in AI capability withinn

[1:40:04] snowflake to do the types

[1:40:07] of analytics that we are

[1:40:09] currently not able to do whether

[1:40:12] it's predictive analytics or

[1:40:15] cost modeling future

[1:40:17] modeling that we're not able to

[1:40:19] do today and especially because

[1:40:23] it requires individuals to have

[1:40:24] certain level of skill whether

[1:40:27] it's python or SQl coding we

[1:40:28] wanted

[1:40:31] the ability for folks who are

[1:40:33] I will use the word junior

[1:40:36] junior data analysts or

[1:40:38] business intelligence analysts

[1:40:40] to be able to leverage a tool

[1:40:41] where they can use plain

[1:40:44] language to ask the questions in

[1:40:46] in plain language and get the

[1:40:48] kind of analytic output from a

[1:40:50] platform. So that's the intent

[1:40:52] and at this time this is not in

[1:40:55] production but we are very

[1:40:56] interested in exploring what it

[1:40:56] can do for the department.

[1:41:00] no ther one of those really

[1:41:02] exciting put all the information

[1:41:04] together and let's see how we

[1:41:05] can't use this so I

[1:41:09] that's how I like to use Ai too

[1:41:12] so I think this is brilliant.

[1:41:13] questions, concerns

[1:41:17] I don't see any

[1:41:22] what's next? I think that's all

[1:41:24] of the systems yeah I was like

[1:41:26] I've I've got them all crossed

[1:41:29] off on my list so wonderful.

[1:41:31] I took note of a couple of

[1:41:33] unanswered questions. I just

[1:41:34] want to say thank you so much

[1:41:37] for being willing to really

[1:41:39] dig in I think sometimes we ask

[1:41:41] a question and we don't always

[1:41:44] get the detail of answer that

[1:41:45] we want and so I just really

[1:41:47] appreciate you digging in

[1:41:49] looking forward to hearing about

[1:41:50] how you're keeping

[1:41:55] up with the cybersecurity and

[1:41:56] pretty much all the systems but

[1:41:58] in the family but that's pretty

[1:41:59] much all the systems is where

[1:42:03] that came from and then also

[1:42:05] looking at how we keep our data

[1:42:08] from training clad or in fact

[1:42:10] any of the vendor models that we

[1:42:13] are sharing with so

[1:42:16] appreciate that

[1:42:20] and with that we will dismiss

[1:42:22] you guys. thank you so much.

[1:42:23] thank you madam Chair you madam

[1:42:25] members of the committee thank

[1:42:26] you thank you for the

[1:42:27] presentation did a really good

[1:42:28] job.

[1:42:33] we have up next Department of

[1:42:37] Public Safety. I don't know that

[1:42:38] I see our colleagues here

[1:42:47] they are up at 11:15 I would

[1:42:49] say I'll give us all 20 minutes

[1:42:52] but I'd prefer for them to come

[1:42:53] here because we're going to have

[1:42:56] two parts of this one part is

[1:42:57] going to be an executive session

[1:43:01] and one part is not and so

[1:43:04] for that I would we'll we'll

[1:43:05] take a five minute break. why

[1:43:07] don't we come back here at

[1:43:11] is my clock wrong? what time

[1:43:12] is your say?

[1:43:17] it says 10:4410:45. All right

[1:43:21] we'll come back at 10:50 and

[1:43:22] maybe folks will be here.

[1:43:22] thanks.

[1:50:09] OK, we'll get started. we

[1:50:13] have CDPS in front of us today

[1:50:14] they have

[1:50:15] two sections that they are

[1:50:18] going to cover today. One will

[1:50:24] be one will be in public

[1:50:27] that'll be about the ALPR as

[1:50:30] well as the 911 Aicoms coach,

[1:50:32] those two things and then we

[1:50:33] will be going into

[1:50:36] executive session

[1:50:41] for the following two issues

[1:50:43] which are the motorolamotorola

[1:50:47] camera upgrades and life raft

[1:50:48] and we'll talk about that as we

[1:50:48] get there

[1:50:52] So yeah

[1:50:58] Oh and we're also I missed that.

[1:51:00] thank you. We're also going to

[1:51:03] publicly discuss case guards

[1:51:05] so we'll talk about acction

[1:51:06] Oh and we're also I missed that.

[1:51:07] thank you. We're also going to

[1:51:07] publicly discuss case guards

[1:51:08] so we'll talk about acinovors

[1:51:09] and caseguard all togetherther

[1:51:11] and then we'll go and new

[1:51:13] executive session and we'll talk

[1:51:15] about the other two components

[1:51:20] so OK, very good. Let's get

[1:51:20] started.

[1:51:22] do you have everyone you need

[1:51:30] are we gonna start with you?

[1:51:35] you're just doing the slides.

[1:51:37] very good. So we have chiefef

[1:51:38] Matthew Packer

[1:51:40] or no

[1:51:43] this is oh we've got aaron

[1:51:45] Coning up first gott you. thank

[1:51:47] you. Come on up in your erran.

[1:51:48] Very good.

[1:51:54] No, it's on the it's on the desk

[1:51:55] I believe or on the yeah

[1:51:59] now you're green. very good.

[1:52:02] thank you madam chair and good

[1:52:03] morning members of the

[1:52:06] jointchnology Committee. my name

[1:52:06] is aaron Coney.

[1:52:09] and I am the quality director

[1:52:11] for CBI forensic Services and

[1:52:14] today I'm here to present on

[1:52:15] Caseguard software which is a

[1:52:16] document redaction application.

[1:52:18] go to the next slide

[1:52:23] first of all I want to stress

[1:52:25] that we have not yet deployed

[1:52:27] this software but are still in

[1:52:28] the process of evaluation

[1:52:32] and to give you some context on

[1:52:33] why we are exploring this

[1:52:35] technology our laboratory runs a

[1:52:37] rigorous quality assurance

[1:52:39] program to identify investigate

[1:52:42] and remediate errors and

[1:52:44] potential nonconformances we

[1:52:45] currently document these

[1:52:48] workflows electronically and in

[1:52:49] order to improve external

[1:52:52] transparency and accountability

[1:52:53] we intend to publish these

[1:52:55] workflows to our public facing

[1:52:56] website

[1:52:59] However these documents contain

[1:53:01] sensitive information such as

[1:53:04] names, dates of birth, agency

[1:53:05] information and casespecific

[1:53:09] data that falls under personally

[1:53:11] identifiable information or PIi

[1:53:12] and sieges protection

[1:53:14] we are currently evaluating

[1:53:15] Caseguard to see if it can help

[1:53:18] make this reduction process more

[1:53:19] efficient and accurate

[1:53:21] Once again it's important to

[1:53:23] note that we are still in the

[1:53:25] evaluation phase and have not

[1:53:26] yet deployed this application to

[1:53:27] date.

[1:53:30] Furthermore, any automated

[1:53:31] process will be followed by a

[1:53:34] comprehensive human review to

[1:53:37] ensure absolute accuracy before

[1:53:38] any information is published.

[1:53:39] excellent

[1:53:43] so what value does this tool

[1:53:46] offer? First it pro it provides

[1:53:49] automated first pass redaction

[1:53:51] capable of instantly recognizing

[1:53:53] over 30 categories of PIi and

[1:53:56] other sensitive information and

[1:53:58] can also recognize complex

[1:54:00] patterns like lab and agency

[1:54:02] case numbers. We think this can

[1:54:03] reduce the risk of human error

[1:54:05] that can happen during manual

[1:54:08] scanning and second we think it

[1:54:10] can offer significant

[1:54:10] operational efficiency

[1:54:13] we estimate that this will save

[1:54:14] our team

[1:54:17] up to about 20 hours of labor

[1:54:19] per month while helping us

[1:54:21] process about75 of these

[1:54:22] documents within that time

[1:54:22] frame.

[1:54:26] Finally I want to emphasize that

[1:54:27] this is a nonconsequential tool

[1:54:31] it is purely for operational

[1:54:33] support. it does not make

[1:54:35] autonomous decisions regarding

[1:54:37] public access nor does it affect

[1:54:38] our core operations

[1:54:41] it will simply help us manage

[1:54:42] our workload more effectively.

[1:54:43] Next slide

[1:54:48] and finally I want to address

[1:54:51] security and managements we will

[1:54:53] have a human in the loop review

[1:54:55] which means that we'll

[1:54:55] have a meaningful human

[1:54:57] verification step that's

[1:55:00] required after every automatic

[1:55:01] reaction to s to ensure accuracy

[1:55:05] from a security standpoint case

[1:55:07] guard is a strong fit because it

[1:55:09] runs fully locally. All

[1:55:11] processing happens on our own

[1:55:14] machines no data is transmitted

[1:55:16] to external servers regarding

[1:55:18] management access is strictly

[1:55:20] restricted to the CBI

[1:55:20] forensicervicesuality unit

[1:55:23] the financial commitment is

[1:55:25] minimal it's about $280 per

[1:55:28] month with no long term contract

[1:55:28] required

[1:55:31] and in terms of data life cycle

[1:55:33] unredacted files remain secure

[1:55:35] and our internal CDpS drives

[1:55:38] while only the fully redacted

[1:55:40] versions are moved to our public

[1:55:40] facing website

[1:55:43] and so in summary we hope to

[1:55:45] utilize caseguard to increase

[1:55:47] operational accuracy and

[1:55:48] efficiency and it will not be

[1:55:50] used to make any independent

[1:55:51] decisions on information

[1:55:52] security.

[1:55:57] very good.

[1:55:59] Repkelty has a question I

[1:56:00] saw.

[1:56:05] thank you madam chair. Yeah umm

[1:56:07] apologize I didn't catch your

[1:56:08] your title but I I do have a

[1:56:09] question about this and I think

[1:56:11] this goes across the board with

[1:56:12] anything that we're using as far

[1:56:15] as applications or any type of

[1:56:17] new IT whatever that we're

[1:56:21] using is coming into into the

[1:56:23] realm I guess you could say. so

[1:56:24] as far as you saying this isn't

[1:56:27] an evaluation period for this

[1:56:29] how can you tell me how involved

[1:56:31] with the evaluation of the

[1:56:32] software is our IT

[1:56:35] security or or IT apps

[1:56:37] department when when you're

[1:56:38] going through this evaluation

[1:56:43] that's my question there you go

[1:56:46] and Rep Kelty this is aaron

[1:56:49] Conan quality director of CBI

[1:56:50] Forensic Services Mroning

[1:56:51] thank you

[1:56:53] Yeah thank youre representative

[1:56:55] for the question. yeah that's

[1:56:57] a great question. so our IT

[1:57:00] department is very involved they

[1:57:02] were involved very closely

[1:57:04] with the selection process of

[1:57:06] this software we evaluated

[1:57:09] several different redaction

[1:57:11] software applications and our IT

[1:57:15] department helps us select this

[1:57:17] particular one and so they have

[1:57:19] been very closely involved with

[1:57:21] the selection process and we

[1:57:25] plan to consult with them during

[1:57:26] the evaluation process as well.

[1:57:29] answer I appreciate it. thank

[1:57:30] you.

[1:57:40] OK you said75 documents per

[1:57:42] month. how much time or money

[1:57:45] did you say was saved? you said

[1:57:46] something but I don't remember

[1:57:47] what you said. Do you remember?

[1:57:50] I think it was let me go back to

[1:57:51] my number it was about 20

[1:57:53] hours of labor per month that we

[1:57:55] estimate so you're correct we

[1:57:58] have we estimate about75 of

[1:58:00] these workflows in electronic

[1:58:04] format each month and so we

[1:58:04] estimate about 20 hours within

[1:58:08] the scope of that month on time

[1:58:08] savings

[1:58:11] OK so kind of a higher level

[1:58:13] question that the people at this

[1:58:14] table may not be able to answer

[1:58:16] but I just say cause I know ohIT

[1:58:18] is in the room but

[1:58:21] this would be a good example if

[1:58:23] we have one reduction software

[1:58:25] that's working really well we

[1:58:27] should probably have one that

[1:58:29] works really well for the state

[1:58:31] of Colorado unless there's some

[1:58:34] compelling reason to have75 for

[1:58:37] each75 departments so I just put

[1:58:39] that on the record because this

[1:58:42] is just an example of common

[1:58:44] government work that you have to

[1:58:47] redact documents and so I would

[1:58:48] be very interested and

[1:58:51] in seeing at a higher level,

[1:58:53] not you guys but at a higher

[1:58:55] level how many reduction

[1:58:57] softwares do we have and I'm

[1:58:59] seeing a thumbs up from the back

[1:59:02] so very good. Thank you. Seeing

[1:59:04] no more questions. we thank

[1:59:07] you for that review of Caguard

[1:59:09] and the redaction. It it'll come

[1:59:12] live at some point what are the

[1:59:14] determinations for how you know

[1:59:16] when you're gonna go live I

[1:59:16] guess is a question

[1:59:22] we we we had a position that we

[1:59:23] were going to bring on board

[1:59:26] that one of the duties of

[1:59:28] that this new position was going

[1:59:30] to be to assist with a manual

[1:59:32] review of the redaction process.

[1:59:34] unfortunately we were not

[1:59:35] successful with bringing that

[1:59:38] person on board and so we have

[1:59:39] identified another individual

[1:59:42] with some capacity to do that

[1:59:43] and and so that really that's

[1:59:45] what we were waiting for

[1:59:46] before we redact and put these

[1:59:49] make these publicly available on

[1:59:51] our website and so I would

[1:59:53] anticipate within the next

[1:59:55] several months that we will

[1:59:59] evaluate this and if successful

[2:00:01] we hope to go live with

[2:00:03] making these publicly available

[2:00:04] hopefully early in 27.

[2:00:09] on der thank you so much for

[2:00:10] your presentation

[2:00:14] todayam you. right now we'll

[2:00:16] hear from Chief Packard chief

[2:00:19] thank you very much madam Chair

[2:00:21] and members of the committee

[2:00:23] it's a pleasure to be with you

[2:00:24] on this wonderful Tuesday

[2:00:26] morning. so we'll we'll go

[2:00:27] ahead and start the presentation

[2:00:29] we're as you outlined we'll

[2:00:31] go over the first two systems

[2:00:33] here and happy to take any

[2:00:35] questions and madam Chair I did

[2:00:36] receive some of the questions

[2:00:38] you sent in advance and I'll

[2:00:39] jump into those two when we're

[2:00:40] talking about Axon

[2:00:42] but I just want to start

[2:00:44] high levell and this really does

[2:00:46] apply to all four of the systems

[2:00:48] that we'll talk about today and

[2:00:49] you'll see the slide on the

[2:00:51] screen but overall and going

[2:00:52] through the decision points that

[2:00:54] the committee provided for us to

[2:00:55] address in the presentation

[2:00:59] first off like we don't have any

[2:01:00] computer systems that are making

[2:01:02] actual decisions for our members

[2:01:05] these systems are providing

[2:01:07] inf are prompting members the

[2:01:08] opportunity to make a decision

[2:01:09] based on the technology they are

[2:01:11] there but any enforcement

[2:01:13] related decision or contact

[2:01:15] decision or any of those the

[2:01:17] systems are presenting the

[2:01:19] opportunity in any again any

[2:01:20] consequential decision is

[2:01:21] actually being made by a human

[2:01:22] that works for the Colra State

[2:01:23] patrol in this context

[2:01:26] all of the data is governed by

[2:01:27] strict cigious compliance

[2:01:28] depending on the system whether

[2:01:31] it's through Cis through OIT

[2:01:33] data protocols depending on

[2:01:35] which system that we're talking

[2:01:37] about and where it lives all of

[2:01:38] that is under strict security

[2:01:40] and standard protocol whether

[2:01:40] it's again through the FBI with

[2:01:44] CI or state OIT compliance as

[2:01:46] I already kind of alluded to

[2:01:47] all of the systems that we'll

[2:01:48] talk about have a human in the

[2:01:51] loop the LPRs is a great

[2:01:52] conversation to talk about and

[2:01:52] how that human engagement

[2:01:55] and when I talk about the com

[2:01:56] training you'll understand that

[2:01:57] it's a supervisor that it's

[2:01:58] engaging in those things but

[2:01:59] there will always be that

[2:02:03] human in the loop. talked a

[2:02:04] little bit about security and

[2:02:05] risk as well about through all

[2:02:08] the on the procurement side and

[2:02:09] the first system that we're

[2:02:10] going to talk about is the coms

[2:02:13] coach and that was a long

[2:02:14] onboarding process in fact the

[2:02:16] training component they'll talk

[2:02:17] about we haven't even officially

[2:02:18] started because of all of the

[2:02:20] testing once the vetting

[2:02:22] happened to to on board the

[2:02:22] system so

[2:02:26] doing a lot of work around

[2:02:28] vetting identifying and then

[2:02:31] mitigating risk we have audit

[2:02:33] processes built in sometimes

[2:02:35] that's internal with supervision

[2:02:36] sometimes it's a position

[2:02:38] sometimes it's groups that we

[2:02:39] work with but there's

[2:02:39] independent audits that are

[2:02:40] involved there as well

[2:02:43] and all of our contracts are

[2:02:44] built in a manner that if we

[2:02:45] need to get out we can get out

[2:02:48] I mean all cards on the table

[2:02:50] technology is kind of this place

[2:02:52] whereas you're in and it

[2:02:53] advances and it turns into

[2:02:54] somewhat of a budget decision

[2:02:56] about whether we stay with a

[2:02:58] particular vendor or system or

[2:02:59] try to restart with somebody

[2:03:01] else but the contracts are built

[2:03:04] in in a manner that that we we

[2:03:05] have an out that gets us if we

[2:03:06] need to go in a different

[2:03:08] direction and then all of the

[2:03:09] data that the patrol is putting

[2:03:10] into these systems

[2:03:13] or takes is owned by us and

[2:03:15] we have if we ever choose to and

[2:03:17] I'm not aware of one right now

[2:03:18] where we're actually sending

[2:03:19] that data out so from a data

[2:03:21] security and ownership and life

[2:03:23] cycle we own the data and then

[2:03:24] have the opportunity to make

[2:03:24] decisions on how long we keep

[2:03:25] that data.

[2:03:29] so first off to get more

[2:03:31] specific we'll talk about the

[2:03:32] comms coach by Govorks and again

[2:03:33] this is a system that was on

[2:03:36] boarded specifically number

[2:03:37] one to help with two things

[2:03:39] train our new communications

[2:03:41] officers our dispatchers and

[2:03:42] also to do this quality

[2:03:44] assurance program and so one of

[2:03:46] the challenges particularly

[2:03:48] in in emergency communications

[2:03:49] is recreating an environment

[2:03:51] that's realistic without

[2:03:54] actually talking to callers and

[2:03:55] troopers on the radio and all of

[2:03:56] those things where life could

[2:03:57] be

[2:03:59] at stake and so this software

[2:04:01] allows us to create a training

[2:04:03] environment that is it's really

[2:04:05] quite remarkable what it can do

[2:04:07] realistic voices, realistic

[2:04:09] scenarios emotion all of those

[2:04:11] things that are in the call that

[2:04:13] happened in the real world. this

[2:04:15] system helps us develop and then

[2:04:17] the Ai the intelligence in it

[2:04:19] can then react to how our

[2:04:20] trainee is is you know

[2:04:21] handling that particular call

[2:04:23] for service so pretty compelling

[2:04:25] stuff with that what what we'll

[2:04:26] be able to do with that again we

[2:04:26] haven't actually turned that on

[2:04:27] for training yet

[2:04:29] but we have turned on the

[2:04:31] quality assurance programme

[2:04:33] and this is a normal it's a

[2:04:35] pretty standard practice in

[2:04:36] emergency communications where

[2:04:37] you go back through and have

[2:04:39] some some sort of audit process

[2:04:41] on how call taking is being

[2:04:43] handled and how dispatch calls

[2:04:45] are being dispatched and just

[2:04:47] to give you an idea and we're

[2:04:49] relatively new into using this

[2:04:51] but normally this is a task

[2:04:52] that was assigned to supervision

[2:04:55] in our com centers and on a good

[2:04:56] month we could get about 250

[2:04:57] reviews done a month.

[2:05:00] the first month we turned this

[2:05:02] on we're able to do 13,000

[2:05:04] reviews and so that is going

[2:05:05] through the roop and and

[2:05:06] thanks to the good

[2:05:08] capptainaththle I just got some

[2:05:09] fresh data this morning that

[2:05:12] our we've had high scores like

[2:05:14] we passed with a 96%rcom

[2:05:15] officers so they're doing really

[2:05:17] really incredible work and as it

[2:05:19] turns out and this seems

[2:05:21] predictable the calls where they

[2:05:22] were struggling more were less

[2:05:24] routine calls that are not

[2:05:25] necessarily in the patrol's

[2:05:26] wheelhouse for example, state

[2:05:26] troopers don't do a lot

[2:05:29] of trespassing calls and so our

[2:05:31] com officers might not be

[2:05:31] involved in those much so the

[2:05:33] questions they're asking how

[2:05:34] they're handling although

[2:05:35] they've been trained or not as

[2:05:37] fresh as it would be for example

[2:05:38] as an injury crash or a hit and

[2:05:39] run or a traffic stop or those

[2:05:40] types of things.

[2:05:44] this is this slide up there

[2:05:45] and I'll just kind of let you

[2:05:46] look at this because it's a

[2:05:47] bunch of the more the technical

[2:05:49] aspects of the systems of how

[2:05:51] it's using but again what I

[2:05:54] really want to stress here is as

[2:05:55] we're using this and as we

[2:05:57] planned it roll this out we can

[2:06:00] develop scenarios we we put that

[2:06:02] into the system and then the

[2:06:03] scoring that happens whether

[2:06:05] that's on a QA or on a training

[2:06:07] scenario the results of that are

[2:06:09] all come back to us so it's not

[2:06:10] like the system is taking what

[2:06:10] we're putting in based on s

[2:06:13] ce na ri o s and then using that

[2:06:15] in other places we do own all of

[2:06:17] that and the Ai that it's using

[2:06:19] is based on internal data that's

[2:06:21] put in there by members of the

[2:06:22] state patrol and our training

[2:06:22] staff

[2:06:26] just a couple of quick notes

[2:06:28] there building off that first

[2:06:29] slide that I talked about number

[2:06:31] one accessibility was as as

[2:06:33] noted with as required by

[2:06:34] House211110.

[2:06:36] and then like the cost that

[2:06:38] we talked about of the we have a

[2:06:39] wonderful relationship with the

[2:06:41] AG's office and one of my

[2:06:43] favorite attorneys on the planet

[2:06:44] is our contract attorney there

[2:06:46] and Laurie does great work for

[2:06:47] us and we give her plenty to

[2:06:49] look at but on the strength of

[2:06:51] our contracts with that

[2:06:53] partnership has been a very

[2:06:55] strong and as noted comorks is a

[2:06:56] software program it's no

[2:06:57] hardware on site for us.

[2:07:01] and then again just kind of a

[2:07:02] summary on this one I think this

[2:07:04] is a real value add if if

[2:07:04] nothing else I mean the the

[2:07:06] efficiency is a huge deal for us

[2:07:08] of course but I think the the

[2:07:10] biggest benefit here is we're

[2:07:12] going to be able to have folks

[2:07:13] that are going to get on live

[2:07:15] call taking and live dispatching

[2:07:16] and having gone through the

[2:07:18] system that transition will be

[2:07:19] much more seamless than it would

[2:07:21] have than it had been before. so

[2:07:22] the first time that they're

[2:07:24] dealing with a life saving you

[2:07:26] know a life at risk won't feel

[2:07:27] like the first time anymore and

[2:07:28] that's a pretty compelling thing

[2:07:29] for

[2:07:29] us.

[2:07:31] maybe I'll stop there for any

[2:07:32] questions on the comors.

[2:07:37] Super cool I went and visited

[2:07:40] a group of doctors last summer

[2:07:43] and they're also using

[2:07:44] technology to train people and

[2:07:45] little did I know that I would

[2:07:47] be getting to be trained in this

[2:07:49] real life scenario of giving

[2:07:53] helping deliver a baby with

[2:07:56] this amazing technology but it

[2:07:59] exists because of the real world

[2:08:00] stuff the crises what's the

[2:08:01] patient saying things you're

[2:08:02] not

[2:08:04] expecting there's someone else

[2:08:06] in the room now how do you deal

[2:08:07] with that person? That is a

[2:08:11] perfect example of how to use Ai

[2:08:13] in my mind and just a really

[2:08:16] great use I wanted to be

[2:08:18] clear on something you said that

[2:08:22] you used to do 250 units per

[2:08:23] month and now you're doing

[2:08:26] 13,000 units. what's that unit?

[2:08:28] Is it a training scenario? Is

[2:08:30] that what that is? Thank you

[2:08:31] madam Chair that's on the

[2:08:32] quality assurance piece and so

[2:08:32] that's

[2:08:35] that's 250 reviews of a call. so

[2:08:37] it's one call is one review and

[2:08:39] now we're able to review a much

[2:08:41] much larger review of thosew or

[2:08:43] sampling of those I should say

[2:08:45] that's great thank you I don't

[2:08:47] see any other questions so we

[2:08:48] can move to the next one

[2:08:51] right thanks and we'll move on

[2:08:53] to the the conversation about

[2:08:55] axon and go ahead and go to the

[2:08:56] next slide for me please. one of

[2:08:57] the things in the the the prompt

[2:09:01] to bring us here todays spoke

[2:09:02] specifically about fleet re

[2:09:04] which is our incar camera system

[2:09:06] and then how ALPr can be an

[2:09:08] incorporated to that but I did

[2:09:09] want to like talk a little bit

[2:09:11] more broadly about Axon and the

[2:09:12] system that we have and where

[2:09:14] there are other systems and

[2:09:16] Aibas technology that we're

[2:09:18] using within that Axon platform

[2:09:19] if that's ok I'm obviously a

[2:09:20] bulk of the what we'll talk

[2:09:20] about is

[2:09:23] is around the fleet re andlPR

[2:09:25] but the first thing to talk

[2:09:27] about again is like the

[2:09:29] different products that we have

[2:09:31] with Axon. so the the one that I

[2:09:33] think that is most compelling is

[2:09:35] and we're about to turn on

[2:09:36] again actually I don't think any

[2:09:37] of these are actually

[2:09:38] functioning yet

[2:09:40] other than the actual incar

[2:09:42] camera system but translation

[2:09:45] there are some media coverage

[2:09:46] down and I believe it was down

[2:09:47] in Parker with aarker has an

[2:09:49] axon body worn camera and they

[2:09:51] were able to show the video of a

[2:09:53] of a police officer on a traffic

[2:09:54] stop having a conversation with

[2:09:55] a gentleman that was speaking in

[2:09:58] Mandarin to my knowledge I don't

[2:09:59] think I have a trooper and

[2:10:01] anywhere in the state that

[2:10:02] speaks Mandarin I might have one

[2:10:03] or two that I just don't know

[2:10:05] about. but our intention is

[2:10:06] to turn on this technology so

[2:10:07] that we can have live

[2:10:11] interpretation like right

[2:10:12] there on the roadside and so

[2:10:13] that's I think that's a game

[2:10:15] changer for us that allows us to

[2:10:16] have more thorough

[2:10:18] investigations have more

[2:10:19] complete conversations on the

[2:10:21] side of the road which is a

[2:10:23] that's going to be a really

[2:10:25] really big win for us. The

[2:10:27] other thing that we'll use

[2:10:28] within the axon system also

[2:10:29] mostly off the body worn camera

[2:10:33] is that transcription process

[2:10:34] and so we'll be able to this

[2:10:37] this kind of sometimes gets

[2:10:38] overlapped with what

[2:10:39] they call I think it's called

[2:10:43] draft one axon draft one that is

[2:10:45] their whole reporting system and

[2:10:47] the patrol is not currently

[2:10:48] using that and so there are some

[2:10:50] agencies around the country that

[2:10:51] will actually write an entire

[2:10:55] police report based on what

[2:10:57] the body worn cameras record and

[2:11:00] do that and I not a popular

[2:11:01] opinion amongst everybody but

[2:11:03] I've been very very hesitant

[2:11:05] about turning that on for us. I

[2:11:07] have concerns about taking away

[2:11:09] the human in the context of this

[2:11:10] hearing the human in the loop to

[2:11:10] actually

[2:11:13] describe what they saw like so

[2:11:14] many of the decisions that

[2:11:15] police officers are being asked

[2:11:17] to make in split seconds and all

[2:11:18] of the case law that supports

[2:11:20] those decisions are based on

[2:11:21] that officer's ability to

[2:11:23] articulate their action in that

[2:11:25] moment. a great example that

[2:11:27] I give is because obviously the

[2:11:28] colorado state patrol does a lot

[2:11:30] of work on impaired driving

[2:11:31] and you and I could watch the

[2:11:33] same video of the same person

[2:11:35] and because of my training and

[2:11:37] my experience they're a cues

[2:11:39] that I can pick up on that are

[2:11:40] an indicator that I've been

[2:11:40] trained or an indicator of

[2:11:41] impairment

[2:11:43] and I'm not confident that that

[2:11:45] the system is ready to do all of

[2:11:47] those things and on the most

[2:11:48] significant cases where there's

[2:11:49] in fact like an officer involved

[2:11:50] shooting or something like that.

[2:11:52] I don't want that to be to be

[2:11:54] the first time that a trooper is

[2:11:55] articulating what they did and

[2:11:56] why they did it. so we want to

[2:11:57] make sure that they're in the

[2:11:58] habit of articulating those

[2:12:00] actions that being said

[2:12:02] transcription for a lot of

[2:12:04] other reasons can be a very very

[2:12:05] valuable tool in a time saver

[2:12:08] for us and so we are starting to

[2:12:10] turn that on not widely

[2:12:10] spread used but again testing

[2:12:11] this

[2:12:13] to talk about interviews to

[2:12:15] just transcribe not a completed

[2:12:17] report and Axon has a

[2:12:19] transcription service built into

[2:12:20] its software that we can use to

[2:12:21] do that whether that's an

[2:12:22] interview whether it's an

[2:12:24] internal interview or out there

[2:12:25] on the street and it looks like

[2:12:26] there might be a question on

[2:12:27] that. I have a question what was

[2:12:31] the name of the setting that is

[2:12:32] basically cause I think it's a

[2:12:35] vendor setting if I'm

[2:12:36] understanding you correctacim

[2:12:38] will allow you to turn on or

[2:12:40] off? do you want to create a

[2:12:40] police report from this

[2:12:43] Currently we have it set to no

[2:12:46] what's the name of that field or

[2:12:48] that program did you say fleet

[2:12:50] or something like that you

[2:12:51] madamir it's don't know if it's

[2:12:53] actually a a flip of a switch

[2:12:55] cause way Axon works sometimes

[2:12:57] it's a flip switch sometimes

[2:12:58] it's not but the system is

[2:12:59] called draft oneraft yeah and

[2:13:02] it's as I understand it it's

[2:13:02] more connected to

[2:13:05] their like RMS system that

[2:13:06] they have that we also don't use

[2:13:11] but isrmS stand for? Records

[2:13:13] managementment system and we

[2:13:14] don't use the record there are

[2:13:15] records management system. Do we

[2:13:16] have our own internal

[2:13:19] proprietary RMS thank you

[2:13:21] madamir and with the help of the

[2:13:23] JTC last session and the

[2:13:24] joint budget committee we are

[2:13:26] anticipating going to RFP later

[2:13:29] this fiscal year for a new RMS

[2:13:30] for us because we don't really

[2:13:32] have one don't remember that but

[2:13:34] I'm going to trust you on that

[2:13:36] there were a lot of I know I

[2:13:38] know I know perfect. Well

[2:13:38] rett Pascal

[2:13:43] thank you madam chairir. just

[2:13:44] wondering is there a particular

[2:13:46] reason you guys don't just turn

[2:13:47] this one on and you want to use

[2:13:48] a different records management

[2:13:50] system or what's the thinking

[2:13:53] there or is this a potential RFP

[2:13:55] before you answer that I'm super

[2:13:57] glad we keep that internally

[2:13:59] because I personally don't love

[2:14:00] the fact thatacxam would have

[2:14:03] all of our records so I'm glad

[2:14:05] that we use an internal records

[2:14:07] management system but please

[2:14:09] Chief Packard answer. thank you

[2:14:10] madamir andrett Pasco

[2:14:14] we don't know what that's gonna

[2:14:15] look like yet so we're in the

[2:14:16] process of drafting our

[2:14:18] requirements and of axon is one

[2:14:20] of the organizations that

[2:14:23] provides a bid or a proposal

[2:14:25] for us to consider we'll do that

[2:14:26] but whether or not we go with

[2:14:28] them or not will be based on all

[2:14:29] of the different settings and

[2:14:31] there's a requirements I should

[2:14:32] say that will be vast

[2:14:36] Rep to tone

[2:14:41] thank you madam chairir. one

[2:14:44] question I had is about sharing

[2:14:44] information with

[2:14:48] with other organizations and

[2:14:51] that's been something that has

[2:14:53] been topic at the legislature

[2:14:57] is you know flock and and a

[2:14:59] on the all operate basically the

[2:15:00] same way

[2:15:04] of just sharing this data and

[2:15:05] even selling it to

[2:15:09] other places without a

[2:15:11] warrant and that's something

[2:15:13] that I'm concerned about.

[2:15:14] what, what are your policies on

[2:15:14] that?

[2:15:19] cheap packard thank you madamir

[2:15:22] andpresentative to tone so

[2:15:23] with Axon

[2:15:26] the sharing of data that we have

[2:15:27] on and talking about like what

[2:15:29] we've addressed so far and and

[2:15:31] we can move into the LPr

[2:15:33] conversation which might be more

[2:15:35] in step with the the root of the

[2:15:38] question rep to tone but

[2:15:39] generally speaking the data that

[2:15:41] we collect with Axon stays

[2:15:44] within the the ecosystem within

[2:15:45] axon and then we have total

[2:15:46] control of where that goes.

[2:15:50] so right now for example a body

[2:15:53] worn camera review or or

[2:15:55] video or or even a video of a

[2:15:57] fleet from fleet 3 there's an

[2:15:59] incar camera where that might go

[2:16:00] there's a couple of different

[2:16:02] options obviously we will

[2:16:03] share that with other partner

[2:16:05] agencies because of we're on the

[2:16:09] same case and so a great example

[2:16:10] I stopped with a car crash an

[2:16:11] injury crash in a municipal

[2:16:13] jurisdiction a few months ago.

[2:16:15] my body worn camera was on the

[2:16:16] patrol was not investigating

[2:16:17] that in

[2:16:19] ci d ent I just happened to be

[2:16:21] there. my body worn camera video

[2:16:23] then was shared and sent over to

[2:16:25] the investigating agency so they

[2:16:26] can ingest that into their

[2:16:27] records and presumably that

[2:16:29] ultimately went to a district

[2:16:30] attorney's office for some sort

[2:16:30] of prosecution

[2:16:33] we would do the same thing and

[2:16:35] that's probably the number one

[2:16:38] recipient of our all of our

[2:16:40] video which is the most of our

[2:16:43] ecosystem is videobased most of

[2:16:45] it our primary receiver of

[2:16:47] that goes to the district

[2:16:49] attorney's office and there is a

[2:16:50] sharing function in the system

[2:16:52] that we can provide that like

[2:16:53] directly to them so that they

[2:16:55] can have that for their for for

[2:16:57] their responsibilities the

[2:16:59] only other place that it might

[2:17:00] be shared is when it comes

[2:17:02] under you know things like co

[2:17:04] request or CCGRA analysis where

[2:17:06] members of the public are

[2:17:08] requesting video and we'll go

[2:17:09] through the factors outlined in

[2:17:11] law about what should and what

[2:17:12] can and can be cannot be

[2:17:13] released what should be redacted

[2:17:15] and what cannot be redacted

[2:17:17] video reaction as a whole

[2:17:17] another that's a huge

[2:17:21] laborious conversation as well

[2:17:24] but those things would would

[2:17:26] go through and of course as

[2:17:28] pursuant to law like we make

[2:17:29] sure that folks attach that

[2:17:31] they're not ingesting that for

[2:17:33] pecuniary gain like if they're

[2:17:34] doing that it's for education

[2:17:35] for some other reason that would

[2:17:38] be accessible via CCGRA or

[2:17:38] Cora request.

[2:17:47] OK, very good. Please continue.

[2:17:50] OK, moving on to the

[2:17:51] conversation specifically around

[2:17:52] fleet re

[2:17:55] so again I'll just tell you

[2:17:57] that we have not turned this on

[2:17:59] yet. so but I I wanna take a

[2:18:01] little opportunity to also to

[2:18:02] speak more holistically about

[2:18:05] LPRs and how we're doing that

[2:18:07] we do have ALPRs at the colorado

[2:18:08] sea patrol. we've actually had

[2:18:10] them functioning for well over a

[2:18:11] decade and right now we are

[2:18:14] eva evaluating whether or not to

[2:18:15] activate the ALpR function

[2:18:16] within fleet re.

[2:18:19] and so going through some of

[2:18:21] the the considerations here.

[2:18:24] data sharing is one specifically

[2:18:26] to fleet 3 data sharing anything

[2:18:28] any license plate read that

[2:18:30] happens in a in a cholera sea

[2:18:33] patrol run system stays within

[2:18:37] the patrols ecosystem it does

[2:18:38] not automatically go out

[2:18:39] specifically with an axon. We

[2:18:41] talk about some of the other

[2:18:42] ALpR systems that we have we

[2:18:45] have two other vendors one of

[2:18:46] them we're not sharing

[2:18:49] anything anywhere yet. we

[2:18:51] actually made a decision in 2025

[2:18:52] that we turned off all sharing

[2:18:55] agreements so we were the only

[2:18:56] people that had access to that

[2:18:56] information

[2:18:59] if another agency had an

[2:19:01] investigatory investigatory need

[2:19:03] to see if we had contact with

[2:19:04] there's something along those

[2:19:06] lines that an agent or or an

[2:19:09] agent like a a police officer

[2:19:10] or a detective or something

[2:19:12] could call us and talk to a

[2:19:14] state patrol employee and said

[2:19:15] hey we're looking for this

[2:19:17] information on this thing and

[2:19:19] then our process is to take log

[2:19:20] like this is the case number

[2:19:22] this is the crime that there's

[2:19:23] an internal vetting before we

[2:19:25] would ever look in providing

[2:19:26] the information related to that

[2:19:26] but an important

[2:19:29] distinction here is nobody

[2:19:31] has their own access to our

[2:19:33] systems it's a it's a phone

[2:19:34] conversation they have to have

[2:19:35] with one of our folks before

[2:19:37] they might have access to that

[2:19:41] the the value of fleet re

[2:19:43] with Axon is

[2:19:45] it's multilane scanning we can

[2:19:46] load hot lists into that

[2:19:50] and that's that's the huge

[2:19:53] value for fleet re is we have

[2:19:54] the tech we have the opportunity

[2:19:55] to have a hot list that comes

[2:19:58] from NCIC or CCIC in in layman's

[2:20:00] term that is the system where

[2:20:02] vehicles are are noted as being

[2:20:03] stolen or for some other reason

[2:20:05] that they're a vehicle of

[2:20:07] interest and we are our system

[2:20:09] is set up to well can be set up

[2:20:11] to refresh that hot list every4

[2:20:13] hours and so that would come in

[2:20:16] and so the that's there

[2:20:16] and so as troopers are driving

[2:20:19] down the road if they receive a

[2:20:21] hit on their ALPR they will

[2:20:23] they'll hit that and then they

[2:20:25] the hit will appear they can

[2:20:26] look for the vehicle that is

[2:20:27] supposed to evidently close to

[2:20:29] them because it hit on their

[2:20:31] incar camera. they'll find that

[2:20:33] vehicle and then they'll go

[2:20:34] through their own manual

[2:20:35] verification process number one

[2:20:37] that is the car that is the

[2:20:39] plate they'll obtain a

[2:20:40] clearance independently back

[2:20:41] through the system to make sure

[2:20:43] it didn't fall off NCIC or CCIC

[2:20:45] in the last four hours to make

[2:20:46] sure that it is still a wanted

[2:20:46] vehicle

[2:20:49] for whatever reason that might

[2:20:51] be and then in the initiate

[2:20:53] whatever tactic they might do to

[2:20:55] to make a traffic stop to call

[2:20:56] in additional units or whatever

[2:20:57] that might be based on the

[2:20:59] nature of the hit. there's

[2:21:01] other opportunities to load into

[2:21:03] the system like vehicles of

[2:21:07] interest so for example a an

[2:21:08] ammber alert comes out quickly

[2:21:09] we can have the opportunity to

[2:21:12] put in a hot list that goes

[2:21:13] in the system independently

[2:21:15] within the patrol our policy is

[2:21:16] that we come through an

[2:21:19] investigative investigative

[2:21:20] group to make sure this is

[2:21:21] something we are ok with going

[2:21:25] in live into our system. if

[2:21:26] it's not touched otherwise that

[2:21:27] is a thirtyday life cycle so

[2:21:29] it's in there for 30 days and

[2:21:32] then drops back off. there's

[2:21:33] a review process that goes

[2:21:37] through that. The the audit

[2:21:38] process so we currently have an

[2:21:40] ALPr coordinator that works for

[2:21:40] us it's a position

[2:21:43] and that person ensures that all

[2:21:45] of the folks are trained

[2:21:47] appropriately to use that that

[2:21:49] also goes through and audits the

[2:21:50] logs if there's a query of

[2:21:51] license plate reads to make sure

[2:21:52] that there's appropriate

[2:21:53] justification provided for each.

[2:21:55] if there's not that's when

[2:21:56] investigation can start to go

[2:21:57] through and to validate whatever

[2:21:59] look that might whatever reason

[2:22:01] for that query might have been

[2:22:02] if it was a clerical error or if

[2:22:03] it was in fact a violation of

[2:22:04] policy or something along those

[2:22:06] lines so that person can do that

[2:22:09] I'm gonna go through some of

[2:22:11] your questions here on ALPRs as

[2:22:12] well

[2:22:15] but you talked about

[2:22:17] flagged do you how the question

[2:22:18] was how is flagged vehicle data

[2:22:18] retained

[2:22:21] and so again that is through the

[2:22:24] NCIC or CCI system that fourho

[2:22:27] update there's also a4

[2:22:29] there's there's a period of time

[2:22:30] in NCIC for a record to stay

[2:22:33] valid. I think it's4 years

[2:22:35] but once the vehicle is found

[2:22:37] that it will drop off of what's

[2:22:39] called a locate is done in that

[2:22:41] NCIc or CCIC system once the

[2:22:44] locate is done then the next

[2:22:47] refresh of the ALPr system will

[2:22:48] drop that off of the hit list. I

[2:22:48] already

[2:22:51] mentioned to you the 30 day rule

[2:22:53] for internal general or I'm

[2:22:55] sorry internally generated

[2:22:57] vehicles of interest I

[2:22:59] suspect that question is also

[2:23:00] more about the actual license

[2:23:03] plate reads for fleet re we

[2:23:04] haven't made that decision yet

[2:23:06] and and we are in fact I had a

[2:23:07] conversation this morning about

[2:23:09] where we are in this process and

[2:23:11] while we've been using ALPRs

[2:23:13] in a more limited version

[2:23:15] specifically an investigative

[2:23:17] areas like going through with

[2:23:18] more cameras we want to make

[2:23:18] sure that everybody

[2:23:21] has sufficient training the

[2:23:22] policy is where it needs to be

[2:23:23] and we've made thoughtful

[2:23:25] decisions about things like

[2:23:27] sharing retention and all of

[2:23:29] those things right now the

[2:23:30] conversation we're thinking

[2:23:32] about is for fleet 3 that system

[2:23:34] will probably stay within Axon

[2:23:36] and it's it's a different value

[2:23:37] than for example a fixed

[2:23:39] location and then it would be

[2:23:41] for the mobileca mobile reads

[2:23:43] but a lot of these decisions are

[2:23:44] yet to be made. Our current

[2:23:46] policy for our current systems

[2:23:48] is a license plate read stays in

[2:23:48] our system

[2:23:51] for 180 days and there's a lot

[2:23:53] of conversation about that and

[2:23:54] I anticipate having that

[2:23:55] conversation lots more over the

[2:23:57] next several weeks and months

[2:24:00] but it's a our we are currently

[2:24:03] set at 180 days and then that is

[2:24:05] it the system drops off and our

[2:24:06] auditor our our internal auditor

[2:24:07] also goes there and verifies

[2:24:08] that

[2:24:11] to be clear on the data sharing

[2:24:15] component to this we have we

[2:24:16] should internally with other

[2:24:16] Department of Public Safety

[2:24:17] entities they have access to

[2:24:20] that we're also moving down

[2:24:21] the path that we have redone an

[2:24:23] MOU to make sure that it's with

[2:24:25] current law and retention and

[2:24:27] reasoning for searching

[2:24:27] specifically around the Colorado

[2:24:29] law about information about

[2:24:31] immigration enforcement to make

[2:24:32] sure it won't be used for that.

[2:24:34] each agency is required to

[2:24:37] reup and sign their MOU with us

[2:24:38] and that will be limited

[2:24:38] initially

[2:24:41] to agencies that we partner with

[2:24:42] through our auto theft task

[2:24:43] forcece so those are capppaw

[2:24:45] funded entities the Colorado

[2:24:46] Auto Theft Prevention Authority

[2:24:49] and then what we call battle

[2:24:51] is is a is an auto theft task

[2:24:53] force so the battle task force

[2:24:55] works so those agencies that are

[2:24:57] involved in that which have

[2:24:59] their own level of because of

[2:25:00] their engagement with Capaw and

[2:25:01] battle their own level of like

[2:25:03] contractual agreements and and

[2:25:05] acknowledgement of law and then

[2:25:07] an added layer with the MOU

[2:25:08] specifically within the A

[2:25:13] L p R data. yes ma'am. OK so

[2:25:14] you're saying that

[2:25:21] every agency must sign an MOU

[2:25:25] with CSP through the cat paw is

[2:25:28] that every agency that is

[2:25:33] using ALPr cameras or accent

[2:25:37] cameras or just people that are

[2:25:38] a part of the cat paw. I'm

[2:25:40] trying to understand what you

[2:25:42] mean like what who do you have

[2:25:43] MU's with?

[2:25:45] thank you madam chair and so the

[2:25:47] answer is right now I'm not sure

[2:25:48] we have one all the way

[2:25:51] completed yet but the initial

[2:25:53] offer of that MOU is with other

[2:25:54] agencies that are affiliated

[2:25:55] with battle the the auto theft

[2:25:58] task force with the cat paw. so

[2:25:59] not beyond that at that point

[2:26:01] there was a specific question

[2:26:02] about federal government

[2:26:03] agencies and we're not currently

[2:26:05] sharing with any federal entity

[2:26:06] that data as well.

[2:26:12] and then do you know the so

[2:26:15] yeah 180 days you're still well

[2:26:17] within the range we have nothing

[2:26:19] on the books that's any smaller

[2:26:22] than 3 years requirement. so

[2:26:25] a half a year is fine. I do

[2:26:26] think we'll be having having

[2:26:28] some conversations about the

[2:26:31] length of that retention

[2:26:37] but this this this all

[2:26:39] this all makes sense to me. I

[2:26:40] don't really have questions. I

[2:26:41] do think that rep to tone

[2:26:45] I think yes rep vice chair to

[2:26:46] tone you're up.

[2:26:47] thank you ma'amirir.

[2:26:53] so I I would be curious to to

[2:26:55] know what your policies are

[2:26:56] about abusing

[2:27:00] data cause I I know I've seen a

[2:27:00] lot of

[2:27:04] cases where officers have been

[2:27:05] using

[2:27:08] these ALPRs to track people

[2:27:13] ex's and people of interest

[2:27:14] that they just want to know

[2:27:15] where they are

[2:27:18] and I would like to know that

[2:27:19] you know what's your policies

[2:27:21] are and I would hope that you

[2:27:22] would

[2:27:22] have quite a a

[2:27:26] a robust policy against that

[2:27:29] because I don't want to see

[2:27:31] that kind of thing abused

[2:27:32] because that you know

[2:27:34] these kinds of cameras can

[2:27:38] help and assist in a crime. I

[2:27:38] don't

[2:27:42] deny that that is a you know

[2:27:43] something that it's possible but

[2:27:46] abusing it is very easy to do

[2:27:47] and I want to make sure that

[2:27:48] you're preventing that

[2:27:53] there was a another thing I was

[2:27:54] gonna ask about

[2:27:57] when it comes to

[2:28:02] a license plate that's not read

[2:28:04] correctly and I've seen some of

[2:28:05] these cases where

[2:28:09] there's a family being pulled

[2:28:10] out of a vehicle

[2:28:10] and

[2:28:14] guns are being drawn by multiple

[2:28:15] officers with

[2:28:19] that putting these innocent

[2:28:20] people in a very difficult

[2:28:20] situation.

[2:28:23] What what kinds of policies do

[2:28:25] you have for these and

[2:28:29] does the system give you a full

[2:28:31] confidence rating in in the

[2:28:32] reading of the plate so you know

[2:28:33] that like

[2:28:33] it's

[2:28:38] you know it's it's very sure

[2:28:40] it's got the match or it's not

[2:28:40] quite as

[2:28:43] as good of a match and does that

[2:28:46] go into the response that you

[2:28:47] take

[2:28:49] on those particular stops

[2:28:51] because you know you don't wanna

[2:28:53] accidentally shoot an innocent

[2:28:55] person whose license plates one

[2:28:56] letter off because that would be

[2:28:56] a

[2:28:59] pretty embarrassing thing to

[2:29:00] have happen.

[2:29:03] yeah Chief Packard so two things

[2:29:05] one would you be willing or able

[2:29:07] to send us a copy of your

[2:29:10] policies for the organization

[2:29:13] within5 days. Is that OK?ritten

[2:29:14] your written policies

[2:29:17] thank you madamir yeah we can

[2:29:19] show you that policy and I think

[2:29:22] if I can't continue with well

[2:29:24] was going to say I also am

[2:29:26] curious about how often you're

[2:29:28] checking the error rate of your

[2:29:29] vendor contracts so yeah

[2:29:33] play that in with what reptonene

[2:29:34] just asked please.

[2:29:36] thank you madamir madam vice

[2:29:38] chairir so our first stop with

[2:29:40] I'll start with the conversation

[2:29:42] about what policy says about

[2:29:45] misuse of the system the

[2:29:47] language is in fact I think it's

[2:29:48] in the front first page of the

[2:29:50] policy does any misuse of this

[2:29:52] system is subject to you know

[2:29:53] personnel action under the state

[2:29:55] personnel board and obviously

[2:29:57] because of the rules within the

[2:29:59] personnel board that language is

[2:30:00] intentionally broad

[2:30:03] but I just offer hopefully with

[2:30:04] some degree of comfort

[2:30:07] for you in the process that

[2:30:09] we're currently undertaking

[2:30:10] with before we're turning on and

[2:30:13] implementing fleet re and

[2:30:15] this is a robust process because

[2:30:17] not only would it be

[2:30:17] embarrassing it would be

[2:30:19] unlawful and it would be

[2:30:20] horrible if someone lost the

[2:30:21] life because of an improper

[2:30:23] action that a member of our

[2:30:24] organization or any organization

[2:30:27] took based on ALPR reader or

[2:30:29] anything else and so we're going

[2:30:31] to go through a robust training

[2:30:33] program for each member that

[2:30:34] would have access to this and in

[2:30:34] fact our current

[2:30:35] policy states that if you have

[2:30:37] not been trained that you were

[2:30:39] not permitted to use it and

[2:30:40] that training talks about the

[2:30:43] process to go through to vet the

[2:30:44] reid as it shows up on the

[2:30:47] system and so there's a couple

[2:30:49] of different things that could

[2:30:51] happen here. there could be

[2:30:53] an improper entry into NCIc or

[2:30:55] CCIC and so that plate might

[2:30:56] actually be an accurate read

[2:30:59] but then we train our members

[2:31:01] to make decisions based on what

[2:31:03] they're seeing as well and make

[2:31:06] oper enforcement decisions and

[2:31:07] tactical decisions based on the

[2:31:09] totality of the circumstances so

[2:31:11] the license plate read whether

[2:31:13] it's correct or not is but one

[2:31:15] factor in this overall analysis

[2:31:18] of how we might respond to a s a

[2:31:20] specific incident and we'll

[2:31:21] train through as many of those

[2:31:22] scenarios as we possibly can

[2:31:25] with our members there is

[2:31:26] another scenario that might

[2:31:26] happen is

[2:31:29] like

[2:31:32] the maybe the best way to do

[2:31:35] this I I personally a personal

[2:31:37] story this was a long time

[2:31:38] ago when I was still a real

[2:31:40] trooper and I stopped on the

[2:31:41] side of the road with the

[2:31:43] vehicle and gave the plate

[2:31:45] license plate to the dispatcher

[2:31:47] who sent off all the alert

[2:31:49] tones on our radio this says hey

[2:31:50] this is a possible stolen

[2:31:51] vehicle. you know you need to

[2:31:52] you know

[2:31:55] handle cord right? well it

[2:31:57] was a mom and daughter with a

[2:31:59] flat tire and we are in the

[2:32:01] process of helping them change

[2:32:02] their flat tire when I received

[2:32:04] that information. Now obviously

[2:32:05] evaluating the entire scenario

[2:32:07] I'm like I wasn't too worried

[2:32:08] about the facts given at that

[2:32:09] moment. I was a different level

[2:32:11] of awareness and being more

[2:32:13] aware of my surroundings but

[2:32:14] talk through it well it turns

[2:32:17] out in that particular case

[2:32:18] it was a stolen license plate

[2:32:21] and not a vehicle and so and as

[2:32:22] I talked through the the the

[2:32:22] woman

[2:32:25] that was owned the vehicle that

[2:32:26] turns out their plate had been

[2:32:27] stolen and that's why that entry

[2:32:28] had been into the system

[2:32:30] and so the way we can work

[2:32:32] through those in an ALPR

[2:32:35] setting for example is an ALpr

[2:32:37] hit shows up on the computer we

[2:32:39] look at that and like I

[2:32:41] mentioned before locate the

[2:32:43] vehicle and then start

[2:32:44] looking what's around, what's

[2:32:45] going on are there other

[2:32:47] indicators of of potential

[2:32:50] criminal behavior rerun the

[2:32:51] license plate and so we can give

[2:32:53] that to our dispatcher you can

[2:32:55] the systems that our troopers

[2:32:56] have access to in their cars to

[2:32:57] say is this really a current

[2:33:01] stolen vehicle or whatever

[2:33:02] else and and then they can come

[2:33:03] back and say sometimes it could

[2:33:05] be a wanted person. well does

[2:33:07] the physical description of the

[2:33:08] person match the physical

[2:33:10] description of the warrant is it

[2:33:11] a stolen license plate or is it

[2:33:14] a stolen vehicle all of these

[2:33:16] things that can be evaluated

[2:33:17] while we're just driving

[2:33:18] alongside before we make an

[2:33:18] enforcement decision.

[2:33:21] and then once all of those

[2:33:23] facts are at play then that

[2:33:24] operational decision gets put

[2:33:26] into play and so there are many

[2:33:29] different steps along the way to

[2:33:30] make sure that there's an

[2:33:32] appropriate level response

[2:33:33] based on the situation that

[2:33:34] we're dealing with in that

[2:33:36] moment and that is all based

[2:33:38] on training and a review process

[2:33:41] that happens anytime that

[2:33:42] you know if there's a level of

[2:33:44] force that's used or the normal

[2:33:46] supervisor review of an arrest

[2:33:47] and investigation in some sort

[2:33:48] of incident so I spoke a lot

[2:33:51] there and madam vice chair maybe

[2:33:52] I'll pause there to make sure I

[2:33:52] answered all the questions you

[2:33:53] were looking for there.

[2:33:59] that's fine for now. thank you.

[2:34:00] very good

[2:34:05] Oh dewey, I'm sorry repelty

[2:34:11] thank you ma'amreau thank you

[2:34:13] and thank you sir for being here

[2:34:14] today. I do have a question

[2:34:16] it's very similar to rupted

[2:34:19] tonennes in in that that

[2:34:21] direction I'm gonna just tell

[2:34:23] you personally what happened and

[2:34:26] see if there's if you can

[2:34:28] help with the situation. I know

[2:34:30] it's a third party that does the

[2:34:32] the cameras for you know

[2:34:32] speeding and stuff like that

[2:34:35] that's around the area but

[2:34:38] myself I was I had to hurry

[2:34:38] around someone that was driving

[2:34:39] extreme

[2:34:41] ly dangerously and I just wanted

[2:34:42] to get away from them and in

[2:34:43] doing so I ended up getting a

[2:34:46] ticket because I went two miles

[2:34:49] or I was 11 miles over the speed

[2:34:50] you know to get around him and I

[2:34:51] got to the other camera because

[2:34:53] it's surveilling me from camera

[2:34:56] to camera and so it got to

[2:34:58] the other camera too soon and it

[2:35:00] just issued a ticket and

[2:35:01] normally in a situation like

[2:35:03] that an officer would pull you

[2:35:04] over you can explain to me hey

[2:35:05] that guy over there and that guy

[2:35:07] would be the one that the

[2:35:08] officer could go after you know

[2:35:09] the the the bad driver

[2:35:11] rather than the person trying to

[2:35:13] flee or get away from them.

[2:35:15] what type of policies do you

[2:35:16] guys have for something like

[2:35:17] that because trying to deal with

[2:35:19] this third party company that

[2:35:21] you that's being utilized for

[2:35:23] this is impossible you there is

[2:35:25] no way to object. there is no

[2:35:27] way to say hey you know there

[2:35:28] was a dangerous situation I had

[2:35:30] to get out of it I had to speed

[2:35:31] for a minute just to get out of

[2:35:33] it and it made me do 11 miles an

[2:35:36] hour over you know for or got me

[2:35:38] to the other camera faster than

[2:35:39] I should have, yes, but it

[2:35:41] was because of a certain

[2:35:43] situation. there's no way for

[2:35:45] our people for the for the

[2:35:47] drivers to be able to to say

[2:35:49] you know this is the situation

[2:35:50] the only way you can do it is go

[2:35:51] to court and then you have to

[2:35:53] pay the fees and then it ends up

[2:35:55] being rather than75 dollars fee

[2:35:56] which I had to pay. I just paid

[2:35:57] it just to get rid of it because

[2:35:58] that was during session I didn't

[2:35:59] have time for all that but other

[2:36:01] people have busy lives as well

[2:36:03] but they're just going to pay it

[2:36:04] rather than have to go to court,

[2:36:06] take time off of work, you know

[2:36:08] and hundreds of dollars later

[2:36:08] what kind of policy do you guys

[2:36:09] have

[2:36:11] or can you come up with a policy

[2:36:13] for people like in my situation

[2:36:15] where there was a very good

[2:36:17] reason as far as safety was

[2:36:19] concerned that I had to do make

[2:36:21] that decision what do you

[2:36:23] have for that for for then I can

[2:36:24] go back to my people and say hey

[2:36:27] you can you can do this cause

[2:36:28] right now it's a problem it's a

[2:36:28] big problem

[2:36:32] great really great example

[2:36:34] cheapef Packard thank you

[2:36:36] madamir and madam Representative

[2:36:39] I so I don't want to feel

[2:36:40] like I'm dodging your question,

[2:36:41] but the patrol does not use

[2:36:42] automated vehicle enforcement

[2:36:43] and so

[2:36:45] you can't get an automated

[2:36:47] citation from the Colorado State

[2:36:49] Patrol. Now there are other and

[2:36:51] there's obviously Cdot uses that

[2:36:53] technology in their

[2:36:54] enforcement in the managed

[2:36:55] lanes and some other scenarios

[2:36:57] and I so I don't have a

[2:36:58] strong awareness of the specific

[2:37:01] process that you have to go

[2:37:03] through to contest a violation

[2:37:05] that was received. I know a

[2:37:07] member of my family who I loved

[2:37:08] dearly and have've been married

[2:37:10] to for many years received

[2:37:14] one of those citations and

[2:37:15] of course I knew where she was

[2:37:17] and understood things and so we

[2:37:20] we paid the violation but my

[2:37:21] understanding is that there is

[2:37:22] instructions provided to that

[2:37:24] but

[2:37:27] honestly ma'am, I don't know

[2:37:28] that there's an opportunity to

[2:37:31] advocate for a justifiable

[2:37:32] reason to violate the law of the

[2:37:34] system and that and in the

[2:37:35] context of automated vehicle

[2:37:36] enforcement and again I'm not

[2:37:38] the expert here it's more about

[2:37:39] whether or not it was your

[2:37:40] vehicle and it was an actual

[2:37:41] violation of law as opposed to a

[2:37:43] justifiable exemption that you

[2:37:45] might be able to provide to a

[2:37:46] human being on the side of the

[2:37:46] road

[2:37:52] and it could have been Rep Kelty

[2:37:53] it could have been either an

[2:37:55] avis or an ALPR system

[2:37:58] you know

[2:37:59] and they're both

[2:38:02] yeah well I'm sorry what were

[2:38:03] you gonna say I didn't mean to

[2:38:03] interrupt

[2:38:06] apologize and I guess that

[2:38:08] the problem is is no one knows

[2:38:10] it's just a a thing mailed to

[2:38:13] you someone in the mail and you

[2:38:14] know everyone getting these

[2:38:16] tickets left and right and and

[2:38:18] there there is no justification

[2:38:20] to say a how are things

[2:38:21] calibratedb the certain

[2:38:23] situation was arried you know or

[2:38:25] see there was a fire there was a

[2:38:26] dangerous driver there was you

[2:38:28] know and there's a problem with

[2:38:31] these systems and I I guess it's

[2:38:32] apologize I assumed that was

[2:38:32] your guys's department that that

[2:38:33] that

[2:38:36] handled all that but again it's

[2:38:38] that nobody knows it is a nobody

[2:38:39] knows and each agency in

[2:38:42] Colorado who has these cameras

[2:38:44] manages their own contracts

[2:38:47] directly with the vendors and so

[2:38:50] repelty you've really touched

[2:38:53] the tip of the iceberg on this

[2:38:54] one for sure.

[2:38:57] so I appreciate that. Any other

[2:39:01] questions on the ALPR or the we

[2:39:02] were just talking kind of avis

[2:39:02] stuff too

[2:39:07] OK, seeing none if my folks

[2:39:10] online would hang online we

[2:39:11] are going

[2:39:14] to first of all I want to

[2:39:16] make sure we've exhausted all of

[2:39:17] the questions related to this

[2:39:19] because we will not be asking

[2:39:21] questions when we go into

[2:39:23] executive session related to

[2:39:25] anything but what is going to be

[2:39:27] presented to us so I just want

[2:39:29] to ask the committee to make

[2:39:30] sure we don't have any

[2:39:30] additional questions

[2:39:35] seeing none what we're going

[2:39:39] to do is go into executive

[2:39:40] session

[2:39:42] we

[2:39:46] have the ability to go into

[2:39:48] executive session for these last

[2:39:50] two issues because they are a

[2:39:52] matter related to capital

[2:39:56] complex security measures so

[2:40:00] this does require a 2/3 members

[2:40:02] approval so it will require4 of

[2:40:04] us to vote yes to go into

[2:40:07] executive session we will not

[2:40:09] have to vote to come out of

[2:40:10] executive session we will not be

[2:40:10] taking

[2:40:14] any formal action or polling we

[2:40:16] will just be listening to

[2:40:19] information and asking questions

[2:40:20] when we're done with the

[2:40:23] executive session, I will

[2:40:24] announce that we're done and

[2:40:26] then we will come back here and

[2:40:26] I will adjourn the meeting.

[2:40:30] does anyone have any questions

[2:40:31] before I entertain emotion?

[2:40:36] OK seeing none, I will entertain

[2:40:38] a motion reppasal

[2:40:42] I moved that the jointchnology

[2:40:44] committee meet in executive

[2:40:45] session in accordance with

[2:40:51] section 2464023A4 for the

[2:40:53] purpose of matters related to

[2:40:55] capital complex security

[2:40:56] measures

[2:40:58] and that's a proper motion Mr

[2:40:59] Gravy will you please pull the

[2:41:00] committee

[2:41:05] senators and representatives

[2:41:06] eisley is excusedba

[2:41:07] i

[2:41:09] kelty

[2:41:14] Pascal yes

[2:41:16] vice chair at home

[2:41:18] madam chair

[2:41:24] that passes at this time I am

[2:41:26] going to ask that the only

[2:41:28] people who will remain on the

[2:41:31] zoom or in the room are

[2:41:33] committee members, committee

[2:41:35] staff and department personnel

[2:41:46] does OIT support your work?

[2:41:48] OIT can stay

[2:41:57] OK

[2:55:06] OIT coming forward and we'll

[2:55:08] look forward to seeing you guys

[2:55:09] shortly. thank you so much.

[2:55:12] Thank you

Joint Technology Committee [Sep 08, 2026]

Agenda

Transcript