Tips for doing DIY predictive analytics right

Michael Johnson, a decision support data scientist at Bend, Oregon-based St. Charles Health System, has only worked in healthcare for a couple years. Before that, he'd spent most of his career doing data modeling and predictive analytics in higher education and in the military.

During his short time so far in this data-intensive industry, Johnson, who will speak at the HIMSS Big Data and Healthcare Analytics Forum on June 13, says he's been impressed by the scope and variety of challenges that can be solved by smart applications of predictive modeling.

But the best way to make that happen, he said, is to learn how to do that hard work in-house. While many health systems rely on outside companies to develop and manage their algorithms,

DIY predictive analytic projects are well worth the effort it takes to get them off the ground.

"I'm not going to kid you," said Johnson. "You have to roll up your sleeves and make some decisions." You'll probably need to make some investments, too – in technology and in staff training."

But the benefits of having complete control over your analytics are hard to overstate.

"There are many tangible benefits to doing it in-house, but there are even more intangible benefits. Things like being able to have spinoff models that are closely related, but not exactly the same, that can answer a similar question,” Johnson said. “Or more closely involving the stakeholders, so they have increased confidence in the way the model was created and how it should be used."

Compare that to what he calls the "cookie-cutter" approach to outside model development, whose algorithms he says are sometimes only slightly more predictive than flipping a coin.

Sure, paying someone else to produce these models, developed with someone else's data, makes sense at least on some level. It's a heavy lift, with a lot of detailed and complicated work, to effectively perform number-crunching such as this.

"There really are pros and cons to having an outside vendor create your models and run through those," said Johnson.

But creating your own models, with your data and your people, sensitive to your own organization's needs, is easier than you may think.

"You can do this," he said.

OK, so how?

Most providers opt to rely on an outside vendor simply because predictive modeling appears to be such a challenge to get a handle on, and they have so many other competing priorities, said Johnson: "This seems like a hard problem, how do we do this with our resources?"

But as he has proven at St. Charles Health, it's doable, with help. Choosing a good technology partner is an import early step, he said

"There are a lot of software choices that are out there and some are better than others. And you can hit the ground faster with some than with others – systems with less coding involved, less maintenance, so it's a much more sustainable process."

With some software, "you can spend more of your time assessing the results, visualizing the interactions of the factors and spend more time implementing the model rather than writing code and making sure that you've got the right variables involved."

That said, it's OK to stick with what you know, at least in the early going.

"Let's say at your hospital you have someone who has used the software SPSS since they were an undergrad, and they've been using if for a really long time and are really comfortable," said Johnson. "Maybe that's where you start off. Have 'em write the code and spit out a model and there you go."

Because people are much more important in this endeavor than any technology, at any rate.

First, you need a "champion – someone to start the process. Someone who understands predictive modeling and regression to some degree. But you don't have to have a stats PhD," he said. "I think the real key to success is that it's a group effort. You have to rely on your subject matter experts to understand the real problem. You have to rely on your data analysts and your IT folks: Do we understand where the data is coming from and is it clean and is it representing what we think it is?”

Finally, hospitals need the right people, operationally, to make decisions about how to implement the result, so you can benefit from it.

"You need somebody who can stand back and look at the big picture. There's a real tendency to say, 'This is all data analytics and that's all you need – here's your equation,'" he explained. "But somebody needs to be looking at the bigger picture. How do we get it off the ground, how do we create the model, how do we implement it and how do we assess how we're doing."

One of the most important benefits of DIY in-house analytics is the easy ability to adjust the models as needed according to different departments' specific needs over time.

So it's important to make sure you're staying in touch with what various stakeholders are looking for to ensure those predictive capabilities are optimized, said Johnson.

Whether it's managing problematic chronic conditions, spotting ED frequent-fliers, reducing readmissions and lengths-of-stay, or assessing populations of pre-surgical patients, the data needs of complex health systems vary widely – and there's no shortage of problems to solve and processes to improve.

"A lot of times you have a great model, but if you don't know how to use it, or even just how to convey the results in a way that people understand, then you may as well just put it on the shelf," said Johnson. "Go to the user. Ask, 'What information do you need, and how is it going to help you improve the care of the patient?' Tailor your output to meet those specific needs."

So, no need to hire armies of extra data geeks to handle all that new algorithm-building?

"I think if you just comb your current staff, you have smart enough people, when equipped with the right toolkit, to get this off the ground," said Johnson. "And that goes for whether you're right at the beginning or you're a seasoned expert with math modeling and regression.”

Yes, it's hard work, and requires a lot of strategizing. 

"You need to figure out how you're going to use this, and then use an iterative process to get to the right tool in the right manner with the right content on the right schedule. But by going through the process, you're just going to get better,” Johnson said. "The first one is tough, the second one is easier. And pretty soon you'll find people across your hospital saying, 'Maybe we need a predictive model for this, to find out the causal factors.' When you start hearing those conversations, you say, 'Wow, we're there.'"

Twitter: @MikeMiliardHITN
Email the writer: [email protected]

Source: Read Full Article