Pharmaceutics & Novel Drug Delivery Systems

The holy grail in health care is not fancier technology and tools, it is physician and patient behavior change. Machine learning will truly come of age when it can systematically and reliably do one of two things – improve the decision- making of clinicians and patients or improve their efficiency in carrying out the actions that follow from those decisions” (Jean Drouin, M.D., Founder and CEO - Clarify Health Solutions, 2018).

The quote above presents well the current state of machine learning in the healthcare industry. Every aspect of the area seems to be influenced by some set of models and their results; however, with now almost every analytics organization leveraging machine learning algorithms to provide insights into healthcare decision-making processes, there is an ever-increasing need for establishing a set of guidelines for machine learning research to aid data scientists with the ability to validate and replicate the applied algorithms and models.

         The discussion has been often focused on how to accurately identify at-risk patients to aid their disease education, diagnoses, and treatment, but also how to accurately attribute the patient population to physicians to ensure proper care for these patients. The application of such algorithms span from personal promotion triggering to addressable TV targeting, and patient journey / treatment identification. Often healthcare data along with sociodemographic variables are leveraged  to  predict  at- risk patients or their specific treatment pathways, noting the variables of significance that predict those currently in the at-risk group or their next treatment steps. However, more and more data scientists question the relevance, validity,  and directionality of the machine learning algorithm insights. Given the fact that someone’s diagnosis or treatment pathway might be impacted by the insights from the algorithms, there is an increased need for these models to be scrutinized and validated, when leveraged in the decision-making process. For example, can being on a diabetes medication be a predictor of breast cancer diagnosis or is it merely descriptors of the selected patient cohort that can help inform, but not predict the outcome?

            Further, questions are being posed on the ability to replicate results from the machine learning algorithms. Can an independent third party using the same data and assumptions arrive at the same results / final set of algorithms? Validation of the research has come to question in the recent couple of years, with a few cases of un- successful applications of the machine learning algorithms (i.e. IBM Watson (Mary Chris Jaklevic, 2017)). As a result, there is a growing concern in the scientific community about applying these techniques in certain areas of the healthcare industry due to the pitfalls listed above.

           Trying to think through on how to improve the process at-risk patient prediction along with the ability to validate and replicate the modeling outcomes, this presentation will review case studies in which we propose techniques and business reasonings to inform objective evaluation of the algorithms and their application in healthcare. We will outline selected benchmark rules that we think can help in validation of the research, along with examples of applications for case studies with the validated outcomes. As a result, the audience will be able to learn and then apply same rules and techniques when working on machine learning projects to ensure the results are not only informed in science and valid but can also be replicated if needed by others.