Photo by National Cancer Institute on Unsplash

Evidence-based medicine is the “conscientious, explicit, judicious and reasonable use of modern, best evidence in making decisions about the care of individual patients” (Masic I et al. 2008) and has completely transformed the way that medicine is practiced. Patient-centered care is another philosophy of medical care that seeks to focus on the individual’s specific health needs and health outcomes, and in doing so elevates the patient’s role in medical treatment. Unfortunately these two paradigms have often not been well integrated in the way that medicine is practiced. …

Monte Carlo Estimation Part II

Photo by Jean-Louis Paulin on Unsplash

At the heart of any Monte Carlo technique, is randomization. We have already learned about the bootstrap technique for estimating a statistical parameter. This can be used to compare two distributions (using a t-test for example), but there is another Monte-Carlo technique used to determine if two populations are different, know as permutation testing.

To illustrate this, imagine we’re in a casino and we’ve been on a long losing streak. We want to determine if the dealer is shuffling the cards randomly, or is actually (gasp!) cheating!

Monte Carlo Estimation Part I

Photo by Moritz Kindler on Unsplash

Many digital experiments carried out these days can be happily analyzed using frequentist statistical methods since we expect the data to be normally distributed. However, there are certain metrics that are not so easy. Take, for example, an experiment designed to increase tax collections, or to pay an invoice. If all the invoices were either fully paid, or not paid at all, this could be analyzed with a binomial proportion test or a chi-squared test. However, a certain fraction of the invoices will be partially paid, making these tests unsuitable. …

Photo by Joshua Ness on Unsplash

Imagine you could build a system that can automatically reach out to users to collect feedback? This is possible today and this blog post will outline how to do it.

In this post I will take you through how you build a chatbot using Rasa Open Source. The post was inspired by a series of great blog posts on this topic, but adds in a little of my own experience in doing the deployment.

Rasa provides a framework that is understandable and intuitive, avoiding blackbox approaches and making it possible to get something up and running very quickly. It also…

Authors: Nathan Begbie, Charles Copley, Eli Grant

Photo by Hush Naidoo on Unsplash

Nurses are vital to any national health service, so supporting them is a key component of the health system. One example of this is the NurseConnect programme for South Africa’s National Department of Health (NDOH).

NurseConnect uses WhatsApp to send weekly messages of support and instruction to nurses’ phones. More in-depth information is accessible on a mobisite, with articles and knowledge-testing quizzes. We learned that enrolling nurses onto our platforms and evaluating how to improve the user experience and content is key to a platform’s success.

This blog is about an experiment we…

A guide to the credit system for the newly landed…

Photo by on Unsplash

My fiancé and I moved to the US in November 2018. I came on a K1 Fiancé visa (a story for another day!) and when I arrived I had NO credit history in the US AT ALL! As anyone will tell you this is a real pain! It effects EVERYTHING! From getting a job to apartment rentals to buying a car. I set about fixing this and tried to increase my credit score to excellent as quickly as I could. By the end of 2019 my score was >750. This is my story…

Photo by Dmitry Ratushny on Unsplash

Authors: Charles Copley, Eli Grant, Monika Obrocka, Bearnard Hibbins, Nathan Begbie, Themba Gqaza, Susan Nzenze, Martin Odendaal, Erik Harding

Rapid feedback services have become possible in the public health sphere with the advent of digital platforms. Previously, the potential to respond to user behaviour and feedback in health system was limited by the cost and time involved in data collection, however, with digital platforms data collection can be automated and done in near real-time with low associated financial costs. Taking advantage of this opportunity and building continual improvement mechanisms that evolve to keep pace with user preferences are an essential…

Authors: Charles Copley, Eli Grant

Photo by Jean Gerber on Unsplash

Collecting feedback from users is a common problem in any service. This is often achieved through focus group discussions or surveys, however these are expensive and slow to carry out. With the advent of IP messaging it is, however, possible to carry these out digitally over platforms such as WhatsApp, Telegram, signal and others.

One of the issues with such groups, is that certain identifiable characteristics are shared with other members of the group. In particular, the telephone number of all participants would be accessible to other members of the group. …

AUTHORS: Charles Copley, Eli Grant

Photo by Jen Theodore on Unsplash

Surveys are often used to get feedback from users of a service. However, the kinds of people that respond to surveys might not be the kind of person whose feedback you really want! This is known as selection bias. Reducing this bias requires increasing the proportion of people that respond. One way to do so is to offer financial incentives. Another might be to vary the length of the invitation message. Perhaps someone is more able to process a shorter message, making them more likely to respond to these?

In order to test the effect…

Authors: Nathan Begbie, Charles Copley, Eli Grant

Photo by Hush Naidoo on Unsplash

When an experiment is conducted on a population that interact with each other, experimentation using individual random assignment (i.e. a participant is randomly assigned to a treatment) runs the risk of a participant from one experimental condition describing their experience with another participant. This is known as contamination. Contamination can affect the behaviour of individuals not assigned to the experimental treatment, including those in the control group, thereby making it more difficult to correctly measure the impact of the treatment. For example, a participant assigned to the control group could feel cause resentment…

Charles Copley

Experimentation at @charles_copley |

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store