As you have read on our blog before, Flatiron's mission is to serve cancer patients by dramatically improving treatment and accelerating research. An important aspect of that mission is trying to increase the number of patients who participate in clinical trials. As a patient, finding a clinical trial to participate in can be a daunting and confusing experience, but it can often literally save someone's life.
Flatiron holds quarterly hackathons where everyone in the company is invited to participate, including both technical and non-technical staff. It gives us the opportunity to take a step back from our daily grind to work on something we wouldn't normally take the time to do. I really enjoy hackathons because our daily work usually tends to involve incremental improvements to existing products and services, so it's refreshing to be able to start from a blank slate and come up with something new.
After two days of hacking, each team presents their project to the entire company, many of which then become prioritized projects across the company. Our first quarter's hackathon had the theme "For the Patients", which meant that every hack was geared towards directly helping patients. Over 100 employees participated in this hackathon (one quarter of our company!), splitting into almost 30 teams (some larger than others). I had the opportunity to work on a project that will make it easier for some colorectal cancer patients to find suitable clinical trials.
The National Institutes of Health (NIH) maintains a database of clinical trials and a website that allows patients, doctors and researchers to search for available trials. There are currently over 240,000 trials listed on this site, and although there are a variety of search tools available, without clinical context or experience it is easy to miss trials that might be beneficial, and there can be lots of noise to wade through.
For example, a search for "colorectal cancer" (CRC) AND "New York" in the basic search box returns 82 studies as of early April 2017, but many are not going to be relevant to a late-stage CRC patient searching for a life-saving treatment option. One study that is returned is trying to demonstrate the feasibility of recruiting people at risk for CRC to a physical activity intervention trial. Another study is a long-term interventional trial to measure the effectiveness of colonoscopy screening. Neither of these would address the current, immediate need of a stage IV CRC patient. Conversely, there are pan-tumor clinical trials that may not mention CRC specifically but could still be a good fit.
There are some websites that list a curated set of clinical trials for specific diseases, such as BreastCancerTrials.org, the Pancreatic Cancer Action Network, or Support for People with Head and Neck Cancer.
Another resource has been the Colorectal MSS Trials Finder, a spreadsheet converted into an online app that lists trials for stage IV microsatellite stable (MSS) CRC patients. This list of clinical trials is curated by oncology researcher, advocate and cancer patient Tom Marsilje.
Tom, after being diagnosed himself with stage IV MSS-CRC, first started curating CRC trial information by inputting details into a spreadsheet of trials that he manually found on the NIH's clinicaltrials.gov. He began sharing that spreadsheet directly with interested parties, and when that became unsustainable, he worked with a couple of friends who he met online to set up a webpage using AppSheet, a tool that makes it possible to turn a Google Sheet into a website. The functionality of the AppSheet site is pretty limited, and is hard to customize. But, the popularity of Tom's spreadsheet grew as more and more late-stage, MSS CRC patients turned to it for help finding clinical trials.
Flatiron decided that as part of our quarterly hackathon in March, we would partner with Tom and Fight Colorectal Cancer, a CRC research and advocacy group that Tom chose to partner with on this project, to upgrade the site by creating a proper web application, which enabled the team to improve the user experience, set the stage for future enhancements, and automate some of the trial curation process. In the process, we also made it possible for the management and curation of the site to be handed off from Tom to Fight CRC so that they could ensure the longevity of this important resource for CRC patients.
The original Colorectal MSS Trials Finder website
Besides creating a "real" web application to host the list of trials, the other main focus of the hackathon project was streamlining the actual curation process.
Tom had been maintaining his Google Sheet by manually performing some specific searches on clinicaltrials.gov, following a curation methodology he came up with that put the patient viewpoint first. He was primarily looking for immunotherapy trials that have the potential to radically improve the prognosis for late-stage patients for whom chemotherapy no longer works (these are generally not CRC-specific, so they can be difficult to find), or low-risk non-immunotherapy trials that have demonstrated some degree of effectiveness against CRC in Phase 1 testing (a Phase 1 clinical trial is an early, small-scale trial primarily intended to ensure safety of the intervention).
Historically, Tom would review each trial, and if he deemed it to be one that he would benefit from, he would copy and paste some of the details into the Google Sheet. This process was tedious and error-prone, and was not amenable to collaboration. Over the course of the two day hackathon, our team was able to automate all of the drudgery in this process, and provide a tool that allows trained volunteers working with Tom and Fight CRC to collaborate on the curation process so that it is no longer solely dependent on Tom.
Landing page of the new website
How did we accomplish all of this in a two-day hackathon? We took advantage of the diverse experience, specialization and knowledge of the hackathon team to break up the project into various workstreams that focused on different aspects of the problem, and we also tried to be thoughtful about the tools we used to avoid reinventing the wheel.
First, we split into two sub-teams, one focused on building a web application, and the other focused on automating the trial curation process. The web application team, made up of software engineers, further split out into groups to work on the front-end and back-end code. I worked on this team along with Ovadia Harary, Jeremy Kohansimeh, Nick Moloney, Joseph Delgado and Shannon Lee.
The team automating the trial curation process was more cross-disciplinary, made up of Chhaya Shadra (a medical informaticist), Alda Pontes (a data insights engineer), Vineeta Agarwala (a product manager), Ben Kies (a software engineer) and Srinivas Pai (a sales operations manager and CRC survivor). This team also distributed their work with some members working on writing the queries to search for relevant trials and on developing classifiers that identify trials as either immunotherapy or low-risk CRC-specific; meanwhile, other members with more clinical expertise analyzed the results of these queries, while the software engineer helped productionise and automate the queries.
Even with this degree of parallelization, we never would have been able to complete the project without taking advantage of modern cloud platforms and application frameworks that allowed us to deliver a production-ready web app without worrying about details like deploying servers, security or implementing user management functionality.
We selected the Heroku cloud platform to host the application, which not only reduces the complexity of deployment to simply pushing a new version of code to Git but also offers a rich ecosystem of add-ons to make it easy to add functionality like logging, monitoring and email integration.
For the application framework we chose the Django Python framework, which is very well-supported on Heroku and comes with a handy built-in administrative interface. We took advantage of this for the curation interface, but it also enables the Fight CRC team to update pages like the FAQs and other bits of text without needing to involve a programmer.
The flow for how a trial gets automatically added to the site works like this:
-
A script runs daily via the Heroku scheduler that looks for any new potentially relevant trials that have been added since the last time the script ran successfully.
-
These trials are added to the database with a status of "Not Reviewed."
-
A volunteer with clinical expertise periodically reviews new trials using the curation interface to confirm if it should be made available for searching, following Tom's curation methodology. If so, the volunteer can also add additional information about the trial that is not available from the clinicaltrials.gov database, such as drug brand names, relevant publications or if prior immunotherapy use would be an exclusion criteria.
-
Once the trial has been reviewed, the volunteer marks the "Reviewed" checkbox and saves the trial, at which point it is immediately searchable.
-
If the trial is not relevant, the volunteer can simply delete the trial. Over time, if a pattern of trials that are not relevant emerges, the query that looks for new trials can be refined to take that pattern into account.
-
Once a trial has been added to the site, a separate scheduled script runs daily to check for any updates to the trial in the clinicaltrials.gov database, such as the trial being made available in additional states, and applies those updates automatically.
Smiles all around as Tom & the Fight CRC team see the project for the first time
The finished website is available at TrialFinder.FightCRC.org. One of the most rewarding aspects of working at Flatiron is knowing that the work we do is directly supporting patients. Helping more patients find a clinical trial that can improve the course of their disease is just one way we are tackling our mission, and this hackathon is a great example of how we are doing it.
We cannot wait to see the impact that this new tool has on the CRC community!
(Some of) The hackathon team doing the #StrongArm with Tom Marsilje at Flatiron's HQ in New York City
To read more about our partnership, check out Tom's blog post here.
