Clinical Trial Data Evaluation

How to turn clinical data into real insight in your clinical evaluation? Let’s walk through it step by step.↴ 1. What is “clinical data”? Clinical data refers to any information regarding the safety or performance of a medical device that comes from: → Clinical investigations of the device itself → Other investigations or studies found in scientific literature → Post-market surveillance, especially PMCF → Peer-reviewed literature about similar clinical experience It’s not just “data”, it’s data that shows how the device behaves in the real world, in real hands, with real patients. 2. Clinical data must be relevant Relevance is contextual. That means: relevant to the medical device and its intended purpose. So: → Always match your data to a specific device and a specific clinical question → Exclude nonhuman, vague or off-label data unless justified → Avoid just dumping SOTA or competitor data unless it supports your argument → Carefully define search terms and clearly identify the device under discussion 3. Clinical data must be high quality Not all data is equal. High-quality clinical data should be: → Scientifically rigorous → Complete and methodologically sound → Controlled and peer-reviewed Use validated frameworks like: ✓ PICO (Population, Intervention, Comparator, Outcome) ✓Cochrane Handbook ✓ PRISMA ✓ MOOSE You can find guidance regarding appraisal of data in IMDRF, MDCG 2020-6, and Meddev 2.7/1. Sources from PubMed, Google Scholar, or Cochrane are preferred, especially if they’re top-tier journals. 4. Clinical data must be sufficient Here’s where many stumble. → Small sample size? Not enough → Low-quality studies or incomplete data? Weak evidence → Case reports and posters? Not generalizable → Not on EU population? Risky Sufficient data means: → Enough for an expert to form an opinion → Strong enough for the device’s risk level → Justified by hierarchy of evidence High-risk = needs deep literature and real-world experience Low-risk = might rely on limited, lower-quality data 5. So… when is clinical data enough? When it supports a clear, justified, and ethical approach to the device’s clinical evaluation. Risk-based is the key: → Higher risk = more data → Lower risk = less, if PMS and RM show acceptable benefit-risk

Treatment switching, which occurs when patients in a #clinical_trial move from the control arm to the experimental therapy, is often ethically necessary in oncology and it creates major statistical challenges. It can obscure the true effect of a new treatment and weaken the evidence used for regulatory and reimbursement decisions. A newly published paper by Campbell and colleagues addresses this issue by introducing a more robust way to recover unbiased survival estimates without discarding valuable trial data. 🔍 What’s new? The authors propose Augmented Two‑Stage Estimation (ATSE), a method that blends traditional trial data with carefully selected external #real_world or historical data. ATSE dynamically reduces the influence of external data when it appears less compatible, which means it improves precision only when the data genuinely support it. 💡 Why it matters: Regulators and HTA bodies expect credible and unbiased survival estimates. Real‑world evidence (#RWE) is playing an expanding role in trial design and evaluation. ATSE creates a practical connection between rigorous causal inference and pragmatic data integration. 📈 Impact: Simulation results show that ATSE can reduce bias and improve precision compared with conventional two‑stage adjustment and external‑control‑only approaches, provided the external data meet reasonable assumptions. This work demonstrates that thoughtful integration of external data can strengthen clinical trial conclusions.

QUIZ: What are the in-use compatibility studies needed to ensure that critical product quality attributes are maintained when a drug product is in contact with administration components? What is needed when there is a change in these components during clinical trials? This excellent, brand new paper by Mason et. al. describes a practical tool for risk-based in-use compatibility assessments. Quoting from the abstract: "In drug development, in-use compatibility studies are crucial steps to ensure that the critical quality attributes of the drug product are maintained when in contact with administration components. But once the drug is in clinical trials, unanticipated variations in these components can stretch limited resources and lengthen timelines to market, as these changes must be assessed and approved to ensure continued patient safety. It's desirable to use a science-based risk evaluation to determine the extent of data and testing needed in these situations, but there is no standard for how such evaluations are done. We have developed an Excel™-based semi-quantitative risk assessment tool to determine whether in-use testing is needed when drug delivery sites or components are changed during a clinical trial. We developed the tool based on our multi-company experience with compatibility studies for many types of drug products targeted for various geographic regions. We have employed the tool as a means to expedite decision-making and, if appropriate, reduce testing in low-risk situations. The tool can save significant time and effort (our estimate is approximately at least 6-9 months off the development cycle) and can minimize pitfalls in clinical administration. While we have designed the tool for our drug products and for use with parenteral dosing regimens, the tool can be adapted for other situations as needed. It will be especially useful for companies with more limited resources."

What if your data analysis could help shape the future of cancer treatments? It’s not just about crunching numbers—it’s about saving lives. Here is how: Clinical trials generate massive amounts of data, and we can use it! Take T-Cell Engaging Therapy therapy, a life saving cancer treatment. But it can also cause Cytokine Release Syndrome (CRS). It is when your immune system goes into overdrive and releases too many chemical signals (called cytokines) all at once, which can make you feel really sick with fever, fatigue, or even life threatening consequences. During a project with Sanofi, we developed a model to predict the pre-infusion risk of significant CRS (sCRS) for patients treated with TCE therapies. We used aggregated and anonymzed data from the Medidata Enterprise Data Store. Our results? • Patients with the highest risk quartile developed sCRS at >4 times the rate in the lowest risk quartile.  • CRS risk stratification may facilitate patient selection for TCE therapy and tailored pre-treatment and monitoring of CRS, with potential to maximize treatment efficacy, patient safety, and resource allocation. 💡Aggregated data like this helps us understand patterns and improve treatment outcomes. CC: Pénélope Lafeuille , William A. Blumentals, Claire Brulle-Wohlhueter, Weixi Chen , Chao Sang, Sydney Manning, Silvy Saltzman, Jan Canvin, Susan Richards , Cris Kamperschroer , Giovanni Abbadessa, Aniketh Talwai , Caroline Der-Nigoghossian, Yahav Itzkovich, Vibhu Agarwal, Rahul Jain, Tanmay Jain, Jacob Aptekar, Stephen Grupp, Sheila Diamond, MS, CGC #MedidataResearchAlliance  #clinicalresearch ----------------------------------------- 👉 Interested in collaborating or learning more? Reach out to me directly or email researchalliance@medidata.com!

As data management evolves in healthcare and life sciences, clinical expertise is becoming crucial in assessing data quality anomalies and ensuring the accuracy and relevance of data. By leveraging clinical insights, data management systems can go beyond technical validations to more nuanced, clinically meaningful evaluations. This trend is enhancing the way organizations manage data related to treatment regimens, patient pathways, and outcomes. Let us look at how the clinical expertise enhances Data Quality Management: Contextual Understanding of Anomalies: Clinical experts can differentiate between real data anomalies and acceptable clinical variances, ensuring that flagged issues reflect genuine concerns rather than natural clinical variations. As an example: A spike in blood pressure might not be an anomaly but a known side effect of a treatment regimen. Validation of Complex Treatment Pathways: Experts can verify that complex treatment pathways are accurately reflected in data, ensuring outcomes match clinical reality. In oncology, clinical experts can ensure that drug sequences and combinations are captured correctly. Reducing False Positives: Clinical insights can help minimize false positives by distinguishing normal clinical variations from real data quality issues, reducing unnecessary investigations. As an example, experts know lab values can fluctuate based on time of day, preventing unnecessary alerts. Enhanced Anomaly Detection: Clinical experts can help refine machine learning models, improving their ability to detect significant data quality issues. Experts can train models to recognize drug interactions or side effects that may cause data deviations. Deeper Insights into Patient Pathways: Clinical experts can interpret data within the context of patient care journeys, leading to better insights into treatment efficacy. As an example: Experts may see changes in a medication regimen as an indicator of disease progression. Optimized Data Stewardship: Clinical insights can guide data stewards to focus on issues that affect patient care, rather than irrelevant data points. In rare diseases, clinical experts help prioritize quality issues affecting treatment efficacy. As can be seen above, integrating clinical experts into data management processes can add a critical layer of expertise that enhances data accuracy, reduces false positives, and provides more relevant insights into patient care. This collaboration has the potential to significantly improve data quality, leading to better healthcare outcomes and more informed decisions.

Clinical investigations are one of the most underestimated challenges in your FDA compliance journey. Most manufacturers focus heavily on QMS documentation and 510(k) submissions. But it’s your clinical evidence that often becomes the real bottleneck. Why? Because it requires both scientific credibility and regulatory precision. And the FDA’s scrutiny of clinical data integrity has never been higher. The stakes are clear: insufficient clinical evidence means delayed clearance, additional information requests, or even denial. So, what does a strong, FDA-compliant clinical evaluation actually look like? Here are 5 essential elements every MedTech leader should master: Clinical Study Plan (CSP) ↳ This isn’t just a document; it’s your compliance foundation. ↳ Define clear endpoints that support your intended use and device claims. ↳ Vague objectives often lead to costly deficiencies during FDA review. Literature & Predicate Analysis ↳ Simply referencing published data or prior clearances isn’t enough. ↳ Develop a systematic literature review and justify your predicate device selection. ↳ Clearly explain inclusion and exclusion criteria; the FDA expects transparency. Clinical Data Adequacy ↳ “Adequate clinical evidence” is not one-size-fits-all. ↳ Define what “adequate” means for your device class and risk level. ↳ pre-2018 studies may not align with today’s expectations for data integrity and diversity. Post-Market Surveillance (PMS) ↳ This is not an afterthought; it’s part of your total product lifecycle. ↳ The FDA expects proactive monitoring of adverse events and trend analysis. ↳ “No complaints” is not a compliance strategy; data-driven vigilance is. Substantial Equivalence Justification ↳ The FDA’s bar for equivalence has evolved. ↳ You must clearly demonstrate similarity in design, materials, and intended use. ↳ Unsupported equivalence claims can trigger a complete response letter. Clinical evaluation isn’t a one-time milestone. It’s a living process from design and validation through post-market oversight. The manufacturers who succeed are those who embed clinical thinking into every stage of their device development. P.S. What’s been your biggest challenge in demonstrating clinical evidence for FDA clearance? At M.E. Dorat Consulting, we help medical device and biotech companies navigate FDA requirements with confidence, from QMS implementation to clinical data readiness. Let’s connect to strengthen your compliance strategy.

Statistical Methods in Clinical Trials: 1)Analysis of Variance (ANOVA): Purpose: Compares means between multiple groups to determine significant differences. Application: Assessing treatment efficacy across different dosage levels or arms. SAS Syntax: PROC ANOVA with CLASS and MODEL statements. **SAS Syntax:** PROC ANOVA data=trial_data; CLASS treatment_group; MODEL efficacy_outcome = treatment_group; RUN; 2)T-tests: Purpose: Compares means between two independent groups. Application: Comparing treatment and control groups. SAS Syntax: PROC TTEST with CLASS and VAR statements. **SAS Syntax:** PROC TTEST data=trial_data; CLASS treatment_group; VAR efficacy_outcome; RUN; 3)Chi-square test: Purpose: Analyzes categorical data to find significant associations. Application: Comparing response rates between treatment groups. SAS Syntax: PROC FREQ with TABLES statement for CHISQ test. **SAS Syntax:** PROC FREQ data=trial_data; TABLES treatment_group * response_category / CHISQ; RUN; 4)Survival Analysis: Purpose: Analyzes time-to-event data, such as survival rates. Application: Assessing progression-free survival or overall survival. SAS Syntax: PROC LIFETEST for Kaplan-Meier curves and PROC PHREG for Cox proportional hazards model. **Kaplan-Meier Curve SAS Syntax:** PROC LIFETEST data=trial_data; TIME time_to_event*event(0); STRATA treatment_group; RUN; **Cox Proportional Hazards Model SAS Syntax:** PROC PHREG data=trial_data; CLASS treatment_group; MODEL time_to_event*event(0) = treatment_group; RUN; 5)Linear Regression: Purpose: Evaluates relationships between predictor variables and continuous outcomes. Application: Assessing impact of covariates on efficacy outcomes. SAS Syntax: PROC REG with MODEL statement. **SAS Syntax:** PROC REG data=trial_data; MODEL efficacy_outcome = predictor_variable1 predictor_variable2; RUN; 6)Logistic Regression: Purpose: Analyzes binary outcome variables (e.g., response vs. non-response). Application: Analyzing categorical outcomes in clinical trials. SAS Syntax: PROC LOGISTIC with CLASS and MODEL statements for binary outcomes. **SAS Syntax:** PROC LOGISTIC data=trial_data; CLASS treatment_group; MODEL binary_outcome(event='1') = treatment_group; RUN; 7)Repeated Measures Analysis: Purpose: Examines data collected over time, considering within-subject changes. Application: Assessing longitudinal efficacy outcomes across multiple time points. SAS Syntax: PROC MIXED with MODEL statement and RANDOM statement for repeated measures. **SAS Syntax:** PROC MIXED data=trial_data; CLASS subject_id timepoint; MODEL efficacy_outcome = treatment_group timepoint treatment_group*timepoint / SOLUTION; RANDOM subject_id; RUN; #sas #clinicalsas #clinicalsasprogrammimg #biostatastic #statasticalprogrammimg #cdisc #adam #tlf #sdtm #clinicaltrial #r #sasprogrammimg

How to review a Case Report Form (CRF) as a Biostatistician. The Biostatistician MUST ensure all the necessary data for statistical analysis is captured correctly. It is very easy for the biostatistician to brush this task off as they are not the formal owner of the CRFs and they are typically working on other high importance items during this time such as randomization lists and initial versions of the SAP. DO NOT BRUSH THIS TASK OFF. A good and proper review of the CRFs can be the difference maker between a well conducted and a poorly conducted trial. As a reminder: - The biostatistician is ultimately responsible for ALL analyses conducted in the clinical trial - All analyses are dependent upon the data they are run on - All data comes from some collection vehicle - primarily the CRFs Review the CRFs - I beg you Here are best practices for a biostatistician reviewing CRFs: - Start with what is most important: The primary endpoint. Review the protocol and write down every variable needed to conduct that analysis. This can be difficult at the beginning of the study as you don't yet have a statistical analysis plan. So take your time and really begin to think about what variables you will need to analyze the primary endpoint - will you be using a model statement? If so, what variables would you need/want to include. (This is bonus points as you are naturally getting a head start on your SAP development 😎) - Once you have your list of variables, you need to determine the time points you need those values at. Are you conducting a repeated measures analysis? If so, what timepoints are to be looked at? Do this for all the variables required for the primary endpoint analysis. - At this point you have a list of all the variables you need and a list of all the timepoints you will need those variables to be collected. - You are now ready to do your formal CRF review! - Be sure to ask for a copy of the blank annotated CRFs (please note this is not CDISC annotations at this point - this is the annotations explaining what the Field OIDs represent). - Go through each page noting down when you see a variable being collected that you need and confirming that it is collected at the correct timepoint/visit. - Once you go through the entire CRF you should be able to clearly determine if everything you need is being collected and at the timepoints/visits you need it to be collected. "But this is only for the primary endpoint!" you say. And yes, you are correct, but the beauty of this process flow is that it can be repeated for secondary, exploratory and safety endpoints! Simply follow the above steps for all the other endpoints within the trial and you will be able to quickly and efficiently review the CRFs to ensure all the data you need to conduct your analyses is captured correctly. Let me know if any of the above is not clear and Happy CRF Reviewing! Happy Friday

This recent research paper from Nature discusses the development and evaluation of TrialGPT, an end-to-end framework using large language models (LLMs) for zero-shot patient-to-trial matching. TrialGPT consists of three modules: TrialGPT-Retrieval, TrialGPT-Matching, and TrialGPT-Ranking. The framework aims to address the challenges of patient recruitment for clinical trials by improving efficiency and accuracy. TrialGPT filters out irrelevant clinical trials from a large initial collection, achieving over 90% recall using less than 6% of the initial trials. TrialGPT-Matching predicts patient eligibility for each trial criterion, providing explanations and locating relevant sentences in patient notes. It achieves an accuracy of 87.3%, close to expert performance. TrialGPT-Ranking aggregates these predictions to generate trial-level scores, outperforming other models by 43.8% in ranking and excluding trials. The evaluation done by the team involved three cohorts of 183 synthetic patients with over 75,000 trial annotations. TrialGPT-Retrieval generated effective keywords for trial filtering, outperforming human clinicians. TrialGPT-Matching was evaluated on 1015 patient-criterion pairs, showing high accuracy in relevance explanations, sentence location, and eligibility prediction. TrialGPT-Ranking's trial-level scores correlated well with expert annotations, effectively distinguishing eligible and ineligible trials. A pilot user study at the National Cancer Institute demonstrated that TrialGPT reduced screening time by 42.6%, enhancing the efficiency of patient recruitment. The study highlights the potential of LLMs in clinical trial matching, emphasizing the importance of human oversight in AI deployments. The paper also discusses the architecture of TrialGPT, the patient cohorts used for evaluation, and the methods for keyword generation, criterion-level prediction, and trial-level scoring. Future areas of improvement could be, including integrating data from electronic health records and exploring other LLMs. Overall, TrialGPT shows promise in streamlining the patient-to-trial matching process, providing accurate and explainable predictions, and significantly reducing the time required for patient recruitment. Very interesting approach to applying LLMs in Patient Trial matching. #TrialGPT #Largelanguagemodels #Patienttotrialmatching #Clinicaltrials #Patientrecruitment #TrialGPTRetrieval #TrialGPTMatching #TrialGPTRanking #Zeroshot #Eligibilitycriteria #Syntheticpatients #Keywordgeneration #Hybridfusionretrieval #Criterionlevelpredictions #Triallevelscores #Accuracy #Explainability #NationalCancerInstitute #Naturallanguageprocessing #Evaluation #Medicalexperts #Screeningtime #Electronichealthrecords Source: https://lnkd.in/ekw62HtM Disclaimer: The opinions are not mine and not of employer's

This paper proposes a benchmark, expand, and calibration (BenchExCal) approach to trial emulation and describes the design and process for evaluating the performance of the approach through both simulation studies; five planned empirical examples are also described. https://lnkd.in/eqzC6Nki #bigdata #datascience #epidemiology #biostatistics #realworlddata #rwd #rwe #realworldevidence #causalinference #clinicaltrials #targettrialemulation #clinicalresearch #realworldoutcomes #patientoutcomes #pharmacoepidemiology #healthoutcomes #clinicalpharmacology International Society for Pharmacoepidemiology