Graphs of Simulation Results

To enable swift visualization and analysis of the simulation results, FACTS has a number of pre-defined graphs it can display. Full and detailed simulation results are available in ‘.csv’ format files that can be loaded into other analysis tools allowing any aspect of the simulation to be explored. These files are described below.

The graphs in the multiple endpoint engine are largely the same as the graphs in the single endpoint engine, except that the graphs from the single endpoint core engines that show responses for an endpoint now have a control allowing for the selection of which endpoint the plot should be made for.

This page will attempt to highlight differences between the single endpoint and the multiple endpoint engines, largely focusing on plots that do not exist in the single endpoint output. See Continuous/Dichotomous Simulation Output for full details about the single endpoint graphs in FACTS.

Per Scenario Graphs

Many of the multiple endpoint graphs are repeated from single endpoint, potentially with the ability to view for any of the simulated endpoints. When the endpoint to use can be selected in the controls for a plot, it is common that the utility can also be selected instead of an endpoint. This will allow for assessment of the distribution of the combined utility in addition to the response estimates for the individual endpoints. The repeated graphs are listed here:

Utility and Subjects Allocation

Figure 1: Utility and Subject Allocation.

This graph displays a histogram of the mean subject allocation and a line graph of the mean estimated utility. These plots show:

The blue bars show the mean number of subjects allocated to each arm over the simulations.
The green dashed line is the average of the mean estimate of utility over the simulations
The black line is the true utility of the combination of the underlying response profiles being simulated for all the endpoints.
The green vertical lines around the green dashed line shows the range of the central 95% of mean estimates of utility over the simulations.

Estimated utility and Target Selection graphs

These plots show the utility of the true simulated response profiles and the mean and 95% spread of the mean estimate of utility ratios along with bars showing the proportion of simulations the different treatment arms are the finally selected target:

The dose with the maximum utility (left)
The Minimum Acceptable Utility Dose (Umin), the dose most likely to be the minimum dose that exceeds the Clinically Significant Minimum Utility (CSMU) of 0.5 (right)

Utility Boxplot

This box and whisker plot shows the distribution of the estimate of utility for each dose:

Per Sim: Utility and Subject Allocation

This set of graphs show the final utility analysis at the end of individual simulations.

Figure 4: Estimated Utility for all endpoints and overall, as well as the number of subjects allocated to each arm.

These graphs includes a control that allows the user to select which simulation to graph the results from.

On each graph

The black line shows the ‘True’ overall utility of the response profiles being simulated.
The green solid line shows the estimate of overall utility
The green dashed lines show the boundaries of the 95% credible interval for the estimate of utility.
The additional lines (here khaki, blue and brown) show the estimates of utility of the individual endpoints.
The blue bars show the number of subjects finally allocated to each arm.

Per Sim: Utility and QOI

This set of graphs show the final utility analysis at the end of individual simulations. This is the graph that’s shown when the Endpoint control is chosen to be “Utility” in the “Per Sim: Utility/Response Dist” plotting controls.

Figure 5: Estimated Utility for all endpoints and overall, as well as the estimate of a chosen QOI

On each graph

The black line shows the ‘True’ overall utility of the response profiles being simulated.
The green solid line shows the estimate of overall utility
The green dashed lines show the boundaries of the 95% credible interval for the estimate of utility.
The additional lines (here khaki, blue and brown) show the estimates of utility of the individual endpoints.
The blue bars show the final values of a selected Utility QOI.

Per interim: Utility Alloc, Utility Dist

This is an identical set of graphs to the Per Sim: Utility and Subject Allocation and Per Sim: Utility and QOI plots, except that in addition to the control to select which simulation to graph the results from, there is a control to select which interim within the simulation to graph the results from. These graphs are only available for those simulations for which ‘weeks’ files have been output (by default the first 100). number of files to be read and subsequent processing of the data.

Across Scenario Graphs

The Across Scenario graphs for the multiple endpoint engine are the same as the graphs available in the individual continuous and dichotomous engines.

For reference, these are:

The only differences from the single endpoint across scenario graphs are:

QOI Box Plots graph now has Utility as a selectable endpoint and the utility QOIs are available to be plotted
Respnnse/Utility is the same as the single endpoint Response graph, except that Utility is a selectable endpoint
Interim vs Final Scatter Plot now has utility QOIs
Receiver Operating Characteristics plots now have utility QOIs.

Viewing Simulation Results in the Simulations Tab

After simulations have been run, FACTS provides a table for assessing the output from the simulations. While everything that FACTS displays in the table on the Simulation tab is available in output files, FACTS does a substantial amount of “beautifying” of results in its built-in table. This entails changes like converting numeric outcome scores to readable text and using descriptive names for QOI columns.

The main table has 1 line per scenario, and shows the summary of all simulations run for each scenario. These lines show values that are stored in the summary.csv file.

More detailed simulation output, providing a row per simulation run, is accessibile by double clicking on any row of the output table. This row-per-simulation output is what is stored in the simulations.csv files.

Additional detail is available for simulations that have a weeks file by right clicking on a simulation in the simulation summary window and selecting “Weeks Results”. See the Results Output section of the Simulation tab to specify the number of weeks files to be output.

Summary per Scenario

By default, only a subset of all columns of output are provided when simulations are complete. This set of columns is called “Highlights.” To get access to further columns, you can right click anywhere on the simulation output table and click “Summary Results: ….” for whichever set of columns you would like to see. There is a “Summary Results: All” option that will provide a pop up containing every column available for summary results.

The following sections break down which columns are available by sorting to which column subset.

Highlights

These are the columns displayed on the simulations tab after simulations are completed, the can also be displayed in the separate “Highlights” results window.

The columns that are displayed by default in the simulations tab after multiple endpoint simulations have been run.
Column Title	Number of columns	Description
Select	1	Not an output column, this column contains check box to allow the user to select which scenario to simulate. The ‘Select All’ button causes them all to be checked.
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ‘Virtual Subject Response > Explicitly Defined > Dose Response’, ’Virtual Subject Response > Explicitly Defined > Control Hazard rates, ‘Virtual Subject Response > Explicitly Defined > Predictor’, This is the same name as used for the results directory
Random Number Seed	1	Base random number seed used to perform the simulations.
Num Sims	1	The number of simulations that were run to produce the displayed results.
Mean Subj.	1	This is the mean (over the simulations) of the number of subjects recruited in this scenario.
Ppn Overall Success	1	This is the proportion of simulations that stopped for success, either early success or late success (as defined below).
Ppn Early Success	1	This is the proportion of simulations that stopped early for success (and did not regress to futility in the final analysis – though they might have regressed to ‘inconclusive’).
Ppn Late Success	1	This is the proportion of simulations that did not stop early but were successful in the final analysis.
Ppn Overall Futility	1	This is the proportion of simulations that stopped for futility, either early futility or late futility (as defined below).
Ppn Late Futility	1	This is the proportion of simulations that did not stop early but were futile in the final analysis.
Ppn Early Futility	1	This is the proportion of simulations that stopped early for futility (and did not regress to success in the final analysis – though they might have regressed to ‘inconclusive’).
Ppn Succ -> Fut Flipflop	1	This is the proportion of simulations that stopped early for success but regressed to futility in the final analysis.
Ppn Fut -> Succ Flipflop	1	This is the proportion of simulations that stopped early for futility but regressed to success in the final analysis.
Ppn Inconclusive	1	This is the proportion of simulations that did not stop early and were neither successful nor futile in the final analysis.
Early Success Time	1	This is the average time to an early decision to stop for success (i.e. the time excluding final follow up) over those simulations that did decide early to stop for success.
Mean Trt. <Dose>	One per arm	This is the mean (over the simulations) of the hazard ratio of the treatment arms to control. The HZ for the control arm is always 1.
Mean Duration	1	The mean over the simulations of the total duration from the start of enrolment to end of follow up.
PPn Arms Drop: <Dose>	One per arm	This is the number of times (over the simulations) that each arm was dropped.
Mean LP Enrolled	1	This is the mean, in weeks (over the simulations) of the duration of the trial from first patient first visit to Last Patient First Visit (i.e. the duration of accrual).
Ppn Correct Arm	1	The proportion of simulations that met the success criteria and selected (by the target QOI specified on the Study > Variant tab) one of the arms marked as “should succeed” on the Virtual Subject Response > Explicitly Defined > Dose Response tab
Ppn Incorrect Arm	1	The proportion of simulations that met the success criteria and selected (by the target QOI specified on the Study > Variant tab) one of the arms not marked as “should succeed” on the Virtual Subject Response > Explicitly Defined > Dose Response tab
Version	1	The FACTS version number at the time the simulations were run.

Allocation

By right clicking and selecting the allocation columns, a pop-out will appear that provides the following columns.

The allocation columns provided in FACTS output for multiple endpoint trials.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ‘Virtual Subject Response > Explicitly Defined > Dose Response’, ’Virtual Subject Response > Explicitly Defined > Control Hazard rates, ‘Virtual Subject Response > Explicitly Defined > Predictor’, This is the same name as used for the results directory
Mean Subj.	1	This is the mean (over the simulations) of the number of subjects recruited in this scenario.
SD Mean Subj.	1	This is the standard deviation across the simulations of the number of subjects recruited.
80%-ile Subj	1	This is the eightieth percentile across the simulations of the number of subjects recruited into the trial.
Mean Alloc.: <Dose>	One per arm	This is the mean (over the simulations) of the number of subjects recruited into each arm in this scenario.
SD Alloc.: <Dose>	One per arm	This is the SD of the means (over the simulations) of the number of subjects allocated to each treatment arm.

Utility

The utility columns provided in FACTS output for multiple endpoint trials.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Utility: <Dose>	One per arm	This is the mean (over the simulations) of the posterior estimate of the overall utility of this dose, this is the utility of each endpoint at the dose, combined using the specified utility combination method.
SD Utility: <Dose>	One per arm	This is the SD (over the simulations) of the posterior estimate of the overall utility of this dose.

Probabilities

The Utility QOIs, which are visible when Probabilities columns are selected in simulation output for multiple endpoint trials.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
<QOI> <Dose>	One per arm per Utility QOI	For each Utility Posterior Probability QOI this is the mean over the simulations of the estimate of the posterior probability of the QOI for each dose. For each Utility Target QOI this is the proportion of simulations where this dose was selected at the end of the trial as the dose with the greatest probability of meeting the target condition. The probability that each dose is the target at the end of a simulated trial is its marginal probability (the number of times it was the dose closest to the target in the MCMC sampling of the analysis at the end of the trial). The Max Utility target is always identifiable, so the Ppn(target) should sum to 1 across the doses. A Umin target is not guaranteed to be identifiable, if no dose meets the CSMU criteria in any MCMC sample so all doses have a 0 probability of having greater utility than Control then no dose is the Umin. So the sum of each Ppn(target) QOI across the doses should sum to between 0 and 1 inclusive.

<Endpoint> Response

The columns that appear when selecting <Endpoint> Response vary depending on whether the selected endpoint is continuous or dichotomous.

Continuous
Dichotomous

The columns provided if selecting endpoint specific response for a continuous endpoint.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Response (<Endpoint>): <Dose>	One per arm	This is the mean (over the simulations) of the estimate of the response on this endpoint on this dose.
SD Response (<Endpoint>): <Dose>	One per arm	This is the standard deviation (over the simulations) of the estimate of the response on this endpoint on this dose.
Mean Sigma (<Endpoint>)	1	This is the mean (over the simulations) of the estimate of the standard deviation of the response (across all doses) on this endpoint.
SD Mean Sigma (<Endpoint>)	1	This is the standard deviation (over the simulations) of the estimate of the standard deviation of the response (across all doses) on this endpoint
True Mean Resp (<Endpoint>): <Dose>	One per arm	This is the true change from baseline / response rate from which the simulation data was sampled.
True SD Resp.: <Dose>	1	If the endpoint is continuous these columns are included and report the true standard deviation of the simulated response. If baseline and “baseline adjustment of the simulated subject response” is used, this reported SD will include that impact of that and be different from the value(s) for sigma entered on the VSR > Dose Response tab.
Mean Utility (<Endpoint>): <Dose>	One per arm	This is the mean (over the simulations) of the posterior estimate of the utility of this endpoint on this dose.
SD Utility (<Endpoint>): <Dose>	One per arm	This is the SD (over the simulations) of the posterior estimate of the overall utility of this dose.

The columns provided if selecting endpoint specific response for a dichotomous endpoint.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Response (<Endpoint>): <Dose>	One per arm	This is the mean (over the simulations) of the estimate of the response on this endpoint on this dose.
SD Response (<Endpoint>): <Dose>	One per arm	This is the standard deviation (over the simulations) of the estimate of the response on this endpoint on this dose.
True Mean Resp (<Endpoint>): <Dose>	One per arm	This is the true change from baseline / response rate from which the simulation data was sampled.
Mean Utility (<Endpoint>): <Dose>	One per arm	This is the mean (over the simulations) of the posterior estimate of the utility of this endpoint on this dose.
SD Utility (<Endpoint>): <Dose>	One per arm	This is the SD (over the simulations) of the posterior estimate of the overall utility of this dose.

<Endpoint> Probabilities

The endpoint specific QOI columns for simulation output in the multiple endpoint engine.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
<QOI> <Dose>	One per arm per QOI defined on this endpoint	For each Posterior Probability and Predictive Probability QOI defined on this endpoint, this is the mean over the simulations of the estimate of the probability of the QOI for each dose. (P-value QOIs are reported in the frequentist results table). For each Target QOI this is the proportion of simulations where this dose was selected at the end of the trial as the dose with the greatest probability of meeting the target condition. The probability that each dose is the target at the end of a simulated trial is its marginal probability (the number of times it was the dose closest to the target in the MCMC sampling of the analysis at the end of the trial). The target of having the Max response on this endpoint, or some fraction of it (EDq) is always identifiable, so the Ppn(target) for these QOIs will sum to 1 across the doses. An MED target is not guaranteed to be identifiable, if no dose meets the CSD criteria in any MCMC sample so all doses have a 0 probability of having a response greater than Control (or AC) by the CSD then no dose is the MED. So the sum of each Ppn(target) QOI across the doses should sum to between 0 and 1 inclusive. Decision QOI’s are not reported here separately, but their component QOIs – the vector of values and target QOI used to select from them are.

<Endpoint> Baseline – Continuous Endpoints Only

The baseline columns shown when <Endpoint> Baseline is selected. Only available for continuous endpoints.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Baseline Beta (<Endpoint>)	1	If a baseline adjusted dose response model is being used then this is the mean (over the simulations) of the estimate of the baseline adjustment parameter ‘Beta’.
SD Baseline Beta (<Endpoint>)	1	If a baseline adjusted dose response model is being used then this is the standard deviation (over the simulations) of the estimate of the baseline adjustment parameter ‘Beta’.
Mean Baseline (<Endpoint>)	1	This is the mean (over the simulations) of the mean observed baseline score.
SD Baseline (<Endpoint>)	1	This is the standard deviation (over the simulations) of the mean observed baseline score.
True Mean Baseline (<Endpoint>)	1	This is the true mean of the baseline distribution used to simulate the baseline scores.
True SD Baseline (<Endpoint>)	1	This is the true standard deviation of the distribution used to simulate the baseline scores.

<Endpoint> Observed

The columns shown when <Endpoint> Observed is selected.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Complete <Dose>	One per arm	This is the mean (over the simulations) of the number of subjects recruited per arm which have had their endpoint observed in this scenario.
Mean Complete Info <Dose>	One per arm	This is the mean (over the simulations) of information observed per arm as defined on the Interims tab (Subjects enrolled, Complete Data at Specified Visit, Opportunity to Complete at Specified Visit) in this scenario.
Mean Dropouts <Dose> <Visit>	One per arm per visit	This is the mean (over the simulations) of dropouts per arm per visit in this scenario.

<Endpoint> Model Params

The columns shown when <Endpoint> Model Params is selected for a specific endpoint.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
BAC Mean (<Endpoint>)	1	This is the mean (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
SD BAC Mean (<Endpoint>)	1	This is the SD (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAC Tau (<Endpoint>)	1	This is the mean (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
SD BAC Tau (<Endpoint>)	1	This is the SD (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAAC Mean (<Endpoint>)	1	This is the mean (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
SD BAAC Mean (<Endpoint>)	1	This is the SD (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean control responses.
BAAC Tau (<Endpoint>)	1	This is the mean (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
SD BAAC Tau (<Endpoint>)	1	This is the SD (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.

Frequentist Results (<Missingness> <Endpoint>)

The columns available when selecting Frequentist results for a specific endpoint and missingness handling method.
Column Title	Number of columns	Description
Status	1	This column reports on the current status of simulations: Completed, Running, No Results, Out of date, Error. It is updated automatically.
Scenario	1	This column gives the name of the scenario, concatenating together the profile names from the following tabs: ‘Execution > Accrual’, ‘Execution > Dropout Rate’, ’Virtual Subject Response > Composite, This is the same name as used for the results directory
Mean Theta (<Missingness> <Endpoint>) <Dose>		The mean response per dose.
SD Mean Theta (<Missingness> <Endpoint>) <Dose>		The standard error of the response per dose
Ppn Signif (<Missingness> <Endpoint>) <Dose>	D	For each treatment arm, the proportion of \| simulations where the unadjusted p-value is \| less than the user specified one-sided alpha. \| \| The marginal probability of significance per \| dose (using unadjusted p-values).
Bias (<Missingness> <Endpoint>) <Dose>	D	The difference between the estimated mean response and the true (simulated) mean response per dose
Coverage (<Missingness> <Endpoint>) <Dose>	D	The proportion of simulations where the unadjusted confidence interval for the treatment difference compared to control contains the true mean treatment difference compared to control used to simulate subject responses.
Ppn Signif Dunnett (<Missingness> <Endpoint>)	1	For each treatment arm, the proportion of simulations where at least one of the Dunnett adjusted p-value is less than the user specified one-sided alpha.
Ppn Signif Dunnett (<Missingness> <Endpoint>) <Dose>	D	For each treatment arm, the proportion of simulations where the Dunnett adjusted p-value is less than the user specified one-sided alpha.
Coverage Dunnett (<Missingness> <Endpoint>) <Dose>	D	The proportion of simulations where the Dunnett adjusted confidence interval for the treatment difference compared to control contains the true mean treatment difference compared to control used to simulate subjects responses.
Ppn Signif Bonf (<Missingness> <Endpoint>)	1	The proportion of simulations where at least one of the Bonferroni adjusted p-values is less than the user specified one-sided alpha.
Ppn Signif Bonf (<Missingness> <Endpoint>) <Dose>	D	For each treatment arm, the proportion of simulations where the Bonferroni adjusted p-value is less than the user specified one-sided alpha.
Coverage Bonf (<Missingness> <Endpoint>) <Dose>	D	The proportion of simulations where the Bonferroni adjusted confidence interval contains the true mean response used to simulate subjects responses.
Ppn Signif Trend (<Missingness> <Endpoint>)	1	The proportion of simulations where the trend test p-value is less than the user specified one-sided alpha.

Detailed Per Simulation Results

After simulation has completed and simulation results have been loaded, the user may examine detailed results for any scenario with simulations output in the table by double-clicking on the appropriate row. A separate window pops up and displays the results for each individual simulation. This is a “prettier” view of the contents of the “simulations.csv” file, which is described below. The simulations results are partitioned into the same results groupings as the summary results. These can be accessed from the “right click” menu, along with opening the results folder, opening the weeks file for a particular simulation and opening the simulated patients file for a particular simulation (where a “weeks” file or a “patients” file has been output).

Figure 6: The simulation results pop out viewer showing simulation highlights.

Output Files

Instead of viewing simulation output on the Simulation tab within FACTS, the output is all contained within easily accessible files saved the the user’s local machine. FACTS stores the results of simulations as ‘.csv’ files under a Results folder. For each row in the simulations table, there is a folder named by the profiles that make up the scenario, which contains the corresponding ‘.csv’ files.

Figure 7: The windows file explorer when opened to the directory containing simulation output.

These files can be opened using Microsoft Excel or any text editor. Excel, and many other apps, takes out a file lock on any file it has open, while a FACTS results file is open in another piece of software it cannot be deleted or modified by FACTS. The most common cause for an error to be reported when simulating trials in FACTS is because the user has one of the previous results files is still open in Excel.

In the scenario directory there are the following types of results file:

Summary.csv Contains a single row of data that summarizes the simulation results. This is the source of the shown on the simulations tab.

Summary_freq_bocf_<Endpoint>.csv, Continuous only. Summary_freq_locf_<Endpoint>.csv, Summary_freq_ignore_<Endpoint>.csv, Summary_freq_fail_<Endpoint>.csv, Dichotomous only. Contains a single row of data that summarizes the frequentist results. A file is output for each endpoint for which frequentist analysis is enabled, and for each of the different method for dealing with missing data that are selected on the Frequentist Analysis tab: BOCF if the endpoint is continuous and the baseline score is being simulated, “fail” for dichotomous, LOCF and “ignore” (referred to as PP (Per-Protocol) in the GUI).

Simulations.csv Contains one row per simulation describing the final state of each simulation for every trial simulated.

Simulations_freq_bocf_<Endpoint>.csv, Continuous only. Simulations_freq_locf_<Endpoint>.csv, Simulations_freq_ignore_<Endpoint>.csv, Simulations_freq_fail_<Endpoint>.csv, Dichotomous only. Contains one row per simulation describing the final frequentist analysis of each simulation for every trial simulated. A file is output for each endpoint for which frequentist analysis is enabled, and for each of the different methods for dealing with missing data that are applicable

PatientsNNNNN.csv Contains one row per patient in a simulation, where NNNNN is the number of the simulation. By default this file is written out only for the first simulation, but this can be changed on the simulations tab.

WeeksNNNNN.csv Contains one row for each cohort during a simulation where NNN is the number of the simulation. By default this file is written only for the first 100 simulations, but this can be changed via the simulation tab. The values in the last row of the cohorts file will be the same as the final values for that simulation in the simulations file.

weeks_freq_bocf_<Endpoint>_NNNNN.csv, Continuous only. weeks_freq_locf_<Endpoint>_NNNN.csv, weeks_freq_ignore_<Endpoint>_NNNN.csv, weeks_freq_fail_<Endpoint>_NNNNN.csv, Dichotomous only. Contains one row for each interim during a simulation where NNNNN is the number of the simulation. A file is output for each endpoint for which frequentist analysis is enabled, for each simulation for which a frequentist “weeks file” is to be output, and for each of the different methods for dealing with missing data that are applicable.

Contents of summary.csv

The first two lines are header lines, starting with a ‘#’, containing the column headings. The first line contains the lengthy “human readable” form of the QOI names, the second line contains the shorter “computer readable” alternate QOI names. For columns that are not QOI values, the column names are the same in the two rows.

Contents of the summary.csv file for the multiple endpoint engine.
Column Title	Number of columns	Description
Project	1	The name of the “.facts” file – not including the file extension.
Scenario	1	The scenario name as used in FACTS – made by concatenating the names of the various profiles that make up the scenario
Timestamp	1	The time the simulations started
Version	1	The overall version of FACTS that ran the simulations (not the version of the specific design engine)
NSim	1	The number of simulation runs.
No. Subj	1	The mean, over the simulations, of the total number of subjects recruited in the trial.
SE Subj.	1	The standard error of the total number of subjects recruited into the trials.
No.subj 80%-ile	1	The 80^th percentile, over the simulations, of the total number of subjects recruited in the trial.
P(ES)	1	The proportion of simulations that stopped early for success.
P(LS)	1	The proportion of simulations that did not stop early, and were successful on final evaluation.
P(LF)	1	The proportion of simulations that did not stop early, and were futile on final evaluation.
P(EF)	1	The proportion of simulations that stopped early for futility.
SFFF	1	The proportion of simulations that stopped early for success, but at final analysis were futile (Success-Futility ‘flip-flops’).
FSFF	1	The proportion of simulations that stopped early for futility, but at final analysis were successful (Futility- Success ‘flip-flops’).
Undec.	1	The proportion of simulations that did not stop early, and met neither the success or futility final evaluation criteria and hence are counted as ‘undecided’.
Unused	3	Three unused columns for other outcome types.
Early Success Time	1	This is the mean early decision time (over the simulations stopped early for success), it is the time from the start of the trial to the interim where early success was declared. It does not include the subsequent follow up time.
Mean Alloc <Dose>	D	The mean (over the simulations) of the number of subjects allocated to each treatment arm.
SE Alloc <Dose>	D	The standard error (over the simulations) of the number of subjects allocated to each treatment arm.
No. Dropouts <Dose>	D	The mean (over the simulations) of the number of dropouts for each treatment arm.
Mean Util <Dose>	D	This is the mean (over the simulations) of the posterior estimate of the utility of this dose.
SE Util <Dose>	D	This is the SD (over the simulations) of the posterior estimate of the utility of this dose.
Mean Resp <Endpoint><Dose>	D per endpoint	This is the mean (over the simulations) of the estimate of the response on this endpoint on this dose.
SE Resp <Dose>	D per endpoint	This is the standard deviation (over the simulations) of the estimate of the response on this endpoint on this dose.
Mean Sigma	1 per continuous endpoint	This is the mean (over the simulations) of the estimate of the standard deviation of the response (across all doses) on this endpoint.
SE Mean Sigma	1 per continuous endpoint	This is the standard deviation (over the simulations) of the estimate of the standard deviation of the response (across all doses) on this endpoint
True Mean resp <Endpoint> <Dose>	D per endpoint	This is the true change from baseline / response rate from which the simulation data was sampled.
True SD resp <Endpoint> <Dose>	D per endpoint	If the endpoint is continuous these columns are included and report the true standard deviation of the simulated response. If baseline and “baseline adjustment of the simulated subject response” is used, this reported SD will include that impact of that and be different from the value(s) for sigma entered on the VSR > Dose Response tab.
Mean Longmod Resp: <Dose> <Visit>	One per arm per visit per endpoint	These are only calculated and written out if the Time Course Hierarchical or ITP longitudinal models are being used. This is the mean (over the simulations) of the estimate of the mean response for a particular dose at a particular visit, based on the independent estimate of the dose response (omega_d), scaled by the longitudinal model’s estimate of the proportion of the final effect observed at the visit.
SE Mean Longmod Resp: <Dose> <Visit>	One per arm per visit per endpoint	This is the SD ( over the simulations) of the longitudinal model estimate of the response at intermediate visits (see above)
Mean Util <Endpoint> <Dose>	D per endpoint	This is the mean (over the simulations) of the mean estimate of the utility of this endpoint on this dose.
SE Util <Endpoint> <Dose>	D per endpoint	This is the standard error (over the simulations) of the mean estimate of the utility of this endpoint on this dose.
Mean Beta <Endpoint>	1 per continuous endpoint	If a baseline adjusted dose response model is being used then this is the mean (over the simulations) of the estimate of the baseline adjustment parameter ‘Beta’.
SE Beta <Endpoint>	1 per continuous endpoint	If a baseline adjusted dose response model is being used then this is the standard error (over the simulations) of the estimate of the baseline adjustment parameter ‘Beta’.
Mean Baseline <Endpoint>	1 per continuous endpoint	This is the mean (over the simulations) of the mean observed baseline score.
SE Mean Baseline <Endpoint>	1 per continuous endpoint	This is the standard error (over the simulations) of the mean observed baseline score.
SD Baseline <Endpoint>	1 per continuous endpoint	This is the mean (over the simulations) of the SD of the observed baseline score.
SE SD Baseline <Endpoint>	1 per continuous endpoint	This is the standard error (over the simulations) of the mean of the observed baseline score.
True Mean Baseline <Endpoint>	1 per continuous endpoint	This is the true mean of the baseline distribution used to simulate the baseline scores.
True SD Baseline <Endpoint>	1 per continuous endpoint	This is the true SD of the baseline distribution used to simulate the baseline score.
BAC Mean (<Endpoint>)	1	If a hierarchical prior for Control is being used this is the mean (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
SE of BAC Mean (<Endpoint>)	1	If a hierarchical prior for Control is being used this is the standard error (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAC Tau (<Endpoint>)	1	If a hierarchical prior for Control is being used this is the mean (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
SE of BAC Tau (<Endpoint>)	1	If a hierarchical prior for Control is being used this is the standard error (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAAC Mean (<Endpoint>)	1	If a hierarchical prior for Active Comparator is being used this is the mean (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
SE of BAAC Mean (<Endpoint>)	1	If a hierarchical prior for Active Comparator is being used this is the standard error (over the simulations) of the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean control responses.
BAAC Tau (<Endpoint>)	1	If a hierarchical prior for Active Comparator is being used this is the mean (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
SE of BAAC Tau (<Endpoint>)	1	If a hierarchical prior for Active Comparator is being used this is the standard error (over the simulations) of the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
Mean Duration	1	The mean, over the simulations, of the overall duration of the trials, from first person first sit, to last person last visit.
Ppn Arms Dropped <Dose>	D	The proportion of the simulations in which each arm was dropped. These will be zero if arm dropping is not selected as the means of adaptive location.
Mean LPFV time	1	The mean, over the simulations, of the accrual period of the trials, from first person first sit, to last person first visit (LPFV).
QOI Columns

The QOI Columns depend on the QOIs that have been defined for this design. The columns are grouped, so that first all of the Utility QOIs are listed, then all of the Endpoint 1 QOIs are listed, and so on. Within the utility or an endpoint group, the QOIs are arranged in the following order:

Statistic	Description
Posterior Probabilities	The summary file contains the mean (over the simulations) of the posterior probability for each dose.
Predictive Probabilities	The summary file contains the mean (over the simulations) of the predictive probability for each dose.
P-Values	The summary file contains the mean (over the simulations) of the p-value for each dose.
Target Probabilities	The summary file contains the proportion of times (over the simulations) each dose had the greatest probability of being the target.
Decision QOIs	The summary file contains the mean (over the simulations) of the final value of the decision QOI at the target.

Contents of Summary_freq_<missing>_<Endpoint>.csv

There is a frequentist summary file for each type of treatment of missing values for each endpoint.

The first line is a header line, starting with a ‘#’, containing the FACTS GUI version number, the name of the FACTS file, the name of the scenario, and the time stamp of the start of the simulation.

The second line is a header line, starting with a ‘#’, containing the following column headings.

The contents of the Summary_freq_<missing>_<Endpoint>.csv file.
Column Title	Number of columns	Description
Mean Theta <Dose>	D	The mean response per dose.
SE Mean Theta <Dose>	D	The standard error of the response per dose
Mean Beta (baseline)	1 if endpoint continuous	The mean of the estimate of the baseline coefficient.
SE Mean Beta (baseline)	1 if endpoint continuous	The standard error of the estimate of the baseline coefficient
Ppn Signif	1	The proportion of simulations where at least one of the unadjusted p-values is less than the user specified one-sided alpha.
Ppn Signif <Dose>	D	For each treatment arm, the proportion of simulations where the unadjusted p-value is less than the user specified one-sided alpha. The marginal probability of significance per dose (using unadjusted p-values.
Bias <Dose>	D	The difference between the mean response and the true (simulated) response per dose
Coverage <Dose>	D	The proportion of simulations where the unadjusted confidence interval for the response contains the true response rate used to simulate subject responses.
Ppn Signif Dunnett	1	The proportion of simulations where at least one of the Dunnett adjusted p-values is less than the user specified one-sided alpha.
Ppn Signif Dunnett <Dose>	D	For each treatment arm, the proportion of simulations where the Dunnett adjusted p-value is less than the user specified one-sided alpha.
Coverage Dunnett <Dose>	D	The proportion of simulations where the Dunnett adjusted confidence interval contains the true response rate used to simulate subjects responses.
Ppn Signif Bonf	1	The proportion of simulations where at least one of the Bonferroni adjusted p-values is less than the user specified one-sided alpha.
Ppn Signif Bonf <Dose>	D	For each treatment arm, the proportion of simulations where the Bonferroni adjusted p-value is less than the user specified one-sided alpha.
Coverage Bonf <Dose>	D	The proportion of simulations where the Bonferroni adjusted confidence interval contains the true response rate used to simulate subjects responses.
Ppn Signif Trend	1	The proportion of simulations where the trend test p-value is less than the user specified one-sided alpha.

Contents of simulations.csv and weeksNNNNN.csv

The simulations.csv file holds the FACTS summary of the final analysis for each simulation (one per row).

The weeksNNNNN.csv file holds the FACTS summary of every analysis for the NNNNN^th simulation. It contains a row for each interim in the trial and a row for the final analysis (Interim number 999).

The final analysis occurs after all the planned data is collected. If the trial does not stop early, the final analysis occurs after full follow-up for all subjects. If the trial does stop early at an interim analysis, then the timing of the final analysis depends on the status of the check boxes on the Interims tab. If follow-up is collected after the particular interim decision, then an analysis after full follow-up of the subjects recruited up to the point of the interim when the stopping decision was taken. If there is no follow-up intended after the interim analysis decision, then the final analysis occurs immediately (at the same time as the interim analysis), and the final analysis criteria are checked.

The second line is a header line, starting with a ‘#’, containing the following column headings.

Most of the columns are common to the two file types, but the weeks file does not contain columns for the ‘final’ values of the evaluation criteria.

The contents of the simulations.csv file and the weeksNNNNN.csv file for the multiple endpoint engine.
Column Title	Number of columns	In simulations file	In weeks file	Description
# Weeks (Duration)	1		✔	The week of the analysis.
#Sim	1	✔		The number of the simulation.
LastInterim	1	✔		The index of the last interim performed the index of the interim immediately before the final interim (index 999). Note this is not necessarily the interim when the trial stopped if the design includes follow-up after stopping.
#Subjects	1	✔	✔	The number of subjects recruited in the simulation.
Outcome	1	✔	✔	A flag categorizing final study outcome: = Early success = Late success = Late futility = Early futility = Success to futility flip-flop = Futility to success flip-flop = Inconclusive
Mean Utility <Dose>	D	✔	✔	The mean estimate of utility per dose
SD Utility <Dose>	D	✔	✔	The SD of the estimate of utility
Early Success	1	✔	✔	This is the time to the early success decision (-9999 if there was no early success decision). The from the start of the trial to the interim where the early success conditions were first met. It does not include the subsequent follow up time.
Alloc <Dose>	D	✔	✔	The number of subjects allocated to each arm.
Pr(Alloc) <Dose>	D	✔	✔	The probability of allocation to the different arms following the interim.
Complete <Dose>	D	✔	✔	The number of subjects on each dose that have completed – final endpoint data is available.
Complete Information <Dose>	D	✔	✔	The number of subject who count as complete for the purposes of timing interims – whether complete or opportunity to complete and at which visit is as defined on the Interims tab. If interims are by enrolment then CompleteInformation is the number of subjects that have completed – final endpoint data is available.
#Dropouts <Dose> <Visit>	D*V	✔	✔	The number of subjects that have dropped out on each arm at each visit
Mean resp <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The estimated response of each treatment arm.
SD resp <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The standard deviation of the estimate of response of each treatment arm.
True Mean resp <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The true mean response of each treatment arm for this simulation.
True SD resp <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The true SD of the response of each treatment arm for this simulation
Sigma <Endpoint>	1 per continuous endpoint	✔	✔	The pooled estimate of the SD of the response across all the treatment arms
SD Sigma <Endpoint>	1 per continuous endpoint	✔	✔	The SD of the pooled estimate of the SD of the response across all the treatment arms
Beta <Endpoint>	1 per continuous endpoint	✔	✔	The estimate of Beta the baseline adjustment coefficient, if baseline adjustment is being used.
SD Beta <Endpoint>	1 per continuous endpoint	✔	✔	The SD of the estimate of Beta the baseline adjustment coefficient.
Mean Baseline <Endpoint>	1 per continuous endpoint	✔	✔	The estimate of the mean baseline score.
SE Mean Baseline <Endpoint>	1 per continuous endpoint	✔	✔	The standard error of the estimate of the mean baseline score.
SD Baseline <Endpoint>	1 per continuous endpoint	✔	✔	The estimate of the SD of the baseline score.
True Mean Baseline <Endpoint>	1 per continuous endpoint	✔	✔	The true mean of the simulated baseline score
True SD Baseline <Endpoint>	1 per continuous endpoint	✔	✔	The true SD of the simulated baseline score
Mean Utility <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The mean estimate of the utility of this endpoint
SD Utility <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The SD of the estimate of utility of this endpoint
Mean Raw Response <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The mean raw response based solely on the observed data on each dose.
SE mean Raw Response <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The SE of the estimate of the mean raw response on each dose.
Complete <Endpoint> <Dose>	D per continuous endpoint	✔	✔	The number of subjects with final data on this endpoint on each arm at the end of the trial / time of the interim.
DR Param <Endpoint> <Param>	10 per continuous endpoint	✔	✔	The estimate of the mean value of each of the response model parameters. The response models require up to 10 parameters, most less. The model parameters are labeled α₁, α₂, .. the subscripts corresponding to the column index <Param>.
Sd DR Param <Endpoint> <Param>	10 per continuous endpoint	✔	✔	The standard deviation of the estimate of the mean value of the corresponding response model parameter.
Longmod <Endpoint> <Model> <Param> <Visit>	LMLMPV per continuous endpoint	✔	✔	The estimate of the mean value of each of the longitudinal model parameters. The number of models depends on the number of model instances the user has specified. For linear regression the parameters reported are: Alpha – per visit – the mean estimate of the constant offset in the change in response from this visit to the final visit Beta – per visit – the mean estimate of the coefficient of change in response from this visit to the final visit Lambda – per visit – the mean estimate of the SD of the residual error term in the predicted final visit response based on the response at this visit. For time course hierarchical the parameters reported are: Alpha – per visit – the mean estimate of the exponential coefficient of the proportion of the dose response and per-subject response seen at this visit.. Lambda – per visit – the mean estimate of the SD of the residual error term in the predicted final visit response based on the response at this visit. Tau – the mean estimate of the SD of the per subject random effect For ITP the parameters are: K – per model – the mean estimate of the ITP shape parameter Tau – per model - the mean estimate of the SD of the per subject random effect Lambda – per model – the mean estimate of the Sd of the residual error. Omega – per treatment arm – the mean estimate of the mean treatment arm effect. For LOCF & Kernel Density there are no estimated parameters.
Longmod Resp <Endpoint> <Dose> <Visit>	D*V per continuous endpoint	✔	✔	These are only calculated and written out if the Time Course Hierarchical or ITP longitudinal models are being used. This is the mean response for a particular dose at a particular visit.
Mean resp <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The estimated response of each treatment arm.
SD resp <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The standard deviation of the estimate of response of each treatment arm.
Mean resp (loser CI) <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The lower bound of the 95% Credible Interval of the estimate of response for each treatment arm.
Mean resp (upper CI) <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The upper bound of the 95% Credible Interval of the estimate of response for each treatment arm.
True Mean resp <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The true response rate of each treatment arm for this simulation.
Mean Utility <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The mean estimate of the utility of this endpoint
SD Utility <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The SD of the estimate of utility of this endpoint
Mean Raw Response <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The mean raw response based solely on the observed data on each dose.
Complete <Endpoint> <Dose>	D per dichotomous endpoint	✔	✔	The number of subjects with final data on each arm at the end of the trial / time of the interim.
DR Param <Endpoint> <Param>	4 per dichotomous endpoint	✔	✔	The estimate of the mean value of each of the response model parameters. The response models require up to 4 parameters, some less. The model parameters are labeled α₁, α₂, .. the subscripts corresponding to the column their value appears in here.
Sd DR Param <Endpoint> <Param>	4 per dichotomous endpoint	✔	✔	The standard deviation of the estimate of the mean value of the corresponding response model parameter.
Longmod <Endpoint> <Model> <Visit>	LMLMPV per dichotomous endpoint	✔	✔	The estimate of the mean value of each of the longitudinal model parameters. The number of models depends on the number of model instances the user has specified. For Beta-Binomial the parameters reported are: Alpha0 & Beta 0 – The beta binomial alpha & beta parameters for the probability that the final result will be 1 if the result at the visit is 0 Prob01 – the mean probability of the Beta(Alpha0, Beta0) – the probability of 1 being the final result if the result at the visit is 0 Alpha1 & Beta 1 – The beta binomial alpha & beta parameters for the probability that the final result will be 1 if the result at the visit is 1 Prob11 – the mean probability of the Beta(Alpha1, Beta1) – the probability of 1 being the final result if the result at the visit is 1 For logistic regression the parameters reported are: Prob11 – the probability of 1 being the final result if the result at the visit is 1and Prob01 – the probability of 1 being the final result if the result at the visit is 0 For restricted Markov the parameters reported are: Alpha0 – Alpha for state 0 AlphaS – Alpha for stable state Alpha1 – Alpha for state 1 Prob0 – Transition probability to state 0 ProbS – Probability of remaining stable Prob1 – Transition probability to state 1 (values are for the transition to the next visit, so there are no values for the final visit)
BAC Mean <Endpoint>	1	✔	✔	This is the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAC Mean SD <Endpoint>	1	✔	✔	This is the SD of the posterior estimate of the mean of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAC Tau <Endpoint>	1	✔	✔	This is the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAC Tau SD <Endpoint>	1	✔	✔	This is the SD of the posterior estimate of the SD of the Bayesian Augmented Control (Hierarchical Prior) distribution of mean control responses.
BAAC Mean <Endpoint>	1	✔	✔	This is the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
BAAC Mean SD <Endpoint>	1	✔	✔	This is the SD of the posterior estimate of the mean of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean control responses.
BAAC Tau <Endpoint>	1	✔	✔	This is the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
SD BAAC Tau <Endpoint>	1	✔	✔	This is the SD of the posterior estimate of the SD of the Bayesian Augmented Active Comparator (Hierarchical Prior) distribution of mean active comparator responses.
Duration	1	✔	✔	The time in weeks to the analysis shown.
Arm Drop Time <Dose>	D	✔	✔	For each treatment arm, the time in weeks when it was decided to drop the arm.
LPFV	1	✔	✔	The time in weeks to the last patient first visit.
QOI Columns

QOI Type	Description
Posterior Probabilities	The posterior probability for each QOI for each dose.
Predictive Probabilities	The predictive probability for each QOI for each dose.
P-Values	The p-value for each QOI for each dose.
Target Probabilities	The probability of being the target for each QOI for each dose.
Decision QOIs	The value of the decision QOI at the target.
Success Futile	A flag for each QOI decision criteria indicating if the decision QOI was evaluated and compared to a threshold at the interim (weeks file) or final analysis (simulation file). The flag value is -1 if the decision QOI was not evaluated, 0 if it did not meet the threshold and 1 if it did.
Success / Futile Combined	Flags indicating if the interim or final analysis determined success or futility taking all the factors defined for Success/Futility into account: 0 if the conditions were not met, 1 it the conditions were met. If success and futility conditions have been defined that are not mutually exclusive and both sets of combined conditions are met, FACTS will pick one of the outcomes as met but not the other. In order to discourage defining Success and Futility rules that can both be true FACTS does not guarantee which outcome will be selected.

Contents of Simulations_freq_<missingness>_<Endpoint>.csv, Weeks_freq_<missingness>_<Endpoint>.csv

These file contain the frequentist results after an interim analysis (weeks file) or final analysis (simulations file), apart for the first column, the columns are the same in the two types of file, and the same across the different methods for treating missing data and different endpoints.

Column	Number of columns	Description
# Trial	1	The number of the simulation (simulations_freq_…csv only)
# Weeks	1	The week of the interim (weeks_freq_ only)
Theta	D	Unadjusted estimate of response per treatment arm
Theta SD	D	The SD of the estimate of response per treatment arm
Trt Effect	D	The relative size of the treatment effect compared to control per arm (treatment effect for control is always 0). The direction of the comparison takes into account whether a response represents a subject’s condition improving or worsening, so that a positive treatment effect shows the response on the dose is better than on control.
t-Stat	D	The t-test statistic per treatment arm (for control this is always 0)
p-Value	D	The unadjusted t-test p-value per arm (for control this is always 0)
lower CI	D	The unadjusted lower bound of the alpha confidence interval of the estimated treatment effect per treatment arm (for control this is always 0).
upper CI	D	The unadjusted upper bound of the alpha confidence interval of the estimated treatment effect per treatment arm (for control this is always 0).
Min p-Value	1	The smallest of the t-test p-values
p-Value Dunnett	D	The Dunnett adjusted t-test p-value per arm (for control this is always 0).
lower CI Dunnett	D	The Dunnett adjusted lower bound of the alpha confidence interval of the estimated treatment effect (for control this is always 0).
upper CI Dunnett	D	The Dunnett adjusted upper bound of the alpha confidence interval of the estimated treatment effect (for control this is always 0).
Min p-Value Dunnett	1	The smallest of the Dunnett adjusted p-values.
p-Value Bonf	D	The Bonferroni adjusted t-test p-value per arm.
lower-CI Bonf	D	The Bonferroni adjusted lower bound of the alpha confidence interval of the estimated treatment effect (for control this is always 0).
upper-CI Bonf	D	The Bonferroni adjusted upper bound of the alpha confidence interval of the estimated treatment effect (for control this is always 0).
Min p-Value Bonf	1	The smallest of the Bonferroni adjusted t-test value per arm.
t-Stat trend	1	The t statistic from the trend test.
p-Value trend	1	The p-value from the trend test.

Contents of PatientsNNNNN.csv

Column Title	Number of columns	Description
#Subject	1	The subject id number, starting at 1.
Region	1	Which region the subject was recruited in
DateInWeeks	1	The date, in weeks, from the start of the trial, of the subject’s baseline visit and randomization.
Dose	1	The index number (1, …) of the treatment arm the subject belongs to.
LastVisit Overall	1	The time of the last observation of the subject – the index of the visit of the observation from the overall visit list. NOTE this is the index of the last visit using the overall visit index (index of all visits). E.g. for a patient with 6 visits (index: 1, 2, …6), and an endpoint observed on 3 visits: visits 2, 4 and 6 of the overall schedule, if that patients last observed data was the second time this endpoint was observed (last visit for that endpoint will be “2”), the last visit overall will be “4”.
Dropout	1	Whether the subject dropped out of the trial (1) or not (0) before reaching their final visit.
LastVisit <Endpoint>	1	The time of the last observation of this endpoint – the index of the visit of the observation from the visit list for the endpoint
Baseline <Endpoint>	1	The baseline score for the subject, if baseline is simulated. If no baseline, or the endpoint is dichotomous, then this column is -9999.
Response <Endpoint> <Visit>	V	One column per visit from the visit list from the endpoint, recording the observed endpoint value at that visit.

Contents of MCMCNNNNN.csv

The MCMC file, if requested for output by the user, contains all the MCMC samples for the estimated responses for each arm on each endpoint. There is one row per sample (including the burnin) and the samples from all the analyses in the simulation are included. The first two columns are the analysis index and the sample (within analysis) index. The next set of columns are the MCMC samples for the first endpoint. This includes response rate estimates and, if the endpoint is continuous, a sigma. Then, various longitudinal model parameters are reported if used for that endpoint.

Then second endpoint’s MCMC columns begin, again providing response estimates with or without sigma and longitudinal model parameters (if used). The other endpoints proceed in kind.

The first two columns are:

Column	Number of columns	Description
Analysis	1	The index of the analysis (interim) in the simulation.
Sample	1	The index of the sample within the analysis.

Then, the endpoint specific output is provided for each endpoint. For a particular endpoint index, <Endpoint>, the columns output are:

Continuous
Dichotomous

Column	Number of columns	Description
Theta_<Endpoint> <Dose>	D	The estimate of the mean response for each dose, based on the dose response model fitted.
Sigma_<Endpoint>	1	The estimate of the SD of the endpoint.
Beta_<Endpoint>	1	If baseline is included in the simulated data and using the “Baseline Adjusted Model”, this is the value of “Beta”, the baseline coefficient.
A<1-8>_<Endpoint> / Tau_<Endpoint>	P	If the dose response model estimates parameters then the samples of these are listed next. The number and name of the parameters will vary depending on the model being fitted. Check the Design > Dose Response tab for a listing of the parameters.
Longmod_<Endpoint> <model> <Param> <Visit>	LPV	If the analysis includes longitudinal modelling then the samples of the parameters of the longitudinal models are output. The number of models depends on the number of different longitudinal fits the user has specified (one over all arms, one for control and one for all treatment arms, model all arms separately, etc.), the number of parameters the model has, and if these are fitted per visit (which depends on the model being fitted), then the number of visits.

If using a dichotomized continuous variable, then the parameters for the LM are the same as for the continuous model MCMC output listed in the Continuous tab.

If using a dichotomous longitudinal model, then the following parameters are output in the MCMC file.

Column	Number of columns	Description
Pi_<Endpoint> <Dose>	D	The estimate of the response rate for each dose, based on the dose response model fitted.
A<1-8>_<Endpoint> / Tau_<Endpoint>	P	If the dose response model estimates parameters then the samples of these are listed next. The number and name of the parameters will vary depending on the model being fitted. Check the Design > Dose Response tab for a listing of the parameters.
Longmod_<Endpoint> <model> <Param> <Visit>	LPV	If the analysis includes longitudinal modelling then the samples of the parameters of the longitudinal models are output. The number of models depends on the number of different longitudinal fits the user has specified (one over all arms, one for control and one for all treatment arms, model all arms separately, etc.), the number of parameters the model has, and if these are fitted per visit (which depends on the model being fitted), then the number of visits.

If using a restricted markov model for the endpoint, then the following columns are output in the MCMC file for that endpoint.

Column	Number of columns	Description
Pi_<Endpoint> <Dose>	D	The estimate of the response rate for each dose, based on the dose response model fitted.
A<1-8><Endpoint> / Tau<Endpoint>	P	If the dose response model estimates parameters then the samples of these are listed next. The number and name of the parameters will vary depending on the model being fitted. Check the Design > Dose Response tab for a listing of the parameters.
Longmod_<Endpoint> <model> <Param> <Visit>	LPV	Currently the values of Alpha0, Alpha1, AlphaS (the parameters to the Dirichlet distribution) are (unnecessarily – these values are not sampled) output, along with the values for P0, P1 and PS the transition probabilities of each model at each visit.